Обсуждение: TCL_ARRAYS code in libpgtcl is pretty seriously broken
There is some code in libpgtcl that purports to convert Postgres array data values into Tcl lists. Is anyone prepared to argue that that code does something useful in its present state? I can name half a dozen bugs in it without breathing hard: 1. Blithely assumes that any data value beginning with '{' and ending with '}' must represent an array value. Should have some more robust way of discovering whether a column is array type. (In fairness, this might require a FE/BE protocol change, unless arrayness can be determined from the tuple descriptors provided by the backend, ie, field type OID, size, and attmod. Anybody know a way to do that?) 2. Applies a translation that converts all backslash escape sequences defined for C string constants into their equivalent single characters. Since neither the backend nor Tcl generate anything close to C-string escapes, the point of this is difficult to determine. It does however result in unexpected output, eg disappearing backslashes. 3. Applies said translation even when processing a non-array data value. 4. Doesn't actually manage to produce a valid Tcl list, if the data contains anything Tcl considers a special character. What it *should* be doing is quoting, not de-quoting. 5. Fails to modify \\, \{, and \} (thus quite unintentionally doing almost the right thing...) when these sequences appear inside an array value, because "they will be unescaped by Tcl in Tcl_AppendElement". But in fact Tcl_AppendElement is not invoked on the results of this code. 6. Modifies the string returned by libpq *in place*. This would be a const-ness violation if we had been more careful about declaring things const. More importantly, it means that re-examining the same tuple of the PGresult will yield a different result. Not cool. 7. The TCL_ARRAYS code is only invoked in the "-assign" variant of the pgtcl pg_result statement, not in any of the other paths that allow tuple values to be examined. This is presumably an oversight, not the intended behavior. 8. Does not cope with MULTIBYTE strings. (But I don't think Tcl does either, so it's not clear that this can be called a bug.) I am strongly inclined to rip this code out, because it is responsible for several behaviors that were correctly called bugs when backslash- handling was discussed on pgsql-interfaces back in August. If we don't rip it out, it needs a complete rewrite. Unless there is a bulletproof solution to problem #1 (how to tell whether a field's data type is array), I do not think it is appropriate for the basic pg_result code to be applying any such transform. Perhaps it would be reasonable to invent a separate string-formatting function, say "pg_arraytolist", that would perform the conversion. It would then be the application writer's responsibility to know which fields were arrays and apply the conversion if he wanted it. Comments? regards, tom lane
> 1. Blithely assumes that any data value beginning with '{' and ending > with '}' must represent an array value. Should have some more robust > way of discovering whether a column is array type. (In fairness, this > might require a FE/BE protocol change, unless arrayness can be > determined from the tuple descriptors provided by the backend, ie, > field type OID, size, and attmod. Anybody know a way to do that?) > Comments? Postgres seems to use a convention that a type name which starts with an underscore is the array type for the corresponding non-underscore, non-array type. Also, the typelem field in pg_type is non-zero for array types. This isn't a definitive answer and there may be another way to discover array-ness but it's where I would look. Not sure if you'd be happy having to do a select on pg_type for every query unless you're doing it already... - Tom
> There is some code in libpgtcl that purports to convert Postgres array > data values into Tcl lists. Is anyone prepared to argue that that code > does something useful in its present state? I can name half a dozen > bugs in it without breathing hard: > > 1. Blithely assumes that any data value beginning with '{' and ending > with '}' must represent an array value. Should have some more robust > way of discovering whether a column is array type. (In fairness, this > might require a FE/BE protocol change, unless arrayness can be > determined from the tuple descriptors provided by the backend, ie, > field type OID, size, and attmod. Anybody know a way to do that?) > > 2. Applies a translation that converts all backslash escape sequences > defined for C string constants into their equivalent single characters. > Since neither the backend nor Tcl generate anything close to C-string > escapes, the point of this is difficult to determine. It does however > result in unexpected output, eg disappearing backslashes. > > 3. Applies said translation even when processing a non-array data value. > > 4. Doesn't actually manage to produce a valid Tcl list, if the data > contains anything Tcl considers a special character. What it *should* > be doing is quoting, not de-quoting. > > 5. Fails to modify \\, \{, and \} (thus quite unintentionally doing > almost the right thing...) when these sequences appear inside an array > value, because "they will be unescaped by Tcl in Tcl_AppendElement". > But in fact Tcl_AppendElement is not invoked on the results of this > code. > > 6. Modifies the string returned by libpq *in place*. This would be a > const-ness violation if we had been more careful about declaring things > const. More importantly, it means that re-examining the same tuple of > the PGresult will yield a different result. Not cool. > > 7. The TCL_ARRAYS code is only invoked in the "-assign" variant of > the pgtcl pg_result statement, not in any of the other paths that allow > tuple values to be examined. This is presumably an oversight, not > the intended behavior. > > 8. Does not cope with MULTIBYTE strings. (But I don't think Tcl does > either, so it's not clear that this can be called a bug.) > > > I am strongly inclined to rip this code out, because it is responsible > for several behaviors that were correctly called bugs when backslash- > handling was discussed on pgsql-interfaces back in August. If we don't > rip it out, it needs a complete rewrite. > > Unless there is a bulletproof solution to problem #1 (how to tell > whether a field's data type is array), I do not think it is appropriate > for the basic pg_result code to be applying any such transform. Perhaps > it would be reasonable to invent a separate string-formatting function, > say "pg_arraytolist", that would perform the conversion. It would then > be the application writer's responsibility to know which fields were > arrays and apply the conversion if he wanted it. I have just started to learn TCL, and have Practical Tcl and TK by Brent Welch on my desk. Sounds like our interface needs fixing. I am sure we currently do not handle full protocol correctly. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
"Thomas G. Lockhart" <lockhart@alumni.caltech.edu> writes: > This isn't a definitive answer and there may be another way to discover > array-ness but it's where I would look. Not sure if you'd be happy > having to do a select on pg_type for every query unless you're doing it > already... I think that won't fly for performance reasons. I know I wouldn't want my Tcl applications paying an additional frontend-to-backend round trip for every SELECT result... I was actually a tad surprised to realize that the column-type info sent by the backend couldn't answer such a basic question as "is this an array?". Something to fix if we ever rev the FE/BE protocol again. Don't think I'd propose a protocol rev just for this, though. In the meantime, I'm inclined to take the fallback approach I suggested yesterday: provide the array-to-list reformatting function as a separate Tcl statement that the application programmer can decide to invoke. The app writer is likely to know perfectly well where he needs that feature anyway. regards, tom lane