Re: [GENERAL] Gripe: bytea_output default => data corruption

ljb wrote:
> tgl@sss.pgh.pa.us wrote:
> > I think we should simply remove the description of *how* the escaping is
> > performed, and state only that the function produces a suitably escaped
> > literal string.  Anything else is not future-proof, and could someday
> > break the way this wording did.
>
> Perhaps it would be best to remove escaping details here. But the
> description of PQescapeBytea() might need to be rewritten, too. Without
> describing exactly what PQescapeByteaConn() does, it is hard to understand
> what PQescapeBytea() does not do, and why it therefore "might give the
> wrong results".
>
> I think the actual function behavior should be documented somewhere. Even
> though it might change again.

Based on this report, I have created the attached documentation patch
which clarifies the libpq behavior for escaping bytea.  I am planning to
backpatch this to 9.0 as well.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 7b70970..26f5140 100644
*** a/doc/src/sgml/libpq.sgml
--- b/doc/src/sgml/libpq.sgml
*************** unsigned char *PQescapeByteaConn(PGconn
*** 3419,3434 ****
        </para>

        <para>
!        Certain byte values <emphasis>must</emphasis> be escaped (but all
!        byte values <emphasis>can</emphasis> be escaped) when used as part
!        of a <type>bytea</type> literal in an <acronym>SQL</acronym>
!        statement. In general, to escape a byte, it is converted into the
!        three digit octal number equal to the octet value, and preceded by
!        usually two backslashes. The single quote (<literal>'</>) and backslash
!        (<literal>\</>) characters have special alternative escape
!        sequences. See <xref linkend="datatype-binary"> for more
!        information. <function>PQescapeByteaConn</function> performs this
!        operation, escaping only the minimally required bytes.
        </para>

        <para>
--- 3419,3429 ----
        </para>

        <para>
!        Certain byte values must be escaped when used as part of a
!        <type>bytea</type> literal in an <acronym>SQL</acronym> statement.
!        <function>PQescapeByteaConn</function> escapes such bytes using
!        either hex encoding or backslash escaping.  See <xref
!        linkend="datatype-binary"> for more information.
        </para>

        <para>
*************** unsigned char *PQescapeBytea(const unsig
*** 3486,3495 ****
         The only difference from <function>PQescapeByteaConn</> is that
         <function>PQescapeBytea</> does not take a <structname>PGconn</>
         parameter.  Because of this, it cannot adjust its behavior
!        depending on the connection properties (in particular, whether
!        standard-conforming strings are enabled) and therefore
!        <emphasis>it might give the wrong results</>.  Also, it has no
!        way to return an error message on failure.
        </para>

        <para>
--- 3481,3490 ----
         The only difference from <function>PQescapeByteaConn</> is that
         <function>PQescapeBytea</> does not take a <structname>PGconn</>
         parameter.  Because of this, it cannot adjust its behavior
!        depending on the connection properties (e.g.
!        <literal>standard_conforming_strings</>, <literal>bytea_output</a>)
!        and therefore <emphasis>might give the wrong results</>.  Also,
!        it has no way to return an error message on failure.
        </para>

        <para>
diff --git a/doc/src/sgml/release-9.0.sgml b/doc/src/sgml/release-9.0.sgml
index 2288f1b..c43f142 100644
*** a/doc/src/sgml/release-9.0.sgml
--- b/doc/src/sgml/release-9.0.sgml
***************
*** 2342,2348 ****
        whether hex or traditional format is used for <type>bytea</>
        output.  Libpq's <function>PQescapeByteaConn()</> function automatically
        uses the hex format when connected to <productname>PostgreSQL</> 9.0
!       or newer servers.
       </para>

       <para>
--- 2342,2349 ----
        whether hex or traditional format is used for <type>bytea</>
        output.  Libpq's <function>PQescapeByteaConn()</> function automatically
        uses the hex format when connected to <productname>PostgreSQL</> 9.0
!       or newer servers.  However, pre-9.0 libpq versions will not
!       correctly process hex format from newer servers.
       </para>

       <para>
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 7b70970..26f5140 100644
*** a/doc/src/sgml/libpq.sgml
--- b/doc/src/sgml/libpq.sgml
*************** unsigned char *PQescapeByteaConn(PGconn
*** 3419,3434 ****
        </para>

        <para>
!        Certain byte values <emphasis>must</emphasis> be escaped (but all
!        byte values <emphasis>can</emphasis> be escaped) when used as part
!        of a <type>bytea</type> literal in an <acronym>SQL</acronym>
!        statement. In general, to escape a byte, it is converted into the
!        three digit octal number equal to the octet value, and preceded by
!        usually two backslashes. The single quote (<literal>'</>) and backslash
!        (<literal>\</>) characters have special alternative escape
!        sequences. See <xref linkend="datatype-binary"> for more
!        information. <function>PQescapeByteaConn</function> performs this
!        operation, escaping only the minimally required bytes.
        </para>

        <para>
--- 3419,3429 ----
        </para>

        <para>
!        Certain byte values must be escaped when used as part of a
!        <type>bytea</type> literal in an <acronym>SQL</acronym> statement.
!        <function>PQescapeByteaConn</function> escapes such bytes using
!        either hex encoding or backslash escaping.  See <xref
!        linkend="datatype-binary"> for more information.
        </para>

        <para>
*************** unsigned char *PQescapeBytea(const unsig
*** 3486,3495 ****
         The only difference from <function>PQescapeByteaConn</> is that
         <function>PQescapeBytea</> does not take a <structname>PGconn</>
         parameter.  Because of this, it cannot adjust its behavior
!        depending on the connection properties (in particular, whether
!        standard-conforming strings are enabled) and therefore
!        <emphasis>it might give the wrong results</>.  Also, it has no
!        way to return an error message on failure.
        </para>

        <para>
--- 3481,3490 ----
         The only difference from <function>PQescapeByteaConn</> is that
         <function>PQescapeBytea</> does not take a <structname>PGconn</>
         parameter.  Because of this, it cannot adjust its behavior
!        depending on the connection properties (e.g.
!        <literal>standard_conforming_strings</>, <literal>bytea_output</a>)
!        and therefore <emphasis>might give the wrong results</>.  Also,
!        it has no way to return an error message on failure.
        </para>

        <para>