Обсуждение: help with error message from perl Pg

We have an unusual problem with some perl code that is processing data
via DBD facility.  Basically, the code consists of various subroutines
that are identified in a hash.  The primary script then references the
subroutines through a &$prog(...) syntax.  What appears to happen is
that all subroutines will run just fine if they do not need to do any
deletes or updates to the database.  Any subroutine that does a delete
or update runs fine, but the subsequent routine fails with the following
error:

prgcaphlg: FATAL:  invalid frontend message type 78
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

Searching google for 'invalid frontend message type 78' returns no hits,
which is a scary thing.

Any insights would be greatly appreciated.

--
Until later, Geoffrey

Any society that would give up a little liberty to gain a little
security will deserve neither and lose both.  - Benjamin Franklin

Martijn van Oosterhout wrote:
> On Tue, Jun 20, 2006 at 09:33:13AM -0400, Geoffrey wrote:
>> We have an unusual problem with some perl code that is processing data
>> via DBD facility.  Basically, the code consists of various subroutines
>> that are identified in a hash.  The primary script then references the
>> subroutines through a &$prog(...) syntax.  What appears to happen is
>> that all subroutines will run just fine if they do not need to do any
>> deletes or updates to the database.  Any subroutine that does a delete
>> or update runs fine, but the subsequent routine fails with the following
>> error:
>>
>> prgcaphlg: FATAL:  invalid frontend message type 78
>
> Errors in the FE-BE protocol are usually caused by users trying to use
> the same connection from multiple threads simultaneously. libpq doesn't
> handle that and so ends up sending things out of order.
>
> Are you using threads?

Not intentionally.  I'm assuming I would have to specify a threaded
process explicitly.

> Also, the database connection, how is the handle passed around? Youre
> not closing it accedently somewhere?

We considered that and have verified that we are not closing it.  But,
the question came up, should we be passing it by reference or value?  We
are doing the following:

my $conn = Pg::connectdb ("dbname=$db port=$port");
.
.
my $retVal = &$prog($conn, @args);

Question is, should we be doing:

my $retVal = &$prog(\$conn, @args);


FYI, there's no expectation of changing $conn in anyway within the sub
routines.

--
Until later, Geoffrey

Any society that would give up a little liberty to gain a little
security will deserve neither and lose both.  - Benjamin Franklin

Geoffrey <esoteric@3times25.net> writes:
> ... Any subroutine that does a delete
> or update runs fine, but the subsequent routine fails with the following
> error:

> prgcaphlg: FATAL:  invalid frontend message type 78
> server closed the connection unexpectedly
>     This probably means the server terminated abnormally
>     before or while processing the request.

> Searching google for 'invalid frontend message type 78' returns no hits,
> which is a scary thing.

78 would be ASCII 'N', but that's not really significant AFAICS.  The
problem here is that the frontend and backend have lost sync: the server
is expecting to find a message beginning at a place in the frontend data
stream that evidently isn't the start of a message.  In short, the
frontend has sent corrupted data of some sort.

Martijn's theory of inadequately locked threaded access is certainly one
likely way this can happen, but it's not the only one.  It might be
useful for you to capture the data stream (with something like tcpdump)
and try to get more information about the nature of the corruption.
Frequently, if you can identify "ah-hah, THIS data is being inserted
into the middle of THAT" or whatever, the cause becomes obvious.

Also, before you spend too much time on this, make sure your DBI and
DBD::Pg modules are up-to-date.  If it's a bug in that level, it'd be
foolish to waste much of your own time chasing it.

            regards, tom lane

Err...

On Tue, Jun 20, 2006 at 10:20:19AM -0400, Geoffrey wrote:
> Martijn van Oosterhout wrote:
> >On Tue, Jun 20, 2006 at 09:33:13AM -0400, Geoffrey wrote:
> >>We have an unusual problem with some perl code that is processing data
> >>via DBD facility.  Basically, the code consists of various subroutines
        ^^^

> my $conn = Pg::connectdb ("dbname=$db port=$port");
             ^^^^^^^^^^^^^
Are you using Pg or DBI?

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

On Tue, Jun 20, 2006 at 09:33:13AM -0400, Geoffrey wrote:
> We have an unusual problem with some perl code that is processing data
> via DBD facility.  Basically, the code consists of various subroutines
> that are identified in a hash.  The primary script then references the
> subroutines through a &$prog(...) syntax.  What appears to happen is
> that all subroutines will run just fine if they do not need to do any
> deletes or updates to the database.  Any subroutine that does a delete
> or update runs fine, but the subsequent routine fails with the following
> error:
>
> prgcaphlg: FATAL:  invalid frontend message type 78

Errors in the FE-BE protocol are usually caused by users trying to use
the same connection from multiple threads simultaneously. libpq doesn't
handle that and so ends up sending things out of order.

Are you using threads?

Also, the database connection, how is the handle passed around? Youre
not closing it accedently somewhere?

Have a ncie day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

On Tue, June 20, 2006 10:44 am, Tom Lane wrote:
> 78 would be ASCII 'N', but that's not really significant AFAICS.  The
> problem here is that the frontend and backend have lost sync: the server is
> expecting to find a message beginning at a place in the frontend data
> stream that evidently isn't the start of a message.  In short, the
> frontend has sent corrupted data of some sort.
>
> Martijn's theory of inadequately locked threaded access is certainly one
> likely way this can happen, but it's not the only one.  It might be useful
> for you to capture the data stream (with something like tcpdump) and try
> to get more information about the nature of the corruption. Frequently, if
> you can identify "ah-hah, THIS data is being inserted into the middle of
> THAT" or whatever, the cause becomes obvious.
>
>
> Also, before you spend too much time on this, make sure your DBI and
> DBD::Pg modules are up-to-date.  If it's a bug in that level, it'd be
> foolish to waste much of your own time chasing it.

If you look at his example code, he's not even using DBI- he's using Pg.pm
which is for all practical purposes deprecated and supplanted by DBD::Pg.