Re: comunication protocol

Merlin Moncure wrote:
> On 8/9/07, Omar Bettin <o.bettin@tiscali.it> wrote:
>   
>> Merlin Moncure wrote:
>>     
>>> AFAIK, the fastest possible way to get data off the server, skipping
>>> all data and text processing is to write a SPI routine, and stream the
>>> data out locally to the server.   I am doing exactly this in a
>>> particular problem that requires high performance and I can tell you
>>> that SPI is fast.
>>>
>>>       
>>     Interesting,do you thing that is possible to implement some send()
>> in the interface?
>>     
>
> SPI is an interface which allows you to make sql calls from C code
> (PostgreSQL allows you to link C code compiled as a .so to the server
> and call -- see numerous examples in contrib).  The routine you need
> to exploit is SPI_getbinval which gives you Datum (essentially a
> variant) pointing to the internal binary representation of your field.
>  In theory you could collect the data into a buffer and send() it off
> although thats a lot of work IMO.  Also, I would only advise this for
> fast dumps from a single table (no joins, etc).
>   
why not joins?
>   
>>    I have tried ZeosLib and for me is unusable (too slow), I use a
>> strong modified PostgresDAC.
>>     
>
> I'm suprised -- I know the ZeosLib internals and it's a very thin
> layer over libpq.  Here is what I suggest:
>
> * turn on statement logging in the server (set log_min_duration_statement)
> * make some operations in the app which you suggest are slow  -- they
> will show up in the log
> * 'explain analyze' the query from the psql console
>
> make note of the times and post back (maybe move this thread to the
> -performance list)
>
> merlin
>
>   
So, you aren't agree with the compression...

I have sow the network statistics and in some cases, the network traffic 
is very big.
Probably a better setup could increase the performance by a few points 
percent but
I think a compressed protocol could increase the communication by 2/3 times.

I think the biggest bottleneck in the whole system is just that.

:..try to get a query from a remote server with a 56k modem! :)

Omar

> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>                http://archives.postgresql.org
>
>