Re: [SQL] Dilema.

>   I have a problem that I'm hoping someone can help me with. I've built a
> web-pased application that is back-ended by PGsql.. It works great, I
> couldn't ask for better preformance.

    And  Bruce  just  said  (on the hackers list) that big tuples
    aren't an item of big pressure.  Reading this  less  than  24
    hours past tells another story.

> However, I just recently coded another module for it and this module needed
> to accept text files (or just large text blocks lets say). I ran into the 8k
> query limit barrier. I need these blocks of text to be searchable and am
> having to do this with a flat-file based solution right now, it's slow and
> cumbersome. I would much rather have the text stored in a PG field so I
> don't have to stray from the design of the search engine (which is designed
> to search only the pg database, but has been hacked to search flat files
> too).. I could break the texts up but they really all need to be in a single
> field, not broken up. I can't think up a way to do it with the 8k query
> limit.

    If  you  really  need  to  store  big  texts  into a table, a
    combination of a view, update  rules  and  PL  functions  can
    definitely  do  it. It might look a little complex first, but
    after understanding the trick, it's pretty neat.

    Look at this:

        pgsql=> \d bttest
              View "bttest"
         Attribute | Type | Extra
        -----------+------+-------
         key       | int4 |
         content   | text |
        View definition: SELECT bttest_shadow."key",
        bigtext_fetch(bttest_shadow.btid) AS content FROM
        bttest_shadow;

        pgsql=> select key, length(content) from bttest;
         key | length
        -----+--------
           1 |      3
           2 |      9
           4 | 135840
        (3 rows)

    Yes, the row with key=4 really contains a text of that  size.
    And I can get it back exactly as it was inserted.  And yes, I
    can do INSERT/UPDATE/DELETE on bttest. Anything behaves as if
    it where a regular table.

    But  it  needs  the  unlimited  querysize,  supported only in
    CURRENT development tree, so you'll  have  to  wait  for  7.0
    anyway.

> Also feel free to suggest some kind of indexing system for the flat files,
> the only problem with that will be integration into my existing application
> but I've left some hooks open so I can deal with it..

    That's another story, and Tom's comment on it says it all.


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#========================================= wieck@debis.com (Jan Wieck) #