Обсуждение: MemoryContextAlloc: invalid request size 1934906735

Поиск
Список
Период
Сортировка

MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
I have been getting the subject message ever since upgrading to 7.2.1.  I 
tried 7.2.2 with the same thing.  It seems to be related to my chkpass type 
(see contrib) as it only happens on tables with that type.  I tried it on a 
new database with a very simple table and still see it.  After compiling 
chkpass.c and running the SQL to create the type create a table with one 
field with chkpass type.  Add a number of rows, I did 24, then vacuum it.  
You get something similar to the above.  Sometimes you get "Memory exhausted 
in AllocSetAlloc(929654141)" instead and once in a while there is no error.

Given table x with field c as chkpass run "UPDATE x SET c = ':a';" on it.  
This never fails.  Now try UPDATE x SET c = 'a';" and let chkpass crypt the 
value.  This usually fails with one of the above messages.  The number is 
constant until you run the UPDATE again.

Somehow the value of the password string is polluting the size storage.  I 
know this because every time this happens, the first 2nd, 3rd and 4th bytes 
(after adjusting for endianness) of the integer are the 6th, 7th and 8th 
characters of the encrypted password.

I have another type which is built like this except that it is an indexable 
type and that doesn't seem to have any problem.  It is constructed the same 
way otherwise.  The palloc calls appear to be correct.  Can anyone see why 
this would suddenly be a problem?

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
Tom Lane
Дата:
"D'Arcy J.M. Cain" <darcy@druid.net> writes:
> I have been getting the subject message ever since upgrading to 7.2.1.  I 
> tried 7.2.2 with the same thing.  It seems to be related to my chkpass type 
> (see contrib) as it only happens on tables with that type.

FWIW, I couldn't see any problem in CVS tip.  Could you provide an exact
sequence-to-reproduce?
        regards, tom lane


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 27, 2002 02:01 am, Tom Lane wrote:
> "D'Arcy J.M. Cain" <darcy@druid.net> writes:
> > I have been getting the subject message ever since upgrading to 7.2.1.  I
> > tried 7.2.2 with the same thing.  It seems to be related to my chkpass
> > type (see contrib) as it only happens on tables with that type.
>
> FWIW, I couldn't see any problem in CVS tip.  Could you provide an exact
> sequence-to-reproduce?

Surely.  Create a database (chkpass_test) and, after loading the chkpass 
type, follow this bouncing ball.

1. CREATE TABLE x (c chkpass);
2. INSERT INTO x VALUES ('a'); [Repeat 20 times]
3. VACUUM ANALYZE x;
4. UPDATE x SET c = ':a';
5. VACUUM ANALYZE x;
6. UPDATE x SET c = 'a';
7. VACUUM ANALYZE x;
8. GOTO 4

Note that 3 and 7 should fail most (95%?) of the time.  When it does, convert 
the invalid size to hex and compare the bytes of the integer with the 
encrypted value in the table.

Note that the fact that 5 does not fail has nothing to do with the path 
through chkpass.c.  If you take a failing password and insert it raw with 
"UPDATE x SET c = ':2w3dhratDt7xo';" then you get the same failure (even the 
same number - 1952543340 in my case) as you did when chkpass encrypted with 
that salt.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
Tom Lane
Дата:
"D'Arcy J.M. Cain" <darcy@druid.net> writes:
> On August 27, 2002 02:01 am, Tom Lane wrote:
>> FWIW, I couldn't see any problem in CVS tip.  Could you provide an exact
>> sequence-to-reproduce?

> Surely.  Create a database (chkpass_test) and, after loading the chkpass 
> type, follow this bouncing ball.
> ...
> Note that 3 and 7 should fail most (95%?) of the time.

Zero failures in a dozen iterations here.  Anyone else see it?
        regards, tom lane


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 27, 2002 09:13 am, Tom Lane wrote:
> "D'Arcy J.M. Cain" <darcy@druid.net> writes:
> > On August 27, 2002 02:01 am, Tom Lane wrote:
> >> FWIW, I couldn't see any problem in CVS tip.  Could you provide an exact
> >> sequence-to-reproduce?
> >
> > Surely.  Create a database (chkpass_test) and, after loading the chkpass
> > type, follow this bouncing ball.
> > ...
> > Note that 3 and 7 should fail most (95%?) of the time.
>
> Zero failures in a dozen iterations here.  Anyone else see it?

NetBSD issue?  It does it on all the NetBSD systems I tried it on.  I will be 
putting 7.2.2 on AIX shortly.  I can try again.  Can you think of any OS 
issue here?

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
Tom Lane
Дата:
"D'Arcy J.M. Cain" <darcy@druid.net> writes:
> On August 27, 2002 09:13 am, Tom Lane wrote:
>> Zero failures in a dozen iterations here.  Anyone else see it?

> NetBSD issue?  It does it on all the NetBSD systems I tried it on.

Hm.  The first thing I thought was "portability problem" --- I had been
testing on HPUX.  But I just tried it on a Linux Intel box and see no
failure there either.

> I will be putting 7.2.2 on AIX shortly.

Note I'm testing CVS tip, not 7.2.  Could we have fixed whatever the bug
is?  Seems unlikely, if it just appeared between 7.2 and 7.2.1 for you.

Another possible reason for differences is configuration.  I used a
pretty plain-vanilla configure:

./configure --with-CXX --with-tcl --enable-cassert --enable-debug

How about you?
        regards, tom lane


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 27, 2002 03:39 pm, Tom Lane wrote:
> "D'Arcy J.M. Cain" <darcy@druid.net> writes:
> > NetBSD issue?  It does it on all the NetBSD systems I tried it on.
>
> Hm.  The first thing I thought was "portability problem" --- I had been
> testing on HPUX.  But I just tried it on a Linux Intel box and see no
> failure there either.

Well, Linux != NetBSD except for processor.

> Note I'm testing CVS tip, not 7.2.  Could we have fixed whatever the bug
> is?  Seems unlikely, if it just appeared between 7.2 and 7.2.1 for you.

Actually it was between 7.1.2 and 7.2.1.  I upgraded to 7.2.2 to see if it 
would go away.  I can try tip on another box.

> Another possible reason for differences is configuration.  I used a
> pretty plain-vanilla configure:
>
> ./configure --with-CXX --with-tcl --enable-cassert --enable-debug
>
> How about you?

I am using the NetBSD pkgsrc system.  It generates this configuration.
How about that --enable-multibyte?

--enable-multibyte --disable-odbc --without-java --without-perl 
--without-python --without-tcl --without-tk 
--includedir=/usr/pkg/include/pgsql 
--with-htmldir=/usr/pkg/share/doc/html/postgresql --disable-readline 
--enable-locale --enable-syslog --with-CXX --with-template=netbsd 
--with-openssl=/usr --host=i386--netbsdelf --prefix=/usr/pkg

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
Tom Lane
Дата:
"D'Arcy J.M. Cain" <darcy@druid.net> writes:
> How about that --enable-multibyte?

--enable-multibyte is default (indeed only) option on CVS tip,
so that's not it.  Ditto locale.

Could you try CVS tip on one of the boxes where you see the failure?
That'd help to narrow down the issue.
        regards, tom lane


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 27, 2002 06:20 pm, Tom Lane wrote:
> "D'Arcy J.M. Cain" <darcy@druid.net> writes:
> > How about that --enable-multibyte?
>
> --enable-multibyte is default (indeed only) option on CVS tip,
> so that's not it.  Ditto locale.
>
> Could you try CVS tip on one of the boxes where you see the failure?
> That'd help to narrow down the issue.

Same issue.  It must be a NetBSD issue but I can't think what.  Could there 
be some data size issues where NetBSD is different than Linux, etc?  And why 
only on the chkpass type?  I have other user defined types and they work fine.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
Tom Lane
Дата:
"D'Arcy J.M. Cain" <darcy@druid.net> writes:
>> Could you try CVS tip on one of the boxes where you see the failure?
>> That'd help to narrow down the issue.

> Same issue.  It must be a NetBSD issue but I can't think what.

The behavior looks a lot like a memory clobber, so perhaps the key
variable is some difference in malloc's allocation strategy, causing
two items to be adjacent in NetBSD where they are not on the other
platforms we've tried.

I eyeballed the chkpass code and didn't see any sign of buffer overruns,
but maybe it needs a harder look.

Hm --- I guess another possible variable is behavior of the local
crypt() function.  Is NetBSD's crypt perhaps willing to return strings
longer than 13 chars?

Did you try CVS tip both with and without --enable-cassert?  That turns
on memory context checking which might alter the failure in interesting
ways.
        regards, tom lane


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 28, 2002 09:23 am, Tom Lane wrote:

> The behavior looks a lot like a memory clobber, so perhaps the key
> variable is some difference in malloc's allocation strategy, causing
> two items to be adjacent in NetBSD where they are not on the other
> platforms we've tried.

Hmm.  I might try adding some buffer in MemoryContextAlloc() and see if that 
changes anything.  One thing that may be different is that NetBSD is 64 bit 
clean.  I don't think the other i386 systems are.

> I eyeballed the chkpass code and didn't see any sign of buffer overruns,
> but maybe it needs a harder look.

It's pretty simple.  Not even indexing.  In fact, I wondered if I should add 
some just like my other type that does do indexing.  That seemed to be the 
only real difference between the two and the other works.

> Hm --- I guess another possible variable is behavior of the local
> crypt() function.  Is NetBSD's crypt perhaps willing to return strings
> longer than 13 chars?

Well, the value that it stores is the correct 13 character DES string.

> Did you try CVS tip both with and without --enable-cassert?  That turns
> on memory context checking which might alter the failure in interesting
> ways.

No difference.  It seems that PostgreSQL is too good at catching the problem 
before the assert macros see it.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 28, 2002 12:48 pm, D'Arcy J.M. Cain wrote:
> On August 28, 2002 09:23 am, Tom Lane wrote:
> > The behavior looks a lot like a memory clobber, so perhaps the key
> > variable is some difference in malloc's allocation strategy, causing
> > two items to be adjacent in NetBSD where they are not on the other
> > platforms we've tried.
>
> Hmm.  I might try adding some buffer in MemoryContextAlloc() and see if

Nope.  Tried adding "size += 64;" into MemoryContextAlloc() and it made no 
difference.  

I had also tried changing palloc.h and mcxt.c to turn MemoryContextAlloc() 
into a macro that called a modified version of the real one to try to narrow 
down where it was being called from but that wouldn't even run.  Is there 
another file I have to modify as well if I try that?  Here are the changes I 
tried.  See any reason that it shouldn't have worked if I rebuilt everything?

*** postgresql-server/work.i386/postgresql-7.2.2/src/backend/utils/mmgr/mcxt.c
Wed Aug 28 14:01:31 2002
--- mcxt.c  Mon Aug 26 21:51:31 2002
***************
*** 409,416 ****  * nodes/memnodes.h into postgres.h which seems a bad idea.  */ void *
! MemoryContextAlloc(MemoryContext context, Size size) {   AssertArg(MemoryContextIsValid(context));
   if (!AllocSizeIsValid(size))
--- 409,417 ----  * nodes/memnodes.h into postgres.h which seems a bad idea.  */ void *
! _MemoryContextAlloc(MemoryContext context, Size size, const char *f, int l) {
+   elog(NOTICE, "_MemoryContextAlloc called from %s line %d", f, l);   AssertArg(MemoryContextIsValid(context));
   if (!AllocSizeIsValid(size))

*** postgresql-server/work.i386/postgresql-7.2.2/src/include/utils/palloc.h 
Mon
Nov  5 12:46:36 2001
--- palloc.h    Mon Aug 26 21:51:46 2002
***************
*** 45,51 **** /*  * Fundamental memory-allocation operations (more are in utils/memutils.h)  */
! extern void *MemoryContextAlloc(MemoryContext context, Size size);
 #define palloc(sz)    MemoryContextAlloc(CurrentMemoryContext, (sz))

--- 45,52 ---- /*  * Fundamental memory-allocation operations (more are in utils/memutils.h)  */
! extern void *_MemoryContextAlloc(MemoryContext context, Size size, const 
char
*s, int l);
! #define MemoryContextAlloc(context, size) _MemoryContextAlloc(context, 
size, __FILE__, __LINE__)
 #define palloc(sz)    MemoryContextAlloc(CurrentMemoryContext, (sz))


> that changes anything.  One thing that may be different is that NetBSD is
> 64 bit clean.  I don't think the other i386 systems are.
>
> > I eyeballed the chkpass code and didn't see any sign of buffer overruns,
> > but maybe it needs a harder look.
>
> It's pretty simple.  Not even indexing.  In fact, I wondered if I should
> add some just like my other type that does do indexing.  That seemed to be
> the only real difference between the two and the other works.
>
> > Hm --- I guess another possible variable is behavior of the local
> > crypt() function.  Is NetBSD's crypt perhaps willing to return strings
> > longer than 13 chars?
>
> Well, the value that it stores is the correct 13 character DES string.
>
> > Did you try CVS tip both with and without --enable-cassert?  That turns
> > on memory context checking which might alter the failure in interesting
> > ways.
>
> No difference.  It seems that PostgreSQL is too good at catching the
> problem before the assert macros see it.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 28, 2002 02:07 pm, D'Arcy J.M. Cain wrote:
> I had also tried changing palloc.h and mcxt.c to turn MemoryContextAlloc()
> into a macro that called a modified version of the real one to try to
> narrow down where it was being called from but that wouldn't even run.  Is
> there another file I have to modify as well if I try that?  Here are the
> changes I tried.  See any reason that it shouldn't have worked if I rebuilt
> everything?

I forgot to mention the error when I do this.  The server runs but when I try 
to connect to a database I get this:

psql: could not receive server response to SSL negotiation packet: 
Inappropriate ioctl for device

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 28, 2002 09:23 am, Tom Lane wrote:
> The behavior looks a lot like a memory clobber, so perhaps the key
> variable is some difference in malloc's allocation strategy, causing
> two items to be adjacent in NetBSD where they are not on the other
> platforms we've tried.

Here's some other wackiness.  The following is various encrypted passwords 
split for convenience of analyzing them along with the hex version of the big 
number in the error.  The OK ones were ones that didn't trigger the error.  
The first line following that is the last byte in the number in hex and 
binary.  The next is the first character of the second chunk of the password. Note how the integer basically is the 4
bytesof the second chunk except for 
 
the first byte which differs in a somewhat regular way.  Looks like some sort 
of bit mask operation somewhere.

wvx8 42kQ 34jyY (OK)
0x34 00110100

Q/Jz mdRb HSwE. 0x62526471
0x71 01110001
0x6d 01101101

ccIx mriB VsviU 0x42697271
0x71 01110001
0x6d 01101101

wsnr TAub uIelw 0x62754158
0x58 01011000
0x54 01010100

tGep W3d5 EX5pU 0x3564335b
0x5b 01011011
0x57 01010111

gJTk uYzh fb3LM 0x687a5979
0x79 01111001
0x75 01110101

EfFt qWDL RgVjY 0x4c445775
0x75 01110101
0x71 01110001

My2J GCTv 8A3GI 0x7654434b
0x4b 01001011
0x47 01000111

uWPk 7xcQ ZpTi. 0x5163783b
0x3b 00111011
0x37 00110111

AlD5 naNP oDKdc (OK)
0x6e 01101110

>
> I eyeballed the chkpass code and didn't see any sign of buffer overruns,
> but maybe it needs a harder look.

Hmm.  I did give it a harder look and look what jumped out.  Both chkpass_out 
and chkpass_rout return PG_RETURN_CSTRING but chkpass_out builds a standard c 
string while chkpass_rout builds a variable text structure.  That can't be 
right.  It's odd that this always worked before.  It seems to me that 
chkpass_rout should be changed to build a c string like chkpass_out given the 
name of the return macro.  I tried that and it made no difference.  I'm not 
entirely surprised since I never used the chkpass_rout function in any of the 
tests.

Is it possible that my thinking is wrong and I should be creating a text type 
for both?

Still doesn't explain why no one else sees this though.

Oh, one more datapoint - the error only happens on vacuum analyze, not just 
vacuum.  Not sure what that means exactly.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: MemoryContextAlloc: invalid request size 1934906735

От
Tom Lane
Дата:
"D'Arcy J.M. Cain" <darcy@druid.net> writes:
> Hmm.  I did give it a harder look and look what jumped out.  Both
> chkpass_out and chkpass_rout return PG_RETURN_CSTRING but chkpass_out
> builds a standard c string while chkpass_rout builds a variable text
> structure.  That can't be right.

It's not, but chkpass_rout is declared to return text, so it should be
saying PG_RETURN_TEXT_P.  It turns out both macros do the same thing,
so this is just a cosmetic issue.

> Oh, one more datapoint - the error only happens on vacuum analyze, not just 
> vacuum.  Not sure what that means exactly.

That is odd.  You only have the chkpass operators shown in the contrib
module, right?  No "chkpass = chkpass" operator?  Without one, vacuum
analyze should pretty much ignore the chkpass column ...
        regards, tom lane


Re: MemoryContextAlloc: invalid request size 1934906735

От
"D'Arcy J.M. Cain"
Дата:
On August 28, 2002 11:07 pm, Tom Lane wrote:
> "D'Arcy J.M. Cain" <darcy@druid.net> writes:
> > Hmm.  I did give it a harder look and look what jumped out.  Both
> > chkpass_out and chkpass_rout return PG_RETURN_CSTRING but chkpass_out
> > builds a standard c string while chkpass_rout builds a variable text
> > structure.  That can't be right.
>
> It's not, but chkpass_rout is declared to return text, so it should be
> saying PG_RETURN_TEXT_P.  It turns out both macros do the same thing,
> so this is just a cosmetic issue.

OK, I will make that cosmetic change.

> > Oh, one more datapoint - the error only happens on vacuum analyze, not
> > just vacuum.  Not sure what that means exactly.
>
> That is odd.  You only have the chkpass operators shown in the contrib
> module, right?  No "chkpass = chkpass" operator?  Without one, vacuum
> analyze should pretty much ignore the chkpass column ...

YES!  Well, sort of.  I didn't have any other operators but while I thought 
that both were the same (after all, I contributed it) someone must have fixed 
the one in CVS before adding it.  The one I was working with had the 
operators working with chkpass on both sides.  As soon as I fixed that it 
worked again.

In 7.2 the cstring and chkpass types fail in the function definitions because 
they have not been defined so I had to stay with opaque.  In fact, how will 
that work in 7.3 anyway?  We declare the functions to take or return a 
chkpass before we define it.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Type definition process (was Re: MemoryContextAlloc: invalid request size 1934906735)

От
Tom Lane
Дата:
"D'Arcy J.M. Cain" <darcy@druid.net> writes:
> YES!  Well, sort of.  I didn't have any other operators but while I thought 
> that both were the same (after all, I contributed it) someone must have fixed
> the one in CVS before adding it.  The one I was working with had the 
> operators working with chkpass on both sides.  As soon as I fixed that it 
> worked again.

Ah-hah, so vacuum was trying to use the "chkpass = text" operator to
compare two chkpass values.  That explains the whole problem --- the
text code of course would take the first four bytes of the chkpass
string as a length word.

> In 7.2 the cstring and chkpass types fail in the function definitions because
> they have not been defined so I had to stay with opaque.  In fact, how will 
> that work in 7.3 anyway?  We declare the functions to take or return a 
> chkpass before we define it.

Yeah, you'll get warnings about the type not being defined yet, but it
will take them anyway.  There's a fundamental circularity involved in
defining these things with any sort of accuracy, so we're going to have
to live with either warnings or kluges :-(.

I suppose that if the warnings really irritate people, we could think
about exposing the shell-type-entry mechanism more explicitly.  For
example, if you did something like
-- make a shell pg_type entryCREATE TYPE chkpass;
-- make the I/O functionsCREATE FUNCTION chkpass_in(cstring) RETURNS chkpass ...;
CREATE FUNCTION chkpass_out(chkpass) RETURNS cstring ...;
-- replace shell entry with real oneCREATE TYPE chkpass(input = chkpass_in, output = ...);

This looks rather ugly to me but it would be pretty easy to make it
work and not give any warnings.  Comments?
        regards, tom lane


Re: Type definition process (was Re: MemoryContextAlloc: invalid request size 1934906735)

От
"D'Arcy J.M. Cain"
Дата:
On August 29, 2002 09:45 am, Tom Lane wrote:
> "D'Arcy J.M. Cain" <darcy@druid.net> writes:
> > YES!  Well, sort of.  I didn't have any other operators but while I
> > thought that both were the same (after all, I contributed it) someone
> > must have fixed the one in CVS before adding it.  The one I was working
> > with had the operators working with chkpass on both sides.  As soon as I
> > fixed that it worked again.
>
> Ah-hah, so vacuum was trying to use the "chkpass = text" operator to
> compare two chkpass values.  That explains the whole problem --- the
> text code of course would take the first four bytes of the chkpass
> string as a length word.

Exactly.

> > In 7.2 the cstring and chkpass types fail in the function definitions
> > because they have not been defined so I had to stay with opaque.  In
> > fact, how will that work in 7.3 anyway?  We declare the functions to take
> > or return a chkpass before we define it.
>
> Yeah, you'll get warnings about the type not being defined yet, but it
> will take them anyway.  There's a fundamental circularity involved in
> defining these things with any sort of accuracy, so we're going to have
> to live with either warnings or kluges :-(.
>
> I suppose that if the warnings really irritate people, we could think
> about exposing the shell-type-entry mechanism more explicitly.  For
> example, if you did something like
>
>     -- make a shell pg_type entry
>     CREATE TYPE chkpass;
>
>     -- make the I/O functions
>     CREATE FUNCTION chkpass_in(cstring) RETURNS chkpass ...;
>
>     CREATE FUNCTION chkpass_out(chkpass) RETURNS cstring ...;
>
>     -- replace shell entry with real one
>     CREATE TYPE chkpass(input = chkpass_in, output = ...);
>
> This looks rather ugly to me but it would be pretty easy to make it
> work and not give any warnings.  Comments?

Well, magic (DWIM) parsing would be nice but this doesn't seem all that ugly 
to me.  One thing I do see though is that there is a completion issue.  Maybe 
we should specify that this can only happen within a transaction and add some 
code to the transaction handling.  Some simple rules (not to suggest that 
they are necessarily simple to implement of course) I see are;
1. An incomplete CREATE TYPE raises an error if not inside a transaction 
block.2. CREATE TYPE and CREATE FUNCTION will be backed out on an abort.3. Closing a transaction aborts if an
incompletetype has not been completed.
 

I think that this closes the loop without leaving functions defined on 
incomplete types.  With enough clever programming perhaps we can even make 
the incomplete declarion automatic when it is used in the CREATE FUNCTION.  
We just don't raise an error until the COMMIT.
 BEGIN TRANSACTION
 -- an incomplete type "chkpass" is conditionally created here CREATE FUNCTION chkpass_in(cstring) RETURNS chkpass;
 -- the existing conditional type is used here CREATE FUNCTION chkpass_out(chkpass) RETURNS cstring;
 -- define the actual type CREATE TYPE chkpass(input = chkpass_in, output = chkpass_out);
 END TRANSACTION

Does this make sense?

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.


Re: Type definition process (was Re: MemoryContextAlloc: invalid request size 1934906735)

От
Tom Lane
Дата:
"D'Arcy J.M. Cain" <darcy@druid.net> writes:
> One thing I do see though is that there is a completion issue.

Well, (a) the shell type can't be used for anything till you turn it
into a real type, and (b) the completion issue already exists, and has
for a long time; you've always been able to create a shell type by using
a not-yet-known type name as the return type of a function.  It's just
not well documented.

>  1. An incomplete CREATE TYPE raises an error if not inside a transaction 
> block.

I have no intention of implementing this.  (1) It wouldn't really
simplify life anyway, since we'd still need all the same guard code to
prevent you from using the shell type within the creating transaction.
(2) It would break existing pg_dump scripts, which don't know they'd
need to do this.

Wrapping the sequence inside a transaction is a good practice, but
I don't feel that we have to try to force good practice on people.
        regards, tom lane


Re: Type definition process (was Re: MemoryContextAlloc: invalid request size 1934906735)

От
"D'Arcy J.M. Cain"
Дата:
On August 29, 2002 03:37 pm, Tom Lane wrote:
> "D'Arcy J.M. Cain" <darcy@druid.net> writes:
> > One thing I do see though is that there is a completion issue.
>
> Well, (a) the shell type can't be used for anything till you turn it
> into a real type, and (b) the completion issue already exists, and has
> for a long time; you've always been able to create a shell type by using
> a not-yet-known type name as the return type of a function.  It's just
> not well documented.

And gives a warning, right?

> >  1. An incomplete CREATE TYPE raises an error if not inside a transaction
> > block.
>
> I have no intention of implementing this.  (1) It wouldn't really
> simplify life anyway, since we'd still need all the same guard code to
> prevent you from using the shell type within the creating transaction.
> (2) It would break existing pg_dump scripts, which don't know they'd
> need to do this.

Yes, I see your point.

> Wrapping the sequence inside a transaction is a good practice, but
> I don't feel that we have to try to force good practice on people.

OK but how about a little reward if they do.  Do everything as we do now 
except that if they wrap it in a transaction then they don't get the warning 
unless they exit the transaction without completing the type?  Some people 
(e.g. me) like to code as if warnings were as bad as errors.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.