Обсуждение: Re: [ADMIN] Repeatable crash in pg_dump (with -d2 info)
David Schnur <dnschnur@gmail.com> writes:
> I probably can't get a stack trace, but I was able to reproduce it with
> just that function. Without the function, pg_dump works fine. I can DROP
> the function, pg_dump works, then add it back again and pg_dump crashes.
Hmph. I still can't reproduce this here, which seems to mean one of
three things:
1. We fixed this since 8.3.15 (still doesn't seem likely).
2. It's specific to Windows.
3. It's specific to the pg_dump build you are using.
I wonder whether anyone else can get this to happen on the strength of
David's description?
> Here are my steps:
> initdb -A md5 --no-locale -E UTF8 -U testuser -D
> "C:\Users\David\Desktop\testdb" -W
> pg_ctl -U testuser -D "C:\Users\David\Desktop\testdb" start
> CREATE DATABASE testdb OWNER testuser;
> CREATE OR REPLACE FUNCTION datastore_unpack(
> data_times TIMESTAMP WITH TIME ZONE[],
> data_values DOUBLE PRECISION[],
> OUT data_time TIMESTAMP WITH TIME ZONE,
> OUT data_value DOUBLE PRECISION
> ) RETURNS SETOF RECORD AS $$
> SELECT $1[rowx] AS data_time, $2[rowx] AS data_value
> FROM generate_series(1, array_upper($1, 1)) AS rowx;
> $$ LANGUAGE SQL STABLE;
> pg_dump -v -F c -x -O -f "C:\Users\David\Desktop\dumptest" -U testuser
> testdb
> Here's the output from pg_dump:
> pg_dump: reading schemas
> pg_dump: reading user-defined functions
> pg_dump: reading user-defined types
> pg_dump: reading procedural languages
> pg_dump: reading user-defined aggregate functions
> pg_dump: reading user-defined operators
> pg_dump: reading user-defined operator classes
> pg_dump: reading user-defined text search parsers
> pg_dump: reading user-defined text search templates
> pg_dump: reading user-defined text search dictionaries
> pg_dump: reading user-defined text search configurations
> pg_dump: reading user-defined operator families
> pg_dump: reading user-defined conversions
> pg_dump: reading user-defined tables
> pg_dump: reading table inheritance information
> pg_dump: reading rewrite rules
> pg_dump: reading type casts
> pg_dump: finding inheritance relationships
> pg_dump: reading column info for interesting tables
> pg_dump: flagging inherited columns in subtables
> pg_dump: reading indexes
> pg_dump: reading constraints
> pg_dump: reading triggers
> pg_dump: reading dependency data
> pg_dump: saving encoding = UTF8
> pg_dump: saving standard_conforming_strings = off
> pg_dump: saving database definition
> And here's the output from the postmaster:
> LOG: statement: SET DATESTYLE = ISO
> LOG: statement: SET extra_float_digits TO 2
> LOG: statement: SET synchronize_seqscans TO off
> LOG: statement: BEGIN
> LOG: statement: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
> LOG: statement: SET search_path = pg_catalog
> LOG: statement: SELECT tableoid, oid, nspname, (SELECT rolname FROM
> pg_catalog.pg_roles WHERE oid = nspowner) as rolname, nspacl FROM
> pg_namespace
> LOG: statement: SELECT tableoid, oid, proname, prolang, pronargs,
> proargtypes, prorettype, proacl, pronamespace,(SELECT rolname FROM
> pg_catalog.pg_roles WHERE oid = proowner) as rolname FROM pg_proc WHERE NOT
> proisagg AND pronamespace != (select oid from pg_namespace where nspname =
> 'pg_catalog')
> LOG: statement: SELECT tableoid, oid, typname, typnamespace, (SELECT
> rolname FROM pg_catalog.pg_roles WHERE oid = typowner) as rolname,
> typinput::oid as typinput, typoutput::oid as typoutput, typelem, typrelid,
> CASE WHEN typrelid = 0 THEN ' '::"char" ELSE (SELECT relkind FROM pg_class
> WHERE oid = typrelid) END as typrelkind, typtype, typisdefined, typname[0]
> = '_' AND typelem != 0 AND (SELECT typarray FROM pg_type te WHERE oid =
> pg_type.typelem) = oid AS isarray FROM pg_type
> LOG: statement: SELECT tableoid, oid, lanname, lanpltrusted,
> lanplcallfoid, lanvalidator, lanacl, (SELECT rolname FROM
> pg_catalog.pg_roles WHERE oid = lanowner) as lanowner FROM pg_language
> WHERE lanispl ORDER BY oid
> LOG: statement: SELECT tableoid, oid, proname as aggname, pronamespace as
> aggnamespace, pronargs, proargtypes, (SELECT rolname FROM
> pg_catalog.pg_roles WHERE oid = proowner) as rolname, proacl as aggacl FROM
> pg_proc WHERE proisagg AND pronamespace != (select oid from pg_namespace
> where nspname = 'pg_catalog')
> LOG: statement: SELECT tableoid, oid, oprname, oprnamespace, (SELECT
> rolname FROM pg_catalog.pg_roles WHERE oid = oprowner) as rolname,
> oprcode::oid as oprcode FROM pg_operator
> LOG: statement: SELECT tableoid, oid, opcname, opcnamespace, (SELECT
> rolname FROM pg_catalog.pg_roles WHERE oid = opcowner) as rolname FROM
> pg_opclass
> LOG: statement: SELECT tableoid, oid, prsname, prsnamespace,
> prsstart::oid, prstoken::oid, prsend::oid, prsheadline::oid,
> prslextype::oid FROM pg_ts_parser
> LOG: statement: SELECT tableoid, oid, tmplname, tmplnamespace,
> tmplinit::oid, tmpllexize::oid FROM pg_ts_template
> LOG: statement: SELECT tableoid, oid, dictname, dictnamespace, (SELECT
> rolname FROM pg_catalog.pg_roles WHERE oid = dictowner) as rolname,
> dicttemplate, dictinitoption FROM pg_ts_dict
> LOG: statement: SELECT tableoid, oid, cfgname, cfgnamespace, (SELECT
> rolname FROM pg_catalog.pg_roles WHERE oid = cfgowner) as rolname,
> cfgparser FROM pg_ts_config
> LOG: statement: SELECT tableoid, oid, opfname, opfnamespace, (SELECT
> rolname FROM pg_catalog.pg_roles WHERE oid = opfowner) as rolname FROM
> pg_opfamily
> LOG: statement: SELECT tableoid, oid, conname, connamespace, (SELECT
> rolname FROM pg_catalog.pg_roles WHERE oid = conowner) as rolname FROM
> pg_conversion
> LOG: statement: SELECT c.tableoid, c.oid, relname, relacl, relkind,
> relnamespace, (SELECT rolname FROM pg_catalog.pg_roles WHERE oid =
> relowner) as rolname, relchecks, reltriggers, relhasindex, relhasrules,
> relhasoids, d.refobjid as owning_tab, d.refobjsubid as owning_col, (SELECT
> spcname FROM pg_tablespace t WHERE t.oid = c.reltablespace) AS
> reltablespace, array_to_string(c.reloptions, ', ') as reloptions from
> pg_class c left join pg_depend d on (c.relkind = 'S' and d.classid =
> c.tableoid and d.objid = c.oid and d.objsubid = 0 and d.refclassid =
> c.tableoid and d.deptype = 'a') where relkind in ('r', 'S', 'v', 'c') order
> by c.oid
> LOG: statement: SELECT inhrelid, inhparent from pg_inherits
> LOG: statement: SELECT tableoid, oid, rulename, ev_class as ruletable,
> ev_type, is_instead, ev_enabled FROM pg_rewrite ORDER BY oid
> LOG: statement: SELECT tableoid, oid, castsource, casttarget, castfunc,
> castcontext FROM pg_cast ORDER BY 3,4
> LOG: statement: SELECT loid FROM pg_largeobject LIMIT 1
> LOG: statement: SELECT classid, objid, refclassid, refobjid, deptype FROM
> pg_depend WHERE deptype != 'p' ORDER BY 1,2
> LOG: statement: SELECT tableoid, oid, (SELECT rolname FROM
> pg_catalog.pg_roles WHERE oid = datdba) as dba,
> pg_encoding_to_char(encoding) as encoding, (SELECT spcname FROM
> pg_tablespace t WHERE t.oid = dattablespace) as tablespace,
> shobj_description(oid, 'pg_database') as description FROM pg_database WHERE
> datname = 'testdb'
> LOG: statement: SELECT description, classoid, objoid, objsubid FROM
> pg_catalog.pg_description ORDER BY classoid, objoid, objsubid
> LOG: statement: SET search_path = public, pg_catalog
> LOG: statement: SELECT proretset, prosrc, probin, proallargtypes,
> proargmodes, proargnames, provolatile, proisstrict, prosecdef, proconfig,
> procost, prorows, (SELECT lanname FROM pg_catalog.pg_language WHERE oid =
> prolang) as lanname FROM pg_catalog.pg_proc WHERE oid =
> '16386'::pg_catalog.oid
> LOG: statement: SELECT pg_catalog.format_type('1185'::pg_catalog.oid, NULL)
> LOG: statement: SELECT pg_catalog.format_type('1022'::pg_catalog.oid, NULL)
> LOG: statement: SELECT pg_catalog.format_type('1184'::pg_catalog.oid, NULL)
> LOG: statement: SELECT pg_catalog.format_type('701'::pg_catalog.oid, NULL)
> LOG: statement: SELECT pg_catalog.format_type('1185'::pg_catalog.oid, NULL)
> LOG: statement: SELECT pg_catalog.format_type('1022'::pg_catalog.oid, NULL)
> LOG: statement: SELECT pg_catalog.format_type('2249'::pg_catalog.oid, NULL)
> LOG: could not receive data from client: No connection could be made
> because the target machine actively refused it.
> LOG: unexpected EOF on client connection
regards, tom lane
I finally had time to test this further on a variety of systems, and was unable to reproduce on any non-Windows platform. The dump even works fine on Windows XP; just not Windows 7.
This prompted me to do a little more research, and this time I found this thread from Sept. 2011:
From Tom Lane in the above thread:
Hmm. I can see how that would happen if you're using one of the Windows
environments wherein malloc's done inside libpq have to be free'd inside
libpq. (The PQExpBuffer support code is in libpq...)
However, the flaw in that explanation is that it would basically mean
pg_dump doesn't work at all on Windows, at least not if you have any
user-defined functions, and probably some other cases too because there
seem to be multiple instances of the dubious coding. It's a bit hard to
believe that nobody's noticed that before.
This appears to describe exactly the issue I'm encountering, and my build is in fact linked against the static runtime. I guess the reason this hasn't come up sooner is because most Windows users either use the 'official' binaries rather than compiling from source, or link against the dynamic runtime.
Is this something I could expect to be fixed in the near future, or is it enough of an edge case that I should come up with some solution or work-around on my own? Thanks,
David
On Mon, Nov 28, 2011 at 8:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Hmph. I still can't reproduce this here, which seems to mean one ofthree things:
1. We fixed this since 8.3.15 (still doesn't seem likely).
2. It's specific to Windows.
3. It's specific to the pg_dump build you are using.
On Tue, Jan 17, 2012 at 04:46:50PM -0500, David Schnur wrote: > I finally had time to test this further on a variety of systems, and was unable > to reproduce on any non-Windows platform. The dump even works fine on Windows > XP; just not Windows 7. > > This prompted me to do a little more research, and this time I found this > thread from Sept. 2011: > > http://postgresql.1045698.n5.nabble.com/ > BUG-6233-pg-dump-hangs-with-Access-Violation-C0000005-td4851598.html > > From Tom Lane in the above thread: > > > Hmm. I can see how that would happen if you're using one of the Windows > environments wherein malloc's done inside libpq have to be free'd inside > libpq. (The PQExpBuffer support code is in libpq...) > > > > However, the flaw in that explanation is that it would basically mean > pg_dump doesn't work at all on Windows, at least not if you have any > user-defined functions, and probably some other cases too because there > seem to be multiple instances of the dubious coding. It's a bit hard to > believe that nobody's noticed that before. > > > This appears to describe exactly the issue I'm encountering, and my build is in > fact linked against the static runtime. I guess the reason this hasn't come up > sooner is because most Windows users either use the 'official' binaries rather > than compiling from source, or link against the dynamic runtime. > > Is this something I could expect to be fixed in the near future, or is it > enough of an edge case that I should come up with some solution or work-around > on my own? Thanks, Late reply, but I don't see any way we could fix this easily. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
On Mon, Aug 27, 2012 at 9:58 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Jan 17, 2012 at 04:46:50PM -0500, David Schnur wrote: >> I finally had time to test this further on a variety of systems, and was unable >> to reproduce on any non-Windows platform. The dump even works fine on Windows >> XP; just not Windows 7. >> >> This prompted me to do a little more research, and this time I found this >> thread from Sept. 2011: >> >> http://postgresql.1045698.n5.nabble.com/ >> BUG-6233-pg-dump-hangs-with-Access-Violation-C0000005-td4851598.html >> >> From Tom Lane in the above thread: >> >> >> Hmm. I can see how that would happen if you're using one of the Windows >> environments wherein malloc's done inside libpq have to be free'd inside >> libpq. (The PQExpBuffer support code is in libpq...) >> >> >> >> However, the flaw in that explanation is that it would basically mean >> pg_dump doesn't work at all on Windows, at least not if you have any >> user-defined functions, and probably some other cases too because there >> seem to be multiple instances of the dubious coding. It's a bit hard to >> believe that nobody's noticed that before. >> >> >> This appears to describe exactly the issue I'm encountering, and my build is in >> fact linked against the static runtime. I guess the reason this hasn't come up >> sooner is because most Windows users either use the 'official' binaries rather >> than compiling from source, or link against the dynamic runtime. >> >> Is this something I could expect to be fixed in the near future, or is it >> enough of an edge case that I should come up with some solution or work-around >> on my own? Thanks, > > Late reply, but I don't see any way we could fix this easily. To me it seems like mostly a case of chasing down all the places where this happens. It's not impossible to do; it's just a bunch of work that nobody's gotten excited about doing yet. We've fixed similar issues in many other cases, IIUC. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
> On Mon, Aug 27, 2012 at 9:58 AM, Bruce Momjian <bruce@momjian.us> wrote:
>> From Tom Lane in the above thread:
>>> Hmm. I can see how that would happen if you're using one of the Windows
>>> environments wherein malloc's done inside libpq have to be free'd inside
>>> libpq. (The PQExpBuffer support code is in libpq...)
>> Late reply, but I don't see any way we could fix this easily.
> To me it seems like mostly a case of chasing down all the places where
> this happens. It's not impossible to do; it's just a bunch of work
> that nobody's gotten excited about doing yet. We've fixed similar
> issues in many other cases, IIUC.
Well, the problem with what I suspect you're thinking of is that even
after we fixed all the existing trouble spots, it would keep on
breaking. Unless we found some mechanical way to warn about unsafe
coding; which in itself would be more work than I want to put into this.
However, a plan B occurs to me: what about preventing pg_dump from using
libpq's copy of PQExpBuffer? IIUC, a copy of the PQExpBuffer code
statically linked into pg_dump would be free of this problem. There's
not so much code there that this would be an intolerable price to pay.
(Besides, we could probably arrange for the extra copy to happen only on
Windows.)
regards, tom lane