Обсуждение: Re: Re: pg_restore fails on Windows
Magnus Hagander wrote > Tom Lane wrote: > > =?us-ascii?Q?Tom=20Tom?= <cobold@seznam.cz> writes: > >> Magnus Hagander wrote: > >>> Attached is a pg_restore.exe off CVS tip today, which should include the > >>> patch. Please try this one. > > > >> I tested the restore using the provided pg_restore.exe. The output is: > > > >> pg_restore: [archiver (db)] could not execute query: could not send data to > server: No buffer space available (0x00002747/10055) > > > > According to > > http://support.microsoft.com/kb/201213 > > this is an acknowledged bug that's been broken since Windows 95, so > > I suppose we should conclude that M$ is unwilling or incompetent to > > fix it. > > Yup, I was just reading that one when I saw your email. I finally got > around to building a libpq with this change in it - attached here. Tom > (not Lane), can you test this please? > > It shouldn't be this one really, since it doesn't list any modern > Windows versions as having this issue, but it's worth a try. Tested. The restore comes through successfuly with the patched libpq. So I take it that it's caused by the MS issue. Again, we are using WinXP Professional SP2. Perhaps the system buffer space was _increased_ in XP (10MB comes through easily), still if the block is too large, it occurs (speculation). Since I don't know the implementation details of the patch I'd like to ask: 1.This is not official patch, didn't pass the review/test cycle; do you think that it can be used in the production environment (any side effects or so..)? If not, is the patch due for a next version? 2.Our production PG version is 8.1.3. For some reasons it is not possible to upgrade to the LATEST; I tested the libpq also on this version and it worked. Is it OK? I mean, did it worked by chance or the library API & contracts didn't change between this version and latest? > > One other thing not asked here btw - is this over SSL or not? SSL is out of game, we are not using it. > > > > Possibly the best workaround is something like > > > > + #ifndef WIN32 > > sent = pqsecure_write(conn, ptr, len); > > + #else > > + /* Windows tends to fail on large sends, see KB 20213 */ > > + sent = pqsecure_write(conn, ptr, Min(len, 65536)); > > + #endif > > > > in pqSendSome(). The backend seems to not be subject to a similar > > problem because it's already filtering its output through a limited-size > > buffer. > > The other option would be to turn off buffering in the sockets per the > third bulletpoint. But I think that's an even uglier solution. > Regards, Tomas
Tom Tom wrote: > Magnus Hagander wrote >> Tom Lane wrote: >>> =?us-ascii?Q?Tom=20Tom?= <cobold@seznam.cz> writes: >>>> Magnus Hagander wrote: >>>>> Attached is a pg_restore.exe off CVS tip today, which should include the >>>>> patch. Please try this one. >>>> I tested the restore using the provided pg_restore.exe. The output is: >>>> pg_restore: [archiver (db)] could not execute query: could not send data to >> server: No buffer space available (0x00002747/10055) >>> According to >>> http://support.microsoft.com/kb/201213 >>> this is an acknowledged bug that's been broken since Windows 95, so >>> I suppose we should conclude that M$ is unwilling or incompetent to >>> fix it. >> Yup, I was just reading that one when I saw your email. I finally got >> around to building a libpq with this change in it - attached here. Tom >> (not Lane), can you test this please? >> >> It shouldn't be this one really, since it doesn't list any modern >> Windows versions as having this issue, but it's worth a try. > > Tested. The restore comes through successfuly with the patched libpq. > So I take it that it's caused by the MS issue. Again, we are using WinXP Professional SP2. Perhaps the > system buffer space was _increased_ in XP (10MB comes through easily), > still if the block is too large, it occurs (speculation). Yes, that sounds quite likely. They fixed the symptoms, but not the underlying problem. > Since I don't know the implementation details of the patch I'd like to ask: > 1.This is not official patch, didn't pass the review/test cycle; do you think that it can be used in the > production environment (any side effects or so..)? If not, is the patch due for a next version? I plan to apply it to HEAD and supported back-branches (8.3 and 8.2) now that you have verified that it works, so it will be in the next versions. The only potential side-effect is that it will be slightly slower on packets >64kb, but I doubt that's even measurable in most cases. So yes, it should be safe to use in production. > 2.Our production PG version is 8.1.3. For some reasons it is not possible to upgrade to the LATEST; > I tested the libpq also on this version and it worked. Is it OK? I mean, did it worked by chance or the library > API & contracts didn't change between this version and latest? Note that libpq is only the *client* side. There is no patch necessary on the server. It might be easier to upgrade than the server? Did you test it with the pg_restore that I sent, or with the one from 8.1? The pg_restore I sent was for HEAD, as well as the libpq I sent, so you shouldn't use those in production long-term. For binaries, we don't provide backpatches for 8.1 any more (it's not a supported platform on Windows!), but you might be able to use the latest 8.2 libpq with the 8.1 pg_restore - you'll have to try that once the release is eventually out. Or you can just apply the patch to the latest 8.1 libpq and build it yourself, of course. I think it should apply just fine. //Magnus
Magnus Hagander wrote > Tom Tom wrote: > > Magnus Hagander wrote > >> Tom Lane wrote: > >>> =?us-ascii?Q?Tom=20Tom?= <cobold@seznam.cz> writes: > >>>> Magnus Hagander wrote: > >>>>> Attached is a pg_restore.exe off CVS tip today, which should include the > >>>>> patch. Please try this one. > >>>> I tested the restore using the provided pg_restore.exe. The output is: > >>>> pg_restore: [archiver (db)] could not execute query: could not send data > to > >> server: No buffer space available (0x00002747/10055) > >>> According to > >>> http://support.microsoft.com/kb/201213 > >>> this is an acknowledged bug that's been broken since Windows 95, so > >>> I suppose we should conclude that M$ is unwilling or incompetent to > >>> fix it. > >> Yup, I was just reading that one when I saw your email. I finally got > >> around to building a libpq with this change in it - attached here. Tom > >> (not Lane), can you test this please? > >> > >> It shouldn't be this one really, since it doesn't list any modern > >> Windows versions as having this issue, but it's worth a try. > > > > Tested. The restore comes through successfuly with the patched libpq. > > So I take it that it's caused by the MS issue. Again, we are using WinXP > Professional SP2. Perhaps the > > system buffer space was _increased_ in XP (10MB comes through easily), > > still if the block is too large, it occurs (speculation). > > Yes, that sounds quite likely. They fixed the symptoms, but not the > underlying problem. > > > > Since I don't know the implementation details of the patch I'd like to ask: > > 1.This is not official patch, didn't pass the review/test cycle; do you think > that it can be used in the > > production environment (any side effects or so..)? If not, is the patch due > for a next version? > > I plan to apply it to HEAD and supported back-branches (8.3 and 8.2) now > that you have verified that it works, so it will be in the next > versions. The only potential side-effect is that it will be slightly > slower on packets >64kb, but I doubt that's even measurable in most cases. > > So yes, it should be safe to use in production. > > > > 2.Our production PG version is 8.1.3. For some reasons it is not possible to > upgrade to the LATEST; > > I tested the libpq also on this version and it worked. Is it OK? I mean, did > it worked by chance or the library > > API & contracts didn't change between this version and latest? > > Note that libpq is only the *client* side. There is no patch necessary > on the server. It might be easier to upgrade than the server? This I didn't know/realize. It's good enough for us to use only the *client* side from the HEAD. I tried the pg_restore from HEAD + patched libpq (on 8.1 installation) and it complained about missing zlib1 library. When supplied, next was libintl3 dll. Further I didn't check. Obviously the library dependencies have changed since the 8.1. How can I tell, which libraries/executables/resources of the installation are part of the *client* side (namely pg_restore), so that I can use it independently from the server version? > > Did you test it with the pg_restore that I sent, or with the one from > 8.1? The pg_restore I sent was for HEAD, as well as the libpq I sent, so > you shouldn't use those in production long-term. > > For binaries, we don't provide backpatches for 8.1 any more (it's not a > supported platform on Windows!), but you might be able to use the latest > 8.2 libpq with the 8.1 pg_restore - you'll have to try that once the > release is eventually out. > > Or you can just apply the patch to the latest 8.1 libpq and build it > yourself, of course. I think it should apply just fine. > Tomas
>>> 2.Our production PG version is 8.1.3. For some reasons it is not possible to >> upgrade to the LATEST; >>> I tested the libpq also on this version and it worked. Is it OK? I mean, did >> it worked by chance or the library >>> API & contracts didn't change between this version and latest? >> Note that libpq is only the *client* side. There is no patch necessary >> on the server. It might be easier to upgrade than the server? > > This I didn't know/realize. It's good enough for us to use only the *client* side from the HEAD. > I tried the pg_restore from HEAD + patched libpq (on 8.1 installation) and it complained about missing zlib1 library. When > supplied, next was libintl3 dll. Further I didn't check. Obviously the library dependencies have changed since the 8.1. > How can I tell, which libraries/executables/resources of the installation are part of the *client* side (namely pg_restore), > so that I can use it independently from the server version? That is correct, there have been a lot of changes there - mainly in that 8.3 and up are compiled with MSVC which changes some dependencies. I would suggest you install the latest 8.3 from the MSI installer. You can then select the client side only. It is no problem to have both 8.1 and 8.3 installed on the same machine at the same time (or you can install it on a different machine and just copy the necessary files over) //Magnus
Magnus Hagander wrote: > >>> 2.Our production PG version is 8.1.3. For some reasons it is not possible > to > >> upgrade to the LATEST; > >>> I tested the libpq also on this version and it worked. Is it OK? I mean, > did > >> it worked by chance or the library > >>> API & contracts didn't change between this version and latest? > >> Note that libpq is only the *client* side. There is no patch necessary > >> on the server. It might be easier to upgrade than the server? > > > > This I didn't know/realize. It's good enough for us to use only the *client* > side from the HEAD. > > I tried the pg_restore from HEAD + patched libpq (on 8.1 installation) and it > complained about missing zlib1 library. When > > supplied, next was libintl3 dll. Further I didn't check. Obviously the library > dependencies have changed since the 8.1. > > How can I tell, which libraries/executables/resources of the installation are > part of the *client* side (namely pg_restore), > > so that I can use it independently from the server version? > > That is correct, there have been a lot of changes there - mainly in that > 8.3 and up are compiled with MSVC which changes some dependencies. I > would suggest you install the latest 8.3 from the MSI installer. You can > then select the client side only. It is no problem to have both 8.1 and > 8.3 installed on the same machine at the same time (or you can install > it on a different machine and just copy the necessary files over) > OK, that is fine. Thank you for your support, we are happy with the provided solution. You are doing a great job. Tomas