Обсуждение: LIBPQ patches ...
Does anyone have anything against me applying this to the current source
tree?
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
---------- Forwarded message ----------
Date: Fri, 17 Dec 1999 13:51:50 -0800 (PST)
From: Alfred Perlstein <bright@wintelcom.net>
To: The Hermit Hacker <scrappy@hub.org>
Subject: Re: pctrackd updates and such
On Fri, 17 Dec 1999, The Hermit Hacker wrote:
>
> Okay, first thing...can you redo these as context diffs? We generally
> refuse *any* patches that aren't context...
sure.
>
> Second, are these against a reasonably current snapshot of PostgreSQL
> (aka. the upcoming v7), or v6.5.3 release? If v6.5.3, we're gonna need to
> get these v7.x ready before we can commit them...
they are against a checked out cvs copy as of a couple days ago,
and should apply cleanly to what's in the current repo.
> Once both of the above conditions are in place, and after I get back from
> BC, I'll work on getting these into the v7.0 release...or, at least,
> talked/commented about if there are any objections...
>
> I'm outta here for 10 days...Happy Holidays and talk with ya when I get
> back...
ok, cool see you soon. :)
-Alfred
don't forget the problem with sending queries that may occur:
i'm not sure if handlesendfailure() can cope with only sending
a 'Q' to the backend, we may have to work out reservations or
something for space, another idea would be to implement a
pqWritev() of some sort that would take an array of pointers
and lengths to send to the backend and only allow any data to
go into the backend if the entire string can fit.
then again, handlesendfailure may work, but doing reservations
for the send buffer seems cleaner...
diff's contexted against pgsql-'current':
Index: fe-connect.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.108
diff -u -c -r1.108 fe-connect.c
cvs diff: conflicting specifications of output style
*** fe-connect.c 1999/12/02 00:26:15 1.108
--- fe-connect.c 1999/12/14 09:42:24
***************
*** 595,625 **** return 0; }
-
- /* ----------
- * connectMakeNonblocking -
- * Make a connection non-blocking.
- * Returns 1 if successful, 0 if not.
- * ----------
- */
- static int
- connectMakeNonblocking(PGconn *conn)
- {
- #ifndef WIN32
- if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
- #else
- if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
- #endif
- {
- printfPQExpBuffer(&conn->errorMessage,
- "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
- errno, strerror(errno));
- return 0;
- }
-
- return 1;
- }
- /* ---------- * connectNoDelay - * Sets the TCP_NODELAY socket option.
--- 595,600 ----
***************
*** 792,798 **** * Ewan Mellor <eem21@cam.ac.uk>. * ---------- */ #if (!defined(WIN32) ||
defined(WIN32_NON_BLOCKING_CONNECTIONS))&& !defined(USE_SSL)
! if (!connectMakeNonblocking(conn)) goto connect_errReturn; #endif
--- 767,773 ---- * Ewan Mellor <eem21@cam.ac.uk>. * ---------- */ #if (!defined(WIN32) ||
defined(WIN32_NON_BLOCKING_CONNECTIONS))&& !defined(USE_SSL)
! if (PQsetnonblocking(conn, TRUE) != 0) goto connect_errReturn; #endif
***************
*** 904,910 **** /* This makes the connection non-blocking, for all those cases which forced us not to do it
above.*/ #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! if (!connectMakeNonblocking(conn)) goto connect_errReturn; #endif
--- 879,885 ---- /* This makes the connection non-blocking, for all those cases which forced us not to do it
above.*/ #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! if (PQsetnonblocking(conn, TRUE) != 0) goto connect_errReturn; #endif
***************
*** 1702,1707 ****
--- 1677,1683 ---- conn->inBuffer = (char *) malloc(conn->inBufSize); conn->outBufSize = 8 * 1024;
conn->outBuffer= (char *) malloc(conn->outBufSize);
+ conn->nonblocking = FALSE; initPQExpBuffer(&conn->errorMessage); initPQExpBuffer(&conn->workBuffer);
if(conn->inBuffer == NULL ||
***************
*** 1811,1816 ****
--- 1787,1793 ---- conn->lobjfuncs = NULL; conn->inStart = conn->inCursor = conn->inEnd = 0; conn->outCount
=0;
+ conn->nonblocking = FALSE; }
Index: fe-exec.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.86
diff -u -c -r1.86 fe-exec.c
cvs diff: conflicting specifications of output style
*** fe-exec.c 1999/11/11 00:10:14 1.86
--- fe-exec.c 1999/12/14 05:55:11
***************
*** 13,18 ****
--- 13,19 ---- */ #include <errno.h> #include <ctype.h>
+ #include <fcntl.h> #include "postgres.h" #include "libpq-fe.h"
***************
*** 24,30 **** #include <unistd.h> #endif
- /* keep this in same order as ExecStatusType in libpq-fe.h */ const char *const pgresStatus[] = {
"PGRES_EMPTY_QUERY",
--- 25,30 ----
***************
*** 574,580 ****
--- 574,588 ---- * we will NOT block waiting for more input. */ if (pqReadData(conn) < 0)
+ {
+ /*
+ * try to flush the send-queue otherwise we may never get a
+ * resonce for something that may not have already been sent
+ * because it's in our write buffer!
+ */
+ pqFlush(conn); return 0;
+ } /* Parsing of the data waits till later. */ return 1; }
***************
*** 1088,1095 ****
--- 1096,1112 ---- { PGresult *result; PGresult *lastResult;
+ bool savedblocking; /*
+ * we assume anyone calling PQexec wants blocking behaviour,
+ * we force the blocking status of the connection to blocking
+ * for the duration of this function and restore it on return
+ */
+ savedblocking = PQisnonblocking(conn);
+ PQsetnonblocking(conn, FALSE);
+
+ /* * Silently discard any prior query result that application didn't * eat. This is probably poor
design,but it's here for backward * compatibility.
***************
*** 1102,1115 **** PQclear(result); printfPQExpBuffer(&conn->errorMessage,
"PQexec:you gotta get out of a COPY state yourself.\n");
! return NULL; } PQclear(result); } /* OK to send the message */ if
(!PQsendQuery(conn,query))
! return NULL; /* * For backwards compatibility, return the last result if there are
--- 1119,1133 ---- PQclear(result); printfPQExpBuffer(&conn->errorMessage,
"PQexec:you gotta get out of a COPY state yourself.\n");
! /* restore blocking status */
! goto errout; } PQclear(result); } /* OK to send the message */ if
(!PQsendQuery(conn,query))
! goto errout; /* * For backwards compatibility, return the last result if there are
***************
*** 1142,1148 ****
--- 1160,1172 ---- result->resultStatus == PGRES_COPY_OUT) break; }
+
+ PQsetnonblocking(conn, savedblocking); return lastResult;
+
+ errout:
+ PQsetnonblocking(conn, savedblocking);
+ return NULL; }
***************
*** 1431,1438 **** "PQendcopy() -- I don't think there's a copy in progress.\n"); return 1; }
! (void) pqFlush(conn); /* make sure no data is waiting to be sent */ /* Return to active duty */
conn->asyncStatus= PGASYNC_BUSY;
--- 1455,1468 ---- "PQendcopy() -- I don't think there's a copy in progress.\n"); return 1; }
+
+ /* make sure no data is waiting to be sent */
+ if (pqFlush(conn))
+ return (1);
! /* non blocking connections may have to abort at this point. */
! if (PQisnonblocking(conn) && PQisBusy(conn))
! return (1); /* Return to active duty */ conn->asyncStatus = PGASYNC_BUSY;
***************
*** 2025,2028 ****
--- 2055,2126 ---- return 1; else return 0;
+ }
+
+ /* PQsetnonblocking:
+ sets the PGconn's database connection non-blocking if the arg is TRUE
+ or makes it non-blocking if the arg is FALSE, this will not protect
+ you from PQexec(), you'll only be safe when using the non-blocking
+ API
+ Needs to be called only on a connected database connection.
+ */
+
+ int
+ PQsetnonblocking(PGconn *conn, int arg)
+ {
+ int fcntlarg;
+
+ arg = (arg == TRUE) ? 1 : 0;
+ if (arg == conn->nonblocking)
+ return (0);
+
+ #ifdef USE_SSL
+ if (conn->ssl)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "PQsetnonblocking() -- not supported when using SSL\n");
+ return (-1);
+ }
+ #endif /* USE_SSL */
+
+ #ifndef WIN32
+ fcntlarg = fcntl(conn->sock, F_GETFL, 0);
+ if (fcntlarg == -1)
+ return (-1);
+
+ if ((arg == TRUE &&
+ fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
+ (arg == FALSE &&
+ fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1))
+ #else
+ fcntlarg = arg;
+ if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
+ #endif
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "PQsetblocking() -- unable to set nonblocking status to %s\n",
+ arg == TRUE ? "TRUE" : "FALSE");
+ return (-1);
+ }
+
+ conn->nonblocking = arg;
+ return (0);
+ }
+
+ /* return the blocking status of the database connection, TRUE == nonblocking,
+ FALSE == blocking
+ */
+ int
+ PQisnonblocking(PGconn *conn)
+ {
+
+ return (conn->nonblocking);
+ }
+
+ /* try to force data out, really only useful for non-blocking users */
+ int
+ PQflush(PGconn *conn)
+ {
+
+ return (pqFlush(conn)); }
Index: fe-misc.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
retrieving revision 1.33
diff -u -c -r1.33 fe-misc.c
cvs diff: conflicting specifications of output style
*** fe-misc.c 1999/11/30 03:08:19 1.33
--- fe-misc.c 1999/12/14 08:21:09
***************
*** 86,91 ****
--- 86,119 ---- { size_t avail = Max(conn->outBufSize - conn->outCount, 0);
+ /*
+ * if we are non-blocking and the send queue is too full to buffer this
+ * request then try to flush some and return an error
+ */
+ if (PQisnonblocking(conn) && nbytes > avail && pqFlush(conn))
+ {
+ /*
+ * even if the flush failed we may still have written some
+ * data, recalculate the size of the send-queue relative
+ * to the amount we have to send, we may be able to queue it
+ * afterall even though it's not sent to the database it's
+ * ok, any routines that check the data coming from the
+ * database better call pqFlush() anyway.
+ */
+ if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "pqPutBytes -- pqFlush couldn't flush enough"
+ " data: space available: %d, space needed %d\n",
+ Max(conn->outBufSize - conn->outCount, 0), nbytes);
+ return EOF;
+ }
+ }
+
+ /*
+ * the non-blocking code above makes sure that this isn't true,
+ * essentially this is no-op
+ */ while (nbytes > avail) { memcpy(conn->outBuffer + conn->outCount, s, avail);
***************
*** 548,553 ****
--- 576,589 ---- return EOF; }
+ /*
+ * don't try to send zero data, allows us to use this function
+ * without too much worry about overhead
+ */
+ if (len == 0)
+ return (0);
+
+ /* while there's still data to send */ while (len > 0) { /* Prevent being SIGPIPEd if backend has
closedthe connection. */
***************
*** 556,561 ****
--- 592,598 ---- #endif int sent;
+ #ifdef USE_SSL if (conn->ssl) sent = SSL_write(conn->ssl, ptr, len);
***************
*** 585,590 ****
--- 622,629 ---- case EWOULDBLOCK: break; #endif
+ case EINTR:
+ continue; case EPIPE: #ifdef ECONNRESET
***************
*** 616,628 **** ptr += sent; len -= sent; } if (len > 0) {
/*We didn't send it all, wait till we can send more */
- /* At first glance this looks as though it should block. I think
- * that it will be OK though, as long as the socket is
- * non-blocking. */ if (pqWait(FALSE, TRUE, conn)) return EOF; }
--- 655,685 ---- ptr += sent; len -= sent; }
+ if (len > 0) { /* We didn't send it all, wait till we can send more */
+
+ /*
+ * if the socket is in non-blocking mode we may need
+ * to abort here
+ */
+ #ifdef USE_SSL
+ /* can't do anything for our SSL users yet */
+ if (conn->ssl == NULL)
+ {
+ #endif
+ if (PQisnonblocking(conn))
+ {
+ /* shift the contents of the buffer */
+ memmove(conn->outBuffer, ptr, len);
+ conn->outCount = len;
+ return EOF;
+ }
+ #ifdef USE_SSL
+ }
+ #endif if (pqWait(FALSE, TRUE, conn)) return EOF; }
Index: libpq-fe.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.53
diff -u -c -r1.53 libpq-fe.h
cvs diff: conflicting specifications of output style
*** libpq-fe.h 1999/11/30 03:08:19 1.53
--- libpq-fe.h 1999/12/14 01:30:01
***************
*** 269,274 ****
--- 269,281 ---- extern int PQputnbytes(PGconn *conn, const char *buffer, int nbytes); extern int
PQendcopy(PGconn*conn);
+ /* Set blocking/nonblocking connection to the backend */
+ extern int PQsetnonblocking(PGconn *conn, int arg);
+ extern int PQisnonblocking(PGconn *conn);
+
+ /* Force the write buffer to be written (or at least try) */
+ extern int PQflush(PGconn *conn);
+ /* * "Fast path" interface --- not really recommended for application * use
Index: libpq-int.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
retrieving revision 1.14
diff -u -c -r1.14 libpq-int.h
cvs diff: conflicting specifications of output style
*** libpq-int.h 1999/11/30 03:08:19 1.14
--- libpq-int.h 1999/12/14 01:30:01
***************
*** 215,220 ****
--- 215,223 ---- int inEnd; /* offset to first position after avail
* data */
+ int nonblocking; /* whether this connection is using a blocking
+ * socket to the backend or not */
+ /* Buffer for data not yet sent to backend */ char *outBuffer; /* currently allocated buffer */
int outBufSize; /* allocated size of buffer */
Looks fine. I have talked to someone about doing no-blocking
connections in the past. Maybe this the same person.
I will let someone else comment on whether the protocol changes are
correct.
>
> Does anyone have anything against me applying this to the current source
> tree?
>
>
> Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
> Systems Administrator @ hub.org
> primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
>
> ---------- Forwarded message ----------
> Date: Fri, 17 Dec 1999 13:51:50 -0800 (PST)
> From: Alfred Perlstein <bright@wintelcom.net>
> To: The Hermit Hacker <scrappy@hub.org>
> Subject: Re: pctrackd updates and such
>
> On Fri, 17 Dec 1999, The Hermit Hacker wrote:
>
> >
> > Okay, first thing...can you redo these as context diffs? We generally
> > refuse *any* patches that aren't context...
>
> sure.
>
> >
> > Second, are these against a reasonably current snapshot of PostgreSQL
> > (aka. the upcoming v7), or v6.5.3 release? If v6.5.3, we're gonna need to
> > get these v7.x ready before we can commit them...
>
> they are against a checked out cvs copy as of a couple days ago,
> and should apply cleanly to what's in the current repo.
>
> > Once both of the above conditions are in place, and after I get back from
> > BC, I'll work on getting these into the v7.0 release...or, at least,
> > talked/commented about if there are any objections...
> >
> > I'm outta here for 10 days...Happy Holidays and talk with ya when I get
> > back...
>
> ok, cool see you soon. :)
>
> -Alfred
>
> don't forget the problem with sending queries that may occur:
>
> i'm not sure if handlesendfailure() can cope with only sending
> a 'Q' to the backend, we may have to work out reservations or
> something for space, another idea would be to implement a
> pqWritev() of some sort that would take an array of pointers
> and lengths to send to the backend and only allow any data to
> go into the backend if the entire string can fit.
>
> then again, handlesendfailure may work, but doing reservations
> for the send buffer seems cleaner...
>
> diff's contexted against pgsql-'current':
>
>
> Index: fe-connect.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
> retrieving revision 1.108
> diff -u -c -r1.108 fe-connect.c
> cvs diff: conflicting specifications of output style
> *** fe-connect.c 1999/12/02 00:26:15 1.108
> --- fe-connect.c 1999/12/14 09:42:24
> ***************
> *** 595,625 ****
> return 0;
> }
>
> -
> - /* ----------
> - * connectMakeNonblocking -
> - * Make a connection non-blocking.
> - * Returns 1 if successful, 0 if not.
> - * ----------
> - */
> - static int
> - connectMakeNonblocking(PGconn *conn)
> - {
> - #ifndef WIN32
> - if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
> - #else
> - if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
> - #endif
> - {
> - printfPQExpBuffer(&conn->errorMessage,
> - "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
> - errno, strerror(errno));
> - return 0;
> - }
> -
> - return 1;
> - }
> -
> /* ----------
> * connectNoDelay -
> * Sets the TCP_NODELAY socket option.
> --- 595,600 ----
> ***************
> *** 792,798 ****
> * Ewan Mellor <eem21@cam.ac.uk>.
> * ---------- */
> #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! if (!connectMakeNonblocking(conn))
> goto connect_errReturn;
> #endif
>
> --- 767,773 ----
> * Ewan Mellor <eem21@cam.ac.uk>.
> * ---------- */
> #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! if (PQsetnonblocking(conn, TRUE) != 0)
> goto connect_errReturn;
> #endif
>
> ***************
> *** 904,910 ****
> /* This makes the connection non-blocking, for all those cases which forced us
> not to do it above. */
> #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! if (!connectMakeNonblocking(conn))
> goto connect_errReturn;
> #endif
>
> --- 879,885 ----
> /* This makes the connection non-blocking, for all those cases which forced us
> not to do it above. */
> #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! if (PQsetnonblocking(conn, TRUE) != 0)
> goto connect_errReturn;
> #endif
>
> ***************
> *** 1702,1707 ****
> --- 1677,1683 ----
> conn->inBuffer = (char *) malloc(conn->inBufSize);
> conn->outBufSize = 8 * 1024;
> conn->outBuffer = (char *) malloc(conn->outBufSize);
> + conn->nonblocking = FALSE;
> initPQExpBuffer(&conn->errorMessage);
> initPQExpBuffer(&conn->workBuffer);
> if (conn->inBuffer == NULL ||
> ***************
> *** 1811,1816 ****
> --- 1787,1793 ----
> conn->lobjfuncs = NULL;
> conn->inStart = conn->inCursor = conn->inEnd = 0;
> conn->outCount = 0;
> + conn->nonblocking = FALSE;
>
> }
>
> Index: fe-exec.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
> retrieving revision 1.86
> diff -u -c -r1.86 fe-exec.c
> cvs diff: conflicting specifications of output style
> *** fe-exec.c 1999/11/11 00:10:14 1.86
> --- fe-exec.c 1999/12/14 05:55:11
> ***************
> *** 13,18 ****
> --- 13,19 ----
> */
> #include <errno.h>
> #include <ctype.h>
> + #include <fcntl.h>
>
> #include "postgres.h"
> #include "libpq-fe.h"
> ***************
> *** 24,30 ****
> #include <unistd.h>
> #endif
>
> -
> /* keep this in same order as ExecStatusType in libpq-fe.h */
> const char *const pgresStatus[] = {
> "PGRES_EMPTY_QUERY",
> --- 25,30 ----
> ***************
> *** 574,580 ****
> --- 574,588 ----
> * we will NOT block waiting for more input.
> */
> if (pqReadData(conn) < 0)
> + {
> + /*
> + * try to flush the send-queue otherwise we may never get a
> + * resonce for something that may not have already been sent
> + * because it's in our write buffer!
> + */
> + pqFlush(conn);
> return 0;
> + }
> /* Parsing of the data waits till later. */
> return 1;
> }
> ***************
> *** 1088,1095 ****
> --- 1096,1112 ----
> {
> PGresult *result;
> PGresult *lastResult;
> + bool savedblocking;
>
> /*
> + * we assume anyone calling PQexec wants blocking behaviour,
> + * we force the blocking status of the connection to blocking
> + * for the duration of this function and restore it on return
> + */
> + savedblocking = PQisnonblocking(conn);
> + PQsetnonblocking(conn, FALSE);
> +
> + /*
> * Silently discard any prior query result that application didn't
> * eat. This is probably poor design, but it's here for backward
> * compatibility.
> ***************
> *** 1102,1115 ****
> PQclear(result);
> printfPQExpBuffer(&conn->errorMessage,
> "PQexec: you gotta get out of a COPY state yourself.\n");
> ! return NULL;
> }
> PQclear(result);
> }
>
> /* OK to send the message */
> if (!PQsendQuery(conn, query))
> ! return NULL;
>
> /*
> * For backwards compatibility, return the last result if there are
> --- 1119,1133 ----
> PQclear(result);
> printfPQExpBuffer(&conn->errorMessage,
> "PQexec: you gotta get out of a COPY state yourself.\n");
> ! /* restore blocking status */
> ! goto errout;
> }
> PQclear(result);
> }
>
> /* OK to send the message */
> if (!PQsendQuery(conn, query))
> ! goto errout;
>
> /*
> * For backwards compatibility, return the last result if there are
> ***************
> *** 1142,1148 ****
> --- 1160,1172 ----
> result->resultStatus == PGRES_COPY_OUT)
> break;
> }
> +
> + PQsetnonblocking(conn, savedblocking);
> return lastResult;
> +
> + errout:
> + PQsetnonblocking(conn, savedblocking);
> + return NULL;
> }
>
>
> ***************
> *** 1431,1438 ****
> "PQendcopy() -- I don't think there's a copy in progress.\n");
> return 1;
> }
>
> ! (void) pqFlush(conn); /* make sure no data is waiting to be sent */
>
> /* Return to active duty */
> conn->asyncStatus = PGASYNC_BUSY;
> --- 1455,1468 ----
> "PQendcopy() -- I don't think there's a copy in progress.\n");
> return 1;
> }
> +
> + /* make sure no data is waiting to be sent */
> + if (pqFlush(conn))
> + return (1);
>
> ! /* non blocking connections may have to abort at this point. */
> ! if (PQisnonblocking(conn) && PQisBusy(conn))
> ! return (1);
>
> /* Return to active duty */
> conn->asyncStatus = PGASYNC_BUSY;
> ***************
> *** 2025,2028 ****
> --- 2055,2126 ----
> return 1;
> else
> return 0;
> + }
> +
> + /* PQsetnonblocking:
> + sets the PGconn's database connection non-blocking if the arg is TRUE
> + or makes it non-blocking if the arg is FALSE, this will not protect
> + you from PQexec(), you'll only be safe when using the non-blocking
> + API
> + Needs to be called only on a connected database connection.
> + */
> +
> + int
> + PQsetnonblocking(PGconn *conn, int arg)
> + {
> + int fcntlarg;
> +
> + arg = (arg == TRUE) ? 1 : 0;
> + if (arg == conn->nonblocking)
> + return (0);
> +
> + #ifdef USE_SSL
> + if (conn->ssl)
> + {
> + printfPQExpBuffer(&conn->errorMessage,
> + "PQsetnonblocking() -- not supported when using SSL\n");
> + return (-1);
> + }
> + #endif /* USE_SSL */
> +
> + #ifndef WIN32
> + fcntlarg = fcntl(conn->sock, F_GETFL, 0);
> + if (fcntlarg == -1)
> + return (-1);
> +
> + if ((arg == TRUE &&
> + fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
> + (arg == FALSE &&
> + fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1))
> + #else
> + fcntlarg = arg;
> + if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
> + #endif
> + {
> + printfPQExpBuffer(&conn->errorMessage,
> + "PQsetblocking() -- unable to set nonblocking status to %s\n",
> + arg == TRUE ? "TRUE" : "FALSE");
> + return (-1);
> + }
> +
> + conn->nonblocking = arg;
> + return (0);
> + }
> +
> + /* return the blocking status of the database connection, TRUE == nonblocking,
> + FALSE == blocking
> + */
> + int
> + PQisnonblocking(PGconn *conn)
> + {
> +
> + return (conn->nonblocking);
> + }
> +
> + /* try to force data out, really only useful for non-blocking users */
> + int
> + PQflush(PGconn *conn)
> + {
> +
> + return (pqFlush(conn));
> }
> Index: fe-misc.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
> retrieving revision 1.33
> diff -u -c -r1.33 fe-misc.c
> cvs diff: conflicting specifications of output style
> *** fe-misc.c 1999/11/30 03:08:19 1.33
> --- fe-misc.c 1999/12/14 08:21:09
> ***************
> *** 86,91 ****
> --- 86,119 ----
> {
> size_t avail = Max(conn->outBufSize - conn->outCount, 0);
>
> + /*
> + * if we are non-blocking and the send queue is too full to buffer this
> + * request then try to flush some and return an error
> + */
> + if (PQisnonblocking(conn) && nbytes > avail && pqFlush(conn))
> + {
> + /*
> + * even if the flush failed we may still have written some
> + * data, recalculate the size of the send-queue relative
> + * to the amount we have to send, we may be able to queue it
> + * afterall even though it's not sent to the database it's
> + * ok, any routines that check the data coming from the
> + * database better call pqFlush() anyway.
> + */
> + if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
> + {
> + printfPQExpBuffer(&conn->errorMessage,
> + "pqPutBytes -- pqFlush couldn't flush enough"
> + " data: space available: %d, space needed %d\n",
> + Max(conn->outBufSize - conn->outCount, 0), nbytes);
> + return EOF;
> + }
> + }
> +
> + /*
> + * the non-blocking code above makes sure that this isn't true,
> + * essentially this is no-op
> + */
> while (nbytes > avail)
> {
> memcpy(conn->outBuffer + conn->outCount, s, avail);
> ***************
> *** 548,553 ****
> --- 576,589 ----
> return EOF;
> }
>
> + /*
> + * don't try to send zero data, allows us to use this function
> + * without too much worry about overhead
> + */
> + if (len == 0)
> + return (0);
> +
> + /* while there's still data to send */
> while (len > 0)
> {
> /* Prevent being SIGPIPEd if backend has closed the connection. */
> ***************
> *** 556,561 ****
> --- 592,598 ----
> #endif
>
> int sent;
> +
> #ifdef USE_SSL
> if (conn->ssl)
> sent = SSL_write(conn->ssl, ptr, len);
> ***************
> *** 585,590 ****
> --- 622,629 ----
> case EWOULDBLOCK:
> break;
> #endif
> + case EINTR:
> + continue;
>
> case EPIPE:
> #ifdef ECONNRESET
> ***************
> *** 616,628 ****
> ptr += sent;
> len -= sent;
> }
> if (len > 0)
> {
> /* We didn't send it all, wait till we can send more */
>
> - /* At first glance this looks as though it should block. I think
> - * that it will be OK though, as long as the socket is
> - * non-blocking. */
> if (pqWait(FALSE, TRUE, conn))
> return EOF;
> }
> --- 655,685 ----
> ptr += sent;
> len -= sent;
> }
> +
> if (len > 0)
> {
> /* We didn't send it all, wait till we can send more */
> +
> + /*
> + * if the socket is in non-blocking mode we may need
> + * to abort here
> + */
> + #ifdef USE_SSL
> + /* can't do anything for our SSL users yet */
> + if (conn->ssl == NULL)
> + {
> + #endif
> + if (PQisnonblocking(conn))
> + {
> + /* shift the contents of the buffer */
> + memmove(conn->outBuffer, ptr, len);
> + conn->outCount = len;
> + return EOF;
> + }
> + #ifdef USE_SSL
> + }
> + #endif
>
> if (pqWait(FALSE, TRUE, conn))
> return EOF;
> }
> Index: libpq-fe.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
> retrieving revision 1.53
> diff -u -c -r1.53 libpq-fe.h
> cvs diff: conflicting specifications of output style
> *** libpq-fe.h 1999/11/30 03:08:19 1.53
> --- libpq-fe.h 1999/12/14 01:30:01
> ***************
> *** 269,274 ****
> --- 269,281 ----
> extern int PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
> extern int PQendcopy(PGconn *conn);
>
> + /* Set blocking/nonblocking connection to the backend */
> + extern int PQsetnonblocking(PGconn *conn, int arg);
> + extern int PQisnonblocking(PGconn *conn);
> +
> + /* Force the write buffer to be written (or at least try) */
> + extern int PQflush(PGconn *conn);
> +
> /*
> * "Fast path" interface --- not really recommended for application
> * use
> Index: libpq-int.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
> retrieving revision 1.14
> diff -u -c -r1.14 libpq-int.h
> cvs diff: conflicting specifications of output style
> *** libpq-int.h 1999/11/30 03:08:19 1.14
> --- libpq-int.h 1999/12/14 01:30:01
> ***************
> *** 215,220 ****
> --- 215,223 ----
> int inEnd; /* offset to first position after avail
> * data */
>
> + int nonblocking; /* whether this connection is using a blocking
> + * socket to the backend or not */
> +
> /* Buffer for data not yet sent to backend */
> char *outBuffer; /* currently allocated buffer */
> int outBufSize; /* allocated size of buffer */
>
>
>
>
>
> ************
>
-- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610)
853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill,
Pennsylvania19026
On Sat, 8 Jan 2000, Bruce Momjian wrote:
> Looks fine. I have talked to someone about doing no-blocking
> connections in the past. Maybe this the same person.
>
> I will let someone else comment on whether the protocol changes are
> correct.
Okay, if I haven't heard anything major by Sunday, I'm going to include
these, whic still gives us a month before beta (well, not quite, but
close) and then the beta period in order to clean it up...
>
>
> >
> > Does anyone have anything against me applying this to the current source
> > tree?
> >
> >
> > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
> > Systems Administrator @ hub.org
> > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
> >
> > ---------- Forwarded message ----------
> > Date: Fri, 17 Dec 1999 13:51:50 -0800 (PST)
> > From: Alfred Perlstein <bright@wintelcom.net>
> > To: The Hermit Hacker <scrappy@hub.org>
> > Subject: Re: pctrackd updates and such
> >
> > On Fri, 17 Dec 1999, The Hermit Hacker wrote:
> >
> > >
> > > Okay, first thing...can you redo these as context diffs? We generally
> > > refuse *any* patches that aren't context...
> >
> > sure.
> >
> > >
> > > Second, are these against a reasonably current snapshot of PostgreSQL
> > > (aka. the upcoming v7), or v6.5.3 release? If v6.5.3, we're gonna need to
> > > get these v7.x ready before we can commit them...
> >
> > they are against a checked out cvs copy as of a couple days ago,
> > and should apply cleanly to what's in the current repo.
> >
> > > Once both of the above conditions are in place, and after I get back from
> > > BC, I'll work on getting these into the v7.0 release...or, at least,
> > > talked/commented about if there are any objections...
> > >
> > > I'm outta here for 10 days...Happy Holidays and talk with ya when I get
> > > back...
> >
> > ok, cool see you soon. :)
> >
> > -Alfred
> >
> > don't forget the problem with sending queries that may occur:
> >
> > i'm not sure if handlesendfailure() can cope with only sending
> > a 'Q' to the backend, we may have to work out reservations or
> > something for space, another idea would be to implement a
> > pqWritev() of some sort that would take an array of pointers
> > and lengths to send to the backend and only allow any data to
> > go into the backend if the entire string can fit.
> >
> > then again, handlesendfailure may work, but doing reservations
> > for the send buffer seems cleaner...
> >
> > diff's contexted against pgsql-'current':
> >
> >
> > Index: fe-connect.c
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
> > retrieving revision 1.108
> > diff -u -c -r1.108 fe-connect.c
> > cvs diff: conflicting specifications of output style
> > *** fe-connect.c 1999/12/02 00:26:15 1.108
> > --- fe-connect.c 1999/12/14 09:42:24
> > ***************
> > *** 595,625 ****
> > return 0;
> > }
> >
> > -
> > - /* ----------
> > - * connectMakeNonblocking -
> > - * Make a connection non-blocking.
> > - * Returns 1 if successful, 0 if not.
> > - * ----------
> > - */
> > - static int
> > - connectMakeNonblocking(PGconn *conn)
> > - {
> > - #ifndef WIN32
> > - if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
> > - #else
> > - if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
> > - #endif
> > - {
> > - printfPQExpBuffer(&conn->errorMessage,
> > - "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
> > - errno, strerror(errno));
> > - return 0;
> > - }
> > -
> > - return 1;
> > - }
> > -
> > /* ----------
> > * connectNoDelay -
> > * Sets the TCP_NODELAY socket option.
> > --- 595,600 ----
> > ***************
> > *** 792,798 ****
> > * Ewan Mellor <eem21@cam.ac.uk>.
> > * ---------- */
> > #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> > ! if (!connectMakeNonblocking(conn))
> > goto connect_errReturn;
> > #endif
> >
> > --- 767,773 ----
> > * Ewan Mellor <eem21@cam.ac.uk>.
> > * ---------- */
> > #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> > ! if (PQsetnonblocking(conn, TRUE) != 0)
> > goto connect_errReturn;
> > #endif
> >
> > ***************
> > *** 904,910 ****
> > /* This makes the connection non-blocking, for all those cases which forced us
> > not to do it above. */
> > #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> > ! if (!connectMakeNonblocking(conn))
> > goto connect_errReturn;
> > #endif
> >
> > --- 879,885 ----
> > /* This makes the connection non-blocking, for all those cases which forced us
> > not to do it above. */
> > #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> > ! if (PQsetnonblocking(conn, TRUE) != 0)
> > goto connect_errReturn;
> > #endif
> >
> > ***************
> > *** 1702,1707 ****
> > --- 1677,1683 ----
> > conn->inBuffer = (char *) malloc(conn->inBufSize);
> > conn->outBufSize = 8 * 1024;
> > conn->outBuffer = (char *) malloc(conn->outBufSize);
> > + conn->nonblocking = FALSE;
> > initPQExpBuffer(&conn->errorMessage);
> > initPQExpBuffer(&conn->workBuffer);
> > if (conn->inBuffer == NULL ||
> > ***************
> > *** 1811,1816 ****
> > --- 1787,1793 ----
> > conn->lobjfuncs = NULL;
> > conn->inStart = conn->inCursor = conn->inEnd = 0;
> > conn->outCount = 0;
> > + conn->nonblocking = FALSE;
> >
> > }
> >
> > Index: fe-exec.c
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
> > retrieving revision 1.86
> > diff -u -c -r1.86 fe-exec.c
> > cvs diff: conflicting specifications of output style
> > *** fe-exec.c 1999/11/11 00:10:14 1.86
> > --- fe-exec.c 1999/12/14 05:55:11
> > ***************
> > *** 13,18 ****
> > --- 13,19 ----
> > */
> > #include <errno.h>
> > #include <ctype.h>
> > + #include <fcntl.h>
> >
> > #include "postgres.h"
> > #include "libpq-fe.h"
> > ***************
> > *** 24,30 ****
> > #include <unistd.h>
> > #endif
> >
> > -
> > /* keep this in same order as ExecStatusType in libpq-fe.h */
> > const char *const pgresStatus[] = {
> > "PGRES_EMPTY_QUERY",
> > --- 25,30 ----
> > ***************
> > *** 574,580 ****
> > --- 574,588 ----
> > * we will NOT block waiting for more input.
> > */
> > if (pqReadData(conn) < 0)
> > + {
> > + /*
> > + * try to flush the send-queue otherwise we may never get a
> > + * resonce for something that may not have already been sent
> > + * because it's in our write buffer!
> > + */
> > + pqFlush(conn);
> > return 0;
> > + }
> > /* Parsing of the data waits till later. */
> > return 1;
> > }
> > ***************
> > *** 1088,1095 ****
> > --- 1096,1112 ----
> > {
> > PGresult *result;
> > PGresult *lastResult;
> > + bool savedblocking;
> >
> > /*
> > + * we assume anyone calling PQexec wants blocking behaviour,
> > + * we force the blocking status of the connection to blocking
> > + * for the duration of this function and restore it on return
> > + */
> > + savedblocking = PQisnonblocking(conn);
> > + PQsetnonblocking(conn, FALSE);
> > +
> > + /*
> > * Silently discard any prior query result that application didn't
> > * eat. This is probably poor design, but it's here for backward
> > * compatibility.
> > ***************
> > *** 1102,1115 ****
> > PQclear(result);
> > printfPQExpBuffer(&conn->errorMessage,
> > "PQexec: you gotta get out of a COPY state yourself.\n");
> > ! return NULL;
> > }
> > PQclear(result);
> > }
> >
> > /* OK to send the message */
> > if (!PQsendQuery(conn, query))
> > ! return NULL;
> >
> > /*
> > * For backwards compatibility, return the last result if there are
> > --- 1119,1133 ----
> > PQclear(result);
> > printfPQExpBuffer(&conn->errorMessage,
> > "PQexec: you gotta get out of a COPY state yourself.\n");
> > ! /* restore blocking status */
> > ! goto errout;
> > }
> > PQclear(result);
> > }
> >
> > /* OK to send the message */
> > if (!PQsendQuery(conn, query))
> > ! goto errout;
> >
> > /*
> > * For backwards compatibility, return the last result if there are
> > ***************
> > *** 1142,1148 ****
> > --- 1160,1172 ----
> > result->resultStatus == PGRES_COPY_OUT)
> > break;
> > }
> > +
> > + PQsetnonblocking(conn, savedblocking);
> > return lastResult;
> > +
> > + errout:
> > + PQsetnonblocking(conn, savedblocking);
> > + return NULL;
> > }
> >
> >
> > ***************
> > *** 1431,1438 ****
> > "PQendcopy() -- I don't think there's a copy in progress.\n");
> > return 1;
> > }
> >
> > ! (void) pqFlush(conn); /* make sure no data is waiting to be sent */
> >
> > /* Return to active duty */
> > conn->asyncStatus = PGASYNC_BUSY;
> > --- 1455,1468 ----
> > "PQendcopy() -- I don't think there's a copy in progress.\n");
> > return 1;
> > }
> > +
> > + /* make sure no data is waiting to be sent */
> > + if (pqFlush(conn))
> > + return (1);
> >
> > ! /* non blocking connections may have to abort at this point. */
> > ! if (PQisnonblocking(conn) && PQisBusy(conn))
> > ! return (1);
> >
> > /* Return to active duty */
> > conn->asyncStatus = PGASYNC_BUSY;
> > ***************
> > *** 2025,2028 ****
> > --- 2055,2126 ----
> > return 1;
> > else
> > return 0;
> > + }
> > +
> > + /* PQsetnonblocking:
> > + sets the PGconn's database connection non-blocking if the arg is TRUE
> > + or makes it non-blocking if the arg is FALSE, this will not protect
> > + you from PQexec(), you'll only be safe when using the non-blocking
> > + API
> > + Needs to be called only on a connected database connection.
> > + */
> > +
> > + int
> > + PQsetnonblocking(PGconn *conn, int arg)
> > + {
> > + int fcntlarg;
> > +
> > + arg = (arg == TRUE) ? 1 : 0;
> > + if (arg == conn->nonblocking)
> > + return (0);
> > +
> > + #ifdef USE_SSL
> > + if (conn->ssl)
> > + {
> > + printfPQExpBuffer(&conn->errorMessage,
> > + "PQsetnonblocking() -- not supported when using SSL\n");
> > + return (-1);
> > + }
> > + #endif /* USE_SSL */
> > +
> > + #ifndef WIN32
> > + fcntlarg = fcntl(conn->sock, F_GETFL, 0);
> > + if (fcntlarg == -1)
> > + return (-1);
> > +
> > + if ((arg == TRUE &&
> > + fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
> > + (arg == FALSE &&
> > + fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1))
> > + #else
> > + fcntlarg = arg;
> > + if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
> > + #endif
> > + {
> > + printfPQExpBuffer(&conn->errorMessage,
> > + "PQsetblocking() -- unable to set nonblocking status to %s\n",
> > + arg == TRUE ? "TRUE" : "FALSE");
> > + return (-1);
> > + }
> > +
> > + conn->nonblocking = arg;
> > + return (0);
> > + }
> > +
> > + /* return the blocking status of the database connection, TRUE == nonblocking,
> > + FALSE == blocking
> > + */
> > + int
> > + PQisnonblocking(PGconn *conn)
> > + {
> > +
> > + return (conn->nonblocking);
> > + }
> > +
> > + /* try to force data out, really only useful for non-blocking users */
> > + int
> > + PQflush(PGconn *conn)
> > + {
> > +
> > + return (pqFlush(conn));
> > }
> > Index: fe-misc.c
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
> > retrieving revision 1.33
> > diff -u -c -r1.33 fe-misc.c
> > cvs diff: conflicting specifications of output style
> > *** fe-misc.c 1999/11/30 03:08:19 1.33
> > --- fe-misc.c 1999/12/14 08:21:09
> > ***************
> > *** 86,91 ****
> > --- 86,119 ----
> > {
> > size_t avail = Max(conn->outBufSize - conn->outCount, 0);
> >
> > + /*
> > + * if we are non-blocking and the send queue is too full to buffer this
> > + * request then try to flush some and return an error
> > + */
> > + if (PQisnonblocking(conn) && nbytes > avail && pqFlush(conn))
> > + {
> > + /*
> > + * even if the flush failed we may still have written some
> > + * data, recalculate the size of the send-queue relative
> > + * to the amount we have to send, we may be able to queue it
> > + * afterall even though it's not sent to the database it's
> > + * ok, any routines that check the data coming from the
> > + * database better call pqFlush() anyway.
> > + */
> > + if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
> > + {
> > + printfPQExpBuffer(&conn->errorMessage,
> > + "pqPutBytes -- pqFlush couldn't flush enough"
> > + " data: space available: %d, space needed %d\n",
> > + Max(conn->outBufSize - conn->outCount, 0), nbytes);
> > + return EOF;
> > + }
> > + }
> > +
> > + /*
> > + * the non-blocking code above makes sure that this isn't true,
> > + * essentially this is no-op
> > + */
> > while (nbytes > avail)
> > {
> > memcpy(conn->outBuffer + conn->outCount, s, avail);
> > ***************
> > *** 548,553 ****
> > --- 576,589 ----
> > return EOF;
> > }
> >
> > + /*
> > + * don't try to send zero data, allows us to use this function
> > + * without too much worry about overhead
> > + */
> > + if (len == 0)
> > + return (0);
> > +
> > + /* while there's still data to send */
> > while (len > 0)
> > {
> > /* Prevent being SIGPIPEd if backend has closed the connection. */
> > ***************
> > *** 556,561 ****
> > --- 592,598 ----
> > #endif
> >
> > int sent;
> > +
> > #ifdef USE_SSL
> > if (conn->ssl)
> > sent = SSL_write(conn->ssl, ptr, len);
> > ***************
> > *** 585,590 ****
> > --- 622,629 ----
> > case EWOULDBLOCK:
> > break;
> > #endif
> > + case EINTR:
> > + continue;
> >
> > case EPIPE:
> > #ifdef ECONNRESET
> > ***************
> > *** 616,628 ****
> > ptr += sent;
> > len -= sent;
> > }
> > if (len > 0)
> > {
> > /* We didn't send it all, wait till we can send more */
> >
> > - /* At first glance this looks as though it should block. I think
> > - * that it will be OK though, as long as the socket is
> > - * non-blocking. */
> > if (pqWait(FALSE, TRUE, conn))
> > return EOF;
> > }
> > --- 655,685 ----
> > ptr += sent;
> > len -= sent;
> > }
> > +
> > if (len > 0)
> > {
> > /* We didn't send it all, wait till we can send more */
> > +
> > + /*
> > + * if the socket is in non-blocking mode we may need
> > + * to abort here
> > + */
> > + #ifdef USE_SSL
> > + /* can't do anything for our SSL users yet */
> > + if (conn->ssl == NULL)
> > + {
> > + #endif
> > + if (PQisnonblocking(conn))
> > + {
> > + /* shift the contents of the buffer */
> > + memmove(conn->outBuffer, ptr, len);
> > + conn->outCount = len;
> > + return EOF;
> > + }
> > + #ifdef USE_SSL
> > + }
> > + #endif
> >
> > if (pqWait(FALSE, TRUE, conn))
> > return EOF;
> > }
> > Index: libpq-fe.h
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
> > retrieving revision 1.53
> > diff -u -c -r1.53 libpq-fe.h
> > cvs diff: conflicting specifications of output style
> > *** libpq-fe.h 1999/11/30 03:08:19 1.53
> > --- libpq-fe.h 1999/12/14 01:30:01
> > ***************
> > *** 269,274 ****
> > --- 269,281 ----
> > extern int PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
> > extern int PQendcopy(PGconn *conn);
> >
> > + /* Set blocking/nonblocking connection to the backend */
> > + extern int PQsetnonblocking(PGconn *conn, int arg);
> > + extern int PQisnonblocking(PGconn *conn);
> > +
> > + /* Force the write buffer to be written (or at least try) */
> > + extern int PQflush(PGconn *conn);
> > +
> > /*
> > * "Fast path" interface --- not really recommended for application
> > * use
> > Index: libpq-int.h
> > ===================================================================
> > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
> > retrieving revision 1.14
> > diff -u -c -r1.14 libpq-int.h
> > cvs diff: conflicting specifications of output style
> > *** libpq-int.h 1999/11/30 03:08:19 1.14
> > --- libpq-int.h 1999/12/14 01:30:01
> > ***************
> > *** 215,220 ****
> > --- 215,223 ----
> > int inEnd; /* offset to first position after avail
> > * data */
> >
> > + int nonblocking; /* whether this connection is using a blocking
> > + * socket to the backend or not */
> > +
> > /* Buffer for data not yet sent to backend */
> > char *outBuffer; /* currently allocated buffer */
> > int outBufSize; /* allocated size of buffer */
> >
> >
> >
> >
> >
> > ************
> >
>
>
> --
> Bruce Momjian | http://www.op.net/~candle
> maillist@candle.pha.pa.us | (610) 853-3000
> + If your life is a hard drive, | 830 Blythe Avenue
> + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
>
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
The Hermit Hacker <scrappy@hub.org> writes:
> Does anyone have anything against me applying this to the current source
> tree?
I'm not particularly comfortable with it --- it looks like the semantics
need more careful thought, particularly concerning when the output buffer
gets flushed and what happens if we can't send data right away. The
insertion of a pqFlush into PQconsumeInput, in particular, looks like
an ill-thought-out hack that could break some applications.
I also object strongly to the lack of documentation. Patches that
change public APIs and come without doco updates should be rejected
out of hand, IMNSHO. Keeping the documentation up to date should
not be considered optional --- especially not when you're talking
about something that makes subtle and pervasive changes to library
behavior.
regards, tom lane
On Sat, 8 Jan 2000, Tom Lane wrote:
> The Hermit Hacker <scrappy@hub.org> writes:
> > Does anyone have anything against me applying this to the current source
> > tree?
>
> I'm not particularly comfortable with it --- it looks like the semantics
> need more careful thought, particularly concerning when the output buffer
> gets flushed and what happens if we can't send data right away. The
> insertion of a pqFlush into PQconsumeInput, in particular, looks like
> an ill-thought-out hack that could break some applications.
Well, at least we have more discussion on this then the previous two posts
about it, so it should give something for Alfred to address :) Is there
anyone workign with libpq that could comment on possible a better way of
it being implemented?
> I also object strongly to the lack of documentation. Patches that
> change public APIs and come without doco updates should be rejected
> out of hand, IMNSHO. Keeping the documentation up to date should
> not be considered optional --- especially not when you're talking
> about something that makes subtle and pervasive changes to library
> behavior.
Agreed here...Alfred and I talked about that on the phone tonight...I
posted the patches tonight so that he could get some feedback on them...if
we could figure out what needs to be fixed/improved, and he has an
indication that he's working in the right direction, then documentation is
forthcoming...
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
* Tom Lane <tgl@sss.pgh.pa.us> [000108 14:56] wrote: > The Hermit Hacker <scrappy@hub.org> writes: > > Does anyone have anything against me applying this to the current source > > tree? > > I'm not particularly comfortable with it --- it looks like the semantics > need more careful thought, particularly concerning when the output buffer > gets flushed and what happens if we can't send data right away. Could you be more specific? My patches address the fact that although there is work to make libpq non-blocking you can easily block while sending large queries expecially because of the select() that is done in pqFlush(). The problem is that libpq doesn't reserve space in the send buffer and will just block if waiting for the socket to the backend to drain. This needs to be fixed if libpq is trully going to offer non-blocking behavior. Unless you reserve the space in the buffer you have to block otherwise if you abort (so as not to block) then libpq may have sent a partial query down the pipe or just buffered part of some data you've sent to the backend. At this point you will be out of sync with the backend. If you are in 'normal mode' (blocking) then the behavior shouldn't be any different, if you are non-blocking then if you attempt to send data and it's not possible you'll get an error without potentially sending a partial line to the backend. > The > insertion of a pqFlush into PQconsumeInput, in particular, looks like > an ill-thought-out hack that could break some applications. I think I agree, the code I was using would attempt an PQconsumeInput() before doing a PQendcopy(), there could be data in the send buffer that would make PQconsumeInput() never succeed hence the need for a flush. I'm going to try it without the PQconsumeInput() before the PQendcopy() my modifications for PQendcopy() should make it non-blocking safe. but in the meanwhile here's my (probably wrong) reasoning behind this 'hack': No, IMHO it's needed, the problem is that there may be datain the send buffer that hasn't been sent yet, it could bepartof a request to the backend that you are explicitlywaiting for a result from. This can happen when doing a COPY into the database. What happens is that you send data, then when you send the'end copy' it can get buffered, then you loop foreverattemptingto read a result for a query that was neversent. In regards to it breaking applications, the send buffershould be opaque to the libpq user, libpq never has offereda trulynon-blocking api, and even when using non-blockingthe flush will fail if it can't be done and PQconsumeInput()will errorout accordingly. Old applications can be snagged by the Flush since in theoryPQconsumeInput shouldn't block, however I'm not sure ifthere'sa real workaround for this except 1.. saving the blocking status of the connection, 2.. setting it non-blocking and attempting a flush and then3.. restoringthe blocking status. It seems that old applications can break (looping on anunsent line) regardless because of the not-flushed-queryproblem. If you can figure an occasion where this might actually happen (with the exception of my accidentaly abuse of libpq) then it may need to be revisited. I'll get back to you guys on the PQendcopy before PQconsumeInput tests. > I also object strongly to the lack of documentation. Patches that > change public APIs and come without doco updates should be rejected > out of hand, IMNSHO. Keeping the documentation up to date should > not be considered optional --- especially not when you're talking > about something that makes subtle and pervasive changes to library > behavior. I agree with you about the documentation issues, I will try to add some documentation to the patches. I think I can also take out the visibility of the PQflush() function as normal applications really shouldn't need it. How do you feel about the explicit PQsetnonblocking and PQisnonblocking functions that I added as well as the additional field 'nonblocking' added to PGconn? IMO the user shouldn't set the socket non-blocking without informing the library about it otherwise it gets really ugly because we have to constantly poll the socket's flags to make sure we DTRT. I also apologize for my not indented patches, is there a way to indent according to postgresql standards on a FreeBSD system? The patches for pgindent are a bit out of date and I get floating point exceptions when attempting to pgindent. Thanks for the feedback. -Alfred
* Alfred Perlstein <bright@wintelcom.net> [000108 16:08] wrote:
> * Tom Lane <tgl@sss.pgh.pa.us> [000108 14:56] wrote:
>
> > The
> > insertion of a pqFlush into PQconsumeInput, in particular, looks like
> > an ill-thought-out hack that could break some applications.
>
> I think I agree, the code I was using would attempt an PQconsumeInput()
> before doing a PQendcopy(), there could be data in the send buffer
> that would make PQconsumeInput() never succeed hence the need for a
> flush.
>
> I'm going to try it without the PQconsumeInput() before the PQendcopy()
> my modifications for PQendcopy() should make it non-blocking safe.
> but in the meanwhile here's my (probably wrong) reasoning behind this
> 'hack':
>
> No, IMHO it's needed, the problem is that there may be data
> in the send buffer that hasn't been sent yet, it could be
> part of a request to the backend that you are explicitly
> waiting for a result from.
>
> This can happen when doing a COPY into the database.
>
> What happens is that you send data, then when you send the
> 'end copy' it can get buffered, then you loop forever
> attempting to read a result for a query that was never
> sent.
>
> In regards to it breaking applications, the send buffer
> should be opaque to the libpq user, libpq never has offered
> a truly non-blocking api, and even when using non-blocking
> the flush will fail if it can't be done and PQconsumeInput()
> will error out accordingly.
>
> Old applications can be snagged by the Flush since in theory
> PQconsumeInput shouldn't block, however I'm not sure if
> there's a real workaround for this except
>
> 1.. saving the blocking status of the connection,
> 2.. setting it non-blocking and attempting a flush and then
> 3.. restoring the blocking status.
>
> It seems that old applications can break (looping on an
> unsent line) regardless because of the not-flushed-query
> problem.
>
> If you can figure an occasion where this might actually happen
> (with the exception of my accidentaly abuse of libpq) then it
> may need to be revisited.
>
> I'll get back to you guys on the PQendcopy before PQconsumeInput
> tests.
I just remebered where the problem is (sorry it's been about 2 weeks
since i've read through the code) it's a bit different
and messier than I thought:
I begin my COPY commands with a PQsendQuery() which has this block
of code in it:
/* send the query to the backend; *//* the frontend-backend protocol uses 'Q' to designate queries */if
(pqPutnchar("Q",1, conn) || pqPuts(query, conn) || pqFlush(conn)){ handleSendFailure(conn); return 0;}
It can get really hairy for non-blocking connections if any of the
functions in the 'if' conditional fail, any ideas on a workaround?
One that comes to mind is using the low/high watermarks in sockets,
if we do that then a write-ready true condition would garantee that
we have X number of bytes available in our send buffer and we can
safely queue the data. This doesn't seem portable and would be
pretty complex.
Another is to attempt a flush beforehand aborting early if it fails
however we still need to flush after we pqPutnchar and pqPuts,
if that fails we are back to needing to call pqFlush from PQconsumeInput
beccause the only failure is that the backend's pipe is potentially full
and we may have queued something.
This 'hack' would be to allow the last flush to fail and always call
pqFlush in PQconsumeInput when the connection is non-blocking because
with the new code it shouldn't block, PQconsumeInput will function to
drive data to the backend in that situation. We'll only do the
flush in PQconsumeInput() for non-blocking connections because
the conditional in PQsendQuery shouldn't fail for blocking connections
unless something is seriously wrong.
I hate to say it, but I like the hack approach, it is somewhat wierd
but looks like it would work quite well as any error returned from
the backend would be delayed until PQconsumeInput and a broken connection
would still be returned immediatly from PQsendQuery.
Your opinion?
--
-Alfred Perlstein - [bright@rush.net|alfred@freebsd.org]
Wintelcom systems administrator and programmer - http://www.wintelcom.net/ [bright@wintelcom.net]
Hello all. I wanted to throw up an idea for a
XML interface to PostgreSQL.
The underlying information model for SGML documents
is called the "groves" model. The model's primary
prupose is to handle multiple content hierarchites
within a single network; as this is the primary
problem with multimedia hypertext markup (HyTime).
Anyway, what is interesting is that the core of the
groves model seems to be the following seemingly
simple production:
node := character | map( string, list(node) )
It is an alternation between a map (unordered
set) and a list (ordered bag). Anyway, the discovery
made me sit back in my chair and go hmmmmmm. As it
seems to me; this model is the essence of a multi-level
result set, and could point towards a rather natural
mapping of a multi-hierarchy result set onto a tag
based markup language like XML.
For starters, here is a row...
row-node := map (string, list(characters))
For example, here is an order, and two order-line rows:
order-row-node := { id = '345' , buyer = 'Clark Evans' }
order-line-row := { order-id = '345' , product = 'Bag of rice' , quantity
='1' }
order-line-row := { order-id = '345' , product = 'Tofo' , quantity = '3'
And here, is a table:
relation-node := map('relation-name',list(row-node))
line-relation := { order-line = [ { order-id = '345'
, product = 'Bag of rice' , quantity = '1' } ,
... , { order-id = '345' ,
product = 'Tofo' , quantity = '1' } ]
}
Here is the origonal production again:
node := character | map( string, list(node) )
It could then be used to return a nested
result set like:
SELECT * FROM ORDER WHERE ID = '345' with this sub-query nested...
SELECT PRODUCT, QUANTITY FROM ORDER_LINE WHERE ORDER_ID = ID;
my-order =: { id = '345' , buyer = 'Clark Evans' , lines = [
{ order-id = '345' , product = 'Bag of rice' } , quantity
='1' } , { order-id = '345' ,
product= 'Tofo' , quantity = '3' } ]
}
Here is a mapping to a simple markup language
(http://www.egroups.com/list/sml-dev/info.html)
<order> <id>345</id> <buyer>Clark Evans</buyer> <order-line-list> <order-line> <product>Bag of rice</product>
<quantity>1</quantity> </order-line> <order-line> <product>Tofo</product> <quantity>3</quantity>
</order-line></order-line-list>
</order>
So, if you notice the even levels are maps,
and the odd levels are lists. Kinda cool.
Of course, you can shorten it with attribute notation...
<order id="345" buyer="Clark Evans"> <order-line product="Bag of rice" quantity="1" /> <order-line product="Tofo"
quantity="3" />
</order>
(The syntax is more brief, but it only allows for one list per map)
Anyway, this could allow for some nice automated
conversions in between a database and SML/XML/SGML.
Indeed, I was thinking that a binary version of
of the former syntax would make for a great
interface into/outof the database. Add a simple
text/binary adapter and PostgreSQL would be
XML ready... without the need for any particular
mapping tool.
Best,
Clark
At 05:27 PM 1/8/00 -0500, Tom Lane wrote: >I also object strongly to the lack of documentation. Patches that >change public APIs and come without doco updates should be rejected >out of hand, IMNSHO. Keeping the documentation up to date should >not be considered optional --- especially not when you're talking >about something that makes subtle and pervasive changes to library >behavior. Boy, Tom's really laid it out in excellent style. If the author of such changes doesn't document them, chances are that the documentation won't get done. That's very bad. The automatic rejection of undocumented patches that change the API or other user-visible behavior shouldn't be controversial. I know there are some folks who aren't native-english speakers, so perhaps you don't want to require that the implementor of such patches provide the final documentation wording. But the information should be there and spelled out in a form that can be very easily moved to the docs. - Don Baccus, Portland OR <dhogaza@pacifier.com> Nature photos, on-line guides, Pacific Northwest Rare Bird Alert Serviceand other goodies at http://donb.photo.net.
Don Baccus <dhogaza@pacifier.com> writes:
> At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
>> I also object strongly to the lack of documentation.
> ... I know there are some folks who aren't native-english speakers, so
> perhaps you don't want to require that the implementor of such patches
> provide the final documentation wording. But the information should
> be there and spelled out in a form that can be very easily moved to
> the docs.
Oh, absolutely. Thomas, our master of the docs, has always had the
policy of "give me some words, I'll take care of formatting and
editing..."
I was probably too harsh on Alfred last night, since in fact his code
was fairly well commented, and some minimal doco could have been
extracted from the routine headers. But on a change like this, I think
some paragraphs of coherent high-level explanation are needed: what it
does, when and why you'd use it. I didn't see that anywhere...
regards, tom lane
> At 05:27 PM 1/8/00 -0500, Tom Lane wrote: > > >I also object strongly to the lack of documentation. Patches that > >change public APIs and come without doco updates should be rejected > >out of hand, IMNSHO. Keeping the documentation up to date should > >not be considered optional --- especially not when you're talking > >about something that makes subtle and pervasive changes to library > >behavior. > > Boy, Tom's really laid it out in excellent style. If the author of > such changes doesn't document them, chances are that the documentation > won't get done. That's very bad. > > The automatic rejection of undocumented patches that change the API > or other user-visible behavior shouldn't be controversial. I know > there are some folks who aren't native-english speakers, so perhaps > you don't want to require that the implementor of such patches provide > the final documentation wording. But the information should be there > and spelled out in a form that can be very easily moved to the docs. If it is missing, we get back to them before final release and ask for doc patches. They get in there one way or another. -- Bruce Momjian | http://www.op.net/~candle maillist@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Sun, 9 Jan 2000, Don Baccus wrote:
> At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
>
> >I also object strongly to the lack of documentation. Patches that
> >change public APIs and come without doco updates should be rejected
> >out of hand, IMNSHO. Keeping the documentation up to date should
> >not be considered optional --- especially not when you're talking
> >about something that makes subtle and pervasive changes to library
> >behavior.
>
> Boy, Tom's really laid it out in excellent style. If the author of
> such changes doesn't document them, chances are that the documentation
> won't get done. That's very bad.
>
> The automatic rejection of undocumented patches that change the API
> or other user-visible behavior shouldn't be controversial. I know
> there are some folks who aren't native-english speakers, so perhaps
> you don't want to require that the implementor of such patches provide
> the final documentation wording. But the information should be there
> and spelled out in a form that can be very easily moved to the docs.
These patches were originally submited before Xmas by Alfred, asking for
feedback on them and possibly pointing out errors in implementation...he
wanted to get a feel whether or not it was *worth* him putting further
work into them. They fell on silent ears.
Personally, I wouldn't waste my time on documenting something that, in the
end, I'd be the only one using...I'd get feedback on the usefulness first,
and then deal with building up the documentation after I've found out that
its worth it...
Alfred didn't ask "do I need to add documentation?" ... he knew that ...
he asked whether or not the implementation was appropriate, and was worth
his time to continue working ...
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
* Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote:
> Don Baccus <dhogaza@pacifier.com> writes:
> > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> >> I also object strongly to the lack of documentation.
>
> > ... I know there are some folks who aren't native-english speakers, so
> > perhaps you don't want to require that the implementor of such patches
> > provide the final documentation wording. But the information should
> > be there and spelled out in a form that can be very easily moved to
> > the docs.
>
> Oh, absolutely. Thomas, our master of the docs, has always had the
> policy of "give me some words, I'll take care of formatting and
> editing..."
>
> I was probably too harsh on Alfred last night, since in fact his code
> was fairly well commented, and some minimal doco could have been
> extracted from the routine headers. But on a change like this, I think
> some paragraphs of coherent high-level explanation are needed: what it
> does, when and why you'd use it. I didn't see that anywhere...
I've actually been trying to work on the sgml and failing miserably,
I have no clue how this stuff works (sgml compilation) are you asking
for a couple of paragraphs that describe the proposed changes?
If so I hope this suffices, if not some help on building the sgml
would be much appreciated:
--------
Summary:
First and foremost, great pains have been taken so that there
are _no_ compatibility issues.
If a 6.5.3 libpq program should not behave any differently
with this patches in place, all they do is offer a step closer
to a truly non-blocking connection to the backend and address
some issues with non-blocking connections.
----
Added functions:
int PQisnonblocking(static PGconn *conn);
returns whether or not the socket is in blocking mode, however... it doesn't actually check the socket flags, it
relieson the user to call 'PQsetnonblocking()' to keep the internal state of libpq sane. users should no longer use
'PQsocket()'to retrieve the socket and 'manually' ioctl/fcntl it to non-blocking
returns TRUE if the socket has been set to blocking more, FALSE if the socket is blocking
int PQflush(PGconn *conn);
flush the send-queue to the backend, just make this visible to the user for convience, as the internal function works,
0for success, EOF for any failure.
int PQsetnonblocking(PGconn *conn, int arg);
actually set the connection to the backend to blocking or non-blocking arg should be set to TRUE to set the connection
tonon-blocking or FALSE to set it blocking.
there's an implied blocking flush of the send-queue which is really ok as the user is either 'going into' or
'returningfrom' a blocking state
returns 0 for success, -1 for failure
---
New functionality:
PQsetblocking() allows libpq to know what behavior the user really wants, the user will not block sending data to the
backend,potentially if i had a constant stream of data and was doing a COPYIN it'd never finish because unless the
backendlost the connection I would block while sending until the backend can take more data.
---
Implementation changes:
none should be visible to programs based on 6.5.3's libpq.
programs based on later versions of libpq will notice that the non-blocking connection functions will set the state of
theconnection to non-blocking automatically.
when the connection is set non-blocking pqFlush() will not block if the sendqueue would be filled by new data inserted
intothe the queue.
functions that poll for data from the backend implicitly _try_ flush the send queue if set to non-blocking. This
allowsthe polling to act as a context for pushing queued data to the backend.
---
Problems:
We need some sort of send-queue commit reservations so that there's no chance of us sending a partial queury down the
pipeto the backend, right now this is hacked around by potentially blocking in non-blocking mode if something 'goes
terriblywrong' I plan to fix this.
---
Quirks:
PQexec() assumes the caller wants blocking behavior and will set the connection to blocking for the duration of the
PQexec()call, it will then restore it
---
Internal changes:
new field in PGconn 'int nonblocking' set to 1 if the connection is nonblocking, 0 if blocking (default)
macro pqIsnonblocking(PGconn) to avoid a function call to check blocking status (only visible if libpq-int.h is
included)
the internal function connectMakeNonblocking() has been replaced with PQsetblocking()
restart a send if EINTR is reported during a flush.
---
Lastly:
This is work in progress, I will be working towards making libpq better at not blocking.
here are the diffs:
Index: fe-connect.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.109
diff -u -c -IHeader -I$Id: -r1.109 fe-connect.c
cvs diff: conflicting specifications of output style
*** fe-connect.c 2000/01/14 05:33:15 1.109
--- fe-connect.c 2000/01/14 18:36:54
***************
*** 594,624 **** return 0; }
-
- /* ----------
- * connectMakeNonblocking -
- * Make a connection non-blocking.
- * Returns 1 if successful, 0 if not.
- * ----------
- */
- static int
- connectMakeNonblocking(PGconn *conn)
- {
- #ifndef WIN32
- if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
- #else
- if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
- #endif
- {
- printfPQExpBuffer(&conn->errorMessage,
- "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
- errno, strerror(errno));
- return 0;
- }
-
- return 1;
- }
- /* ---------- * connectNoDelay - * Sets the TCP_NODELAY socket option.
--- 594,599 ----
***************
*** 789,795 **** * Ewan Mellor <eem21@cam.ac.uk>. * ---------- */ #if (!defined(WIN32) ||
defined(WIN32_NON_BLOCKING_CONNECTIONS))&& !defined(USE_SSL)
! if (!connectMakeNonblocking(conn)) goto connect_errReturn; #endif
--- 764,770 ---- * Ewan Mellor <eem21@cam.ac.uk>. * ---------- */ #if (!defined(WIN32) ||
defined(WIN32_NON_BLOCKING_CONNECTIONS))&& !defined(USE_SSL)
! if (PQsetnonblocking(conn, TRUE) != 0) goto connect_errReturn; #endif
***************
*** 898,904 **** /* This makes the connection non-blocking, for all those cases which forced us not to do it
above.*/ #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! if (!connectMakeNonblocking(conn)) goto connect_errReturn; #endif
--- 873,879 ---- /* This makes the connection non-blocking, for all those cases which forced us not to do it
above.*/ #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! if (PQsetnonblocking(conn, TRUE) != 0) goto connect_errReturn; #endif
***************
*** 1702,1707 ****
--- 1677,1683 ---- conn->inBuffer = (char *) malloc(conn->inBufSize); conn->outBufSize = 8 * 1024;
conn->outBuffer= (char *) malloc(conn->outBufSize);
+ conn->nonblocking = FALSE; initPQExpBuffer(&conn->errorMessage); initPQExpBuffer(&conn->workBuffer);
if(conn->inBuffer == NULL ||
***************
*** 1812,1817 ****
--- 1788,1794 ---- conn->lobjfuncs = NULL; conn->inStart = conn->inCursor = conn->inEnd = 0; conn->outCount
=0;
+ conn->nonblocking = FALSE; }
Index: fe-exec.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.86
diff -u -c -IHeader -I$Id: -r1.86 fe-exec.c
cvs diff: conflicting specifications of output style
*** fe-exec.c 1999/11/11 00:10:14 1.86
--- fe-exec.c 2000/01/14 22:47:07
***************
*** 13,18 ****
--- 13,19 ---- */ #include <errno.h> #include <ctype.h>
+ #include <fcntl.h> #include "postgres.h" #include "libpq-fe.h"
***************
*** 24,30 **** #include <unistd.h> #endif
- /* keep this in same order as ExecStatusType in libpq-fe.h */ const char *const pgresStatus[] = {
"PGRES_EMPTY_QUERY",
--- 25,30 ----
***************
*** 514,526 **** conn->curTuple = NULL; /* send the query to the backend; */
! /* the frontend-backend protocol uses 'Q' to designate queries */
! if (pqPutnchar("Q", 1, conn) ||
! pqPuts(query, conn) ||
! pqFlush(conn)) {
! handleSendFailure(conn);
! return 0; } /* OK, it's launched! */
--- 514,566 ---- conn->curTuple = NULL; /* send the query to the backend; */
!
! /*
! * in order to guarantee that we don't send a partial query
! * where we would become out of sync with the backend and/or
! * block during a non-blocking connection we must first flush
! * the send buffer before sending more data
! *
! * an alternative is to implement 'queue reservations' where
! * we are able to roll up a transaction
! * (the 'Q' along with our query) and make sure we have
! * enough space for it all in the send buffer.
! */
! if (pqIsnonblocking(conn)) {
! /*
! * the buffer must have emptied completely before we allow
! * a new query to be buffered
! */
! if (pqFlush(conn))
! return 0;
! /* 'Q' == queries */
! /* XXX: if we fail here we really ought to not block */
! if (pqPutnchar("Q", 1, conn) ||
! pqPuts(query, conn))
! {
! handleSendFailure(conn);
! return 0;
! }
! /*
! * give the data a push, ignore the return value as
! * ConsumeInput() will do any aditional flushing if needed
! */
! (void) pqFlush(conn);
! }
! else
! {
! /*
! * the frontend-backend protocol uses 'Q' to
! * designate queries
! */
! if (pqPutnchar("Q", 1, conn) ||
! pqPuts(query, conn) ||
! pqFlush(conn))
! {
! handleSendFailure(conn);
! return 0;
! } } /* OK, it's launched! */
***************
*** 574,580 ****
--- 614,630 ---- * we will NOT block waiting for more input. */ if (pqReadData(conn) < 0)
+ {
+ /*
+ * for non-blocking connections
+ * try to flush the send-queue otherwise we may never get a
+ * responce for something that may not have already been sent
+ * because it's in our write buffer!
+ */
+ if (pqIsnonblocking(conn))
+ (void) pqFlush(conn); return 0;
+ } /* Parsing of the data waits till later. */ return 1; }
***************
*** 1088,1093 ****
--- 1138,1153 ---- { PGresult *result; PGresult *lastResult;
+ bool savedblocking;
+
+ /*
+ * we assume anyone calling PQexec wants blocking behaviour,
+ * we force the blocking status of the connection to blocking
+ * for the duration of this function and restore it on return
+ */
+ savedblocking = pqIsnonblocking(conn);
+ if (PQsetnonblocking(conn, FALSE) == -1)
+ return NULL; /* * Silently discard any prior query result that application didn't
***************
*** 1102,1115 **** PQclear(result); printfPQExpBuffer(&conn->errorMessage,
"PQexec:you gotta get out of a COPY state yourself.\n");
! return NULL; } PQclear(result); } /* OK to send the message */ if
(!PQsendQuery(conn,query))
! return NULL; /* * For backwards compatibility, return the last result if there are
--- 1162,1176 ---- PQclear(result); printfPQExpBuffer(&conn->errorMessage,
"PQexec:you gotta get out of a COPY state yourself.\n");
! /* restore blocking status */
! goto errout; } PQclear(result); } /* OK to send the message */ if
(!PQsendQuery(conn,query))
! goto errout; /* restore blocking status */ /* * For backwards compatibility, return the last
resultif there are
***************
*** 1142,1148 ****
--- 1203,1217 ---- result->resultStatus == PGRES_COPY_OUT) break; }
+
+ if (PQsetnonblocking(conn, savedblocking) == -1)
+ return NULL; return lastResult;
+
+ errout:
+ if (PQsetnonblocking(conn, savedblocking) == -1)
+ return NULL;
+ return NULL; }
***************
*** 1431,1438 **** "PQendcopy() -- I don't think there's a copy in progress.\n"); return 1; }
! (void) pqFlush(conn); /* make sure no data is waiting to be sent */ /* Return to active duty */
conn->asyncStatus= PGASYNC_BUSY;
--- 1500,1516 ---- "PQendcopy() -- I don't think there's a copy in progress.\n"); return 1; }
+
+ /*
+ * make sure no data is waiting to be sent,
+ * abort if we are non-blocking and the flush fails
+ */
+ if (pqFlush(conn) && pqIsnonblocking(conn))
+ return (1);
! /* non blocking connections may have to abort at this point. */
! if (pqIsnonblocking(conn) && PQisBusy(conn))
! return (1); /* Return to active duty */ conn->asyncStatus = PGASYNC_BUSY;
***************
*** 2025,2028 ****
--- 2103,2192 ---- return 1; else return 0;
+ }
+
+ /* PQsetnonblocking:
+ sets the PGconn's database connection non-blocking if the arg is TRUE
+ or makes it non-blocking if the arg is FALSE, this will not protect
+ you from PQexec(), you'll only be safe when using the non-blocking
+ API
+ Needs to be called only on a connected database connection.
+ */
+
+ int
+ PQsetnonblocking(PGconn *conn, int arg)
+ {
+ int fcntlarg;
+
+ arg = (arg == TRUE) ? 1 : 0;
+ /* early out if the socket is already in the state requested */
+ if (arg == conn->nonblocking)
+ return (0);
+
+ /*
+ * to guarantee constancy for flushing/query/result-polling behavior
+ * we need to flush the send queue at this point in order to guarantee
+ * proper behavior.
+ * this is ok because either they are making a transition
+ * _from_ or _to_ blocking mode, either way we can block them.
+ */
+ /* if we are going from blocking to non-blocking flush here */
+ if (!pqIsnonblocking(conn) && pqFlush(conn))
+ return (-1);
+
+
+ #ifdef USE_SSL
+ if (conn->ssl)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "PQsetnonblocking() -- not supported when using SSL\n");
+ return (-1);
+ }
+ #endif /* USE_SSL */
+
+ #ifndef WIN32
+ fcntlarg = fcntl(conn->sock, F_GETFL, 0);
+ if (fcntlarg == -1)
+ return (-1);
+
+ if ((arg == TRUE &&
+ fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
+ (arg == FALSE &&
+ fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1))
+ #else
+ fcntlarg = arg;
+ if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
+ #endif
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "PQsetblocking() -- unable to set nonblocking status to %s\n",
+ arg == TRUE ? "TRUE" : "FALSE");
+ return (-1);
+ }
+
+ conn->nonblocking = arg;
+
+ /* if we are going from non-blocking to blocking flush here */
+ if (pqIsnonblocking(conn) && pqFlush(conn))
+ return (-1);
+
+ return (0);
+ }
+
+ /* return the blocking status of the database connection, TRUE == nonblocking,
+ FALSE == blocking
+ */
+ int
+ PQisnonblocking(const PGconn *conn)
+ {
+
+ return (pqIsnonblocking(conn));
+ }
+
+ /* try to force data out, really only useful for non-blocking users */
+ int
+ PQflush(PGconn *conn)
+ {
+
+ return (pqFlush(conn)); }
Index: fe-misc.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
retrieving revision 1.33
diff -u -c -IHeader -I$Id: -r1.33 fe-misc.c
cvs diff: conflicting specifications of output style
*** fe-misc.c 1999/11/30 03:08:19 1.33
--- fe-misc.c 2000/01/12 03:12:14
***************
*** 86,91 ****
--- 86,122 ---- { size_t avail = Max(conn->outBufSize - conn->outCount, 0);
+ /*
+ * if we are non-blocking and the send queue is too full to buffer this
+ * request then try to flush some and return an error
+ */
+ if (pqIsnonblocking(conn) && nbytes > avail && pqFlush(conn))
+ {
+ /*
+ * even if the flush failed we may still have written some
+ * data, recalculate the size of the send-queue relative
+ * to the amount we have to send, we may be able to queue it
+ * afterall even though it's not sent to the database it's
+ * ok, any routines that check the data coming from the
+ * database better call pqFlush() anyway.
+ */
+ if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "pqPutBytes -- pqFlush couldn't flush enough"
+ " data: space available: %d, space needed %d\n",
+ Max(conn->outBufSize - conn->outCount, 0), nbytes);
+ return EOF;
+ }
+ }
+
+ /*
+ * is the amount of data to be sent is larger than the size of the
+ * output buffer then we must flush it to make more room.
+ *
+ * the code above will make sure the loop conditional is never
+ * true for non-blocking connections
+ */ while (nbytes > avail) { memcpy(conn->outBuffer + conn->outCount, s, avail);
***************
*** 548,553 ****
--- 579,592 ---- return EOF; }
+ /*
+ * don't try to send zero data, allows us to use this function
+ * without too much worry about overhead
+ */
+ if (len == 0)
+ return (0);
+
+ /* while there's still data to send */ while (len > 0) { /* Prevent being SIGPIPEd if backend has
closedthe connection. */
***************
*** 556,561 ****
--- 595,601 ---- #endif int sent;
+ #ifdef USE_SSL if (conn->ssl) sent = SSL_write(conn->ssl, ptr, len);
***************
*** 585,590 ****
--- 625,632 ---- case EWOULDBLOCK: break; #endif
+ case EINTR:
+ continue; case EPIPE: #ifdef ECONNRESET
***************
*** 616,628 **** ptr += sent; len -= sent; } if (len > 0) {
/*We didn't send it all, wait till we can send more */
- /* At first glance this looks as though it should block. I think
- * that it will be OK though, as long as the socket is
- * non-blocking. */ if (pqWait(FALSE, TRUE, conn)) return EOF; }
--- 658,688 ---- ptr += sent; len -= sent; }
+ if (len > 0) { /* We didn't send it all, wait till we can send more */
+
+ /*
+ * if the socket is in non-blocking mode we may need
+ * to abort here
+ */
+ #ifdef USE_SSL
+ /* can't do anything for our SSL users yet */
+ if (conn->ssl == NULL)
+ {
+ #endif
+ if (pqIsnonblocking(conn))
+ {
+ /* shift the contents of the buffer */
+ memmove(conn->outBuffer, ptr, len);
+ conn->outCount = len;
+ return EOF;
+ }
+ #ifdef USE_SSL
+ }
+ #endif if (pqWait(FALSE, TRUE, conn)) return EOF; }
Index: libpq-fe.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.54
diff -u -c -IHeader -I$Id: -r1.54 libpq-fe.h
cvs diff: conflicting specifications of output style
*** libpq-fe.h 2000/01/14 05:33:15 1.54
--- libpq-fe.h 2000/01/14 22:45:33
***************
*** 261,266 ****
--- 261,273 ---- extern int PQgetlineAsync(PGconn *conn, char *buffer, int bufsize); extern int
PQputnbytes(PGconn*conn, const char *buffer, int nbytes); extern int PQendcopy(PGconn *conn);
+
+ /* Set blocking/nonblocking connection to the backend */
+ extern int PQsetnonblocking(PGconn *conn, int arg);
+ extern int PQisnonblocking(const PGconn *conn);
+
+ /* Force the write buffer to be written (or at least try) */
+ extern int PQflush(PGconn *conn); /* * "Fast path" interface --- not really recommended for
application
Index: libpq-int.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
retrieving revision 1.15
diff -u -c -IHeader -I$Id: -r1.15 libpq-int.h
cvs diff: conflicting specifications of output style
*** libpq-int.h 2000/01/14 05:33:15 1.15
--- libpq-int.h 2000/01/14 18:32:51
***************
*** 214,219 ****
--- 214,222 ---- int inEnd; /* offset to first position after avail
* data */
+ int nonblocking; /* whether this connection is using a blocking
+ * socket to the backend or not */
+ /* Buffer for data not yet sent to backend */ char *outBuffer; /* currently allocated buffer */
int outBufSize; /* allocated size of buffer */
***************
*** 297,301 ****
--- 300,310 ---- #define strerror(A) (sys_errlist[(A)]) #endif /* sunos4 */ #endif /* !strerror */
+
+ /*
+ * this is so that we can check is a connection is non-blocking internally
+ * without the overhead of a function call
+ */
+ #define pqIsnonblocking(conn) (conn->nonblocking) #endif /* LIBPQ_INT_H */
on a side note miscadmin.h causes problems on FreeBSD because it uses
pid_t without having included sys/types.h
thanks!
--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
The FreeBSD Documentation Project (FDP) has excellent references to get a general idea on building sgml docs. First, you can install the textproc/docproj port or, if you're not running freebsd, refer to the website to see which programs you need. Second, you can read the FDP Primer which details how everything comes together: http://www.freebsd.org/tutorials/docproj-primer/ Furthermore, again if you happen to be running FreeBSD, you can grab the doc src using cvsup. The proper reference is also documented somewhere in the Primer or in the Synchronisation chapter in the Handbook. Keep at it, sgml and the docbook stylesheets are really worthwhile when you start getting the hang of it. Marc > * Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote: > > Don Baccus <dhogaza@pacifier.com> writes: > > > At 05:27 PM 1/8/00 -0500, Tom Lane wrote: > > >> I also object strongly to the lack of documentation. > > > > > ... I know there are some folks who aren't native-english speakers, so > > > perhaps you don't want to require that the implementor of such patches > > > provide the final documentation wording. But the information should > > > be there and spelled out in a form that can be very easily moved to > > > the docs. > > > > Oh, absolutely. Thomas, our master of the docs, has always had the > > policy of "give me some words, I'll take care of formatting and > > editing..." > > > > I was probably too harsh on Alfred last night, since in fact his code > > was fairly well commented, and some minimal doco could have been > > extracted from the routine headers. But on a change like this, I think > > some paragraphs of coherent high-level explanation are needed: what it > > does, when and why you'd use it. I didn't see that anywhere... > > I've actually been trying to work on the sgml and failing miserably, > I have no clue how this stuff works (sgml compilation) are you asking > for a couple of paragraphs that describe the proposed changes? > > If so I hope this suffices, if not some help on building the sgml > would be much appreciated: > > -------- > > Summary: > > First and foremost, great pains have been taken so that there > are _no_ compatibility issues. > > If a 6.5.3 libpq program should not behave any differently > with this patches in place, all they do is offer a step closer > to a truly non-blocking connection to the backend and address > some issues with non-blocking connections. > > ---- > > Added functions: > > int PQisnonblocking(static PGconn *conn); > > returns whether or not the socket is in blocking mode, however... > it doesn't actually check the socket flags, it relies on the user > to call 'PQsetnonblocking()' to keep the internal state of libpq > sane. users should no longer use 'PQsocket()' to retrieve the > socket and 'manually' ioctl/fcntl it to non-blocking > > returns TRUE if the socket has been set to blocking more, FALSE > if the socket is blocking > > int PQflush(PGconn *conn); > > flush the send-queue to the backend, just make this visible to the > user for convience, as the internal function works, 0 for success, > EOF for any failure. > > int PQsetnonblocking(PGconn *conn, int arg); > > actually set the connection to the backend to blocking or > non-blocking arg should be set to TRUE to set the connection to > non-blocking or FALSE to set it blocking. > > there's an implied blocking flush of the send-queue which is > really ok as the user is either 'going into' or 'returning from' > a blocking state > > returns 0 for success, -1 for failure > > --- > > New functionality: > > PQsetblocking() allows libpq to know what behavior the user really > wants, the user will not block sending data to the backend, > potentially if i had a constant stream of data and was doing a > COPYIN it'd never finish because unless the backend lost the > connection I would block while sending until the backend can take > more data. > > --- > > Implementation changes: > > none should be visible to programs based on 6.5.3's libpq. > > programs based on later versions of libpq will notice that > the non-blocking connection functions will set the state of > the connection to non-blocking automatically. > > when the connection is set non-blocking pqFlush() will not block > if the sendqueue would be filled by new data inserted into the > the queue. > > functions that poll for data from the backend implicitly _try_ > flush the send queue if set to non-blocking. This allows the > polling to act as a context for pushing queued data to the backend. > > --- > > Problems: > > We need some sort of send-queue commit reservations so that > there's no chance of us sending a partial queury down the pipe > to the backend, right now this is hacked around by potentially > blocking in non-blocking mode if something 'goes terribly wrong' > I plan to fix this. > > --- > > Quirks: > > PQexec() assumes the caller wants blocking behavior and will set the > connection to blocking for the duration of the PQexec() call, it will > then restore it > > --- > > Internal changes: > > new field in PGconn 'int nonblocking' set to 1 if the connection is > nonblocking, 0 if blocking (default) > > macro pqIsnonblocking(PGconn) to avoid a function call to check blocking > status (only visible if libpq-int.h is included) > > the internal function connectMakeNonblocking() has been replaced with > PQsetblocking() > > restart a send if EINTR is reported during a flush. > > --- > > Lastly: > > This is work in progress, I will be working towards making libpq > better at not blocking. > > here are the diffs: > > Index: fe-connect.c > =================================================================== > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v > retrieving revision 1.109 > diff -u -c -IHeader -I$Id: -r1.109 fe-connect.c > cvs diff: conflicting specifications of output style > *** fe-connect.c 2000/01/14 05:33:15 1.109 > --- fe-connect.c 2000/01/14 18:36:54 > *************** > *** 594,624 **** > return 0; > } > > - > - /* ---------- > - * connectMakeNonblocking - > - * Make a connection non-blocking. > - * Returns 1 if successful, 0 if not. > - * ---------- > - */ > - static int > - connectMakeNonblocking(PGconn *conn) > - { > - #ifndef WIN32 > - if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0) > - #else > - if (ioctlsocket(conn->sock, FIONBIO, &on) != 0) > - #endif > - { > - printfPQExpBuffer(&conn->errorMessage, > - "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n", > - errno, strerror(errno)); > - return 0; > - } > - > - return 1; > - } > - > /* ---------- > * connectNoDelay - > * Sets the TCP_NODELAY socket option. > --- 594,599 ---- > *************** > *** 789,795 **** > * Ewan Mellor <eem21@cam.ac.uk>. > * ---------- */ > #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL) > ! if (!connectMakeNonblocking(conn)) > goto connect_errReturn; > #endif > > --- 764,770 ---- > * Ewan Mellor <eem21@cam.ac.uk>. > * ---------- */ > #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL) > ! if (PQsetnonblocking(conn, TRUE) != 0) > goto connect_errReturn; > #endif > > *************** > *** 898,904 **** > /* This makes the connection non-blocking, for all those cases which forced us > not to do it above. */ > #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL) > ! if (!connectMakeNonblocking(conn)) > goto connect_errReturn; > #endif > > --- 873,879 ---- > /* This makes the connection non-blocking, for all those cases which forced us > not to do it above. */ > #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL) > ! if (PQsetnonblocking(conn, TRUE) != 0) > goto connect_errReturn; > #endif > > *************** > *** 1702,1707 **** > --- 1677,1683 ---- > conn->inBuffer = (char *) malloc(conn->inBufSize); > conn->outBufSize = 8 * 1024; > conn->outBuffer = (char *) malloc(conn->outBufSize); > + conn->nonblocking = FALSE; > initPQExpBuffer(&conn->errorMessage); > initPQExpBuffer(&conn->workBuffer); > if (conn->inBuffer == NULL || > *************** > *** 1812,1817 **** > --- 1788,1794 ---- > conn->lobjfuncs = NULL; > conn->inStart = conn->inCursor = conn->inEnd = 0; > conn->outCount = 0; > + conn->nonblocking = FALSE; > > } > > Index: fe-exec.c > =================================================================== > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v > retrieving revision 1.86 > diff -u -c -IHeader -I$Id: -r1.86 fe-exec.c > cvs diff: conflicting specifications of output style > *** fe-exec.c 1999/11/11 00:10:14 1.86 > --- fe-exec.c 2000/01/14 22:47:07 > *************** > *** 13,18 **** > --- 13,19 ---- > */ > #include <errno.h> > #include <ctype.h> > + #include <fcntl.h> > > #include "postgres.h" > #include "libpq-fe.h" > *************** > *** 24,30 **** > #include <unistd.h> > #endif > > - > /* keep this in same order as ExecStatusType in libpq-fe.h */ > const char *const pgresStatus[] = { > "PGRES_EMPTY_QUERY", > --- 25,30 ---- > *************** > *** 514,526 **** > conn->curTuple = NULL; > > /* send the query to the backend; */ > ! /* the frontend-backend protocol uses 'Q' to designate queries */ > ! if (pqPutnchar("Q", 1, conn) || > ! pqPuts(query, conn) || > ! pqFlush(conn)) > { > ! handleSendFailure(conn); > ! return 0; > } > > /* OK, it's launched! */ > --- 514,566 ---- > conn->curTuple = NULL; > > /* send the query to the backend; */ > ! > ! /* > ! * in order to guarantee that we don't send a partial query > ! * where we would become out of sync with the backend and/or > ! * block during a non-blocking connection we must first flush > ! * the send buffer before sending more data > ! * > ! * an alternative is to implement 'queue reservations' where > ! * we are able to roll up a transaction > ! * (the 'Q' along with our query) and make sure we have > ! * enough space for it all in the send buffer. > ! */ > ! if (pqIsnonblocking(conn)) > { > ! /* > ! * the buffer must have emptied completely before we allow > ! * a new query to be buffered > ! */ > ! if (pqFlush(conn)) > ! return 0; > ! /* 'Q' == queries */ > ! /* XXX: if we fail here we really ought to not block */ > ! if (pqPutnchar("Q", 1, conn) || > ! pqPuts(query, conn)) > ! { > ! handleSendFailure(conn); > ! return 0; > ! } > ! /* > ! * give the data a push, ignore the return value as > ! * ConsumeInput() will do any aditional flushing if needed > ! */ > ! (void) pqFlush(conn); > ! } > ! else > ! { > ! /* > ! * the frontend-backend protocol uses 'Q' to > ! * designate queries > ! */ > ! if (pqPutnchar("Q", 1, conn) || > ! pqPuts(query, conn) || > ! pqFlush(conn)) > ! { > ! handleSendFailure(conn); > ! return 0; > ! } > } > > /* OK, it's launched! */ > *************** > *** 574,580 **** > --- 614,630 ---- > * we will NOT block waiting for more input. > */ > if (pqReadData(conn) < 0) > + { > + /* > + * for non-blocking connections > + * try to flush the send-queue otherwise we may never get a > + * responce for something that may not have already been sent > + * because it's in our write buffer! > + */ > + if (pqIsnonblocking(conn)) > + (void) pqFlush(conn); > return 0; > + } > /* Parsing of the data waits till later. */ > return 1; > } > *************** > *** 1088,1093 **** > --- 1138,1153 ---- > { > PGresult *result; > PGresult *lastResult; > + bool savedblocking; > + > + /* > + * we assume anyone calling PQexec wants blocking behaviour, > + * we force the blocking status of the connection to blocking > + * for the duration of this function and restore it on return > + */ > + savedblocking = pqIsnonblocking(conn); > + if (PQsetnonblocking(conn, FALSE) == -1) > + return NULL; > > /* > * Silently discard any prior query result that application didn't > *************** > *** 1102,1115 **** > PQclear(result); > printfPQExpBuffer(&conn->errorMessage, > "PQexec: you gotta get out of a COPY state yourself.\n"); > ! return NULL; > } > PQclear(result); > } > > /* OK to send the message */ > if (!PQsendQuery(conn, query)) > ! return NULL; > > /* > * For backwards compatibility, return the last result if there are > --- 1162,1176 ---- > PQclear(result); > printfPQExpBuffer(&conn->errorMessage, > "PQexec: you gotta get out of a COPY state yourself.\n"); > ! /* restore blocking status */ > ! goto errout; > } > PQclear(result); > } > > /* OK to send the message */ > if (!PQsendQuery(conn, query)) > ! goto errout; /* restore blocking status */ > > /* > * For backwards compatibility, return the last result if there are > *************** > *** 1142,1148 **** > --- 1203,1217 ---- > result->resultStatus == PGRES_COPY_OUT) > break; > } > + > + if (PQsetnonblocking(conn, savedblocking) == -1) > + return NULL; > return lastResult; > + > + errout: > + if (PQsetnonblocking(conn, savedblocking) == -1) > + return NULL; > + return NULL; > } > > > *************** > *** 1431,1438 **** > "PQendcopy() -- I don't think there's a copy in progress.\n"); > return 1; > } > > ! (void) pqFlush(conn); /* make sure no data is waiting to be sent */ > > /* Return to active duty */ > conn->asyncStatus = PGASYNC_BUSY; > --- 1500,1516 ---- > "PQendcopy() -- I don't think there's a copy in progress.\n"); > return 1; > } > + > + /* > + * make sure no data is waiting to be sent, > + * abort if we are non-blocking and the flush fails > + */ > + if (pqFlush(conn) && pqIsnonblocking(conn)) > + return (1); > > ! /* non blocking connections may have to abort at this point. */ > ! if (pqIsnonblocking(conn) && PQisBusy(conn)) > ! return (1); > > /* Return to active duty */ > conn->asyncStatus = PGASYNC_BUSY; > *************** > *** 2025,2028 **** > --- 2103,2192 ---- > return 1; > else > return 0; > + } > + > + /* PQsetnonblocking: > + sets the PGconn's database connection non-blocking if the arg is TRUE > + or makes it non-blocking if the arg is FALSE, this will not protect > + you from PQexec(), you'll only be safe when using the non-blocking > + API > + Needs to be called only on a connected database connection. > + */ > + > + int > + PQsetnonblocking(PGconn *conn, int arg) > + { > + int fcntlarg; > + > + arg = (arg == TRUE) ? 1 : 0; > + /* early out if the socket is already in the state requested */ > + if (arg == conn->nonblocking) > + return (0); > + > + /* > + * to guarantee constancy for flushing/query/result-polling behavior > + * we need to flush the send queue at this point in order to guarantee > + * proper behavior. > + * this is ok because either they are making a transition > + * _from_ or _to_ blocking mode, either way we can block them. > + */ > + /* if we are going from blocking to non-blocking flush here */ > + if (!pqIsnonblocking(conn) && pqFlush(conn)) > + return (-1); > + > + > + #ifdef USE_SSL > + if (conn->ssl) > + { > + printfPQExpBuffer(&conn->errorMessage, > + "PQsetnonblocking() -- not supported when using SSL\n"); > + return (-1); > + } > + #endif /* USE_SSL */ > + > + #ifndef WIN32 > + fcntlarg = fcntl(conn->sock, F_GETFL, 0); > + if (fcntlarg == -1) > + return (-1); > + > + if ((arg == TRUE && > + fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) || > + (arg == FALSE && > + fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1)) > + #else > + fcntlarg = arg; > + if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0) > + #endif > + { > + printfPQExpBuffer(&conn->errorMessage, > + "PQsetblocking() -- unable to set nonblocking status to %s\n", > + arg == TRUE ? "TRUE" : "FALSE"); > + return (-1); > + } > + > + conn->nonblocking = arg; > + > + /* if we are going from non-blocking to blocking flush here */ > + if (pqIsnonblocking(conn) && pqFlush(conn)) > + return (-1); > + > + return (0); > + } > + > + /* return the blocking status of the database connection, TRUE == nonblocking, > + FALSE == blocking > + */ > + int > + PQisnonblocking(const PGconn *conn) > + { > + > + return (pqIsnonblocking(conn)); > + } > + > + /* try to force data out, really only useful for non-blocking users */ > + int > + PQflush(PGconn *conn) > + { > + > + return (pqFlush(conn)); > } > Index: fe-misc.c > =================================================================== > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v > retrieving revision 1.33 > diff -u -c -IHeader -I$Id: -r1.33 fe-misc.c > cvs diff: conflicting specifications of output style > *** fe-misc.c 1999/11/30 03:08:19 1.33 > --- fe-misc.c 2000/01/12 03:12:14 > *************** > *** 86,91 **** > --- 86,122 ---- > { > size_t avail = Max(conn->outBufSize - conn->outCount, 0); > > + /* > + * if we are non-blocking and the send queue is too full to buffer this > + * request then try to flush some and return an error > + */ > + if (pqIsnonblocking(conn) && nbytes > avail && pqFlush(conn)) > + { > + /* > + * even if the flush failed we may still have written some > + * data, recalculate the size of the send-queue relative > + * to the amount we have to send, we may be able to queue it > + * afterall even though it's not sent to the database it's > + * ok, any routines that check the data coming from the > + * database better call pqFlush() anyway. > + */ > + if (nbytes > Max(conn->outBufSize - conn->outCount, 0)) > + { > + printfPQExpBuffer(&conn->errorMessage, > + "pqPutBytes -- pqFlush couldn't flush enough" > + " data: space available: %d, space needed %d\n", > + Max(conn->outBufSize - conn->outCount, 0), nbytes); > + return EOF; > + } > + } > + > + /* > + * is the amount of data to be sent is larger than the size of the > + * output buffer then we must flush it to make more room. > + * > + * the code above will make sure the loop conditional is never > + * true for non-blocking connections > + */ > while (nbytes > avail) > { > memcpy(conn->outBuffer + conn->outCount, s, avail); > *************** > *** 548,553 **** > --- 579,592 ---- > return EOF; > } > > + /* > + * don't try to send zero data, allows us to use this function > + * without too much worry about overhead > + */ > + if (len == 0) > + return (0); > + > + /* while there's still data to send */ > while (len > 0) > { > /* Prevent being SIGPIPEd if backend has closed the connection. */ > *************** > *** 556,561 **** > --- 595,601 ---- > #endif > > int sent; > + > #ifdef USE_SSL > if (conn->ssl) > sent = SSL_write(conn->ssl, ptr, len); > *************** > *** 585,590 **** > --- 625,632 ---- > case EWOULDBLOCK: > break; > #endif > + case EINTR: > + continue; > > case EPIPE: > #ifdef ECONNRESET > *************** > *** 616,628 **** > ptr += sent; > len -= sent; > } > if (len > 0) > { > /* We didn't send it all, wait till we can send more */ > > - /* At first glance this looks as though it should block. I think > - * that it will be OK though, as long as the socket is > - * non-blocking. */ > if (pqWait(FALSE, TRUE, conn)) > return EOF; > } > --- 658,688 ---- > ptr += sent; > len -= sent; > } > + > if (len > 0) > { > /* We didn't send it all, wait till we can send more */ > + > + /* > + * if the socket is in non-blocking mode we may need > + * to abort here > + */ > + #ifdef USE_SSL > + /* can't do anything for our SSL users yet */ > + if (conn->ssl == NULL) > + { > + #endif > + if (pqIsnonblocking(conn)) > + { > + /* shift the contents of the buffer */ > + memmove(conn->outBuffer, ptr, len); > + conn->outCount = len; > + return EOF; > + } > + #ifdef USE_SSL > + } > + #endif > > if (pqWait(FALSE, TRUE, conn)) > return EOF; > } > Index: libpq-fe.h > =================================================================== > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v > retrieving revision 1.54 > diff -u -c -IHeader -I$Id: -r1.54 libpq-fe.h > cvs diff: conflicting specifications of output style > *** libpq-fe.h 2000/01/14 05:33:15 1.54 > --- libpq-fe.h 2000/01/14 22:45:33 > *************** > *** 261,266 **** > --- 261,273 ---- > extern int PQgetlineAsync(PGconn *conn, char *buffer, int bufsize); > extern int PQputnbytes(PGconn *conn, const char *buffer, int nbytes); > extern int PQendcopy(PGconn *conn); > + > + /* Set blocking/nonblocking connection to the backend */ > + extern int PQsetnonblocking(PGconn *conn, int arg); > + extern int PQisnonblocking(const PGconn *conn); > + > + /* Force the write buffer to be written (or at least try) */ > + extern int PQflush(PGconn *conn); > > /* > * "Fast path" interface --- not really recommended for application > Index: libpq-int.h > =================================================================== > RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v > retrieving revision 1.15 > diff -u -c -IHeader -I$Id: -r1.15 libpq-int.h > cvs diff: conflicting specifications of output style > *** libpq-int.h 2000/01/14 05:33:15 1.15 > --- libpq-int.h 2000/01/14 18:32:51 > *************** > *** 214,219 **** > --- 214,222 ---- > int inEnd; /* offset to first position after avail > * data */ > > + int nonblocking; /* whether this connection is using a blocking > + * socket to the backend or not */ > + > /* Buffer for data not yet sent to backend */ > char *outBuffer; /* currently allocated buffer */ > int outBufSize; /* allocated size of buffer */ > *************** > *** 297,301 **** > --- 300,310 ---- > #define strerror(A) (sys_errlist[(A)]) > #endif /* sunos4 */ > #endif /* !strerror */ > + > + /* > + * this is so that we can check is a connection is non-blocking internally > + * without the overhead of a function call > + */ > + #define pqIsnonblocking(conn) (conn->nonblocking) > > #endif /* LIBPQ_INT_H */ > > > on a side note miscadmin.h causes problems on FreeBSD because it uses > pid_t without having included sys/types.h > > thanks! > > -- > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > > ************ >
* admin <admin@wtbwts.com> [000114 11:35] wrote: > Alfred wrote: > > * Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote: > > > Don Baccus <dhogaza@pacifier.com> writes: > > > > At 05:27 PM 1/8/00 -0500, Tom Lane wrote: > > > >> I also object strongly to the lack of documentation. > > > > > > > ... I know there are some folks who aren't native-english speakers, so > > > > perhaps you don't want to require that the implementor of such patches > > > > provide the final documentation wording. But the information should > > > > be there and spelled out in a form that can be very easily moved to > > > > the docs. > > > > > > Oh, absolutely. Thomas, our master of the docs, has always had the > > > policy of "give me some words, I'll take care of formatting and > > > editing..." > > > > > > I was probably too harsh on Alfred last night, since in fact his code > > > was fairly well commented, and some minimal doco could have been > > > extracted from the routine headers. But on a change like this, I think > > > some paragraphs of coherent high-level explanation are needed: what it > > > does, when and why you'd use it. I didn't see that anywhere... > > > > I've actually been trying to work on the sgml and failing miserably, > > I have no clue how this stuff works (sgml compilation) are you asking > > for a couple of paragraphs that describe the proposed changes? > > > > If so I hope this suffices, if not some help on building the sgml > > would be much appreciated: > > > > -------- > > > The FreeBSD Documentation Project (FDP) has excellent references to get a > general idea on building sgml docs. First, you can install the > textproc/docproj port or, if you're not running freebsd, refer to the > website to see which programs you need. Second, you can read the FDP > Primer which details how everything comes together: > http://www.freebsd.org/tutorials/docproj-primer/ > > Furthermore, again if you happen to be running FreeBSD, you can grab the > doc src using cvsup. The proper reference is also documented somewhere in > the Primer or in the Synchronisation chapter in the Handbook. > > Keep at it, sgml and the docbook stylesheets are really worthwhile when > you start getting the hang of it. > Marc 'course I run freebsd. :) I even have the docproj port installed, however it seems that there's some things missing here, (see the end of this message). I really have no problem with commenting my code nor do I have a problem with producing documentation for these changes, however I'm _extremely_ pressed for time with this project, haven't slept in 2 days and I and don't have time to fight with building the sgml files to check that my changes/additions are valid, I'd much rather focus on working on the rest of libpq for blocking issues and getting my app into test mode. Perhaps someone can offer a step-by-step to building _postgresql's_ doc files, or maybe there's a machine out there where this will build properly and someone can give me an account on it? If so then I'll be glad to update the docs myself, otherwise I'd also be happy to provide coupious amounts of plaintext docs and comments in my code like I have been so far. thanks, -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] ~/pgcvs/pgsql/doc/src % gmake gmake all gmake[1]: Entering directory `/home/bright/pgcvs/pgsql/doc/src' gmake -C sgml clean gmake[2]: Entering directory `/home/bright/pgcvs/pgsql/doc/src/sgml' (rm -rf HTML.manifest *.html *.htm *.1 *.l man1 manl manpage*) gmake[2]: Leaving directory `/home/bright/pgcvs/pgsql/doc/src/sgml' gmake -C sgml admin.html gmake[2]: Entering directory `/home/bright/pgcvs/pgsql/doc/src/sgml' (rm -rf *.htm) jade -D ref -D ../graphics -V %use-id-as-filename% -d /home/users/t/thomas/db118.d/docbook/html/docbook.dsl -t sgml admin.sgml ^^^^^^^^^^^^---- huh? ~/pgcvs/pgsql/doc % find . -name "*.dsl" ~/pgcvs/pgsql/doc % continues... jade:admin.sgml:26:59:W: cannot generate system identifier for public text "-//Davenport//DTD DocBook V3.0//EN" jade:admin.sgml:51:0:E: reference to entity "BOOK" for which no system identifier could be generated jade:admin.sgml:26:0: entity was defined here jade:admin.sgml:51:0:E: DTD did not contain element declaration for document type name jade:admin.sgml:53:9:E: there is no attribute "ID" jade:admin.sgml:53:16:E: element "BOOK" undefined jade:admin.sgml:57:7:E: element "TITLE" undefined jade:admin.sgml:58:10:E: element "BOOKINFO" undefined jade:admin.sgml:59:14:E: element "RELEASEINFO" undefined jade:admin.sgml:60:13:E: element "BOOKBIBLIO" undefined jade:admin.sgml:61:15:E: element "AUTHORGROUP" undefined jade:admin.sgml:62:15:E: element "CORPAUTHOR" undefined jade:admin.sgml:67:10:E: element "EDITOR" undefined jade:admin.sgml:68:14:E: element "FIRSTNAME" undefined jade:admin.sgml:69:12:E: element "SURNAME" undefined jade:admin.sgml:70:16:E: element "AFFILIATION" undefined jade:admin.sgml:71:13:E: element "ORGNAME" undefined jade:admin.sgml:82:8:E: element "DATE" undefined jade:admin.sgml:85:14:E: element "LEGALNOTICE" undefined jade:admin.sgml:86:8:E: element "PARA" undefined jade:admin.sgml:87:16:E: element "PRODUCTNAME" undefined jade:admin.sgml:87:56:E: general entity "copy" not defined and no default entity jade:admin.sgml:107:13:E: there is no attribute "ID" jade:admin.sgml:107:22:E: element "PREFACE" undefined jade:admin.sgml:108:8:E: element "TITLE" undefined jade:admin.sgml:110:7:E: element "PARA" undefined jade:admin.sgml:111:15:E: element "PRODUCTNAME" undefined jade:admin.sgml:117:15:E: element "PRODUCTNAME" undefined jade:intro-ag.sgml:1:13:E: there is no attribute "ID" jade:intro-ag.sgml:1:23:E: element "CHAPTER" undefined jade:intro-ag.sgml:2:8:E: element "TITLE" undefined jade:intro-ag.sgml:4:7:E: element "PARA" undefined jade:intro-ag.sgml:6:14:E: there is no attribute "URL" jade:intro-ag.sgml:6:38:E: element "ULINK" undefined jade:intro-ag.sgml:6:51:E: element "PRODUCTNAME" undefined jade:intro-ag.sgml:10:15:E: element "PRODUCTNAME" undefined jade:intro-ag.sgml:11:74:E: element "ULINK" undefined jade:intro-ag.sgml:12:16:E: element "PRODUCTNAME" undefined jade:intro-ag.sgml:13:19:E: element "PRODUCTNAME" undefined jade:intro-ag.sgml:15:55:E: element "ACRONYM" undefined jade:intro-ag.sgml:16:33:E: element "ACRONYM" undefined jade:intro-ag.sgml:17:23:E: element "ACRONYM" undefined jade:info.sgml:1:6:E: element "SECT1" undefined jade:info.sgml:2:7:E: element "TITLE" undefined jade:info.sgml:4:6:E: element "PARA" undefined jade:info.sgml:8:14:E: element "VARIABLELIST" undefined jade:info.sgml:9:15:E: element "VARLISTENTRY" undefined jade:info.sgml:10:8:E: element "TERM" undefined jade:info.sgml:11:12:E: element "LISTITEM" undefined jade:info.sgml:12:9:E: element "PARA" undefined jade:info.sgml:18:15:E: element "VARLISTENTRY" undefined jade:info.sgml:19:8:E: element "TERM" undefined jade:info.sgml:20:12:E: element "LISTITEM" undefined jade:info.sgml:21:9:E: element "PARA" undefined jade:info.sgml:27:15:E: element "VARLISTENTRY" undefined jade:info.sgml:28:8:E: element "TERM" undefined jade:info.sgml:29:12:E: element "LISTITEM" undefined jade:info.sgml:30:9:E: element "PARA" undefined jade:info.sgml:38:15:E: element "VARLISTENTRY" undefined jade:info.sgml:39:8:E: element "TERM" undefined jade:info.sgml:40:12:E: element "LISTITEM" undefined jade:info.sgml:41:9:E: element "PARA" undefined jade:info.sgml:47:15:E: element "VARLISTENTRY" undefined jade:info.sgml:48:8:E: element "TERM" undefined jade:info.sgml:49:12:E: element "LISTITEM" undefined jade:info.sgml:50:9:E: element "PARA" undefined jade:info.sgml:51:33:E: element "PRODUCTNAME" undefined jade:info.sgml:53:17:E: element "PRODUCTNAME" undefined jade:info.sgml:55:15:E: element "CITETITLE" undefined jade:info.sgml:56:41:E: element "CITETITLE" undefined jade:info.sgml:61:15:E: element "VARLISTENTRY" undefined jade:info.sgml:62:8:E: element "TERM" undefined jade:info.sgml:63:12:E: element "LISTITEM" undefined jade:info.sgml:64:9:E: element "PARA" undefined jade:info.sgml:66:41:E: element "CITETITLE" undefined jade:info.sgml:72:6:E: element "PARA" undefined jade:info.sgml:74:14:E: element "PRODUCTNAME" undefined jade:info.sgml:77:14:E: element "VARIABLELIST" undefined jade:info.sgml:78:15:E: element "VARLISTENTRY" undefined jade:info.sgml:79:8:E: element "TERM" undefined jade:info.sgml:80:12:E: element "LISTITEM" undefined jade:info.sgml:81:9:E: element "PARA" undefined jade:info.sgml:87:15:E: element "VARLISTENTRY" undefined jade:info.sgml:88:8:E: element "TERM" undefined jade:info.sgml:89:12:E: element "LISTITEM" undefined jade:info.sgml:90:9:E: element "PARA" undefined jade:info.sgml:97:15:E: element "VARLISTENTRY" undefined jade:info.sgml:98:8:E: element "TERM" undefined jade:info.sgml:99:12:E: element "LISTITEM" undefined jade:info.sgml:100:9:E: element "PARA" undefined jade:info.sgml:106:15:E: element "VARLISTENTRY" undefined jade:info.sgml:107:8:E: element "TERM" undefined jade:info.sgml:108:12:E: element "LISTITEM" undefined jade:info.sgml:109:9:E: element "PARA" undefined jade:info.sgml:111:32:E: element "ULINK" undefined jade:info.sgml:111:45:E: element "PRODUCTNAME" undefined jade:info.sgml:113:28:E: element "PRODUCTNAME" undefined jade:info.sgml:119:15:E: element "VARLISTENTRY" undefined jade:info.sgml:120:8:E: element "TERM" undefined jade:info.sgml:121:12:E: element "LISTITEM" undefined jade:info.sgml:122:9:E: element "PARA" undefined jade:info.sgml:124:53:E: element "ULINK" undefined jade:info.sgml:125:67:E: element "ULINK" undefined jade:info.sgml:133:15:E: element "VARLISTENTRY" undefined jade:info.sgml:134:8:E: element "TERM" undefined jade:info.sgml:135:12:E: element "LISTITEM" undefined jade:info.sgml:136:9:E: element "PARA" undefined jade:info.sgml:137:17:E: element "PRODUCTNAME" undefined jade:info.sgml:139:37:E: element "PRODUCTNAME" undefined jade:info.sgml:147:9:E: element "PARA" undefined jade:info.sgml:151:50:E: element "ULINK" undefined jade:info.sgml:152:64:E: element "ULINK" undefined jade:notation.sgml:1:10:E: there is no attribute "ID" jade:notation.sgml:1:23:E: element "SECT1" undefined jade:notation.sgml:2:7:E: element "TITLE" undefined jade:notation.sgml:4:6:E: element "PARA" undefined jade:notation.sgml:6:12:E: element "FIRSTTERM" undefined jade:notation.sgml:8:14:E: element "PRODUCTNAME" undefined jade:notation.sgml:10:14:E: element "PRODUCTNAME" undefined jade:notation.sgml:13:14:E: element "PRODUCTNAME" undefined jade:notation.sgml:16:6:E: element "PARA" undefined jade:notation.sgml:18:14:E: element "PRODUCTNAME" undefined jade:notation.sgml:18:48:E: element "FIRSTTERM" undefined jade:notation.sgml:19:32:E: element "REPLACEABLE" undefined jade:notation.sgml:20:27:E: element "PRODUCTNAME" undefined jade:notation.sgml:24:31:E: element "PRODUCTNAME" undefined jade:notation.sgml:26:28:E: element "PRODUCTNAME" undefined jade:notation.sgml:27:11:E: element "EMPHASIS" undefined jade:notation.sgml:28:73:E: element "FIRSTTERM" undefined jade:notation.sgml:29:66:E: element "FIRSTTERM" undefined jade:notation.sgml:33:6:E: element "PARA" undefined jade:notation.sgml:35:12:E: element "FIRSTTERM" undefined jade:notation.sgml:36:13:E: element "ACRONYM" undefined jade:notation.sgml:37:14:E: element "PRODUCTNAME" undefined jade:notation.sgml:41:14:E: element "APPLICATION" undefined jade:notation.sgml:44:6:E: element "PARA" undefined jade:notation.sgml:45:18:E: element "APPLICATION" undefined jade:notation.sgml:47:21:E: element "PRODUCTNAME" undefined jade:notation.sgml:48:51:E: element "APPLICATION" undefined jade:notation.sgml:50:38:E: element "APPLICATION" undefined jade:notation.sgml:56:6:E: element "PARA" undefined jade:notation.sgml:57:18:E: element "PRODUCTNAME" undefined jade:notation.sgml:58:45:E: element "APPLICATION" undefined jade:notation.sgml:60:14:E: element "PRODUCTNAME" undefined jade:notation.sgml:68:20:E: element "SECT1" undefined jade:notation.sgml:69:7:E: element "TITLE" undefined jade:notation.sgml:71:6:E: element "PARA" undefined jade:notation.sgml:72:8:E: element "QUOTE" undefined jade:notation.sgml:72:33:E: element "FILENAME" undefined jade:notation.sgml:74:26:E: element "PRODUCTNAME" undefined jade:notation.sgml:77:6:E: element "PARA" undefined jade:notation.sgml:79:9:E: element "QUOTE" undefined jade:notation.sgml:79:30:E: element "QUOTE" undefined jade:notation.sgml:81:9:E: element "QUOTE" undefined jade:notation.sgml:81:30:E: element "QUOTE" undefined jade:notation.sgml:81:78:E: element "QUOTE" undefined jade:notation.sgml:85:6:E: element "PARA" undefined jade:notation.sgml:86:34:E: element "QUOTE" undefined jade:notation.sgml:86:55:E: element "QUOTE" undefined jade:notation.sgml:87:22:E: element "QUOTE" undefined jade:notation.sgml:90:6:E: element "PARA" undefined jade:notation.sgml:92:71:E: element "QUOTE" undefined jade:notation.sgml:92:73:E: general entity "gt" not defined and no default entity jade:notation.sgml:93:41:E: element "PRODUCTNAME" undefined jade:notation.sgml:94:49:E: element "QUOTE" undefined jade:notation.sgml:96:8:E: element "QUOTE" undefined jade:notation.sgml:97:10:E: element "ACRONYM" undefined jade:notation.sgml:97:63:E: element "QUOTE" undefined jade:notation.sgml:101:6:E: element "NOTE" undefined jade:notation.sgml:102:7:E: element "PARA" undefined jade:notation.sgml:103:39:E: element "PRODUCTNAME" undefined jade:notation.sgml:106:42:E: element "ULINK" undefined jade:y2k.sgml:1:15:E: element "SECT1" undefined jade:y2k.sgml:2:7:E: element "TITLE" undefined jade:y2k.sgml:4:6:E: element "NOTE" undefined jade:y2k.sgml:5:8:E: element "TITLE" undefined jade:y2k.sgml:7:7:E: element "PARA" undefined jade:y2k.sgml:9:50:E: element "ULINK" undefined jade:y2k.sgml:14:6:E: element "PARA" undefined jade:y2k.sgml:15:18:E: element "PRODUCTNAME" undefined jade:y2k.sgml:16:18:E: element "PRODUCTNAME" undefined jade:y2k.sgml:21:14:E: element "ITEMIZEDLIST" undefined jade:y2k.sgml:22:11:E: element "LISTITEM" undefined jade:y2k.sgml:23:8:E: element "PARA" undefined jade:y2k.sgml:24:65:E: element "PRODUCTNAME" undefined jade:y2k.sgml:26:36:E: element "PRODUCTNAME" undefined jade:y2k.sgml:31:11:E: element "LISTITEM" undefined jade:y2k.sgml:32:8:E: element "PARA" undefined jade:y2k.sgml:36:19:E: element "PRODUCTNAME" undefined jade:y2k.sgml:42:11:E: element "LISTITEM" undefined jade:y2k.sgml:43:8:E: element "PARA" undefined jade:y2k.sgml:47:65:E: element "ULINK" undefined jade:y2k.sgml:50:15:E: element "QUOTE" undefined jade:y2k.sgml:50:57:E: element "QUOTE" undefined jade:y2k.sgml:51:18:E: element "QUOTE" undefined jade:y2k.sgml:51:60:E: element "QUOTE" undefined jade:y2k.sgml:55:11:E: element "LISTITEM" undefined jade:y2k.sgml:56:8:E: element "PARA" undefined jade:y2k.sgml:59:16:E: element "PRODUCTNAME" undefined jade:y2k.sgml:64:6:E: element "PARA" undefined jade:y2k.sgml:66:56:E: element "ULINK" undefined jade:y2k.sgml:68:53:E: element "ULINK" undefined jade:I: maximum number of errors (200) reached; change with -E option jade:E: cannot open "/home/users/t/thomas/db118.d/docbook/html/docbook.dsl" (No such file or directory) jade:E: specification document does not have the DSSSL architecture as a base architecture PostgreSQL Administrator's Guide Covering v6.5 for general release The PostgreSQL Development Team Thomas Lockhart Caltech/JPL (last updated 1999-06-01) PostgreSQL is Copyright 1996-9 by the Postgres Global Development Group. Summary Postgres, developed originally in the UC Berkeley Computer Science Department, pioneered many of the object-relationalconcepts now becoming available in some commercial databases. It provides SQL92/SQL3 language support, transaction integrity, and type extensibility. PostgreSQL is an open-source descendant of this original Berkeleycode. Introduction This document is the Administrator's Manual for the PostgreSQL database management system, originally developedat the University of California at Berkeley. PostgreSQL is based on Postgres release 4.2. The Postgres project, led by Professor Michael Stonebraker, wassponsored by the Defense Advanced Research Projects Agency (DARPA), the Army Research Office (ARO), the NationalScience Foundation (NSF), and ESL, Inc. Resources This manual set is organized into several parts: Tutorial An introduction for new users. Does not cover advanced features. User's Guide General information for users, including available commands and data types. Programmer's Guide Advanced information for application programmers. Topics include type and function extensibility,library interfaces, and application design issues. Administrator's Guide Installation and management information. List of supported machines. Developer's Guide Information for Postgres developers. This is intended for those who are contributing tothe Postgres project; application development information should appear in the Programmer's Guide. Currentlyincluded in the Programmer's Guide. Reference Manual Detailed reference information on command syntax. Currently included in the User's Guide. In addition to this manual set, there are other resources to help you with Postgres installation and use: man pages The man pages have general information on command syntax. FAQs The Frequently Asked Questions (FAQ) documents address both general issues and some platform-specificissues. READMEs README files are available for some contributed packages. Web Site The Postgres web site might have some information not appearing in the distribution. Thereis a mhonarc catalog of mailing list traffic which is a rich resource for many topics. Mailing Lists The pgsql-general (archive) mailing list is a good place to have user questions answered. Other mailing lists are available; consult the Info Central section of the PostgreSQL web site for details. Yourself! Postgres is an open source product. As such, it depends on the user community for ongoing support. As you begin to use Postgres, you will rely on others for help, either through the documentation or throughthe mailing lists. Consider contributing your knowledge back. If you learn something which is not in the documentation,write it up and contribute it. If you add features to the code, contribute it. Even those without a lot of experience can provide corrections and minor changes in the documentation, and thatis a good way to start. The pgsql-docs (archive) mailing list is the place to get going. Terminology In the following documentation, site may be interpreted as the host machine on which Postgres is installed. Since it ispossible to install more than one set of Postgres databases on a single host, this term more precisely denotes any particularset of installed Postgres binaries and databases. The Postgres superuser is the user named postgres who owns the Postgres binaries and database files. As the databasesuperuser, all protection mechanisms may be bypassed and any data accessed arbitrarily. In addition, the Postgressuperuser is allowed to execute some support programs which are generally not available to all users. Note that thePostgres superuser is not the same as the Unix superuser (which will be referred to as root). The superuser should havea non-zero user identifier (UID) for security reasons. The database administrator or DBA, is the person who is responsible for installing Postgres with mechanisms to enforcea security policy for a site. The DBA can add new users by the method described below and maintain a set of templatedatabases for use by createdb. The postmaster is the process that acts as a clearing-house for requests to the Postgres system. Frontend applicationsconnect to the postmaster, which keeps tracks of any system errors and communication between the backend processes. The postmaster can take several command-line arguments to tune its behavior. However, supplying arguments is necessaryonly if you intend to run multiple sites or a non-default site. The Postgres backend (the actual executable program postgres) may be executed directly from the user shell by the Postgressuper-user (with the database name as an argument). However, doing this bypasses the shared buffer pool and locktable associated with a postmaster/site, therefore this is not recommended in a multiuser site. Notation ... or /usr/local/pgsql/ at the front of a file name is used to represent the path to the Postgres superuser's home directory. In a command synopsis, brackets ([ and ]) indicate an optional phrase or keyword. Anything in braces ({ and }) and containingvertical bars (|) indicates that you must choose one. In examples, parentheses (( and )) are used to group boolean expressions. | is the boolean operator OR. Examples will show commands executed from various accounts and programs. Commands executed from the root account will bepreceeded with . Commands executed from the Postgres superuser account will be preceeded with %, while commands executedfrom an unprivileged user's account will be preceeded with $. SQL commands will be preceeded with = or will haveno leading prompt, depending on the context. At the time of writing (Postgres v6.5) the notation for flagging commands is not universally consistant throughoutthe documentation set. Please report problems to the Documentation Mailing List. Y2K Statement Author Written by Thomas Lockhart on 1998-10-22. The PostgreSQL Global Development Team provides the Postgres software code tree as a public service, without warranty andwithout liability for it's behavior or performance. However, at the time of writing: The author of this statement, a volunteer on the Postgres support team since November, 1996, is not aware of any problems in the Postgres code base related to time transitions around Jan 1, 2000 (Y2K). The author of this statement is not aware of any reports of Y2K problems uncovered in regression testing orin other field use of recent or current versions of Postgres. We might have expected to hear about problems if theyexisted, given the installed base and the active participation of users on the support mailing lists. To the best of the author's knowledge, the assumptions Postgres makes about dates specified with a two-digit year are documented in the current User's Guide in the chapter on data types. For two-digit years, the significanttransition year is 1970, not 2000; e.g. 70-01-01 is interpreted as 1970-01-01, whereas 69-01-01 is interpretedas 2069-01-01. Any Y2K problems in the underlying OS related to obtaining "the current time" may propagate into apparent Y2K problemsin Postgres. Refer to The Gnu Project and gmake[2]: *** [admin.html] Error 1 gmake[2]: Leaving directory `/home/bright/pgcvs/pgsql/doc/src/sgml' gmake[1]: *** [admin.tar] Error 2 gmake[1]: Leaving directory `/home/bright/pgcvs/pgsql/doc/src' gmake: *** [install] Error 2 oy!
On Fri, 14 Jan 2000, Alfred Perlstein wrote: > > If so then I'll be glad to update the docs myself, otherwise I'd > also be happy to provide coupious amounts of plaintext docs and > comments in my code like I have been so far. That's all you need to do...as long as we have documentation that can be included, it will be included ... if in sgml, all the better, but plaintext works also...
> > > I've actually been trying to work on the sgml and failing miserably,
> > > I have no clue how this stuff works (sgml compilation) are you asking
> > > for a couple of paragraphs that describe the proposed changes?
> > > If so I hope this suffices, if not some help on building the sgml
> > > would be much appreciated:
> 'course I run freebsd. :) I even have the docproj port installed,
> however it seems that there's some things missing here, (see the
> end of this message).
> Perhaps someone can offer a step-by-step to building _postgresql's_
> doc files, or maybe there's a machine out there where this will
> build properly and someone can give me an account on it?
You are probably very near having something working. The Postgres docs
have an appendix on "Documentation", which contains some information
on getting jade built and running. The makefile in doc/src/sgml has
comments which indicate what probably needs to be changed, and how to
do it (the parameters are set so the stuff builds on postgresql.org,
but I have a couple of lines in my Makefile.custom to get things to
work at home).
In particular, the .dsl files are somewhere in your jade installation
tree (it is in /usr/lib/sgml/stylesheets/nwalsh-modular/{print,html}
on my Linux box).
Ask more specific questions and we'll help you through it, but only
after you get some sleep :)
I'm out of town through the weekend, but will be on-list Monday night
afaik.
- Thomas
> ~/pgcvs/pgsql/doc/src % gmake
> gmake all
> gmake[1]: Entering directory `/home/bright/pgcvs/pgsql/doc/src'
> gmake -C sgml clean
> gmake[2]: Entering directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
> (rm -rf HTML.manifest *.html *.htm *.1 *.l man1 manl manpage*)
> gmake[2]: Leaving directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
> gmake -C sgml admin.html
> gmake[2]: Entering directory `/home/bright/pgcvs/pgsql/doc/src/sgml'
> (rm -rf *.htm)
> jade -D ref -D ../graphics -V %use-id-as-filename% -d /home/users/t/thomas/db118.d/docbook/html/docbook.dsl -t sgml
admin.sgml
>
> ^^^^^^^^^^^^---- huh?
> ~/pgcvs/pgsql/doc % find . -name "*.dsl"
> ~/pgcvs/pgsql/doc %
>
> continues...
--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California
FreeBSD postgresql doc-HOWTO was: Re: [HACKERS] Revised nonblocking patches + quasi docs
От
Alfred Perlstein
Дата:
* Thomas Lockhart <lockhart@alumni.caltech.edu> [000114 19:42] wrote: > > > > I've actually been trying to work on the sgml and failing miserably, > > > > I have no clue how this stuff works (sgml compilation) are you asking > > > > for a couple of paragraphs that describe the proposed changes? > > > > If so I hope this suffices, if not some help on building the sgml > > > > would be much appreciated: > > 'course I run freebsd. :) I even have the docproj port installed, > > however it seems that there's some things missing here, (see the > > end of this message). > > Perhaps someone can offer a step-by-step to building _postgresql's_ > > doc files, or maybe there's a machine out there where this will > > build properly and someone can give me an account on it? > > You are probably very near having something working. Yes, I can feel it, I just ran to the store to pick up the goat blood and candles. :) With the help of a friend Jeroen Ruigrok van der Werven, one of the FreeBSD'doc folks I got it working here's what I needed to do: install the textproc/docproj and textproc/docbook from /usr/ports, maybe more packages are needed (dsssl-docbook-modular, dtd-catalogs), i was installing everything hoping to get it to work... setup my enviornment... (this ought to be mentioned in the docs) export SMGL_ROOT=/usr/local/share/sgml SGML_CATALOG_FILES=/usr/local/share/sgml/jade/catalog SGML_CATALOG_FILES=/usr/local/share/sgml/html/catalog:$SGML_CATALOG_FILES SGML_CATALOG_FILES=/usr/local/share/sgml/iso8879/catalog:$SGML_CATALOG_FILES SGML_CATALOG_FILES=/usr/local/share/sgml/transpec/catalog:$SGML_CATALOG_FILES SGML_CATALOG_FILES=/usr/local/share/sgml/docbook/catalog:$SGML_CATALOG_FILES export SGML_CATALOG_FILES then in the pgsql/doc/src dir: gmake all \HSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/html/ \PSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/print/\ wait a good long time... viola. Ok, I should have the docs for my code along with some help for hapless FreeBSD users trying to work on this stuff in a bit. I guess this means that Marc can't weasel his way out of doing documentation anymore, or was that the point all along? :) Btw, does anyone have some fixes so gvim doesn't barf doing syntax highlighting on these sgml files? -Alfred
* Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote:
> Don Baccus <dhogaza@pacifier.com> writes:
> > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> >> I also object strongly to the lack of documentation.
>
> > ... I know there are some folks who aren't native-english speakers, so
> > perhaps you don't want to require that the implementor of such patches
> > provide the final documentation wording. But the information should
> > be there and spelled out in a form that can be very easily moved to
> > the docs.
>
> Oh, absolutely. Thomas, our master of the docs, has always had the
> policy of "give me some words, I'll take care of formatting and
> editing..."
>
> I was probably too harsh on Alfred last night, since in fact his code
> was fairly well commented, and some minimal doco could have been
> extracted from the routine headers. But on a change like this, I think
> some paragraphs of coherent high-level explanation are needed: what it
> does, when and why you'd use it. I didn't see that anywhere...
Here's the revised patch, it includes sgml docs and changes to
ensure that old style connections behave the way they are expected
to:
Index: doc/src/sgml/libpq.sgml
===================================================================
RCS file: /home/pgcvs/pgsql/doc/src/sgml/libpq.sgml,v
retrieving revision 1.25
diff -u -c -r1.25 libpq.sgml
*** doc/src/sgml/libpq.sgml 2000/01/14 05:33:13 1.25
--- doc/src/sgml/libpq.sgml 2000/01/17 03:40:30
***************
*** 377,382 ****
--- 377,386 ---- changed in the future. </para> <para>
+ These functions leave the socket in a non-blocking state as if
+ <function>PQsetnonblocking</function> had been called.
+ </para>
+ <para> These functions are not thread-safe. </para> </listitem>
***************
*** 1168,1175 ****
--- 1172,1229 ---- Applications that do not like these limitations can instead use the underlying functions that
<function>PQexec</function>is built from: <function>PQsendQuery</function> and <function>PQgetResult</function>.
+ </para>
+ <para>
+ Older programs that used this functionality as well as
+ <function>PQputline</function> and <function>PQputnbytes</function>
+ could block waiting to send data to the backend, to
+ address that issue, the function <function>PQsetnonblocking</function>
+ was added.
+ </para>
+ <para>
+ Old applications can neglect to use <function>PQsetnonblocking</function>
+ and get the older potentially blocking behavior. Newer programs can use
+ <function>PQsetnonblocking</function> to achieve a completely non-blocking
+ connection to the backend. <itemizedlist>
+ <listitem>
+ <para>
+ <function>PQsetnonblocking</function> Sets the state of the connection
+ to non-blocking.
+ <synopsis>
+ int PQsetnonblocking(PGconn *conn)
+ </synopsis>
+ this function will ensure that calls to
+ <function>PQputline</function>, <function>PQputnbytes</function>,
+ <function>PQsendQuery</function> and <function>PQendcopy</function>
+ will not block but instead return an error if they need to be called
+ again.
+ </para>
+ <para>
+ When a database connection has been set to non-blocking mode and
+ <function>PQexec</function> is called, it will temporarily set the state
+ of the connection to blocking until the <function>PQexec</function>
+ completes.
+ </para>
+ <para>
+ More of libpq is expected to be made safe for
+ <function>PQsetnonblocking</function> functionality in the near future.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ <function>PQisnonblocking</function>
+ Returns the blocking status of the database connection.
+ <synopsis>
+ int PQisnonblocking(const PGconn *conn)
+ </synopsis>
+ Returns TRUE if the connection is set to non-blocking mode,
+ FALSE if blocking.
+ </para>
+ </listitem>
+ <listitem> <para> <function>PQsendQuery</function>
***************
*** 1267,1286 **** <listitem> <para> <function>PQsocket</function> Obtain the file descriptor number for the
backendconnection socket.
! A valid descriptor will be >= 0; a result of -1 indicates that no backend connection is currently open.
<synopsis>int PQsocket(const PGconn *conn); </synopsis> <function>PQsocket</function> should be used to obtain the
backendsocket descriptor in preparation for executing <function>select</function>(2). This allows an
! application to wait for either backend responses or other conditions. If the result of <function>select</function>(2)
indicatesthat data can be read from the backend socket, then <function>PQconsumeInput</function> should be called to
readthe data; after which, <function>PQisBusy</function>, <function>PQgetResult</function>, and/or
<function>PQnotifies</function>can be used to process the response. </para> </listitem>
--- 1321,1363 ---- <listitem> <para>
+ <function>PQflush</function> Attempt to flush any data queued to the backend,
+ returns 0 if successful (or if the send queue is empty) or EOF if it failed for
+ some reason.
+ <synopsis>
+ int PQflush(PGconn *conn);
+ </synopsis>
+ <function>PQflush</function> needs to be called on a non-blocking connection
+ before calling <function>select</function> to determine if a responce has
+ arrived. If 0 is returned it ensures that there is no data queued to the
+ backend that has not actually been sent. Only applications that have used
+ <function>PQsetnonblocking</function> have a need for this.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para> <function>PQsocket</function> Obtain the file descriptor number for the backend connection socket.
! A valid descriptor will be >= 0; a result of -1 indicates that no backend connection is currently
open.<synopsis> int PQsocket(const PGconn *conn); </synopsis> <function>PQsocket</function> should be used to obtain
thebackend socket descriptor in preparation for executing <function>select</function>(2). This allows an
! application using a blocking connection to wait for either backend responses or
! other conditions. If the result of <function>select</function>(2) indicates that data can be read from the backend
socket,then <function>PQconsumeInput</function> should be called to read the data; after which,
<function>PQisBusy</function>,<function>PQgetResult</function>, and/or <function>PQnotifies</function> can be used to
processthe response.
+ </para>
+ <para>
+ Non-blocking connections (that have used <function>PQsetnonblocking</function>)
+ should not use <function>select</function> until <function>PQflush</function>
+ has returned 0 indicating that there is no buffered data waiting to be sent
+ to the backend. </para> </listitem>
Index: src/interfaces/libpq/fe-connect.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.111
diff -u -c -r1.111 fe-connect.c
*** src/interfaces/libpq/fe-connect.c 2000/01/16 21:18:52 1.111
--- src/interfaces/libpq/fe-connect.c 2000/01/17 02:35:56
***************
*** 594,624 **** return 0; }
-
- /* ----------
- * connectMakeNonblocking -
- * Make a connection non-blocking.
- * Returns 1 if successful, 0 if not.
- * ----------
- */
- static int
- connectMakeNonblocking(PGconn *conn)
- {
- #ifndef WIN32
- if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
- #else
- if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
- #endif
- {
- printfPQExpBuffer(&conn->errorMessage,
- "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
- errno, strerror(errno));
- return 0;
- }
-
- return 1;
- }
- /* ---------- * connectNoDelay - * Sets the TCP_NODELAY socket option.
--- 594,599 ----
***************
*** 789,795 **** * Ewan Mellor <eem21@cam.ac.uk>. * ---------- */ #if (!defined(WIN32) ||
defined(WIN32_NON_BLOCKING_CONNECTIONS))&& !defined(USE_SSL)
! if (!connectMakeNonblocking(conn)) goto connect_errReturn; #endif
--- 764,770 ---- * Ewan Mellor <eem21@cam.ac.uk>. * ---------- */ #if (!defined(WIN32) ||
defined(WIN32_NON_BLOCKING_CONNECTIONS))&& !defined(USE_SSL)
! if (PQsetnonblocking(conn, TRUE) != 0) goto connect_errReturn; #endif
***************
*** 898,904 **** /* This makes the connection non-blocking, for all those cases which forced us not to do it
above.*/ #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! if (!connectMakeNonblocking(conn)) goto connect_errReturn; #endif
--- 873,879 ---- /* This makes the connection non-blocking, for all those cases which forced us not to do it
above.*/ #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
! if (PQsetnonblocking(conn, TRUE) != 0) goto connect_errReturn; #endif
***************
*** 1720,1725 ****
--- 1695,1701 ---- conn->inBuffer = (char *) malloc(conn->inBufSize); conn->outBufSize = 8 * 1024;
conn->outBuffer= (char *) malloc(conn->outBufSize);
+ conn->nonblocking = FALSE; initPQExpBuffer(&conn->errorMessage); initPQExpBuffer(&conn->workBuffer);
if(conn->inBuffer == NULL ||
***************
*** 1830,1835 ****
--- 1806,1812 ---- conn->lobjfuncs = NULL; conn->inStart = conn->inCursor = conn->inEnd = 0; conn->outCount
=0;
+ conn->nonblocking = FALSE; }
Index: src/interfaces/libpq/fe-exec.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
retrieving revision 1.86
diff -u -c -r1.86 fe-exec.c
*** src/interfaces/libpq/fe-exec.c 1999/11/11 00:10:14 1.86
--- src/interfaces/libpq/fe-exec.c 2000/01/14 22:47:07
***************
*** 13,18 ****
--- 13,19 ---- */ #include <errno.h> #include <ctype.h>
+ #include <fcntl.h> #include "postgres.h" #include "libpq-fe.h"
***************
*** 24,30 **** #include <unistd.h> #endif
- /* keep this in same order as ExecStatusType in libpq-fe.h */ const char *const pgresStatus[] = {
"PGRES_EMPTY_QUERY",
--- 25,30 ----
***************
*** 514,526 **** conn->curTuple = NULL; /* send the query to the backend; */
! /* the frontend-backend protocol uses 'Q' to designate queries */
! if (pqPutnchar("Q", 1, conn) ||
! pqPuts(query, conn) ||
! pqFlush(conn)) {
! handleSendFailure(conn);
! return 0; } /* OK, it's launched! */
--- 514,566 ---- conn->curTuple = NULL; /* send the query to the backend; */
!
! /*
! * in order to guarantee that we don't send a partial query
! * where we would become out of sync with the backend and/or
! * block during a non-blocking connection we must first flush
! * the send buffer before sending more data
! *
! * an alternative is to implement 'queue reservations' where
! * we are able to roll up a transaction
! * (the 'Q' along with our query) and make sure we have
! * enough space for it all in the send buffer.
! */
! if (pqIsnonblocking(conn)) {
! /*
! * the buffer must have emptied completely before we allow
! * a new query to be buffered
! */
! if (pqFlush(conn))
! return 0;
! /* 'Q' == queries */
! /* XXX: if we fail here we really ought to not block */
! if (pqPutnchar("Q", 1, conn) ||
! pqPuts(query, conn))
! {
! handleSendFailure(conn);
! return 0;
! }
! /*
! * give the data a push, ignore the return value as
! * ConsumeInput() will do any aditional flushing if needed
! */
! (void) pqFlush(conn);
! }
! else
! {
! /*
! * the frontend-backend protocol uses 'Q' to
! * designate queries
! */
! if (pqPutnchar("Q", 1, conn) ||
! pqPuts(query, conn) ||
! pqFlush(conn))
! {
! handleSendFailure(conn);
! return 0;
! } } /* OK, it's launched! */
***************
*** 574,580 ****
--- 614,630 ---- * we will NOT block waiting for more input. */ if (pqReadData(conn) < 0)
+ {
+ /*
+ * for non-blocking connections
+ * try to flush the send-queue otherwise we may never get a
+ * responce for something that may not have already been sent
+ * because it's in our write buffer!
+ */
+ if (pqIsnonblocking(conn))
+ (void) pqFlush(conn); return 0;
+ } /* Parsing of the data waits till later. */ return 1; }
***************
*** 1088,1093 ****
--- 1138,1153 ---- { PGresult *result; PGresult *lastResult;
+ bool savedblocking;
+
+ /*
+ * we assume anyone calling PQexec wants blocking behaviour,
+ * we force the blocking status of the connection to blocking
+ * for the duration of this function and restore it on return
+ */
+ savedblocking = pqIsnonblocking(conn);
+ if (PQsetnonblocking(conn, FALSE) == -1)
+ return NULL; /* * Silently discard any prior query result that application didn't
***************
*** 1102,1115 **** PQclear(result); printfPQExpBuffer(&conn->errorMessage,
"PQexec:you gotta get out of a COPY state yourself.\n");
! return NULL; } PQclear(result); } /* OK to send the message */ if
(!PQsendQuery(conn,query))
! return NULL; /* * For backwards compatibility, return the last result if there are
--- 1162,1176 ---- PQclear(result); printfPQExpBuffer(&conn->errorMessage,
"PQexec:you gotta get out of a COPY state yourself.\n");
! /* restore blocking status */
! goto errout; } PQclear(result); } /* OK to send the message */ if
(!PQsendQuery(conn,query))
! goto errout; /* restore blocking status */ /* * For backwards compatibility, return the last
resultif there are
***************
*** 1142,1148 ****
--- 1203,1217 ---- result->resultStatus == PGRES_COPY_OUT) break; }
+
+ if (PQsetnonblocking(conn, savedblocking) == -1)
+ return NULL; return lastResult;
+
+ errout:
+ if (PQsetnonblocking(conn, savedblocking) == -1)
+ return NULL;
+ return NULL; }
***************
*** 1431,1438 **** "PQendcopy() -- I don't think there's a copy in progress.\n"); return 1; }
! (void) pqFlush(conn); /* make sure no data is waiting to be sent */ /* Return to active duty */
conn->asyncStatus= PGASYNC_BUSY;
--- 1500,1516 ---- "PQendcopy() -- I don't think there's a copy in progress.\n"); return 1; }
+
+ /*
+ * make sure no data is waiting to be sent,
+ * abort if we are non-blocking and the flush fails
+ */
+ if (pqFlush(conn) && pqIsnonblocking(conn))
+ return (1);
! /* non blocking connections may have to abort at this point. */
! if (pqIsnonblocking(conn) && PQisBusy(conn))
! return (1); /* Return to active duty */ conn->asyncStatus = PGASYNC_BUSY;
***************
*** 2025,2028 ****
--- 2103,2192 ---- return 1; else return 0;
+ }
+
+ /* PQsetnonblocking:
+ sets the PGconn's database connection non-blocking if the arg is TRUE
+ or makes it non-blocking if the arg is FALSE, this will not protect
+ you from PQexec(), you'll only be safe when using the non-blocking
+ API
+ Needs to be called only on a connected database connection.
+ */
+
+ int
+ PQsetnonblocking(PGconn *conn, int arg)
+ {
+ int fcntlarg;
+
+ arg = (arg == TRUE) ? 1 : 0;
+ /* early out if the socket is already in the state requested */
+ if (arg == conn->nonblocking)
+ return (0);
+
+ /*
+ * to guarantee constancy for flushing/query/result-polling behavior
+ * we need to flush the send queue at this point in order to guarantee
+ * proper behavior.
+ * this is ok because either they are making a transition
+ * _from_ or _to_ blocking mode, either way we can block them.
+ */
+ /* if we are going from blocking to non-blocking flush here */
+ if (!pqIsnonblocking(conn) && pqFlush(conn))
+ return (-1);
+
+
+ #ifdef USE_SSL
+ if (conn->ssl)
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "PQsetnonblocking() -- not supported when using SSL\n");
+ return (-1);
+ }
+ #endif /* USE_SSL */
+
+ #ifndef WIN32
+ fcntlarg = fcntl(conn->sock, F_GETFL, 0);
+ if (fcntlarg == -1)
+ return (-1);
+
+ if ((arg == TRUE &&
+ fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
+ (arg == FALSE &&
+ fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1))
+ #else
+ fcntlarg = arg;
+ if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
+ #endif
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "PQsetblocking() -- unable to set nonblocking status to %s\n",
+ arg == TRUE ? "TRUE" : "FALSE");
+ return (-1);
+ }
+
+ conn->nonblocking = arg;
+
+ /* if we are going from non-blocking to blocking flush here */
+ if (pqIsnonblocking(conn) && pqFlush(conn))
+ return (-1);
+
+ return (0);
+ }
+
+ /* return the blocking status of the database connection, TRUE == nonblocking,
+ FALSE == blocking
+ */
+ int
+ PQisnonblocking(const PGconn *conn)
+ {
+
+ return (pqIsnonblocking(conn));
+ }
+
+ /* try to force data out, really only useful for non-blocking users */
+ int
+ PQflush(PGconn *conn)
+ {
+
+ return (pqFlush(conn)); }
Index: src/interfaces/libpq/fe-misc.c
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
retrieving revision 1.33
diff -u -c -r1.33 fe-misc.c
*** src/interfaces/libpq/fe-misc.c 1999/11/30 03:08:19 1.33
--- src/interfaces/libpq/fe-misc.c 2000/01/12 03:12:14
***************
*** 86,91 ****
--- 86,122 ---- { size_t avail = Max(conn->outBufSize - conn->outCount, 0);
+ /*
+ * if we are non-blocking and the send queue is too full to buffer this
+ * request then try to flush some and return an error
+ */
+ if (pqIsnonblocking(conn) && nbytes > avail && pqFlush(conn))
+ {
+ /*
+ * even if the flush failed we may still have written some
+ * data, recalculate the size of the send-queue relative
+ * to the amount we have to send, we may be able to queue it
+ * afterall even though it's not sent to the database it's
+ * ok, any routines that check the data coming from the
+ * database better call pqFlush() anyway.
+ */
+ if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
+ {
+ printfPQExpBuffer(&conn->errorMessage,
+ "pqPutBytes -- pqFlush couldn't flush enough"
+ " data: space available: %d, space needed %d\n",
+ Max(conn->outBufSize - conn->outCount, 0), nbytes);
+ return EOF;
+ }
+ }
+
+ /*
+ * is the amount of data to be sent is larger than the size of the
+ * output buffer then we must flush it to make more room.
+ *
+ * the code above will make sure the loop conditional is never
+ * true for non-blocking connections
+ */ while (nbytes > avail) { memcpy(conn->outBuffer + conn->outCount, s, avail);
***************
*** 548,553 ****
--- 579,592 ---- return EOF; }
+ /*
+ * don't try to send zero data, allows us to use this function
+ * without too much worry about overhead
+ */
+ if (len == 0)
+ return (0);
+
+ /* while there's still data to send */ while (len > 0) { /* Prevent being SIGPIPEd if backend has
closedthe connection. */
***************
*** 556,561 ****
--- 595,601 ---- #endif int sent;
+ #ifdef USE_SSL if (conn->ssl) sent = SSL_write(conn->ssl, ptr, len);
***************
*** 585,590 ****
--- 625,632 ---- case EWOULDBLOCK: break; #endif
+ case EINTR:
+ continue; case EPIPE: #ifdef ECONNRESET
***************
*** 616,628 **** ptr += sent; len -= sent; } if (len > 0) {
/*We didn't send it all, wait till we can send more */
- /* At first glance this looks as though it should block. I think
- * that it will be OK though, as long as the socket is
- * non-blocking. */ if (pqWait(FALSE, TRUE, conn)) return EOF; }
--- 658,688 ---- ptr += sent; len -= sent; }
+ if (len > 0) { /* We didn't send it all, wait till we can send more */
+
+ /*
+ * if the socket is in non-blocking mode we may need
+ * to abort here
+ */
+ #ifdef USE_SSL
+ /* can't do anything for our SSL users yet */
+ if (conn->ssl == NULL)
+ {
+ #endif
+ if (pqIsnonblocking(conn))
+ {
+ /* shift the contents of the buffer */
+ memmove(conn->outBuffer, ptr, len);
+ conn->outCount = len;
+ return EOF;
+ }
+ #ifdef USE_SSL
+ }
+ #endif if (pqWait(FALSE, TRUE, conn)) return EOF; }
Index: src/interfaces/libpq/libpq-fe.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.55
diff -u -c -r1.55 libpq-fe.h
*** src/interfaces/libpq/libpq-fe.h 2000/01/15 05:37:21 1.55
--- src/interfaces/libpq/libpq-fe.h 2000/01/17 02:35:56
***************
*** 263,268 ****
--- 263,275 ---- extern int PQputnbytes(PGconn *conn, const char *buffer, int nbytes); extern int
PQendcopy(PGconn*conn);
+ /* Set blocking/nonblocking connection to the backend */
+ extern int PQsetnonblocking(PGconn *conn, int arg);
+ extern int PQisnonblocking(const PGconn *conn);
+
+ /* Force the write buffer to be written (or at least try) */
+ extern int PQflush(PGconn *conn);
+ /* * "Fast path" interface --- not really recommended for application * use
Index: src/interfaces/libpq/libpq-int.h
===================================================================
RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
retrieving revision 1.16
diff -u -c -r1.16 libpq-int.h
*** src/interfaces/libpq/libpq-int.h 2000/01/15 05:37:21 1.16
--- src/interfaces/libpq/libpq-int.h 2000/01/17 02:35:56
***************
*** 214,219 ****
--- 214,222 ---- int inEnd; /* offset to first position after avail
* data */
+ int nonblocking; /* whether this connection is using a blocking
+ * socket to the backend or not */
+ /* Buffer for data not yet sent to backend */ char *outBuffer; /* currently allocated buffer */
int outBufSize; /* allocated size of buffer */
***************
*** 299,303 ****
--- 302,312 ---- #define strerror(A) (sys_errlist[(A)]) #endif /* sunos4 */ #endif /* !strerror */
+
+ /*
+ * this is so that we can check is a connection is non-blocking internally
+ * without the overhead of a function call
+ */
+ #define pqIsnonblocking(conn) (conn->nonblocking) #endif /* LIBPQ_INT_H */
Re: FreeBSD postgresql doc-HOWTO was: Re: [HACKERS] Revised nonblocking patches + quasi docs
От
Thomas Lockhart
Дата:
> setup my enviornment... (this ought to be mentioned in the docs)
But afaik this isn't required for me to run on postgresql.org, a
FreeBSD machine set up by Marc/scrappy.
> then in the pgsql/doc/src dir:
> gmake all \
> HSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/html/ \
> PSTYLE=/usr/local/share/sgml/docbook/dsssl/modular/print/ \
That works too. I usually just set up a src/Makefile.custom with the
two lines defining HSTYLE and PSTYLE.
> Btw, does anyone have some fixes so gvim doesn't barf doing syntax
> highlighting on these sgml files?
Let us know when you find them; I can help with emacs...
- Thomas
--
Thomas Lockhart lockhart@alumni.caltech.edu
South Pasadena, California
Applied.
> * Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote:
> > Don Baccus <dhogaza@pacifier.com> writes:
> > > At 05:27 PM 1/8/00 -0500, Tom Lane wrote:
> > >> I also object strongly to the lack of documentation.
> >
> > > ... I know there are some folks who aren't native-english speakers, so
> > > perhaps you don't want to require that the implementor of such patches
> > > provide the final documentation wording. But the information should
> > > be there and spelled out in a form that can be very easily moved to
> > > the docs.
> >
> > Oh, absolutely. Thomas, our master of the docs, has always had the
> > policy of "give me some words, I'll take care of formatting and
> > editing..."
> >
> > I was probably too harsh on Alfred last night, since in fact his code
> > was fairly well commented, and some minimal doco could have been
> > extracted from the routine headers. But on a change like this, I think
> > some paragraphs of coherent high-level explanation are needed: what it
> > does, when and why you'd use it. I didn't see that anywhere...
>
> Here's the revised patch, it includes sgml docs and changes to
> ensure that old style connections behave the way they are expected
> to:
>
> Index: doc/src/sgml/libpq.sgml
> ===================================================================
> RCS file: /home/pgcvs/pgsql/doc/src/sgml/libpq.sgml,v
> retrieving revision 1.25
> diff -u -c -r1.25 libpq.sgml
> *** doc/src/sgml/libpq.sgml 2000/01/14 05:33:13 1.25
> --- doc/src/sgml/libpq.sgml 2000/01/17 03:40:30
> ***************
> *** 377,382 ****
> --- 377,386 ----
> changed in the future.
> </para>
> <para>
> + These functions leave the socket in a non-blocking state as if
> + <function>PQsetnonblocking</function> had been called.
> + </para>
> + <para>
> These functions are not thread-safe.
> </para>
> </listitem>
> ***************
> *** 1168,1175 ****
> --- 1172,1229 ----
> Applications that do not like these limitations can instead use the
> underlying functions that <function>PQexec</function> is built from:
> <function>PQsendQuery</function> and <function>PQgetResult</function>.
> + </para>
> + <para>
> + Older programs that used this functionality as well as
> + <function>PQputline</function> and <function>PQputnbytes</function>
> + could block waiting to send data to the backend, to
> + address that issue, the function <function>PQsetnonblocking</function>
> + was added.
> + </para>
> + <para>
> + Old applications can neglect to use <function>PQsetnonblocking</function>
> + and get the older potentially blocking behavior. Newer programs can use
> + <function>PQsetnonblocking</function> to achieve a completely non-blocking
> + connection to the backend.
>
> <itemizedlist>
> + <listitem>
> + <para>
> + <function>PQsetnonblocking</function> Sets the state of the connection
> + to non-blocking.
> + <synopsis>
> + int PQsetnonblocking(PGconn *conn)
> + </synopsis>
> + this function will ensure that calls to
> + <function>PQputline</function>, <function>PQputnbytes</function>,
> + <function>PQsendQuery</function> and <function>PQendcopy</function>
> + will not block but instead return an error if they need to be called
> + again.
> + </para>
> + <para>
> + When a database connection has been set to non-blocking mode and
> + <function>PQexec</function> is called, it will temporarily set the state
> + of the connection to blocking until the <function>PQexec</function>
> + completes.
> + </para>
> + <para>
> + More of libpq is expected to be made safe for
> + <function>PQsetnonblocking</function> functionality in the near future.
> + </para>
> + </listitem>
> +
> + <listitem>
> + <para>
> + <function>PQisnonblocking</function>
> + Returns the blocking status of the database connection.
> + <synopsis>
> + int PQisnonblocking(const PGconn *conn)
> + </synopsis>
> + Returns TRUE if the connection is set to non-blocking mode,
> + FALSE if blocking.
> + </para>
> + </listitem>
> +
> <listitem>
> <para>
> <function>PQsendQuery</function>
> ***************
> *** 1267,1286 ****
>
> <listitem>
> <para>
> <function>PQsocket</function>
> Obtain the file descriptor number for the backend connection socket.
> ! A valid descriptor will be >= 0; a result of -1 indicates that
> no backend connection is currently open.
> <synopsis>
> int PQsocket(const PGconn *conn);
> </synopsis>
> <function>PQsocket</function> should be used to obtain the backend socket descriptor
> in preparation for executing <function>select</function>(2). This allows an
> ! application to wait for either backend responses or other conditions.
> If the result of <function>select</function>(2) indicates that data can be read from
> the backend socket, then <function>PQconsumeInput</function> should be called to read the
> data; after which, <function>PQisBusy</function>, <function>PQgetResult</function>,
> and/or <function>PQnotifies</function> can be used to process the response.
> </para>
> </listitem>
>
> --- 1321,1363 ----
>
> <listitem>
> <para>
> + <function>PQflush</function> Attempt to flush any data queued to the backend,
> + returns 0 if successful (or if the send queue is empty) or EOF if it failed for
> + some reason.
> + <synopsis>
> + int PQflush(PGconn *conn);
> + </synopsis>
> + <function>PQflush</function> needs to be called on a non-blocking connection
> + before calling <function>select</function> to determine if a responce has
> + arrived. If 0 is returned it ensures that there is no data queued to the
> + backend that has not actually been sent. Only applications that have used
> + <function>PQsetnonblocking</function> have a need for this.
> + </para>
> + </listitem>
> +
> + <listitem>
> + <para>
> <function>PQsocket</function>
> Obtain the file descriptor number for the backend connection socket.
> ! A valid descriptor will be >= 0; a result of -1 indicates that
> no backend connection is currently open.
> <synopsis>
> int PQsocket(const PGconn *conn);
> </synopsis>
> <function>PQsocket</function> should be used to obtain the backend socket descriptor
> in preparation for executing <function>select</function>(2). This allows an
> ! application using a blocking connection to wait for either backend responses or
> ! other conditions.
> If the result of <function>select</function>(2) indicates that data can be read from
> the backend socket, then <function>PQconsumeInput</function> should be called to read the
> data; after which, <function>PQisBusy</function>, <function>PQgetResult</function>,
> and/or <function>PQnotifies</function> can be used to process the response.
> + </para>
> + <para>
> + Non-blocking connections (that have used <function>PQsetnonblocking</function>)
> + should not use <function>select</function> until <function>PQflush</function>
> + has returned 0 indicating that there is no buffered data waiting to be sent
> + to the backend.
> </para>
> </listitem>
>
> Index: src/interfaces/libpq/fe-connect.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-connect.c,v
> retrieving revision 1.111
> diff -u -c -r1.111 fe-connect.c
> *** src/interfaces/libpq/fe-connect.c 2000/01/16 21:18:52 1.111
> --- src/interfaces/libpq/fe-connect.c 2000/01/17 02:35:56
> ***************
> *** 594,624 ****
> return 0;
> }
>
> -
> - /* ----------
> - * connectMakeNonblocking -
> - * Make a connection non-blocking.
> - * Returns 1 if successful, 0 if not.
> - * ----------
> - */
> - static int
> - connectMakeNonblocking(PGconn *conn)
> - {
> - #ifndef WIN32
> - if (fcntl(conn->sock, F_SETFL, O_NONBLOCK) < 0)
> - #else
> - if (ioctlsocket(conn->sock, FIONBIO, &on) != 0)
> - #endif
> - {
> - printfPQExpBuffer(&conn->errorMessage,
> - "connectMakeNonblocking -- fcntl() failed: errno=%d\n%s\n",
> - errno, strerror(errno));
> - return 0;
> - }
> -
> - return 1;
> - }
> -
> /* ----------
> * connectNoDelay -
> * Sets the TCP_NODELAY socket option.
> --- 594,599 ----
> ***************
> *** 789,795 ****
> * Ewan Mellor <eem21@cam.ac.uk>.
> * ---------- */
> #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! if (!connectMakeNonblocking(conn))
> goto connect_errReturn;
> #endif
>
> --- 764,770 ----
> * Ewan Mellor <eem21@cam.ac.uk>.
> * ---------- */
> #if (!defined(WIN32) || defined(WIN32_NON_BLOCKING_CONNECTIONS)) && !defined(USE_SSL)
> ! if (PQsetnonblocking(conn, TRUE) != 0)
> goto connect_errReturn;
> #endif
>
> ***************
> *** 898,904 ****
> /* This makes the connection non-blocking, for all those cases which forced us
> not to do it above. */
> #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! if (!connectMakeNonblocking(conn))
> goto connect_errReturn;
> #endif
>
> --- 873,879 ----
> /* This makes the connection non-blocking, for all those cases which forced us
> not to do it above. */
> #if (defined(WIN32) && !defined(WIN32_NON_BLOCKING_CONNECTIONS)) || defined(USE_SSL)
> ! if (PQsetnonblocking(conn, TRUE) != 0)
> goto connect_errReturn;
> #endif
>
> ***************
> *** 1720,1725 ****
> --- 1695,1701 ----
> conn->inBuffer = (char *) malloc(conn->inBufSize);
> conn->outBufSize = 8 * 1024;
> conn->outBuffer = (char *) malloc(conn->outBufSize);
> + conn->nonblocking = FALSE;
> initPQExpBuffer(&conn->errorMessage);
> initPQExpBuffer(&conn->workBuffer);
> if (conn->inBuffer == NULL ||
> ***************
> *** 1830,1835 ****
> --- 1806,1812 ----
> conn->lobjfuncs = NULL;
> conn->inStart = conn->inCursor = conn->inEnd = 0;
> conn->outCount = 0;
> + conn->nonblocking = FALSE;
>
> }
>
> Index: src/interfaces/libpq/fe-exec.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-exec.c,v
> retrieving revision 1.86
> diff -u -c -r1.86 fe-exec.c
> *** src/interfaces/libpq/fe-exec.c 1999/11/11 00:10:14 1.86
> --- src/interfaces/libpq/fe-exec.c 2000/01/14 22:47:07
> ***************
> *** 13,18 ****
> --- 13,19 ----
> */
> #include <errno.h>
> #include <ctype.h>
> + #include <fcntl.h>
>
> #include "postgres.h"
> #include "libpq-fe.h"
> ***************
> *** 24,30 ****
> #include <unistd.h>
> #endif
>
> -
> /* keep this in same order as ExecStatusType in libpq-fe.h */
> const char *const pgresStatus[] = {
> "PGRES_EMPTY_QUERY",
> --- 25,30 ----
> ***************
> *** 514,526 ****
> conn->curTuple = NULL;
>
> /* send the query to the backend; */
> ! /* the frontend-backend protocol uses 'Q' to designate queries */
> ! if (pqPutnchar("Q", 1, conn) ||
> ! pqPuts(query, conn) ||
> ! pqFlush(conn))
> {
> ! handleSendFailure(conn);
> ! return 0;
> }
>
> /* OK, it's launched! */
> --- 514,566 ----
> conn->curTuple = NULL;
>
> /* send the query to the backend; */
> !
> ! /*
> ! * in order to guarantee that we don't send a partial query
> ! * where we would become out of sync with the backend and/or
> ! * block during a non-blocking connection we must first flush
> ! * the send buffer before sending more data
> ! *
> ! * an alternative is to implement 'queue reservations' where
> ! * we are able to roll up a transaction
> ! * (the 'Q' along with our query) and make sure we have
> ! * enough space for it all in the send buffer.
> ! */
> ! if (pqIsnonblocking(conn))
> {
> ! /*
> ! * the buffer must have emptied completely before we allow
> ! * a new query to be buffered
> ! */
> ! if (pqFlush(conn))
> ! return 0;
> ! /* 'Q' == queries */
> ! /* XXX: if we fail here we really ought to not block */
> ! if (pqPutnchar("Q", 1, conn) ||
> ! pqPuts(query, conn))
> ! {
> ! handleSendFailure(conn);
> ! return 0;
> ! }
> ! /*
> ! * give the data a push, ignore the return value as
> ! * ConsumeInput() will do any aditional flushing if needed
> ! */
> ! (void) pqFlush(conn);
> ! }
> ! else
> ! {
> ! /*
> ! * the frontend-backend protocol uses 'Q' to
> ! * designate queries
> ! */
> ! if (pqPutnchar("Q", 1, conn) ||
> ! pqPuts(query, conn) ||
> ! pqFlush(conn))
> ! {
> ! handleSendFailure(conn);
> ! return 0;
> ! }
> }
>
> /* OK, it's launched! */
> ***************
> *** 574,580 ****
> --- 614,630 ----
> * we will NOT block waiting for more input.
> */
> if (pqReadData(conn) < 0)
> + {
> + /*
> + * for non-blocking connections
> + * try to flush the send-queue otherwise we may never get a
> + * responce for something that may not have already been sent
> + * because it's in our write buffer!
> + */
> + if (pqIsnonblocking(conn))
> + (void) pqFlush(conn);
> return 0;
> + }
> /* Parsing of the data waits till later. */
> return 1;
> }
> ***************
> *** 1088,1093 ****
> --- 1138,1153 ----
> {
> PGresult *result;
> PGresult *lastResult;
> + bool savedblocking;
> +
> + /*
> + * we assume anyone calling PQexec wants blocking behaviour,
> + * we force the blocking status of the connection to blocking
> + * for the duration of this function and restore it on return
> + */
> + savedblocking = pqIsnonblocking(conn);
> + if (PQsetnonblocking(conn, FALSE) == -1)
> + return NULL;
>
> /*
> * Silently discard any prior query result that application didn't
> ***************
> *** 1102,1115 ****
> PQclear(result);
> printfPQExpBuffer(&conn->errorMessage,
> "PQexec: you gotta get out of a COPY state yourself.\n");
> ! return NULL;
> }
> PQclear(result);
> }
>
> /* OK to send the message */
> if (!PQsendQuery(conn, query))
> ! return NULL;
>
> /*
> * For backwards compatibility, return the last result if there are
> --- 1162,1176 ----
> PQclear(result);
> printfPQExpBuffer(&conn->errorMessage,
> "PQexec: you gotta get out of a COPY state yourself.\n");
> ! /* restore blocking status */
> ! goto errout;
> }
> PQclear(result);
> }
>
> /* OK to send the message */
> if (!PQsendQuery(conn, query))
> ! goto errout; /* restore blocking status */
>
> /*
> * For backwards compatibility, return the last result if there are
> ***************
> *** 1142,1148 ****
> --- 1203,1217 ----
> result->resultStatus == PGRES_COPY_OUT)
> break;
> }
> +
> + if (PQsetnonblocking(conn, savedblocking) == -1)
> + return NULL;
> return lastResult;
> +
> + errout:
> + if (PQsetnonblocking(conn, savedblocking) == -1)
> + return NULL;
> + return NULL;
> }
>
>
> ***************
> *** 1431,1438 ****
> "PQendcopy() -- I don't think there's a copy in progress.\n");
> return 1;
> }
>
> ! (void) pqFlush(conn); /* make sure no data is waiting to be sent */
>
> /* Return to active duty */
> conn->asyncStatus = PGASYNC_BUSY;
> --- 1500,1516 ----
> "PQendcopy() -- I don't think there's a copy in progress.\n");
> return 1;
> }
> +
> + /*
> + * make sure no data is waiting to be sent,
> + * abort if we are non-blocking and the flush fails
> + */
> + if (pqFlush(conn) && pqIsnonblocking(conn))
> + return (1);
>
> ! /* non blocking connections may have to abort at this point. */
> ! if (pqIsnonblocking(conn) && PQisBusy(conn))
> ! return (1);
>
> /* Return to active duty */
> conn->asyncStatus = PGASYNC_BUSY;
> ***************
> *** 2025,2028 ****
> --- 2103,2192 ----
> return 1;
> else
> return 0;
> + }
> +
> + /* PQsetnonblocking:
> + sets the PGconn's database connection non-blocking if the arg is TRUE
> + or makes it non-blocking if the arg is FALSE, this will not protect
> + you from PQexec(), you'll only be safe when using the non-blocking
> + API
> + Needs to be called only on a connected database connection.
> + */
> +
> + int
> + PQsetnonblocking(PGconn *conn, int arg)
> + {
> + int fcntlarg;
> +
> + arg = (arg == TRUE) ? 1 : 0;
> + /* early out if the socket is already in the state requested */
> + if (arg == conn->nonblocking)
> + return (0);
> +
> + /*
> + * to guarantee constancy for flushing/query/result-polling behavior
> + * we need to flush the send queue at this point in order to guarantee
> + * proper behavior.
> + * this is ok because either they are making a transition
> + * _from_ or _to_ blocking mode, either way we can block them.
> + */
> + /* if we are going from blocking to non-blocking flush here */
> + if (!pqIsnonblocking(conn) && pqFlush(conn))
> + return (-1);
> +
> +
> + #ifdef USE_SSL
> + if (conn->ssl)
> + {
> + printfPQExpBuffer(&conn->errorMessage,
> + "PQsetnonblocking() -- not supported when using SSL\n");
> + return (-1);
> + }
> + #endif /* USE_SSL */
> +
> + #ifndef WIN32
> + fcntlarg = fcntl(conn->sock, F_GETFL, 0);
> + if (fcntlarg == -1)
> + return (-1);
> +
> + if ((arg == TRUE &&
> + fcntl(conn->sock, F_SETFL, fcntlarg | O_NONBLOCK) == -1) ||
> + (arg == FALSE &&
> + fcntl(conn->sock, F_SETFL, fcntlarg & ~O_NONBLOCK) == -1))
> + #else
> + fcntlarg = arg;
> + if (ioctlsocket(conn->sock, FIONBIO, &fcntlarg) != 0)
> + #endif
> + {
> + printfPQExpBuffer(&conn->errorMessage,
> + "PQsetblocking() -- unable to set nonblocking status to %s\n",
> + arg == TRUE ? "TRUE" : "FALSE");
> + return (-1);
> + }
> +
> + conn->nonblocking = arg;
> +
> + /* if we are going from non-blocking to blocking flush here */
> + if (pqIsnonblocking(conn) && pqFlush(conn))
> + return (-1);
> +
> + return (0);
> + }
> +
> + /* return the blocking status of the database connection, TRUE == nonblocking,
> + FALSE == blocking
> + */
> + int
> + PQisnonblocking(const PGconn *conn)
> + {
> +
> + return (pqIsnonblocking(conn));
> + }
> +
> + /* try to force data out, really only useful for non-blocking users */
> + int
> + PQflush(PGconn *conn)
> + {
> +
> + return (pqFlush(conn));
> }
> Index: src/interfaces/libpq/fe-misc.c
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/fe-misc.c,v
> retrieving revision 1.33
> diff -u -c -r1.33 fe-misc.c
> *** src/interfaces/libpq/fe-misc.c 1999/11/30 03:08:19 1.33
> --- src/interfaces/libpq/fe-misc.c 2000/01/12 03:12:14
> ***************
> *** 86,91 ****
> --- 86,122 ----
> {
> size_t avail = Max(conn->outBufSize - conn->outCount, 0);
>
> + /*
> + * if we are non-blocking and the send queue is too full to buffer this
> + * request then try to flush some and return an error
> + */
> + if (pqIsnonblocking(conn) && nbytes > avail && pqFlush(conn))
> + {
> + /*
> + * even if the flush failed we may still have written some
> + * data, recalculate the size of the send-queue relative
> + * to the amount we have to send, we may be able to queue it
> + * afterall even though it's not sent to the database it's
> + * ok, any routines that check the data coming from the
> + * database better call pqFlush() anyway.
> + */
> + if (nbytes > Max(conn->outBufSize - conn->outCount, 0))
> + {
> + printfPQExpBuffer(&conn->errorMessage,
> + "pqPutBytes -- pqFlush couldn't flush enough"
> + " data: space available: %d, space needed %d\n",
> + Max(conn->outBufSize - conn->outCount, 0), nbytes);
> + return EOF;
> + }
> + }
> +
> + /*
> + * is the amount of data to be sent is larger than the size of the
> + * output buffer then we must flush it to make more room.
> + *
> + * the code above will make sure the loop conditional is never
> + * true for non-blocking connections
> + */
> while (nbytes > avail)
> {
> memcpy(conn->outBuffer + conn->outCount, s, avail);
> ***************
> *** 548,553 ****
> --- 579,592 ----
> return EOF;
> }
>
> + /*
> + * don't try to send zero data, allows us to use this function
> + * without too much worry about overhead
> + */
> + if (len == 0)
> + return (0);
> +
> + /* while there's still data to send */
> while (len > 0)
> {
> /* Prevent being SIGPIPEd if backend has closed the connection. */
> ***************
> *** 556,561 ****
> --- 595,601 ----
> #endif
>
> int sent;
> +
> #ifdef USE_SSL
> if (conn->ssl)
> sent = SSL_write(conn->ssl, ptr, len);
> ***************
> *** 585,590 ****
> --- 625,632 ----
> case EWOULDBLOCK:
> break;
> #endif
> + case EINTR:
> + continue;
>
> case EPIPE:
> #ifdef ECONNRESET
> ***************
> *** 616,628 ****
> ptr += sent;
> len -= sent;
> }
> if (len > 0)
> {
> /* We didn't send it all, wait till we can send more */
>
> - /* At first glance this looks as though it should block. I think
> - * that it will be OK though, as long as the socket is
> - * non-blocking. */
> if (pqWait(FALSE, TRUE, conn))
> return EOF;
> }
> --- 658,688 ----
> ptr += sent;
> len -= sent;
> }
> +
> if (len > 0)
> {
> /* We didn't send it all, wait till we can send more */
> +
> + /*
> + * if the socket is in non-blocking mode we may need
> + * to abort here
> + */
> + #ifdef USE_SSL
> + /* can't do anything for our SSL users yet */
> + if (conn->ssl == NULL)
> + {
> + #endif
> + if (pqIsnonblocking(conn))
> + {
> + /* shift the contents of the buffer */
> + memmove(conn->outBuffer, ptr, len);
> + conn->outCount = len;
> + return EOF;
> + }
> + #ifdef USE_SSL
> + }
> + #endif
>
> if (pqWait(FALSE, TRUE, conn))
> return EOF;
> }
> Index: src/interfaces/libpq/libpq-fe.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-fe.h,v
> retrieving revision 1.55
> diff -u -c -r1.55 libpq-fe.h
> *** src/interfaces/libpq/libpq-fe.h 2000/01/15 05:37:21 1.55
> --- src/interfaces/libpq/libpq-fe.h 2000/01/17 02:35:56
> ***************
> *** 263,268 ****
> --- 263,275 ----
> extern int PQputnbytes(PGconn *conn, const char *buffer, int nbytes);
> extern int PQendcopy(PGconn *conn);
>
> + /* Set blocking/nonblocking connection to the backend */
> + extern int PQsetnonblocking(PGconn *conn, int arg);
> + extern int PQisnonblocking(const PGconn *conn);
> +
> + /* Force the write buffer to be written (or at least try) */
> + extern int PQflush(PGconn *conn);
> +
> /*
> * "Fast path" interface --- not really recommended for application
> * use
> Index: src/interfaces/libpq/libpq-int.h
> ===================================================================
> RCS file: /home/pgcvs/pgsql/src/interfaces/libpq/libpq-int.h,v
> retrieving revision 1.16
> diff -u -c -r1.16 libpq-int.h
> *** src/interfaces/libpq/libpq-int.h 2000/01/15 05:37:21 1.16
> --- src/interfaces/libpq/libpq-int.h 2000/01/17 02:35:56
> ***************
> *** 214,219 ****
> --- 214,222 ----
> int inEnd; /* offset to first position after avail
> * data */
>
> + int nonblocking; /* whether this connection is using a blocking
> + * socket to the backend or not */
> +
> /* Buffer for data not yet sent to backend */
> char *outBuffer; /* currently allocated buffer */
> int outBufSize; /* allocated size of buffer */
> ***************
> *** 299,303 ****
> --- 302,312 ----
> #define strerror(A) (sys_errlist[(A)])
> #endif /* sunos4 */
> #endif /* !strerror */
> +
> + /*
> + * this is so that we can check is a connection is non-blocking internally
> + * without the overhead of a function call
> + */
> + #define pqIsnonblocking(conn) (conn->nonblocking)
>
> #endif /* LIBPQ_INT_H */
>
>
> ************
>
-- Bruce Momjian | http://www.op.net/~candle pgman@candle.pha.pa.us | (610)
853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill,
Pennsylvania19026
Applied, or did I already say that? > * Tom Lane <tgl@sss.pgh.pa.us> [000109 08:18] wrote: > > Don Baccus <dhogaza@pacifier.com> writes: > > > At 05:27 PM 1/8/00 -0500, Tom Lane wrote: > > >> I also object strongly to the lack of documentation. > > > > > ... I know there are some folks who aren't native-english speakers, so > > > perhaps you don't want to require that the implementor of such patches > > > provide the final documentation wording. But the information should > > > be there and spelled out in a form that can be very easily moved to > > > the docs. > > > > Oh, absolutely. Thomas, our master of the docs, has always had the > > policy of "give me some words, I'll take care of formatting and > > editing..." > > > > I was probably too harsh on Alfred last night, since in fact his code > > was fairly well commented, and some minimal doco could have been > > extracted from the routine headers. But on a change like this, I think > > some paragraphs of coherent high-level explanation are needed: what it > > does, when and why you'd use it. I didn't see that anywhere... > > Here's the revised patch, it includes sgml docs and changes to > ensure that old style connections behave the way they are expected > to: > -- Bruce Momjian | http://www.op.net/~candle pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
* Bruce Momjian <pgman@candle.pha.pa.us> [000118 11:49] wrote: > Applied, or did I already say that? Just one mail was sent, but I cc'd patches and hackers as well as yourself on the message, sorry for duplicates, but since the mailing contained my revised patch I sent to -patches as well. I'll be a bit less zealous in the future. :) sorry, -Alfred
> * Bruce Momjian <pgman@candle.pha.pa.us> [000118 11:49] wrote: > > Applied, or did I already say that? > > Just one mail was sent, but I cc'd patches and hackers as well as > yourself on the message, sorry for duplicates, but since the mailing > contained my revised patch I sent to -patches as well. > > I'll be a bit less zealous in the future. :) No, that is fine. I usually catch that, but I was not sure in this case. Seems I only sent out one, and had not already sent it. It is good to hit multiple lists with something that has been discussed this much. -- Bruce Momjian | http://www.op.net/~candle pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026