Re: Database Failure on Windows XP Pro psql (PostgreSQL)

Shelby Cain wrote:
> I'm just throwing this out here... but the usual suspects for strange connection issues like these would be a
softwarefirewall or antivirus (especially AV software that has anti-worm features). 
>
> Also, Windows XP Pro isn't really viable as a server OS as Microsoft has intentionally limited certain features of
theTCP/IP stack (e.g.: low limits for the TCP connection backlog).  In theory, if you had a flood of incoming
connectionsonly some of them would be serviced correctly before the backlog limit was reached. 
>
> Regards,
>
> Shelby Cain
>
> ----- Original Message ----
> From: Oisin Glynn <me@oisinglynn.com>
> To: pgsql-general@postgresql.org
> Sent: Monday, January 8, 2007 1:33:54 PM
> Subject: [GENERAL] Database Failure on Windows XP Pro psql (PostgreSQL) 8.1.4
>
> I am running postgres on Windows XP and have been for quite a while as
>
> the database engine for our application.
>
> On an ODBC connection the following error has started happening. The gap
> in activity is normal for our application. but the connection failing is
> a first we have this application live in approx 5 locations some for as
> long as 12 months (though only 2 live are on XP)
>
> We did enable the system to check for Windows updates automatically
> recently not sure if this could be a factor.
> There is another one of our apps connecting through the C++ API which is
> not seeing any loss of service.  I suspect some sort of tcp-ip issue as
> I cannot connect with PgAdminIII locally on the server once the
> condition occurs. I have had this 3 times now and it seems to be
> happening very 2-3hours of operation. I am not getting any eventvwr or
> other Windows errors telling me I am hitting a limit or such?  And when
> I restarted the Postgresql service I got the following as part of my
> log(See attached logs)
>
> 2007-01-05 14:38:28 LOG:  pgpipe failed to bind: 10055i
> 2007-01-05 14:38:28 FATAL:  could not create pipe for statistics buffer:
> No error
>
> I am attaching my postgresql.conf (perhaps I can turn on some more
> logging to make things easier to spot) and the log files I have from the
> first occurrence. These show my attempts to restart the postgresql
> service and the errors I received.
>
> This is a snipit of my applications log file showing the error pop up at
> 14:11:07
>
> 01/05/07 13:58:57.453 2296 DEBUG  - Call No: 10131  SELECT   SQL =
> SELECT * FROM zfunc_access_mailbox_cleanup(10131)
> 01/05/07 13:58:57.468 2296 DEBUG  - Call No: 10131  SELECT Complete
> Return Code = $VC1
> 01/05/07 13:58:57.468 2296 DEBUG  - BX_SVR_TRP009 Sent TRP_IVR_ODBCOK
> for call no= 10131
> 01/05/07 13:58:57.484 2296 DEBUG  - Call No: 10131  DISCONNECT
> 01/05/07 13:58:57.484 2296 DEBUG  - Call No: 10131  DISCONNECT Free
> Statement Handle
> 01/05/07 13:58:57.484 2296 DEBUG  - Call No: 10131DISCONNECT OK Voicemail
> 01/05/07 13:58:57.484 2296 DEBUG  - BX_SVR_TRP009 Sent TRP_IVR_ODBCOK
> for call no= 10131
> 01/05/07 14:11:07.734 2296 DEBUG  - Call No: 10132  CONNECT Voicemail
> 01/05/07 14:11:07.734 2296 DEBUG  - Call No: 10132  CONNECT Allocate DBC
> Handle Voicemail
> 01/05/07 14:11:07.734 2296 DEBUG  - Call No: 10132  CONNECT Connect
> Voicemail
> 01/05/07 14:11:07.750 2296 DEBUG  - Call No: 10132CONNECT SQLConnect
> failed   08001  could not connect to server: No buffer space available
> (0x00002747/10055)
>     Is the server running on host "127.0.0.1" and accepting
>     TCP/IP connections on port 5432?
>  Error Message = 165Voicemail
>
> If there is anything I can do to generate better logging I am all ears,
>
> Oisin
>
>
>
>
>
>
> # -----------------------------
> # PostgreSQL configuration file
> # -----------------------------
> #
> # This file consists of lines of the form:
> #
> #   name = value
> #
> # (The '=' is optional.) White space may be used. Comments are introduced
> # with '#' anywhere on a line. The complete list of option names and
> # allowed values can be found in the PostgreSQL documentation. The
> # commented-out settings shown in this file represent the default values.
> #
> # Please note that re-commenting a setting is NOT sufficient to revert it
> # to the default value, unless you restart the postmaster.
> #
> # Any option can also be given as a command line switch to the
> # postmaster, e.g. 'postmaster -c log_connections=on'. Some options
> # can be changed at run-time with the 'SET' SQL command.
> #
> # This file is read on postmaster startup and when the postmaster
> # receives a SIGHUP. If you edit the file on a running system, you have
> # to SIGHUP the postmaster for the changes to take effect, or use
> # "pg_ctl reload". Some settings, such as listen_addresses, require
> # a postmaster shutdown and restart to take effect.
>
>
> #---------------------------------------------------------------------------
> # FILE LOCATIONS
> #---------------------------------------------------------------------------
>
> # The default values of these variables are driven from the -D command line
> # switch or PGDATA environment variable, represented here as ConfigDir.
>
> #data_directory = 'ConfigDir'        # use data in another directory
> #hba_file = 'ConfigDir/pg_hba.conf'    # host-based authentication file
> #ident_file = 'ConfigDir/pg_ident.conf'    # IDENT configuration file
>
> # If external_pid_file is not explicitly set, no extra pid file is written.
> #external_pid_file = '(none)'        # write an extra pid file
>
>
> #---------------------------------------------------------------------------
> # CONNECTIONS AND AUTHENTICATION
> #---------------------------------------------------------------------------
>
> # - Connection Settings -
>
> #listen_addresses = 'localhost'        # what IP address(es) to listen on;
>                     # comma-separated list of addresses;
>                     # defaults to 'localhost', '*' = all
> port = 5432
> max_connections = 100
> # note: increasing max_connections costs ~400 bytes of shared memory per
> # connection slot, plus lock space (see max_locks_per_transaction).  You
> # might also need to raise shared_buffers to support more connections.
> #superuser_reserved_connections = 2
> #unix_socket_directory = ''
> #unix_socket_group = ''
> #unix_socket_permissions = 0777        # octal
> #bonjour_name = ''            # defaults to the computer name
>
> # - Security & Authentication -
>
> #authentication_timeout = 60        # 1-600, in seconds
> #ssl = off
> #password_encryption = on
> #db_user_namespace = off
>
> # Kerberos
> #krb_server_keyfile = ''
> #krb_srvname = 'postgres'
> #krb_server_hostname = ''        # empty string matches any keytab entry
> #krb_caseins_users = off
>
> # - TCP Keepalives -
> # see 'man 7 tcp' for details
>
> #tcp_keepalives_idle = 0        # TCP_KEEPIDLE, in seconds;
>                     # 0 selects the system default
> #tcp_keepalives_interval = 0        # TCP_KEEPINTVL, in seconds;
>                     # 0 selects the system default
> #tcp_keepalives_count = 0        # TCP_KEEPCNT;
>                     # 0 selects the system default
>
>
> #---------------------------------------------------------------------------
> # RESOURCE USAGE (except WAL)
> #---------------------------------------------------------------------------
>
> # - Memory -
>
> shared_buffers = 1000            # min 16 or max_connections*2, 8KB each
> #temp_buffers = 1000            # min 100, 8KB each
> #max_prepared_transactions = 5        # can be 0 or more
> # note: increasing max_prepared_transactions costs ~600 bytes of shared memory
> # per transaction slot, plus lock space (see max_locks_per_transaction).
> #work_mem = 1024            # min 64, size in KB
> #maintenance_work_mem = 16384        # min 1024, size in KB
> #max_stack_depth = 2048            # min 100, size in KB
>
> # - Free Space Map -
>
> #max_fsm_pages = 20000            # min max_fsm_relations*16, 6 bytes each
> #max_fsm_relations = 1000        # min 100, ~70 bytes each
>
> # - Kernel Resource Usage -
>
> #max_files_per_process = 1000        # min 25
> #preload_libraries = ''
>
> # - Cost-Based Vacuum Delay -
>
> #vacuum_cost_delay = 0            # 0-1000 milliseconds
> #vacuum_cost_page_hit = 1        # 0-10000 credits
> #vacuum_cost_page_miss = 10        # 0-10000 credits
> #vacuum_cost_page_dirty = 20        # 0-10000 credits
> #vacuum_cost_limit = 200        # 0-10000 credits
>
> # - Background writer -
>
> #bgwriter_delay = 200            # 10-10000 milliseconds between rounds
> #bgwriter_lru_percent = 1.0        # 0-100% of LRU buffers scanned/round
> #bgwriter_lru_maxpages = 5        # 0-1000 buffers max written/round
> #bgwriter_all_percent = 0.333        # 0-100% of all buffers scanned/round
> #bgwriter_all_maxpages = 5        # 0-1000 buffers max written/round
>
>
> #---------------------------------------------------------------------------
> # WRITE AHEAD LOG
> #---------------------------------------------------------------------------
>
> # - Settings -
>
> #fsync = on                # turns forced synchronization on or off
> #wal_sync_method = fsync        # the default is the first option
>                     # supported by the operating system:
>                     #   open_datasync
>                     #   fdatasync
>                     #   fsync
>                     #   fsync_writethrough
>                     #   open_sync
> #full_page_writes = on            # recover from partial page writes
> #wal_buffers = 8            # min 4, 8KB each
> #commit_delay = 0            # range 0-100000, in microseconds
> #commit_siblings = 5            # range 1-1000
>
> # - Checkpoints -
>
> #checkpoint_segments = 3        # in logfile segments, min 1, 16MB each
> #checkpoint_timeout = 300        # range 30-3600, in seconds
> #checkpoint_warning = 30        # in seconds, 0 is off
>
> # - Archiving -
>
> #archive_command = ''            # command to use to archive a logfile
>                     # segment
>
>
> #---------------------------------------------------------------------------
> # QUERY TUNING
> #---------------------------------------------------------------------------
>
> # - Planner Method Configuration -
>
> #enable_bitmapscan = on
> #enable_hashagg = on
> #enable_hashjoin = on
> #enable_indexscan = on
> #enable_mergejoin = on
> #enable_nestloop = on
> #enable_seqscan = on
> #enable_sort = on
> #enable_tidscan = on
>
> # - Planner Cost Constants -
>
> #effective_cache_size = 1000        # typically 8KB each
> #random_page_cost = 4            # units are one sequential page fetch
>                     # cost
> #cpu_tuple_cost = 0.01            # (same)
> #cpu_index_tuple_cost = 0.001        # (same)
> #cpu_operator_cost = 0.0025        # (same)
>
> # - Genetic Query Optimizer -
>
> #geqo = on
> #geqo_threshold = 12
> #geqo_effort = 5            # range 1-10
> #geqo_pool_size = 0            # selects default based on effort
> #geqo_generations = 0            # selects default based on effort
> #geqo_selection_bias = 2.0        # range 1.5-2.0
>
> # - Other Planner Options -
>
> #default_statistics_target = 10        # range 1-1000
> #constraint_exclusion = off
> #from_collapse_limit = 8
> #join_collapse_limit = 8        # 1 disables collapsing of explicit
>                     # JOINs
>
>
> #---------------------------------------------------------------------------
> # ERROR REPORTING AND LOGGING
> #---------------------------------------------------------------------------
>
> # - Where to Log -
>
> log_destination = 'stderr'        # Valid values are combinations of
>                     # stderr, syslog and eventlog,
>                     # depending on platform.
>
> # This is used when logging to stderr:
> redirect_stderr = on            # Enable capturing of stderr into log
>                     # files
>
> # These are only used if redirect_stderr is on:
> #log_directory = 'pg_log'        # Directory where log files are written
>                     # Can be absolute or relative to PGDATA
> #log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # Log file name pattern.
>                     # Can include strftime() escapes
> #log_truncate_on_rotation = off # If on, any existing log file of the same
>                     # name as the new log file will be
>                     # truncated rather than appended to. But
>                     # such truncation only occurs on
>                     # time-driven rotation, not on restarts
>                     # or size-driven rotation. Default is
>                     # off, meaning append to existing files
>                     # in all cases.
> #log_rotation_age = 1440        # Automatic rotation of logfiles will
>                     # happen after so many minutes.  0 to
>                     # disable.
> #log_rotation_size = 10240        # Automatic rotation of logfiles will
>                     # happen after so many kilobytes of log
>                     # output.  0 to disable.
>
> # These are relevant when logging to syslog:
> #syslog_facility = 'LOCAL0'
> #syslog_ident = 'postgres'
>
>
> # - When to Log -
>
> #client_min_messages = notice        # Values, in order of decreasing detail:
>                     #   debug5
>                     #   debug4
>                     #   debug3
>                     #   debug2
>                     #   debug1
>                     #   log
>                     #   notice
>                     #   warning
>                     #   error
>
> #log_min_messages = notice        # Values, in order of decreasing detail:
>                     #   debug5
>                     #   debug4
>                     #   debug3
>                     #   debug2
>                     #   debug1
>                     #   info
>                     #   notice
>                     #   warning
>                     #   error
>                     #   log
>                     #   fatal
>                     #   panic
>
> #log_error_verbosity = default        # terse, default, or verbose messages
>
> #log_min_error_statement = panic    # Values in order of increasing severity:
>                      #   debug5
>                     #   debug4
>                     #   debug3
>                     #   debug2
>                     #   debug1
>                      #   info
>                     #   notice
>                     #   warning
>                     #   error
>                     #   panic(off)
>
> #log_min_duration_statement = -1    # -1 is disabled, 0 logs all statements
>                     # and their durations, in milliseconds.
>
> #silent_mode = off            # DO NOT USE without syslog or
>                     # redirect_stderr
>
> # - What to Log -
>
> #debug_print_parse = off
> #debug_print_rewritten = off
> #debug_print_plan = off
> #debug_pretty_print = off
> #log_connections = off
> #log_disconnections = off
> #log_duration = off
> log_line_prefix = '%t '            # Special values:
>                     #   %u = user name
>                     #   %d = database name
>                     #   %r = remote host and port
>                     #   %h = remote host
>                     #   %p = PID
>                     #   %t = timestamp (no milliseconds)
>                     #   %m = timestamp with milliseconds
>                     #   %i = command tag
>                     #   %c = session id
>                     #   %l = session line number
>                     #   %s = session start timestamp
>                     #   %x = transaction id
>                     #   %q = stop here in non-session
>                     #        processes
>                     #   %% = '%'
>                     # e.g. '<%u%%%d> '
> #log_statement = 'none'            # none, mod, ddl, all
> #log_hostname = off
>
>
> #---------------------------------------------------------------------------
> # RUNTIME STATISTICS
> #---------------------------------------------------------------------------
>
> # - Statistics Monitoring -
>
> #log_parser_stats = off
> #log_planner_stats = off
> #log_executor_stats = off
> #log_statement_stats = off
>
> # - Query/Index Statistics Collector -
>
> stats_start_collector = on
> #stats_command_string = off
> #stats_block_level = off
> stats_row_level = on
> #stats_reset_on_server_start = off
>
>
> #---------------------------------------------------------------------------
> # AUTOVACUUM PARAMETERS
> #---------------------------------------------------------------------------
>
> autovacuum = on            # enable autovacuum subprocess?
> #autovacuum_naptime = 60        # time between autovacuum runs, in secs
> #autovacuum_vacuum_threshold = 1000    # min # of tuple updates before
>                     # vacuum
> #autovacuum_analyze_threshold = 500    # min # of tuple updates before
>                     # analyze
> #autovacuum_vacuum_scale_factor = 0.4    # fraction of rel size before
>                     # vacuum
> #autovacuum_analyze_scale_factor = 0.2    # fraction of rel size before
>                     # analyze
> #autovacuum_vacuum_cost_delay = -1    # default vacuum cost delay for
>                     # autovac, -1 means use
>                     # vacuum_cost_delay
> #autovacuum_vacuum_cost_limit = -1    # default vacuum cost limit for
>                     # autovac, -1 means use
>                     # vacuum_cost_limit
>
>
> #---------------------------------------------------------------------------
> # CLIENT CONNECTION DEFAULTS
> #---------------------------------------------------------------------------
>
> # - Statement Behavior -
>
> #search_path = '$user,public'        # schema names
> #default_tablespace = ''        # a tablespace name, '' uses
>                     # the default
> #check_function_bodies = on
> #default_transaction_isolation = 'read committed'
> #default_transaction_read_only = off
> #statement_timeout = 0            # 0 is disabled, in milliseconds
>
> # - Locale and Formatting -
>
> #datestyle = 'iso, mdy'
> #timezone = unknown            # actually, defaults to TZ
>                     # environment setting
> #australian_timezones = off
> #extra_float_digits = 0            # min -15, max 2
> #client_encoding = sql_ascii        # actually, defaults to database
>                     # encoding
>
> # These settings are initialized by initdb -- they might be changed
> lc_messages = 'C'            # locale for system error message
>                     # strings
> lc_monetary = 'C'            # locale for monetary formatting
> lc_numeric = 'C'            # locale for number formatting
> lc_time = 'C'                # locale for time formatting
>
> # - Other Defaults -
>
> #explain_pretty_print = on
> #dynamic_library_path = '$libdir'
>
>
> #---------------------------------------------------------------------------
> # LOCK MANAGEMENT
> #---------------------------------------------------------------------------
>
> #deadlock_timeout = 1000        # in milliseconds
> #max_locks_per_transaction = 64        # min 10
> # note: each lock table slot uses ~220 bytes of shared memory, and there are
> # max_locks_per_transaction * (max_connections + max_prepared_transactions)
> # lock table slots.
>
>
> #---------------------------------------------------------------------------
> # VERSION/PLATFORM COMPATIBILITY
> #---------------------------------------------------------------------------
>
> # - Previous Postgres Versions -
>
> #add_missing_from = off
> #backslash_quote = safe_encoding    # on, off, or safe_encoding
> #default_with_oids = off
> #escape_string_warning = off
> #regex_flavor = advanced        # advanced, extended, or basic
> #sql_inheritance = on
>
> # - Other Platforms & Clients -
>
> #transform_null_equals = off
>
>
> #---------------------------------------------------------------------------
> # CUSTOMIZED OPTIONS
> #---------------------------------------------------------------------------
>
> #custom_variable_classes = ''        # list of custom variable class names
>
>
> 2007-01-05 14:37:02 LOG:  database system was shut down at 2007-01-05 14:36:54 Eastern Standard Time
> 2007-01-05 14:37:02 LOG:  checkpoint record is at 0/E1B0328
> 2007-01-05 14:37:02 LOG:  redo record is at 0/E1B0328; undo record is at 0/0; shutdown TRUE
> 2007-01-05 14:37:02 LOG:  next transaction ID: 780781; next OID: 17780
> 2007-01-05 14:37:02 LOG:  next MultiXactId: 1; next MultiXactOffset: 0
> 2007-01-05 14:37:02 LOG:  database system is ready
> 2007-01-05 14:37:02 LOG:  transaction ID wrap limit is 2147484148, limited by database "postgres"
> 2007-01-05 14:37:02 LOG:  pgpipe failed to bind: 10055i
> 2007-01-05 14:37:02 FATAL:  could not create pipe for statistics buffer: No error
> 2007-01-05 14:37:02 LOG:  autovacuum: processing database "template1"
> 2007-01-05 14:38:02 LOG:  autovacuum: processing database "template1"
> 2007-01-05 14:38:02 LOG:  pgpipe failed to bind: 10055i
> 2007-01-05 14:38:02 FATAL:  could not create pipe for statistics buffer: No error
> 2007-01-05 14:38:25 LOG:  received fast shutdown request
> 2007-01-05 14:38:25 LOG:  shutting down
> 2007-01-05 14:38:25 LOG:  database system is shut down
> 2007-01-05 14:38:26 LOG:  logger shutting down
>
>
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>        choose an index scan if your joining column's datatypes do not
>        match
>

I have exceptions in the FW for postgresql.
I know XP Pro is not a server OS but it is a $$ question (I know *nix is
cheaper but it is not an option without a major rewrite). The Server is
really more like a network appliance it is not expecting to have extra
connections all over. In fact the DB is only accessible on 127.0.0.1

I have been checking for XP limits coming into play but the kicker is
that there are no errors appearing (which other people are reporting
lots of with SP2)
The 3 limits I am aware of are
a) There is a new limit on numbers of outbound connections being created
per second (people using e mule and torrents etc are having issues) We
are not creating masses of outbound requests per second (at least AFAIK
unless something in PG or something else is? without us noticing)
b)  There is a max open TCP/IP connections which is not physically
enforced but may legally be inforced......(according to MS)
c) There is a hard limit on NAMED_PIPE and file shares that is allegedly
at 10. It is unclear exactly what is being counted here as I had
NAMED_PIPES and 5 file shares from different PC's open earlier trying to
cause a failure.

We have run netstat to check for odd looking connections on each
occurrence but never seen any?
Are there pieces of logging I should turn on in my .conf that would be
helpful?

Oisin