unused tuples constantly increasing

Поиск
Список
Период
Сортировка
От Chantal Ackermann
Тема unused tuples constantly increasing
Дата
Msg-id 3E1EC843.4010701@biomax.de
обсуждение исходный текст
Ответы Re: unused tuples constantly increasing  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-general
hello all,

With PostgreSQL version 7.3, I have the following problem while
updating, and vacuuming a table containing over 2,000,000 rows:

What I am doing: I want to build a tsearch index for one of the columns
(called "sentence") in this table. For this I added a new column called
"sentenceidx" which now has to be populated with the parsed text from
"sentence", using txt2txtidx (comes with contrib/tsearch).

The script I use updates 2000 rows at once, then runs a VACUUM VERBOSE
on the table. The update of the 2000 rows takes always less then a
second (more likely about 0.6 sec) which I think is ok. The VACUUM,
though, takes always more than 110 secs. Thus, the script has been
running for quite some time, now.

What concerns me even more is that the value for "UnUsed" in the vacuum
output is constantly increasing, even though I increased max_fsm_pages
four times (to be 40000) (and restarted the postmaster).

I have searched the net, but I am obviously searching for the wrong
keywords as I have not found a solution I can understand and apply.
Probably all this is just the result of a poor optimization of the
database: while this script runs, postgres writes constantly to disc,
while "top" tells:

   PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
19217 postgres  10   0  124M 124M  123M R     8.7 24.2   9:14 postmaster



I am pasting three consecutive outputs from "VACUUM VERBOSE sentences;",
part of postgresql.conf, and some system properties.

The 2 million rows are only a little part of the whole data that should
populate the db in the future, so I am very interested in advice on how
to optimize postgres well.

regards,

Chantal

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Here is the output from VACUUM:

UPDATE 2000
psql:1202: INFO:  --Relation public.sentences--
psql:1202: INFO:  Index sentences_pkey: Pages 10644; Tuples 2262126:
Deleted 2000.
         CPU 0.95s/1.18u sec elapsed 39.75 sec.
psql:1202: INFO:  Removed 2000 tuples in 51 pages.
         CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:1202: INFO:  Pages 80239: Changed 143, Empty 0; Tup 2262126: Vac
2000, Keep 0, UnUsed 518555.
         Total CPU 6.84s/1.99u sec elapsed 117.63 sec.
psql:1202: INFO:  --Relation pg_toast.pg_toast_462586--
psql:1202: INFO:  Pages 1: Changed 0, Empty 0; Tup 5: Vac 0, Keep 0,
UnUsed 0.
         Total CPU 0.00s/0.00u sec elapsed 0.06 sec.
VACUUM
UPDATE 2000
psql:1204: INFO:  --Relation public.sentences--
psql:1204: INFO:  Index sentences_pkey: Pages 10650; Tuples 2262126:
Deleted 2000.
         CPU 0.92s/1.16u sec elapsed 38.84 sec.
psql:1204: INFO:  Removed 2000 tuples in 53 pages.
         CPU 0.01s/0.01u sec elapsed 0.01 sec.
psql:1204: INFO:  Pages 80280: Changed 147, Empty 0; Tup 2262126: Vac
2000, Keep 0, UnUsed 519456.
         Total CPU 6.99s/1.93u sec elapsed 117.15 sec.
psql:1204: INFO:  --Relation pg_toast.pg_toast_462586--
psql:1204: INFO:  Pages 1: Changed 0, Empty 0; Tup 5: Vac 0, Keep 0,
UnUsed 0.
         Total CPU 0.00s/0.00u sec elapsed 0.01 sec.
VACUUM
UPDATE 2000
psql:1206: INFO:  --Relation public.sentences--
psql:1206: INFO:  Index sentences_pkey: Pages 10656; Tuples 2262126:
Deleted 2000.
         CPU 0.90s/1.05u sec elapsed 39.61 sec.
psql:1206: INFO:  Removed 2000 tuples in 50 pages.
         CPU 0.01s/0.00u sec elapsed 0.00 sec.
psql:1206: INFO:  Pages 80314: Changed 137, Empty 0; Tup 2262126: Vac
2000, Keep 0, UnUsed 520277.
         Total CPU 6.94s/1.73u sec elapsed 116.77 sec.
psql:1206: INFO:  --Relation pg_toast.pg_toast_462586--
psql:1206: INFO:  Pages 1: Changed 0, Empty 0; Tup 5: Vac 0, Keep 0,
UnUsed 0.
         Total CPU 0.00s/0.00u sec elapsed 0.01 sec.
VACUUM
UPDATE 2000

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

This is the part of postgresql.conf which might be relevant:

#
#       Shared Memory Size
#
shared_buffers = 15200          # 2*max_connections, min 16, typically
                # 8KB each, default 64
max_fsm_relations = 200         # min 10, fsm is free space map, ~40
                # bytes, default 100
max_fsm_pages = 40000           # min 1000, fsm is free space map, ~6
                # bytes, default 10000
#max_locks_per_transaction = 64 # min 10
#wal_buffers = 32               # min 4, typically 8KB each, default 32

#
#       Non-shared Memory Sizes
#
#sort_mem = 1024                # min 64, size in KB
vacuum_mem = 16384              # min 1024, size in KB, default 8192

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

System: Linux 2.2.18, i686
Postgres: 7.3
CPU: 933MHz
RAM: 524Mb

shmall: 16777216 (I did not manage to increase it)
shmmax: 134217728



В списке pgsql-general по дате отправления:

Предыдущее
От: "karthikeyan.balasubramanian"
Дата:
Сообщение: PostgreSQL on Windows
Следующее
От: Andreas Schlegel
Дата:
Сообщение: if exists ...