Re: [COMMITTERS] pgsql-server/ /configure /configure.in rc/incl ...
От | Sean Chittenden |
---|---|
Тема | Re: [COMMITTERS] pgsql-server/ /configure /configure.in rc/incl ... |
Дата | |
Msg-id | 20030307214630.GI79234@perrin.int.nxad.com обсуждение исходный текст |
Ответ на | Re: [COMMITTERS] pgsql-server/ /configure /configure.in rc/incl ... (Tom Lane <tgl@sss.pgh.pa.us>) |
Список | pgsql-performance |
> > Absolutely! Is this a simple benchmark? Yup. Do I think it > > simulates PostgreSQL? Eh, not particularly. I think quite a few of these Q's would have been answered by reading the code/Makefile.... > This would be on what OS? FreeBSD, but it shouldn't matter. Any reasonably written VM should have similar numbers (though BSD is generally regarded as having the best VM, which, I think Linux poached not that long ago, iirc ::grimace::). > What hardware? My ultra-pathetic laptop with some fine - overly-noisy and can hardly buildworld - IDE drives. > What size test file? In this case, only 72K. I've just updated the test program to use an array of files though. > Do the "iterations" mean so many reads of the entire file, or so > many buffer-sized read requests? In some cases, yes. With the file mmap()'ed, sorta. One of the test cases (the one that did it in ~8s), mmap()'ed and munmap()'ed the file every iteration and was twice as fast as the vanilla read() call. > Did the mmap case actually *read* anything, or just map and unmap > the file? Nope, read it and wrote it out to stdout (which was redirected to /dev/null). > Also, what did you do to normalize for the effects of the test file > being already in kernel disk cache after the first test? That honestly doesn't matter too much since I wasn't testing the rate of reading in files from my hard drive, only the OS's ability to read/write pages of data around. In any case, I've updated my test case to iterate through an array of files instead of just reading in a copy of /etc/services. My laptop is generally a poor benchmark for disk read performance given it takes 8hrs to buildworld, over 12hrs to build mozilla, 18 for KDE, and about 48hrs for Open Office. :) Someone with faster disks may want to try this and report back, but it doesn't matter much in terms of relevancy for considering the benefits of mmap(). The point is that there are calls that can be used that substantially speed up read()'s and write()'s by allowing the VM to align pages of data and give hints about its usage. For the sake of argument re: the previously done tests, I'll reverse the order in which I ran them and I bet dime to dollar that the times will be identical. % make ~/open_source/mmap_test cp -f /etc/services ./services gcc -O3 -finline-functions -fkeep-inline-functions -funroll-loops -DDEFAULT_READSIZE=1 -DDO_MMAP=1 -DDO_MMAP_ONCE=1 -o mmap-testmmap-test.c /usr/bin/time ./mmap-test > /dev/null Beginning tests with file: services Page size: 4096 File read size is the same as the file size Number of iterations: 100000 Start time: 1047064672.276544 Time: 1.281477 Completed tests 1.29 real 0.10 user 0.92 sys gcc -O3 -finline-functions -fkeep-inline-functions -funroll-loops -DDEFAULT_READSIZE=1 -DDO_MMAP=1 -o mmap-test mmap-test.c /usr/bin/time ./mmap-test > /dev/null Beginning tests with file: services Page size: 4096 File read size is the same as the file size Number of iterations: 100000 Start time: 1047064674.266191 Time: 7.486622 Completed tests 7.49 real 0.41 user 6.01 sys gcc -O3 -finline-functions -fkeep-inline-functions -funroll-loops -DDEFAULT_READSIZE=1 -o mmap-test mmap-test.c /usr/bin/time ./mmap-test > /dev/null Beginning tests with file: services Page size: 4096 File read size is default read size: 65536 Number of iterations: 100000 Start time: 1047064682.288637 Time: 19.35214 Completed tests 19.04 real 0.88 user 15.43 sys gcc -O3 -finline-functions -fkeep-inline-functions -funroll-loops -o mmap-test mmap-test.c /usr/bin/time ./mmap-test > /dev/null Beginning tests with file: services Page size: 4096 File read size is the same as the file size Number of iterations: 100000 Start time: 1047064701.867031 Time: 82.4294540875 Completed tests 81.57 real 2.10 user 69.55 sys Here's the updated test that iterates through. Ooh! One better, the files I've used are actual data files from ~pgsql. The new benchmark iterates through the list of files and and calls bench() once for each file and restarts at the first file after reaching the end of its list (ARGV). Whoa, if these tests are even close to real world, then we at the very least should be mmap()'ing the file every time we read it (assuming we're reading more than just a handful of bytes): find /usr/local/pgsql/data -type f | /usr/bin/xargs /usr/bin/time ./mmap-test > /dev/null Page size: 4096 File read size is the same as the file size Number of iterations: 100000 Start time: 1047071143.463360 Time: 12.109530 Completed tests 12.11 real 0.36 user 6.80 sys find /usr/local/pgsql/data -type f | /usr/bin/xargs /usr/bin/time ./mmap-test > /dev/null Page size: 4096 File read size is default read size: 65536 Number of iterations: 100000 .... [been waiting here for >40min now....] Ah well, if these tests finish this century, I'll post the results in a bit, but it's pretty clearly a win. In terms of the data that I'm copying, I'm copying ~700MB of data from my test DB on my laptop. I only have 256MB of RAM so I can pretty much promise you that the data isn't in my system buffers. If anyone else would like to run the tests or look at the results, please check it out: o1 and o2 should be the only targets used if FILES is bigger than the RAM on the system. o3's by far and away the fastest, but only in rare cases will a DBA have more RAM than data. But, as mentioned earlier, the LRU cache could easily be modified to munmap() infrequently accessed files to keep the size of mmap()'ed data down to a reasonable level. The updated test programs are at: http://people.FreeBSD.org/~seanc/mmap_test/ -sc -- Sean Chittenden
Вложения
В списке pgsql-performance по дате отправления: