Hi!
Still, "writing" speed to disk:
With "fio" 3 GB / s.
With PostgreSQL: 350 -500 MB / s, also with a partitioned table.
(and something similar with an other DBMS).
Monitored with dstat.
With about 12 threads, then it does not get better anymore.
Same results with data split to 4 partitions, in a partitioned table. CPU load increased, but not full yet.
Same results with open_datasync.
BR Sam
On pe, lokak. 12, 2018 at 19:27, Sam R.
<samruohola@yahoo.com> wrote:
Hi!
Could someone discuss about following? It would be great to hear comments!
There is a good storage. According to "fio", write speed could be e.g. 3 GB/s.
(It is First time using the command for me, so I am not certain of the real speed with "fio". E.g. with --bs=100m, direct=1, in fio. The measurement result may be faulty also. So, checking still.)
Currently there is one table without partitioning. The table contains json data. In PostgreSQL, in Linux.
Write speed can be e.g. 300 - 600 MB/s, through PostgreSQL. Measured with dstat while inserting. Shared buffers is large is PostgreSQL.
With a storage/disk which "scales", is there some way to write faster to the disk in the system through PostgreSQL?
Inside same server.
Does splitting data help? Partitioned table / splitting to smaller tables? Should I test it?
Change settings somewhere? Block sizes? 8 KB / 16 KB, ... "Dangerous" to change?
2nd question, sharding:
If the storage / "disk" scales, could better *disk writing speed* be achieved (in total) with sharding kind of splitting of data? (Same NAS storage, which scales, in use in all shards.)
Sharding or use only one server? From pure disk writing speed point of view.
BR Sam