Обсуждение: arrays, inline to pointer
Hi, I have some whopper tables for machine learning. One table has a handful of 200-500 double precision arrays (representing feature vectors). It's a 9.5 on a SSD (over USB3). Each table has 5-15M rows in them. For each array I've added, and populated, any dealings with the table has become way way slower. I can only assume this is because the array data is inline in the datablock on disk that stores the row. Is it possible to "have a pointer instead", or is the only workaround a child table with vector only in it? Best regards, Marcus
On 5/3/2016 1:21 AM, Marcus Engene wrote:
For each array I've added, and populated, any dealings with the table has become way way slower. I can only assume this is because the array data is inline in the datablock on disk that stores the row.
any field on a table thats more than a few dozen bytes gets 'toasted' and stored out of line in special 'toast' tables.
-- john r pierce, recycling bits in santa cruz
In fact, disabling toast compression will probably improve the performance (the indirection will still take place). A float array is not usually very compressible anyway.
On May 3, 2016 10:37 AM, "John R Pierce" <pierce@hogranch.com> wrote:
On 5/3/2016 1:21 AM, Marcus Engene wrote:For each array I've added, and populated, any dealings with the table has become way way slower. I can only assume this is because the array data is inline in the datablock on disk that stores the row.any field on a table thats more than a few dozen bytes gets 'toasted' and stored out of line in special 'toast' tables.
-- john r pierce, recycling bits in santa cruz