Re: Add test module for Table Access Method

Поиск
Список
Период
Сортировка
От Mats Kindahl
Тема Re: Add test module for Table Access Method
Дата
Msg-id CA+14424qbsh4yQuQdsi90xOhqzk+12CoJp6w=edd_jpwJoyLCQ@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Add test module for Table Access Method  (Aleksander Alekseev <aleksander@timescale.com>)
Список pgsql-hackers
Hi all,

On Tue, Jan 16, 2024 at 10:40 AM Aleksander Alekseev <aleksander@timescale.com> wrote:
Hi,

> > I think a test module for a table AM will really help developers. Just
> > to add to the above list - how about the table AM implementing a
> > simple in-memory (columnar if possible) database storing tables
> > in-memory and subsequently providing readers with the access to the
> > tables?
>
> That's a good idea.

Personally I would be careful with this idea.

Practice shows that when you show the first incomplete, limited and
buggy PoC it ends up being in the production environment the next day
:) In other words sooner or later there will be users demanding a full
in-memory columnar storage support from Postgres. I believe it would
be a problem. Last time I checked TAM was not extremely good for
implementing proper columnar storages, and there are lots of open
questions when it comes to in-memory tables (e.g. what to do with
foreign keys, inherited tables, etc).

All in all I don't think we should provide something that can look /
be interpreted as first-class alternative storage but in fact is not.

I tossed together a table access method for in-memory storage in column format for experimental purposes over the holidays (I actually have a row-based one as well, but that is in no shape to share at this point). It's available under https://github.com/mkindahl/pg_arrow. The intention was mostly to have something simple to play and experiment with. It is loosely based on the Apache Arrow Columnar format, but the normal data structures are not suitable for storing in shared memory so I have tweaked it a little.
 
> How about adding the same kind of coverage as dummy_index_am with a
> couple of reloptions then?  That can serve as a point of reference
> when a table AM needs a few custom options.  A second idea would be to
> show how to use toast relations when implementing your new AM, where a
> toast table could be created even in cases where we did not want one
> with heap, when it comes to size limitations with char and/or varchar,
> and that makes for a simpler needs_toast_table callback.

Good ideas. Additionally we could provide a proxy TAM for a heap TAM
which does nothing but logging used TAM methods, its arguments and
return values. This would be a good example and also potentially can
be used as a debugging tool.

We wrote a table access method for experimenting with and to be able to trace what happens while executing various statements. It is available under https://github.com/timescale/pg_traceam for anybody who is interested.

Best wishes,
Mats Kindahl
 

--
Best regards,
Aleksander Alekseev


В списке pgsql-hackers по дате отправления:

Предыдущее
От: "Anton A. Melnikov"
Дата:
Сообщение: Re: ResourceOwner refactoring
Следующее
От: Andrey Borodin
Дата:
Сообщение: Re: UUID v7