Re: WIP: Access method extendability

Поиск
Список
Период
Сортировка
От Alexander Korotkov
Тема Re: WIP: Access method extendability
Дата
Msg-id CAPpHfdvadZq4cmb=JsiBzjympMdyU+=XWGz+gnmEhGBEn1hSWQ@mail.gmail.com
обсуждение исходный текст
Ответ на Re: WIP: Access method extendability  (Simon Riggs <simon@2ndQuadrant.com>)
Ответы Re: WIP: Access method extendability
Re: WIP: Access method extendability
Список pgsql-hackers
Дерево обсуждения
On Tue, Oct 28, 2014 at 8:04 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
On 28 October 2014 14:22, Simon Riggs <simon@2ndquadrant.com> wrote:

> Or put it another way, it will be easier to write new index AMs
> because we'll be able to skip the WAL part until we know we want it.

To be clear: I am suggesting you do *less* work, not more.

By allowing AMs to avoid writing WAL we get
* higher performance unlogged indexes
* we get fewer bugs in early days of new AMs
* writers of new AMs are OK to avoid majority of hard work and hard testing

So overall, we get new AMs working faster because we can skip writing
the WAL code until we are certain the new AM code is useful and bug
free.

For example, if GIN had avoided implementing WAL it would have been
easier to change on-disk representation.

Major problem of changing on-disk representation is that we have to support both binary formats because of pg_upgrade. This problem is even burdened with WAL, because WAL record redo function also have to support both formats. However, it's also quite independent of WAL.

Having access methods as extensions can significantly improves situations here. Imagine, GIN was an extension. One day we decide to change its binary format. Then we can issue new extension, GIN2 for instance. User can upgrade from GIN to GIN2 in following steps:
  1. CREATE EXTENSION gin2;
  2. CREATE INDEX CONCURRENTLY [new_index] USING gin2 ([index_def]);
  3. DROP INDEX  CONCURRENTLY [old_index];
  4. DROP EXTENSION gin;
No need to write and debug the code which reads both old and new binary format. For sure, we need to support old GIN extension for some time. But, we have to support old in-core versions too. 

Unfortunately, I didn't mention this in the first post because I consider this as a serious argument for extensible AMs.

Also, I'm not sure that many users have enough of courage to use unlogged AMs. Absence of durability is only half of trouble, another half is lack of streaming replication. I think if we have unlogged GIN then external indexing engines would be used by majority of users instead of GIN.

------
With best regards,
Alexander Korotkov. 

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Jim Nasby
Дата:
Сообщение: Re: WIP: Access method extendability
Следующее
От: Kevin Grittner
Дата:
Сообщение: Re: Directory/File Access Permissions for COPY and Generic File Access Functions