Обсуждение: [GENERAL] Call for users to talk about table partitioning

Поиск
Список
Период
Сортировка

[GENERAL] Call for users to talk about table partitioning

От
Shirley Wang
Дата:
Hello!

We're a team from Pivotal, working with members of the Postgres community on table partitioning in pgAdmin4. We're looking to chat with some Postgres users on their expectations with table partitioning within pgAdmin4. 

If you have some availability next week, we'd love to set up some time to learn about your needs with this feature. Let us know some times that work for you and we'll send over details for the call.

Thanks!
Shirley

Re: [GENERAL] Call for users to talk about table partitioning

От
Melvin Davidson
Дата:
Shirley,
I am a bit confused. AFAIK, it is PostgreSQL that is responsible for table partitioning. PgAdmin4 is just an administrative tool.
Are you saying that PgAdmin4 now can make partition tables automatically?

On Thu, May 18, 2017 at 2:10 PM, Shirley Wang <swang@pivotal.io> wrote:
Hello!

We're a team from Pivotal, working with members of the Postgres community on table partitioning in pgAdmin4. We're looking to chat with some Postgres users on their expectations with table partitioning within pgAdmin4. 

If you have some availability next week, we'd love to set up some time to learn about your needs with this feature. Let us know some times that work for you and we'll send over details for the call.

Thanks!
Shirley



--
Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.

Re: [GENERAL] Call for users to talk about table partitioning

От
Robert Eckhardt
Дата:
All the code for creating and managing partitions is part of the core Postgres code. What we are interested in looking into is what that work flow might look like and how that workflow can be supported with a GUI management tool. 

-- Rob

On Thu, May 18, 2017 at 3:21 PM, Melvin Davidson <melvin6925@gmail.com> wrote:
Shirley,
I am a bit confused. AFAIK, it is PostgreSQL that is responsible for table partitioning. PgAdmin4 is just an administrative tool.
Are you saying that PgAdmin4 now can make partition tables automatically?

On Thu, May 18, 2017 at 2:10 PM, Shirley Wang <swang@pivotal.io> wrote:
Hello!

We're a team from Pivotal, working with members of the Postgres community on table partitioning in pgAdmin4. We're looking to chat with some Postgres users on their expectations with table partitioning within pgAdmin4. 

If you have some availability next week, we'd love to set up some time to learn about your needs with this feature. Let us know some times that work for you and we'll send over details for the call.

Thanks!
Shirley



--
Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.


Re: [GENERAL] Call for users to talk about table partitioning

От
Scott Marlowe
Дата:
On Thu, May 18, 2017 at 1:21 PM, Melvin Davidson <melvin6925@gmail.com> wrote:
Shirley,
I am a bit confused. AFAIK, it is PostgreSQL that is responsible for table partitioning. PgAdmin4 is just an administrative tool.
Are you saying that PgAdmin4 now can make partition tables automatically?

I think maybe she just means that their GUI is pgadmin4 for reference as opposed to the partitioning being IN pgadmin4.

If she'd like to set up a discussion thread HERE on partitioning I'm more than willing to add to it.

More importantly, while I think the postgresql documentation on partitioning gives you everything you need to roll your own, it doesn't inform you on all the ways to do partitioning for various jobs. That's where partitioning gets interesting and requires business requirements and all that fun stuff.

Do you hand off maintenance jobs to cron or do you check and run it every time a function's called etc.

Do you put materialized views on top to stop scanning all the partitions ever?

fun fun.

Re: [GENERAL] Call for users to talk about table partitioning

От
John R Pierce
Дата:
On 5/18/2017 2:29 PM, Robert Eckhardt wrote:
> All the code for creating and managing partitions is part of the core
> Postgres code. What we are interested in looking into is what that
> work flow might look like and how that workflow can be supported with
> a GUI management tool.


only thing I'd expect from a GUI management tool would be to allow me to
create partitioned tables and its partitions, and display their
attributes appropriately.



--
john r pierce, recycling bits in santa cruz



Re: [GENERAL] Call for users to talk about table partitioning

От
Scott Marlowe
Дата:
I would say that the best thing to do is to run 9.6 grab pgadmin4 and do all the examples in the doc page on partitioning.

https://www.postgresql.org/docs/9.6/static/ddl-partitioning.html

If that works well then the question becomes are there any esoteric cases where pgadmin4 won't quite get you there?

Re: [GENERAL] Call for users to talk about table partitioning

От
Scott Marlowe
Дата:
On Thu, May 18, 2017 at 3:40 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:
> I would say that the best thing to do is to run 9.6 grab pgadmin4 and do all
> the examples in the doc page on partitioning.
>
> https://www.postgresql.org/docs/9.6/static/ddl-partitioning.html
>
> If that works well then the question becomes are there any esoteric cases
> where pgadmin4 won't quite get you there?

Or maybe what can we automate from pgadmin4 that you currently need to
script etc?


Re: [GENERAL] Call for users to talk about table partitioning

От
Melvin Davidson
Дата:

On Thu, May 18, 2017 at 5:41 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:
On Thu, May 18, 2017 at 3:40 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:
> I would say that the best thing to do is to run 9.6 grab pgadmin4 and do all
> the examples in the doc page on partitioning.
>
> https://www.postgresql.org/docs/9.6/static/ddl-partitioning.html
>
> If that works well then the question becomes are there any esoteric cases
> where pgadmin4 won't quite get you there?

Or maybe what can we automate from pgadmin4 that you currently need to
script etc?

FWIW, It's not much of a problem creating partitioned tables. You simply follow the
documentation in https://www.postgresql.org/docs/9.6/static/ddl-partitioning.html

The problem occurs when you are already in production and have data in the master
table. In that case, you need a function (or script) that reads records from the old master
and  inserts them  into the appropriate child tables. Verification of success is needed,
and then the old master table needs to be truncated. This involves a timeout period in
production. One technique which minimizes that, which I personally have done, is to create
a new master that is empty, and then create all children from that. When inserts are complete
and verified (from old master), then the old master is renamed and the new master is renamed in
place of the old master. That means the only downtime is during the renaming, which
is minimal. In the event a problem occurs, you can easily reverse the renames, and since
the old master will still  contain all original records, the risk is minimal.

--
Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.