Обсуждение: Postgresql Write cluster needed

Поиск
Список
Период
Сортировка

Postgresql Write cluster needed

От
fahad basheer
Дата:
Hi ALL,

I need to scale postgresql with write capabilities, please help me.

Regards
Fahad

Re: Postgresql Write cluster needed

От
Mario Splivalo
Дата:
Unless you shard (partition) your data, writes can't be scaled.

Mario

On November 21, 2014 12:29:38 PM CET, fahad basheer <fahad.a.basheer@gmail.com> wrote:
Hi ALL,

I need to scale postgresql with write capabilities, please help me.

Regards
Fahad

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Postgresql Write cluster needed

От
Sergei Sheinin
Дата:
It may be possible to write to text files then use COPY command for bulk inserts.

On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
Unless you shard (partition) your data, writes can't be scaled.

Mario


On November 21, 2014 12:29:38 PM CET, fahad basheer <fahad.a.basheer@gmail.com> wrote:
Hi ALL,

I need to scale postgresql with write capabilities, please help me.

Regards
Fahad

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Postgresql Write cluster needed

От
desmodemone
Дата:


2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:
It may be possible to write to text files then use COPY command for bulk inserts.

On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
Unless you shard (partition) your data, writes can't be scaled.

Mario


On November 21, 2014 12:29:38 PM CET, fahad basheer <fahad.a.basheer@gmail.com> wrote:
Hi ALL,

I need to scale postgresql with write capabilities, please help me.

Regards
Fahad

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.



Hi,
         I think we need more informations about this problem and how it's the enviroment (OLTP, dwh ? ) . For example if some tables not need to be full ACID you could use  unlogged table or  for example you could relax the durability of the commit , disabling synchronous commit  .
Regards

--
Matteo Durighetto

- - - - - - - - - - - - - - - - - - - - - - -

Italian PostgreSQL User Group
Italian Community for Geographic Free/Open-Source Software

Re: Postgresql Write cluster needed

От
Koichi Suzuki
Дата:
Write scalability for OLTP workload is available with Postgres-XC.
---
Koichi Suzuki


2014-11-21 21:15 GMT+09:00 desmodemone <desmodemone@gmail.com>:
>
>
> 2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:
>>
>> It may be possible to write to text files then use COPY command for bulk
>> inserts.
>>
>> On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
>>>
>>> Unless you shard (partition) your data, writes can't be scaled.
>>>
>>> Mario
>>>
>>>
>>> On November 21, 2014 12:29:38 PM CET, fahad basheer
>>> <fahad.a.basheer@gmail.com> wrote:
>>>>
>>>> Hi ALL,
>>>>
>>>> I need to scale postgresql with write capabilities, please help me.
>>>>
>>>> Regards
>>>> Fahad
>>>
>>>
>>> --
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>>
>
>
> Hi,
>          I think we need more informations about this problem and how it's
> the enviroment (OLTP, dwh ? ) . For example if some tables not need to be
> full ACID you could use  unlogged table or  for example you could relax the
> durability of the commit , disabling synchronous commit  .
>
> Regards
>
>
> --
> Matteo Durighetto
>
> - - - - - - - - - - - - - - - - - - - - - - -
>
> Italian PostgreSQL User Group
> Italian Community for Geographic Free/Open-Source Software


Re: Postgresql Write cluster needed

От
fahad basheer
Дата:
Hi All,

Thanks for the fast response

Database is used as OLTP database.

Frontend application used is  OpenERP.

I have tried postgresql xc,but its not supported with OpenERP.




On Mon, Nov 24, 2014 at 5:58 AM, Koichi Suzuki <koichi.dbms@gmail.com> wrote:
Write scalability for OLTP workload is available with Postgres-XC.
---
Koichi Suzuki


2014-11-21 21:15 GMT+09:00 desmodemone <desmodemone@gmail.com>:
>
>
> 2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:
>>
>> It may be possible to write to text files then use COPY command for bulk
>> inserts.
>>
>> On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
>>>
>>> Unless you shard (partition) your data, writes can't be scaled.
>>>
>>> Mario
>>>
>>>
>>> On November 21, 2014 12:29:38 PM CET, fahad basheer
>>> <fahad.a.basheer@gmail.com> wrote:
>>>>
>>>> Hi ALL,
>>>>
>>>> I need to scale postgresql with write capabilities, please help me.
>>>>
>>>> Regards
>>>> Fahad
>>>
>>>
>>> --
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>>
>
>
> Hi,
>          I think we need more informations about this problem and how it's
> the enviroment (OLTP, dwh ? ) . For example if some tables not need to be
> full ACID you could use  unlogged table or  for example you could relax the
> durability of the commit , disabling synchronous commit  .
>
> Regards
>
>
> --
> Matteo Durighetto
>
> - - - - - - - - - - - - - - - - - - - - - - -
>
> Italian PostgreSQL User Group
> Italian Community for Geographic Free/Open-Source Software


Regards,
Fahad

Re: Postgresql Write cluster needed

От
Mario Splivalo
Дата:
On 11/24/2014 10:30 AM, fahad basheer wrote:
> Hi All,
>
> Thanks for the fast response
>
> Database is used as OLTP database.
>
> Frontend application used is  OpenERP.
>
> I have tried postgresql xc,but its not supported with OpenERP.
>
>

I don't see why OpenERP wouldn't work with pg-xc, but I'm also not sure
if you would benefit from it as deciding on what to partition your data
on is not a simple task.

What is your main performance issue?

In theory, if you want to scale writes you need to scale your data. So,
you'd, for instance, put all the even IDs on server A and all the odd
IDs on server B. That way you double the writing throughput. But, when
querying your data you need to somehow 'merge' both partitions, the one
on server A and the one on server B.
This is, actually, where Postgres-XC comes in - it will allow you to
create partitions of your data in a way that is oblivious to the
application itself. But for OpenERP I'm not sure if you will gain
anything, and your setup will be quite complicated.

You either need better server (with better I/O performance - add more
spindles if you're on RAID10, or switch to ssd), or partition your data.
If you have multiple tenants in your OpenERP maybe you could move each
of them to their separate servers?

    Mario
--
Mario Splivalo
mario@splivalo.hr

"I can do it quick, I can do it cheap, I can do it well. Pick any two."


Re: Postgresql Write cluster needed

От
Koichi Suzuki
Дата:
I hope you've already visited some of PGXC sites for scaling.   If you
haven't, please try to visit
http://sourceforge.net/p/postgres-xc/xc-wiki/Presentation_Materials/
and other materials which describes how PGXC scales both read and
write.

Regards;
---
Koichi Suzuki


2014-11-24 19:21 GMT+09:00 Mario Splivalo <mario@splivalo.hr>:
> On 11/24/2014 10:30 AM, fahad basheer wrote:
>> Hi All,
>>
>> Thanks for the fast response
>>
>> Database is used as OLTP database.
>>
>> Frontend application used is  OpenERP.
>>
>> I have tried postgresql xc,but its not supported with OpenERP.
>>
>>
>
> I don't see why OpenERP wouldn't work with pg-xc, but I'm also not sure
> if you would benefit from it as deciding on what to partition your data
> on is not a simple task.
>
> What is your main performance issue?
>
> In theory, if you want to scale writes you need to scale your data. So,
> you'd, for instance, put all the even IDs on server A and all the odd
> IDs on server B. That way you double the writing throughput. But, when
> querying your data you need to somehow 'merge' both partitions, the one
> on server A and the one on server B.
> This is, actually, where Postgres-XC comes in - it will allow you to
> create partitions of your data in a way that is oblivious to the
> application itself. But for OpenERP I'm not sure if you will gain
> anything, and your setup will be quite complicated.
>
> You either need better server (with better I/O performance - add more
> spindles if you're on RAID10, or switch to ssd), or partition your data.
> If you have multiple tenants in your OpenERP maybe you could move each
> of them to their separate servers?
>
>         Mario
> --
> Mario Splivalo
> mario@splivalo.hr
>
> "I can do it quick, I can do it cheap, I can do it well. Pick any two."
>
>
> --
> Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-cluster-hackers