Re: Horizontal scalability/sharding

Поиск
Список
Период
Сортировка
От Tomas Vondra
Тема Re: Horizontal scalability/sharding
Дата
Msg-id 55E84DD8.6000703@2ndquadrant.com
обсуждение исходный текст
Ответ на Re: Horizontal scalability/sharding  (Amit Kapila <amit.kapila16@gmail.com>)
Ответы Re: Horizontal scalability/sharding  (Bruce Momjian <bruce@momjian.us>)
Список pgsql-hackers
Hi,

On 09/03/2015 05:02 AM, Amit Kapila wrote:
> On Thu, Sep 3, 2015 at 8:28 AM, Bruce Momjian <bruce@momjian.us
> <mailto:bruce@momjian.us>> wrote:
>  >
>  > On Wed, Sep  2, 2015 at 07:50:25PM -0700, Joshua Drake wrote:
>  > > >Can you explain why logical replication is better than binary
>  > > >replication for this use-case?
>  > > >
>  > >
>  > > Selectivity?
>  >
>> I was assuming you would just create identical slaves to handle
>> failure, rather than moving selected data around.
>  >
>
> Yes, I also think so, otherwise when the shard goes down and it's
> replica has to take the place of shard, it will take more time to
> make replica available as it won't have all the data as original
> shard had.

Not really, the idea is that you don't need to create the replica 
immediately. The system recognizes that primary shard location is 
unavailable and redirects the tasks to the "replicas." So the time to 
recreate the failed node is not that critical.

It needs to be done in a smart way to prevent some typical issues like 
suddenly doubling the load on replicas due to failure of the primary 
location. By using different group of nodes for each "data segment" you 
can eliminate this, because the group of nodes to handle the additional 
load will be larger.

The other issue then of course is that the groups of nodes must not be 
entirely random, otherwise the cluster would suffer data loss in case of 
outage of arbitrary group of K nodes (where K is the number of replicas 
for each piece of data).

It's also non-trivial to do this when you have to consider racks, data 
centers etc.

With regular slaves you can't do any of this - no matter what you do, 
you have to load balance the additional load only on the slaves.

regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



В списке pgsql-hackers по дате отправления:

Предыдущее
От: Anastasia Lubennikova
Дата:
Сообщение: Re: [PATCH] Microvacuum for gist.
Следующее
От: Michael Paquier
Дата:
Сообщение: Re: Fwd: Core dump with nested CREATE TEMP TABLE