On Thu, Jun 11, 2009 at 12:59 PM, Brett Henderson<brett@bretth.com> wrote:
> Greg Stark wrote:
>> Another option to consider would be including a boolean column
>> "dumped" defaulted to false. Then you could have a partial index on
>> the primary key or date "WHERE NOT dumped". Then when you dump you can
>> "SELECT FOR UPDATE * WHERE NOT dumped" and then when you're done
>> "UPDATE SET dumped = 't' ". Alternately you could use "UPDATE SET
>> dumped='t' WHERE NOT dumped RETURNING *" which is basically
>> equivalent.
>>
>
> I have a couple of hesitations with using this approach:
> 1. We can only run the replicator once.
> 2. We can only run a single replicator.
> 3. It requires write access to the db.
4. it requires a schema change to support this use case.
again, this isn't a massive issue. but for the time being we're at
least making an effort to not extend the schema beyond what the rails
code itself requires.
> 3 is also a major consideration, it makes everybody's life easier if we can
> avoid updates being made to the db by the replicator.
for safety's sake i think this makes a lot of sense.
> I hope I don't sound too negative. My gut also tells me that what we're
> doing is not the "right" solution and I've had fairly similar arguments with
> Matt already :-) But having spent some time playing with it I can't find
> any reason why it won't work, and from a performance point of view I suspect
> it will win ...
it seems right to me to use postgres' existing features to support
this. we've got pretty close to a working system without needing
schema changes, so it would be a shame if they turn out to be
necessary.
cheers,
matt