> Christopher Browne wrote:
>> Marc Munro <marc at bloodnok.com> writes:
>>> It seems to me that extending slony to be able to do this would be
>>> possible, though perhaps not easy.  The bigger questions might be:
>>> - Is this desirable?
>>> - Will it open a huge can of worms?
>>> - Are there better ways of achieving the same ends?
>>
>> It's an interesting idea.  Seems nontrivial to me to accomplish
>> this...
>>
>> Another way to regard this would be say that you have some set of
>> partitions, where events internal to the partition are not exposed
>> outside of the partition.
>>
>> That would indeed admit a way to scale the number of nodes higher;
>> having (say) clusters of 3 nodes at each site, where the local events
>> only stay at the site, would mean that the present situation where
>> each event must be confirmed by *ALL* nodes would be alleviated.
>
> I'd say the problem is not so much that every node needs to confirm every
> event,
> but rather that all those confirmations must travel back to originating
> node.
> Couldn't slony be made smarter in this aread? I guess it would be
> sufficient,
> if a node remebered an event it generated only until all it's "neighbours"
> have
> acknowledged that event, not until every other node has ack'ed it.
> This sounds quite simple in theory, not taking the interesting race
> conditions
> into account that might occur if the cluster topology changes suddenly ;-)
>
> Or does it already work like that, and am I being stupid?

It is NOT sufficient to simply keep memory until neighbours have
acknowledged the event; that could lead to arbitrary amounts of data loss
in the case of a failover.

The trouble is that at failover time, subscriptions get reshaped
temporarily to "who's got the latest data?" and if you're forgetting
events too aggressively, the answer to that gets increasingly vague.

The thing I'm thinking about, here, is the fact that event propagation is
used to determine when you're allowed to purge data from
sl_log_1/sl_log_2.

It seems to me that there may be optimizations to allow event
confirmations to be handled a lot more efficiently.  For instance,
returning several at once seems like a plausible improvement.  But I don't
think it's good enough to keep them at the "neighbour" level...