On 11-02-03 11:15 AM, Jan Wieck wrote:

>> subscribe set(set id=1,provider=1,receiver=2)
>> subscribe set(set id=2,provider=2,receiver=1)
>> wait for event(origin=1,confirmed=2,wait on=1)
>> wait for event(origin=2,confirmed=1,wait on=2)
>> subscribe set(set id=1,origin=1,receiver=3)
>> subscribe set(set id=2,origin=2,receiver=3)
>> #
>> # subscribing to set 3 takes a LONG time
>> # because it is in a remote data centre
>> #
>> # while it is subscribing I discover
>> # I need to make an emergency schema change
>> # via EXECUTE SCRIPT such that I can't wait
>> # for node 3 to finish subscribing before
>> # making the change on node 1 and 2.
>>
>> If i use node 1 or node 2 as the event node it might get applied on node
>> 3 before the set from the other node finishes.
>
> Again, if using the event node where the affected objects originate,
> there will be no conflict and the order in which the things are applied
> doesn't matter.

the node which objects originate on? I have two sets.

My EXECUTE SCRIPT might be selecting from tables in both sets and 
inserting into a local non-replicated table.  I can ensure consistency 
by taking an application outage and making sure nodes 1+2 are up to date 
with other.  This still doesn't ensure that node 3 applies it in the 
right order.
>>
>> create set(id=1, origin=1)
>> set add table(set id=1, origin=1, fully qualified table='public.foo');
>> #commands execute, dba notices a mistake
>> drop set(set id=1,event node=1);
>> wait for event(origin=1,confirmed=3,wait on=1);
>> create set(id=2, origin=2)
>> set add table(set id=2,origin=3);
>
> I don't think this should be possible because set add table makes only
> sense on the set origin.

I hadn't realized that set add table only does something on the 
subscribe set.

I then modify my example to:

create set(id=1, origin=1)
set add table(set id=1, origin=1, fully qualified table='public.foo');
subscribe set(set id=1, provider=1,receiver=2);
# I would wait for the subscription to finish
# but it doesn't fix anything, it just makes the
# race condition less likely
drop set(set id=1,event node=1);
wait for event(origin=1,confirmed=3,wait on=1);
create set(id=2, origin=3)
set add table(set id=2,origin=3);
subscribe set(set id=2, provider=3,receiver=2)

The subscribe set for set id=2 might be processed on node 3 before the 
drop set for set 1 is processed from node 1.

Also consider the case where the second 'create set' is recreating a set 
with id=1 but using node 3 as the origin.


I have added a disorder based test case to my github repo at 
https://github.com/ssinger/slony1-engine/commit/dbed2a39062975366b1a2e7294a80a4e1b3f388b 
that demonstrates this.

The slon on node 3 will report:
  ESTFATAL  storeSubscribe: set 2 not found

and loop for ever.







>
> Jan
>