Jerry Sievers jerry at jerrysievers.com
Tue Jul 31 15:54:39 PDT 2007
Christopher Browne <cbbrowne at ca.afilias.info> writes:

> Bill Moran wrote:
> 
> > In response to Jerry Sievers <jerry at jerrysievers.com>:
> >
> >
> >> Bill Moran <wmoran at collaborativefusion.com> writes:
> >>
> >>
> >>> In response to Jerry Sievers <jerry at jerrysievers.com>:
> >>>
> >>>
> >>>> Slony 1.1.5
> >>>>
> >>>> Assuming a cascaded cluster of 3 nodes arranged as follows;
> >>>>
> >>>> Master
> >>>>         Slave1
> >>>>                 Slave2
> >>>>
> >>>> If the middle tier Slave1 is totally lost, is there a way to get
> >>>> Slave2 pointing to Master without data loss, or at all?
> >>>>
> >>>> I am unable to do it with Slonik but had thought this was
> >>>> workable.
> >>> What, exactly, did you try to do?
> >>>
> >>> You should be able to do it by simply issuing a new subscribe set ()
> >>> command with the correct parameters to subscribe it to Master.
> >>>
> >> That was my assumption and I guess also believe that Slonik would feed
> >> the new subscription info directly to the now cut=-off slave. .  This
> >> does not seem to be the case.
> >>
> >> original subscription looked like set 1,
> >> provider 2,
> >> receiver 3
> >>
> >> New subscription;
> >>
> >> set 1,
> >> provider 1, receiver 3
> >>
> >> But with the node that was in the middle feeding slave 2 down, I think
> >> there's no way slave 2 ever hears of the new subscription and it keeps
> >> trying to connect to the slave that used to be feeding it.
> >>
> >> I've queried the sl_* tables on the  node that I want to reposition
> >> and see that nothing is  changing.
> >>
> >> Any ideas?
> >>
> >
> > This may come across harsh.  I hope not, but I fear that there's no way
> > to describe this without sounding harsh.
> >
> > If you tell us _what_you_did_ instead of _describing_what_you_think_
> > you_did_, it's much more likely that people on the list will be able
> > to help.
> >
> > This is a common problem.  Lots of people say, "I entered the correct
> > commands", but refuse (for some unknown reason) to simply cut/paste
> > the commands into their email.
> >
> > Unfortunately, this strategy comes up against the fact that 99% of the
> > problems that people have are operator error, and usually boil down to
> > a very specific misuse of a command.  Without knowing exactly what those
> > commands are, we'll never find the problem.
> >
> >
> Well put.
> > That being said, I'm not sure that this is _supposed_ to work.  It' may
> > be necessary to "DROP NODE" of node 2 in order for Slony to understand
> > that it's no longer participating.  Haven't tried this personally, but
> > Slony should automatically reshape paths when node 2 is officially removed.
> >
> > Of course, someone who knows better may correct me on this.
> >
> Dropping out node #2 will actually NOT work, because node #2 is still
> regarded as data provider for node #3 in some places.
> 
> I would expect it to be useful to submit STORE PATH entries for the
> paths that are still functioning, as that would "bump" the remaining
> nodes, telling them all to regenerate listen path (sl_listen) entries.
> 
> I think the first thing I'd try to do is to drop whatever PATH
> references to node #2 that I could, and add all the valid ones that I
> can.
> 
> Here are all the possible ones:
> 
>   DROP PATH ( SERVER = 1, CLIENT = 2 );
>   DROP PATH ( SERVER = 2, CLIENT = 1 );
>   DROP PATH ( SERVER = 3, CLIENT = 2 );
>   DROP PATH ( SERVER = 2, CLIENT = 3 );
> 
> By default, those will be injected at node #1; submitting them to node
> #3 would be a useful alternative:
> 
>   DROP PATH ( SERVER = 1, CLIENT = 2, EVENT NODE = 3 );
>   DROP PATH ( SERVER = 2, CLIENT = 1, EVENT NODE = 3 );
>   DROP PATH ( SERVER = 3, CLIENT = 2, EVENT NODE = 3 );
>   DROP PATH ( SERVER = 2, CLIENT = 3, EVENT NODE = 3 );
> 
> I expect that some of those would fail due to appearing to cut off
> communications; all the same, the more of them that *do* go away, the
> likelier that useful communications will be re-established between
> nodes 1 and 3.
> 
> Supposing I have a preamble (e.g. - CLUSTER NAME and ADMIN CONNINFO
> statements) in the file /tmp/preamble.slonik, then the following shell
> script could do the trick:
> 
> #!/bin/sh
> PREAMBLE=`cat /tmp/preamble.slonik`
> for eventnode in 1 3; do
>     for server in 1 2 3; do
>     for client in 1 2 3; do
>         echo "${PREAMBLE}" > /tmp/slonik-script.$$
>         echo "drop path (server = ${server}, client = ${client}, event
> node = ${eventnode});" | grep 2 >> /tmp/slonik-script.$$
>         slonik /tmp/slonik-script.$$
>      done
>      done
> done
> 
> That'll remove all the paths it can, initially, which will trigger
> rebuilding of listen paths.
> 
> Submit STORE PATH requests for all the non-related-to-node-2 paths and
> that may also do good.
> 
> Restarting the slons may also help.
> 
> This is all a bit messy, but the situation is, if anything, that...

Chris, this is very helpful.  Thanks!> 

Basically, I misunderstood the documentation and believe that what I
was trying was supported and straightforward to do.  You have affirmed
that it is not. 

We're still using Slony 1.1.5 on some large and very busy production
systems due to running EnterpriseDB 8.1 which came bundled with Slony
1.1.5.  Their 8.2 version that we're testing has Slony 1.2.6.

I have one related question here;

I find that even with all nodes running, that on 1.1.5, a new
subscription to do only provider change, does *not* affect the
sl_listen table entries on slaves.  Only the master will be changed to
what should be correct.

In the example of the 3 node cluster being discussed here...

Moving  Slave2 to have Master as provider does indeed update
sl_subscribe tables on all nodes, however the only sl_listen table
then showing   a listen path as;

origin 1, provider 1, receiver 3

...Is Master system.  Both slave nodes continue to have the related
sl_listen record looking as follows.

origin 1, provider 2, receiver 3

Appologies if this is a known defficiency in 1.1.5.

-- 
-------------------------------------------------------------------------------
Jerry Sievers   732 365-2844 (work)     Production Database Administrator
                305 321-1144 (mobil	WWW E-Commerce Consultant


More information about the Slony1-general mailing list