Well I certainly look forward to hearing more about log shipping, and
paddling around with it when some building code is available.

In the meantime, given that I have fairly short time-frame, I guess I'm
back to my original question about these listeners:

If I have a MASTER and subscribers A and B, and I know that neither A
nor B will ever become the set origin, do A and B strictly need to
listen to each other? I see in the code where slon at different points
is waiting for syncs from other nodes, but I haven't figured out yet
what bad thing happens if no listen record exists for a given node pair.

What I would like would be to just have listen relationships just
between each edge node and the MASTER, so A<->MASTER and B<->MASTER, but
no A<->B.

I am going to go ahead and try this, and look at the code some more, but
if somebody has an idea exactly where in the code would blow up in this
scenario, that would help to focus my search.

Thanks!

- DAP

>-----Original Message-----
>From: Christopher Browne [mailto:cbbrowne at ca.afilias.info] 
>Sent: Wednesday, January 26, 2005 2:50 PM
>To: David Parker
>Cc: slony1-general at gborg.postgresql.org
>Subject: Re: [Slony1-general] quadratic listener count growth
>
>David Parker wrote:
>
>>>One of the "high priority items" (fairly urgent, and considered very 
>>>important, at least here at Afilias) is the implementation of Log 
>>>Shipping, which ought to be an answer to your problem.
>>>
>>>The idea of Log Shipping is that you can have a node that, 
>instead of 
>>>syncing into a database, generates a series of files containing the 
>>>SQL queries to do the updates.  That series of files may then be 
>>>distributed to remote sites, and applied into a database, 
>giving you a 
>>>replica that is as up to date as the series of files.
>>>
>>>One such "log shipping" node can feed as many replicas as 
>you want it 
>>>to.  There is no means of feeding anything back to the 
>origin, but it 
>>>can certainly scale.
>>>
>>>So if what you want is to have (say) four reliable boxes at 
>a central 
>>>site that talk to one another, each of which could take over in a 
>>>pinch, and then have a whole bunch of copies at different sites, log 
>>>shipping would allow that to work.
>>>
>>>It's not implemented yet, but our urgency level on it is 
>rather high, 
>>>so there oughta be something at least beta-like in the not 
>too distant 
>>>future.
>>>    
>>>
>>
>>Thanks! That does sound promising. Is there a spec anywhere that I 
>>could take a look at? Some questions that occur to me initially:
>>
>>1) Is there still a subscription mechanism, for instance allowing the 
>>specification of a "type" of subscription?
>>
>>  
>>
>Yes.  Jan and I have both done some "fighting" with the 
>problem and independently discovered some of the same bits of 
>trickiness to setting up subscriptions.
>
>>2) Does the slon daemon handle the "shipping"?
>>
>>  
>>
>It may be a special slon daemon, or log shipping code may be 
>added in; not yet sure.
>
>>3) How does a given subscriber node determine it's sync state 
>vis a vis 
>>the log shipper?
>>  
>>
>What appears to be the case (at this point; call it "preliminary
>analysis") is that setting up a subscription involves going to 
>some existing subscriber node (which implies you need at least 
>one subscriber already in order to start log shipping), and 
>grab a Slony-I-customized analogue to a pg_dump which contains 
>the sync status of that subscriber at the time the dump starts.
>
>Shipped logs are then "deltas" from that dump.
>
>>Has code for this gone into cvs HEAD for the project, or are 
>you folks 
>>just working on it locally? So many questions....
>>  
>>
>It's not yet at the point of putting something in CVS HEAD...
>
>>If it is in HEAD, I would be interested in taking on some 
>tasks, if you 
>>need help. Failing that, I would also be eager to help with testing, 
>>etc., since this is a feature I really need!
>>  
>>
>As soon as it's far enough along for it to make sense to try 
>to distribute tasks, I'll see about pestering you.  I'm sure 
>there will be room for extra hands.  Log shipping would solve 
>a number of problems that we've got, including some that have 
>been persistently thorny, so I'm pretty certain that I'm as 
>eager as you are.
>--
><http://linuxdatabases.info/info/slony.html>
>