Category “Skytools” — 11 articles

Hannu just gave me a good idea in this email on -hackers, proposing that pg_basebackup should get the xlog files again and again in a loop for the whole duration of the base backup. That’s now done in the aforementioned tool, whose options got a little more useful now: Usage: pg_basebackup.py [-v] [-f] [-j jobs] "dsn" dest Options: -h, --help show this help message and exit --version show version and quit -x, --pg_xlog backup the pg_xlog files -v, --verbose be verbose and about processing progress -d, --debug show debug information, including SQL queries -f, --force remove destination directory if it exists -j JOBS, --jobs=JOBS how many helper jobs to launch -D DELAY, --delay=DELAY pg_xlog subprocess loop delay, see -x -S, --slave auxilliary process --stdin get list of files to backup from stdin Yeah, as implementing the xlog idea required having some kind of parallelism, I built on it and the script now has a --jobs option for you to setup how many processes to launch in parallel, all fetching some base backup files in its own standard ( libpq) PostgreSQL connection, in compressed chunks of 8 MB (so that’s not 8 MB chunks sent over).

I can’t really compare PgCon 2009 with previous years versions, last time I enjoyed the event it was in 2006, in Toronto. But still I found the experience to be a great one, and I hope I’ll be there next year too! I’ve met a lot of known people in the community, some of them I already had the chance to run into at Toronto or Prato, but this was the first time I got to talk to many of them about interresting projects and ideas.

One of the difficulties in getting to understand and configure londiste reside in the relation between the ticker and the replication. This question was raised once more on IRC yesterday, so I made a new FAQ entry about it: How do this ticker thing relates to londiste?

In the page about Skytools I’ve encouraged people to ask some more questions in order for me to be able to try and answer them. That just happened, as usual on the #postgresql IRC, and the question is What does londiste lack that slony has?

So, you’re using londiste and the ticker has not been running all night long, due to some restart glitch in your procedures, and the on call admin didn’t notice the restart failure. If you blindly restart the replication daemon, it will load in memory all those events produced during the night, at once, because you now have only one tick where to put them all. The following query allows you to count how many events that represents, with the magic tick numbers coming from pgq.