7 Articles tagged “backup”

If you don’t remember about what pg_staging is all about, it’s a central console from where to control all your PostgreSQL databases. Typically you use it to manage your development and pre-production setup, where developers ask you pretty often to install them some newer dump from the production, and you want that operation streamlined and easy. Usage The typical session would be something like this: pg_staging> databases foodb.dev foodb foodb_20100824 :5432 foodb_20100209 foodb_20100209 :5432 foodb_20100824 foodb_20100824 :5432 pgbouncer pgbouncer :6432 postgres postgres :5432 pg_staging> dbsizes foodb.

Hannu just gave me a good idea in this email on -hackers, proposing that pg_basebackup should get the xlog files again and again in a loop for the whole duration of the base backup. That’s now done in the aforementioned tool, whose options got a little more useful now: Usage: pg_basebackup.py [-v] [-f] [-j jobs] "dsn" dest Options: -h, --help show this help message and exit --version show version and quit -x, --pg_xlog backup the pg_xlog files -v, --verbose be verbose and about processing progress -d, --debug show debug information, including SQL queries -f, --force remove destination directory if it exists -j JOBS, --jobs=JOBS how many helper jobs to launch -D DELAY, --delay=DELAY pg_xlog subprocess loop delay, see -x -S, --slave auxilliary process --stdin get list of files to backup from stdin Yeah, as implementing the xlog idea required having some kind of parallelism, I built on it and the script now has a --jobs option for you to setup how many processes to launch in parallel, all fetching some base backup files in its own standard ( libpq) PostgreSQL connection, in compressed chunks of 8 MB (so that’s not 8 MB chunks sent over).

After reading Simon’s blog post, I can’t help but try to give some details about what it is exactly that I’m working on. As he said, there are several aspects to extensions in PostgreSQL, it all begins here: Chapter 35. Extending SQL. It’s possible, and mostly simple enough, to add your own code or behavior to PostgreSQL, so that it will use your code and your semantics while solving user queries.

These days, thanks to my community oriented job, I’m working full time on a PostgreSQL patch to terminate basic support for extending SQL. First thing I want to share is that patching the backend code is not as hard as one would think. Second one is that git really is helping. “Not as hard as one would think, are you kidding me?”, I hear some say. Well, that’s true. It’s C code in there, but with a very good layer of abstractions so that you’re not dealing with subtle problems that much.

This year’s edition has been the best pgcon ever for me. Granted, it’s only my third time, but still :) As Josh said the “Hall Track” in particular was very good, and the Dev Meeting has been very effective! Extensions This time I prepared some slides to present the extension design and I tried hard to make it so that we get to agree on a plan, even recognizing it’s not solving all of our problems from the get go.