Yes, you read it well, el-get currently features 174 recipes, and is now reaching the 1.1 release. The reason for this release is mainly that I have two big chunks of code to review and the current code has been very stable for awhile. It seems better to do a release with the stable code that exists now before to shake it this much. If you're wondering when to jump in the water and switch to using el-get, now is a pretty good time.

 

Dynamic Triggers in PLpgSQL

November, 24 2010

You certainly know that implementing dynamic triggers in PLpgSQL is impossible. But I had a very bad night, being up from as soon as 3:30 am today, so that when a developer asked me about reusing the same trigger function code from more than one table and for a dynamic column name, I didn't remember about it being impossible.

 

pg_basebackup

November, 07 2010

Hannu just gave me a good idea in this email on -hackers, proposing that pg_basebackup should get the xlog files again and again in a loop for the whole duration of the base backup. That's now done in the aforementioned tool, whose options got a little more useful now:

 

Introducing Extensions

October, 21 2010

After reading Simon's blog post, I can't help but try to give some details about what it is exactly that I'm working on. As he said, there are several aspects to extensions in PostgreSQL, it all begins here: Chapter 35. Extending SQL.

 

These days, thanks to my community oriented job, I'm working full time on a PostgreSQL patch to terminate basic support for extending SQL. First thing I want to share is that patching the backend code is not as hard as one would think. Second one is that git really is helping.

 

Date puzzle for starters

October, 08 2010

The PostgreSQL IRC channel is a good place to be, for all the very good help you can get there, because people are always wanting to remain helpful, because of the off-topics discussions sometime, or to get to talk with community core members. And to start up your day too.

 

Yeah I'm back on working on my part of the extension thing in PostgreSQL.

 

el-get reaches 1.0

October, 07 2010

It's been a week since the last commits in the el-get repository, and those were all about fixing and adding recipes, and about notifications. Nothing like core plumbing you see. Also, 0.9 was released on 2010-08-24 and felt pretty complete already, then received lots of improvements. It's high time to cross the line and call it 1.0!

 

The major reason why I dislike perl so much, and ruby too, and the thing I'd want different in the Emacs Lisp API so far is how they set developers mind into using regexp. You know the quote, don't you?

 

The previous article about M-x mailq has raised several mails asking me details about the Postfix setup I'm talking about. The problem we're trying to solve is having a local MTA to send mails, so that any old-style Unix tool just works, instead of only the MUA you've spent time setting up.

 

Scratch that itch: M-x mailq

September, 23 2010

Nowadays, most people would think that email is something simple, you just setup your preferred client (that's called a MUA) with some information such as the smtp host you want it to talk to (that's call a MTA and this one is your relayhost). Then there's all the receiving mails part, and that's smtp again on the server side. Then there's how to get those mail, read them, flag them, manage them, and that's better served by IMAP. Let's talk about sending mails in smtp for this entry.

 

switch-window reaches 0.8

September, 13 2010

I wanted to play with the idea of using the whole keyboard for my switch-window utility, but wondered how to get those keys in the right order and all. Finally found quail-keyboard-layout which seems to exists for such uses, as you can see:

 

The drawback of hosting a static only website is, obviously, the lack of comments. What happens actually, though, is that I receive very few comments by direct mail. As I don't get another spam source to cleanup, I'm left unconvinced that's such a drawback. I still miss the low probability of seeing blog readers exchange directly, but I think a tapoueh.org mailing list would be my answer, here...

 

Window Functions example

September, 09 2010

So, when 8.4 came out there was all those comments about how getting window functions was an awesome addition. Now, it seems that a lot of people seeking for help in #postgresql just don't know what kind of problem this feature helps solving. I've already been using them in some cases here in this blog, for getting some nice overview about Partitioning: relation size per “group”.

 

Synchronous Replication

September, 06 2010

Although the new asynchronous replication facility that ships with 9.0 ain't released to the wide public yet, our hackers hero are already working on the synchronous version of it. A part of the facility is rather easy to design, we want something comparable to DRBD flexibility, but specific to our database world. So synchronous would either mean recv, fsync or apply, depending on what you need the standby to have already done when the master acknowledges the COMMIT. Let's call that the service level.

 

Yes, that's another el-get related entry. It seems to take a lot of my attention these days. After having setup the git repository so that you can update el-get from within itself (so that it's self-contained), the next logical step is providing recipes.

 

Happy Numbers

August, 30 2010

After discovering the excellent Gwene service, which allows you to subscribe to newsgroups to read RSS content ( blogs, planets, commits, etc), I came to read this nice article about Happy Numbers. That's a little problem that fits well an interview style question, so I first solved it yesterday evening in Emacs Lisp as that's the language I use the most those days.

 

A very good remark from some users: installing and managing el-get should be simpler. They wanted both an easy install of the thing, and a way to be able to manage it afterwards (like, update the local copy against the authoritative source). So I decided it was high time for getting the code out of my ~/.emacs.d git repository and up to a public place: http://github.com/dimitri/el-get.

 

Playing with bit strings

August, 26 2010

The idea of the day ain't directly from me, I'm just helping with a very thin subpart of the problem. The problem, I can't say much about, let's just assume you want to reduce the storage of MD5 in your database, so you want to abuse bit strings. A solution to use them works fine, but the datatype is still missing some facilities, for example going from and to hexadecimal representation in text.

 

el-get news

August, 26 2010

I've been receiving some requests for el-get, some of them even included a patch. So now there's support for bzr, CSV and http-tar, augmenting the existing support for git, git-svn, apt-get, fink and ELPA formats.

 

Thanks to you readers of Planet Emacsen taking the time to try those pieces of emacs lisp found in my blog, and also the time to comment on them, some bugs have been fixed, and new releases appeared.

 

We're using constants in some constraints here, for example in cases where several servers are replicating to the same federating one: each origin server has his own schema, and all is replicated nicely on the central host, thanks to Londiste, as you might have guessed already.

 

In trying to help an extension debian packaging effort, I've once again proposed to handle it. That's because I now begin to know how to do it, as you can see in my package overview page at debian QA facility. There's a reason why I proposed myself here, it's that yet another tool of mine is now to be found in debian, and should greatly help extension packaging there. You can already check for the postgresql-server-dev-all package page if you're that impatient!

 

Some user on IRC was reading the releases notes in order to plan for a minor upgrade of his 8.3.3 installation, and was puzzled about potential needs for rebuilding GIST indexes. That's from the 8.3.5 release notes, and from the 8.3.8 notes you see that you need to consider hash indexes on interval columns too. Now the question is, how to find out if any such beasts are in use in your database?

 

el-get

August, 04 2010

I've been using emacs for a long time, and a long time it took me to consider learning Emacs Lisp. Before that, I didn't trust my level of understanding enough to be comfortable in managing my setup efficiently.

 

Today I'm being told once again about SQLite as an embedded database software. That one ain't a database server but a software library that you can use straight into your main program. I'm yet to use it, but it looks like its SQL support is good enough for simple things — and that covers loads of things. I guess read-only cache and configuration storage would be the obvious ones, because it seems that SQLite use cases aren't including mixed concurrency, that is workloads with concurrent readers and writers.

 

This time, we are trying to figure out where is the bulk of the data on disk. The trick is that we're using DDL partitioning, but we want a “nice” view of size per partition set. Meaning that if you have for example a parent table foo with partitions foo_201006 and foo_201007, you would want to see a single category foo containing the accumulated size of all the partitions underneath foo.

 

Thanks to amazing readers of planet emacsen, two annoyances of switch-window.el have already been fixed! The first is that handling of C-g isn't exactly an option after all, and the other is that you want to avoid the buffer creation in the simple cases (1 or 2 windows only), because it's the usual case.

 

dim-switch-window.el

July, 25 2010

So it's Sunday and I'm thinking I'll get into el-get sometime later. Now is the time to present dim-switch-window.el which implements a visual C-x o. I know of only one way to present a visual effect, and that's with a screenshot:

 

If you don't know about ClusterSSH, it's a project that builds on M-x term and ssh to offer a nice and simple way to open remote terminals. It's available in ELPA and developed at github cssh repository.

 

Emacs and PostgreSQL

July, 22 2010

Those are my two all times favorite Open Source Software. Or Free Software in the GNU sense of the world, as both the BSD and the GPL are labeled free there. Even if I prefer the The Debian Free Software Guidelines as a global definition and the WTFPL license. But that's a digression.

 

Background writers

July, 19 2010

There's currently a thread on hackers about bg worker: overview and a series of 6 patches. Thanks a lot Markus! This is all about generalizing a concept already in use in the autovacuum process, where you have an independent subsystem that require having an autonomous daemon running and able to start its own workers.

 

Logs analysis

July, 13 2010

Nowadays to analyze logs and provide insights, the more common tool to use is pgfouine, which does an excellent job. But there has been some improvements in logs capabilities that we're not benefiting from yet, and I'm thinking about the CSV log format.

 

There's a big trend nowadays about using column storage as opposed to what PostgreSQL is doing, which would be row storage. The difference is that if you have the same column value in a lot of rows, you could get to a point where you have this value only once in the underlying storage file. That means high compression. Then you tweak the executor to be able to load this value only once, not once per row, and you win another huge source of data traffic (often enough, from disk).

 

MVCC in the Cloud

July, 06 2010

At CHAR(10) Markus had a talk about Using MVCC for Clustered Database Systems and explained how Postgres-R does it. The scope of his project is to maintain a set of database servers in the same state, eventually.

 

Back from CHAR(10)

July, 05 2010

It surely does not feel like a full month and some more went by since we were enjoying PGCon 2010, but in fact it was already the time for CHAR(10). The venue was most excellent, as Oxford is a very beautiful city. Also, the college was like a city in the city, and having the accomodation all in there really smoothed it all.

 

Back from PgCon2010

May, 27 2010

This year's edition has been the best pgcon ever for me. Granted, it's only my third time, but still :) As Josh said the "Hall Track" in particular was very good, and the Dev Meeting has been very effective!

 

So, following previous blog entries about importing fixed width data, from Postgres Online Journal and David (perl) Fetter, I couldn't resist following the meme and showing how to achieve the same thing with pgloader.

 

Yes. This pgloader project is still maintained and somewhat active. Development happens when I receive a complaint, either about a bug in existing code or a feature in yet-to-write code. If you have a bug to report, just send me an email!

 

This time we're having a database where sequences were used, but not systematically as a default value of a given column. It's mainly an historic bad idea, but you know the usual excuse with bad ideas and bad code: the first 6 months it's experimental, after that it's historic.

 

Emacs Muse hacking

March, 04 2010

Now you know what piece of software is used to publish this blog. I really like it, the major mode makes it a great experience to be using this tool, and the fact that you produce the HTML and rsync it all from within Emacs ( C-c C-p then C-c C-r with some easy elisp code) is a big advantage as far as I'm concerned. No need to resort to shell and Makefile.

 

So, if you followed the previous blog entry, now you have a new database containing all the static tables encoded in UTF-8 rather than SQL_ASCII. Because if it was not yet the case, you now severely distrust this non-encoding.

 

It happens that you have to manage databases designed by your predecessor, and it even happens that the team used to not have a DBA. Those histerical raisins can lead to having a SQL_ASCII database. The horror!

 

So, after restoring a production dump with intermediate filtering, none of our sequences were set to the right value. I could have tried to review the process of filtering the dump here, but it's a one-shot action and you know what that sometimes mean. With some pressure you don't script enough of it and you just crawl more and more.