Back to our series about pgloader. The previous articles detailed How To Use PgLoader then How to Setup pgloader, then what to expect from a parallel pgloader setup. This article will detail how to reformat input columns so that what PostgreSQL sees is not what’s in the data file, but the result of a transformation from this data into something acceptable as an input for the target data type.
Tsung is an open-source multi-protocol distributed load testing tool and a mature project. It’s been available for about 10 years and is built with the Erlang system. It supports several protocols, including the PostgreSQL one.
When you want to benchmark your own application, to know how many more clients it can handle or how much gain you will see with some new shiny hardware, Tsung is the tool to use. It will allow you to record a number of sessions then replay them at high scale.
This article continues the series that began with How To Use PgLoader then detailed How to Setup pgloader. We have some more fine points to talk about here, today’s article is about loading your data in parallel with pgloader.
In a previous article we
detailed
how to use pgloader,
let’s now see how to write the pgloader.conf
that
instructs pgloader about what to
do.
When you do partition your tables monthly, then comes the question of when to create next partitions. I tend to create them just the week before next month and I have some nice nagios scripts to alert me in case I’ve forgotten to do so. How to check that by hand in the end of a month?