module-postgresql/manifests/validate_db_connection.pp
Ken Barber 5df36cf1f7 (GH-198) Fix race condition on postgresql startup
This patch is a fix for the race condition that keeps occuring during
postgresql setup. Its very rare on its own, but when you are using this
module in a CI environment it happens quite frequently.

Basically what happens is that sometimes the service will announce the
database has started, but really it is still working in the background.
Sometimes the unix socket may not be listening, and sometimes the
system is still loading and you get a weird client error.

The fix itself is a modification to postgresql::validate_db_connection
so that it is able to connect on the local unix socket, plus retry
until the database is available.

This new and improved validate_db_connection can then be put into the
build pipeline (in the service class in particular) to ensure the
database is started before continuing on with the remaining steps.

This in effect blocks the puppet module from continuing until the
postgresql database is fully started and able to receive connections
which is perfect.

Tests and documentation provided.

Signed-off-by: Ken Barber <ken@bob.sh>
2013-10-24 00:33:45 +01:00

74 lines
2.5 KiB
Puppet

# This type validates that a successful postgres connection can be established
# between the node on which this resource is run and a specified postgres
# instance (host/port/user/password/database name).
#
# See README.md for more details.
define postgresql::validate_db_connection(
$database_host = undef,
$database_name = 'postgres',
$database_password = undef,
$database_username = undef,
$database_port = undef,
$run_as = undef,
$sleep = 2,
$tries = 10,
$create_db_first = true
) {
require postgresql::client
$psql_path = $postgresql::params::psql_path
$cmd_init = "${psql_path} --tuples-only --quiet "
$cmd_host = $database_host ? {
default => "-h ${database_host} ",
undef => ""
}
$cmd_user = $database_username ? {
default => "-U ${database_username} ",
undef => ""
}
$cmd_port = $database_port ? {
default => "-p ${database_port} ",
undef => ""
}
$cmd_dbname = $database_name ? {
default => "--dbname ${database_name} ",
undef => ""
}
$env = $database_password ? {
default => "PGPASSWORD=${database_password}",
undef => undef,
}
$cmd = join([$cmd_init, $cmd_host, $cmd_user, $cmd_port, $cmd_dbname])
$validate_cmd = "/usr/local/bin/validate_postgresql_connection.sh ${sleep} ${tries} '${cmd}'"
# This is more of a safety valve, we add a little extra to compensate for the
# time it takes to run each psql command.
$timeout = (($sleep + 2) * $tries)
$exec_name = "validate postgres connection for ${database_host}/${database_name}"
exec { $exec_name:
command => "echo 'Unable to connect to defined database using: ${cmd}' && false",
unless => $validate_cmd,
cwd => '/tmp',
environment => $env,
logoutput => 'on_failure',
user => $run_as,
path => '/bin',
timeout => $timeout,
require => Package['postgresql-client'],
}
# This is a little bit of puppet magic. What we want to do here is make
# sure that if the validation and the database instance creation are being
# applied on the same machine, then the database resource is applied *before*
# the validation resource. Otherwise, the validation is guaranteed to fail
# on the first run.
#
# We accomplish this by using Puppet's resource collection syntax to search
# for the Database resource in our current catalog; if it exists, the
# appropriate relationship is created here.
if($create_db_first) {
Postgresql::Server::Database<|title == $database_name|> -> Exec[$exec_name]
}
}