I'd like to see this patch included ASAP -- the desirable default could
be manage_pg_ident_conf => true, but one could already manage this file
manually : we don't want to wipe it.
Please switch the default from false to true at the next major release
and write a line about this in the release notes.
This defined type helps create database schemas, and assign them to an
`owner`. It is closely modeled after Postgresql::Server::Tablespace.
It uses PostgreSQL's builtin IF NOT EXISTS to guarantee idempotency.
(>= 9.3, else it checks pg_namespace).
n.b.: This defined type *requires* that a `db` is passed. This is a
concious design decision, since we find it rather useless to create such
schemas in the default `postgres` database, and if *were* useful, one
can always "over-specify".
This addresses MODULES-1098.
This is likely to be a controversial change so I wanted to put some
explanation of our reasoning into the commit message. This gets
kind of complex so I'll start with the problem and then the reasoning.
Problem:
We rely heavily on the ability to uninstall and reinstall postgres
throughout our testing code, testing features like "can I move from the
distribution packages to the upstream packages through the module" and
over time we've learnt that the uninstall code simply doesn't work a lot
of the time. It leaves traces of postgres behind or fails to remove
certain packages on Ubuntu, and generally causes bits to be left on your
system that you didn't expect.
When we then reinstall things fail because it's not a true clean slate,
and this causes us enormous problems during test. We've spent weeks and
months working on these tests and they simply don't hold up well across
the full range of PE platforms.
Reasoning:
Due to all these problems we've decided to take a stance on uninstalling
in general. We feel that in 2014 it's completely reasonable and normal
to have a good provisioning pipeline combined with your configuration
management and the "correct" way to uninstall a fully installed service
like postgresql is to simply reprovision the server without it in the
first place. As a general rule this is how I personally like to work
and I think is a good practice.
WAIT A MINUTE:
We understand that there are environments and situations in which it's
not easy to do that. What if you accidently deployed Postgres on
100,000 nodes? When this work is finished I'm going to take a look at
building some example 'profiles' to be found under examples/ within this
module that can uninstall postgres on popular platforms. These can be
modified and used in your specific case to uninstall postgresql. They
will be much more brute force and reliant on deleting entire directories
and require you to do more work up front in specifying where things are
installed but we think it'll prove to be a much cleaner mechanism for
this kind of thing rather than trying to weave it into the main module
logic itself.
This doesn't fix the root cause of the issue, such as the fact that
dpkg can't do wildcard removals, and the uninstall fails when you're
passing in a version number like this, but THIS test doesn't care, it
just wants to make sure the deprecation warning appears in the first
place.
This does however make the tests pass on 14.04.
The validate_db_connection class takes a user to connect as, but if we're
using the progresql::server::db defined type then the user might not be
created yet, and might not have any permissions granted yet. This patch
users a collector to ensure that the that the user and grants are active
before validating.
We now test if service_ensure is 'running' or 'stopped' but it was
actually picking up the default value of ensure in params.pp which
was true, not present.
Fix this and thereby fix the failing test.
Adjusting the version is explicitly done though the postgresql::globals
class, as this affects many parts of the module. This parameter did not
function correctly on systems that did not have a default, as described
in the ticket.
On FreeBSD systems the $user variable is not 'postgres' so does not
match the default database correctly. These changes use the existing
default_database parameter to replace instances where $user is passed as
the database to be connected to.
These changes are in server::database, server::role and
server::grant.
FreeBSD needs /usr/local/bin in PATH in order to find bash. This does
require that the node has the bash port installed. It might be desired
that a separate script was provided for FreeBSD which used its /bin/sh,
this could be done by changing the for loop in the script to use
for c in $(jot $TRIES)
in place of
for (( c=1; c<=$TRIES; c++ ))
Allows for OS specific $user and $group value specification. For most of
the target operating systems these will both be 'postgres'. For FreeBSD
however these values are 'pgsql'.
At present, the ownership of pg_hba.conf is hardwired to be uid 0. It should have the same ownership as all of the other postgressql configuration files in the same cluster so that they can be managed/edited by the postgres role user (system) account.
The warnings are as follows:
Warning: Scope(Concat::Fragment[pg_hba_rule_deny access to postgresql user]): The $mode parameter to concat::fragment is deprecated and has no effect
Warning: Scope(Concat::Fragment[pg_hba_rule_deny access to postgresql user]): The $owner parameter to concat::fragment is deprecated and has no effect
This commit modifies postgresql::validate_db_connection to use the
default_database parameter from postgresql::params rather than
hard-coding a local default value of "postgres".
This makes the variable consistent with the manner in which most/all of
the rest of the postgresql module currently works.
Commit also adds the new param to the README file.
E.g. pe-postgresql does NOT use postgres as the default database name.
It uses pe-postgres. So if there is no way to specify a default database
name, the postgesql::validate_db_connection resource in
postgresql::server::service will ALWAYS fail. This commit exposes the
parameter in order to avoid that situation.
Since the class is now throwing an error when you use the class directly,
I'm just removing it.
We left this in from the last rewrite as someone reported an issue a long
time ago, but alas we have been unable to prove its a problem.
Signed-off-by: Ken Barber <ken@bob.sh>
This patch is a fix for the race condition that keeps occuring during
postgresql setup. Its very rare on its own, but when you are using this
module in a CI environment it happens quite frequently.
Basically what happens is that sometimes the service will announce the
database has started, but really it is still working in the background.
Sometimes the unix socket may not be listening, and sometimes the
system is still loading and you get a weird client error.
The fix itself is a modification to postgresql::validate_db_connection
so that it is able to connect on the local unix socket, plus retry
until the database is available.
This new and improved validate_db_connection can then be put into the
build pipeline (in the service class in particular) to ensure the
database is started before continuing on with the remaining steps.
This in effect blocks the puppet module from continuing until the
postgresql database is fully started and able to receive connections
which is perfect.
Tests and documentation provided.
Signed-off-by: Ken Barber <ken@bob.sh>
The stdlib join() function takes an array and a string as parameters, it then joins all the elements of the array using the string as a separator.
On Centos 6.4 the join() function fails if given just an array but not a string.
This change will allow to test beta and release candidate version of
postgres from apt.postgresql.org.
For stable version this change will make available the libpq package
for that specific version (that will not hurts).
This patch provides fixes to help tests pass on Debian. The failures were in
relation to the lack of proper removal of the server and contrib parts. Setting
the ensure => purged parameter for removal seems to have solved a lot of this.
Also, the ordering for contrib being (un)installed needed some work.
I've also made sure we use 'purged' a lot more in tests to make sure we clean
up after ourselves properly.
Signed-off-by: Ken Barber <ken@bob.sh>
This is a very very large change to the module. It started out as a fix to add
postgresl::server::config_entry, and quickly became a rewrite to fix a lot of
ordering issues inherent in the API.
Since this changes the Public API it is considered a backwards compatible
change.
See the upgrading guide in README.md for more details as to what has been
modified in this patch.
Signed-off-by: Ken Barber <ken@bob.sh>
This includes the following changes:
- Hooking up pgconf module to postgresql to manage postgresql.conf
- pgconf includes a type and provider for postgresql.conf the provider
is a simple parsed file following basic ini parsing.
- Add config_entry.pp which wraps the pgconf module.
- Replace file_line resources by postgresql::config_entry in beforeservice.pp
- Adding unit tests for the newly introduced functionality
Most of these changes are just simple bits but I've wrapped a bunch of
stuff around 80 lines so it's slightly more readable. These should all
be no-op changes.
The postgresql::role defined type was not idempotent when passed cleartext
passwords. This is because we were comparing it with its md5 equivalent in
the db.
This patch converts any cleartext passwords to md5 before comparison, but
only if they are cleartext (ie. not starting with md5).
I also bumped the version of rspec-system-puppet to get use of the refresh
method, plus did some cleanup because the old tests were a bit dusty, again
taking advantage of refresh plus changing some matchers for clarity.
Signed-off-by: Ken Barber <ken@bob.sh>
This allows non-standard users (pe-postgres) to change passwords. Also
added a function to do escaping of the password, added system tests
and rspec tests for the function.
The code to grant permissions databases and their objects has been
abstracted to `postgresql::grant` and is used by both
`postgresql::database_grant` and `postgresql::table_grant`
Previously we only created a new user, any updates to the defined resource
would not update the role. This patch adds extra logic to modify a role
whenever a parameter is changed.
System tests have also been added to support this.
This patch just adds some new tests for the unknown OS patch, and cleans up
some existing tests to look for the new warning message.
Also, change the warning message for $osfamily and manage_package_repo to
reflect the parameter at fault.
Signed-off-by: Ken Barber <ken@bob.sh>
Prior to this commit, if you attempted to use the module to
manage postgres on any OS other than Redhat/Debian, there
was an explicit check for that, and a call to `fail`.
In reality, the OS family is only used to build up defaults
for various path and package names, which are all exposed
as parameters. If the user is willing to explicitly pass
in all of those parameters, there's no reason we should
fail based on OS family.
This commit adds checks to the 'default' osfamily case
such that we now only fail if they're on a non-Redhat-or-Debian
system *and* they haven't explicitly passed in values for
all of the required parameters.
This patch ports all of the existing system tests to use rspec-system instead.
To assist with this conversion some patches were made to fix OS compatibility
where necessary. We also added an ensure parameter to the postgresql::server
class to assist with removing PostgreSQL configuration to aid with testing
cleanups.
The documentation has been updated to indicate test usage with rspec-system,
we've also renamed the 'tests' directory to 'examples'.
Signed-off-by: Ken Barber <ken@bob.sh>
This fixes some mistakes in the .travis.yml file:
* Corrects matches so the minor gem revisions get rounded, this means 2.6.0
for example is used.
* Adds Ruby 2.0.0 tests
* Allows 2.6.0 to fail for now, as it doesn't have create_resources.
* Removes trailing commas and arrays in functions for 2.6.0.
Signed-off-by: Ken Barber <ken@bob.sh>
This removes the 'include' parameter for PostgreSQL 8.1 as it was failing on
Centos 5.
Also added Centos 5 system tests using our new Vagrant boxes.
Signed-off-by: Ken Barber <ken@bob.sh>
While this worked fine in Ubuntu, it failed silently in Centos.
The script is really designed to be ran as root, so removing the user
property. This was failing our new pg_hba_rule tests without it.
Signed-off-by: Ken Barber <ken@bob.sh>
This patch provides a more advanced way of managing pg_hba rules, by providing a
defined resource to manage a pg_hba file, and a defined resource for managing
rules within such a file (pg_hba_rule).
These new resources are wrappers around ripinaar-concat, and utilise file
assemblies instead of a template to compose the pg_hba.conf file.
I've provided a function that interprets the old ip4|6acl arrays and converts
them to this new format for backwards compatibility as well.
I slightly reformatted our documentation to allow for better documentation of
defined resources in 'Usage' as well, and provided examples of how to use this
new resource.
This hopefully should go a long way to solving the PR's related to lack of full
functionality for pg_hba.conf.
Signed-off-by: Ken Barber <ken@bob.sh>
Modify params.pp to abstract the package name for the postgresql JDBC
connector and add Class['postgresql::java']. Also update the README.md
to mention the availability of this class.
This adds the parameter 'locale' to the 'postgresql' class so we have a global
default, and adds it two the defined resources 'postgresql::db' and
'postgresql::database'. This allows users to either:
* Defined a global default for the cluster
* Define a per-database default
As a side-effect I had to make sure 'charset' was also exposed in a similar
manner as some locales need a particular charset to work.
Tests were added to test both the 'createdb' case and 'initdb' case for Redhat,
and some refactoring was done to make the existing non_default test area use
heredocs so my manifests and test code was kept close together. As apposed to
entirely different files and places in the directory structure.
I cleaned up the related execs a little bit, adding logoutput => on_failure
where needed so we can debug failures. Beforehand execs just 'failed', but
now we should be able to get better feedback from failed execs helping support.
I also add intention comments in parts of the Puppet code that I touched where
it made sense.
Signed-off-by: Ken Barber <ken@bob.sh>
In postgres 8.1, the 'CONNECT' privilege doesn't exist, which
would result in an error if you tried to use the 'database'
type. This commit conditions the revoke statement to use the
'ALL' privilege on 8.1.
