This change will allow to test beta and release candidate version of
postgres from apt.postgresql.org.
For stable version this change will make available the libpq package
for that specific version (that will not hurts).
This patch provides fixes to help tests pass on Debian. The failures were in
relation to the lack of proper removal of the server and contrib parts. Setting
the ensure => purged parameter for removal seems to have solved a lot of this.
Also, the ordering for contrib being (un)installed needed some work.
I've also made sure we use 'purged' a lot more in tests to make sure we clean
up after ourselves properly.
Signed-off-by: Ken Barber <ken@bob.sh>
This is a very very large change to the module. It started out as a fix to add
postgresl::server::config_entry, and quickly became a rewrite to fix a lot of
ordering issues inherent in the API.
Since this changes the Public API it is considered a backwards compatible
change.
See the upgrading guide in README.md for more details as to what has been
modified in this patch.
Signed-off-by: Ken Barber <ken@bob.sh>
This includes the following changes:
- Hooking up pgconf module to postgresql to manage postgresql.conf
- pgconf includes a type and provider for postgresql.conf the provider
is a simple parsed file following basic ini parsing.
- Add config_entry.pp which wraps the pgconf module.
- Replace file_line resources by postgresql::config_entry in beforeservice.pp
- Adding unit tests for the newly introduced functionality
Most of these changes are just simple bits but I've wrapped a bunch of
stuff around 80 lines so it's slightly more readable. These should all
be no-op changes.
The postgresql::role defined type was not idempotent when passed cleartext
passwords. This is because we were comparing it with its md5 equivalent in
the db.
This patch converts any cleartext passwords to md5 before comparison, but
only if they are cleartext (ie. not starting with md5).
I also bumped the version of rspec-system-puppet to get use of the refresh
method, plus did some cleanup because the old tests were a bit dusty, again
taking advantage of refresh plus changing some matchers for clarity.
Signed-off-by: Ken Barber <ken@bob.sh>
This allows non-standard users (pe-postgres) to change passwords. Also
added a function to do escaping of the password, added system tests
and rspec tests for the function.
The code to grant permissions databases and their objects has been
abstracted to `postgresql::grant` and is used by both
`postgresql::database_grant` and `postgresql::table_grant`
Previously we only created a new user, any updates to the defined resource
would not update the role. This patch adds extra logic to modify a role
whenever a parameter is changed.
System tests have also been added to support this.
This patch just adds some new tests for the unknown OS patch, and cleans up
some existing tests to look for the new warning message.
Also, change the warning message for $osfamily and manage_package_repo to
reflect the parameter at fault.
Signed-off-by: Ken Barber <ken@bob.sh>
Prior to this commit, if you attempted to use the module to
manage postgres on any OS other than Redhat/Debian, there
was an explicit check for that, and a call to `fail`.
In reality, the OS family is only used to build up defaults
for various path and package names, which are all exposed
as parameters. If the user is willing to explicitly pass
in all of those parameters, there's no reason we should
fail based on OS family.
This commit adds checks to the 'default' osfamily case
such that we now only fail if they're on a non-Redhat-or-Debian
system *and* they haven't explicitly passed in values for
all of the required parameters.
This patch ports all of the existing system tests to use rspec-system instead.
To assist with this conversion some patches were made to fix OS compatibility
where necessary. We also added an ensure parameter to the postgresql::server
class to assist with removing PostgreSQL configuration to aid with testing
cleanups.
The documentation has been updated to indicate test usage with rspec-system,
we've also renamed the 'tests' directory to 'examples'.
Signed-off-by: Ken Barber <ken@bob.sh>
This fixes some mistakes in the .travis.yml file:
* Corrects matches so the minor gem revisions get rounded, this means 2.6.0
for example is used.
* Adds Ruby 2.0.0 tests
* Allows 2.6.0 to fail for now, as it doesn't have create_resources.
* Removes trailing commas and arrays in functions for 2.6.0.
Signed-off-by: Ken Barber <ken@bob.sh>
This removes the 'include' parameter for PostgreSQL 8.1 as it was failing on
Centos 5.
Also added Centos 5 system tests using our new Vagrant boxes.
Signed-off-by: Ken Barber <ken@bob.sh>
While this worked fine in Ubuntu, it failed silently in Centos.
The script is really designed to be ran as root, so removing the user
property. This was failing our new pg_hba_rule tests without it.
Signed-off-by: Ken Barber <ken@bob.sh>
This patch provides a more advanced way of managing pg_hba rules, by providing a
defined resource to manage a pg_hba file, and a defined resource for managing
rules within such a file (pg_hba_rule).
These new resources are wrappers around ripinaar-concat, and utilise file
assemblies instead of a template to compose the pg_hba.conf file.
I've provided a function that interprets the old ip4|6acl arrays and converts
them to this new format for backwards compatibility as well.
I slightly reformatted our documentation to allow for better documentation of
defined resources in 'Usage' as well, and provided examples of how to use this
new resource.
This hopefully should go a long way to solving the PR's related to lack of full
functionality for pg_hba.conf.
Signed-off-by: Ken Barber <ken@bob.sh>
Modify params.pp to abstract the package name for the postgresql JDBC
connector and add Class['postgresql::java']. Also update the README.md
to mention the availability of this class.
This adds the parameter 'locale' to the 'postgresql' class so we have a global
default, and adds it two the defined resources 'postgresql::db' and
'postgresql::database'. This allows users to either:
* Defined a global default for the cluster
* Define a per-database default
As a side-effect I had to make sure 'charset' was also exposed in a similar
manner as some locales need a particular charset to work.
Tests were added to test both the 'createdb' case and 'initdb' case for Redhat,
and some refactoring was done to make the existing non_default test area use
heredocs so my manifests and test code was kept close together. As apposed to
entirely different files and places in the directory structure.
I cleaned up the related execs a little bit, adding logoutput => on_failure
where needed so we can debug failures. Beforehand execs just 'failed', but
now we should be able to get better feedback from failed execs helping support.
I also add intention comments in parts of the Puppet code that I touched where
it made sense.
Signed-off-by: Ken Barber <ken@bob.sh>
In postgres 8.1, the 'CONNECT' privilege doesn't exist, which
would result in an error if you tried to use the 'database'
type. This commit conditions the revoke statement to use the
'ALL' privilege on 8.1.
Previously we hadn't supported using the official PostgreSQL repos for deb-based
packaging. This commit fixes that by introducing the management of the apt
repositories on the host as required, and using the correct package and service
name.
Signed-off-by: Ken Barber <ken@bob.sh>
Previously the validate_db_connection defined type was trying to use inherits
like a class. This of course would fail.
After analyzing its usage, I've removed the need for the top-level params
inherit by just including the 'postgresql' module which pulls in the client
package and the params class as well. It also avoids resource duplication
for the client package as well.
To ensure we don't get regression on this I've added system tests that test
this defined type in a success and failure state.
Signed-off-by: Ken Barber <ken@bob.sh>
When the psql command runs from a directory it does not have permission to
access, it outputs an error. This error trips up the unless SQL command,
causing the other SQL commands to run even if not needed. Rather than ignore
stderr (which might hide something else), or use an arbitrary directory like
/tmp, this code sets the cwd to the data directory, which will exist and be
owned by the postgres user. If someone uses the postgres_psql type and
customises the psql_user parameter, they should also set an appropriate cwd.
The GPG key for yum.postgresql.org was in the wrong directory,
and as written, would have only supported postgres 9.2 anyway.
I looked at the yum rpms for a few other versions besides 9.2,
and they appear to be using the same GPG key. So this commit
assumes that they all are doing so, and it sets things up
accordingly. Hopefully this will make the yumrepo support
compatible with other versions of postgres besides 9.2, but I
haven't added that to the actual test matrix yet.
This commit fixes up the `postgres_default_version` fact so that
it doesn't use apt/yum (slow), and instead just has a hard-coded
list of default postgres versions for various OS versions. We
will need to add new OS versions to this fact over time, but that
seems preferable to the previous implementation which was causing
slower puppet runs on all nodes (regardless of whether they were
actually using postgres or not).
Thanks to some tricks I learned from Nan Liu and Dan Bode, I was
able to figure out a way to move all of the new version-related stuff
back into the params class, and clean up some of the if/_real stuff.
Basic tests for centos6 + pg 9.2 are passing.
Nan showed me a trick that will let us keep all of that param stuff
inside of params.pp, make it a parameterized class, and still support
the ability for users to specify a custom (non-system-default) pg
version. This commit takes the first step towards that pattern by
consolidating platform.pp and params.pp. (Everything old is new again!)
This commit creates a new class called `package_source_info`,
which has some initial framework for managing the postgresql.org
yumrepo. It also serves as a container for the 'version'
variable that is needed by the 'platform' class in order to
use other versions of postgres besides the system default.
The service provider / status stuff got a little broken during
the refactors. This should mostly fix it but there is still
one spec failing, and I will probably also refactor the
paths.pp and packages.pp into a single file together eventually.
This commit does a fairly major refactor of how the spec tests
are laid out. The main goal was to make it easier to run
a subset of the tests--e.g., the ability to only run tests
on a single OS via a simple rspec command.
The test logic is now defined in some shared examples in the
`support` directory. There are now spec folders for each
distro, which contain some stubs to include the shared examples
as well as a Vagrantfile for the particular distro.
Also, the system-default postgres package tests now run
successfully against the CentOS6 VM that is defined by the
Vagrantfile.
In the exec resource to reload postgres add path parameter and set it to
"/usr/bin:/usr/sbin:/bin:/sbin", so that the service command can be
called correctly on different platforms.
notify that exec after changes to pg_hba.conf. This avoids restarting
the whole postgresql cluster for configuration changes in pg_hba.conf
which only need a reload.
This commit provides a working implementation of a ruby
type/provider (`postgresql_psql` for handling the PSQL
commands. This is a little more flexible than doing it
via Exec resources, which is what the `postgresql::psql`
type was doing.
The old type is still present but now includes a
deprecation notification, and all of the other types
that were using the `::psql` type have been ported over
to use the `postgresql_psql` type instead.
This will allow characters such as '-' in database role
names. Additionally, escaping of '"' characters now applies to all '"'
characters, not just the first in a sql command.
The postgresql init script on debian/ubuntu returns 0 no matter whether postgresql is started or stopped, so puppet has no way of knowing whether to start postgresql when the service is set to 'ensure => running'. This commit adds a param to the params class called $service_status which is set to the status piped to an egrep on debian/ubuntu, which reliably returns 0 if there are clusters running and 1 if there are none. The output before and after this patch can be seen below. If the init script is fixed at some point, the logic would need to be revisited.
Before patch postgresql remains stopped after puppet run.
$ # service postgresql stop
$ # * Stopping PostgreSQL 9.1 database server
$ # puppet apply -e "class {'postgresql::server':}"
$ # notice: Finished catalog run in 0.15 seconds
$ # service postgresql status
$ # Running clusters:
After patch postgresql is started after puppet run.
$ # service postgresql stop
$ # * Stopping PostgreSQL 9.1 database server
$ # puppet apply -e "class {'postgresql::server':}"
$ # notice: /Stage[main]/Postgresql::Server/Service[postgresqld]/ensure: ensure changed 'stopped' to 'running'
$ # notice: Finished catalog run in 2.26 seconds
$ # service postgresql status
$ # Running clusters: 9.1/main
The title I'd used for the persist-firewall resource
was too generic, and fairly likely to collide with
resources in other modules. This commit simply
renames it to be a bit more explicit about belonging
to this module, to reduce the likelihood of a title
collision.
This commit adds a new resource type that can be used to validate
that a successful postgres connection can be established from a
puppet node to a (potentially remote) postgres database instance.
Ubuntu's precise ships with init scripts, not with upstart service definitions. Declaring the provider to be upstart will cause service management on postgres to fail. Leaving it undefined will allow puppet to select the correct (init) provider.
This commit adds a postgresql::db type for convenience;
it mirrors the 'db' type from the mysql module, which
allows you to create a database instance and user plus
grant privileges to that user all in one succint
resource.
This commit also improves security in the following ways:
* Revoke "CONNECT" privilege from the 'public' role for
newly created databases; without this, any database
created via this module will allow connections from
any database user, and will allow them to do things
like create tables.
* Change to a 'reject'-based policy for dealing with
remote connections by the postgres user in pg_hba.conf.
Prior to this commit, if you tried to restrict access
to the postgres user by IP, the rule would simply not
match for disallowed IPs; then it would fall through
to the rule for "all" users, which could still match
and thus allow the postgres user to connect remotely.
Renamed a few files and made some tweaks to try to get
database_grant, database_user, and database types into
a state where they work very similarly to the ones in
the mysql module. Also introduced a "postgresql_password"
function that can be used to generate an md5 password
hash for a postgres user.
This commit adds some configuration management for
postgres, to allow users to get a more complete
setup from their initial install. Prior to this
commit, we were basically only ensuring that the
package was installed and the service was running.
Now, we support limited configuration for the
pg_hba.conf file to enable md5 authentication for
remote hosts, and for the postgresql.conf file
to specify the listener addresses where TCP
connections should be accepted. Without these
two changes the initial postgres configuration
doesn't allow *any* connections from outside of the
local host.
This commit also adds an option for opening up the
postgres port in the firewall on redhat-based systems,
and an option to allow setting the password for the
'postgres' database user.
As of this commit, this module now has dependencies
on puppetlabs-stdlib (version > 2.3.4, which includes
the new 'match' parameter for the 'file_line' resource
type), and on puppetlabs-firewall.
This is a first working version of postgresql::server.
It includes a very simple test manifest, which has
been tried out on CentOS6 and Ubuntu 10.04; initial
tests were successful both from a clean state and
for subsequent runs.
Includes a new fact called 'postgres_default_version',
which detects what the default version of postgres is
for a given OS. This is needed because some of the
commands and directory names include this version string.
Current implementation *only* supports managing the
system default version; in the future it would be nice
to allow the user to explicitly specify a postgres version,
but that isn't yet supported.
The "postgresql::server" class includes a call to postgres's
initdb command on redhat systems, because they don't do
this automatically when the package is installed.
