Logging changes to postgresql.conf
It's Sunday morning here in Japan, which in my case means it's an excellent time for a round of database server updates without interrupting production flow (lucky me). None of the databases in question are directly vulnerable to the recent security issue as for some crazy reason I prefer not to have port 5432 swinging in the Internet breeze for all and sundry to probe. However updates are updates, and the sooner applied the better - you never know what creative attack vectors all and sundry will dream up.
While I was updating, I was taking the opportunity to perform the odd bit of administrative TLC, which involves editing the postgresql.conf file, which involves manually checking in the changed version into source control, which is mildly onerous. Also, it's not convenient for tracking changes to individual configuration items over time. And it would be kind of handy to record the database settings in the database itself. Also, if it's possible to record exactly which configuration file the setting was taken from (a potential issue if the 'include' directive is used), it might be helpful when tracking down errors.
Anyway, it occurred to me that pg_settings stores displays information about which configuration items were set from values explicitly defined in postgresql.conf(and also notes which line in which file they were set on), so it should be possible to track changes on the basis of pg_settings' output:
1. Create and pre-populate a log table:
CREATE TABLE pg_settings_log AS
SELECT name,
setting,
unit,
sourcefile,
sourceline,
CAST('INSERT' AS VARCHAR(6)) AS op,
CURRENT_TIMESTAMP AS recorded_ts
FROM pg_settings WHERE source='configuration file'
2. Create a view to show the most recent state of log table entries:
CREATE OR REPLACE VIEW pg_settings_log_current
AS SELECT psl.*
FROM pg_settings_log psl
LEFT JOIN pg_settings_log psl_ref
ON (psl.name = psl_ref.name
AND psl.recorded_ts < psl_ref.recorded_ts)
WHERE psl_ref.name IS NULL
3. Create a function to update the log table to record any changes to pg_settings
:
CREATE OR REPLACE FUNCTION pg_settings_logger()
RETURNS BOOLEAN
LANGUAGE plpgsql
AS $$
DECLARE
changed BOOLEAN := FALSE;
settings_rec RECORD;
BEGIN
FOR settings_rec IN
WITH pg_settings_log_current AS (
SELECT *
FROM pg_settings_log_current
ORDER BY name
)
SELECT 'UPDATE' AS op,
ps.name,
ps.setting,
ps.unit,
ps.sourcefile,
ps.sourceline
FROM pg_settings ps
INNER JOIN pg_settings_log_current psl ON (psl.name=ps.name AND psl.setting != ps.setting)
WHERE ps.source ='configuration file'
UNION
SELECT 'INSERT' AS op,
ps.name,
ps.setting,
ps.unit,
ps.sourcefile,
ps.sourceline
FROM pg_settings ps
WHERE ps.source ='configuration file'
AND NOT EXISTS (SELECT NULL
FROM pg_settings_log_current psl
WHERE psl.name = ps.name
)
UNION
SELECT 'DELETE' AS op,
psl.name,
psl.setting,
psl.unit,
psl.sourcefile,
psl.sourceline
FROM pg_settings_log_current psl
WHERE EXISTS (SELECT NULL
FROM pg_settings ps
WHERE ps.name = psl.name
AND ps.source ='default'
)
AND psl.op != 'DELETE'
LOOP
INSERT INTO pg_settings_log
(name,
setting,
unit,
sourcefile,
sourceline,
op,
recorded_ts
)
VALUES(settings_rec.name,
settings_rec.setting,
settings_rec.unit,
settings_rec.sourcefile,
settings_rec.sourceline,
settings_rec.op,
CURRENT_TIMESTAMP
);
changed = TRUE;
END LOOP;
RETURN changed;
END;
$$
If needed, stop the common riffraff from doing anything with these three freshly-minted objects:
REVOKE ALL ON pg_settings_log FROM public; REVOKE ALL ON pg_settings_log_current FROM public; REVOKE ALL ON FUNCTION pg_settings_logger() FROM public;The table will now look like this (settings from a test DB on a laptop, in case anyone's wondering about the general weirdness):
postgres=# SELECT * FROM pg_settings_log ORDER BY name;
name | setting | unit | sourcefile | sourceline | op | recorded_ts
----------------------------------+---------------------------------+------+---------------------------------+------------+--------+-------------------------------
DateStyle | ISO, DMY | ¤ | /opt/data/pg92b/postgresql.conf | 496 | INSERT | 2013-04-07 19:12:41.426134+09
TimeZone | Japan | ¤ | /opt/data/pg92b/postgresql.conf | 498 | INSERT | 2013-04-07 19:12:41.426134+09
checkpoint_segments | 5 | | /opt/data/pg92b/postgresql.conf | 184 | INSERT | 2013-04-07 19:12:41.426134+09
default_text_search_config | pg_catalog.english | ¤ | /opt/data/pg92b/postgresql.conf | 518 | INSERT | 2013-04-07 19:12:41.426134+09
lc_messages | en_GB.UTF-8 | ¤ | /opt/data/pg92b/postgresql.conf | 511 | INSERT | 2013-04-07 19:12:41.426134+09
lc_monetary | en_GB.UTF-8 | ¤ | /opt/data/pg92b/postgresql.conf | 513 | INSERT | 2013-04-07 19:12:41.426134+09
lc_numeric | en_GB.UTF-8 | ¤ | /opt/data/pg92b/postgresql.conf | 514 | INSERT | 2013-04-07 19:12:41.426134+09
lc_time | en_GB.UTF-8 | ¤ | /opt/data/pg92b/postgresql.conf | 515 | INSERT | 2013-04-07 19:12:41.426134+09
listen_addresses | localhost | ¤ | /opt/data/pg92b/postgresql.conf | 59 | INSERT | 2013-04-07 19:12:41.426134+09
log_line_prefix | ['foo'] | ¤ | /opt/data/pg92b/postgresql.conf | 392 | INSERT | 2013-04-07 19:12:41.426134+09
log_timezone | Japan | ¤ | /opt/data/pg92b/postgresql.conf | 417 | INSERT | 2013-04-07 19:12:41.426134+09
max_connections | 20 | | /opt/data/pg92b/postgresql.conf | 64 | INSERT | 2013-04-07 19:12:41.426134+09
pg_stat_statements.track_utility | off | ¤ | /opt/data/pg92b/postgresql.conf | 574 | INSERT | 2013-04-07 19:12:41.426134+09
shared_buffers | 150 | 8kB | /opt/data/pg92b/postgresql.conf | 113 | INSERT | 2013-04-07 19:12:41.426134+09
shared_preload_libraries | pg_hint_plan,pg_stat_statements | ¤ | /opt/data/pg92b/postgresql.conf | 135 | INSERT | 2013-04-07 19:12:41.426134+09
ssl | off | ¤ | /opt/data/pg92b/postgresql.conf | 80 | INSERT | 2013-04-07 19:12:41.426134+09
(16 rows)
Following changes are then made to postgresql.conf
:
- comment out entry "ssl=off "
- change entry "checkpoint_segments " to 10
- restart server
postgres=# SELECT pg_settings_logger();
pg_settings_logger
--------------------
t
(1 row)
postgres=# SELECT * FROM pg_settings_log_current ORDER BY name;
name | setting | unit | sourcefile | sourceline | op | recorded_ts
----------------------------------+---------------------------------+------+---------------------------------+------------+--------+-------------------------------
DateStyle | ISO, DMY | ¤ | /opt/data/pg92b/postgresql.conf | 496 | INSERT | 2013-04-07 19:12:41.426134+09
TimeZone | Japan | ¤ | /opt/data/pg92b/postgresql.conf | 498 | INSERT | 2013-04-07 19:12:41.426134+09
checkpoint_segments | 10 | | /opt/data/pg92b/postgresql.conf | 184 | UPDATE | 2013-04-07 19:17:11.499642+09
default_text_search_config | pg_catalog.english | ¤ | /opt/data/pg92b/postgresql.conf | 518 | INSERT | 2013-04-07 19:12:41.426134+09
lc_messages | en_GB.UTF-8 | ¤ | /opt/data/pg92b/postgresql.conf | 511 | INSERT | 2013-04-07 19:12:41.426134+09
lc_monetary | en_GB.UTF-8 | ¤ | /opt/data/pg92b/postgresql.conf | 513 | INSERT | 2013-04-07 19:12:41.426134+09
lc_numeric | en_GB.UTF-8 | ¤ | /opt/data/pg92b/postgresql.conf | 514 | INSERT | 2013-04-07 19:12:41.426134+09
lc_time | en_GB.UTF-8 | ¤ | /opt/data/pg92b/postgresql.conf | 515 | INSERT | 2013-04-07 19:12:41.426134+09
listen_addresses | localhost | ¤ | /opt/data/pg92b/postgresql.conf | 59 | INSERT | 2013-04-07 19:12:41.426134+09
log_line_prefix | ['foo'] | ¤ | /opt/data/pg92b/postgresql.conf | 392 | INSERT | 2013-04-07 19:12:41.426134+09
log_timezone | Japan | ¤ | /opt/data/pg92b/postgresql.conf | 417 | INSERT | 2013-04-07 19:12:41.426134+09
max_connections | 20 | | /opt/data/pg92b/postgresql.conf | 64 | INSERT | 2013-04-07 19:12:41.426134+09
pg_stat_statements.track_utility | off | ¤ | /opt/data/pg92b/postgresql.conf | 574 | INSERT | 2013-04-07 19:12:41.426134+09
shared_buffers | 150 | 8kB | /opt/data/pg92b/postgresql.conf | 113 | INSERT | 2013-04-07 19:12:41.426134+09
shared_preload_libraries | pg_hint_plan,pg_stat_statements | ¤ | /opt/data/pg92b/postgresql.conf | 135 | INSERT | 2013-04-07 19:12:41.426134+09
ssl | off | ¤ | /opt/data/pg92b/postgresql.conf | 80 | DELETE | 2013-04-07 19:17:11.499642+09
Here we see the setting for "checkpoint_segments" has been updated, and the entry for "ssl" is marked with 'DELETE' to indicate the setting is no longer specified inpostgresql.conf.
Note that 'DELETE' implies the configuration setting has been removed from (or commented out in) the postgresql.conf file. The 'settings' value is the same as the last registered change; exclude rows marked with 'DELETE' from the query to show the current configuration status. If the setting is subsequently reinstated inpostgresql.conf, it will be registered as an 'INSERT'.
In postgresql.conf, "checkpoint_segments" is now changed to 15 and the configuration file reloaded:
postgres=# SELECT pg_reload_conf();
pg_reload_conf
----------------
t
(1 row)
postgres=# SELECT pg_settings_logger();
pg_settings_logger
--------------------
t
(1 row)
postgres=# SELECT * FROM pg_settings_log WHERE name='checkpoint_segments' ORDER BY recorded_ts DESC;
name | setting | unit | sourcefile | sourceline | op | recorded_ts
---------------------+---------+------+---------------------------------+------------+--------+-------------------------------
checkpoint_segments | 15 | | /opt/data/pg92b/postgresql.conf | 184 | UPDATE | 2013-04-07 19:21:10.680142+09
checkpoint_segments | 10 | | /opt/data/pg92b/postgresql.conf | 184 | UPDATE | 2013-04-07 19:17:11.499642+09
checkpoint_segments | 5 | | /opt/data/pg92b/postgresql.conf | 184 | INSERT | 2013-04-07 19:12:41.426134+09
(3 rows)
and voila, full change history for that parameter.
This should work with all PostgreSQL versions as far back as 8.4. Unfortunately (as far as I'm aware) there's no built-in way to trigger this function automatically whenever the config file is reloaded. While it would be possible to have pg_settings_logger() call pg_reload_config(), that doesn't cover all cases when the configuration settings are changed, e.g. for settings requiring a server restart.
A cronjob would do the trick, but feels like overkill - it's not as if postgresql.conf gets changed on a regular basis. For 9.3, a custom background worker might be an option, or if it becomes possible to alter the postgresql.conf file via SQL - maybe there'll be some way of hooking into that mechanism.
Caveats:
- this is something I knocked together over a late breakfast, and has not been exhaustively tested, and there's probably a better way of doing it
- pg_settings_logger() should be called from a fresh session which hasn't overridden any configuration settings
- it doesn't record who made the changes (though if you're not sure who's making changes on your database server, you might have a whole bunch of other problems...)
- Using source control, it's easy to add a commit comment ("added moar walsenders") to summarize the changes made, which is not possible here. It would be simple enough to add a comment parameter to pg_settings_logger() which stores the comment either with each changed row, or in a separate table
- probably someone has already come up with a better solution already which has totally escaped my notice