Each time ltcluster or ltclusterd perform a significant event, a record
of that event is written into the ltcluster.events
table together with
a timestamp, an indication of failure or success, and further details
if appropriate. This is useful for gaining an overview of events
affecting the replication cluster. However note that this table has
advisory character and should be used in combination with the ltcluster
and LightDB logs to obtain details of any events.
Example output after a primary was registered and a standby cloned and registered:
ltcluster=# SELECT * from ltcluster.events ; node_id | event | successful | event_timestamp | details ---------+------------------+------------+-------------------------------+------------------------------------------------------------------------------------- 1 | primary_register | t | 2016-01-08 15:04:39.781733+09 | 2 | standby_clone | t | 2016-01-08 15:04:49.530001+09 | Cloned from host 'ltcluster_node1', port 5432; backup method: lt_basebackup; --force: N 2 | standby_register | t | 2016-01-08 15:04:50.621292+09 | (3 rows)
Alternatively, use ltcluster cluster event to output a formatted list of events.
Additionally, event notifications can be passed to a user-defined program
or script which can take further action, e.g. send email notifications.
This is done by setting the event_notification_command
parameter in
ltcluster.conf
.
The following format placeholders are provided for all event notifications:
%n
node ID
%e
event type
%s
success (1) or failure (0)
%t
timestamp
%d
details
The values provided for %t
and %d
may contain spaces, so should be quoted in the provided command
configuration, e.g.:
event_notification_command='/path/to/some/script %n %e %s "%t" "%d"'
The following parameters are provided for a subset of event notifications:
%p
node ID of the current primary (ltcluster standby register and ltcluster standby follow)
node ID of the demoted primary (ltcluster standby switchover only)
%c
conninfo
string of the primary node
(ltcluster standby register and ltcluster standby follow)
%a
name of the current primary node (ltcluster standby register and ltcluster standby follow)
The values provided for %c
and %a
will probably contain spaces, so should always be quoted.
By default, all notification types will be passed to the designated script;
the notification types can be filtered to explicitly named ones using the
event_notifications
parameter, e.g.:
event_notifications='primary_register,standby_register,witness_register'
Events generated by the ltcluster command:
Events generated by ltclusterd (streaming replication mode):
ltclusterd_start
ltclusterd_shutdown
ltclusterd_reload
ltclusterd_failover_promote
ltclusterd_failover_follow
ltclusterd_failover_aborted
ltclusterd_standby_reconnect
ltclusterd_promote_error
ltclusterd_local_disconnect
ltclusterd_local_reconnect
ltclusterd_upstream_disconnect
ltclusterd_upstream_reconnect
standby_disconnect_manual
standby_failure
standby_recovery
child_node_disconnect
child_node_reconnect
child_node_new_connect
child_nodes_disconnect_command
Note that under some circumstances (e.g. when no replication cluster primary
could be located), it will not be possible to write an entry into the
ltcluster.events
table, in which case executing a script via event_notification_command
can serve as a fallback by generating some form of notification.