patroni.postgresql package¶
Submodules¶
- patroni.postgresql.bootstrap module
- patroni.postgresql.callback_executor module
- patroni.postgresql.cancellable module
- patroni.postgresql.citus module
- patroni.postgresql.config module
- patroni.postgresql.connection module
- patroni.postgresql.misc module
- patroni.postgresql.postmaster module
- patroni.postgresql.rewind module
- patroni.postgresql.slots module
- patroni.postgresql.sync module
- patroni.postgresql.validator module
Module contents¶
-
class
patroni.postgresql.
Postgresql
(config: Dict[str, Any])¶ Bases:
object
-
POSTMASTER_START_TIME
= 'pg_catalog.pg_postmaster_start_time()'¶
-
TL_LSN
= "CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_{0}file_name(pg_catalog.pg_current_{0}_{1}()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_{0}_{1}_diff(pg_catalog.pg_current_{0}{2}_{1}(), '0/0')::bigint END, pg_catalog.pg_{0}_{1}_diff(pg_catalog.pg_last_{0}_replay_{1}(), '0/0')::bigint, pg_catalog.pg_{0}_{1}_diff(COALESCE(pg_catalog.pg_last_{0}_receive_{1}(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_{0}_replay_paused()"¶
-
__init__
(config: Dict[str, Any]) → None¶ Initialize self. See help(type(self)) for accurate signature.
-
_before_stop
() → None¶ Synchronously run a script prior to stopping postgres.
-
_cluster_info_state_get
(name: str) → Optional[Any]¶
-
_do_before_stop
(cmd: str) → None¶
-
_do_stop
(mode: str, block_callbacks: bool, checkpoint: bool, on_safepoint: Optional[Callable[[...], Any]], on_shutdown: Optional[Callable[[...], Any]], before_shutdown: Optional[Callable[[...], Any]], stop_timeout: Optional[int]) → Tuple[bool, bool]¶
-
_get_gucs
() → patroni.collections.CaseInsensitiveSet¶ Get all available GUCs based on
postgres --describe-config
output.Returns: all available GUCs in the local Postgres server.
-
_pre_promote
() → bool¶ Runs a fencing script after the leader lock is acquired but before the replica is promoted. If the script exits with a non-zero code, promotion does not happen and the leader key is removed from DCS.
-
_query
(sql: str, *params) → List[Tuple[Any, ...]]¶ Execute sql query with params and optionally return results.
Parameters: - sql – SQL statement to execute.
- params – parameters to pass.
Returns: a query response as a list of tuples if there is any.
Raises: Error
if had issues while executing sql.PostgresConnectionException
: if had issues while connecting to the database.RetryFailedError
: if it was detected that connection/query failed due to PostgreSQL restart.
-
_version_file_exists
() → bool¶
-
_wait_for_connection_close
(postmaster: patroni.postgresql.postmaster.PostmasterProcess) → None¶
-
_wait_promote
(wait_seconds: int) → Optional[bool]¶
-
static
_wal_position
(is_primary: bool, wal_position: int, received_location: Optional[int], replayed_location: Optional[int]) → int¶
-
available_gucs
¶ GUCs available in this Postgres server.
-
call_nowait
(cb_type: patroni.postgresql.callback_executor.CallbackAction) → None¶ pick a callback command and call it without waiting for it to finish
-
callback
¶
-
can_advance_slots
¶ True
if :attr:major_version
is greater than 110000.
-
can_create_replica_without_replication_connection
(replica_methods: Optional[List[str]]) → bool¶ go through the replication methods to see if there are ones that does not require a working replication connection.
-
cb_called
¶
-
check_for_startup
() → bool¶ Checks PostgreSQL status and returns if PostgreSQL is in the middle of startup.
-
check_startup_state_changed
() → bool¶ Checks if PostgreSQL has completed starting up or failed or still starting.
Should only be called when state == ‘starting’
Returns: True if state was changed from ‘starting’
-
checkpoint
(connect_kwargs: Optional[Dict[str, Any]] = None, timeout: Optional[float] = None) → Optional[str]¶
-
cluster_info_query
¶ Returns the monitoring query with a fixed number of fields.
The query text is constructed based on current state in DCS and PostgreSQL version:
- function names depend on version. wal/lsn for v10+ and xlog/location for pre v10.
- for primary we query timeline_id (extracted from pg_walfile_name()) and pg_current_wal_lsn()
- for replicas we query pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn(), and pg_is_wal_replay_paused()
- for v9.6+ we query primary_slot_name and primary_conninfo from pg_stat_get_wal_receiver()
- for v11+ with permanent logical slots we query from pg_replication_slots and aggregate the result
- for standby_leader node running v9.6+ we also query pg_control_checkpoint to fetch timeline_id
- if sync replication is enabled we query pg_stat_replication and aggregate the result. In addition to that we get current values of synchronous_commit and synchronous_standby_names GUCs.
If some conditions are not satisfied we simply put static values instead. E.g., NULL, 0, ‘’, and so on.
-
configure_server_parameters
() → None¶
-
connection
() → Union[connection3, Connection3[Any]]¶
-
controldata
() → Dict[str, str]¶ return the contents of pg_controldata, or non-True value if pg_controldata call failed
-
create_replica_methods
¶
-
data_dir
¶
-
data_directory_empty
() → bool¶
-
database
¶
-
enforce_hot_standby_feedback
¶
-
ensure_major_version_is_known
() → bool¶ Calls configure_server_parameters() if _major_version is not known
Returns: True if _major_version is set, otherwise False
-
follow
(member: Union[patroni.dcs.Leader, patroni.dcs.Member, None], role: str = 'replica', timeout: Optional[float] = None, do_reload: bool = False) → Optional[bool]¶ Reconfigure postgres to follow a new member or use different recovery parameters.
Method may call on_role_change callback if role is changing.
Parameters: - member – The member to follow
- role – The desired role, normally ‘replica’, but could also be a ‘standby_leader’
- timeout – start timeout, how long should the start() method wait for postgres accepting connections
- do_reload – indicates that after updating postgresql.conf we just need to do a reload instead of restart
Returns: True - if restart/reload were successfully performed, False - if restart/reload failed None - if nothing was done or if Postgres is still in starting state after timeout seconds.
-
get_guc_value
(name: str) → Optional[str]¶
-
get_history
(timeline: int) → List[Union[Tuple[int, int, str], Tuple[int, int, str, str, str]]]¶
-
get_major_version
() → int¶ Reads major version from PG_VERSION file
Returns: major PostgreSQL version in integer format or 0 in case of missing file or errors
-
get_postgres_role_from_data_directory
() → str¶
-
get_primary_timeline
() → int¶ Returns: current timeline if postgres is running as a primary or 0.
-
get_replica_timeline
() → Optional[int]¶
-
get_replication_connection_cursor
(host: Optional[str] = None, port: Union[int, str] = 5432, **kwargs) → Iterator[Union[cursor, Cursor[Any]]]¶
-
global_config
¶
-
handle_parameter_change
() → None¶
-
initdb
(*args, **kwargs) → bool¶ Builds and executes the initdb command.
Parameters: - args – List of arguments to be joined into the initdb command.
- kwargs – Keyword arguments to pass to
subprocess.call
.
Returns: True
if the result ofsubprocess.call`, the exit code, is ``0
.
-
is_healthy
() → bool¶
-
is_primary
() → bool¶
-
is_running
() → Optional[patroni.postgresql.postmaster.PostmasterProcess]¶ Returns PostmasterProcess if one is running on the data directory or None. If most recently seen process is running updates the cached process based on pid file.
-
is_starting
() → bool¶
-
last_operation
() → int¶
-
latest_checkpoint_location
() → Optional[int]¶ Returns checkpoint location for the cleanly shut down primary. But, if we know that the checkpoint was written to the new WAL due to the archive_mode=on, we will return the LSN of prev wal record (SWITCH).
-
lsn_name
¶
-
major_version
¶
-
move_data_directory
() → None¶
-
parse_wal_record
(timeline: str, lsn: str) → Union[Tuple[str, str, str, str], Tuple[None, None, None, None]]¶
-
pending_restart
¶
-
pg_control_exists
() → bool¶
-
pg_control_timeline
() → Optional[int]¶
-
pg_ctl
(cmd: str, *args, **kwargs) → bool¶ Builds and executes pg_ctl command
Returns: !True when return_code == 0, otherwise !False
-
pg_isready
() → str¶ Runs pg_isready to see if PostgreSQL is accepting connections.
Returns: ‘ok’ if PostgreSQL is up, ‘reject’ if starting up, ‘no_resopnse’ if not up.
-
pg_stat_replication
() → List[Dict[str, Any]]¶ Returns: a result set of ‘SELECT * FROM pg_stat_replication’.
-
pg_tblspc_realpaths
() → Dict[str, str]¶ Returns a dict containing the symlink (key) and target (values) for the tablespaces
-
pg_wal_realpath
() → Dict[str, str]¶ Returns a dict containing the symlink (key) and target (value) for the wal directory
-
pgcommand
(cmd: str) → str¶ Return path to the specified PostgreSQL command.
Note
If
postgresql.bin_name.*cmd*
was configured by the user then that binary name is used, otherwise the default binary name cmd is used.Parameters: cmd – the Postgres binary name to get path to. Returns: path to Postgres binary named cmd.
-
postmaster_start_time
() → Optional[str]¶
-
primary_conninfo
() → Optional[str]¶
-
primary_slot_name
() → Optional[str]¶
-
promote
(wait_seconds: int, task: patroni.async_executor.CriticalTask, before_promote: Optional[Callable[[...], Any]] = None) → Optional[bool]¶
-
query
(sql: str, *params, retry: bool = True) → List[Tuple[Any, ...]]¶ Execute sql query with params and optionally return results.
Parameters: - sql – SQL statement to execute.
- params – parameters to pass.
- retry – whether the query should be retried upon failure or given up immediately.
Returns: a query response as a list of tuples if there is any.
Raises: Error
if had issues while executing sql.PostgresConnectionException
: if had issues while connecting to the database.RetryFailedError
: if it was detected that connection/query failed due to PostgreSQL restart or if retry deadline was exceeded.
-
received_location
() → Optional[int]¶
-
received_timeline
() → Optional[int]¶
-
reload
(block_callbacks: bool = False) → bool¶
-
reload_config
(config: Dict[str, Any], sighup: bool = False) → None¶
-
remove_data_directory
() → None¶
-
replay_paused
() → bool¶
-
replayed_location
() → Optional[int]¶
-
replica_cached_timeline
(primary_timeline: Optional[int]) → Optional[int]¶
-
replica_method_can_work_without_replication_connection
(method: str) → bool¶
-
replica_method_options
(method: str) → Dict[str, Any]¶
-
replication_state
() → Optional[str]¶ Checks replication state from pg_stat_get_wal_receiver().
Note
Available only since 9.6
Returns: streaming
,in archive recovery
, orNone
-
replication_state_from_parameters
(is_primary: bool, receiver_state: Optional[str], restore_command: Optional[str]) → Optional[str]¶ Figure out the replication state from input parameters.
Note
This method could be only called when Postgres is up, running and queries are successfuly executed.
Is_primary: True is postgres is not running in recovery Receiver_state: value from pg_stat_get_wal_receiver.state or None if Postgres is older than 9.6 Restore_command: value of restore_command
GUC for PostgreSQL 12+ or postgresql.recovery_conf.restore_command if it is set in Patroni configurationReturns: - None for the primary and for Postgres older than 9.6;
- ’streaming’ if replica is streaming according to the pg_stat_wal_receiver view;
- ’in archive recovery’ if replica isn’t streaming and there is a restore_command
-
reset_cluster_info_state
(cluster: Optional[patroni.dcs.Cluster], nofailover: bool = False, global_config: Optional[GlobalConfig] = None) → None¶ Reset monitoring query cache.
It happens in the beginning of heart-beat loop and on change of synchronous_standby_names.
Parameters: - cluster – currently known cluster state from DCS
- nofailover – whether this node could become a new primary. Important when there are logical permanent replication slots because “nofailover” node could do cascading replication and should enable hot_standby_feedback
- global_config – last known
GlobalConfig
object
-
restart
(timeout: Optional[float] = None, task: Optional[patroni.async_executor.CriticalTask] = None, block_callbacks: bool = False, role: Optional[str] = None, before_shutdown: Optional[Callable[[...], Any]] = None, after_start: Optional[Callable[[...], Any]] = None) → Optional[bool]¶ Restarts PostgreSQL.
When timeout parameter is set the call will block either until PostgreSQL has started, failed to start or timeout arrives.
Returns: True when restart was successful and timeout did not expire when waiting.
-
resume_wal_replay
() → None¶
-
role
¶
-
schedule_sanity_checks_after_pause
() → None¶ After coming out of pause we have to: 1. configure server parameters if necessary 2. sync replication slots, because it might happen that slots were removed 3. get new ‘Database system identifier’ to make sure that it wasn’t changed
-
server_version
¶
-
set_enforce_hot_standby_feedback
(value: bool) → None¶
-
set_pending_restart
(value: bool) → None¶
-
set_role
(value: str) → None¶
-
set_state
(value: str) → None¶
-
slots
() → Dict[str, int]¶
-
start
(timeout: Optional[float] = None, task: Optional[patroni.async_executor.CriticalTask] = None, block_callbacks: bool = False, role: Optional[str] = None, after_start: Optional[Callable[[...], Any]] = None) → Optional[bool]¶ Start PostgreSQL
Waits for postmaster to open ports or terminate so pg_isready can be used to check startup completion or failure.
Returns: True if start was initiated and postmaster ports are open, False if start failed, and None if postgres is still starting up
-
state
¶
-
stop
(mode: str = 'fast', block_callbacks: bool = False, checkpoint: Optional[bool] = None, on_safepoint: Optional[Callable[[...], Any]] = None, on_shutdown: Optional[Callable[[int], Any]] = None, before_shutdown: Optional[Callable[[...], Any]] = None, stop_timeout: Optional[int] = None) → bool¶ Stop PostgreSQL
Supports a callback when a safepoint is reached. A safepoint is when no user backend can return a successful commit to users. Currently this means we wait for user backends to close. But in the future alternate mechanisms could be added.
Parameters: - on_safepoint – This callback is called when no user backends are running.
- on_shutdown – is called when pg_controldata starts reporting Database cluster state: shut down
- before_shutdown – is called after running optional CHECKPOINT and before running pg_ctl stop
-
supports_multiple_sync
¶ True if Postgres version supports more than one synchronous node.
Type: returns
-
synchronous_commit
() → str¶ Returns: “synchronous_commit” GUC value.
-
synchronous_standby_names
() → str¶ Returns: “synchronous_standby_names” GUC value.
-
sysid
¶
-
terminate_postmaster
(postmaster: patroni.postgresql.postmaster.PostmasterProcess, mode: str, stop_timeout: Optional[int]) → Optional[bool]¶
-
terminate_starting_postmaster
(postmaster: patroni.postgresql.postmaster.PostmasterProcess) → None¶ Terminates a postmaster that has not yet opened ports or possibly even written a pid file. Blocks until the process goes away.
-
time_in_state
() → float¶
-
timeline_wal_position
() → Tuple[int, int, Optional[int]]¶
-
wait_for_port_open
(postmaster: patroni.postgresql.postmaster.PostmasterProcess, timeout: float) → bool¶ Waits until PostgreSQL opens ports.
-
wait_for_startup
(timeout: float = 0) → Optional[bool]¶ Waits for PostgreSQL startup to complete or fail.
Returns: True if start was successful, False otherwise
-
wal_dir
¶
-
wal_flush
¶ For PostgreSQL 9.6 onwards we want to use pg_current_wal_flush_lsn()/pg_current_xlog_flush_location().
-
wal_name
¶
-
waldump
(timeline: Union[int, str], lsn: str, limit: int) → Tuple[Optional[bytes], Optional[bytes]]¶
-
-
patroni.postgresql.
null_context
()¶