patroni.config module¶
Facilities related to Patroni configuration.
-
class
patroni.config.
Config
(configfile: str, validator: Optional[Callable[[Dict[str, Any]], List[str]]] = <function default_validator>)¶ Bases:
object
Handle Patroni configuration.
This class is responsible for:
- Building and giving access to
effective_configuration
from:Config.__DEFAULT_CONFIG
– some sane default values;dynamic_configuration
– configuration stored in DCS;local_configuration
– configuration from config.yml or environment.
- Saving and loading
dynamic_configuration
into ‘patroni.dynamic.json’ file located in local_configuration[‘postgresql’][‘data_dir’] directory. This is necessary to be able to restoredynamic_configuration
if DCS was accidentally wiped. - Loading of configuration file in the old format and converting it into new format.
- Mimicking some
dict
interfaces to make it possible to work with it as with the oldconfig
object.
Variables: - PATRONI_CONFIG_VARIABLE – name of the environment variable that can be used to load Patroni configuration from.
- __CACHE_FILENAME – name of the file used to cache dynamic configuration under Postgres data directory.
- __DEFAULT_CONFIG – default configuration values for some Patroni settings.
-
PATRONI_CONFIG_VARIABLE
= 'PATRONI_CONFIGURATION'¶
-
_Config__CACHE_FILENAME
= 'patroni.dynamic.json'¶
-
_Config__DEFAULT_CONFIG
= {'loop_wait': 10, 'postgresql': {'parameters': <CaseInsensitiveDict{'wal_level': 'hot_standby', 'hot_standby': 'on', 'max_connections': 100, 'max_wal_senders': 10, 'max_prepared_transactions': 0, 'max_locks_per_transaction': 64, 'track_commit_timestamp': 'off', 'max_replication_slots': 10, 'max_worker_processes': 8, 'wal_log_hints': 'on'} at 7f011ec27970>, 'use_slots': True}, 'retry_timeout': 10, 'standby_cluster': {'archive_cleanup_command': '', 'create_replica_methods': '', 'host': '', 'port': '', 'primary_slot_name': '', 'recovery_min_apply_delay': '', 'restore_command': ''}, 'ttl': 30}¶
-
_Config__get_and_maybe_adjust_int_value
(config: Dict[str, Any], param: str, min_value: int) → int¶ Get, validate and maybe adjust a param integer value from the config
dict
.Parameters: - config –
dict
object with new global configuration. - param – name of the configuration parameter we want to read/validate/adjust.
- min_value – the minimum possible value that a given param could have.
Returns: an integer value which corresponds to a provided param.
- config –
-
__init__
(configfile: str, validator: Optional[Callable[[Dict[str, Any]], List[str]]] = <function default_validator>) → None¶ Create a new instance of
Config
and validate the loaded configuration using validator.Note
Patroni will read configuration from these locations in this order:
- file or directory path passed as command-line argument (configfile), if it exists and the file or
files found in the directory can be parsed (see
_load_config_path()
), otherwise - YAML file passed via the environment variable (see
PATRONI_CONFIG_VARIABLE
), if the referenced file exists and can be parsed, otherwise - from configuration values defined as environment variables, see
_build_environment_configuration()
.
Parameters: - configfile – path to Patroni configuration file.
- validator – function used to validate Patroni configuration. It should receive a dictionary which represents Patroni configuration, and return a list of zero or more error messages based on validation.
Raises: ConfigParseError
: if any issue is reported by validator.- file or directory path passed as command-line argument (configfile), if it exists and the file or
files found in the directory can be parsed (see
-
_build_effective_configuration
(dynamic_configuration: Dict[str, Any], local_configuration: Dict[str, Union[Dict[str, Any], Any]]) → Dict[str, Any]¶ Build effective configuration by merging dynamic_configuration and local_configuration.
Note
local_configuration takes precedence over dynamic_configuration if a setting is defined in both.
Parameters: - dynamic_configuration – Patroni dynamic configuration.
- local_configuration – Patroni local configuration.
Returns: _description_
-
static
_build_environment_configuration
() → Dict[str, Any]¶ Get local configuration settings that were specified through environment variables.
Returns: dictionary containing the found environment variables and their values, respecting the expected structure of Patroni configuration.
-
_load_cache
() → None¶ Load dynamic configuration from
patroni.dynamic.json
.
-
_load_config_file
() → Dict[str, Any]¶ Load configuration file(s) from filesystem and apply values which were set via environment variables.
Returns: final configuration after merging configuration file(s) and environment variables.
-
_load_config_path
(path: str) → Dict[str, Any]¶ Load Patroni configuration file(s) from path.
If path is a file, load the yml file pointed to by path. If path is a directory, load all yml files in that directory in alphabetical order.
Parameters: path – path to either an YAML configuration file, or to a folder containing YAML configuration files. Returns: configuration after reading the configuration file(s) from path. Raises: ConfigParseError
: if path is invalid.
-
static
_process_postgresql_parameters
(parameters: Dict[str, Any], is_local: bool = False) → Dict[str, Any]¶ Process Postgres parameters.
Note
If is_local configuration discard any setting from parameters that is listed under
CMDLINE_OPTIONS
as those are supposed to be set only through dynamic configuration.When setting parameters from
CMDLINE_OPTIONS
through dynamic configuration their value will be validated as per the validator defined in that very same attribute entry. If the given value cannot be validated, a warning will be logged and the default value of the GUC will be used instead.Some parameters from
CMDLINE_OPTIONS
cannot be set even if not is_local configuration:listen_addresses
: inferred frompostgresql.listen
local configuration or fromPATRONI_POSTGRESQL_LISTEN
environment variable;
port
: inferred frompostgresql.listen
local configuration or fromPATRONI_POSTGRESQL_LISTEN
environment variable;
cluster_name
: set throughscope
local configuration or throughPATRONI_SCOPE
environment- variable;
hot_standby
: always enabled;wal_log_hints
: always enabled.
Parameters: - parameters – Postgres parameters to be processed. Should be the parsed YAML value of
postgresql.parameters
configuration, either from local or from dynamic configuration. - is_local – should be
True
if parameters refers to local configuration, orFalse
if parameters refers to dynamic configuration.
Returns: new value for
postgresql.parameters
after processing and validating parameters.
-
_safe_copy_dynamic_configuration
(dynamic_configuration: Dict[str, Any]) → Dict[str, Any]¶ Create a copy of dynamic_configuration.
Merge dynamic_configuration with
__DEFAULT_CONFIG
(dynamic_configuration takes precedence), and processpostgresql.parameters
from dynamic_configuration through_process_postgresql_parameters()
, if present.Note
The following settings are not allowed in
postgresql
section as they are intended to be local configuration, and are removed if present:connect_address
;proxy_address
;listen
;config_dir
;data_dir
;pgpass
;authentication
;
Besides that any setting present in dynamic_configuration but absent from
__DEFAULT_CONFIG
is discarded.Parameters: dynamic_configuration – Patroni dynamic configuration. Returns: copy of dynamic_configuration, merged with default dynamic configuration and with some sanity checks performed over it.
-
_validate_and_adjust_timeouts
(config: Dict[str, Any]) → None¶ Validate and adjust
loop_wait
,retry_timeout
, andttl
values if necessary.Minimum values:
loop_wait
: 1 second;retry_timeout
: 3 seconds.ttl
: 20 seconds;
Maximum values: In case if values don’t fulfill the following rule,
retry_timeout
andloop_wait
are reduced so that the rule is fulfilled:loop_wait + 2 * retry_timeout <= ttl
Parameters: config – dict
object with new global configuration.
Check
nofailover
/failover_priority
config and warn user if it’s contradictory.Note
To preserve sanity (and backwards compatibility) the
nofailover
tag will still exist. A contradictory configuration is one wherenofailover
isTrue
butfailover_priority > 0
, or wherenofailover
isFalse
, butfailover_priority <= 0
. Essentially,nofailover
andfailover_priority
are communicating different things. This checks for this edge case (which is a misconfiguration on the part of the user) and warns them. The behaviour is as iffailover_priority
were not provided (i.enofailover
is the bedrock source of truth)
-
config_file
¶ Path to Patroni configuration file, if any, else
None
.
-
copy
() → Dict[str, Any]¶ Get a deep copy of effective Patroni configuration.
Returns: a deep copy of the Patroni configuration.
-
dynamic_configuration
¶ Deep copy of cached Patroni dynamic configuration.
-
get
(key: str, default: Optional[Any] = None) → Any¶ Get effective value of
key
setting from Patroni configuration root.Designed to work the same way as
dict.get()
.Parameters: - key – name of the setting.
- default – default value if key is not present in the effective configuration.
Returns: value of key, if present in the effective configuration, otherwise default.
-
classmethod
get_default_config
() → Dict[str, Any]¶ Deep copy default configuration.
Returns: copy of __DEFAULT_CONFIG
-
get_global_config
(cluster: Optional[patroni.dcs.Cluster]) → patroni.config.GlobalConfig¶ Instantiate
GlobalConfig
based on input.Use the configuration from provided cluster (the most up-to-date) or from the local cache if cluster.config is not initialized or doesn’t have a valid config.
Parameters: cluster – the currently known cluster state from DCS. Returns: GlobalConfig
object.
-
local_configuration
¶ Deep copy of cached Patroni local configuration.
Returns: copy of _local_configuration
-
reload_local_configuration
() → Optional[bool]¶ Reload configuration values from the configuration file(s).
Note
Designed to be used when user applies changes to configuration file(s), so Patroni can use the new values with a reload instead of a restart.
Returns: True
if changes have been detected between current local configuration
-
save_cache
() → None¶ Save dynamic configuration to
patroni.dynamic.json
under Postgres data directory.Note
patroni.dynamic.jsonXXXXXX
is created as a temporary file and than renamed topatroni.dynamic.json
, whereXXXXXX
is a random suffix.
-
set_dynamic_configuration
(configuration: Union[patroni.dcs.ClusterConfig, Dict[str, Any]]) → bool¶ Set dynamic configuration values with given configuration.
Parameters: configuration – new dynamic configuration values. Supports dict
for backward compatibility.Returns: True
if changes have been detected between current dynamic configuration and the new dynamic configuration,False
otherwise.
- Building and giving access to
-
class
patroni.config.
GlobalConfig
(config: Dict[str, Any])¶ Bases:
object
A class that wraps global configuration and provides convenient methods to access/check values.
It is instantiated either by calling
get_global_config()
orConfig.get_global_config()
, which picks either a configuration from providedCluster
object (the most up-to-date) or from the local cache ifClusterConfig
is not initialized or doesn’t have a valid config.-
__init__
(config: Dict[str, Any]) → None¶ Initialize
GlobalConfig
object with given config.Parameters: config – current configuration either from ClusterConfig
or fromConfig.dynamic_configuration()
.
-
check_mode
(mode: str) → bool¶ Checks whether the certain parameter is enabled.
Parameters: mode – parameter name, e.g. synchronous_mode
,failsafe_mode
,pause
,check_timeline
, and so on.Returns: True
if parameter mode is enabled in the global configuration.
-
get
(name: str) → Any¶ Gets global configuration value by name.
Parameters: name – parameter name. Returns: configuration value or None
if it is missing.
-
get_int
(name: str, default: int = 0) → int¶ Gets current value of name from the global configuration and try to return it as
int
.Parameters: - name – name of the parameter.
- default – default value if name is not in the configuration or invalid.
Returns: currently configured value of name from the global configuration or default if it is not set or invalid.
-
get_standby_cluster_config
() → Union[Dict[str, Any], Any]¶ Get
standby_cluster
configuration.Returns: a copy of standby_cluster
configuration.
-
is_paused
¶ True
if cluster is in maintenance mode.
-
is_standby_cluster
¶ True
if global configuration has a validstandby_cluster
section.
-
is_synchronous_mode
¶ True
if synchronous replication is requested and it is not a standby cluster config.
-
is_synchronous_mode_strict
¶ True
if at least one synchronous node is required.
-
maximum_lag_on_failover
¶ Currently configured value of
maximum_lag_on_failover
from the global configuration.Assume
1048576
if it is not set or invalid.
-
maximum_lag_on_syncnode
¶ Currently configured value of
maximum_lag_on_syncnode
from the global configuration.Assume
-1
if it is not set or invalid.
-
min_synchronous_nodes
¶ The minimal number of synchronous nodes based on whether
synchronous_mode_strict
is enabled or not.
-
primary_start_timeout
¶ Currently configured value of
primary_start_timeout
from the global configuration.Assume
300
if it is not set or invalid.Note
master_start_timeout
is still supported to keep backward compatibility.
-
primary_stop_timeout
¶ Currently configured value of
primary_stop_timeout
from the global configuration.Assume
0
if it is not set or invalid.Note
master_stop_timeout
is still supported to keep backward compatibility.
-
synchronous_node_count
¶ Currently configured value of
synchronous_node_count
from the global configuration.Assume
1
if it is not set or invalid.
-
-
patroni.config.
default_validator
(conf: Dict[str, Any]) → List[str]¶ Ensure conf is not empty.
Designed to be used as default validator for
Config
objects, if no specific validator is provided.Parameters: conf – configuration to be validated. Returns: an empty list – Config
expects the validator to return a list of 0 or more issues found while validating the configuration.Raises: ConfigParseError
: if conf is empty.
-
patroni.config.
get_global_config
(cluster: Optional[patroni.dcs.Cluster], default: Optional[Dict[str, Any]] = None) → patroni.config.GlobalConfig¶ Instantiates
GlobalConfig
based on the input.Parameters: - cluster – the currently known cluster state from DCS.
- default – default configuration, which will be used if there is no valid cluster.config.
Returns: GlobalConfig
object.