From e65497df8f85ab9b9084c928ff69f384ea729b24 Mon Sep 17 00:00:00 2001 From: Fujii Masao Date: Tue, 3 Mar 2020 12:03:43 +0900 Subject: Report progress of streaming base backup. This commit adds pg_stat_progress_basebackup view that reports the progress while an application like pg_basebackup is taking a base backup. This uses the progress reporting infrastructure added by c16dc1aca5e0, adding support for streaming base backup. Bump catversion. Author: Fujii Masao Reviewed-by: Kyotaro Horiguchi, Amit Langote, Sergei Kornilov Discussion: https://postgr.es/m/9ed8b801-8215-1f3d-62d7-65bff53f6e94@oss.nttdata.com --- doc/src/sgml/monitoring.sgml | 163 +++++++++++++++++++++++++++++++++++- doc/src/sgml/protocol.sgml | 2 +- doc/src/sgml/ref/pg_basebackup.sgml | 16 ++++ 3 files changed, 179 insertions(+), 2 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 87586a7b069..dd4a668eea6 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -376,6 +376,14 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser + + pg_stat_progress_basebackuppg_stat_progress_basebackup + One row for each WAL sender process streaming a base backup, + showing current progress. + See . + + + @@ -3535,7 +3543,10 @@ SELECT pg_stat_get_backend_pid(s.backendid) AS pid, certain commands during command execution. Currently, the only commands which support progress reporting are ANALYZE, CLUSTER, - CREATE INDEX, and VACUUM. + CREATE INDEX, VACUUM, + and (i.e., replication + command that issues to take + a base backup). This may be expanded in the future. @@ -4336,6 +4347,156 @@ SELECT pg_stat_get_backend_pid(s.backendid) AS pid, + + + + Base Backup Progress Reporting + + + Whenever an application like pg_basebackup + is taking a base backup, the + pg_stat_progress_basebackup + view will contain a row for each WAL sender process that is currently + running BASE_BACKUP replication command + and streaming the backup. The tables below describe the information + that will be reported and provide information about how to interpret it. + + + + <structname>pg_stat_progress_basebackup</structname> View + + + + Column + Type + Description + + + + + + pid + integer + Process ID of a WAL sender process. + + + phase + text + Current processing phase. See . + + + backup_total + bigint + + Total amount of data that will be streamed. If progress reporting + is not enabled in pg_basebackup + (i.e., --progress option is not specified), + this is 0. Otherwise, this is estimated and + reported as of the beginning of + streaming database files phase. Note that + this is only an approximation since the database + may change during streaming database files phase + and WAL log may be included in the backup later. This is always + the same value as backup_streamed + once the amount of data streamed exceeds the estimated + total size. + + + + backup_streamed + bigint + + Amount of data streamed. This counter only advances + when the phase is streaming database files or + transfering wal files. + + + + tablespaces_total + bigint + + Total number of tablespaces that will be streamed. + + + + tablespaces_streamed + bigint + + Number of tablespaces streamed. This counter only + advances when the phase is streaming database files. + + + + +
+ + + Base backup phases + + + + Phase + Description + + + + + initializing + + The WAL sender process is preparing to begin the backup. + This phase is expected to be very brief. + + + + waiting for checkpoint to finish + + The WAL sender process is currently performing + pg_start_backup to set up for + taking a base backup, and waiting for backup start + checkpoint to finish. + + + + estimating backup size + + The WAL sender process is currently estimating the total amount + of database files that will be streamed as a base backup. + + + + streaming database files + + The WAL sender process is currently streaming database files + as a base backup. + + + + waiting for wal archiving to finish + + The WAL sender process is currently performing + pg_stop_backup to finish the backup, + and waiting for all the WAL files required for the base backup + to be successfully archived. + If either --wal-method=none or + --wal-method=stream is specified in + pg_basebackup, the backup will end + when this phase is completed. + + + + transferring wal files + + The WAL sender process is currently transferring all WAL logs + generated during the backup. This phase occurs after + waiting for wal archiving to finish phase if + --wal-method=fetch is specified in + pg_basebackup. The backup will end + when this phase is completed. + + + + +
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 80275215e04..f139ba02312 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -2465,7 +2465,7 @@ The commands accepted in replication mode are: - + BASE_BACKUP [ LABEL 'label' ] [ PROGRESS ] [ FAST ] [ WAL ] [ NOWAIT ] [ MAX_RATE rate ] [ TABLESPACE_MAP ] [ NOVERIFY_CHECKSUMS ] BASE_BACKUP diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml index fc9e222f8d0..fc9ba2e8b00 100644 --- a/doc/src/sgml/ref/pg_basebackup.sgml +++ b/doc/src/sgml/ref/pg_basebackup.sgml @@ -104,6 +104,13 @@ PostgreSQL documentation + + + Whenever pg_basebackup is taking a base + backup, the pg_stat_progress_basebackup + view will report the progress of the backup. + See for details. + @@ -459,6 +466,15 @@ PostgreSQL documentation This may make the backup take slightly longer, and in particular it will take longer before the first data is sent. + + Whether this is enabled or not, the + pg_stat_progress_basebackup view + report the progress of the backup in the server side. But note + that the total amount of data that will be streamed is estimated + and reported only when this option is enabled. In other words, + backup_total column in the view always + indicates 0 if this option is disabled. + -- cgit v1.2.3