From 174c480508ac25568561443e6d4a82d5c1103487 Mon Sep 17 00:00:00 2001 From: Robert Haas Date: Wed, 20 Dec 2023 08:41:09 -0500 Subject: Add a new WAL summarizer process. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When active, this process writes WAL summary files to $PGDATA/pg_wal/summaries. Each summary file contains information for a certain range of LSNs on a certain TLI. For each relation, it stores a "limit block" which is 0 if a relation is created or destroyed within a certain range of WAL records, or otherwise the shortest length to which the relation was truncated during that range of WAL records, or otherwise InvalidBlockNumber. In addition, it stores a list of blocks which have been modified during that range of WAL records, but excluding blocks which were removed by truncation after they were modified and never subsequently modified again. In other words, it tells us which blocks need to copied in case of an incremental backup covering that range of WAL records. But this doesn't yet add the capability to actually perform an incremental backup; the next patch will do that. A new parameter summarize_wal enables or disables this new background process. The background process also automatically deletes summary files that are older than wal_summarize_keep_time, if that parameter has a non-zero value and the summarizer is configured to run. Patch by me, with some design help from Dilip Kumar and Andres Freund. Reviewed by Matthias van de Meent, Dilip Kumar, Jakub Wartak, Peter Eisentraut, and Álvaro Herrera. Discussion: http://postgr.es/m/CA+TgmoYOYZfMCyOXFyC-P+-mdrZqm5pP2N7S-r0z3_402h9rsA@mail.gmail.com --- doc/src/sgml/config.sgml | 61 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) (limited to 'doc/src') diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 44cada2b403..ee985850275 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -4150,6 +4150,67 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows + + WAL Summarization + + + + + + summarize_wal (boolean) + + summarize_wal configuration parameter + + + + + Enables the WAL summarizer process. Note that WAL summarization can + be enabled either on a primary or on a standby. WAL summarization + cannot be enabled when wal_level is set to + minimal. This parameter can only be set in the + postgresql.conf file or on the server command line. + The default is off. + + + + + + wal_summary_keep_time (boolean) + + wal_summary_keep_time configuration parameter + + + + + Configures the amount of time after which the WAL summarizer + automatically removes old WAL summaries. The file timestamp is used to + determine which files are old enough to remove. Typically, you should set + this comfortably higher than the time that could pass between a backup + and a later incremental backup that depends on it. WAL summaries must + be available for the entire range of WAL records between the preceding + backup and the new one being taken; if not, the incremental backup will + fail. If this parameter is set to zero, WAL summaries will not be + automatically deleted, but it is safe to manually remove files that you + know will not be required for future incremental backups. + This parameter can only be set in the + postgresql.conf file or on the server command line. + The default is 10 days. If summarize_wal = off, + existing WAL summaries will not be removed regardless of the value of + this parameter, because the WAL summarizer will not run. + + + + + + + + -- cgit v1.2.3