summaryrefslogtreecommitdiff
path: root/src/bin/pg_basebackup
AgeCommit message (Collapse)Author
2020-11-09Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 2ffedf5ea37677f39cdc1eb92a1e78762cd3fb0e
2020-11-07Fix redundant error messages in client toolsPeter Eisentraut
A few client tools duplicate error messages already provided by libpq. Discussion: https://www.postgresql.org/message-id/flat/3e937641-88a1-e697-612e-99bba4b8e5e4%40enterprisedb.com
2020-09-14Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 00c0d74fc1f1f2a831077fdf3655c6ae5eeceac3
2020-08-18Avoid non-constant format string argument to fprintf().Heikki Linnakangas
As Tom Lane pointed out, it could defeat the compiler's printf() format string verification. Backpatch to v12, like that patch that introduced it. Discussion: https://www.postgresql.org/message-id/1069283.1597672779%40sss.pgh.pa.us
2020-08-17Fix printing last progress report line in client programs.Heikki Linnakangas
A number of client programs have a "--progress" option that when printing to a TTY, updates the current line by printing a '\r' and overwriting it. After the last line, '\n' needs to be printed to move the cursor to the next line. pg_basebackup and pgbench got this right, but pg_rewind and pg_checksums were slightly wrong. pg_rewind printed the newline to stdout instead of stderr, and pg_checksums printed the newline even when not printing to a TTY. Fix them, and also add a 'finished' argument to pg_basebackup's progress_report() function, to keep it consistent with the other programs. Backpatch to v12. pg_rewind's newline was broken with the logging changes in commit cc8d415117 in v12, and pg_checksums was introduced in v12. Discussion: https://www.postgresql.org/message-id/82b539e5-ae33-34b0-1aee-22b3379fd3eb@iki.fi
2020-08-10Move connect.h from fe_utils to src/include/common.Noah Misch
Any libpq client can use the header. Clients include backend components postgres_fdw, dblink, and logical replication apply worker. Back-patch to v10, because another fix needs this. In released branches, just copy the header and keep the original.
2020-08-10Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 42620448109473e0d2271f0f0015d3647fbbfff6
2020-06-19Ensure write failure reports no-disk-spaceAlvaro Herrera
A few places calling fwrite and gzwrite were not setting errno to ENOSPC when reporting errors, as is customary; this led to some failures being reported as "could not write file: Success" which makes us look silly. Make a few of these places in pg_dump and pg_basebackup use our customary pattern. Backpatch-to: 9.5 Author: Justin Pryzby <pryzby@telsasoft.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/20200611153753.GU14879@telsasoft.com
2020-06-06Add missing source files to nls.mkPeter Eisentraut
2020-05-18Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 031ca65d7825c3e539a3e62ea9d6630af12e6b6b
2020-05-14Initial pgindent and pgperltidy run for v13.Tom Lane
Includes some manual cleanup of places that pgindent messed up, most of which weren't per project style anyway. Notably, it seems some people didn't absorb the style rules of commit c9d297751, because there were a bunch of new occurrences of function calls with a newline just after the left paren, all with faulty expectations about how the rest of the call would get indented.
2020-05-13Fix pg_recvlogical avoidance of superfluous Standby Status Update.Noah Misch
The defect suppressed a Standby Status Update message when bytes flushed to disk had changed but bytes received had not changed. If pg_recvlogical then exited with no intervening Standby Status Update, the next pg_recvlogical repeated already-flushed records. The defect could also cause superfluous messages, which are functionally harmless. Back-patch to 9.5 (all supported versions). Discussion: https://postgr.es/m/20200502221647.GA3941274@rfd.leadboat.com
2020-05-13In successful pg_recvlogical, end PGRES_COPY_OUT cleanly.Noah Misch
pg_recvlogical merely called PQfinish(), so the backend sent messages after the disconnect. When that caused EPIPE in internal_flush(), before a LogicalConfirmReceivedLocation(), the next pg_recvlogical would repeat already-acknowledged records. Whether or not the defect causes EPIPE, post-disconnect messages could contain an ErrorResponse that the user should see. One properly ends PGRES_COPY_OUT by repeating PQgetCopyData() until it returns a negative value. Augment one of the tests to cover the case of WAL past --endpos. Back-patch to v10, where commit 7c030783a5bd07cadffc2a1018bc33119a4c7505 first appeared. Before that commit, pg_recvlogical never reached PGRES_COPY_OUT. Reported by Thomas Munro. Discussion: https://postgr.es/m/CAEepm=1MzM2Z_xNe4foGwZ1a+MO_2S9oYDq3M5D11=JDU_+0Nw@mail.gmail.com
2020-05-11Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 80d8f54b3c5533ec036404bd3c3b24ff4825d037
2020-05-01Put new command-line options into alphabetical order in help outputPeter Eisentraut
2020-04-27Fix bogus tar-file padding logic for standby.signal.Robert Haas
When pg_basebackup -R is used, we inject standby.signal into the tar file for the main tablespace. The proper thing to do is to pad each file injected into the tar file out to a 512-byte boundary by appending nulls, but here the file is of length 0 and we add 511 zero bytes. Since 0 is already a multiple of 512, we should not add any zero bytes. Do that instead. Patch by me, reviewed by Tom Lane. Discussion: http://postgr.es/m/CA+TgmobWbfReO9-XFk8urR1K4wTNwqoHx_v56t7=T8KaiEoKNw@mail.gmail.com
2020-04-17Fix minor memory leak in pg_basebackup and pg_receivewalMichael Paquier
The result of the query used to retrieve the WAL segment size from the backend was not getting freed in two code paths. Both pg_basebackup and pg_receivewal exit immediately if a failure happened on this query, so this was not an actual problem, but it could be an issue if this code gets used for other tools in different ways, be they future tools in this code tree or external, existing, ones. Oversight in commit fc49e24, so backpatch down to 11. Author: Jie Zhang Discussion: https://postgr.es/m/970ad9508461469b9450b64027842331@G08CNEXMBPEKD06.g08.fujitsu.local Backpatch-through: 11
2020-04-16Disable silently generation of manifests with servers <= 12 in pg_basebackupMichael Paquier
Since 0d8c9c1, pg_basebackup would generate an error if connected to a backend version older than 12 where backup manifests are not supported. Avoiding this error is possible by using the --no-manifest option. This error handling could be confusing for some users, where patching a backup script that interacts with multiple backend versions would cause the addition of --no-manifest to potentially not generate a backup manifest even for Postgres 13 and newer versions. As we want to encourage the use of backup manifests as much as possible, this commit silently disables manifests where not supported, instead of generating an error. While on it, rework a bit the code to make it more consistent with the surroundings when generating the BASE_BACKUP command. Per discussion with Andres Freund, Stephen Frost, Robert Haas, Álvaro Herrera, Kyotaro Horiguchi, Tom Lane, David Steele, and me. Author: Michael Paquier Discussion: https://postgr.es/m/20200410080910.GZ1606@paquier.xyz
2020-04-15Code review for backup manifest.Fujii Masao
This commit prevents pg_basebackup from receiving backup_manifest file when --no-manifest is specified. Previously, when pg_basebackup was writing a tarfile to stdout, it tried to receive backup_manifest file even when --no-manifest was specified, and reported an error. Also remove unused -m option from pg_basebackup. Also fix typo in BASE_BACKUP command documentation. Author: Fujii Masao Reviewed-by: Michael Paquier, Robert Haas Discussion: https://postgr.es/m/01e3ed3a-8729-5aaa-ca84-e60e3ca59db8@oss.nttdata.com
2020-04-03Generate backup manifests for base backups, and validate them.Robert Haas
A manifest is a JSON document which includes (1) the file name, size, last modification time, and an optional checksum for each file backed up, (2) timelines and LSNs for whatever WAL will need to be replayed to make the backup consistent, and (3) a checksum for the manifest itself. By default, we use CRC-32C when checksumming data files, because we are trying to detect corruption and user error, not foil an adversary. However, pg_basebackup and the server-side BASE_BACKUP command now have options to select a different algorithm, so users wanting a cryptographic hash function can select SHA-224, SHA-256, SHA-384, or SHA-512. Users not wanting file checksums at all can disable them, or disable generating of the backup manifest altogether. Using a cryptographic hash function in place of CRC-32C consumes significantly more CPU cycles, which may slow down backups in some cases. A new tool called pg_validatebackup can validate a backup against the manifest. If no checksums are present, it can still check that the right files exist and that they have the expected sizes. If checksums are present, it can also verify that each file has the expected checksum. Additionally, it calls pg_waldump to verify that the expected WAL files are present and parseable. Only plain format backups can be validated directly, but tar format backups can be validated after extracting them. Robert Haas, with help, ideas, review, and testing from David Steele, Stephen Frost, Andrew Dunstan, Rushabh Lathia, Suraj Kharage, Tushar Ahuja, Rajkumar Raghuwanshi, Mark Dilger, Davinder Singh, Jeevan Chalke, Amit Kapila, Andres Freund, and Noah Misch. Discussion: http://postgr.es/m/CA+TgmoZV8dw1H2bzZ9xkKwdrk8+XYa+DC9H=F7heO2zna5T6qg@mail.gmail.com
2020-03-19Make pg_basebackup ask the server to estimate the total backup size, by default.Fujii Masao
This commit changes pg_basebackup so that it specifies PROGRESS option in BASE_BACKUP replication command whether --progress is specified or not. This causes the server to estimate the total backup size and report it in pg_stat_progress_basebackup.backup_total, by default. This is reasonable default because the time required for the estimation would not be so large in most cases. Also this commit adds new option --no-estimate-size to pg_basebackup. This option prevents the server from the estimation, and so is useful to avoid such estimation time if it's too long. Author: Fujii Masao Reviewed-by: Magnus Hagander, Amit Langote Discussion: https://postgr.es/m/CABUevEyDPPSjP7KRvfTXPdqOdY5aWNkqsB5aAXs3bco5ZwtGHg@mail.gmail.com
2020-02-28Add PostgreSQL home page to --help outputPeter Eisentraut
Per emerging standard in GNU programs and elsewhere. Autoconf already has support for specifying a home page, so we can just that. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Discussion: https://www.postgresql.org/message-id/flat/8d389c5f-7fb5-8e48-9a4a-68cec44786fa%402ndquadrant.com
2020-02-28Refer to bug report address by symbol rather than hardcodingPeter Eisentraut
Use the PACKAGE_BUGREPORT macro that is created by Autoconf for referring to the bug reporting address rather than hardcoding it everywhere. This makes it easier to change the address and it reduces translation work. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Discussion: https://www.postgresql.org/message-id/flat/8d389c5f-7fb5-8e48-9a4a-68cec44786fa%402ndquadrant.com
2020-02-24Add prefix checks in exclude lists for pg_rewind, pg_checksums and base backupsMichael Paquier
An instance of PostgreSQL crashing with a bad timing could leave behind temporary pg_internal.init files, potentially causing failures when verifying checksums. As the same exclusion lists are used between pg_rewind, pg_checksums and basebackup.c, all those tools are extended with prefix checks to keep everything in sync, with dedicated checks added for pg_internal.init. Backpatch down to 11, where pg_checksums (pg_verify_checksums in 11) and checksum verification for base backups have been introduced. Reported-by: Michael Banck Author: Michael Paquier Reviewed-by: Kyotaro Horiguchi, David Steele Discussion: https://postgr.es/m/62031974fd8e941dd8351fbc8c7eff60d59c5338.camel@credativ.de Backpatch-through: 11
2020-02-07Revert "Prevent running pg_basebackup as root"Michael Paquier
This reverts commit 7bae0ad, as this is not ideal with the tar format, and we may want to explore more options like what is done by tar with some equivalents of --owner and --group, but for pg_basebackup. Per complaints from Magnus Hagander and Stephen Frost. Discussion: https://postgr.es/m/20200205172259.GW3195@tamriel.snowman.net
2020-02-04Fix fuzzy error handling in pg_basebackup when opening gzFileMichael Paquier
First, this code did not bother checking for a failure when calling dup(). Then, per zlib, gzerror() returns NULL for a NULL input, which can happen if passing down to gzdopen() an invalid file descriptor or if there was an allocation failure. No back-patch is done as this would unlikely be a problem in the field. Per Coverity. Reported-by: Tom Lane
2020-02-01Prevent running pg_basebackup as rootMichael Paquier
Similarly to pg_upgrade, pg_ctl and initdb, a root user is able to use --version and --help, but cannot execute the actual operation to avoid the creation of files with permissions incompatible with the postmaster. This is a behavior change, so not back-patching is done. Author: Ian Barwick Discussion: https://postgr.es/m/CABvVfJVqOdD2neLkYdygdOHvbWz_5K_iWiqY+psMfA=FeAa3qQ@mail.gmail.com
2020-01-01Update copyrights for 2020Bruce Momjian
Backpatch-through: update all files in master, backpatch legal files through 9.4
2019-12-05pg_basebackup: Refactor code for reading COPY and tar data.Robert Haas
Add a new function ReceiveCopyData that does just that, taking a callback as an argument to specify what should be done with each chunk as it is received. This allows a single copy of the logic to be shared between ReceiveTarFile and ReceiveAndUnpackTarFile, and eliminates a few #ifdef conditions based on HAVE_LIBZ. While this is slightly more code, it's arguably clearer, and there is a pending patch that introduces additional calls to ReceiveCopyData. This commit is not intended to result in any functional change. Discussion: http://postgr.es/m/CA+TgmoYZDTHbSpwZtW=JDgAhwVAYvmdSrRUjOd+AYdfNNXVBDg@mail.gmail.com
2019-11-25Make the order of the header file includes consistent.Amit Kapila
Similar to commits 14aec03502, 7e735035f2 and dddf4cdc33, this commit makes the order of header file inclusion consistent in more places. Author: Vignesh C Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CALDaNm2Sznv8RR6Ex-iJO6xAdsxgWhCoETkaYX=+9DW3q0QCfA@mail.gmail.com
2019-11-05Split all OBJS style lines in makefiles into one-line-per-entry style.Andres Freund
When maintaining or merging patches, one of the most common sources for conflicts are the list of objects in makefiles. Especially when the split across lines has been changed on both sides, which is somewhat common due to attempting to stay below 80 columns, those conflicts are unnecessarily laborious to resolve. By splitting, and alphabetically sorting, OBJS style lines into one object per line, conflicts should be less frequent, and easier to resolve when they still occur. Author: Andres Freund Discussion: https://postgr.es/m/20191029200901.vww4idgcxv74cwes@alap3.anarazel.de
2019-10-25Make the order of the header file includes consistent in non-backend modules.Amit Kapila
Similar to commit 7e735035f2, this commit makes the order of header file inclusion consistent for non-backend modules. In passing, fix the case where we were using angle brackets (<>) for the local module includes instead of quotes (""). Author: Vignesh C Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CALDaNm2Sznv8RR6Ex-iJO6xAdsxgWhCoETkaYX=+9DW3q0QCfA@mail.gmail.com
2019-09-25Split out recovery confing-writing code from pg_basebackupAlvaro Herrera
... into a new file, fe_utils/recovery_gen.c. This can later be used by pg_rewind. Authors: Paul Guo, Jimmy Yih, Ashwin Agrawal. A few tweaks by Álvaro Herrera Reviewed-by: Michaël Paquier Discussion: https://postgr.es/m/CAEET0ZEffUkXc48pg2iqARQgGRYDiiVxDu+yYek_bTwJF+q=Uw@mail.gmail.com
2019-09-04Delay fsyncs of pg_basebackup until the end of backupMichael Paquier
Since the addition of fsync requests in bc34223 to make base backup data consistent on disk once pg_basebackup finishes, each tablespace tar file is individually flushed once completed, with an additional flush of the parent directory when the base backup finishes. While holding a connection to the server, a fsync request taking a long time may cause a failure of the base backup, which is annoying for any integration. A recent example of breakage can involve tcp_user_timeout, but wal_sender_timeout can cause similar problems. While reviewing the code, there was a second issue causing too many fsync requests to be done for the same WAL data. As recursive fsyncs are done at the end of the backup for both the plain and tar formats from the base target directory where everything is written, it is fine to disable fsyncs when fetching or streaming WAL. Reported-by: Ryohei Takahashi Author: Michael Paquier Reviewed-by: Ryohei Takahashi Discussion: https://postgr.es/m/OSBPR01MB4550DAE2F8C9502894A45AAB82BE0@OSBPR01MB4550.jpnprd01.prod.outlook.com Backpatch-through: 10
2019-09-03Improve base backup protocol documentationPeter Eisentraut
Document that the tablespace sizes are in units of kilobytes. Make the pg_basebackup source code a bit clearer about this, too. Reviewed-by: Magnus Hagander <magnus@hagander.net>
2019-08-19Fix inconsistencies and typos in the tree, take 11Michael Paquier
This fixes various typos in docs and comments, and removes some orphaned definitions. Author: Alexander Lakhin Discussion: https://postgr.es/m/5da8e325-c665-da95-21e0-c8a99ea61fbf@gmail.com
2019-07-29Handle fsync failures in pg_receivewal and pg_recvlogicalPeter Eisentraut
It is not safe to simply report an fsync error and continue. We must exit the program instead. Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Sehrope Sarkuni <sehrope@jackdb.com> Discussion: https://www.postgresql.org/message-id/flat/9b49fe44-8f3e-eca9-5914-29e9e99030bf@2ndquadrant.com
2019-07-26Don't uselessly escape a string that doesn't need escapingAlvaro Herrera
Per gripe from Ian Barwick Co-authored-by: Ian Barwick <ian@2ndquadrant.com> Discussion: https://postgr.es/m/CABvVfJWNnNKb8cHsTLhkTsvL1+G6BVcV+57+w1JZ61p8YGPdWQ@mail.gmail.com
2019-07-21Fix compilation warning of pg_basebackup with MinGWMichael Paquier
Several buildfarm members have been complaining about that with gcc, like jacana. Weirdly enough, Visual Studio's compilers do not find this issue. Author: Michael Paquier Reviewed-by: Andrew Dunstan Discussion: https://postgr.es/m/20190719050830.GK1859@paquier.xyz
2019-07-09Message style improvementsPeter Eisentraut
2019-07-04Use appendStringInfoString and appendPQExpBufferStr where possibleDavid Rowley
This changes various places where appendPQExpBuffer was used in places where it was possible to use appendPQExpBufferStr, and likewise for appendStringInfo and appendStringInfoString. This is really just a stylistic improvement, but there are also small performance gains to be had from doing this. Discussion: http://postgr.es/m/CAKJS1f9P=M-3ULmPvr8iCno8yvfDViHibJjpriHU8+SXUgeZ=w@mail.gmail.com
2019-07-02Remove redundant newlines from error messagesPeter Eisentraut
These are no longer needed/allowed with the new logging API.
2019-06-17Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 1a710c413ce4c4cd081843e563cde256bb95f490
2019-06-08Update stale comments, and fix comment typos.Noah Misch
2019-06-03Fix typos in various placesMichael Paquier
Author: Andrea Gelmini Reviewed-by: Michael Paquier, Justin Pryzby Discussion: https://postgr.es/m/20190528181718.GA39034@glet
2019-05-22Initial pgperltidy run for v12.Tom Lane
Make all the perl code look nice, too (for some value of "nice").
2019-05-22Phase 2 pgindent run for v12.Tom Lane
Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22Initial pgindent run for v12.Tom Lane
This is still using the 2.0 version of pg_bsd_indent. I thought it would be good to commit this separately, so as to document the differences between 2.0 and 2.1 behavior. Discussion: https://postgr.es/m/16296.1558103386@sss.pgh.pa.us
2019-05-20Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: a20bf6b8a5b4e32450967055eb5b07cee4704edd
2019-05-14Move logging.h and logging.c from src/fe_utils/ to src/common/.Tom Lane
The original placement of this module in src/fe_utils/ is ill-considered, because several src/common/ modules have dependencies on it, meaning that libpgcommon and libpgfeutils now have mutual dependencies. That makes it pointless to have distinct libraries at all. The intended design is that libpgcommon is lower-level than libpgfeutils, so only dependencies from the latter to the former are acceptable. We already have the precedent that fe_memutils and a couple of other modules in src/common/ are frontend-only, so it's not stretching anything out of whack to treat logging.c as a frontend-only module in src/common/. To the extent that such modules help provide a common frontend/backend environment for the rest of common/ to use, it's a reasonable design. (logging.c does not yet provide an ereport() emulation, but one can dream.) Hence, move these files over, and revert basically all of the build-system changes made by commit cc8d41511. There are no places that need to grow new dependencies on libpgcommon, further reinforcing the idea that this is the right solution. Discussion: https://postgr.es/m/a912ffff-f6e4-778a-c86a-cf5c47a12933@2ndquadrant.com
generated by cgit v1.2.3 (git 2.46.0) at 2025-10-25 16:40:29 +0000