summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-07-24pg_upgrade: Retrieve subscription count more efficiently.Nathan Bossart
Presently, pg_upgrade obtains the number of subscriptions in the to-be-upgraded cluster by first querying pg_subscription in every database for the number of subscriptions in only that database. Then, in count_old_cluster_subscriptions(), it adds all the values collected in the first step. This is expensive, especially when there are many databases. Fortunately, there is a better way to retrieve the subscription count. Since pg_subscription is a shared catalog, we only need to connect to a single database and query it once. This commit modifies pg_upgrade to use that approach, which also allows us to trim several lines of code. In passing, move the call to get_db_subscription_count(), which has been renamed to get_subscription_count(), from get_db_rel_and_slot_infos() to the dedicated >= v17 section in check_and_dump_old_cluster(). We may be able to make similar improvements to get_old_cluster_logical_slot_infos(), but that is left as a future exercise. Reviewed-by: Michael Paquier, Amit Kapila Discussion: https://postgr.es/m/ZprQJv_TxccN3tkr%40nathan Backpatch-through: 17
2024-07-24Reset relhassubclass upon attaching table as a partitionAlvaro Herrera
We don't allow inheritance parents as partitions, and have checks to prevent this; but if a table _was_ in the past an inheritance parents and all their children are removed, the pg_class.relhassubclass flag may remain set, which confuses the partition pruning code (most obviously, it results in an assertion failure; in production builds it may be worse.) Fix by resetting relhassubclass on attach. Backpatch to all supported versions. Reported-by: Alexander Lakhin <exclusion@gmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/18550-d5e047e9a897a889@postgresql.org
2024-07-23Detect integer overflow in array_set_slice().Nathan Bossart
When provided an empty initial array, array_set_slice() fails to check for overflow when computing the new array's dimensions. While such overflows are ordinarily caught by ArrayGetNItems(), commands with the following form are accepted: INSERT INTO t (i[-2147483648:2147483647]) VALUES ('{}'); To fix, perform the hazardous computations using overflow-detecting arithmetic routines. As with commit 18b585155a, the added test cases generate errors that include a platform-dependent value, so we again use psql's VERBOSITY parameter to suppress printing the message text. Reported-by: Alexander Lakhin Author: Joseph Koshakow Reviewed-by: Jian He Discussion: https://postgr.es/m/31ad2cd1-db94-bdb3-f91a-65ffdb4bef95%40gmail.com Backpatch-through: 12
2024-07-23Use more consistently int64 for page numbers in SLRU-related codeMichael Paquier
clog.c, async.c and predicate.c included some SLRU page numbers still handled as 4-byte integers, while int64 should be used for this purpose. These holes have been introduced in 4ed8f0913bfd, that has introduced the use of 8-byte integers for SLRU page numbers, still forgot about the code paths updated by this commit. Reported-by: Noah Misch Author: Aleksander Alekseev, Michael Paquier Discussion: https://postgr.es/m/20240626002747.dc.nmisch@google.com Backpatch-through: 17
2024-07-23Improve comments in slru.{c,h} about segment name formatMichael Paquier
slru.h described incorrectly how SLRU segment names are formatted depending on the segment number and if long or short segment names are used. This commit closes the gap with a better description, fitting with the reality. Reported-by: Noah Misch Author: Aleksander Alekseev Discussion: https://postgr.es/m/20240626002747.dc.nmisch@google.com Backpatch-through: 17
2024-07-22Doc: improve description of plpgsql's FETCH and MOVE commands.Tom Lane
We were not being clear about which variants of the "direction" clause are permitted in MOVE. Also, the text seemed to be written with only the FETCH/MOVE NEXT case in mind, so it didn't apply very well to other variants. Also, document that "MOVE count IN cursor" only works if count is a constant. This is not the whole truth, because some other cases such as a parenthesized expression will also work, but we want to push people to use "MOVE FORWARD count" instead. The constant case is enough to cover what we allow in plain SQL, and that seems sufficient to claim support for. Update a comment in pl_gram.y claiming that we don't document that point. Per gripe from Philipp Salvisberg. Discussion: https://postgr.es/m/172155553388.702.7932496598218792085@wrigleys.postgresql.org
2024-07-22Revert "Test that vacuum removes tuples older than OldestXmin"Melanie Plageman
This reverts commit 80c34692e8e674e3b2f150f248ef2002ae2ac3a7. This test proved to be unstable on the buildfarm, timing out before the standby could catch up on 32-bit machines where more rows were required and failing to reliably trigger multiple index vacuum rounds on 64-bit machines where fewer rows should be required. Because the instability is only known to be present on versions of Postgres with TIDStore used for dead TID storage by vacuum, this is only being reverted on master and REL_17_STABLE. As having this coverage may be valuable, there is a discussion on the thread of possible ways to stabilize the test. If that happens, a fixed test can be committed again. Backpatch-through: 17 Reported-by: Tom Lane Discussion: https://postgr.es/m/614152.1721580711%40sss.pgh.pa.us
2024-07-22Initialize wal_level in the initial checkpoint record.Robert Haas
As per Coverity and Tom Lane, commit 402b586d0 (back-patched to v17 as 2b5819e2b) forgot to initialize this new structure member in this code path.
2024-07-22Add missing call to ConditionVariableCancelSleep().Robert Haas
After calling ConditionVariableSleep() or ConditionVariableTimedSleep() one or more times, code is supposed to call ConditionVariableCancelSleep() to remove itself from the waitlist. This code neglected to do so. As far as I know, that had no observable consequences, but let's make the code correct. Discussion: http://postgr.es/m/CA+TgmoYW8eR+KN6zhVH0sin7QH6AvENqw_bkN-bB4yLYKAnsew@mail.gmail.com
2024-07-20Correctly check updatability of columns targeted by INSERT...DEFAULT.Tom Lane
If a view has some updatable and some non-updatable columns, we failed to verify updatability of any columns for which an INSERT or UPDATE on the view explicitly specifies a DEFAULT item (unless the view has a declared default for that column, which is rare anyway, and one would almost certainly not write one for a non-updatable column). This would lead to an unexpected "attribute number N not found in view targetlist" error rather than the intended error. Per bug #18546 from Alexander Lakhin. This bug is old, so back-patch to all supported branches. Discussion: https://postgr.es/m/18546-84a292e759a9361d@postgresql.org
2024-07-19Add overflow checks to money type.Nathan Bossart
None of the arithmetic functions for the the money type handle overflow. This commit introduces several helper functions with overflow checking and makes use of them in the money type's arithmetic functions. Fixes bug #18240. Reported-by: Alexander Lakhin Author: Joseph Koshakow Discussion: https://postgr.es/m/18240-c5da758d7dc1ecf0%40postgresql.org Discussion: https://postgr.es/m/CAAvxfHdBPOyEGS7s%2Bxf4iaW0-cgiq25jpYdWBqQqvLtLe_t6tw%40mail.gmail.com Backpatch-through: 12
2024-07-19Test that vacuum removes tuples older than OldestXminMelanie Plageman
If vacuum fails to prune a tuple killed before OldestXmin, it will decide to freeze its xmax and later error out in pre-freeze checks. Add a test reproducing this scenario to the recovery suite which creates a table on a primary, updates the table to generate dead tuples for vacuum, and then, during the vacuum, uses a replica to force GlobalVisState->maybe_needed on the primary to move backwards and precede the value of OldestXmin set at the beginning of vacuuming the table. This commit is separate from the fix in case there are test stability issues. Author: Melanie Plageman Reviewed-by: Peter Geoghegan Discussion: https://postgr.es/m/CAAKRu_apNU2MPBK96V%2BbXjTq0RiZ-%3DA4ZTaysakpx9jxbq1dbQ%40mail.gmail.com
2024-07-19Ensure vacuum removes all visibly dead tuples older than OldestXminMelanie Plageman
If vacuum fails to remove a tuple with xmax older than VacuumCutoffs->OldestXmin and younger than GlobalVisState->maybe_needed, it may attempt to freeze the tuple's xmax and then ERROR out in pre-freeze checks with "cannot freeze committed xmax". Fix this by having vacuum always remove tuples older than OldestXmin. It is possible for GlobalVisState->maybe_needed to precede OldestXmin if maybe_needed is forced to go backward while vacuum is running. This can happen if a disconnected standby with a running transaction older than VacuumCutoffs->OldestXmin reconnects to the primary after vacuum initially calculates GlobalVisState and OldestXmin. In back branches starting with 14, the first version using GlobalVisState, failing to remove tuples older than OldestXmin during pruning caused vacuum to infinitely loop in lazy_scan_prune(), as investigated on this [1] thread. After 1ccc1e05ae removed the retry loop in lazy_scan_prune() and stopped comparing tuples to OldestXmin, the hang could no longer happen, but we could still attempt to freeze dead tuples with xmax older than OldestXmin -- resulting in an ERROR. Fix this by always removing dead tuples with xmax older than VacuumCutoffs->OldestXmin. This is okay because the standby won't replay the tuple removal until the tuple is removable. Thus, the worst that can happen is a recovery conflict. [1] https://postgr.es/m/20240415173913.4zyyrwaftujxthf2%40awork3.anarazel.de#1b216b7768b5bd577a3d3d51bd5aadee Back-patch through 14 Author: Melanie Plageman Reviewed-by: Peter Geoghegan, Robert Haas, Andres Freund, Heikki Linnakangas, and Noah Misch Discussion: https://postgr.es/m/CAAKRu_bDD7oq9ZwB2OJqub5BovMG6UjEYsoK2LVttadjEqyRGg%40mail.gmail.com
2024-07-19Move resowner from common JitContext to LLVM specificHeikki Linnakangas
Only the LLVM specific code uses it since resource owners were made extensible in commit b8bff07daa85c837a2747b4d35cd5a27e73fb7b2. This is new in v17, so backpatch there to keep the branches from diverging just yet. Author: Andreas Karlsson <andreas@proxel.se> Discussion: https://www.postgresql.org/message-id/fd3a2a00-6605-4e30-a118-48418b478e6e@proxel.se
2024-07-19Propagate query IDs of utility statements in functionsMichael Paquier
For utility statements defined within a function, the query tree is copied to a PlannedStmt as utility commands do not require planning. However, the query ID was missing from the information passed down. This leads to plugins relying on the query ID like pg_stat_statements to not be able to track utility statements within function calls. Tests are added to check this behavior, depending on pg_stat_statements.track. This is an old bug. Now, query IDs for utilities are compiled using their parsed trees rather than the query string since v16 (3db72ebcbe20), leading to less bloat with utilities, so backpatch down only to this version. Author: Anthonin Bonnefoy Discussion: https://postgr.es/m/CAO6_XqrGp-uwBqi3vBPLuRULKkddjC7R5QZCgsFren=8E+m2Sg@mail.gmail.com Backpatch-through: 16
2024-07-18Do not summarize WAL if generated with wal_level=minimal.Robert Haas
To do this, we must include the wal_level in the first WAL record covered by each summary file; so add wal_level to struct Checkpoint and the payload of XLOG_CHECKPOINT_REDO and XLOG_END_OF_RECOVERY. This, in turn, requires bumping XLOG_PAGE_MAGIC and, since the Checkpoint is also stored in the control file, also PG_CONTROL_VERSION. It's not great to do that so late in the release cycle, but the alternative seems to ship v17 without robust protections against this scenario, which could result in corrupted incremental backups. A side effect of this patch is that, when a server with wal_level=replica is started with summarize_wal=on for the first time, summarization will no longer begin with the oldest WAL that still exists in pg_wal, but rather from the first checkpoint after that. This change should be harmless, because a WAL summary for a partial checkpoint cycle can never make an incremental backup possible when it would otherwise not have been. Report by Fujii Masao. Patch by me. Review and/or testing by Jakub Wartak and Fujii Masao. Discussion: http://postgr.es/m/6e30082e-041b-4e31-9633-95a66de76f5d@oss.nttdata.com
2024-07-17Use PqMsg_* macros in more places.Nathan Bossart
Commit f4b54e1ed9, which introduced macros for protocol characters, missed updating a few places. It also did not introduce macros for messages sent from parallel workers to their leader processes. This commit adds a new section in protocol.h for those. Author: Aleksander Alekseev Discussion: https://postgr.es/m/CAJ7c6TNTd09AZq8tGaHS3LDyH_CCnpv0oOz2wN1dGe8zekxrdQ%40mail.gmail.com Backpatch-through: 17
2024-07-17Avoid error in recovery test if history file is not yet presentAndrew Dunstan
Error was detected when testing use of libpq sessions instead of psql for polling queries. Discussion: https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db886@dunslane.net Backpatch to all live branches
2024-07-17SQL/JSON: Rethink c2d93c3802bAmit Langote
This essentially reverts c2d93c3802b except tests. The problem with c2d93c3802b was that it only changed the casting behavior for types with typmod, and had coding issues noted in the post-commit review. This commit changes coerceJsonFuncExpr() to use assignment-level casts instead of explicit casts to coerce the result of JSON constructor functions to the specified or the default RETURNING type. Using assignment-level casts fixes the problem that using explicit casts was leading to the wrong typmod / length coercion behavior -- truncating results longer than the specified length instead of erroring out -- which c2d93c3802b aimed to solve. That restricts the set of allowed target types to string types, the same set that's currently allowed. Discussion: https://postgr.es/m/202406291824.reofujy7xdj3@alvherre.pgsql
2024-07-16When creating materialized views, use REFRESH to load data.Jeff Davis
Previously, CREATE MATERIALIZED VIEW ... WITH DATA populated the MV the same way as CREATE TABLE ... AS. Instead, reuse the REFRESH logic, which locks down security-restricted operations and restricts the search_path. This reduces the chance that a subsequent refresh will fail. Reported-by: Noah Misch Backpatch-through: 17 Discussion: https://postgr.es/m/20240630222344.db.nmisch@google.com
2024-07-15Fix bad indentation introduced in 43cd30bcd1cAndres Freund
Oops. Reported-by: Nathan Bossart <nathandbossart@gmail.com> Discussion: https://postgr.es/m/ZpVZB9rH5tHllO75@nathan Backpatch: 12-, like 43cd30bcd1c
2024-07-15Add missing RestrictSearchPath() calls.Jeff Davis
Reported-by: Noah Misch Backpatch-through: 17 Discussion: https://postgr.es/m/20240630222344.db.nmisch@google.com
2024-07-15Fix type confusion in guc_var_compare()Andres Freund
Before this change guc_var_compare() cast the input arguments to const struct config_generic *. That's not quite right however, as the input on one side is often just a char * on one side. Instead just use char *, the first field in config_generic. This fixes a -Warray-bounds warning with some versions of gcc. While the warning is only known to be triggered for <= 15, the issue the warning points out seems real, so apply the fix everywhere. Author: Nazir Bilal Yavuz <byavuz81@gmail.com> Reported-by: Erik Rijkers <er@xs4all.nl> Suggested-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/a74a1a0d-0fd2-3649-5224-4f754e8f91aa%40xs4all.nl
2024-07-15Use atomics to avoid locking in InjectionPointRun()Heikki Linnakangas
This allows using injection points without having a PGPROC, like early at backend startup, or in the postmaster. The injection points facility is new in v17, so backpatch there. Reviewed-by: Michael Paquier <michael@paquier.xyz> Disussion: https://www.postgresql.org/message-id/4317a7f7-8d24-435e-9e49-29b72a3dc418@iki.fi
2024-07-15Fix unstable tests in partition_merge.sql and partition_split.sql.Fujii Masao
The tests added by commit c086896625 were unstable due to missing schema names when checking pg_tables and pg_indexes. Backpatch to v17. Reported by buildfarm.
2024-07-15Fix tablespace handling in MERGE/SPLIT partition commands.Fujii Masao
As commit ca4103025d stated, new partitions without a specified tablespace should inherit the parent relation's tablespace. However, previously, ALTER TABLE MERGE PARTITIONS and ALTER TABLE SPLIT PARTITION commands always created new partitions in the default tablespace, ignoring the parent's tablespace. This commit ensures new partitions inherit the parent's tablespace. Backpatch to v17 where these commands were introduced. Author: Fujii Masao Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/abaf390b-3320-40a5-8815-ef476db5cfe7@oss.nttdata.com
2024-07-14Avoid unhelpful internal error for incorrect recursive-WITH queries.Tom Lane
checkWellFormedRecursion would issue "missing recursive reference" if a WITH RECURSIVE query contained a single self-reference but that self-reference was inside a top-level WITH, ORDER BY, LIMIT, etc, rather than inside the second arm of the UNION as expected. We already intended to throw more-on-point errors for such cases, but those error checks must be done before examining the UNION arm in order to have the desired results. So this patch need only move some code (and improve the comments). Per bug #18536 from Alexander Lakhin. Back-patch to all supported branches. Discussion: https://postgr.es/m/18536-0a342ec07901203e@postgresql.org
2024-07-13Use correct collate.windows.win1252.outAndrew Dunstan
I inadvertently missed backporting this to Release 17 from commit 291c420747 per offlist reminder from Alexander Lakhin.
2024-07-13Fix new assertion for MERGE view_name ... DO NOTHING.Noah Misch
Such queries don't expand automatically updatable views, and ModifyTable uses the wholerow attribute unconditionally. The user-visible behavior is fine, so change to more-specific assertions. Commit d5f788b41dc2cbdde6e7694c70dda54d829a5ed5 added the wrong assertion. Back-patch to v17, where commit 5f2e179bd31e5f5803005101eb12a8d7bf8db8f3 introduced MERGE view_name. Reported by Alexander Lakhin. Discussion: https://postgr.es/m/e4b40a88-c134-6926-3196-bc4501cb87a2@gmail.com
2024-07-13Don't lose partitioned table reltuples=0 after relhassubclass=f.Noah Misch
ANALYZE sets relhassubclass=f when a partitioned table no longer has partitions. An ANALYZE doing that proceeded to apply the inplace update of pg_class.reltuples to the old pg_class tuple instead of the new tuple, losing that reltuples=0 change if the ANALYZE committed. Non-partitioning inheritance trees were unaffected. Back-patch to v14, where commit 375aed36ad83f0e021e9bdd3a0034c0c992c66dc introduced maintenance of partitioned table pg_class.reltuples. Reported by Alexander Lakhin. Discussion: https://postgr.es/m/a295b499-dcab-6a99-c06e-01cf60593344@gmail.com
2024-07-13Make sure to run pg_isready on correct portAndrew Dunstan
The current code can have pg_isready unexpectedly succeed if there is a server running on the default port. To avoid this we delay running the test until after a node has been created but before it starts, and then use that node's port, so we are fairly sure there is nothing running on the port. Backpatch to all live branches.
2024-07-13Fix lost Windows socket EOF events.Thomas Munro
Winsock only signals an FD_CLOSE event once if the other end of the socket shuts down gracefully. Because each WaitLatchOrSocket() call constructs and destroys a new event handle every time, with unlucky timing we can lose it and hang. We get away with this only if the other end disconnects non-gracefully, because FD_CLOSE is repeatedly signaled in that case. To fix this design flaw in our Windows socket support fundamentally, we'd probably need to rearchitect it so that a single event handle exists for the lifetime of a socket, or switch to completely different multiplexing or async I/O APIs. That's going to be a bigger job and probably wouldn't be back-patchable. This brute force kludge closes the race by explicitly polling with MSG_PEEK before sleeping. Back-patch to all supported releases. This should hopefully clear up some random build farm and CI hang failures reported over the years. It might also allow us to try using graceful shutdown in more places again (reverted in commit 29992a6) to fix instability in the transmission of FATAL error messages, but that isn't done by this commit. Reported-by: Tom Lane <tgl@sss.pgh.pa.us> Tested-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/176008.1715492071%40sss.pgh.pa.us
2024-07-12Add ORDER BY to new test queryAlvaro Herrera
Per buildfarm.
2024-07-12Fix ALTER TABLE DETACH for inconsistent indexesAlvaro Herrera
When a partitioned table has an index that doesn't support a constraint, but a partition has an equivalent index that does, then a DETACH operation would misbehave: a crash in assertion-enabled systems (because we fail to find the constraint in the parent that we expect to), or a broken coninhcount value (-1) in production systems (because we blindly believe that we've successfully detached the parent). While we should reject an ATTACH of a partition with such an index, we have failed to do so in existing releases, so adding an error in stable releases might break the (unlikely) existing applications that rely on this behavior. At this point I don't even want to reject them in master, because it'd break pg_upgrade if such databases exist, and there would be no easy way to fix existing databases without expensive index rebuilds. (Later on we could add ALTER TABLE ... ADD CONSTRAINT USING INDEX to partitioned tables, which would allow the user to fix such patterns. At that point we could add more restrictions to prevent the problem from its root.) Also, add a test case that leaves one table in this condition, so that we can verify that pg_upgrade continues to work if we later decide to change the policy on the master branch. Backpatch to all supported branches. Co-authored-by: Tender Wang <tndrwang@gmail.com> Reported-by: Alexander Lakhin <exclusion@gmail.com> Reviewed-by: Tender Wang <tndrwang@gmail.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/18500-62948b6fe5522f56@postgresql.org
2024-07-12Fix unstable test in 040_pg_createsubscriber.Amit Kapila
The slot synchronization failed because the local slot's (created during slot synchronization) catalog_xmin on standby is ahead of remote slot. This happens because the INSERT before slot synchronization results in the generation of a new xid that could be replicated to the standby. Now before the xmin of the physical slot on the primary catches up via hot_standby_feedback, the test has created a logical slot that got some prior value of catalog_xmin. To fix this we could try to ensure that the physical slot's catalog_xmin is caught up to latest value before creating a logical slot but we took a simpler path to move the INSERT after synchronizing the logical slot. Reported-by: Alexander Lakhin as per buildfarm Diagnosed-by: Amit Kapila, Hou Zhijie, Alexander Lakhin Author: Hou Zhijie Backpatch-through: 17 Discussion: https://postgr.es/m/bde6ac67-69cc-c104-5ab6-dd4f5deadf24@gmail.com
2024-07-11Fix possibility of logical decoding partial transaction changes.Masahiko Sawada
When creating and initializing a logical slot, the restart_lsn is set to the latest WAL insertion point (or the latest replay point on standbys). Subsequently, WAL records are decoded from that point to find the start point for extracting changes in the DecodingContextFindStartpoint() function. Since the initial restart_lsn could be in the middle of a transaction, the start point must be a consistent point where we won't see the data for partial transactions. Previously, when not building a full snapshot, serialized snapshots were restored, and the SnapBuild jumps to the consistent state even while finding the start point. Consequently, the slot's restart_lsn and confirmed_flush could be set to the middle of a transaction. This could lead to various unexpected consequences. Specifically, there were reports of logical decoding decoding partial transactions, and assertion failures occurred because only subtransactions were decoded without decoding their top-level transaction until decoding the commit record. To resolve this issue, the changes prevent restoring the serialized snapshot and jumping to the consistent state while finding the start point. On v17 and HEAD, a flag indicating whether snapshot restores should be skipped has been added to the SnapBuild struct, and SNAPBUILD_VERSION has been bumpded. On backbranches, the flag is stored in the LogicalDecodingContext instead, preserving on-disk compatibility. Backpatch to all supported versions. Reported-by: Drew Callahan Reviewed-by: Amit Kapila, Hayato Kuroda Discussion: https://postgr.es/m/2444AA15-D21B-4CCE-8052-52C7C2DAFE5C%40amazon.com Backpatch-through: 12
2024-07-10Make our back branches compatible with libxml2 2.13.x.Tom Lane
This back-patches HEAD commits 066e8ac6e, 6082b3d5d, e7192486d, and 896cd266f into supported branches. Changes: * Use xmlAddChildList not xmlAddChild in XMLSERIALIZE (affects v16 and up only). This was a flat-out coding mistake that we got away with due to lax checking in previous versions of xmlAddChild. * Use xmlParseInNodeContext not xmlParseBalancedChunkMemory. This is to dodge a bug in xmlParseBalancedChunkMemory in libxm2 releases 2.13.0-2.13.2. While that bug is now fixed upstream and will probably never be seen in any production-oriented distro, it is currently a problem on some more-bleeding-edge-friendly platforms. * Suppress "chunk is not well balanced" errors from libxml2, unless it is the only error. This eliminates an error-reporting discrepancy between 2.13 and older releases. This error is almost always redundant with previous errors, if not flat-out inappropriate, which is why 2.13 changed the behavior and why nobody's likely to miss it. Erik Wienhold and Tom Lane, per report from Frank Streitzig. Discussion: https://postgr.es/m/trinity-b0161630-d230-4598-9ebc-7a23acdb37cb-1720186432160@3c-app-gmx-bap25 Discussion: https://postgr.es/m/trinity-361ba18b-541a-4fe7-bc63-655ae3a7d599-1720259822452@3c-app-gmx-bs01
2024-07-10Use diff's --strip-trailing-cr flag where appropriate on WindowsAndrew Dunstan
Test result files might be checked out using Unix or Windows style line endings, depening on git flags, so on Windows we use the --strip-trailing-cr flag to tell diff to ignore line endings differences. The flag is added to the diff invocation for the test_json_parser module tests and the pg_bsd_indent tests. in pg_regress.c we replace the current use of the "-w" flag, which ignore all white space differences, with this one which only ignores line end differences. Discussion: https://postgr.es/m/20240707052030.r77hbdkid3mwksop@awork3.anarazel.de
2024-07-09Prevent CRLF conversion of inputs in json_parser test moduleAndrew Dunstan
Do this by opening the file in PG_BINARY_R mode. This prevents us from getting wrong byte count from stat(). Per complaint from Andres Freund Discussion: https://postgr.es/m/20240707052030.r77hbdkid3mwksop@awork3.anarazel.de Backpatch to rlease 17 where this code was introduced
2024-07-09Fix missing invalidations for search_path cache.Jeff Davis
Reported-by: Noah Misch Discussion: https://postgr.es/m/20240630223047.1f.nmisch@google.com Backpatch-through: 17
2024-07-09Fix limit block handling in pg_wal_summary_contents().Fujii Masao
Previously, pg_wal_summary_contents() had two issues, causing discrepancies between pg_wal_summary_contents() and the pg_walsummary command on the same WAL summary file: (1) It did not emit the limit block when that's the only data for a particular relation fork. (2) It emitted the same limit block multiple times if the list of block numbers was long enough. This commit fixes these issues. Backpatch to v17 where pg_wal_summary_contents() was added. Author: Fujii Masao Reviewed-by: Robert Haas Discussion: https://postgr.es/m/90980ee6-2da6-42f6-a7b0-b7bae62ae279@oss.nttdata.com
2024-07-08Symlink pg_replslot robustly on Windows in pg_basebackup testAndrew Dunstan
This reverts commit e9f15bc9. Instead of a hacky solution that didn't work on Windows, we avoid trying to move the directory possibly across drives, and instead remove it and recreate it in the new location. Discussion: https://postgr.es/m/20240707070243.sb77kp4ubowauctz@awork3.anarazel.de Backpatch to release 14 like the previous patch.
2024-07-08Choose ports for test servers less likely to result in conflictsAndrew Dunstan
If we choose ports in the range typically used for ephemeral ports there is a danger of encountering a port conflict due to a race condition between the time we choose the port in a range below that typically used to allocate ephemeral ports, but higher than the range typically used by well known services. Author: Jelte Fenema-Nio, with some editing by me. Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee056f2@gmail.com Backpatch to all live branches (12 and up)
2024-07-08Force nodes for SSL tests to start in TCP modeAndrew Dunstan
Currently they are started in unix socket mode in ost cases, and then converted to run in TCP mode. This can result in port collisions, and there is no virtue in startng in unix socket mode, so start as we will be going on. Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee056f2@gmail.com Backpatch to all live branches (12 and up).
2024-07-08Fix scale clamping in numeric round() and trunc().Dean Rasheed
The numeric round() and trunc() functions clamp the scale argument to the range between +/- NUMERIC_MAX_RESULT_SCALE (2000), which is much smaller than the actual allowed range of type numeric. As a result, they return incorrect results when asked to round/truncate more than 2000 digits before or after the decimal point. Fix by using the correct upper and lower scale limits based on the actual allowed (and documented) range of type numeric. While at it, use the new NUMERIC_WEIGHT_MAX constant instead of SHRT_MAX in all other overflow checks, and fix a comment thinko in power_var() introduced by e54a758d24 -- the minimum value of ln_dweight is -NUMERIC_DSCALE_MAX (-16383), not -SHRT_MAX, though this doesn't affect the point being made in the comment, that the resulting local_rscale value may exceed NUMERIC_MAX_DISPLAY_SCALE (1000). Back-patch to all supported branches. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/CAEZATCXB%2BrDTuMjhK5ZxcouufigSc-X4tGJCBTMpZ3n%3DxxQuhg%40mail.gmail.com
2024-07-08Typo fixAmit Langote
Reported-by: Junwang Zhao <zhjwpku@gmail.com> Discussion: https://postgr.es/m/CAEG8a3KPi=LayiTwJ11ikF7bcqnZUrcj8NgX0V8nO1mQKZ9GfQ@mail.gmail.com Backpatch-through: 17
2024-07-08Fix outdated comment after removal of direct SSL fallbackHeikki Linnakangas
The option to fall back from direct SSL to negotiated SSL or a plaintext connection was removed in commit fb5718f35f. Discussion: https://www.postgresql.org/message-id/c82ad227-e049-4e18-8898-475a748b5a5a@iki.fi
2024-07-08Fix right-anti-joins when the inner relation is proven uniqueRichard Guo
For an inner_unique join, we always assume that the executor will stop scanning for matches after the first match. Therefore, for a mergejoin that is inner_unique and whose mergeclauses are sufficient to identify a match, we set the skip_mark_restore flag to true, indicating that the executor need not do mark/restore calls. However, merge-right-anti-join did not get this memo and continues scanning the inner side for matches after the first match. If there are duplicates in the outer scan, we may incorrectly skip matching some inner tuples, which can lead to wrong results. Here we fix this issue by ensuring that merge-right-anti-join also advances to next outer tuple after the first match in inner_unique cases. This also saves cycles by avoiding unnecessary scanning of inner tuples after the first match. Although hash-right-anti-join does not suffer from this wrong results issue, we apply the same change to it as well, to help save cycles for the same reason. Per bug #18522 from Antti Lampinen, and bug #18526 from Feliphe Pozzer. Back-patch to v16 where right-anti-join was introduced. Author: Richard Guo Discussion: https://postgr.es/m/18522-c7a8956126afdfd0@postgresql.org
2024-07-08Re-enable autoruns for for cmd.exe on WindowsMichael Paquier
This acts as a revert of b83747a8a65b and 9886744a361b. As pointed out by Noah, HEAD and REL_17_STABLE are in a weird state where the code paths adding /D would limit the spawn of child processes, but we still have code paths where the spawn of more than one child process would be possible. Let's remove these /D switches for now, to bring back the code into a state consistent with how autorun is configured on a Windows host. Reported-by: Noah Misch Discussion: https://postgr.es/m/20240630021211.f3.nmisch@google.com Backpatch-through: 17
2024-07-06Fix incorrect sentinel byte logic in GenerationRealloc()David Rowley
This only affects MEMORY_CONTEXT_CHECKING builds. This fixes an off-by-one issue in GenerationRealloc() where the fast-path code which tries to reuse the existing allocation if the existing chunk is >= the new requested size. The code there thought it was always ok to use the existing chunk, but when oldsize == size there isn't enough space to store the sentinel byte. If both sizes matched exactly set_sentinel() would overwrite the first byte beyond the chunk and then subsequent GenerationRealloc() calls could then fail the Assert(chunk->requested_size < oldsize) check which is trying to ensure the chunk is large enough to store the sentinel. The same issue does not exist in aset.c as the sentinel checking code only adds a sentinel byte if there's enough space in the chunk. Reported-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/49275921-7b39-41af-5eb8-97b50ce3312e@gmail.com Backpatch-through: 16, where the problem was introduced by 0e480385e