summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-07-04Ensure that creation of an empty relfile is fsync'd at checkpoint.Heikki Linnakangas
If you create a table and don't insert any data into it, the relation file is never fsync'd. You don't lose data, because an empty table doesn't have any data to begin with, but if you crash and lose the file, subsequent operations on the table will fail with "could not open file" error. To fix, register an fsync request in mdcreate(), like we do for mdwrite(). Per discussion, we probably should also fsync the containing directory after creating a new file. But that's a separate and much wider issue. Backpatch to all supported versions. Reviewed-by: Andres Freund, Thomas Munro Discussion: https://www.postgresql.org/message-id/d47d8122-415e-425c-d0a2-e0160829702d%40iki.fi
2023-07-04Re-bin segment when memory pages are freed.Thomas Munro
It's OK to be lazy about re-binning memory segments when allocating, because that can only leave segments in a bin that's too high. We'll search higher bins if necessary while allocating next time, and also eventually re-bin, so no memory can become unreachable that way. However, when freeing memory, the largest contiguous range of free pages might go up, so we should re-bin eagerly to make sure we don't leave the segment in a bin that is too low for get_best_segment() to find. The re-binning code is moved into a function of its own, so it can be called whenever free pages are returned to the segment's free page map. Back-patch to all supported releases. Author: Dongming Liu <ldming101@gmail.com> Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier version) Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/CAL1p7e8LzB2LSeAXo2pXCW4%2BRya9s0sJ3G_ReKOU%3DAjSUWjHWQ%40mail.gmail.com
2023-07-04Fix race in SSI interaction with gin fast path.Thomas Munro
The ginfast.c code previously checked for conflicts in before locking the relevant buffer, leaving a window where a RW conflict could be missed. Re-order. There was also a place where buffer ID and block number were confused while trying to predicate-lock a page, noted by visual inspection. Back-patch to all supported releases. Fixes one more problem discovered with the reproducer from bug #17949, in this case when Dmitry tried other index types. Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com> Reported-by: Dmitry Dolgov <9erthalion6@gmail.com> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
2023-07-04Fix race in SSI interaction with bitmap heap scan.Thomas Munro
When performing a bitmap heap scan, we don't want to miss concurrent writes that occurred after we observed the heap's rs_nblocks, but before we took predicate locks on index pages. Therefore, we can't skip fetching any heap tuples that are referenced by the index, because we need to test them all with CheckForSerializableConflictOut(). The old optimization that would ignore any references to blocks >= rs_nblocks gets in the way of that requirement, because it means that concurrent writes in that window are ignored. Removing that optimization shouldn't affect correctness at any isolation level, because any new tuples shouldn't be visible to an MVCC snapshot. There also shouldn't be any error-causing references to heap blocks past the end, because we should have held at least an AccessShareLock on the table before the index scan. It can't get smaller while our transaction is running. For now, though, we'll keep the optimization at lower levels to avoid making unnecessary changes in a bug fix. Back-patch to all supported releases. In release 11, the code is in a different place but not fundamentally different. Fixes one aspect of bug #17949. Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com> Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
2023-07-04Fix race in SSI interaction with empty btrees.Thomas Munro
When predicate-locking btrees, we have a special case for completely empty btrees, since there is no page to lock. This was racy, because, without buffer lock held, a matching key could be inserted between the _bt_search() and the PredicateLockRelation() calls. Fix, by rechecking _bt_search() after taking the relation-level SIREAD lock, if using SERIALIZABLE isolation and an empty btree is discovered. Back-patch to all supported releases. Fixes one aspect of bug #17949. Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com> Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com> Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
2023-07-03Remove expensive test of postgres_fdw batch insertsTomas Vondra
The test inserted 70k rows into a foreign table, in order to verify correct behavior with more than 65535 parameters, and was added in response to a bug report. However, this is rather expensive, especially when running the tests under valgrind, CLOBBER_CACHE_ALWAYS etc. It doesn't seem worth it to keep running the test, so remove it from all branches (14+). Backpatch-through: 14 Discussion: https://postgr.es/m/2131017.1623451468@sss.pgh.pa.us
2023-07-03Improve pg_basebackup long file name test Windows robustnessAndrew Dunstan
Creation of a file with a very long name can create problems on Windows due to its file path limits. Work around that by creating the file via a symlink with a shorter name. Error displayed by buildfarm animal fairywren.o Backpatch to all live branches
2023-07-03Make PG_TEST_NOCLEAN work for temporary directories in TAP testsMichael Paquier
When set, this environment variable was only effective for data directories but not for all the other temporary files created by PostgreSQL::Test::Utils. Keeping the temporary files after a successful run can be useful for debugging purposes. The documentation is updated to reflect the new behavior, with contents available in doc/ since v16 and in src/test/perl/README since v15. Author: Jacob Champion Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/CAAWbhmgHtDH1SGZ+Fw05CsXtE0mzTmjbuUxLB9mY9iPKgM6cUw@mail.gmail.com Discussion: https://postgr.es/m/YyPd9unV14SX2bLF@paquier.xyz Backpatch-through: 11
2023-07-03Silence "missing contrecord" error.Thomas Munro
Commit dd38ff28ad added a new error message "missing contrecord" when we fail to reassemble a record. Unfortunately that caused noisy messages to be logged by pg_waldump at end of segment, and by walsender when asked to shut down on a segment boundary. Remove the new error message, so that this condition signals end-of- WAL without a message. It's arguably a reportable condition that should not be silenced while performing crash recovery, but fixing that without introducing noise in the other cases will require more research. Back-patch to 15. Reported-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Discussion: https://postgr.es/m/6a1df56e-4656-b3ce-4b7a-a9cb41df8189%40enterprisedb.com
2023-07-02Fix oversight in handling of modifiedCols since f24523672dTomas Vondra
Commit f24523672d fixed a memory leak by moving the modifiedCols bitmap into the per-row memory context. In the case of AFTER UPDATE triggers, the bitmap is however referenced from an event kept until the end of the query, resulting in a use-after-free bug. Fixed by copying the bitmap into the AfterTriggerEvents memory context, which is the one where we keep the trigger events. There's only one place that needs to do the copy, but the memory context may not exist yet. Doing that in a separate function seems more readable. Report by Alexander Pyhalov, fix by me. Backpatch to 13, where the bitmap was added to the event by commit 71d60e2aa0. Reported-by: Alexander Pyhalov Backpatch-through: 13 Discussion: https://postgr.es/m/acddb17c89b0d6cb940eaeda18c08bbe@postgrespro.ru
2023-07-02Fix memory leak in Incremental Sort rescansTomas Vondra
The Incremental Sort had a couple issues, resulting in leaking memory during rescans, possibly triggering OOM. The code had a couple of related flaws: 1. During rescans, the sort states were reset but then also set to NULL (despite the comment saying otherwise). ExecIncrementalSort then sees NULL and initializes a new sort state, leaking the memory used by the old one. 2. Initializing the sort state also automatically rebuilt the info about presorted keys, leaking the already initialized info. presorted_keys was also unnecessarily reset to NULL. Patch by James Coleman, based on patches by Laurenz Albe and Tom Lane. Backpatch to 13, where Incremental Sort was introduced. Author: James Coleman, Laurenz Albe, Tom Lane Reported-by: Laurenz Albe, Zu-Ming Jiang Backpatch-through: 13 Discussion: https://postgr.es/m/b2bd02dff61af15e3526293e2771f874cf2a3be7.camel%40cybertec.at Discussion: https://postgr.es/m/db03c582-086d-e7cd-d4a1-3bc722f81765%40inf.ethz.ch
2023-07-01Fix test of pg_stat_io for wal_level=minimal.Noah Misch
Back-patch to v16, which introduced the test.
2023-06-30doc: PG 16 relnotes, remove "Have initdb use ICU by default"Bruce Momjian
Item reverted. Backpatch-through: 16 only
2023-06-30Add a test case for a316a3bcAmit Langote
a316a3bc fixed the code in build_simpl_rel() that propagates RelOptInfo.userid from parent to child rels so that it works correctly for the child rels of a UNION ALL subquery rel, though no tests were added in that commit. So do so here. As noted in the discussion, coming up with a test case in the core regression suite for this fix has turned out to be tricky, so the test case is added to the postgres_fdw's suite instead. postgresGetForeignRelSize()'s use of user mapping for the user specified in RelOptInfo.userid makes it relatively easier to craft a test case around. Discussion: https://postgr.es/m/CA%2BHiwqH91GaFNXcXbLAM9L%3DzBwUmSyv699Mtv3i1_xtk9Xec_A%40mail.gmail.com Backpatch-through: 16
2023-06-30Fix marking of indisvalid for partitioned indexes at creationMichael Paquier
The logic that introduced partitioned indexes missed a few things when invalidating a partitioned index when these are created, still the code is written to handle recursions: 1) If created from scratch because a mapping index could not be found, the new index created could be itself invalid, if for example it was a partitioned index with one of its leaves invalid. 2) A CCI was missing when indisvalid is set for a parent index, leading to inconsistent trees when recursing across more than one level for a partitioned index creation if an invalidation of the parent was required. This could lead to the creation of a partition index tree where some of the partitioned indexes are marked as invalid, but some of the parents are marked valid, which is not something that should happen (as validatePartitionedIndex() defines, indisvalid is switched to true for a partitioned index iff all its partitions are themselves valid). This patch makes sure that indisvalid is set to false on a partitioned index if at least one of its partition is invalid. The flag is set to true if *all* its partitions are valid. The regression test added in this commit abuses of a failed concurrent index creation, marked as invalid, that maps with an index created on its partitioned table afterwards. Reported-by: Alexander Lakhin Reviewed-by: Alexander Lakhin Discussion: https://postgr.es/m/14987634-43c0-0cb3-e075-94d423607e08@gmail.com Backpatch-through: 11
2023-06-29Arm gen_node_support.pl's nodetag ABI stability check in v16.Tom Lane
Per RELEASE_CHANGES checklist.
2023-06-30Fix pg_depend entry to AMs after ALTER TABLE .. SET ACCESS METHODMichael Paquier
ALTER TABLE .. SET ACCESS METHOD was not registering a dependency to the new access method with the relation altered in its rewrite phase, making possible the drop of an access method even if there are relations that depend on it. During the rewrite, a temporary relation is created to build the new relation files before swapping the new and old files, and, while the temporary relation was registering a correct dependency to the new AM, the old relation did not do that. A dependency on the access method is added when the relation files are swapped, which is the point where pg_class is updated. Materialized views and tables use the same code path, hence both were impacted. Backpatch down to 15, where this command has been introduced. Reported-by: Alexander Lakhin Reviewed-by: Nathan Bossart, Andres Freund Discussion: https://postgr.es/m/18000-9145c25b1af475ca@postgresql.org Backpatch-through: 15
2023-06-29meson: Remove redundant return code checkAndres Freund
run_command(check: true) already would have errorred out before the check is reached. Author: Tristan Partin <tristan@neon.tech> Discussion: CSPIJVUDZFKX.3KHMOAVGF94RV@c3po
2023-06-29Defend against bogus parameterization of join input paths.Tom Lane
An outer join cannot be formed using an input path that is parameterized by a value that is supposed to be nulled by the outer join. This is obviously nonsensical, and it could lead to a bad plan being selected; although currently it seems that we'll hit various sanity-check assertions first. I think that such cases were formerly prevented by the delay_upper_joins mechanism, but now that that's gone we need an explicit check. (Perhaps we should avoid generating baserel paths that could lead to this situation in the first place; but it seems like having a defense at the join level would be a good idea anyway.) Richard Guo and Tom Lane, per report from Jaime Casanova Discussion: https://postgr.es/m/CAJKUy5g2uZRrUDZJ8p-=giwcSHVUn0c9nmdxPSY0jF0Ov8VoEA@mail.gmail.com
2023-06-29Fix order of operations in ExecEvalFieldStoreDeForm().Tom Lane
If the given composite datum is toasted out-of-line, DatumGetHeapTupleHeader will perform database accesses to detoast it. That can invalidate the result of get_cached_rowtype, as documented (perhaps not plainly enough) in that function's API spec; which leads to strange errors or crashes when we try to use the TupleDesc to read the tuple. In short then, trying to update a field of a composite column could fail intermittently if the overall column value is wide enough to require toasting. We can fix the bug at no cost by just changing the order of operations, since we don't need the TupleDesc until after detoasting. (Other callers of get_cached_rowtype appear to get this right already, so there's only one bug.) Note that the added regression test case reveals this bug reliably only with debug_discard_caches/CLOBBER_CACHE_ALWAYS. Per bug #17994 from Alexander Lakhin. Sadly, this patch does not fix the missing-values issue revealed in the bug discussion; we'll need some more work to cover that. Discussion: https://postgr.es/m/17994-5c7100b51b4790e9@postgresql.org
2023-06-29meson: Fix intl misspellingPeter Eisentraut
Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Use a better error message in an impossible casePeter Eisentraut
Meson validates 'choice' options for us, so technically this case is impossible. A better error message helps people reading the code understand what is going on in that branch. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Make some Meson style more consistent with surrounding codePeter Eisentraut
Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Fix some grammar usage in Meson commentsPeter Eisentraut
Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Mention the correct way to disable readline supportPeter Eisentraut
Using false to disable a feature option is incorrect. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Remove old commentPeter Eisentraut
That portion of code is not run in the Windows case already given the structure of the surrounding if statement. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Use the not_found_dep constantPeter Eisentraut
Previously in the build description, a not_found_dep was defined. Make use of it. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Attach colon to keyword argumentPeter Eisentraut
This matches the style found in the rest of the Meson build description. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Use consistent Meson option description formatsPeter Eisentraut
Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Use consistent casing in Meson option descriptionsPeter Eisentraut
Meson itself uses capital letters for option descriptions, so follow that. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29meson: Remove triple-quoted stringsPeter Eisentraut
Triple-quoted strings are for multiline strings in Meson. None of the descriptions that got changed were multiline and the entire file uses single-line descriptions. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CSPIJVUDZFKX.3KHMOAVGF94RV%40c3po
2023-06-29Remove inappropriate raw_expression_tree_walker() codePeter Eisentraut
It was walking into the ColumnDef->compression field, which is not a node but a string. This code is currently not reachable (because the compression field is only set in situations that don't go through raw_expression_tree_walker()), but if it had been, this could have behaved erratically.
2023-06-29Error message wording improvementsPeter Eisentraut
2023-06-28Fix psql \?'s entries for \dp and \z.Nathan Bossart
d913928c9c added support for the "S" modifier to psql's \dp and \z meta-commands, but it missed updating the corresponding entries in \?'s output. Author: Noriyoshi Shinoda Discussion: https://postgr.es/m/DM4PR84MB17342A51B3A1556CFBC7A4B2EE25A%40DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
2023-06-29pg_stat_statements: Fix second comment related to entry resetsMichael Paquier
This should have been part of dc73db6, but it got lost in the mix. Oversight in 6b4d23f. Author: Japin Li Discussion: https://postgr.es/m/MEYP282MB1669FC91C764E277821936D3B624A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM Backpatch-through: 14
2023-06-29pg_stat_statements: Fix incorrect comment with entry resetsMichael Paquier
Oversight in 6b4d23f. Author: Japin Li, Richard Guo Discussion: https://postgr.es/m/MEYP282MB1669FC91C764E277821936D3B624A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM Backpatch-through: 14
2023-06-28Reword error messages for consistencyPeter Eisentraut
2023-06-28Doc: minor wording adjustments in transaction isolation discussion.Tom Lane
Re-word for more clarity, per gripe from Anton Sidyakin. Discussion: https://postgr.es/m/168745911769.2239590.6062411529242609290@wrigleys.postgresql.org
2023-06-28Add timeline ID to file names generated with pg_waldump --save-fullpageMichael Paquier
Not including the timeline IDs to the file names generated by pg_waldump for the individual blocks saved could cause some of these files to be overwritten when scanning segments across multiple timelines. Having this information is also as much useful as the LSNs, to be able to know from exactly which WAL segment a block is comes from. While on it, this fixes a few comments in the tests, where the format of the file was not described as matching with the reality. Reported-by: Fujii Masao Reviewed-by: Kyotaro Horiguchi, David Christensen Discussion: https://postgr.es/m/ZJp921+nITFnvBVS@paquier.xyz
2023-06-28Ignore invalid indexes when enforcing index rules in ALTER TABLE ATTACH ↵Michael Paquier
PARTITION A portion of ALTER TABLE .. ATTACH PARTITION is to ensure that the partition being attached to the partitioned table has a correct set of indexes, so as there is a consistent index mapping between the partitioned table and its new-to-be partition. However, as introduced in 8b08f7d, the current logic could choose an invalid index as a match, which is something that can exist when dealing with more than two levels of partitioning, like attaching a partitioned table (that has partitions, with an index created by CREATE INDEX ON ONLY) to another partitioned table. A partitioned index with indisvalid set to false is equivalent to an incomplete partition tree, meaning that an invalid partitioned index does not have indexes defined in all its partitions. Hence, choosing an invalid partitioned index can create inconsistent partition index trees, where the parent attaching to is valid, but its partition may be invalid. In the report from Alexander Lakhin, this showed up as an assertion failure when validating an index. Without assertions enabled, the partition index tree would be actually broken, as indisvalid should be switched to true for a partitioned index once all its partitions are themselves valid. With two levels of partitioning, the top partitioned table used a valid index and was able to link to an invalid index stored on its partition, itself a partitioned table. I have studied a few options here (like the possibility to switch indisvalid to false for the parent), but came down to the conclusion that we'd better rely on a simple rule: invalid indexes had better never be chosen, so as the partition attached uses and creates indexes that the parent expects. Some regression tests are added to provide some coverage. Note that the existing coverage is not impacted. This is a problem since partitioned indexes exist, so backpatch all the way down to v11. Reported-by: Alexander Lakhin Discussion: https://postgr.es/14987634-43c0-0cb3-e075-94d423607e08@gmail.com Backpatch-through: 11
2023-06-28Remove dependency to query text in JumbleQuery()Michael Paquier
Since 3db72eb, the query ID of utilities is generated using the Query structure, making the use of the query string in JumbleQuery() unnecessary. This commit removes the argument "querytext" from JumbleQuery(). Reported-by: Joe Conway Reviewed-by: Nathan Bossart Discussion: https://postgr.es/m/ZJlQAWE4COFqHuAV@paquier.xyz
2023-06-27Fix comment on clearing padding.Heikki Linnakangas
Author: Japin Li Discussion: https://www.postgresql.org/message-id/MEYP282MB16696317B5DA7D0D92306149B627A@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM
2023-06-26Stamp 16beta2.REL_16_BETA2Tom Lane
2023-06-26doc: PG 16 relnotes, update for beta 2Bruce Momjian
2023-06-26Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: ab77975e9d2cde44da796c18af3ec1a66f0df7ae
2023-06-26Change "..." to cstring in old input/output function comments.Heikki Linnakangas
It was not clear what the "..." meant. Author: Steve Chavez Discussion: https://www.postgresql.org/message-id/CAGRrpzZzeh7zC3yaVG9di%3DydJ%2BiHwdXnFPB3evGFKvC1zf6ajA@mail.gmail.com
2023-06-24Check for interrupts and stack overflow in TParserGet().Tom Lane
TParserGet() recurses for some token types, meaning it's possible to drive it to stack overflow. Since this is a minority behavior, I chose to add the check_stack_depth() call to the two places that recurse rather than doing it during every single call. While at it, add CHECK_FOR_INTERRUPTS(), because this can run unpleasantly long for long inputs. Per bug #17995 from Zuming Jiang. This is old, so back-patch to all supported branches. Discussion: https://postgr.es/m/17995-9f20ff3e6389db4c@postgresql.org
2023-06-23doc: rename "decades" to be more genericBruce Momjian
Reported-by: Michael Paquier Discussion: https://postgr.es/m/ZJTzwD2rTbHWWQ9g@paquier.xyz Backpatch-through: 11
2023-06-23Error message refactoringPeter Eisentraut
Take some untranslatable things out of the message and replace by format placeholders, to reduce translatable strings and reduce translation mistakes.
2023-06-23doc: Improve punctuationPeter Eisentraut