summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-01-30Doc: mention foreign keys can reference unique indexesDavid Rowley
We seem to have only documented a foreign key can reference the columns of a primary key or unique constraint. Here we adjust the documentation to mention columns in a non-partial unique index can be mentioned too. The header comment for transformFkeyCheckAttrs() also didn't mention unique indexes, so fix that too. In passing make that header comment reflect reality in the various other aspects where it deviated from it. Bug: 18295 Reported-by: Gilles PARC Author: Laurenz Albe, David Rowley Discussion: https://www.postgresql.org/message-id/18295-0ed0fac5c9f7b17b%40postgresql.org Backpatch-through: 12
2024-01-29Move is_valid_ascii() to ascii.h.Nathan Bossart
This function requires simd.h, which is a rather large dependency for a widely-used header file like pg_wchar.h. Furthermore, there is a report of a third-party tool that is struggling to use pg_wchar.h due to its dependence on simd.h (presumably because simd.h uses several intrinsics). Moving the function to the much less popular ascii.h resolves these issues for now. This commit is back-patched for the benefit of the aforementioned third-party tool. The simd.h dependency was only added in v16, but we've opted to back-patch to v15 so that is_valid_ascii() lives in the same file for all versions where it exists. This could break existing third-party code that uses the function, but we couldn't find any examples of such code. It should be possible to fix any code that this commit breaks by including ascii.h in the file that uses is_valid_ascii(). Author: Jubilee Young Reviewed-by: Tom Lane, John Naylor, Andres Freund, Eric Ridge Discussion: https://postgr.es/m/CAPNHn3oKJJxMsYq%2BqLYzVJOFrUcOr4OF1EC-KtFT-qh8nOOOtQ%40mail.gmail.com Backpatch-through: 15
2024-01-29Fix incompatibilities with libxml2 >= 2.12.0.Tom Lane
libxml2 changed the required signature of error handler callbacks to make the passed xmlError struct "const". This is causing build failures on buildfarm member caiman, and no doubt will start showing up in the field quite soon. Add a version check to adjust the declaration of xml_errorHandler() according to LIBXML_VERSION. 2.12.x also produces deprecation warnings for contrib/xml2/xpath.c's assignment to xmlLoadExtDtdDefaultValue. I see no good reason for that to still be there, seeing that we disabled external DTDs (at a lower level) years ago for security reasons. Let's just remove it. Back-patch to all supported branches, since they might all get built with newer libxml2 once it gets a bit more popular. (The back branches produce another deprecation warning about xpath.c's use of xmlSubstituteEntitiesDefault(). We ought to consider whether to back-patch all or part of commit 65c5864d7 to silence that. It's less urgent though, since it won't break the buildfarm.) Discussion: https://postgr.es/m/1389505.1706382262@sss.pgh.pa.us
2024-01-29Fix locking when fixing an incomplete split of a GIN internal pageHeikki Linnakangas
ginFinishSplit() expects the caller to hold an exclusive lock on the buffer, but when finishing an earlier "leftover" incomplete split of an internal page, the caller held a shared lock. That caused an assertion failure in MarkBufferDirty(). Without assertions, it could lead to corruption if two backends tried to complete the split at the same time. On master, add a test case using the new injection point facility. Report and analysis by Fei Changhong. Backpatch the fix to all supported versions. Reviewed-by: Fei Changhong, Michael Paquier Discussion: https://www.postgresql.org/message-id/tencent_A3CE810F59132D8E230475A5F0F7A08C8307@qq.com
2024-01-29Add more LOG messages when starting and ending recovery from a backupMichael Paquier
Three LOG messages are added in the recovery code paths, providing information that can be useful to track corruption issues depending on the state of the cluster, telling that: - Recovery has started from a backup_label. - Recovery is restarting from a backup start LSN, without a backup_label. - Recovery has completed from a backup. This was originally applied on HEAD as of 1d35f705e191, and there is consensus that this can be useful for older versions. This applies cleanly down to 15, so do it down to this version for now (older versions have heavily refactored the WAL recovery paths, making the change less straight-forward to do). Author: Andres Freund Reviewed-by: David Steele, Laurenz Albe, Michael Paquier Discussion: https://postgr.es/m/20231117041811.vz4vgkthwjnwp2pp@awork3.anarazel.de Backpatch-through: 15
2024-01-29Fix DROP ROLE when specifying duplicated rolesMichael Paquier
This commit fixes failures with "tuple already updated by self" when listing twice the same role and in a DROP ROLE query. This is an oversight in 6566133c5f52, that has introduced a two-phase logic in DropRole() where dependencies of all the roles to drop are removed in a first phase, with the roles themselves removed from pg_authid in a second phase. The code is simplified to not rely on a List of ObjectAddress built in the first phase used to remove the pg_authid entries in the second phase, switching to a list of OIDs. Duplicated OIDs can be simply avoided in the first phase thanks to that. Using ObjectAddress was not necessary for the roles as they are not used for anything specific to dependency.c, building all the ObjectAddress in the List with AuthIdRelationId as class ID. In 15 and older versions, where a single phase is used, DROP ROLE with duplicated role names would fail on "role \"blah\" does not exist" for the second entry after the CCI() done by the first deletion. This is not really incorrect, but it does not seem worth changing based on a lack of complaints. Reported-by: Alexander Lakhin Reviewed-by: Tender Wang Discussion: https://postgr.es/m/18310-1eb233c5908189c8@postgresql.org Backpatch-through: 16
2024-01-26Compare varnullingrels too in assign_param_for_var().Tom Lane
Oversight in 2489d76c4. Preliminary analysis suggests that the problem may be unreachable --- but if we did have instances of the same column with different varnullingrels, we'd surely need to treat them as different Params. Discussion: https://postgr.es/m/412552.1706203379@sss.pgh.pa.us
2024-01-26Detect Julian-date overflow in timestamp[tz]_pl_interval.Tom Lane
We perform addition of the days field of an interval via arithmetic on the Julian-date representation of the timestamp's date. This step is subject to int32 overflow, and we also should not let the Julian date become very negative, for fear of weird results from j2date. (In the timestamptz case, allow a Julian date of -1 to pass, since it might convert back to zero after timezone rotation.) The additions of the months and microseconds fields could also overflow, of course. However, I believe we need no additional checks there; the existing range checks should catch such cases. The difficulty here is that j2date's magic modular arithmetic could produce something that looks like it's in-range. Per bug #18313 from Christian Maurer. This has been wrong for a long time, so back-patch to all supported branches. Discussion: https://postgr.es/m/18313-64d2c8952d81e84b@postgresql.org
2024-01-25Track LLVM 18 changes.Thomas Munro
A function was given a newly standard name from C++20 in LLVM 16. Then LLVM 18 added a deprecation warning for the old name, and it is about to ship, so it's time to adjust that. Back-patch to all supported releases. Discussion: https://www.postgresql.org/message-id/CA+hUKGLbuVhH6mqS8z+FwAn4=5dHs0bAWmEMZ3B+iYHWKC4-ZA@mail.gmail.com
2024-01-24Fix ALTER TABLE .. ADD COLUMN with complex inheritance treesMichael Paquier
This command, when used to add a column on a parent table with a complex inheritance tree, tried to update multiple times the same tuple in pg_attribute for a child table when incrementing attinhcount, causing failures with "tuple already updated by self" because of a missing CommandCounterIncrement() between two updates. This exists for a rather long time, so backpatch all the way down. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Richard Guo Discussion: https://postgr.es/m/18297-b04cd83a55b51e35@postgresql.org Backpatch-through: 12
2024-01-23Revert "libpqwalreceiver: Convert to libpq-be-fe-helpers.h"Heikki Linnakangas
This reverts commit 728f86fec65537eade8d9e751961782ddb527934. The signal handling was a few bricks shy of a load in that commit, which made the walreceiver non-responsive to SIGTERM while it was waiting for the connection to be established. That prevented a standby from being promoted. Since it was non-essential refactoring, let's revert it to make v16 work the same as earlier releases. I reverted it in 'master' too, to keep the branches in sync. The refactoring was a good idea as such, but it needs a bit more work. Once we have developed a complete patch with this issue fixed, let's re-apply that to 'master'. Reported-by: Kyotaro Horiguchi Backpatch-through: 16 Discussion: https://www.postgresql.org/message-id/20231231.200741.1078989336605759878.horikyota.ntt@gmail.com
2024-01-23Improve stability of recovery test 035_standby_logical_decodingMichael Paquier
This commit tweaks a couple of things in 035_standby_logical_decoding to hopefully stabilize it: - Autovacuum is now disabled, as it could hold a global xmin with a transaction. - Conflicts are generated with command sequences that removed rows (on catalogs, shared or non-shared, or just plain relations) followed by a VACUUM. This was unstable because these did not check that the horizon moved between the SQL commands and the VACUUM. The logic is refactored as follows, to ensure that VACUUM removes dead rows before testing for slot invalidation on a standby (idea suggested by Andres Freund): -- Grab the current horizon. -- Launch SQL commands removing rows. -- Check that the snapshot horizon has been updated. -- Launch VACUUM on the relation whose rows have been removed by the first step. Note that there are still some issues because of standby snapshot WAL records generated by the bgwriter, but this makes the test much more stable. Per reports from buildfarm members dikkop and skink, with analysis and tests from Alexander Lakhin. While on it, fix a couple of incorrect comments. Author: Bertrand Drouvot Reviewed-by: Alexander Lakhin, Michael Paquier Discussion: https://postgr.es/m/OSZPR01MB6310ED3CEDB531BCEDBC6AF2FD479@OSZPR01MB6310.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/bf67e076-b163-9ba3-4ade-b9fc51a3a8f6@gmail.com Backpatch-through: 16
2024-01-22Abort pgbench if script end is reached with an open pipelineAlvaro Herrera
When a pipeline is opened with \startpipeline and not closed, pgbench will either error on the next transaction with a "already in pipeline mode" error or successfully end if this was the last transaction -- despite not sending anything that was piped in the pipeline. Make it an error to reach end of script is reached while there's an open pipeline. Backpatch to 14, where pgbench got support for pipelines. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Reported-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/Za4IObZkDjrO4TcS@paquier.xyz
2024-01-22Re-disallow Memoize for parameterized nested loops with join filtersDavid Rowley
This was previously fixed in 9e215378d but got broken again as a result of 2489d76c4. It seems that commit causes ppi_clauses to contain duplicate clauses and it's no longer safe to check the list_length of that list to determine if there are join conditions other than what's mentioned in ppi_clauses. Here we adjust the check to count the distinct rinfo_serial mentioned in ppi_clauses. We expect that extra->restrictlist won't have duplicate rinfo_serials. Reported-by: Amadeo Gallardo Author: Richard Guo Discussion: https://postgr.es/m/CADFREbW-BLJd7-a5J%2B5wjVumeFG1ByXiSOFzMtkmY_SDWckTxw%40mail.gmail.com Backpatch-through: 16, where 2489d76c4 was introduced.
2024-01-18Fix buildfarm error from commit 5c31669058.Jeff Davis
Skip test when not using unix domain sockets. Discussion: https://postgr.es/m/CALDaNm29-8OozsBWo9H6DN_Tb_3yA1QjRJput-KhaN8ncDJtJA@mail.gmail.com Backpatch-through: 16
2024-01-18Fix plpgsql to allow new-style SQL CREATE FUNCTION as a SQL command.Tom Lane
plpgsql fails on new-style CREATE FUNCTION/PROCEDURE commands within a routine or DO block, because make_execsql_stmt believes that a semicolon token always terminates a SQL command. Now, that's actually been wrong since the day it was written, because CREATE RULE has long allowed multiple rule actions separated by semicolons. But there are few enough people using multi-action rules that there was never an attempt to fix it. New-style SQL functions, though, are popular. psql has this same problem of "does this semicolon really terminate the command?". It deals with CREATE RULE by counting parenthesis nesting depth: a semicolon within parens doesn't end a command. Commits e717a9a18 and 029c5ac03 created a similar heuristic to count matching BEGIN/END pairs (but only within CREATEs, so as not to be fooled by plain BEGIN). That's survived several releases now without trouble reports, so let's just absorb those heuristics into plpgsql. Per report from Samuel Dussault. Back-patch to v14 where new-style SQL function syntax came in. Discussion: https://postgr.es/m/YT2PR01MB88552C3E9AD40A6C038774A781722@YT2PR01MB8855.CANPRD01.PROD.OUTLOOK.COM
2024-01-18Improve handling of dropped partitioned indexes for REINDEX INDEXMichael Paquier
A REINDEX INDEX done on a partitioned index builds a list of the indexes to work on before processing its partitions in individual transactions. When combined with a DROP of the partitioned index, there was a window where it was possible to see some unexpected "could not open relation with OID", synonym of relation lookup error. The code was robust enough to handle the case where the parent relation is missing, but not the case where an index would be gone missing. This is similar to 1d65416661bb. Support for REINDEX on partitioned relations has been introduced in a6642b3ae060, so backpatch down to 14. Author: Fei Changhong Discussion: https://postgr.es/m/tencent_6A52106095ACDE55333E3AD33F304C0C3909@qq.com Backpatch-through: 14
2024-01-18Add try_index_open(), conditional variant of index_open()Michael Paquier
try_index_open() is able to open an index if its relkind fits, except that it would return NULL instead of generated an error if the relation does not exist. This new routine will be used by an upcoming patch to make REINDEX on partitioned relations more robust when an index in a partition tree is dropped. Extracted from a larger patch by the same author. Author: Fei Changhong Discussion: https://postgr.es/m/tencent_6A52106095ACDE55333E3AD33F304C0C3909@qq.com Backpatch-through: 14
2024-01-17Fix incorrect comment on how BackendStatusArray is indexedHeikki Linnakangas
The comment was copy-pasted from the call to ProcSignalInit() in AuxiliaryProcessMain(), which uses a similar scheme of having reserved slots for aux processes after MaxBackends slots for backends. However, ProcSignalInit() indexing starts from 1, whereas BackendStatusArray starts from 0. The code is correct, but the comment was wrong. Discussion: https://www.postgresql.org/message-id/f3ecd4cb-85ee-4e54-8278-5fabfb3a4ed0@iki.fi Backpatch-through: v14
2024-01-17Close socket in case of errors in setting non-blockingDaniel Gustafsson
If configuring the newly created socket non-blocking fails we error out and return INVALID_SOCKET, but the socket that had been created wasn't closed. Fix by issuing closesocket in the errorpath. Backpatch to all supported branches. Author: Ranier Vilela <ranier.vf@gmail.com> Discussion: https://postgr.es/m/CAEudQApmU5CrKefH85VbNYE2y8H=-qqEJbg6RAPU65+vCe+89A@mail.gmail.com Backpatch-through: v12
2024-01-16Don't test already-referenced pointer for nullnessAlvaro Herrera
Commit b8ba7344e9eb added in PQgetResult a derefence to a pointer returned by pqPrepareAsyncResult(), before some other code that was already testing that pointer for nullness. But since commit 618c16707a6d (in Postgres 15), pqPrepareAsyncResult() doesn't ever return NULL (a statically-allocated result is returned if OOM). So in branches 15 and up, we can remove the redundant pointer check with no harm done. However, in branch 14, pqPrepareAsyncResult() can indeed return NULL if it runs out of memory. Fix things there by adding a null pointer check before dereferencing the pointer. This should hint Coverity that the preexisting check is not redundant but necessary. Backpatch to 14, like b8ba7344e9eb. Per Coverity.
2024-01-14Prevent access to an unpinned buffer in BEFORE ROW UPDATE triggers.Tom Lane
When ExecBRUpdateTriggers switches to a new target tuple as a result of the EvalPlanQual logic, it must form a new proposed update tuple. Since commit 86dc90056, that tuple (the result of ExecGetUpdateNewTuple) has been a virtual tuple that might contain pointers to by-ref fields of the new target tuple (in "oldslot"). However, immediately after that we materialize oldslot, causing it to drop its buffer pin, whereupon the by-ref pointers are unsafe to use. This is a live bug only when the new target tuple is in a different page than the original target tuple, since we do still hold a pin on the original one. (Before 86dc90056, there was no bug because the EPQ plantree would hold a pin on the new target tuple; but now that's not assured.) To fix, forcibly materialize the new tuple before we materialize oldslot. This costs nothing since we would have done that shortly anyway. The real-world impact of this is probably minimal. A visible failure could occur if the new target tuple's buffer were recycled for some other page in the short interval before we materialize newslot within the trigger-calling loop; but that's quite unlikely given that we'd just touched that page. There's a larger hazard that some other process could prune and repack that page within the window. We have lock on the new target tuple, but that wouldn't prevent it being moved on the page. Alexander Lakhin and Tom Lane, per bug #17798 from Alexander Lakhin. Back-patch to v14 where 86dc90056 came in. Discussion: https://postgr.es/m/17798-0907404928dcf0dd@postgresql.org
2024-01-13Re-pgindent catcache.c after previous commit.Tom Lane
Discussion: https://postgr.es/m/1393953.1698353013@sss.pgh.pa.us Discussion: https://postgr.es/m/CAGjhLkOoBEC9mLsnB42d3CO1vcMx71MLSEuigeABbQ8oRdA6gw@mail.gmail.com
2024-01-13Cope with catcache entries becoming stale during detoasting.Tom Lane
We've long had a policy that any toasted fields in a catalog tuple should be pulled in-line before entering the tuple in a catalog cache. However, that requires access to the catalog's toast table, and we'll typically do AcceptInvalidationMessages while opening the toast table. So it's possible that the catalog tuple is outdated by the time we finish detoasting it. Since no cache entry exists yet, we can't mark the entry stale during AcceptInvalidationMessages, and instead we'll press forward and build an apparently-valid cache entry. The upshot is that we have a race condition whereby an out-of-date entry could be made in a backend's catalog cache, and persist there indefinitely causing indeterminate misbehavior. To fix, use the existing systable_recheck_tuple code to recheck whether the catalog tuple is still up-to-date after we finish detoasting it. If not, loop around and restart the process of searching the catalog and constructing cache entries from the top. The case is rare enough that this shouldn't create any meaningful performance penalty, even in the SearchCatCacheList case where we need to tear down and reconstruct the whole list. Indeed, the case is so rare that AFAICT it doesn't occur during our regression tests, and there doesn't seem to be any easy way to build a test that would exercise it reliably. To allow testing of the retry code paths, add logic (in USE_ASSERT_CHECKING builds only) that randomly pretends that the recheck failed about one time out of a thousand. This is enough to ensure that we'll pass through the retry paths during most regression test runs. By adding an extra level of looping, this commit creates a need to reindent most of SearchCatCacheMiss and SearchCatCacheList. I'll do that separately, to allow putting those changes in .git-blame-ignore-revs. Patch by me; thanks to Alexander Lakhin for having built a test case to prove the bug is real, and to Xiaoran Wang for review. Back-patch to all supported branches. Discussion: https://postgr.es/m/1393953.1698353013@sss.pgh.pa.us Discussion: https://postgr.es/m/CAGjhLkOoBEC9mLsnB42d3CO1vcMx71MLSEuigeABbQ8oRdA6gw@mail.gmail.com
2024-01-12Fix memory leak in connection string validation.Jeff Davis
Introduced in commit c3afe8cf5a. Discussion: https://postgr.es/m/066a65233d3cb4ea27a9e0778d2f1d0dc764b222.camel@j-davis.com Reviewed-by: Nathan Bossart, Tom Lane Backpatch-through: 16
2024-01-12Re-validate connection string in libpqrcv_connect().Jeff Davis
A superuser may create a subscription with password_required=true, but which uses a connection string without a password. Previously, if the owner of such a subscription was changed to a non-superuser, the non-superuser was able to utilize a password from another source (like a password file or the PGPASSWORD environment variable), which should not have been allowed. This commit adds a step to re-validate the connection string before connecting. Reported-by: Jeff Davis Author: Vignesh C Reviewed-by: Peter Smith, Robert Haas, Amit Kapila Discussion: https://www.postgresql.org/message-id/flat/e5892973ae2a80a1a3e0266806640dae3c428100.camel%40j-davis.com Backpatch-through: 16
2024-01-12pg_regress: Disable autoruns for cmd.exe on WindowsMichael Paquier
This is similar to 9886744a361b, to prevent the execution of other programs due to autorun configurations which could influence the postmaster startup. This was originally applied on HEAD as of 83c75ac7fb69 without a backpatch, but the patch has survived CI and buildfarm cycles. I have checked that cmd /d exists down to Windows XP, which should make this change work correctly in the oldest branches still supported. Discussion: https://postgr.es/m/20230922.161551.320043332510268554.horikyota.ntt@gmail.com Backpatch-through: 12
2024-01-12pg_ctl: Disable autoruns for cmd.exe on WindowsMichael Paquier
On Windows, cmd.exe is used to launch the postmaster process to ease its redirection setup. However, cmd.exe may execute other programs at startup due to autorun configurations, which could influence the postmaster startup. This patch adds /D flag to the launcher cmd.exe command line to disable autorun settings written in the registry. This was originally applied on HEAD as of 9886744a361b without a backpatch, but the patch has survived CI and buildfarm cycles. I have checked that cmd /d exists down to Windows XP, which should make this change work correctly in the oldest branches still supported. Reported-by: Hayato Kuroda Author: Kyotaro Horiguchi Reviewed-by: Robert Haas, Michael Paquier Discussion: https://postgr.es/m/20230922.161551.320043332510268554.horikyota.ntt@gmail.com Backpatch-through: 12
2024-01-11Allow subquery pullup to wrap a PlaceHolderVar in another one.Tom Lane
The code for wrapping subquery output expressions in PlaceHolderVars believed that if the expression already was a PlaceHolderVar, it was never necessary to wrap that in another one. That's wrong if the expression is underneath an outer join and involves a lateral reference to outside that scope: failing to add an additional PHV risks evaluating the expression at the wrong place and hence not forcing it to null when the outer join should do so. This is an oversight in commit 9e7e29c75, which added logic to forcibly wrap lateral-reference Vars in PlaceHolderVars, but didn't see that the adjacent case for PlaceHolderVars needed the same treatment. The test case we have for this doesn't fail before 4be058fe9, but now that I see the problem I wonder if it is possible to demonstrate related errors before that. That's moot though, since all such branches are out of support. Per bug #18284 from Holger Reise. Back-patch to all supported branches. Discussion: https://postgr.es/m/18284-47505a20c23647f8@postgresql.org
2024-01-10Restore initdb's old behavior of always setting the lc_xxx GUCs.Tom Lane
In commit 3e51b278d I (tgl) caused initdb to leave lc_messages and other lc_xxx GUCs commented-out in the installed postgresql.conf file if they were going to be set to 'C'. This was a hack for cosmetic purposes, and it was buggy because lc_messages' wired-in default is not 'C' but '' (empty string). That led to --no-locale not having the expected effect, since the postmaster would then obtain lc_messages from its startup environment. Let's just revert to the prior behavior of always de-commenting the lc_xxx entries; the argument for changing that longstanding behavior was weak in the first place. Also, fix postgresql.conf.sample's erroneous claim that the default value of lc_messages is 'C'. I suspect that was what misled me into making this mistake in the first place. Report and patch by Kyotaro Horiguchi. Back-patch to v16 where the problem was introduced. Discussion: https://postgr.es/m/20231122.162700.1995154567625541112.horikyota.ntt@gmail.com
2024-01-10Handle WindowClause.runCondition in tree walker/mutator functions.Tom Lane
Commit 9d9c02ccd, which added the notion of a "run condition" for window functions, neglected to teach nodeFuncs.c to process the new field. Remarkably, that doesn't seem to have had any ill effects before we invented Var.varnullingrels, but now it can cause visible failures in join-removal scenarios. I have no faith that there's not reachable problems in v15 too, so back-patch the code change to v15 where 9d9c02ccd came in. The test case seems irrelevant to v15, though. Per bug #18277 from Zuming Jiang. Diagnosis and patch by Richard Guo. Discussion: https://postgr.es/m/18277-089ead83b329a2fd@postgresql.org
2024-01-08Fix indentation in ExecParallelHashIncreaseNumBatches()Alexander Korotkov
Backpatch-through: 12
2024-01-07Fix oversized memory allocation in Parallel Hash JoinAlexander Korotkov
During the calculations of the maximum for the number of buckets, take into account that later we round that to the next power of 2. Reported-by: Karen Talarico Bug: #16925 Discussion: https://postgr.es/m/16925-ec96d83529d0d629%40postgresql.org Author: Thomas Munro, Andrei Lepikhov, Alexander Korotkov Reviewed-by: Alena Rybakina Backpatch-through: 12
2024-01-05Fix corruption of local buffer state during extend of temp relationMichael Paquier
A typo has been introduced by 31966b151e6a when updating the state of a local buffer when a temporary relation is extended, for the case of a block included in the relation range extended, when it is already found in the hash table holding the local buffers. In this case, BM_VALID should be cleared, but the buffer state was changed so as BM_VALID remained while clearing the other flags. As reported on the thread, it was possible to corrupt the state of the local buffers on ENOSPC, but the states would be corrupted on any kind of ERROR during the relation extend (like partial writes or some other errno). Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Richard Guo, Alexander Lakhin, Michael Paquier Discussion: https://postgr.es/m/18259-6e256429825dd435@postgresql.org Backpatch-through: 16
2024-01-04Fix use of incorrect TupleTableSlot in DISTINCT aggregatesDavid Rowley
1349d2790 added code to allow DISTINCT and ORDER BY aggregates to work more efficiently by using presorted input. That commit added some code that made use of the AggState's tmpcontext and adjusted the ecxt_outertuple and ecxt_innertuple slots before checking if the current row is distinct from the previously seen row. That code forgot to set the TupleTableSlots back to what they were originally, which could result in errors such as: ERROR: attribute 1 of type record has wrong type This only affects aggregate functions which have multiple arguments when DISTINCT is used. For example: string_agg(DISTINCT col, ', ') Thanks to Tom Lane for identifying the breaking commit. Bug: #18264 Reported-by: Vojtěch Beneš Discussion: https://postgr.es/m/18264-e363593d7e9feb7d@postgresql.org Backpatch-through: 16, where 1349d2790 was added
2024-01-03Avoid masking EOF (no-password-supplied) conditions in auth.c.Tom Lane
CheckPWChallengeAuth() would return STATUS_ERROR if the user does not exist or has no password assigned, even if the client disconnected without responding to the password challenge (as libpq often will, for example). We should return STATUS_EOF in that case, and the lower-level functions do, but this code level got it wrong since the refactoring done in 7ac955b34. This breaks the intent of not logging anything for EOF cases (cf. comments in auth_failed()) and might also confuse users of ClientAuthentication_hook. Per report from Liu Lang. Back-patch to all supported versions. Discussion: https://postgr.es/m/b725238c-539d-cb09-2bff-b5e6cb2c069c@esgyn.cn
2023-12-29In pg_dump, don't dump a stats object unless dumping underlying table.Tom Lane
If the underlying table isn't being dumped, it's useless to dump an extended statistics object; it'll just cause errors at restore. We have always applied similar policies to, say, indexes. (When and if we get cross-table stats objects, it might be profitable to think a little harder about what to do with them. But for now there seems no point in considering a stats object as anything but an appendage of its table.) Rian McGuire and Tom Lane, per report from Rian McGuire. Back-patch to supported branches. Discussion: https://postgr.es/m/7075d3aa-3f05-44a5-b68f-47dc6a8a0550@buildkite.com
2023-12-26Fix failure to verify PGC_[SU_]BACKEND GUCs in pg_file_settings view.Tom Lane
set_config_option() bails out early if it detects that the option to be set is PGC_BACKEND or PGC_SU_BACKEND class and we're reading the config file in a postmaster child; we don't want to apply any new value in such a case. That's fine as far as it goes, but it fails to consider the requirements of the pg_file_settings view: for that, we need to check validity of the value even though we have no intention to apply it. Because we didn't, even very silly values for affected GUCs would be reported as valid by the view. There are only half a dozen such GUCs, which perhaps explains why this got overlooked for so long. Fix by continuing when changeVal is false; this parallels the logic in some other early-exit paths. Also, the check added by commit 924bcf4f1 to prevent GUC changes in parallel workers seems a few bricks shy of a load: it's evidently assuming that ereport(elevel, ...) won't return. Make sure we bail out if it does. The lack of trouble reports suggests that this is only a latent bug, i.e. parallel workers don't actually reach here with elevel < ERROR. (Per the code coverage report, we never reach here at all in the regression suite.) But we clearly don't want to risk proceeding if that does happen. Per report from Rıdvan Korkmaz. These are ancient bugs, so back-patch to all supported branches. Discussion: https://postgr.es/m/2089235.1703617353@sss.pgh.pa.us
2023-12-26Fix mistaken file name in plpython's meson recipe.Tom Lane
Brown-paper-bag bug in commit 58c3151bb. Per buildfarm.
2023-12-26Hide warnings from Python headers when using gcc-compatible compiler.Tom Lane
Like commit 388e80132, use "#pragma GCC system_header" to silence warnings appearing within the Python headers, since newer Python versions no longer worry about some restrictions we still use like -Wdeclaration-after-statement. This patch improves on 388e80132 by inventing a separate wrapper header file, allowing the pragma to be tightly scoped to just the Python headers and not other stuff we have laying about in plpython.h. I applied the same technique to plperl for the same reason: the original patch suppressed warnings for a good deal of our own code, not only the Perl headers. Like the previous commit, back-patch to supported branches. Peter Eisentraut and Tom Lane Discussion: https://postgr.es/m/ae523163-6d2a-4b81-a875-832e48dec502@eisentraut.org
2023-12-23Set readline-relevant ENV vars in interactive_psql(), not caller.Tom Lane
Commit 664d75753 pulled 010_tab_completion.pl's infrastructure for invoking an interactive psql session out into a generally-useful test function, but it didn't move enough stuff. We need to set up various environment variables that readline will look at, both to ensure stability of test results and to prevent test actions from cluttering the calling user's ~/.psql_history. Expecting calling scripts to remember to do that is too failure-prone: the other existing caller 001_password.pl did not do it. Hence, remove those initialization steps from 010_tab_completion.pl and put them into interactive_psql(). Since interactive_psql was already making a local ENV hash, this has no effect on calling scripts. Discussion: https://postgr.es/m/794610.1703182896@sss.pgh.pa.us
2023-12-21Avoid trying to fetch metapage of an SPGist partitioned index.Tom Lane
This is necessary when spgcanreturn() is invoked on a partitioned index, and the failure might be reachable in other scenarios as well. The rest of what spgGetCache() does is perfectly sensible for a partitioned index, so we should allow it to go through. I think the main takeaway from this is that we lack sufficient test coverage for non-btree partitioned indexes. Therefore, I added simple test cases for brin and gin as well as spgist (hash and gist AMs were covered already in indexing.sql). Per bug #18256 from Alexander Lakhin. Although the known test case only fails since v16 (3c569049b), I've got no faith at all that there aren't other ways to reach this problem; so back-patch to all supported branches. Discussion: https://postgr.es/m/18256-0b0e1b6e4a620f1b@postgresql.org
2023-12-21Fix BEFORE ROW trigger handling in cross-partition MERGE update.Dean Rasheed
Fix a bug during MERGE if a cross-partition update is attempted on a partitioned table with a BEFORE DELETE ROW trigger that returns NULL, to prevent the update. This would cause an error to be thrown, or an assert failure in an assert-enabled build. This was an oversight in 9321c79c86, which failed to properly distinguish a DELETE prevented by a trigger from one prevented by a concurrent update. Fix by having ExecDelete() return the TM_Result status to ExecCrossPartitionUpdate(), so that it can distinguish the two cases, and make ExecCrossPartitionUpdate() return the TM_Result status to ExecUpdateAct(), so that it can return the correct status from a concurrent update. In addition, ensure that the command tag is correctly updated by having ExecMergeMatched() pass canSetTag to ExecUpdateAct(), rather than passing false, so that it updates the command tag if it does a cross-partition update, making this code path in ExecMergeMatched() consistent with ExecUpdate(). Per bug #18238 from Alexander Lakhin. Back-patch to v15, where MERGE was introduced. Dean Rasheed, reviewed by Richard Guo and Jian He. Discussion: https://postgr.es/m/18238-2f2bdc7f720180b9%40postgresql.org
2023-12-20Fix unchecked return value from strdupDaniel Gustafsson
The pg_dump compression was using strdup() instead of pg_strdup() and failed to check the returned pointer for out-of-memory before dereferencing it. Fix by using pg_strdup() instead which probably was the intention here in the original patch. Backpatch to v16 where pg_dump compression was introduced. Reviewed-by: Tristan Partin <tristan@neon.tech> Reviewed-by: Nathan Bossart <nathandbossart@gmail.com> Discussion: https://postgr.es/m/CC661D60-6F4C-474D-B9CF-E789ACA3CEFC@yesql.se Backpatch-through: 16
2023-12-15Fix bugs in manipulation of large objects.Tom Lane
In v16 and up (since commit afbfc0298), large object ownership checking has been broken because object_ownercheck() didn't take care of the discrepancy between our object-address representation of large objects (classId == LargeObjectRelationId) and the catalog where their ownership info is actually stored (LargeObjectMetadataRelationId). This resulted in failures such as "unrecognized class ID: 2613" when trying to update blob properties as a non-superuser. Poking around for related bugs, I found that AlterObjectOwner_internal would pass the wrong classId to the PostAlterHook in the no-op code path where the large object already has the desired owner. Also, recordExtObjInitPriv checked for the wrong classId; that bug is only latent because the stanza is dead code anyway, but as long as we're carrying it around it should be less wrong. These bugs are quite old. In HEAD, we can reduce the scope for future bugs of this ilk by changing AlterObjectOwner_internal's API to let the translation happen inside that function, rather than requiring callers to know about it. A more bulletproof fix, perhaps, would be to start using LargeObjectMetadataRelationId as the dependency and object-address classId for blobs. However that has substantial risk of breaking third-party code; even within our own code, it'd create hassles for pg_dump which would have to cope with a version-dependent representation. For now, keep the status quo. Discussion: https://postgr.es/m/2650449.1702497209@sss.pgh.pa.us
2023-12-14Fix description of I/O timing info for shared buffers in EXPLAIN (BUFFERS)Michael Paquier
This fixes an error introduced by efb0ef909f60, that changed the description of this field to "shared/local" while these I/O timings relate to shared buffers. This information is available when track_io_timing is enabled. Note that HEAD has added new counters for local buffers in 295c36c0c1fa, so there is no need to touch it. The description is updated to "shared" to be compatible with HEAD. Per discussion with Nazir Bilal Yavuz and Hubert Depesz Lubaczewski, whose EXPLAIN analyzer tool was not actually able to parse the previous term because of the slash character. Discussion: https://postgr.es/m/ZTCTiUqm_H3iBihl@paquier.xyz Backpatch-through: 15
2023-12-12Prevent tuples to be marked as dead in subtransactions on standbysMichael Paquier
Dead tuples are ignored and are not marked as dead during recovery, as it can lead to MVCC issues on a standby because its xmin may not match with the primary. This information is tracked by a field called "xactStartedInRecovery" in the transaction state data, switched on when starting a transaction in recovery. Unfortunately, this information was not correctly tracked when starting a subtransaction, because the transaction state used for the subtransaction did not update "xactStartedInRecovery" based on the state of its parent. This would cause index scans done in subtransactions to return inconsistent data, depending on how the xmin of the primary and/or the standby evolved. This is broken since the introduction of hot standby in efc16ea52067, so backpatch all the way down. Author: Fei Changhong Reviewed-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/tencent_C4D907A5093C071A029712E73B43C6512706@qq.com Backpatch-through: 12
2023-12-12Fix typo in commentDaniel Gustafsson
Commit 98e675ed7af accidentally mistyped IDENTIFY_SYSTEM as IDENTIFY_SERVER. Backpatch to all supported branches. Reported-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/68138521-5345-8780-4390-1474afdcba1f@gmail.com
2023-12-11Be more wary about OpenSSL not setting errno on error.Tom Lane
OpenSSL will sometimes return SSL_ERROR_SYSCALL without having set errno; this is apparently a reflection of recv(2)'s habit of not setting errno when reporting EOF. Ensure that we treat such cases the same as read EOF. Previously, we'd frequently report them like "could not accept SSL connection: Success" which is confusing, or worse report them with an unrelated errno left over from some previous syscall. To fix, ensure that errno is zeroed immediately before the call, and report its value only when it's not zero afterwards; otherwise report EOF. For consistency, I've applied the same coding pattern in libpq's pqsecure_raw_read(). Bare recv(2) shouldn't really return -1 without setting errno, but in case it does we might as well cope. Per report from Andres Freund. Back-patch to all supported versions. Discussion: https://postgr.es/m/20231208181451.deqnflwxqoehhxpe@awork3.anarazel.de
2023-12-11Fix an undetected deadlock due to apply worker.Amit Kapila
The apply worker needs to update the state of the subscription tables to 'READY' during the synchronization phase which requires locking the corresponding subscription. The apply worker also waits for the subscription tables to reach the 'SYNCDONE' state after holding the locks on the subscription and the wait is done using WaitLatch. The 'SYNCDONE' state is changed by tablesync workers again by locking the corresponding subscription. Both the state updates use AccessShareLock mode to lock the subscription, so they can't block each other. However, a backend can simultaneously try to acquire a lock on the same subscription using AccessExclusiveLock mode to alter the subscription. Now, the backend's wait on a lock can sneak in between the apply worker and table sync worker causing deadlock. In other words, apply_worker waits for tablesync worker which waits for backend, and backend waits for apply worker. This is not detected by the deadlock detector because apply worker uses WaitLatch. The fix is to release existing locks in apply worker before it starts to wait for tablesync worker to change the state. Reported-by: Tomas Vondra Author: Shlok Kyal Reviewed-by: Amit Kapila, Peter Smith Backpatch-through: 12 Discussion: https://postgr.es/m/d291bb50-12c4-e8af-2af2-7bb9bb4d8e3e@enterprisedb.com