summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2024-01-11Fix an intermetant BF failure in 003_logical_slots.Amit Kapila
During upgrade, when pg_restore performs CREATE DATABASE, bgwriter or checkpointer may flush buffers and hold a file handle for pg_largeobject, so later TRUNCATE pg_largeobject command will fail if OS (such as older Windows versions) doesn't remove an unlinked file completely till it's open. The probability of seeing this behavior is higher in this test because we use wal_level as logical via allows_streaming => 'logical' which in turn set shared_buffers as 1MB and make it more probable for bgwriter to hold the file handle. Diagnosed-by: Alexander Lakhin Author: Hayato Kuroda, Amit Kapila Reviewed-by: Alexander Lakhin Discussion: https://postgr.es/m/TYAPR01MB5866AB7FD922CE30A2565B8BF5A8A@TYAPR01MB5866.jpnprd01.prod.outlook.com
2024-01-11pg_regress: Disable autoruns for cmd.exe on WindowsMichael Paquier
This is similar to 9886744a361b, to prevent the execution of other programs due to autorun configurations which could influence the postmaster startup. Like the other change, no backpatch is done. Discussion: https://postgr.es/m/20230922.161551.320043332510268554.horikyota.ntt@gmail.com
2024-01-10Restore initdb's old behavior of always setting the lc_xxx GUCs.Tom Lane
In commit 3e51b278d I (tgl) caused initdb to leave lc_messages and other lc_xxx GUCs commented-out in the installed postgresql.conf file if they were going to be set to 'C'. This was a hack for cosmetic purposes, and it was buggy because lc_messages' wired-in default is not 'C' but '' (empty string). That led to --no-locale not having the expected effect, since the postmaster would then obtain lc_messages from its startup environment. Let's just revert to the prior behavior of always de-commenting the lc_xxx entries; the argument for changing that longstanding behavior was weak in the first place. Also, fix postgresql.conf.sample's erroneous claim that the default value of lc_messages is 'C'. I suspect that was what misled me into making this mistake in the first place. Report and patch by Kyotaro Horiguchi. Back-patch to v16 where the problem was introduced. Discussion: https://postgr.es/m/20231122.162700.1995154567625541112.horikyota.ntt@gmail.com
2024-01-10Allow noise semicolons ending psql \sf, \ef, \sv, \ev commands.Tom Lane
Many psql backslash commands tolerate trailing semicolons, even though that's not part of the official syntax. These did not. They tried to, by passing semicolon = true to psql_scan_slash_option, but that function ignored this parameter in OT_WHOLE_LINE mode. Teach it to do the right thing, and remove the now-duplicative logic in exec_command_help. Discussion: https://postgr.es/m/2012251.1704746912@sss.pgh.pa.us
2024-01-10Fix Asserts in calc_non_nestloop_required_outer().Tom Lane
These were not testing the same thing as the comparable Assert in calc_nestloop_required_outer(), because we neglected to map the given Paths' relids to top-level relids. When considering a partition child join the latter is the correct thing to do. This oversight is old, but since it's only an overly-weak Assert check there doesn't seem to be much value in back-patching. Richard Guo (with cosmetic changes and comment updates by me) Discussion: https://postgr.es/m/CAMbWs49sqbe9GBZ8sy8dSfKRNURgicR85HX8vgzcgQsPF0XY1w@mail.gmail.com
2024-01-10Handle WindowClause.runCondition in tree walker/mutator functions.Tom Lane
Commit 9d9c02ccd, which added the notion of a "run condition" for window functions, neglected to teach nodeFuncs.c to process the new field. Remarkably, that doesn't seem to have had any ill effects before we invented Var.varnullingrels, but now it can cause visible failures in join-removal scenarios. I have no faith that there's not reachable problems in v15 too, so back-patch the code change to v15 where 9d9c02ccd came in. The test case seems irrelevant to v15, though. Per bug #18277 from Zuming Jiang. Diagnosis and patch by Richard Guo. Discussion: https://postgr.es/m/18277-089ead83b329a2fd@postgresql.org
2024-01-10Show the default checkpoint method in the pg_basebackup help messageMagnus Hagander
Author: Michael Banck Reviewed-By: Aleksander Alekseev, Michael Paquier, Peter Eisentraut Discussion: https://postgr.es/m/6530f954.170a0220.5637c.9209@mx.google.com
2024-01-10pg_ctl: Disable autoruns for cmd.exe on WindowsMichael Paquier
On Windows, cmd.exe is used to launch the postmaster process to ease its redirection setup. However, cmd.exe may execute other programs at startup due to autorun configurations, which could influence the postmaster startup. This patch adds /D flag to the launcher cmd.exe command line to disable autorun settings written in the registry. This is arguably a bug, but no backpatch is done now out of caution. Reported-by: Hayato Kuroda Author: Kyotaro Horiguchi Reviewed-by: Robert Haas, Michael Paquier Discussion: https://postgr.es/m/20230922.161551.320043332510268554.horikyota.ntt@gmail.com
2024-01-09Cross-check lists of predefined LWLocks.Nathan Bossart
Both lwlocknames.txt and wait_event_names.txt contain a list of all the predefined LWLocks, i.e., those with predefined positions within MainLWLockArray. It is easy to miss one or the other, especially since the list in wait_event_names.txt omits the "Lock" suffix from all the LWLock wait events. This commit adds a cross- check of these lists to the script that generates lwlocknames.h. If the lists do not match exactly, building will fail. Suggested-by: Robert Haas Reviewed-by: Robert Haas, Michael Paquier, Bertrand Drouvot Discussion: https://postgr.es/m/20240102173120.GA1061678%40nathanxps13
2024-01-09Add new function, PQchangePassword(), to libpqJoe Conway
Essentially this moves the non-interactive part of psql's "\password" command into an exported client function. The password is not sent to the server in cleartext because it is "encrypted" (in the case of scram and md5 it is actually hashed, but we have called these encrypted passwords for a long time now) on the client side. This is good because it ensures the cleartext password is never known by the server, and therefore won't end up in logs, pg_stat displays, etc. In other words, it exists for the same reason as PQencryptPasswordConn(), but is more convenient as it both builds and runs the "ALTER USER" command for you. PQchangePassword() uses PQencryptPasswordConn() to do the password encryption. PQencryptPasswordConn() is passed a NULL for the algorithm argument, hence encryption is done according to the server's password_encryption setting. Also modify the psql client to use the new function. That provides a builtin test case. Ultimately drivers built on top of libpq should expose this function and its use should be generally encouraged over doing ALTER USER directly for password changes. Author: Joe Conway Reviewed-by: Tom Lane Discussion: https://postgr.es/m/flat/b75955f7-e8cc-4bbd-817f-ef536bacbe93%40joeconway.com
2024-01-09An addition to 8c441c08279Alexander Korotkov
Given that now SJE doesn't work with result relation, turn a code dealing with that into an assert that it shouldn't happen.
2024-01-09Forbid SJE with result relationAlexander Korotkov
The target relation for INSERT/UPDATE/DELETE/MERGE has a different behavior than other relations in EvalPlanQual() and RETURNING clause. This is why we forbid target relation to be either source or target relation in SJE. It's not clear if we could ever support this. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/b9e8f460-f9a6-0e9b-e8ba-60d59f0bc22c%40gmail.com
2024-01-09Fix misuse of RelOptInfo.unique_for_rels cache by SJEAlexander Korotkov
When SJE uses RelOptInfo.unique_for_rels cache, it passes filtered quals to innerrel_is_unique_ext(). That might lead to an invalid match to cache entries made by previous non self-join checking calls. Add UniqueRelInfo.self_join flag to prevent such cases. Also, fix that SJE should require a strict match of outerrelids to make sure UniqueRelInfo.extra_clauses are valid. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/4788f781-31bd-9796-d7d6-588a751c8787%40gmail.com
2024-01-08Make dblink interruptible, via new libpqsrv APIs.Noah Misch
This replaces dblink's blocking libpq calls, allowing cancellation and allowing DROP DATABASE (of a database not involved in the query). Apart from explicit dblink_cancel_query() calls, dblink still doesn't cancel the remote side. The replacement for the blocking calls consists of new, general-purpose query execution wrappers in the libpqsrv facility. Out-of-tree extensions should adopt these. Use them in postgres_fdw, replacing a local implementation from which the libpqsrv implementation derives. This is a bug fix for dblink. Code inspection identified the bug at least thirteen years ago, but user complaints have not appeared. Hence, no back-patch for now. Discussion: https://postgr.es/m/20231122012945.74@rfd.leadboat.com
2024-01-08Remove excess #include "utils/wait_event.h".Noah Misch
This simplifies copying libpq/libpq-be-fe-helpers.h into extensions, because some supported PostgreSQL versions lack the other header. Discussion: https://postgr.es/m/20231122012945.74@rfd.leadboat.com
2024-01-08Fix missing word in comment.Noah Misch
2024-01-08Fix indentation in ExecParallelHashIncreaseNumBatches()Alexander Korotkov
Backpatch-through: 12
2024-01-08Allow examine_simple_variable() to work on INSERT RETURNING Vars.Tom Lane
Since commit 599b33b94, this function assumed that every RTE_RELATION RangeTblEntry would have an associated RelOptInfo. But that's not so: we only build RelOptInfos for relations that are scanned by the query. In particular the target of an INSERT won't have one, so that Vars appearing in an INSERT ... RETURNING list will not have an associated RelOptInfo. This apparently wasn't a problem before commit f7816aec2 taught examine_simple_variable() to drill down into CTEs containing INSERT RETURNING, but it is now. To fix, add a fallback code path that gets the userid to use directly from the RTEPermissionInfo associated with the RTE. (Sadly, we must have two code paths, because not every RTE has a RTEPermissionInfo either.) Per report from Alexander Lakhin. No back-patch, since the case is apparently unreachable before f7816aec2. Discussion: https://postgr.es/m/608a4886-6c60-0f9e-97d5-591256bd4150@gmail.com
2024-01-08Strengthen tests for 5ef34a8fc3Alexander Korotkov
The test query in 5ef34a8fc3 is running over the empty emp1 table giving the same (empty) return both with and without the fix. Add one row to that table to make not just the test query plan, but also the test query result different. Reported-by: Richard Guo Bug: #18261 Discussion: https://postgr.es/m/CAMbWs49igjcszLgicb4D1N21_5iNDoxheJ7KFmAcs_z%3DLx6jhg%40mail.gmail.com
2024-01-07Fix oversized memory allocation in Parallel Hash JoinAlexander Korotkov
During the calculations of the maximum for the number of buckets, take into account that later we round that to the next power of 2. Reported-by: Karen Talarico Bug: #16925 Discussion: https://postgr.es/m/16925-ec96d83529d0d629%40postgresql.org Author: Thomas Munro, Andrei Lepikhov, Alexander Korotkov Reviewed-by: Alena Rybakina Backpatch-through: 12
2024-01-06Fix the issue that SJE mistakenly omits qual clausesAlexander Korotkov
When the SJE code handles the transfer of qual clauses from the removed relation to the remaining one, it replaces the Vars of the removed relation with the Vars of the remaining relation for each clause, and then reintegrates these clauses into the appropriate restriction or join clause lists, while attempting to avoid duplicates. However, the code compares RestrictInfo->clause to determine if two clauses are duplicates. This is just flat wrong. Two RestrictInfos with the same clause can have different required_relids, incompatible_relids, is_pushed_down, and so on. This can cause qual clauses to be mistakenly omitted, leading to wrong results. This patch fixes it by comparing the entire RestrictInfos not just their clauses ignoring 'rinfo_serial' field (otherwise almost all RestrictInfos will be unique). Making 'rinfo_serial' equal_ignore would break other code. This is why this commit implements our own comparison function for checking the equality of RestrictInfos. Reported-by: Zuming Jiang Bug: #18261 Discussion: https://postgr.es/m/18261-2a75d748c928609b%40postgresql.org Author: Richard Guo
2024-01-05Clean up some edge cases in plpgsql's %TYPE parsing.Tom Lane
Support referencing a composite-type variable in %TYPE. Remove the undocumented, untested, and pretty useless ability to have the subject of %TYPE be an (unqualified) type name. You get the same result by just not writing %TYPE. Add or adjust some test cases to improve code coverage here. Discussion: https://postgr.es/m/716852.1704402127@sss.pgh.pa.us
2024-01-05Add copyright notices to a few perl scripts that don't have themAndrew Dunstan
2024-01-05Fix corruption of local buffer state during extend of temp relationMichael Paquier
A typo has been introduced by 31966b151e6a when updating the state of a local buffer when a temporary relation is extended, for the case of a block included in the relation range extended, when it is already found in the hash table holding the local buffers. In this case, BM_VALID should be cleared, but the buffer state was changed so as BM_VALID remained while clearing the other flags. As reported on the thread, it was possible to corrupt the state of the local buffers on ENOSPC, but the states would be corrupted on any kind of ERROR during the relation extend (like partial writes or some other errno). Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Richard Guo, Alexander Lakhin, Michael Paquier Discussion: https://postgr.es/m/18259-6e256429825dd435@postgresql.org Backpatch-through: 16
2024-01-04Teach estimate_array_length() to use statistics where available.Tom Lane
If we have DECHIST statistics about the argument expression, use the average number of distinct elements as the array length estimate. (It'd be better to use the average total number of elements, but that is not currently calculated by compute_array_stats(), and it's unclear that it'd be worth extra effort to get.) To do this, we have to change the signature of estimate_array_length to pass the "root" pointer. While at it, also change its result type to "double". That's probably not really necessary, but it avoids any risk of overflow of the value extracted from DECHIST. All existing callers are going to use the result in a "double" calculation anyway. Paul Jungwirth, reviewed by Jian He and myself Discussion: https://postgr.es/m/CA+renyUnM2d+SmrxKpDuAdpiq6FOM=FByvi6aS6yi__qyf6j9A@mail.gmail.com
2024-01-04Add macros for looping through a List without a ListCell.Nathan Bossart
Many foreach loops only use the ListCell pointer to retrieve the content of the cell, like so: ListCell *lc; foreach(lc, mylist) { int myint = lfirst_int(lc); ... } This commit adds a few convenience macros that automatically declare the loop variable and retrieve the current cell's contents. This allows us to rewrite the previous loop like this: foreach_int(myint, mylist) { ... } This commit also adjusts a few existing loops in order to add coverage for the new/adjusted macros. There is presently no plan to bulk update all foreach loops, as that could introduce a significant amount of back-patching pain. Instead, these macros are primarily intended for use in new code. Author: Jelte Fennema-Nio Reviewed-by: David Rowley, Alvaro Herrera, Vignesh C, Tom Lane Discussion: https://postgr.es/m/CAGECzQSwXKnxGwW1_Q5JE%2B8Ja20kyAbhBHO04vVrQsLcDciwXA%40mail.gmail.com
2024-01-04In plpgsql, allow %TYPE and %ROWTYPE to be followed by array decoration.Tom Lane
This provides the useful ability to declare a variable that is an array of the type of some other variable or some table column. Quan Zongliang, Pavel Stehule Discussion: https://postgr.es/m/ec4523e1-9e7e-f3ef-f9ce-bafd680ad6f6@yeah.net
2024-01-04ALTER TABLE command to change generation expressionPeter Eisentraut
This adds a new ALTER TABLE subcommand ALTER COLUMN ... SET EXPRESSION that changes the generation expression of a generated column. The syntax is not standard but was adapted from other SQL implementations. This command causes a table rewrite, using the usual ALTER TABLE mechanisms. The implementation is similar to and makes use of some of the infrastructure of the SET DATA TYPE subcommand (for example, rebuilding constraints and indexes afterwards). The new command requires a new pass in AlterTablePass, and the ADD COLUMN pass had to be moved earlier so that combinations of ADD COLUMN and SET EXPRESSION can work. Author: Amul Sul <sulamul@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b94yyJeGA-5M951_Lr+KfZokOp-2kXicpmEhi5FXhBeTog@mail.gmail.com
2024-01-04Fix use of incorrect TupleTableSlot in DISTINCT aggregatesDavid Rowley
1349d2790 added code to allow DISTINCT and ORDER BY aggregates to work more efficiently by using presorted input. That commit added some code that made use of the AggState's tmpcontext and adjusted the ecxt_outertuple and ecxt_innertuple slots before checking if the current row is distinct from the previously seen row. That code forgot to set the TupleTableSlots back to what they were originally, which could result in errors such as: ERROR: attribute 1 of type record has wrong type This only affects aggregate functions which have multiple arguments when DISTINCT is used. For example: string_agg(DISTINCT col, ', ') Thanks to Tom Lane for identifying the breaking commit. Bug: #18264 Reported-by: Vojtěch Beneš Discussion: https://postgr.es/m/18264-e363593d7e9feb7d@postgresql.org Backpatch-through: 16, where 1349d2790 was added
2024-01-04Track conflict_reason in pg_replication_slots.Amit Kapila
This patch changes the existing 'conflicting' field to 'conflict_reason' in pg_replication_slots. This new field indicates the reason for the logical slot's conflict with recovery. It is always NULL for physical slots, as well as for logical slots which are not invalidated. The non-NULL values indicate that the slot is marked as invalidated. Possible values are: wal_removed = required WAL has been removed. rows_removed = required rows have been removed. wal_level_insufficient = the primary doesn't have a wal_level sufficient to perform logical decoding. The existing users of 'conflicting' column can get the same answer by using 'conflict_reason' IS NOT NULL. Author: Shveta Malik Reviewed-by: Amit Kapila, Bertrand Drouvot, Michael Paquier Discussion: https://postgr.es/m/ZYOE8IguqTbp-seF@paquier.xyz
2024-01-03Update copyright for 2024Bruce Momjian
Reported-by: Michael Paquier Discussion: https://postgr.es/m/ZZKTDPxBBMt3C0J9@paquier.xyz Backpatch-through: 12
2024-01-03Avoid masking EOF (no-password-supplied) conditions in auth.c.Tom Lane
CheckPWChallengeAuth() would return STATUS_ERROR if the user does not exist or has no password assigned, even if the client disconnected without responding to the password challenge (as libpq often will, for example). We should return STATUS_EOF in that case, and the lower-level functions do, but this code level got it wrong since the refactoring done in 7ac955b34. This breaks the intent of not logging anything for EOF cases (cf. comments in auth_failed()) and might also confuse users of ClientAuthentication_hook. Per report from Liu Lang. Back-patch to all supported versions. Discussion: https://postgr.es/m/b725238c-539d-cb09-2bff-b5e6cb2c069c@esgyn.cn
2024-01-03Second attempt at organizing jsonpath operators and methodsPeter Eisentraut
Second attempt at 283a95da923. Since we can't reorder the enum values of JsonPathItemType, instead reorder the switch cases where they are used to generally follow the order of the enum values, for better maintainability.
2024-01-03Revert "Reorganise jsonpath operators and methods"Peter Eisentraut
This reverts commit 283a95da923605c1cc148155db2d865d0801b419. The reordering of JsonPathItemType affects the binary on-disk compatibility of the jsonpath type, so we must not change it. Revert for now and consider.
2024-01-03Fix defects in PrepareForIncrementalBackup.Robert Haas
Swap the arguments to TimestampDifferenceMilliseconds so that we get a positive answer instead of zero. Then use the result of that computation instead of ignoring it. Per reports from Alexander Lakhin. Discussion: http://postgr.es/m/8b686764-7ac1-74c3-70f9-b64685a2535f@gmail.com
2024-01-03Make Perl warnings fatal in newly added TAP testsPeter Eisentraut
New TAP tests added by commits 9a17be1e and 4710b67d missed to convert warnings to FATAL. This commit fixes that. Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2024-01-03Reorganise jsonpath operators and methodsPeter Eisentraut
Various jsonpath operators and methods add various keywords, switch cases, and documentation entries in some order. However, they are not consistent; reorder them for better maintainability or readability. Author: Jeevan Chalke <jeevan.chalke@enterprisedb.com> Discussion: https://www.postgresql.org/message-id/flat/CAM2+6=XjTyqrrqHAOj80r0wVQxJSxc0iyib9bPC55uFO9VKatg@mail.gmail.com
2024-01-03Add numeric_int8_opt_error() to optionally suppress errorsPeter Eisentraut
This matches the existing numeric_int4_opt_error() (see commit 16d489b0fe). It will be used by a future JSON-related patch, which wants to report errors in its own way and thus does not want the internal functions to throw any error. Author: Jeevan Chalke <jeevan.chalke@enterprisedb.com> Discussion: https://www.postgresql.org/message-id/flat/CAM2+6=XjTyqrrqHAOj80r0wVQxJSxc0iyib9bPC55uFO9VKatg@mail.gmail.com
2024-01-03Refactor: separate function to find all objects depending on a columnPeter Eisentraut
Move code from ATExecAlterColumnType() that finds the all the objects that depend on the column to a separate function. A future patch will reuse this code. Author: Amul Sul <sulamul@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/CAAJ_b94yyJeGA-5M951_Lr+KfZokOp-2kXicpmEhi5FXhBeTog@mail.gmail.com
2024-01-03Fix some typosMichael Paquier
Author: Dagfinn Ilmari Mannsåker Reviewed-by: Shubham Khanna Discussion: https://postgr.es/m/87le9fmi01.fsf@wibble.ilmari.org
2024-01-03Remove unnecessary PGDATABASE settings from TAP testsMichael Paquier
Some of the TAP tests have been historically setting the environment variable PGDATABASE to 'postgres', which is not needed because PostgreSQL::Test::Cluster already sets it when initialized. This commit removes these explicit setups. Note that the dependency of cluster -a with PGDATABASE (from 1caef31d9e55) is still documented. Author: Bharath Rupireddy Discussion: https://postgr.es/m/CALj2ACXLAz5dW3ZP+Fec8g6jQMMmDyCVT+qdbye2h7QJJmhsdw@mail.gmail.com
2024-01-02Minor fixes for search path cache code.Tom Lane
Avoid leaving a dangling pointer in the unlikely event that nsphash_create fails. Improve comments, and fix formatting by adding typedefs.list entries. Discussion: https://postgr.es/m/3972900.1704145107@sss.pgh.pa.us
2024-01-02Remove Lock suffix from WALSummarizerLock in wait_event_names.txtRobert Haas
Nathan Bossart Discussion: https://postgr.es/m/20240102173120.GA1061678@nathanxps13
2024-01-02jsonpath_exec: fix typo "absense" -> "absence"Robert Haas
Dagfinn Ilmari Mannsåker, reviewed by Shubham Khanna. Discussion: http://postgr.es/m/87le9fmi01.fsf@wibble.ilmari.org
2024-01-02libpq: fix typo "occurences" -> "occurrences" in testsRobert Haas
Dagfinn Ilmari Mannsåker, reviewed by Shubham Khanna. Discussion: http://postgr.es/m/87le9fmi01.fsf@wibble.ilmari.org
2024-01-02gist: fix typo "split(t)ed" -> "split"Robert Haas
Dagfinn Ilmari Mannsåker, reviewed by Shubham Khanna. Discussion: http://postgr.es/m/87le9fmi01.fsf@wibble.ilmari.org
2024-01-02tsquery: fix typo "rewrited" -> "rewritten"Robert Haas
Dagfinn Ilmari Mannsåker, reviewed by Shubham Khanna. Discussion: http://postgr.es/m/87le9fmi01.fsf@wibble.ilmari.org
2024-01-02Fix typos in comments and in one isolation test.Robert Haas
Dagfinn Ilmari Mannsåker, reviewed by Shubham Khanna. Some subtractions by me. Discussion: http://postgr.es/m/87le9fmi01.fsf@wibble.ilmari.org
2024-01-02Add WALSummarizerLock to wait_event_names.txtRobert Haas
Per report from Nathan Bossart. Discussion: http://postgr.es/m/20231227153647.GA601861@nathanxps13
2024-01-02Fix typos in simplehash.hPeter Eisentraut
Author: Richard Guo <guofenglinux@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/18252-d46d27900a277d87@postgresql.org