summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2023-09-26Fix another bug in parent page splitting during GiST index build.Heikki Linnakangas
Yet another bug in the ilk of commits a7ee7c851 and 741b88435. In 741b88435, we took care to clear the memorized location of the downlink when we split the parent page, because splitting the parent page can move the downlink. But we missed that even *updating* a tuple on the parent can move it, because updating a tuple on a gist page is implemented as a delete+insert, so the updated tuple gets moved to the end of the page. This commit fixes the bug in two different ways (belt and suspenders): 1. Clear the downlink when we update a tuple on the parent page, even if it's not split. This the same approach as in commits a7ee7c851 and 741b88435. I also noticed that gistFindCorrectParent did not clear the 'downlinkoffnum' when it stepped to the right sibling. Fix that too, as it seems like a clear bug even though I haven't been able to find a test case to hit that. 2. Change gistFindCorrectParent so that it treats 'downlinkoffnum' merely as a hint. It now always first checks if the downlink is still at that location, and if not, it scans the page like before. That's more robust if there are still more cases where we fail to clear 'downlinkoffnum' that we haven't yet uncovered. With this, it's no longer necessary to meticulously clear 'downlinkoffnum', so this makes the previous fixes unnecessary, but I didn't revert them because it still seems nice to clear it when we know that the downlink has moved. Also add the test case using the same test data that Alexander posted. I tried to reduce it to a smaller test, and I also tried to reproduce this with different test data, but I was not able to, so let's just include what we have. Backpatch to v12, like the previous fixes. Reported-by: Alexander Lakhin Discussion: https://www.postgresql.org/message-id/18129-caca016eaf0c3702@postgresql.org
2023-09-26Fix behavior of "force" in pgstat_report_wal()Michael Paquier
As implemented in 5891c7a8ed8f, setting "force" to true in pgstat_report_wal() causes the routine to not wait for the pgstat shmem lock if it cannot be acquired, in which case the WAL and I/O statistics finish by not being flushed. The origin of the confusion comes from pgstat_flush_wal() and pgstat_flush_io(), that use "nowait" as sole argument. The I/O stats are new in v16. This is the opposite behavior of what has been used in pgstat_report_stat(), where "force" is the opposite of "nowait". In this case, when "force" is true, the routine sets "nowait" to false, which would cause the routine to wait for the pgstat shmem lock, ensuring that the stats are always flushed. When "force" is false, "nowait" is set to true, and the stats would only not be flushed if the pgstat shmem lock can be acquired, returning immediately without flushing the stats if the lock cannot be acquired. This commit changes pgstat_report_wal() so as "force" has the same behavior as in pgstat_report_stat(). There are currently three callers of pgstat_report_wal(): - Two in the checkpointer where force=true during a shutdown and the main checkpointer loop. Now the code behaves so as the stats are always flushed. - One in the main loop of the bgwriter, where force=false. Now the code behaves so as the stats would not be flushed if the pgstat shmem lock could not be acquired. Before this commit, some stats on WAL and I/O could have been lost after a shutdown, for example. Reported-by: Ryoga Yoshida Author: Ryoga Yoshida, Michael Paquier Discussion: https://postgr.es/m/f87a4d7be70530606b864fd1df91718c@oss.nttdata.com Backpatch-through: 15
2023-09-26Fix edge-case for xl_tot_len broken by bae868ca.Thomas Munro
bae868ca removed a check that was still needed. If you had an xl_tot_len at the end of a page that was too small for a record header, but not big enough to span onto the next page, we'd immediately perform the CRC check using a bogus large length. Because of arbitrary coding differences between the CRC implementations on different platforms, nothing very bad happened on common modern systems. On systems using the _sb8.c fallback we could segfault. Restore that check, add a new assertion and supply a test for that case. Back-patch to 12, like bae868ca. Tested-by: Tom Lane <tgl@sss.pgh.pa.us> Tested-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/CA%2BhUKGLCkTT7zYjzOxuLGahBdQ%3DMcF%3Dz5ZvrjSOnW4EDhVjT-g%40mail.gmail.com
2023-09-25pg_dump: tests: Correct test condition for invalid databasesAndres Freund
For some reason I used not_like = { pg_dumpall_dbprivs => 1, } in the test condition of one of the tests added in in c66a7d75e65. That doesn't make sense for two reasons: 1) not_like isn't a valid test condition 2) the database should not be dumped in any of the tests. Due to 1), the test achieved its goal, but clearly the formulation is confusing. Instead use like => {}, with a comment explaining why. Reported-by: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/3ddf79f2-8b7b-a093-11d2-5c739bc64f86@eisentraut.org Backpatch: 11-, like c66a7d75e65
2023-09-25Collect dependency information for parsed CallStmts.Tom Lane
Parse analysis of a CallStmt will inject mutable information, for instance the OID of the called procedure, so that subsequent DDL may create a need to re-parse the CALL. We failed to detect this for CALLs in plpgsql routines, because no dependency information was collected when putting a CallStmt into the plan cache. That could lead to misbehavior or strange errors such as "cache lookup failed". Before commit ee895a655, the issue would only manifest for CALLs appearing in atomic contexts, because we re-planned non-atomic CALLs every time through anyway. It is now apparent that extract_query_dependencies() probably needs a special case for every utility statement type for which stmt_requires_parse_analysis() returns true. I wanted to add something like Assert(!stmt_requires_parse_analysis(...)) when falling out of extract_query_dependencies_walker without doing anything, but there are API issues as well as a more fundamental point: stmt_requires_parse_analysis is supposed to be applied to raw parser output, so it'd be cheating to assume it will give the correct answer for post-parse-analysis trees. I contented myself with adding a comment. Per bug #18131 from Christian Stork. Back-patch to all supported branches. Discussion: https://postgr.es/m/18131-576854e79c5cd264@postgresql.org
2023-09-25Limit to_tsvector_byid's initial array allocation to something sane.Tom Lane
The initial estimate of the number of distinct ParsedWords is just that: an estimate. Don't let it exceed what palloc is willing to allocate. If in fact we need more entries, we'll eventually fail trying to enlarge the array. But if we don't, this allows success on inputs that currently draw "invalid memory alloc request size". Per bug #18080 from Uwe Binder. Back-patch to all supported branches. Discussion: https://postgr.es/m/18080-d5c5e58fef8c99b7@postgresql.org
2023-09-25vacuumdb: Reword --help message for clarityDaniel Gustafsson
The --help output stated that schemas were specified using PATTERN when they in fact aren't pattern matched but are required to be exact matches. This changes to SCHEMA to make that clear. Backpatch through v16 where this was introduced. Author: Kuwamura Masaki <kuwamura@db.is.i.nagoya-u.ac.jp> Discussion: https://postgr.es/m/CAMyC8qp9mXPQd5D6s6CJxvmignsbTqGZwDDB6VYJOn1A8WG38w@mail.gmail.com Backpatch-through: 16
2023-09-25vacuumdb: Fix excluding multiple schemas with -NDaniel Gustafsson
When specifying multiple schemas to exclude with -N parameters, none of the schemas are actually excluded (a single -N worked as expected). This fixes the catalog query to handle multiple exclusions and adds a test for this case. Backpatch to v16 where this was introduced. Author: Nathan Bossart <nathandbossart@gmail.com> Author: Kuwamura Masaki <kuwamura@db.is.i.nagoya-u.ac.jp> Reported-by: Kuwamura Masaki <kuwamura@db.is.i.nagoya-u.ac.jp> Discussion: https://postgr.es/m/CAMyC8qp9mXPQd5D6s6CJxvmignsbTqGZwDDB6VYJOn1A8WG38w@mail.gmail.com Backpatch-through: 16
2023-09-25pg_upgrade: check for types removed in pg12Alvaro Herrera
Commit cda6a8d01d39 removed a few datatypes, but didn't update pg_upgrade --check to throw error if these types are used. So the users find that pg_upgrade --check tells them that everything is fine, only to fail when the real upgrade is attempted. Reviewed-by: Tristan Partin <tristan@neon.tech> Reviewed-by: Suraj Kharage <suraj.kharage@enterprisedb.com> Discussion: https://postgr.es/m/202309201654.ng4ksea25mti@alvherre.pgsql
2023-09-23Don't use Perl pack('Q') in 039_end_of_wal.pl.Thomas Munro
'Q' for 64 bit integers turns out not to work on 32 bit Perl, as revealed by the build farm. Use 'II' instead, and deal with endianness. Back-patch to 12, like bae868ca. Discussion: https://postgr.es/m/ZQ4r1vHcryBsSi_V%40paquier.xyz
2023-09-23Don't trust unvalidated xl_tot_len.Thomas Munro
xl_tot_len comes first in a WAL record. Usually we don't trust it to be the true length until we've validated the record header. If the record header was split across two pages, previously we wouldn't do the validation until after we'd already tried to allocate enough memory to hold the record, which was bad because it might actually be garbage bytes from a recycled WAL file, so we could try to allocate a lot of memory. Release 15 made it worse. Since 70b4f82a4b5, we'd at least generate an end-of-WAL condition if the garbage 4 byte value happened to be > 1GB, but we'd still try to allocate up to 1GB of memory bogusly otherwise. That was an improvement, but unfortunately release 15 tries to allocate another object before that, so you could get a FATAL error and recovery could fail. We can fix both variants of the problem more fundamentally using pre-existing page-level validation, if we just re-order some logic. The new order of operations in the split-header case defers all memory allocation based on xl_tot_len until we've read the following page. At that point we know that its first few bytes are not recycled data, by checking its xlp_pageaddr, and that its xlp_rem_len agrees with xl_tot_len on the preceding page. That is strong evidence that xl_tot_len was truly the start of a record that was logged. This problem was most likely to occur on a standby, because walreceiver.c recycles WAL files without zeroing out trailing regions of each page. We could fix that too, but it wouldn't protect us from rare crash scenarios where the trailing zeroes don't make it to disk. With reliable xl_tot_len validation in place, the ancient policy of considering malloc failure to indicate corruption at end-of-WAL seems quite surprising, but changing that is left for later work. Also included is a new TAP test to exercise various cases of end-of-WAL detection by writing contrived data into the WAL from Perl. Back-patch to 12. We decided not to put this change into the final release of 11. Author: Thomas Munro <thomas.munro@gmail.com> Author: Michael Paquier <michael@paquier.xyz> Reported-by: Alexander Lakhin <exclusion@gmail.com> Reviewed-by: Noah Misch <noah@leadboat.com> (the idea, not the code) Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Sergei Kornilov <sk@zsrv.org> Reviewed-by: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/17928-aa92416a70ff44a2%40postgresql.org
2023-09-22Avoid potential pfree on NULL on OpenSSL errorsDaniel Gustafsson
Guard against the pointer being NULL before pfreeing upon an error returned from OpenSSL. Also handle errors from X509_NAME_print_ex which also can return -1 on memory allocation errors. Backpatch down to v15 where the code was added. Author: Sergey Shinderuk <s.shinderuk@postgrespro.ru> Discussion: https://postgr.es/m/8db5374d-32e0-6abb-d402-40762511eff2@postgrespro.ru Backpatch-through: v15
2023-09-21Fix COMMIT/ROLLBACK AND CHAIN in the presence of subtransactions.Tom Lane
In older branches, COMMIT/ROLLBACK AND CHAIN failed to propagate the current transaction's properties to the new transaction if there was any open subtransaction (unreleased savepoint). Instead, some previous transaction's properties would be restored. This is because the "if (s->chain)" check in CommitTransactionCommand examined the wrong instance of the "chain" flag and falsely concluded that it didn't need to save transaction properties. Our regression tests would have noticed this, except they used identical transaction properties for multiple tests in a row, so that the faulty behavior was not distinguishable from correct behavior. Commit 12d768e70 fixed the problem in v15 and later, but only rather accidentally, because I removed the "if (s->chain)" test to avoid a compiler warning, while not realizing that the warning was flagging a real bug. In v14 and before, remove the if-test and save transaction properties unconditionally; just as in the newer branches, that's not expensive enough to justify thinking harder. Add the comment and extra regression test to v15 and later to forestall any future recurrence, but there's no live bug in those branches. Patch by me, per bug #18118 from Liu Xiang. Back-patch to v12 where the AND CHAIN feature was added. Discussion: https://postgr.es/m/18118-4b72fcbb903aace6@postgresql.org
2023-09-21Update comment about set_join_pathlist_hook().Etsuro Fujita
The comment introduced by commit e7cb7ee14 was a bit too terse, which could lead to extensions doing different things within the hook function than we intend to allow. Extend the comment to explain what they can do within the hook function. Back-patch to all supported branches. In passing, I rephrased a nearby comment that I recently added to the back branches. Reviewed by David Rowley and Andrei Lepikhov. Discussion: https://postgr.es/m/CAPmGK15SBPA1nr3Aqsdm%2BYyS-ay0Ayo2BRYQ8_A2To9eLqwopQ%40mail.gmail.com
2023-09-21Fix vacuumdb to pass buffer-usage-limit with analyze-only modeDavid Rowley
ae78cae3b added the --buffer-usage-limit to vacuumdb to allow it to include the BUFFER_USAGE_LIMIT option in the VACUUM command. Unfortunately, that commit forgot to adjust the code so the option was added to the ANALYZE command when the -Z command line argument was specified. There were no issues with the -z command as that option just adds ANALYZE to the VACUUM command. In passing adjust the code to escape the --buffer-usage-limit option before passing it to the server. It seems nothing beyond a confusing error message could become this lack of escaping as VACUUM cannot be specified in a multi-command string. Reported-by: Ryoga Yoshida Author: Ryoga Yoshida, David Rowley Discussion: https://postgr.es/m/08930c0b541700a5264e5fbf3a685f5a%40oss.nttdata.com Backpatch-through: 16, where ae78cae3b was introduced.
2023-09-19Fix GiST README's explanation of the NSN cross-check.Heikki Linnakangas
The text got the condition backwards, it's "NSN > LSN", not "NSN < LSN". While we're at it, expand it a little for clarity. Reviewed-by: Daniel Gustafsson Discussion: https://www.postgresql.org/message-id/4cb46e18-e688-524a-0f73-b1f03ed5d6ee@iki.fi
2023-09-19Fix assertion failure with PL/Python exceptionsMichael Paquier
PLy_elog() was not able to handle correctly cases where a SPI called failed, which would fill in a DETAIL string able to trigger an assertion. We may want to improve this infrastructure so as it is able to provide any extra detail information provided by an error stack, but this is left as a future improvement as it could impact existing error stacks and any applications that depend on them. For now, the assertion is removed and a regression test is added to cover the case of a failure with a detail string. This problem exists since 2bd78eb8d51c, so backpatch all the way down with tweaks to the regression tests output added where required. Author: Alexander Lakhin Discussion: https://postgr.es/m/18070-ab9c171cbf4ebb0f@postgresql.org Backpatch-through: 11
2023-09-18Don't crash if cursor_to_xmlschema is used on a non-data-returning Portal.Tom Lane
cursor_to_xmlschema() assumed that any Portal must have a tupDesc, which is not so. Add a defensive check. It's plausible that this mistake occurred because of the rather poorly chosen name of the lookup function SPI_cursor_find(), which in such cases is returning something that isn't very much like a cursor. Add some documentation to try to forestall future errors of the same ilk. Report and patch by Boyu Yang (docs changes by me). Back-patch to all supported branches. Discussion: https://postgr.es/m/dd343010-c637-434c-a8cb-418f53bda3b8.yangboyu.yby@alibaba-inc.com
2023-09-15Track nesting depth correctly when drilling down into RECORD Vars.Tom Lane
expandRecordVariable() failed to adjust the parse nesting structure correctly when recursing to inspect an outer-level Var. This could result in assertion failures or core dumps in corner cases. Likewise, get_name_for_var_field() failed to adjust the deparse namespace stack correctly when recursing to inspect an outer-level Var. In this case the likely result was a "bogus varno" error while deparsing a view. Per bug #18077 from Jingzhou Fu. Back-patch to all supported branches. Richard Guo, with some adjustments by me Discussion: https://postgr.es/m/18077-b9db97c6e0ab45d8@postgresql.org
2023-09-14Revert "Improve error message on snapshot import in snapmgr.c"Michael Paquier
This reverts commit a0d87bcd9b57, following a remark from Andres Frend that the new error can be triggered with an incorrect SET TRANSACTION SNAPSHOT command without being really helpful for the user as it uses the internal file name. Discussion: https://postgr.es/m/20230914020724.hlks7vunitvtbbz4@awork3.anarazel.de Backpatch-through: 11
2023-09-13Fix tracking of temp table relation extensions as writesAndres Freund
Karina figured out that I (Andres) confused BufferUsage.temp_blks_written with BufferUsage.local_blks_written in fcdda1e4b5. Tests in core PG can't easily test this, as BufferUsage is just used for EXPLAIN (ANALYZE, BUFFERS) and pg_stat_statements. Thus this commit adds tests for this to pg_stat_statements. Reported-by: Karina Litskevich <litskevichkarina@gmail.com> Author: Karina Litskevich <litskevichkarina@gmail.com> Author: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CACiT8ibxXA6+0amGikbeFhm8B84XdQVo6D0Qfd1pQ1s8zpsnxQ@mail.gmail.com Backpatch: 16-, where fcdda1e4b5 was merged
2023-09-14Improve error message on snapshot import in snapmgr.cMichael Paquier
When a snapshot file fails to be read in ImportSnapshot(), it would issue an ERROR as "invalid snapshot identifier" when opening a stream for it in read-only mode. This error message is reworded to be the same as all the other messages used in this case on failure, which is useful when debugging this area. Thinko introduced by bb446b689b66 where snapshot imports have been added. A backpatch down to 11 is done as this can improve any work related to snapshot imports in older branches. Author: Bharath Rupireddy Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/CALj2ACWmr=3KdxDkm8h7Zn1XxBoF6hdzq8WQyMn2y1OL5RYFrg@mail.gmail.com Backpatch-through: 11
2023-09-14Refactor error messages for unsupported providers in pg_locale.cMichael Paquier
These code paths should not be reached normally, but if they are an error with "(null)" as information for the collation provider would show up if no locale is set, while we can assume that we are referring to libc. This refactors the code so as the provider is always reported even if no locale is set. The name of the function where the error happens is added, while on it, as it can be helpful for debugging. Issue introduced by d87d548cd030, so backpatch down to 16. Author: Michael Paquier, Ranier Vilela Reviewed-by: Jeff Davis, Kyotaro Horiguchi Discussion: https://postgr.es/m/7073610042fcf97e1bea2ce08b7e0214b5e11094.camel@j-davis.com Backpatch-through: 16
2023-09-14Fix incorrect logic in plan dependency recordingDavid Rowley
Both 50e17ad28 and 29f45e299 mistakenly tried to record a plan dependency on a function but mistakenly inverted the OidIsValid test. This meant that we'd record a dependency only when the function's Oid was InvalidOid. Clearly this was meant to *not* record the dependency in that case. 50e17ad28 made this mistake first, then in v15 29f45e299 copied the same mistake. Reported-by: Tom Lane Backpatch-through: 14, where 50e17ad28 first made this mistake Discussion: https://postgr.es/m/2277537.1694301772@sss.pgh.pa.us
2023-09-13Fix the ALTER SUBSCRIPTION to reflect the change in run_as_owner option.Amit Kapila
Reported-by: Jeff Davis Author: Hou Zhijie Reviewed-by: Amit Kapila Backpatch-through: 16 Discussion: http://postgr.es/m/17b62714fd115bd1899afd922954540a5c6a0467.camel@j-davis.com
2023-09-13Fix exception safety bug in typcache.c.Thomas Munro
If an out-of-memory error was thrown at an unfortunate time, ensure_record_cache_typmod_slot_exists() could leak memory and leave behind a global state that produced an infinite loop on the next call. Fix by merging RecordCacheArray and RecordIdentifierArray into a single array. With only one allocation or re-allocation, there is no intermediate state. Back-patch to all supported releases. Reported-by: "James Pang (chaolpan)" <chaolpan@cisco.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/PH0PR11MB519113E738814BDDA702EDADD6EFA%40PH0PR11MB5191.namprd11.prod.outlook.com
2023-09-13Skip psql's TAP test for query cancellation entirely on WindowsMichael Paquier
This changes 020_cancel.pl so as the test is entirely skipped on Windows. This test was already doing nothing under WIN32, except initializing and starting a node without using it so this shaves a few test cycles. Author: Yugo NAGATA Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/20230810125935.22c2922ea5250ba79358965b@sraoss.co.jp Backpatch-through: 15
2023-09-11Translation updatesAlvaro Herrera
This file was missed in the previous update. Source-Git-URL: ssh://git@git.postgresql.org/pgtranslation/messages.git Source-Git-Hash: de944161c6153124a7bf720cb99823ff31b64bab
2023-09-11Translation updatesAlvaro Herrera
Source-Git-URL: ssh://git@git.postgresql.org/pgtranslation/messages.git Source-Git-Hash: 06696f05da005029a2326e1cbb234917a9286914
2023-09-08Stabilize subscription stats test.Masahiko Sawada
The new test added by commit 68a59f9e9 disables the subscription and manually drops the associated replication slot. However, since disabling the subsubscription doesn't wait for a walsender to release the replication slot and exit, pg_drop_replication_slot() could fail. Avoid failure by adding a wait for the replication slot to become inactive. Reported-by: Hou Zhijie, as per buildfarm Reviewed-by: Hou Zhijie Discussion: https://postgr.es/m/OS0PR01MB571682316378379AA34854F694E9A%40OS0PR01MB5716.jpnprd01.prod.outlook.com Backpatch-through: 15
2023-09-07pg_basebackup: Generate valid temporary slot names under PQbackendPID()Michael Paquier
pgbouncer can cause PQbackendPID() to return negative values due to it filling be_pid with random bytes (even these days pid_max can only be set up to 2^22 on 64b machines on Linux, for example, so this cannot happen with normal PID numbers). When this happens, pg_basebackup may generate a temporary slot name that may not be accepted by the parser, leading to spurious failures, like: pg_basebackup: error: could not send replication command ERROR: replication slot name "pg_basebackup_-1201966863" contains invalid character This commit fixes that problem by formatting the result from PQbackendPID() as an unsigned integer when creating the temporary replication slot name, so as the invalid character is gone and the command can be parsed. Author: Jelte Fennema Reviewed-by: Daniel Gustafsson, Nishant Sharma Discussion: https://postgr.es/m/CAGECzQQOGvYfp8ziF4fWQ_o8s2K7ppaoWBQnTmdakn3s-4Z=5g@mail.gmail.com Backpatch-through: 11
2023-09-07Disable 031_recovery_conflict.pl in 15 and 16.Thomas Munro
This test fails due to known bugs in the test and the server. Those will be fixed in master shortly and possibly back-patched a bit later, but in the meantime it is unhelpful for package maintainers if the tests randomly fail, and it's not a good time to make complex changes in 16. This had already been done for older branches prior to 15's release. Now we're about to release 16, and Debian's test builds are regularly failing on one architecture, so let's do the same for 15 and 16. Reported-by: Christoph Berg <myon@debian.org> Reported-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> Discussion: https://postgr.es/m/CALj2ACVr8au2J_9D88UfRCi0JdWhyQDDxAcSVav0B0irx9nXEg%40mail.gmail.com
2023-09-06Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: c5b5ab1da828e1d7a012431e417f0b75b2450c8f
2023-09-05Unify gratuitously different error messagesPeter Eisentraut
Fixup for commit 37188cea0c.
2023-09-04Fix out-of-bound read in gtsvector_picksplit()Michael Paquier
This could lead to an imprecise choice when splitting an index page of a GiST index on a tsvector, deciding which entries should remain on the old page and which entries should move to a new page. This is wrong since tsearch2 has been moved into core with commit 140d4ebcb46e, so backpatch all the way down. This error has been spotted by valgrind. Author: Alexander Lakhin Discussion: https://postgr.es/m/17950-6c80a8d2b94ec695@postgresql.org Backpatch-through: 11
2023-09-04Fix handling of shared statistics with dropped databasesMichael Paquier
Dropping a database while a connection is attempted on it was able to lead to the presence of valid database entries in shared statistics. The issue is that MyDatabaseId was getting set too early than it should, as, if the connection attempted on the dropped database fails when renamed or dropped, the shutdown callback of the shared statistics would finish by re-inserting a correct entry related to the database already dropped. As analyzed by the bug reporters, this issue could lead to phantom entries in the database list maintained by the autovacuum launcher (in rebuild_database_list()) if the database dropped was part of the database list when it was still valid. After the database was dropped, it would remain the highest on the list of databases to considered by the autovacuum worker as things to process. This would prevent autovacuum jobs to happen on all the other databases still present. The commit fixes this issue by delaying setting MyDatabaseId until the database existence has been re-checked with the second scan on pg_database after getting a shared lock on it, and by switching pgstat_update_dbstats() so as nothing happens if MyDatabaseId is not valid. Issue introduced by 5891c7a8ed8f, so backpatch down to 15. Reported-by: Will Mortensen, Jacob Speidel Analyzed-by: Will Mortensen, Jacob Speidel Author: Andres Freund Discussion: https://postgr.es/m/17973-bca1f7d5c14f601e@postgresql.org Backpatch-through: 15
2023-08-31Report syncscan position at end of scan.Heikki Linnakangas
The comment in heapgettup_advance_block() says that it reports the scan position before checking for end of scan, but that didn't match the code. The code was refactored in commit 7ae0ab0ad9, which inadvertently changed the order of the check and reporting. Change it back. This caused a few regression test failures with a small shared_buffers setting like 10 MB. The 'portals' and 'cluster' tests perform seqscans that are large enough that sync seqscans kick in. When the sync scan position is not updated at end of scan, the next seq scan doesn't start at the beginning of the table, and the test queries are sensitive to that. Reviewed-by: Melanie Plageman, David Rowley Discussion: https://www.postgresql.org/message-id/6f991389-ae22-d844-a9d8-9aceb7c01a9a@iki.fi Backpatch-through: 16
2023-08-30Use actual backend IDs in pg_stat_get_backend_subxact().Nathan Bossart
Unlike the other pg_stat_get_backend* functions, pg_stat_get_backend_subxact() looks up the backend entry by using its integer argument as a 1-based index in an internal array. The other functions look for the entry with the matching session backend ID. These numbers often match, but that isn't reliably true. This commit resolves this discrepancy by introducing pgstat_get_local_beentry_by_backend_id() and using it in pg_stat_get_backend_subxact(). We cannot use pgstat_get_beentry_by_backend_id() because it returns a PgBackendStatus, which lacks the locally computed additions available in LocalPgBackendStatus that are required by pg_stat_get_backend_subxact(). Author: Ian Barwick Reviewed-by: Sami Imseih, Michael Paquier, Robert Haas Discussion: https://postgr.es/m/CAB8KJ%3Dj-ACb3H4L9a_b3ZG3iCYDW5aEu3WsPAzkm2S7JzS1Few%40mail.gmail.com Backpatch-through: 16
2023-08-30Rename some support functions for pgstat* views.Nathan Bossart
Presently, pgstat_fetch_stat_beentry() accepts a session's backend ID as its argument, and pgstat_fetch_stat_local_beentry() accepts a 1-based index in an internal array as its argument. The former is typically used wherever a user must provide a backend ID, and the latter is usually used internally when looping over all entries in the array. This difference was first introduced by d7e39d72ca. Before that commit, both functions accepted a 1-based index to the internal array. This commit renames these two functions to make it clear whether they use the backend ID or the 1-based index to look up the entry. This is preparatory work for a follow-up change that will introduce a function for looking up a LocalPgBackendStatus using a backend ID. Reviewed-by: Ian Barwick, Sami Imseih, Michael Paquier, Robert Haas Discussion: https://postgr.es/m/CAB8KJ%3Dj-ACb3H4L9a_b3ZG3iCYDW5aEu3WsPAzkm2S7JzS1Few%40mail.gmail.com Backpatch-through: 16
2023-08-30Avoid possible overflow with ltsGetFreeBlock() in logtape.cMichael Paquier
nFreeBlocks, defined as a long, stores the number of free blocks in a logical tape. ltsGetFreeBlock() has been using an int to store the value of nFreeBlocks, which could lead to overflows on platforms where long and int are not the same size (in short everything except Windows where long is 4 bytes). The problematic intermediate variable is switched to be a long instead of an int. Issue introduced by c02fdc9223015, so backpatch down to 13. Author: Ranier vilela Reviewed-by: Peter Geoghegan, David Rowley Discussion: https://postgr.es/m/CAEudQApLDWCBR_xmwNjGBrDo+f+S4E87x3s7-+hoaKqYdtC4JQ@mail.gmail.com Backpatch-through: 13
2023-08-29Rename logical_replication_mode to debug_logical_replication_streamingPeter Eisentraut
The logical_replication_mode GUC is intended for testing and debugging purposes, but its current name may be misleading and encourage users to make unnecessary changes. To avoid confusion, renaming the GUC to a less misleading name debug_logical_replication_streaming that casual users are less likely to mistakenly assume needs to be modified in a regular logical replication setup. Author: Hou Zhijie <houzj.fnst@cn.fujitsu.com> Reviewed-by: Peter Smith <smithpb2250@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/d672d774-c44b-6fec-f993-793e744f169a%40eisentraut.org
2023-08-29Reword user-facing message for "power of two"Daniel Gustafsson
While there are numerous instances of using "power of 2" in the code, translated user-facing messages use "power of two". Fix two instances which used "power of 2" instead. This is a backpatch of 95fff2abee in master. Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/20230829.175615.682972421946735863.horikyota.ntt@gmail.com Backpatch-through: v16
2023-08-29Initialize ListenSocket array earlier.Heikki Linnakangas
After commit b0bea38705, syslogger prints 63 warnings about failing to close a listen socket at postmaster startup. That's because the syslogger process forks before the ListenSockets array is initialized, so ClosePostmasterPorts() calls "close(0)" 64 times. The first call succeeds, because fd 0 is stdin. This has been like this since commit 9a86f03b4e in version 13, which moved the SysLogger_Start() call to before initializing ListenSockets. We just didn't notice until commit b0bea38705 added the LOG message. Reported by Michael Paquier and Jeff Janes. Author: Michael Paquier Discussion: https://www.postgresql.org/message-id/ZOvvuQe0rdj2slA9%40paquier.xyz Discussion: https://www.postgresql.org/message-id/ZO0fgDwVw2SUJiZx@paquier.xyz#482670177eb4eaf4c9f03c1eed963e5f Backpatch-through: 13
2023-08-28Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: a58360c10bd5a0f0153fcf75d0b5291ac1acecb0
2023-08-24pg_upgrade: Bump MESSAGE_WIDTH.Nathan Bossart
Commit 7b378237aa added a status message to pg_upgrade that is 60 characters wide. Since the MESSAGE_WIDTH macro is currently set to 60, there is no space between this new status message and the "ok" or "failed" indicator appended when the step completes. To fix this problem, this commit increases the value of MESSAGE_WIDTH to 62. Suggested-by: Bharath Rupireddy Reviewed-by: Peter Eisentraut Discussion: https://postgr.es/m/CALj2ACVVvk1cYLtWVxHv%3DZ1Ubq%3DUES9fhKbUU4c9k4W%2BfEDnbw%40mail.gmail.com Backpatch-through: 16
2023-08-24Avoid unnecessary plancache revalidation of utility statements.Tom Lane
Revalidation of a plancache entry (after a cache invalidation event) requires acquiring a snapshot. Normally that is harmless, but not if the cached statement is one that needs to run without acquiring a snapshot. We were already aware of that for TransactionStmts, but for some reason hadn't extrapolated to the other statements that PlannedStmtRequiresSnapshot() knows mustn't set a snapshot. This can lead to unexpected failures of commands such as SET TRANSACTION ISOLATION LEVEL. We can fix it in the same way, by excluding those command types from revalidation. However, we can do even better than that: there is no need to revalidate for any statement type for which parse analysis, rewrite, and plan steps do nothing interesting, which is nearly all utility commands. To mechanize this, invent a parser function stmt_requires_parse_analysis() that tells whether parse analysis does anything beyond wrapping a CMD_UTILITY Query around the raw parse tree. If that's what it does, then rewrite and plan will just skip the Query, so that it is not possible for the same raw parse tree to produce a different plan tree after cache invalidation. stmt_requires_parse_analysis() is basically equivalent to the existing function analyze_requires_snapshot(), except that for obscure reasons that function omits ReturnStmt and CallStmt. It is unclear whether those were oversights or intentional. I have not been able to demonstrate a bug from not acquiring a snapshot while analyzing these commands, but at best it seems mighty fragile. It seems safer to acquire a snapshot for parse analysis of these commands too, which allows making stmt_requires_parse_analysis and analyze_requires_snapshot equivalent. In passing this fixes a second bug, which is that ResetPlanCache would exclude ReturnStmts and CallStmts from revalidation. That's surely *not* safe, since they contain parsable expressions. Per bug #18059 from Pavel Kulakov. Back-patch to all supported branches. Discussion: https://postgr.es/m/18059-79c692f036b25346@postgresql.org
2023-08-24Fix lack of message pluralizationPeter Eisentraut
2023-08-24Update DECLARE_INDEX documentationPeter Eisentraut
Update source code comment changes belonging to the changes in 6a6389a08b. Discussion: https://www.postgresql.org/message-id/flat/75ae5875-3abc-dafc-8aec-73247ed41cde@eisentraut.org
2023-08-24Rename hook functions for debug_io_direct to match variable name.Peter Eisentraut
Commit 319bae9a renamed the GUC. Rename the check and assign functions to match, and alphabetize. Back-patch to 16. Author: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/2769341e-fa28-c2ee-3e4b-53fdcaaf2271%40eisentraut.org
2023-08-24Fix the error message when failing to restore the snapshot.Amit Kapila
The SnapBuildRestoreContents() used a const value in the error message to indicate the size in bytes it was expecting to read from the serialized snapshot file. Fix it by reporting the size that was actually passed. Author: Hou Zhijie Reviewed-by: Amit Kapila Backpatch-through: 16 Discussion: http://postgr.es/m/OS0PR01MB5716D408364F7DF32221C08D941FA@OS0PR01MB5716.jpnprd01.prod.outlook.com