summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2021-07-10Fix numeric_mul() overflow due to too many digits after decimal point.Dean Rasheed
This fixes an overflow error when using the numeric * operator if the result has more than 16383 digits after the decimal point by rounding the result. Overflow errors should only occur if the result has too many digits *before* the decimal point. Discussion: https://postgr.es/m/CAEZATCUmeFWCrq2dNzZpRj5+6LfN85jYiDoqm+ucSXhb9U2TbA@mail.gmail.com
2021-07-09libpq: Fix sending queries in pipeline aborted stateAlvaro Herrera
When sending queries in pipeline mode, we were careless about leaving the connection in the right state so that PQgetResult would behave correctly; trying to read further results after sending a query after having read a result with an error would sometimes hang. Fix by ensuring internal libpq state is changed properly. All the state changes were being done by the callers of pqAppendCmdQueueEntry(); it would have become too repetitious to have this logic in each of them, so instead put it all in that function and relieve callers of the responsibility. Add a test to verify this case. Without the code fix, this new test hangs sometimes. Also, document that PQisBusy() would return false when no queries are pending result. This is not intuitively obvious, and NULL would be obtained by calling PQgetResult() at that point, which is confusing. Wording by Boris Kolpackov. In passing, fix bogus use of "false" to mean "0", per Ranier Vilela. Backpatch to 14. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reported-by: Boris Kolpackov <boris@codesynthesis.com> Discussion: https://postgr.es/m/boris.20210624103805@codesynthesis.com
2021-07-09Avoid creating a RESULT RTE that's marked LATERAL.Tom Lane
Commit 7266d0997 added code to pull up simple constant function results, converting the RTE_FUNCTION RTE to a dummy RTE_RESULT RTE since it no longer need be scanned. But I forgot to clear the LATERAL flag if the RTE has it set. If the function reduced to a constant, it surely contains no lateral references so this simplification is logically OK. It's needed because various other places will Assert that RESULT RTEs aren't LATERAL. Per bug #17097 from Yaoguang Chen. Back-patch to v13 where the faulty code came in. Discussion: https://postgr.es/m/17097-3372ef9f798fc94f@postgresql.org
2021-07-09Update configure's probe for libldap to work with OpenLDAP 2.5.Tom Lane
The separate libldap_r is gone and libldap itself is now always thread-safe. Unfortunately there seems no easy way to tell by inspection whether libldap is thread-safe, so we have to take it on faith that libldap is thread-safe if there's no libldap_r. That should be okay, as it appears that libldap_r was a standard part of the installation going back at least 20 years. Report and patch by Adrian Ho. Back-patch to all supported branches, since people might try to build any of them with a newer OpenLDAP. Discussion: https://postgr.es/m/17083-a19190d9591946a7@postgresql.org
2021-07-09Reject cases where a query in WITH rewrites to just NOTIFY.Tom Lane
Since the executor can't cope with a utility statement appearing as a node of a plan tree, we can't support cases where a rewrite rule inserts a NOTIFY into an INSERT/UPDATE/DELETE command appearing in a WITH clause of a larger query. (One can imagine ways around that, but it'd be a new feature not a bug fix, and so far there's been no demand for it.) RewriteQuery checked for this, but it missed the case where the DML command rewrites to *only* a NOTIFY. That'd lead to crashes later on in planning. Add the missed check, and improve the level of testing of this area. Per bug #17094 from Yaoguang Chen. It's been busted since WITH was introduced, so back-patch to all supported branches. Discussion: https://postgr.es/m/17094-bf15dff55eaf2e28@postgresql.org
2021-07-09Remove more obsolete comments about semaphores.Thomas Munro
Commit 6753333f stopped using semaphores as the sleep/wake mechanism for heavyweight locks, but some obsolete references to that scheme remained in comments. As with similar commit 25b93a29, back-patch all the way. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Discussion: https://postgr.es/m/CA%2BhUKGLafjB1uzXcy%3D%3D2L3cy7rjHkqOVn7qRYGBjk%3D%3DtMJE7Yg%40mail.gmail.com
2021-07-09Fix incorrect return value in pg_size_pretty(bigint)David Rowley
Due to how pg_size_pretty(bigint) was implemented, it's possible that when given a negative number of bytes that the returning value would not match the equivalent positive return value when given the equivalent positive number of bytes. This was due to two separate issues. 1. The function used bit shifting to convert the number of bytes into larger units. The rounding performed by bit shifting is not the same as dividing. For example -3 >> 1 = -2, but -3 / 2 = -1. These two operations are only equivalent with positive numbers. 2. The half_rounded() macro rounded towards positive infinity. This meant that negative numbers rounded towards zero and positive numbers rounded away from zero. Here we fix #1 by dividing the values instead of bit shifting. We fix #2 by adjusting the half_rounded macro always to round away from zero. Additionally, adjust the pg_size_pretty(numeric) function to be more explicit that it's using division rather than bit shifting. A casual observer might have believed bit shifting was used due to a static function being named numeric_shift_right. However, that function was calculating the divisor from the number of bits and performed division. Here we make that more clear. This change is just cosmetic and does not affect the return value of the numeric version of the function. Here we also add a set of regression tests both versions of pg_size_pretty() which test the values directly before and after the function switches to the next unit. This bug was introduced in 8a1fab36a. Prior to that negative values were always displayed in bytes. Author: Dean Rasheed, David Rowley Discussion: https://postgr.es/m/CAEZATCXnNW4HsmZnxhfezR5FuiGgp+mkY4AzcL5eRGO4fuadWg@mail.gmail.com Backpatch-through: 9.6, where the bug was introduced.
2021-07-05Reduce overhead of cache-clobber testing in LookupOpclassInfo().Tom Lane
Commit 03ffc4d6d added logic to bypass all caching behavior in LookupOpclassInfo when CLOBBER_CACHE_ALWAYS is enabled. It doesn't look like I stopped to think much about what that would cost, but recent investigation shows that the cost is enormous: it roughly doubles the time needed for cache-clobber test runs. There does seem to be value in this behavior when trying to test the opclass-cache loading logic itself, but for other purposes the cost is excessive. Hence, let's back off to doing this only when debug_invalidate_system_caches_always is at least 3; or in older branches, when CLOBBER_CACHE_RECURSIVELY is defined. While here, clean up some other minor issues in LookupOpclassInfo. Re-order the code so we aren't left with broken cache entries (leading to later core dumps) in the unlikely case that we suffer OOM while trying to allocate space for a new entry. (That seems to be my oversight in 03ffc4d6d.) Also, in >= v13, stop allocating one array entry too many. That's evidently left over from sloppy reversion in 851b14b0c. Back-patch to all supported branches, mainly to reduce the runtime of cache-clobbering buildfarm animals. Discussion: https://postgr.es/m/1370856.1625428625@sss.pgh.pa.us
2021-07-05Rethink blocking annotations in detach-partition-concurrently-[34].Tom Lane
In 741d7f104, I tried to make the reports from canceled steps come out after the pg_cancel_backend() steps, since that was the most common ordering before. However, that doesn't ensure that a canceled step doesn't report even later, as shown in a recent failure on buildfarm member idiacanthus. Rather than complicating things even more with additional annotations, let's just force the cancel's effect to be reported first. It's not *that* unnatural-looking. Back-patch to v14 where these test cases appeared. Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2021-07-02%2001%3A40%3A04
2021-07-02Don't try to print data type names in slot_store_error_callback().Tom Lane
The existing code tried to do syscache lookups in an already-failed transaction, which is problematic to say the least. After some consideration of alternatives, the best fix seems to be to just drop type names from the error message altogether. The table and column names seem like sufficient localization. If the user is unsure what types are involved, she can check the local and remote table definitions. Having done that, we can also discard the LogicalRepTypMap hash table, which had no other use. Arguably, LOGICAL_REP_MSG_TYPE replication messages are now obsolete as well; but we should probably keep them in case some other use emerges. (The complexity of removing something from the replication protocol would likely outweigh any savings anyhow.) Masahiko Sawada and Bharath Rupireddy, per complaint from Andres Freund. Back-patch to v10 where this code originated. Discussion: https://postgr.es/m/20210106020229.ne5xnuu6wlondjpe@alap3.anarazel.de
2021-07-01Add --clobber-cache option to initdb, for CCA testing.Tom Lane
Commit 4656e3d66 replaced the "#define CLOBBER_CACHE_ALWAYS" testing mechanism with a GUC, which has been a great help for doing cache-clobber testing in more efficient ways; but there is a gap in the implementation. The only way to do cache-clobber testing during an initdb run is to use the old method with #define, because one can't set the GUC from outside. Improve this by adding a switch to initdb for the purpose. (Perhaps someday we should let initdb pass through arbitrary "-c NAME=VALUE" switches. Quoting difficulties dissuaded me from attempting that right now, though.) Back-patch to v14 where 4656e3d66 came in. Discussion: https://postgr.es/m/1582507.1624227029@sss.pgh.pa.us
2021-07-01Don't reset relhasindex for partitioned tables on ANALYZEAlvaro Herrera
Commit 0e69f705cc1a introduced code to analyze partitioned table; however, that code fails to preserve pg_class.relhasindex correctly. Fix by observing whether any indexes exist rather than accidentally falling through to assuming none do. Backpatch to 14. Author: Alexander Pyhalov <a.pyhalov@postgrespro.ru> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Zhihong Yu <zyu@yugabyte.com> Discussion: https://postgr.es/m/CALNJ-vS1R3Qoe5t4tbzxrkpBtzRbPq1dDcW4RmA_a+oqweF30w@mail.gmail.com
2021-07-01Fix prove_installcheck to use correct paths when used with PGXSAndrew Dunstan
The prove_installcheck recipe in src/Makefile.global.in was emitting bogus paths for a couple of elements when used with PGXS. Here we create a separate recipe for the PGXS case that does it correctly. We also take the opportunity to make the make the file more readable by breaking up the prove_installcheck and prove_check recipes across several lines, and to remove the setting for REGRESS_SHLIB to src/test/recovery/Makefile, which is the only set of tests that actually need it. Backpatch to all live branches Discussion: https://postgr.es/m/f2401388-936b-f4ef-a07c-a0bcc49b3300@dunslane.net
2021-06-30Replace magic constants used in pg_stat_get_replication_slot().Amit Kapila
A few variables have been using 10 as a magic constant while PG_STAT_GET_REPLICATION_SLOT_COLS can be used instead. Author: Masahiko Sawada Reviewed-By: Amit Kapila Backpatch-through: 14, where it was introduced Discussion: https://postgr.es/m/CAD21AoBvqODDfmD17DkEuPCvV2KbruukXQ2Vwrv5Xi-TsAsTJA@mail.gmail.com
2021-06-30Allow streaming the changes after speculative aborts.Amit Kapila
Until now, we didn't allow to stream the changes in logical replication till we receive speculative confirm or the next DML change record after speculative inserts. The reason was that we never use to process speculative aborts but after commit 4daa140a2f it is possible to process them so we can allow streaming once we receive speculative abort after speculative insertion. We decided to backpatch to 14 where the feature for streaming in progress transactions have been introduced as this is a minor change and makes that functionality better. Author: Amit Kapila Reviewed-By: Dilip Kumar Backpatch-through: 14 Discussion: https://postgr.es/m/CAA4eK1KdqmTCtrBR6oFfGELrLLbDLDedL6zACcsUOQuTJBj1vw@mail.gmail.com
2021-06-30Fix incorrect PITR message for transaction ROLLBACK PREPAREDMichael Paquier
Reaching PITR on such a transaction would cause the generation of a LOG message mentioning a transaction committed, not aborted. Oversight in 4f1b890. Author: Simon Riggs Discussion: https://postgr.es/m/CANbhV-GJ6KijeCgdOrxqMCQ+C8QiK657EMhCy4csjrPcEUFv_Q@mail.gmail.com Backpatch-through: 9.6
2021-06-29Fixes for multirange selectivity estimationAlexander Korotkov
* Fix enumeration of the multirange operators in calc_multirangesel() and calc_multirangesel() switches. * Add more regression tests for matching to empty ranges/multiranges. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/c5269c65-f967-77c5-ff7c-15e621c47f6a%40gmail.com Author: Alexander Korotkov Backpatch-through: 14, where multiranges were introduced
2021-06-29Fix libpq state machine in pipeline modeAlvaro Herrera
The original coding required that PQpipelineSync had been called before the first call to PQgetResult, and failure to do that would result in an unexpected NULL result being returned. Fix by setting the right state when a query is sent, rather than leaving it unchanged and having PQpipelineSync apply the necessary state change. A new test case to verify the behavior is added, which relies on the new PQsendFlushRequest() function added by commit a7192326c74d. Backpatch to 14, where pipeline mode was added. Reported-by: Boris Kolpackov <boris@codesynthesis.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/boris.20210616110321@codesynthesis.com
2021-06-29Add PQsendFlushRequest to libpqAlvaro Herrera
This new libpq function allows the application to send an 'H' message, which instructs the server to flush its outgoing buffer. This hasn't been needed so far because the Sync message already requests a buffer; and I failed to realize that this was needed in pipeline mode because PQpipelineSync also causes the buffer to be flushed. However, sometimes it is useful to request a flush without establishing a synchronization point. Backpatch to 14, where pipeline mode was introduced in libpq. Reported-by: Boris Kolpackov <boris@codesynthesis.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/202106252350.t76x73nt643j@alvherre.pgsql
2021-06-29Fix bogus logic for reporting which hash partition conflicts.Tom Lane
Commit efbfb6424 added logic for reporting exactly which existing partition conflicts when complaining that a new hash partition's modulus isn't compatible with the existing ones. However, it misunderstood the partitioning data structure, and would select the wrong partition in some cases, or crash outright due to fetching a bogus table OID in other cases. Per bug #17076 from Alexander Lakhin. Fix by Amit Langote; some further work on the code comments by me. Discussion: https://postgr.es/m/17076-89a16ae835d329b9@postgresql.org
2021-06-28Don't use abort(3) in libpq's fe-print.c.Tom Lane
Causing a core dump on out-of-memory seems pretty unfriendly, and surely is far outside the expected behavior of a general-purpose library. Just print an error message (as we did already) and return. These functions unfortunately don't have an error return convention, but code using them is probably just looking for a quick-n-dirty print method and wouldn't bother to check anyway. Although these functions are semi-deprecated, it still seems appropriate to back-patch this. In passing, also back-patch b90e6cef1, just to reduce cosmetic differences between the branches. Discussion: https://postgr.es/m/3122443.1624735363@sss.pgh.pa.us
2021-06-28Don't depend on -fwrapv semantics in pgbench's random() function.Tom Lane
Instead use the common/int.h functions to check for integer overflow in a more C-standard-compliant fashion. This is motivated by recent failures on buildfarm member moonjelly, where it appears that development-tip gcc is optimizing without regard to the -fwrapv switch. Presumably that's a gcc bug that will be fixed soon, but we might as well install cleaner coding here rather than wait. (This does not address the question of whether we'll ever be able to get rid of using -fwrapv. Testing shows that this spot is the only place where doing so creates visible regression test failures, but unfortunately that proves very little.) Back-patch to v12. The common/int.h functions exist in v11, but that branch doesn't use them in any client-side code. I judge that this case isn't interesting enough in the real world to take even a small risk of issues from being the first such use. Tom Lane and Fabien Coelho Discussion: https://postgr.es/m/73927.1624815543@sss.pgh.pa.us
2021-06-28Pre branch pgindent / pgperltidy runAndrew Dunstan
Along the way make a slight adjustment to src/include/utils/queryjumble.h to avoid an unused typedef.
2021-06-28Message style improvementsPeter Eisentraut
2021-06-28Improve RelationGetIdentityKeyBitmap().Amit Kapila
We were using RelationGetIndexList() to update the relation's replica identity index but instead, we can directly use RelationGetReplicaIndex() which uses the same functionality. This is a minor code readability improvement. Author: Japin Li Reviewed-By: Takamichi Osumi, Amit Kapila Discussion: https://postgr.es/m/4C99A862-69C8-431F-960A-81B1151F1B89@enterprisedb.com
2021-06-28Fix race condition in TransactionGroupUpdateXidStatus().Amit Kapila
When we cannot immediately acquire XactSLRULock in exclusive mode at commit time, we add ourselves to a list of processes that need their XIDs status update. We do this if the clog page where we need to update the current transaction status is the same as the group leader's clog page, otherwise, we allow the caller to clear it by itself. Now, when we can't add ourselves to any group, we were not clearing the current proc if it has already become a member of some group which was leading to an assertion failure when the same proc was assigned to another backend after the current backend exits. Reported-by: Alexander Lakhin Bug: 17072 Author: Amit Kapila Tested-By: Alexander Lakhin Backpatch-through: 11, where it was introduced Discussion: https://postgr.es/m/17072-2f8764857ef2c92a@postgresql.org
2021-06-28Change recovery_init_sync_method to PGC_SIGHUP.Thomas Munro
The setting has no effect except during startup. It's still nice to be able to change it dynamically, which is expected to be pretty useful to an admin following crash recovery when restarting the cluster is not so appealing. Per discussions following commits 2941138e6 and 61752afb2. Author: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/20210529192321.GM2082%40telsasoft.com
2021-06-28Fix variable initialization with ALTER SUBSCRIPTION DROP PUBLICATIONMichael Paquier
copy_data is not a supported option with this sub-command of ALTER SUBSCRIPTION, which would not make a variable related to it initialized after parsing the option set in DefElems. A refresh could then refer to it. Author: Ranier Vilela Reviewed-by: Peter Smith Discussion: https://postgr.es/m/CAEudQAp5P8nr=ze2Gv=BMj=DJFZnrvendZCZcC-fos3QiDe2sg@mail.gmail.com
2021-06-28Add test for CREATE INDEX CONCURRENTLY with not-so-immutable predicateMichael Paquier
83158f7 has improved index_set_state_flags() so as it is possible to use transactional updates when updating pg_index state flags, but there was not really a test case which stressed directly the possibility it fixed. This commit adds such a test, using a predicate that looks valid in appearance but calls a stable function. Author: Andrey Lepikhov Discussion: https://postgr.es/m/9b905019-5297-7372-0ad2-e1a4bb66a719@postgrespro.ru Backpatch-through: 9.6
2021-06-27Remove memory leaks in isolationtester.Tom Lane
specscanner.l leaked a kilobyte of memory per token of the spec file. Apparently somebody thought that the introductory code block would be executed once; but it's once per yylex() call. A couple of functions in isolationtester.c leaked small amounts of memory due to not bothering to free one-time allocations. Might as well improve these so that valgrind gives this program a clean bill of health. Also get rid of an ugly static variable. Coverity complained about one of the one-time leaks, which led me to try valgrind'ing isolationtester, which led to discovery of the larger leak.
2021-06-27Error message refactoringPeter Eisentraut
Take some untranslatable things out of the message and replace by format placeholders, to reduce translatable strings and reduce translation mistakes.
2021-06-26Remove undesirable libpq dependency on stringinfo.c.Tom Lane
Commit c0cb87fbb unwisely introduced a dependency on the StringInfo machinery in fe-connect.c. We must not use that in libpq, because it will do a summary exit(1) if it hits OOM, and that is not appropriate behavior for a general-purpose library. The goal of allowing arbitrary line lengths in service files doesn't seem like it's worth a lot of effort, so revert back to the previous method of using a stack-allocated buffer and failing on buffer overflow. This isn't an exact revert though. I kept that patch's refactoring to have a single exit path, as that seems cleaner than having each error path know what to do to clean up. Also, I made the fixed-size buffer 1024 bytes not 256, just to push off the need for an expandable buffer some more. There is more to do here; in particular the lack of any mechanical check for this type of mistake now seems pretty hazardous. But this fix gets us back to the level of robustness we had in v13, anyway. Discussion: https://postgr.es/m/daeb22ec6ca8ef61e94d766a9b35fb03cabed38e.camel@vmware.com
2021-06-26Remove non-existing variable reference in MSVC's Solution.pmMichael Paquier
The version string is grabbed from PACKAGE_VERSION in pg_config.h in the MSVC build since 8f4fb4c6, but an error message referenced a variable that existed before that. This had no consequences except if one messes up enough with the version number of the build. Author: Anton Voloshin Discussion: https://postgr.es/m/af79ee1b-9962-b299-98e1-f90a289e19e6@postgrespro.ru Backpatch-through: 13
2021-06-26Remove some useless logs from the TAP tests of pgbenchMichael Paquier
002_pgbench_no_server was printing some array pointers instead of the actual contents of those arrays for the expected outputs of stdout and stderr for a tested command. This does not add any new information that can help with debugging as the test names allow to track failure locations, if any. This commit simply removes those logs as the rest of the printed information is redundant with command_checks_all(). Per discussion with Andrew Dunstan and Álvaro Herrera. Discussion: https://postgr.es/m/YNXNFaG7IgkzZanD@paquier.xyz Backpatch-through: 11
2021-06-25Remove unnecessary failure cases in RemoveRoleFromObjectPolicy().Tom Lane
It's not really necessary for this function to open or lock the relation associated with the pg_policy entry it's modifying. The error checks it's making on the rel are if anything counterproductive (e.g., if we don't want to allow installation of policies on system catalogs, here is not the place to prevent that). In particular, it seems just wrong to insist on an ownership check. That has the net effect of forcing people to use superuser for DROP OWNED BY, which surely is not an effect we want. Also there is no point in rebuilding the dependencies of the policy expressions, which aren't being changed. Lastly, locking the table also seems counterproductive; it's not helping to prevent race conditions, since we failed to re-read the pg_policy row after acquiring the lock. That means that concurrent DDL would likely result in "tuple concurrently updated/deleted" errors; which is the same behavior this code will produce, with less overhead. Per discussion of bug #17062. Back-patch to all supported versions, as the failure cases this eliminates seem just as undesirable in 9.6 as in HEAD. Discussion: https://postgr.es/m/1573181.1624220108@sss.pgh.pa.us
2021-06-25Cleanup some code related to pgbench log checks in TAP testsMichael Paquier
This fixes a couple of problems within the so-said code of this commit subject: - Replace the use of open() with slurp_file(), fixing an issue reported by buildfarm member fairywren whose perl installation keep around CRLF characters, causing the matching patterns for the logs to fail. - Remove the eval block, which is not really necessary. This set of issues has come into light after fixing a different issue with c13585fe, and this is wrong since this code has been introduced. Reported-by: Andrew Dunstan, and buildfarm member fairywren Author: Michael Paquier Reviewed-by: Andrew Dunstan Discussion: https://postgr.es/m/0f49303e-7784-b3ee-200b-cbf67be2eb9e@dunslane.net Backpatch-through: 11
2021-06-25Put option listing back into alphabetical orderPeter Eisentraut
2021-06-25Fixes in ALTER SUBSCRIPTION DROP PUBLICATION codePeter Eisentraut
ALTER SUBSCRIPTION DROP PUBLICATION does not actually support copy_data option, so remove it from tab completion. Also, reword the error message that is thrown when all the publications from a subscription are specified to be dropped. Also, made few doc and cosmetic adjustments. Author: Vignesh C <vignesh21@gmail.com> Reviewed-by: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com> Reviewed-by: Japin Li <japinli@hotmail.com> Discussion: https://www.postgresql.org/message-id/flat/CALDaNm21RwsDzs4xj14ApteAF7auyyomHNnp+NEL-sH8m-jMvQ@mail.gmail.com
2021-06-25Remove redundant variable pageSize in gistinitpagePeter Eisentraut
In gistinitpage, pageSize variable looks redundant, instead just pass BLCKSZ. This will be consistent with its peers BloomInitPage, brin_page_init and SpGistInitPage. Author: Bharath Rupireddy <bharath.rupireddy@enterprisedb.com> Discussion: https://www.postgresql.org/message-id/flat/CALj2ACWj=V1k5591eeZK2sOg2FYuBUp6azFO8tMkBtGfXf8PMQ@mail.gmail.com
2021-06-25Add more debugging information with log checks in TAP tests of pgbenchMichael Paquier
fairywren is not happy with the pattern checks introduced by c13585f. I am not sure if this outlines a bug in pgbench or if the regex patterns used in the tests are too restrictive for this buildfarm member's environment. This adds more debugging information to show the log entries that do not match with the expected pattern, to help in finding out what's happening. That seems like a good addition in the long-term anyway as that may not be the only issue in this area. Discussion: https://postgr.es/m/YNUad2HvgW+6eXyo@paquier.xyz
2021-06-25doc: Move remove_temp_files_after_crash to section for developer optionsMichael Paquier
The main goal of this option is to allow inspecting temporary files for debugging purposes, so moving the parameter there is natural. Oversight in cd91de0. Reported-by: Justin Pryzby Author: Euler Taveira Discussion: https://postgr.es/m/20210612004347.GP16435@telsasoft.com
2021-06-25Prepare for forthcoming LLVM 13 API change.Thomas Munro
LLVM 13 (due out in September) has changed the semantics of LLVMOrcAbsoluteSymbols(), so we need to bump some reference counts to avoid a double-free that causes crashes and bad query results. A proactive change seems necessary to avoid having a window of time where our respective latest releases would interact badly. It's possible that the situation could change before then, though. Thanks to Fabien Coelho for monitoring bleeding edge LLVM and Andres Freund for tracking down the change. Back-patch to 11, where the JIT code arrived. Discussion: https://postgr.es/m/CA%2BhUKGLEy8mgtN7BNp0ooFAjUedDTJj5dME7NxLU-m91b85siA%40mail.gmail.com
2021-06-25Fix pattern matching logic for logs in TAP tests of pgbenchMichael Paquier
The logic checking for the format of per-thread logs used grep() with directly "$re", which would cause the test to consider all the logs as a match without caring about their format at all. Using "/$re/" makes grep() perform a regex test, which is what we want here. While on it, improve some of the tests to be more picky with the patterns expected and add more comments to describe the tests. Issue discovered while digging into a separate patch. Author: Fabien Coelho, Michael Paquier Discussion: https://postgr.es/m/YNPsPAUoVDCpPOGk@paquier.xyz Backpatch-through: 11
2021-06-24Another fix to relmapper race condition.Heikki Linnakangas
In previous commit, I missed that relmap_redo() was also not acquiring the RelationMappingLock. Thanks to Thomas Munro for pointing that out. Backpatch-through: 9.6, like previous commit. Discussion: https://www.postgresql.org/message-id/CA%2BhUKGLev%3DPpOSaL3WRZgOvgk217et%2BbxeJcRr4eR-NttP1F6Q%40mail.gmail.com
2021-06-24Prevent race condition while reading relmapper file.Heikki Linnakangas
Contrary to the comment here, POSIX does not guarantee atomicity of a read(), if another process calls write() concurrently. Or at least Linux does not. Add locking to load_relmap_file() to avoid the race condition. Fixes bug #17064. Thanks to Alexander Lakhin for the report and test case. Backpatch-through: 9.6, all supported versions. Discussion: https://www.postgresql.org/message-id/17064-bb0d7904ef72add3@postgresql.org
2021-06-23Allow non-quoted identifiers as isolation test session/step names.Tom Lane
For no obvious reason, isolationtester has always insisted that session and step names be written with double quotes. This is fairly tedious and does little for test readability, especially since the names that people actually choose almost always look like normal identifiers. Hence, let's tweak the lexer to allow SQL-like identifiers not only double-quoted strings. (They're SQL-like, not exactly SQL, because I didn't add any case-folding logic. Also there's no provision for U&"..." names, not that anyone's likely to care.) There is one incompatibility introduced by this change: if you write "foo""bar" with no space, that used to be taken as two identifiers, but now it's just one identifier with an embedded quote mark. I converted all the src/test/isolation/ specfiles to remove unnecessary double quotes, but stopped there because my eyes were glazing over already. Like 741d7f104, back-patch to all supported branches, so that this isn't a stumbling block for back-patching isolation test changes. Discussion: https://postgr.es/m/759113.1623861959@sss.pgh.pa.us
2021-06-23Don't assume GSSAPI result strings are null-terminated.Tom Lane
Our uses of gss_display_status() and gss_display_name() assumed that the gss_buffer_desc strings returned by those functions are null-terminated. It appears that they generally are, given the lack of field complaints up to now. However, the available documentation does not promise this, and some man pages for gss_display_status() show examples that rely on the gss_buffer_desc.length field instead of expecting null termination. Also, we now have a report that on some implementations, clang's address sanitizer is of the opinion that the byte after the specified length is undefined. Hence, change the code to rely on the length field instead. This might well be cosmetic rather than fixing any real bug, but it's hard to be sure, so back-patch to all supported branches. While here, also back-patch the v12 changes that made pg_GSS_error deal honestly with multiple messages available from gss_display_status. Per report from Sudheer H R. Discussion: https://postgr.es/m/5372B6D4-8276-42C0-B8FB-BD0918826FC3@tekenlight.com
2021-06-23Improve display of query results in isolation tests.Tom Lane
Previously, isolationtester displayed SQL query results using some ad-hoc code that clearly hadn't had much effort expended on it. Field values longer than 14 characters weren't separated from the next field, and usually caused misalignment of the columns too. Also there was no visual separation of a query's result from subsequent isolationtester output. This made test result files confusing and hard to read. To improve matters, let's use libpq's PQprint() function. Although that's long since unused by psql, it's still plenty good enough for the purpose here. Like 741d7f104, back-patch to all supported branches, so that this isn't a stumbling block for back-patching isolation test changes. Discussion: https://postgr.es/m/582362.1623798221@sss.pgh.pa.us
2021-06-23Add test case for obsoleting slot with active walsender, take 2Alvaro Herrera
The code to signal a running walsender when its reserved WAL size grows too large is completely uncovered before this commit; this adds coverage for that case. This test involves sending SIGSTOP to walsender and walreceiver, then advancing enough WAL for a checkpoint to trigger, then sending SIGCONT. There's no precedent for STOP signalling in Perl tests, and my reading of relevant manpages says it's likely to fail on Windows. Because of this, this test is always skipped on that platform. This version fixes a couple of rarely hit race conditions in the previous attempt 09126984a263; most notably, both LOG string searches are loops, not just the second one; we acquire the start-of-log position before STOP-signalling; and reference the correct process name in the test description. All per Tom Lane. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/202106102202.mjw4huiix7lo@alvherre.pgsql
2021-06-22Use annotations to reduce instability of isolation-test results.Tom Lane
We've long contended with isolation test results that aren't entirely stable. Some test scripts insert long delays to try to force stable results, which is not terribly desirable; but other erratic failure modes remain, causing unrepeatable buildfarm failures. I've spent a fair amount of time trying to solve this by improving the server-side support code, without much success: that way is fundamentally unable to cope with diffs that stem from chance ordering of arrival of messages from different server processes. We can improve matters on the client side, however, by annotating the test scripts themselves to show the desired reporting order of events that might occur in different orders. This patch adds three types of annotations to deal with (a) test steps that might or might not complete their waits before the isolationtester can see them waiting; (b) test steps in different sessions that can legitimately complete in either order; and (c) NOTIFY messages that might arrive before or after the completion of a step in another session. We might need more annotation types later, but this seems to be enough to deal with the instabilities we've seen in the buildfarm. It also lets us get rid of all the long delays that were previously used, cutting more than a minute off the runtime of the isolation tests. Back-patch to all supported branches, because the buildfarm instabilities affect all the branches, and because it seems desirable to keep isolationtester's capabilities the same across all branches to simplify possible future back-patching of tests. Discussion: https://postgr.es/m/327948.1623725828@sss.pgh.pa.us