summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2019-11-09Fix negative bitmapset member not allowed error in logical replicationPeter Eisentraut
This happens when we add a replica identity column on a subscriber that does not yet exist on the publisher, according to the mapping maintained by the subscriber. Code that checks whether the target relation on the subscriber is updatable would check the replica identity attribute bitmap with a column number -1, which would result in an error. To fix, skip such columns in the bitmap lookup and consider the relation not updatable. The result is consistent with the rule that the replica identity columns on the subscriber must be a subset of those on the publisher, since if the column doesn't exist on the publisher, the column set on the subscriber can't be a subset. Reported-by: Tim Clarke <tim.clarke@minerva.info> Analyzed-by: Jehan-Guillaume de Rorthais <jgdr@dalibo.com> Discussion: https://www.postgresql.org/message-id/flat/a9139c29-7ddd-973b-aa7f-71fed9c38d75%40minerva.info
2019-11-08Fix gratuitous error message variationPeter Eisentraut
2019-11-07Move declaration of ecpg_gettext() to a saner place.Tom Lane
Declaring this in the client-visible header ecpglib.h was a pretty poor decision. It's not meant to be application-callable (and if it was, putting it outside the extern "C" { ... } wrapper means that C++ clients would fail to call it). And the declaration would not even compile for a client, anyway, since it would not have the macro pg_attribute_format_arg(). Fortunately, it seems that no clients have tried to include this header with ENABLE_NLS defined, or we'd have gotten complaints about that. But we have no business putting such a restriction on client code. Move the declaration to ecpglib_extern.h, since in fact nothing outside src/interfaces/ecpg/ecpglib/ needs to call it. The practical effect of this is just that clients can now safely #include ecpglib.h while having ENABLE_NLS defined, but that seems like enough of a reason to back-patch it. Discussion: https://postgr.es/m/20590.1573069709@sss.pgh.pa.us
2019-11-07Fix SET CONSTRAINTS .. DEFERRED on partitioned tablesAlvaro Herrera
SET CONSTRAINTS ... DEFERRED failed on partitioned tables, because of a sanity check that ensures that the affected constraints have triggers. On partitioned tables, the triggers are in the leaf partitions, not in the partitioned relations themselves, so the sanity check fails. Removing the sanity check solves the problem, because the code needed to support the case is already there. Backpatch to 11. Note: deferred unique constraints are not affected by this bug, because they do have triggers in the parent partitioned table. I did not add a test for this scenario. Discussion: https://postgr.es/m/20191105212915.GA11324@alvherre.pgsql
2019-11-07Fix integer-overflow edge case detection in interval_mul and pgbench.Tom Lane
This patch adopts the overflow check logic introduced by commit cbdb8b4c0 into two more places. interval_mul() failed to notice if it computed a new microseconds value that was one more than INT64_MAX, and pgbench's double-to-int64 logic had the same sorts of edge-case problems that cbdb8b4c0 fixed in the core code. To make this easier to get right in future, put the guts of the checks into new macros in c.h, and add commentary about how to use the macros correctly. Back-patch to all supported branches, as we did with the previous fix. Yuya Watari Discussion: https://postgr.es/m/CAJ2pMkbkkFw2hb9Qb1Zj8d06EhWAQXFLy73St4qWv6aX=vqnjw@mail.gmail.com
2019-11-07Fix assertion failure when running pgbench -s.Fujii Masao
If there is the WAL page that the continuation WAL record just fits within (i.e., the continuation record ends just at the end of the page) and the LSN in such page is specified with -s option, previously pg_waldump caused an assertion failure. The cause of this assertion failure was that XLogFindNextRecord() that pg_waldump -s calls mistakenly handled such special WAL page. This commit changes XLogFindNextRecord() so that it can handle such WAL page correctly. Back-patch to all supported versions. Author: Andrey Lepikhov Reviewed-by: Fujii Masao, Michael Paquier Discussion: https://postgr.es/m/99303554-5dd5-06e6-f943-b3005ccd6edd@postgrespro.ru
2019-11-06Fix memory allocation mistakePeter Eisentraut
The previous code was allocating more memory than necessary because the formula used the wrong data type. Reported-by: Jehan-Guillaume de Rorthais <jgdr@dalibo.com> Discussion: https://www.postgresql.org/message-id/20191105172918.3e32a446@firost
2019-11-06Fix timestamp of sent message for write context in logical decodingMichael Paquier
When sending data for logical decoding using the streaming replication protocol via a WAL sender, the timestamp of the sent write message is allocated at the beginning of the message when preparing for the write, and actually computed when the write message is ready to be sent. The timestamp was getting computed after sending the message. This impacts anything using logical decoding, causing for example logical replication to report mostly NULL for last_msg_send_time in pg_stat_subscription. This commit makes sure that the timestamp is computed before sending the message. This is wrong since 5a991ef, so backpatch down to 9.4. Author: Jeff Janes Discussion: https://postgr.es/m/CAMkU=1z=WMn8jt7iEdC5sYNaPgAgOASb_OW5JYv-vMdYaJSL-w@mail.gmail.com Backpatch-through: 9.4
2019-11-06Request small targetlist for input to WindowAgg.Andrew Gierth
WindowAgg will potentially store large numbers of input rows into tuplestores to allow access to other rows in the frame. If the input is coming via an explicit Sort node, then unneeded columns will already have been discarded (since Sort requests a small tlist); but there are idioms like COUNT(*) OVER () that result in the input not being sorted at all, and cases where the input is being sorted by some means other than a Sort; if we don't request a small tlist, then WindowAgg's storage requirement is inflated by the unneeded columns. Backpatch back to 9.6, where the current tlist handling was added. (Prior to that, WindowAgg would always use a small tlist.) Discussion: https://postgr.es/m/87a7ator8n.fsf@news-spur.riddles.org.uk
2019-11-05Avoid logging complaints about abandoned connections when using PAM.Tom Lane
For a long time (since commit aed378e8d) we have had a policy to log nothing about a connection if the client disconnects when challenged for a password. This is because libpq-using clients will typically do that, and then come back for a new connection attempt once they've collected a password from their user, so that logging the abandoned connection attempt will just result in log spam. However, this did not work well for PAM authentication: the bottom-level function pam_passwd_conv_proc() was on board with it, but we logged messages at higher levels anyway, for lack of any reporting mechanism. Add a flag and tweak the logic so that the case is silent, as it is for other password-using auth mechanisms. Per complaint from Yoann La Cancellera. It's been like this for awhile, so back-patch to all supported branches. Discussion: https://postgr.es/m/CACP=ajbrFFYUrLyJBLV8=q+eNCapa1xDEyvXhMoYrNphs-xqPw@mail.gmail.com
2019-11-05Fix "unexpected relkind" error when denying permissions on toast tables.Tom Lane
get_relkind_objtype, and hence get_object_type, failed when applied to a toast table. This is not a good thing, because it prevents reporting of perfectly legitimate permissions errors. (At present, these functions are in fact *only* used to determine the ObjectType argument for acl_error() calls.) It seems best to have them fall back to returning OBJECT_TABLE in every case where they can't determine an object type for a pg_class entry, so do that. In passing, make some edits to alter.c to make it more obvious that those calls of get_object_type() are used only for error reporting. This might save a few cycles in the non-error code path, too. Back-patch to v11 where this issue originated. John Hsu, Michael Paquier, Tom Lane Discussion: https://postgr.es/m/C652D3DF-2B0C-4128-9420-FB5379F6B1E4@amazon.com
2019-11-05Change pg_restore -f- to dump to stdout instead of to ./-Alvaro Herrera
Starting with PostgreSQL 12, pg_restore refuses to run when neither -d nor -f are specified (c.f. commit 413ccaa74d9a), and it also makes "-f -" mean the old implicit behavior of dumping to stdout. However, older branches write to a file called ./- when invoked like that, making it impossible to write pg_restore scripts that work across versions. This is a partial backpatch of the aforementioned commit to all older supported branches, providing an upgrade path. Discussion: https://postgr.es/m/20191006190839.GE18030@telsasoft.com
2019-11-04Stabilize pg_dump output order for similarly-named triggers and policies.Tom Lane
The code only compared two triggers' names and namespaces (the latter being the owning table's schema). This could result in falling back to an OID-based sort of similarly-named triggers on different tables. We prefer to avoid that, so add a comparison of the table names too. (The sort order is thus table namespace, trigger name, table name, which is a bit odd, but it doesn't seem worth contorting the code to work around that.) Likewise for policy objects, in 9.5 and up. Complaint and fix by Benjie Gillam. Back-patch to all supported branches. Discussion: https://postgr.es/m/CAMThMzEEt2mvBbPgCaZ1Ap1N-moGn=Edxmadddjq89WG4NpPtQ@mail.gmail.com
2019-11-04Catch invalid typlens in a couple of placesPeter Eisentraut
Rearrange the logic in record_image_cmp() and record_image_eq() to error out on unexpected typlens (either not supported there or completely invalid due to corruption). Barring corruption, this is not possible today but it seems more future-proof and robust to fix this. Reported-by: Peter Geoghegan <pg@bowt.ie>
2019-11-03Suppress warning from older compilers.Tom Lane
Commit 8af1624e3 introduced a warning about possibly returning without a value, on compilers that don't realize that ereport(ERROR) doesn't return. Tweak the code to avoid that. Per buildfarm. Back-patch to 9.6, like the aforesaid commit.
2019-11-02Validate ispell dictionaries more carefully.Tom Lane
Using incorrect, or just mismatched, dictionary and affix files could result in a crash, due to failure to cross-check offsets obtained from the file. Add necessary validation, as well as some Asserts for future-proofing. Per bug #16050 from Alexander Lakhin. Back-patch to 9.6 where the problem was introduced. Arthur Zakirov, per initial investigation by Tomas Vondra Discussion: https://postgr.es/m/16050-024ae722464ab604@postgresql.org Discussion: https://postgr.es/m/20191013012610.2p2fp3zzpoav7jzf@development
2019-11-02Fix failure when creating cloned indexes for a partitionMichael Paquier
When using CREATE TABLE for a new partition, the partitioned indexes of the parent are created automatically in a fashion similar to LIKE INDEXES. The new partition and its parent use a mapping for attribute numbers for this operation, and while the mapping was correctly built, its length was defined as the number of attributes of the newly-created child, and not the parent. If the parent includes dropped columns, this could cause failures. This is wrong since 8b08f7d which has introduced the concept of partitioned indexes, so backpatch down to 11. Reported-by: Wyatt Alt Author: Michael Paquier Reviewed-by: Amit Langote Discussion: https://postgr.es/m/CAGem3qCcRmhbs4jYMkenYNfP2kEusDXvTfw-q+eOhM0zTceG-g@mail.gmail.com Backpatch-through: 11
2019-11-01Fix race condition at backend exit when deleting element in syncrep queueMichael Paquier
When a backend exits, it gets deleted from the syncrep queue if present. The queue was checked without SyncRepLock taken in exclusive mode, so it would have been possible for a backend to remove itself after a WAL sender already did the job. Fix this issue based on a suggestion from Fujii Masao, by first checking the queue without the lock. Then, if the backend is present in the queue, take the lock and perform an additional lookup check before doing the element deletion. Author: Dongming Liu Reviewed-by: Kyotaro Horiguchi, Fujii Masao, Michael Paquier Discussion: https://postgr.es/m/a0806273-8bbb-43b3-bbe1-c45a58f6ae21.lingce.ldm@alibaba-inc.com Backpatch-through: 9.4
2019-10-29pg_waldump: Fix --bkp-details to not issue spurious newlines for FPWs.Andres Freund
The additional newline seems to have accidentally been introduced in 2c03216d831, in 9.5. The newline is only issued when an FPW is present for the block reference. While there could be an argument that removing the newlines in the back branches could cause a problem for somebody parsing the pg_waldump output, the likelihood of that seems small enough. It seems at least equally likely that the randomness of when newlines are issued causes problems. Author: Andres Freund Discussion: https://postgr.es/m/20191029233341.4gnyau7e5v2lh5sc@alap3.anarazel.de Backpatch: 9.5, like 2c03216d831.
2019-10-29pg_waldump: Fix small memory leak when rmgr->rm_identify returns NULL.Andres Freund
This got broken in 604f7956b94, shortly after rm_identify's introduction. Author: Andres Freund Discussion: https://postgr.es/m/20191029233341.4gnyau7e5v2lh5sc@alap3.anarazel.de Backpatch: 9.5, where rm_identify was introduced
2019-10-25Avoid failure when selecting a namespace node in XMLTABLE.Tom Lane
It appears that libxml2 doesn't bother to set the "children" field of an XML_NAMESPACE_DECL node to null; that field just contains garbage. In v10 and v11, this can result in a crash in XMLTABLE(). The rewrite done in commit 251cf2e27 fixed this, somewhat accidentally, in v12. We're not going to back-patch 251cf2e27, however. The case apparently doesn't have wide use, so rather than risk introducing other problems, just add a safety check to throw an error. Even though no bug manifests in v12/HEAD, add the relevant test case there too, to prevent future regressions. Chapman Flack (per private report)
2019-10-23Clean up properly error_context_stack in autovacuum worker on exceptionMichael Paquier
Any callback set would have no meaning in the context of an exception. As an autovacuum worker exits quickly in this context, this could be only an issue within EmitErrorReport(), where the elog hook is for example called. That's unlikely to going to be a problem, but let's be clean and consistent with other code paths handling exceptions. This is present since 2909419, which introduced autovacuum. Author: Ashwin Agrawal Reviewed-by: Tom Lane, Michael Paquier Discussion: https://postgr.es/m/CALfoeisM+_+dgmAdAOHAu0k-ZpEHHqSSG=GRf3pKJGm8OqWX0w@mail.gmail.com Backpatch-through: 9.4
2019-10-21Deal with yet another issue related to "Norwegian (Bokmål)" locale.Tom Lane
It emerges that recent versions of Windows (at least 2016 Standard) spell this locale name as "Norwegian Bokmål_Norway.1252", defeating our mapping code that translates "Norwegian (Bokmål)_Norway" to something that's all-ASCII (cf commits db29620d4 and aa1d2fc5e). Add another mapping entry to handle this spelling. Per bug #16068 from Robert Ford. Like the previous patches, back-patch to all supported branches. Discussion: https://postgr.es/m/16068-4cb6eeaa7eb46d93@postgresql.org
2019-10-21Select CFLAGS_SL at configure time, not in platform-specific Makefiles.Tom Lane
Move the platform-dependent logic that sets CFLAGS_SL from src/makefiles/Makefile.foo to src/template/foo, so that the value is determined at configure time and thus is available while running configure's tests. On a couple of platforms this might save a few microseconds of build time by eliminating a test that make otherwise has to do over and over. Otherwise it's pretty much a wash for build purposes; in particular, this makes no difference to anyone who might be overriding CFLAGS_SL via a make option. This patch in itself does nothing with the value and thus should not change any behavior, though you'll probably have to re-run configure to get a correctly updated Makefile.global. We'll use the new configure variable in a follow-on patch. Per gripe from Kyotaro Horiguchi. Back-patch to all supported branches, because the follow-on patch is a portability bug fix. Discussion: https://postgr.es/m/20191010.144533.263180400.horikyota.ntt@gmail.com
2019-10-18For PowerPC instruction "addi", use constraint "b".Noah Misch
Without "b", a variant of the tas() code miscompiles on macOS 10.4. This may also fix a compilation failure involving macOS 10.1. Today's compilers have been allocating acceptable registers with or without this change, but this future-proofs the code by precisely conveying the acceptable registers. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. Discussion: https://postgr.es/m/20191009063900.GA4066266@rfd.leadboat.com
2019-10-18Fix failure of archive recovery with recovery_min_apply_delay enabled.Fujii Masao
recovery_min_apply_delay parameter is intended for use with streaming replication deployments. However, the document clearly explains that the parameter will be honored in all cases if it's specified. So it should take effect even if in archive recovery. But, previously, archive recovery with recovery_min_apply_delay enabled always failed, and caused assertion failure if --enable-caasert is enabled. The cause of this problem is that; the ownership of recoveryWakeupLatch that recovery_min_apply_delay uses was taken only when standby mode is requested. So unowned latch could be used in archive recovery, and which caused the failure. This commit changes recovery code so that the ownership of recoveryWakeupLatch is taken even in archive recovery. Which prevents archive recovery with recovery_min_apply_delay from failing. Back-patch to v9.4 where recovery_min_apply_delay was added. Author: Fujii Masao Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAHGQGwEyD6HdZLfdWc+95g=VQFPR4zQL4n+yHxQgGEGjaSVheQ@mail.gmail.com
2019-10-18Fix timeout handling in logical replication workerMichael Paquier
The timestamp tracking the last moment a message is received in a logical replication worker was initialized in each loop checking if a message was received or not, causing wal_receiver_timeout to be ignored in basically any logical replication deployments. This also broke the ping sent to the server when reaching half of wal_receiver_timeout. This simply moves the initialization of the timestamp out of the apply loop to the beginning of LogicalRepApplyLoop(). Reported-by: Jehan-Guillaume De Rorthais Author: Julien Rouhaud Discussion: https://postgr.es/m/CAOBaU_ZHESFcWva8jLjtZdCLspMj7vqaB2k++rjHLY897ZxbYw@mail.gmail.com Backpatch-through: 10
2019-10-17Fix minor bug in logical-replication walsender shutdownAlvaro Herrera
Logical walsender should exit when it catches up with sending WAL during shutdown; but there was a rare corner case when it failed to because of a race condition that puts it back to wait for more WAL instead -- but since there wasn't any, it'd not shut down immediately. It would only continue the shutdown when wal_sender_timeout terminates the sleep, which causes annoying waits during shutdown procedure. Restructure the code so that we no longer forget to set WalSndCaughtUp in that case. This was an oversight in commit c6c333436. Backpatch all the way down to 9.4. Author: Craig Ringer, Álvaro Herrera Discussion: https://postgr.es/m/CAMsr+YEuz4XwZX_QmnX_-2530XhyAmnK=zCmicEnq1vLr0aZ-g@mail.gmail.com
2019-10-17When restoring GUCs in parallel workers, show an error context.Thomas Munro
Otherwise it can be hard to see where an error is coming from, when the parallel worker sets all the GUCs that it received from the leader. Bug #15726. Back-patch to 9.5, where RestoreGUCState() appeared. Reported-by: Tiago Anastacio Reviewed-by: Daniel Gustafsson, Tom Lane Discussion: https://postgr.es/m/15726-6d67e4fa14f027b3%40postgresql.org
2019-10-17Fix bug that could try to freeze running multixacts.Thomas Munro
Commits 801c2dc7 and 801c2dc7 made it possible for vacuum to try to freeze a multixact that is still running. That was prevented by a check, but raised an error. Repair. Back-patch all the way. Author: Nathan Bossart, Jeremy Schneider Reported-by: Jeremy Schneider Reviewed-by: Jim Nasby, Thomas Munro Discussion: https://postgr.es/m/DAFB8AFF-2F05-4E33-AD7F-FF8B0F760C17%40amazon.com
2019-10-16Improve the check for pg_catalog.unknown data type in pg_upgradeTomas Vondra
The pg_upgrade check for pg_catalog.unknown type when upgrading from 9.6 had a couple of issues with domains and composite types - it detected even composite types unused in objects with storage. So for example this was enough to trigger an unnecessary pg_upgrade failure: CREATE TYPE unknown_composite AS (u pg_catalog.unknown) On the other hand, this only happened with composite types directly on the pg_catalog.unknown data type, but not with a domain. So this was not detected CREATE DOMAIN unknown_domain AS pg_catalog.unknown; CREATE TYPE unknown_composite_2 AS (u unknown_domain); unlike the first example. These false positives and inconsistencies are unfortunate, but what's worse we've failed to detected objects using the pg_catalog.unknown type through a domain. So we missed cases like this CREATE TABLE t (u unknown_composite_2); The consequence is clusters broken after a pg_upgrade. This fixes these false positives and false negatives by using the same recursive CTE introduced by eaf900e842 for sql_identifier. Backpatch all the way to 10, where the of pg_catalog.unknown data type was restricted. Author: Tomas Vondra Backpatch-to: 10- Discussion: https://postgr.es/m/16045-673e8fa6b5ace196%40postgresql.org
2019-10-16Improve the check for pg_catalog.line data type in pg_upgradeTomas Vondra
The pg_upgrade check for pg_catalog.line data type when upgrading from 9.3 had a couple of issues with domains and composite types. Firstly, it triggered false positives for composite types unused in objects with storage. This was enough to trigger an unnecessary pg_upgrade failure: CREATE TYPE line_composite AS (l pg_catalog.line) On the other hand, this only happened with composite types directly on the pg_catalog.line data type, but not with a domain. So this was not detected CREATE DOMAIN line_domain AS pg_catalog.line; CREATE TYPE line_composite_2 AS (l line_domain); unlike the first example. These false positives and inconsistencies are unfortunate, but what's worse we've failed to detected objects using the pg_catalog.line data type through a domain. So we missed cases like this CREATE TABLE t (l line_composite_2); The consequence is clusters broken after a pg_upgrade. This fixes these false positives and false negatives by using the same recursive CTE introduced by eaf900e842 for sql_identifier. 9.3 did not support domains on composite types, but we can still have multi-level composite types. Backpatch all the way to 9.4, where the format for pg_catalog.line data type changed. Author: Tomas Vondra Backpatch-to: 9.4- Discussion: https://postgr.es/m/16045-673e8fa6b5ace196%40postgresql.org
2019-10-12AIX: Stop adding option -qsrcmsg.Noah Misch
With xlc v16.1.0, it causes internal compiler errors. With xlc versions not exhibiting that bug, removing -qsrcmsg merely changes the compiler error reporting format. Back-patch to 9.4 (all supported versions). Discussion: https://postgr.es/m/20191003064105.GA3955242@rfd.leadboat.com
2019-10-09Flush logical mapping files with fd opened for read/write at checkpointMichael Paquier
The file descriptor was opened with read-only to fsync a regular file, which would cause EBADFD errors on some platforms. This is similar to the recent fix done by a586cc4b (which was broken by me with 82a5649), except that I noticed this issue while monitoring the backend code for similar mistakes. Backpatch to 9.4, as this has been introduced since logical decoding exists as of b89e151. Author: Michael Paquier Reviewed-by: Andres Freund Discussion: https://postgr.es/m/20191006045548.GA14532@paquier.xyz Backpatch-through: 9.4
2019-10-07Check for too many postmaster children before spawning a bgworker.Tom Lane
The postmaster's code path for spawning a bgworker neglected to check whether we already have the max number of live child processes. That's a bit hard to hit, since it would necessarily be a transient condition; but if we do, AssignPostmasterChildSlot() fails causing a postmaster crash, as seen in a report from Bhargav Kamineni. To fix, invoke canAcceptConnections() in the bgworker code path, as we do in the other code paths that spawn children. Since we don't want the same pmState tests in this case, add a child-process-type parameter to canAcceptConnections() so that it can know what to do. Back-patch to 9.5. In principle the same hazard exists in 9.4, but the code is enough different that this patch wouldn't quite fix it there. Given the tiny usage of bgworkers in that branch it doesn't seem worth creating a variant patch for it. Discussion: https://postgr.es/m/18733.1570382257@sss.pgh.pa.us
2019-10-05Report test_atomic_ops() failures consistently, via macros.Noah Misch
This prints the unexpected value in more failure cases, and it removes forty-eight hand-maintained error messages. Back-patch to 9.5, which introduced these tests. Reviewed (in an earlier version) by Andres Freund. Discussion: https://postgr.es/m/20190915160021.GA24376@alvherre.pgsql
2019-10-05Avoid use of wildcard in pg_waldump's .gitignore.Tom Lane
This would be all right, maybe, if it didn't also match a file that definitely should not be ignored. We don't add rmgrs so often that manual maintenance of this file list is impractical, so just write out the list. (I find the equivalent wildcard use in the Makefile pretty lazy and unsafe as well, but will leave that alone until it actually causes a problem.) Per bug #16042 from Denis Stuchalin. Discussion: https://postgr.es/m/16042-c174ee692ac21cbd@postgresql.org
2019-10-05Disable one more set of tests from c8841199509.Andres Freund
Discussion: https://postgr.es/m/20191004222437.45qmglpto43pd3jb@alap3.anarazel.de Backpatch: 9.6-, just like c8841199509 and 6e61d75f525
2019-10-04Disable one set of tests from c8841199509.Andres Freund
One of the upsert related tests is unstable (sometimes even hanging until isolationtester's step timeout is reached). Based on preliminary analysis that might be a problem outside of just that test, but not really related to EPQ and triggers. Disable for now, to get the buildfarm greener again. Discussion: https://postgr.es/m/20191004222437.45qmglpto43pd3jb@alap3.anarazel.de Backpatch: 9.6-, just like c8841199509.
2019-10-04Add isolation tests for the combination of EPQ and triggers.Andres Freund
As evidenced by bug #16036 this area is woefully under-tested. Add fairly extensive tests for the combination. Backpatch back to 9.6 - before that isolationtester was not capable enough. While we don't backpatch tests all the time, future fixes to trigger.c would potentially look different enough in 12+ from the earlier branches that introducing bugs during backpatching is more likely than normal. Also, it's just a crucial and undertested area of the code. Author: Andres Freund Discussion: https://postgr.es/m/16036-28184c90d952fb7f@postgresql.org Backpatch: 9.6-, the earliest these tests work
2019-10-04Handle spaces in OpenSSL install location for MSVCAndrew Dunstan
First, make sure that the .exe name is quoted when trying to get the version number. Also, don't quote the lib name for using in the project files if it's already been quoted. This second change applies to all libraries, not just OpenSSL. This has clearly been broken forever, so backpatch to all live branches.
2019-10-04Fix bitshiftright()'s zero-padding some more.Tom Lane
Commit 5ac0d9360 failed to entirely fix bitshiftright's habit of leaving one-bits in the pad space that should be all zeroes, because in a moment of sheer brain fade I'd concluded that only the code path used for not-a-multiple-of-8 shift distances needed to be fixed. Of course, a multiple-of-8 shift distance can also cause the problem, so we need to forcibly zero the extra bits in both cases. Per bug #16037 from Alexander Lakhin. As before, back-patch to all supported branches. Discussion: https://postgr.es/m/16037-1d1ebca564db54f4@postgresql.org
2019-10-03Avoid unnecessary out-of-memory errors during encoding conversion.Tom Lane
Encoding conversion uses the very simplistic rule that the output can't be more than 4X longer than the input, and palloc's a buffer of that size. This results in failure to convert any string longer than 1/4 GB, which is becoming an annoying limitation. As a band-aid to improve matters, allow the allocated output buffer size to exceed 1GB. We still insist that the final result fit into MaxAllocSize (1GB), though. Perhaps it'd be safe to relax that restriction, but it'd require close analysis of all callers, which is daunting (not least because external modules might call these functions). For the moment, this should allow a 2X to 4X improvement in the longest string we can convert, which is a useful gain in return for quite a simple patch. Also, once we have successfully converted a long string, repalloc the output down to the actual string length, returning the excess to the malloc pool. This seems worth doing since we can usually expect to give back several MB if we take this path at all. This still leaves much to be desired, most notably that the assumption that MAX_CONVERSION_GROWTH == 4 is very fragile, and yet we have no guard code verifying that the output buffer isn't overrun. Fixing that would require significant changes in the encoding conversion APIs, so it'll have to wait for some other day. The present patch seems safely back-patchable, so patch all supported branches. Alvaro Herrera and Tom Lane Discussion: https://postgr.es/m/20190816181418.GA898@alvherre.pgsql Discussion: https://postgr.es/m/3614.1569359690@sss.pgh.pa.us
2019-10-03Allow repalloc() to give back space when a large chunk is downsized.Tom Lane
Up to now, if you resized a large (>8K) palloc chunk down to a smaller size, aset.c made no attempt to return any space to the malloc pool. That's unpleasant if a really large allocation is resized to a significantly smaller size. I think no such cases existed when this code was designed, and I'm not sure whether they're common even yet, but an upcoming fix to encoding conversion will certainly create such cases. Therefore, fix AllocSetRealloc so that it gives realloc() a chance to do something with the block. This doesn't noticeably increase complexity, we mostly just have to change the order in which the cases are considered. Back-patch to all supported branches. Discussion: https://postgr.es/m/20190816181418.GA898@alvherre.pgsql Discussion: https://postgr.es/m/3614.1569359690@sss.pgh.pa.us
2019-10-03Selectively include window frames in expression walks/mutates.Andrew Gierth
query_tree_walker and query_tree_mutator were skipping the windowClause of the query, without regard for the fact that the startOffset and endOffset in a WindowClause node are expression trees that need to be processed. This was an oversight in commit ec4be2ee6 from 2010 which added the expression fields; the main symptom is that function parameters in window frame clauses don't work in inlined functions. Fix (as conservatively as possible since this needs to not break existing out-of-tree callers) and add tests. Backpatch all the way, since this has been broken since 9.0. Per report from Alastair McKinley; fix by me with kibitzing and review from Tom Lane. Discussion: https://postgr.es/m/DB6PR0202MB2904E7FDDA9D81504D1E8C68E3800@DB6PR0202MB2904.eurprd02.prod.outlook.com
2019-10-02Remove temporary WAL and history files at the end of archive recoveryMichael Paquier
cbc55da has reworked the order of some actions at the end of archive recovery. Unfortunately this overlooked the fact that the startup process needs to remove RECOVERYXLOG (for temporary WAL segment newly recovered from archives) and RECOVERYHISTORY (for temporary history file) at this step, leaving the files around even after recovery ended. Backpatch to 9.5, like the previous commit. Author: Sawada Masahiko Reviewed-by: Fujii Masao, Michael Paquier Discussion: https://postgr.es/m/CAD21AoBO_eDQub6zojFnWtnmutRBWvYf7=cW4Hsqj+U_R26w3Q@mail.gmail.com Backpatch-through: 9.5
2019-09-30Suppress another CR in program outputAndrew Dunstan
This one was exposed by a12c75a10. Backpatch to release 11 where check_pg_config was introduced.
2019-09-29jit: Re-allow JIT compilation of execGrouping.c hashtable comparisons.Andres Freund
In the course of 5567d12ce03, 356687bd8 and 317ffdfeaac, I changed BuildTupleHashTable[Ext]'s call to ExecBuildGroupingEqual to not pass in the parent node, but NULL. Which in turn prevents the tuple equality comparator from being JIT compiled. While that fixes bug #15486, it is not actually necessary after all of the above commits, as we don't re-build the comparator when using the new BuildTupleHashTableExt() interface (as the content of the hashtable are reset, but the TupleHashTable itself is not). Therefore re-allow jit compilation for callers that use BuildTupleHashTableExt with a separate context for "metadata" and content. As in the previous commit, there's ongoing work to make this easier to test to prevent such regressions in the future, but that infrastructure is not going to be backpatchable. The performance impact of not JIT compiling hashtable equality comparators can be substantial e.g. for aggregation queries that aggregate a lot of input rows to few output rows (when there are a lot of output groups, there will be fewer comparisons). Author: Andres Freund Discussion: https://postgr.es/m/20190927072053.njf6prdl3vb7y7qb@alap3.anarazel.de Backpatch: 11, just as 5567d12ce03
2019-09-29Allow SSL TAP tests to run on WindowsAndrew Dunstan
Windows does not enforce key file permissions checks in libpq, and psql can produce CRLF line endings on Windows. Backpatch to Release 12 (CRLF) and Release 11 (permissions check)
2019-09-28Improve stability of partition_prune regression test.Tom Lane
This test already knew that, to get stable test output, it had to hide "loops" counts in EXPLAIN ANALYZE results. But that's not nearly enough: if we get a smaller number of workers than we planned for, then the "Workers Launched" number will change, and so will all the rows and loops counts up to the Gather node. This has resulted in repeated failures in the buildfarm, so adjust the test to filter out all these counts. (Really, we wouldn't bother with EXPLAIN ANALYZE at all here, except that currently the only way to verify that executor-time pruning has happened is to look for '(never executed)' annotations. Those are stable and needn't be filtered out.) Back-patch to v11 where the test was introduced. Discussion: https://postgr.es/m/11952.1569536725@sss.pgh.pa.us