summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2019-05-24Fix typos.Thomas Munro
Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CA%2BhUKGJFWXmtYo6Frd77RR8YXCHz7hJ2mRy5aHV%3D7fJOqDnBHA%40mail.gmail.com
2019-05-23tableam: Rename wrapper functions to match callback names.Andres Freund
Some of the wrapper functions didn't match the callback names. Many of them due to staying "consistent" with historic naming of the wrapped functionality. We decided that for most cases it's more important to be for tableam to be consistent going forward, than with the past. The one exception is beginscan/endscan/... because it'd have looked odd to have systable_beginscan/endscan/... with a different naming scheme, and changing the systable_* APIs would have caused way too much churn (including breaking a lot of external users). Author: Ashwin Agrawal, with some small additions by Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/CALfoeiugyrXZfX7n0ORCa4L-m834dzmaE8eFdbNR6PMpetU4Ww@mail.gmail.com
2019-05-24Fix table dump in pg_dump[all] with backends older than 9.5Michael Paquier
The access method name "amname" can be dumped as of 3b925e90, but queries for backends older than 9.5 forgot to map it to a dummy NULL value, causing the column to not be mapped to a number. As a result, pg_dump was throwing some spurious errors in its stderr output coming from libpq: pg_dump: column number -1 is out of range 0..36 Fix this issue by adding a mapping of "amname" to NULL to all the older queries. Discussion: https://postgr.es/m/20190522083038.GA16837@paquier.xyz Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Andres Freund, Tom Lane
2019-05-23pg_upgrade: Make test.sh's installcheck use to-be-upgraded version's bindir.Andres Freund
On master (after 700538) the old version's installed psql was used - even when the old version might not actually be installed / might be installed into a temporary directory. As commonly the case when just executing make check for pg_upgrade, as $oldbindir is just the current version's $bindir. In the back branches, with --install specified, psql from the new version's temporary installation was used, without --install (e.g for NO_TEMP_INSTALL, cf 47b3c26642), the new version's installed psql was used (which might or might not exist). Author: Andres Freund Discussion: https://postgr.es/m/20190522175150.c26f4jkqytahajdg@alap3.anarazel.de
2019-05-23Fix array size allocation for HashAggregate hash keys.Andrew Gierth
When there were duplicate columns in the hash key list, the array sizes could be miscomputed, resulting in access off the end of the array. Adjust the computation to ensure the array is always large enough. (I considered whether the duplicates could be removed in planning, but I can't rule out the possibility that duplicate columns might have different hash functions assigned. Simpler to just make sure it works at execution time regardless.) Bug apparently introduced in fc4b3dea2 as part of narrowing down the tuples stored in the hashtable. Reported by Colm McHugh of Salesforce, though I didn't use their patch. Backpatch back to version 10 where the bug was introduced. Discussion: https://postgr.es/m/CAFeeJoKKu0u+A_A9R9316djW-YW3-+Gtgvy3ju655qRHR3jtdA@mail.gmail.com
2019-05-23Fix ordering of GRANT commands in pg_dumpall for tablespacesMichael Paquier
This uses a method similar to 68a7c24f and now b8c6014 (applied for database creation), which guarantees that GRANT commands using the WITH GRANT OPTION are dumped in a way so as cascading dependencies are respected. Note that tablespaces do not have support for initial privileges via pg_init_privs, so the same method needs to be applied again. It would be nice to merge all the logic generating ACL queries in dumps under the same banner, but this requires extending the support of pg_init_privs to objects that cannot use it yet, so this is left as future work. Discussion: https://postgr.es/m/20190522071555.GB1278@paquier.xyz Author: Michael Paquier Reviewed-by: Nathan Bossart Backpatch-through: 9.6
2019-05-23Remove -o/--oids from pg_dumpallMichael Paquier
This has been forgotten in 578b229, which has removed support for WITH OIDS. Discussion: https://postgr.es/m/CALAY4q99FcFCoG6ddke0V-AksGe82L_+bhDWgEfgZBakB840zA@mail.gmail.com Author: Surafel Temesgen
2019-05-22Initial pgperltidy run for v12.Tom Lane
Make all the perl code look nice, too (for some value of "nice").
2019-05-22Phase 2 pgindent run for v12.Tom Lane
Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22Initial pgindent run for v12.Tom Lane
This is still using the 2.0 version of pg_bsd_indent. I thought it would be good to commit this separately, so as to document the differences between 2.0 and 2.1 behavior. Discussion: https://postgr.es/m/16296.1558103386@sss.pgh.pa.us
2019-05-22Convert ExecComputeStoredGenerated to use tuple slotsPeter Eisentraut
This code was still using the old style of forming a heap tuple rather than using tuple slots. This would be less efficient if a non-heap access method was used. And using tuple slots is actually quite a bit faster when using heap as well. Also add some test cases for generated columns with null values and with varlena values. This lack of coverage was discovered while working on this patch. Discussion: https://www.postgresql.org/message-id/flat/20190331025744.ugbsyks7czfcoksd%40alap3.anarazel.de
2019-05-23Mention ANALYZE boolean options in documentation.Fujii Masao
Commit 41b54ba78e allowed not only VACUUM but also ANALYZE options to take a boolean argument. But it forgot to update the documentation for ANALYZE. This commit adds the descriptions about those ANALYZE boolean options into the documentation. This patch also updates tab-completion for ANALYZE boolean options. Reported-by: Kyotaro Horiguchi Author: Fujii Masao Reviewed-by: Masahiko Sawada, Michael Paquier Discussion: https://postgr.es/m/CAHGQGwHTUt-kuwgiwe8f0AvTnB+ySqJWh95jvmh-qcoKW9YA9g@mail.gmail.com
2019-05-22Fix O(N^2) performance issue in pg_publication_tables view.Tom Lane
The original coding of this view relied on a correlated IN sub-query. Our planner is not very bright about correlated sub-queries, and even if it were, there's no way for it to know that the output of pg_get_publication_tables() is duplicate-free, making the de-duplicating semantics of IN unnecessary. Hence, rewrite as a LATERAL sub-query. This provides circa 100X speedup for me with a few hundred published tables (the whole regression database), and things would degrade as roughly O(published_relations * all_relations) beyond that. Because the rules.out expected output changes, force a catversion bump. Ordinarily we might not want to do that post-beta1; but we already know we'll be doing a catversion bump before beta2 to fix pg_statistic_ext issues, so it's pretty much free to fix it now instead of waiting for v13. Per report and fix suggestion from PegoraroF10. Discussion: https://postgr.es/m/1551385426763-0.post@n3.nabble.com
2019-05-22Add .gitignore entries for new ecpg test case.Tom Lane
Oversight in commit a1dc6ab465986a62b308dd1bb8da316b5ed9685a.
2019-05-22In transam.h, don't expose static inline functions to frontend code.Tom Lane
That leads to unsatisfied external references if the C compiler fails to elide unused static functions. Apparently, we have no buildfarm members building HEAD that have that issue ... but such compilers still exist in the wild. Need to do something about that. In passing, fix Berkeley-era typo in comment. Discussion: https://postgr.es/m/27054.1558533367@sss.pgh.pa.us
2019-05-22Fix ordering of GRANT commands in pg_dump for database creationMichael Paquier
This uses a method similar to 68a7c24f, which guarantees that GRANT commands using the WITH GRANT OPTION are dumped in a way so as cascading dependencies are respected. As databases do not have support for initial privileges via pg_init_privs, we need to repeat again the same ACL reordering method. ACL for databases have been moved from pg_dumpall to pg_dump in v11, so this impacts pg_dump for v11 and above, and pg_dumpall for v9.6 and v10. Discussion: https://postgr.es/m/15788-4e18847520ebcc75@postgresql.org Author: Nathan Bossart Reviewed-by: Haribabu Kommi Backpatch-through: 9.6
2019-05-21Un-break pg_upgrade regression test.Tom Lane
Commit 5af2e976d removed a bit too much from the test.sh invocation. Per buildfarm.
2019-05-22Implement PREPARE AS statement for ECPG.Michael Meskes
Besides implementing the new statement this change fix some issues with the parsing of PREPARE and EXECUTE statements. The different forms of these statements are now all handled in a ujnified way. Author: Matsumura-san <matsumura.ryo@jp.fujitsu.com>
2019-05-21pg_upgrade: Avoid check target accidentally breaking make's --output-sync.Andres Freund
When $(MAKE) is present in a rule, make assumes that target is a submake, and it doesn't need to buffer its output. But in this case it's a shell script that needs buffered output. Avoid that heuristic, by referring to $(MAKE) via an indirection. Discussion: https://postgr.es/m/20190521004717.qsktdsugj3shagco@alap3.anarazel.de
2019-05-21pg_upgrade: Don't use separate installation for test.Andres Freund
For pg_upgrade's test we (unless prevented by the caller via via NO_TEMP_INSTALL) built a separate installation. That causes an unnecessary slowdown after the infrastructure introduced by dcae5faccab (and unnecessarily duplicates code). Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20190521191918.z7kwnrlj45mk2k67@alap3.anarazel.de https://postgr.es/m/20190521195209.qfzwfxvymguuwlu5@alap3.anarazel.de
2019-05-21Make pg_upgrade's test.sh less chatty.Tom Lane
The use of "set -x" to echo a subset of the test's commands might've been a good idea during development of this test, but it's been stable for long enough now that the extra output isn't very useful. Also our project expectations have been trending towards less output in non-error cases; the fact that "set -x" produces output on stderr is particularly annoying from that standpoint. So get rid of it. Also, pass "-A trust" to initdb explicitly so that it won't issue a warning about "trust" being an insecure default. This matches what the TAP tests have done for a long time, and again gets rid of some noise on stderr. Discussion: https://postgr.es/m/21766.1558397960@sss.pgh.pa.us
2019-05-21Insert temporary debugging output in regression tests.Tom Lane
We're seeing occasional instability in the plans generated for parallel queries on the "a_star" table hierarchy. This suggests that something is changing the planner's stats for those tables, but that should not be happening within a regression test run. To try to gather some information about what's happening, insert additional queries to check the basic page/tuple counts for these tables, as well as whether any vacuums or analyzes have happened on them. (We expect that only the database-wide VACUUM in sanity_check.sql will have touched them.) I added the probes not only in select_parallel.sql itself, but also in stats.sql, bearing in mind that the stats collector's lag may prevent the initial query from reporting current truth. If any extra vacuum/analyze has happened, the recheck in stats.sql definitely ought to see it. This commit can be reverted once we figure out what's going on. Per suggestion from David Rowley, though I changed the queries around. Discussion: https://postgr.es/m/CA+hUKG+0CxrKRWRMf5ymN3gm+BECHna2B-q1w8onKBep4HasUw@mail.gmail.com
2019-05-21tableam: Move heap-specific logic from needs_toast_table below tableam.Robert Haas
This allows table AMs to completely suppress TOAST table creation, or to modify the conditions under which they are created. Patch by me. Reviewed by Andres Freund. Discussion: http://postgr.es/m/CA+Tgmoa4O2n=yphqD2pERUnYmUO84bH1SqMsA-nSxBGsZ7gWfA@mail.gmail.com
2019-05-20Stamp 12beta1.REL_12_BETA1Tom Lane
2019-05-20Fix regression tests broken in fc7c281f87467.Andres Freund
This shouldn't have been committed without even running the tests (nor were the tests added that were suggested). I'm fixing up the results to get the buildfarm back to green, it's quite possible we'll want to revert this later.
2019-05-21Fix comment for issue_xlog_fsync().Fujii Masao
"segno" is the argument for the function, not "log" and "seg". Author: Antonin Houska Discussion: https://postgr.es/m/11863.1558361020@spoje.net
2019-05-21Make VACUUM accept 1 and 0 as a boolean value.Fujii Masao
Commit 41b54ba78e allowed existing VACUUM options to take a boolean argument. It's documented that valid boolean values that VACUUM can accept are true, false, on, off, 1, and 0. But previously the parser failed to accept 1 and 0 as a boolean value in VACUUM syntax because of a lack of NumericOnly clause for vac_analyze_option_arg in gram.y. This commit adds such NumericOnly clause so that VACUUM options can take also 1 and 0 as a boolean value. Discussion: https://postgr.es/m/CAHGQGwGYg82A8UCQxZe7Zn9MnyUBGdyB=1CNpKF3jBny+RbyfA@mail.gmail.com
2019-05-20Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: a20bf6b8a5b4e32450967055eb5b07cee4704edd
2019-05-20Remove bug.template filePeter Eisentraut
It's outdated and not really in use anymore. Discussion: https://www.postgresql.org/message-id/flat/cf7ed2b1-1ebe-83cf-e05e-d5943f67af2d%402ndquadrant.com
2019-05-19Remove outdated comment in copy.c.Andres Freund
2019-05-19Minimally fix partial aggregation for aggregates that don't have one argument.Andres Freund
For partial aggregation combine steps, AggStatePerTrans->numTransInputs was set to the transition function's number of inputs, rather than the combine function's number of inputs (always 1). That lead to partial aggregates with strict combine functions to wrongly check for NOT NULL input as required by strictness. When the aggregate wasn't exactly passed one argument, the strictness check was either omitted (in the 0 args case) or too many arguments were checked. In the latter case we'd read beyond the end of FunctionCallInfoData->args (only in master). AggStatePerTrans->numTransInputs actually has been wrong since since 9.6, where partial aggregates were added. But it turns out to not be an active problem in 9.6 and 10, because numTransInputs wasn't used at all for combine functions: Before c253b722f6 there simply was no NULL check for the input to strict trans functions, and after that the check was simply hardcoded for the right offset in fcinfo, as it's done by code specific to combine functions. In bf6c614a2f2 (11) the strictness check was generalized, with common code doing the strictness checks for both plain and combine transition functions, based on numTransInputs. For combine functions this lead to not emitting an expression step to check for strict input in the 0 arguments case, and in the > 1 arguments case, we'd check too many arguments.Due to the fact that the relevant fcinfo->isnull[2..] was always zero-initialized (more or less by accident, by being part of the AggStatePerTrans struct, which is palloc0'ed), there was no observable damage in the latter case before a9c35cf85ca1f, we just checked too many array elements. Due to the changes in a9c35cf85ca1f, > 1 argument bug became visible, because these days fcinfo is a) dynamically allocated without being zeroed b) exactly the length required for the number of specified arguments (hardcoded to 2 in this case). This commit only contains a fairly minimal fix, setting numTransInputs to a hardcoded 1 when building a pertrans for a combine function. It seems likely that we'll want to clean this up further (e.g. the arguments build_pertrans_for_aggref() aren't particularly meaningful for combine functions). But the wrap date for 12 beta1 is coming up fast, so it seems good to have a minimal fix in place. Backpatch to 11. While AggStatePerTrans->numTransInputs was set wrongly before that, the value was not used for combine functions. Reported-By: Rajkumar Raghuwanshi Diagnosed-By: Kyotaro Horiguchi, Jeevan Chalke, Andres Freund, David Rowley Author: David Rowley, Kyotaro Horiguchi, Andres Freund Discussion: https://postgr.es/m/CAKcux6=uZEyWyLw0N7HtR9OBc-sWEFeByEZC7t-KDf15FKxVew@mail.gmail.com
2019-05-19Fix and improve SnapshotType comments.Andres Freund
The comment for SNAPSHOT_SELF was unfortunately explaining SNAPSHOT_DIRTY, as reported by Sergei. Also expand a few comments, and include a few more comments from heapam_visibility.c, so they're in an AM independent place. Reported-By: Sergei Kornilov Author: Andres Freund Discussion: https://postgr.es/m/9152241558192351@sas1-d856b3d759c7.qloud-c.yandex.net
2019-05-19Revert "In the pg_upgrade test suite, don't write to src/test/regress."Noah Misch
This reverts commit bd1592e8570282b1650af6b8eede0016496daecd. It had multiple defects. Discussion: https://postgr.es/m/12717.1558304356@sss.pgh.pa.us
2019-05-19Don't to predicate lock for analyze scans, refactor scan option passing.Andres Freund
Before this commit, when ANALYZE was run on a table and serializable was used (either by virtue of an explicit BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE, or default_transaction_isolation being set to serializable) a null pointer dereference lead to a crash. The analyze scan doesn't need a snapshot (nor predicate locking), but before this commit a scan only contained information about being a bitmap or sample scan. Refactor the option passing to the scan_begin callback to use a bitmask instead. Alternatively we could have added a new boolean parameter, but that seems harder to read. Even before this issue various people (Heikki, Tom, Robert) suggested doing so. These changes don't change the scan APIs outside of tableam. The flags argument could be exposed, it's not necessary to fix this problem. Also the wrapper table_beginscan* functions encapsulate most of that complexity. After these changes fixing the bug is trivial, just don't acquire predicate lock for analyze style scans. That was already done for bitmap heap scans. Add an assert that a snapshot is passed when acquiring the predicate lock, so this kind of bug doesn't require running with serializable. Also add a comment about sample scans currently requiring predicate locking the entire relation, that previously wasn't remarked upon. Reported-By: Joe Wildish Author: Andres Freund Discussion: https://postgr.es/m/4EA80A20-E9BF-49F1-9F01-5B66CAB21453@elusive.cx https://postgr.es/m/20190411164947.nkii4gaeilt4bui7@alap3.anarazel.de https://postgr.es/m/20190518203102.g7peu2fianukjuxm@alap3.anarazel.de
2019-05-19In the pg_upgrade test suite, don't write to src/test/regress.Noah Misch
When this suite runs installcheck, redirect file creations from src/test/regress to src/bin/pg_upgrade/tmp_check/regress. This closes a race condition in "make -j check-world". If the pg_upgrade suite wrote to a given src/test/regress/results file in parallel with the regular src/test/regress invocation writing it, a test failed spuriously. Even without parallelism, in "make -k check-world", the suite finishing second overwrote the other's regression.diffs. This revealed test "largeobject" assuming @abs_builddir@ is getcwd(), so fix that, too. Buildfarm client REL_10, released forty-five days ago, supports saving regression.diffs from its new location. When an older client reports a pg_upgradeCheck failure, it will no longer include regression.diffs. Back-patch to 9.5, where pg_upgrade moved to src/bin. Reviewed by Andrew Dunstan. Discussion: https://postgr.es/m/20181224034411.GA3224776@rfd.leadboat.com
2019-05-19Improve logrotate test so that it meaningfully exercises syslogger.Tom Lane
Discussion of bug #15804 reveals that this test didn't really prove that the syslogger child process ever launched successfully, much less did anything. It was only checking that the expected log file gets created, and that's done in the postmaster. Moreover, the test assumed it could rename the log file, which is likely to fail on Windows (cf. commit d611175e5). Instead, use the default log file name pattern, which should result in a new file name being chosen after 1 second, and verify that rotation has occurred by checking for a new file name. Also add code to test that messages actually do propagate through the syslogger. In theory this version of the test should work on Windows, so revert d611175e5. Discussion: https://postgr.es/m/15804-3721117bf40fb654@postgresql.org
2019-05-19Revert "postmaster: Start syslogger earlier".Tom Lane
This commit reverts 57431a911d3a650451d198846ad3194900666152. While that's still a good idea in the abstract, we found out that there are multiple crasher bugs in it on Windows builds, making the logging_collector option unusable on Windows. There's no time left to fix these issues before 12beta1, so revert the patch to allow Windows beta testing to proceed. We'll try again at some future date. Per bug #15804 from Yulian Khodorkovskiy and additional investigation by Michael Paquier. Discussion: https://postgr.es/m/15804-3721117bf40fb654@postgresql.org
2019-05-19Fix declarations of couple jsonpath functionsAlexander Korotkov
Make jsonb_path_query_array() and jsonb_path_query_first() use PG_FUNCTION_ARGS macro instead of its expansion.
2019-05-18ANSI-ify a few straggler K&R-style function definitions.Tom Lane
We still had a couple of these left in ancient src/port/ files. Convert them to modern style in preparation for switching to a version of pg_bsd_indent that doesn't cope well with K&R style. Discussion: https://postgr.es/m/16886.1558104483@sss.pgh.pa.us
2019-05-18Make BufFileCreateTemp() ensure that temp tablespaces are set up.Tom Lane
If PrepareTempTablespaces() has never been called in the current transaction, OpenTemporaryFile() will fall back to using the default tablespace, which is a bug if the user wanted temp files placed elsewhere. gistInitBuildBuffers() appears to have this disease already, and it seems like an easy trap for future coders to fall into. We discussed other ways to close this gap, but none of them are prettier or more reliable than just having BufFileCreateTemp do it. In particular, having fd.c do this creates layering issues that we could do without. Per suggestion from Melanie Plageman. Arguably this is a bug fix, but nobody seems very excited about back-patching, so change in HEAD only. Discussion: https://postgr.es/m/CAAKRu_YwzjuGAmmaw4-8XO=OVFGR1QhY_Pq-t3wjb9ribBJb_Q@mail.gmail.com
2019-05-18"A void function may not return a value".Tom Lane
Per buildfarm.
2019-05-17tableam: Avoid relying on relation size to determine validity of tids.Andres Freund
Instead add a tableam callback to do so. To avoid adding per validation overhead, pass a scan to tuple_tid_valid. In heap's case we'd otherwise incurred a RelationGetNumberOfBlocks() call for each tid - which'd have added noticable overhead to nodeTidscan.c. Author: Andres Freund Reviewed-By: Ashwin Agrawal Discussion: https://postgr.es/m/20190515185447.gno2jtqxyktylyvs@alap3.anarazel.de
2019-05-17tableam: Don't assume that every AM uses md.c style storage.Andres Freund
Previously various parts of the code routed size requests through RelationGetNumberOfBlocks[InFork]. That works if md.c is used by the AM, but not otherwise. Add a tableam callback to return the size of the table. As not every AM will use postgres' BLCKSZ, have it return bytes, and have RelationGetNumberOfBlocksInFork() round the byte size up into blocks. To allow code outside of the AM to determine the actual relation size map InvalidForkNumber the total size of a relation, as not every AM might just need the postgres defined forks. A few users of RelationGetNumberOfBlocks() ought to be converted away from that. One case, the use of it to determine whether a tid is valid, will be fixed in a follow up commit. Others will have to wait for v13. Author: Andres Freund Discussion: https://postgr.es/m/20190423225201.3bbv6tbqzkb5w7cw@alap3.anarazel.de
2019-05-17Restructure creation of run-time pruning steps.Tom Lane
Previously, gen_partprune_steps() always built executor pruning steps using all suitable clauses, including those containing PARAM_EXEC Params. This meant that the pruning steps were only completely safe for executor run-time (scan start) pruning. To prune at executor startup, we had to ignore the steps involving exec Params. But this doesn't really work in general, since there may be logic changes needed as well --- for example, pruning according to the last operator's btree strategy is the wrong thing if we're not applying that operator. The rules embodied in gen_partprune_steps() and its minions are sufficiently complicated that tracking their incremental effects in other logic seems quite impractical. Short of a complete redesign, the only safe fix seems to be to run gen_partprune_steps() twice, once to create executor startup pruning steps and then again for run-time pruning steps. We can save a few cycles however by noting during the first scan whether we rejected any clauses because they involved exec Params --- if not, we don't need to do the second scan. In support of this, refactor the internal APIs in partprune.c to make more use of passing information in the GeneratePruningStepsContext struct, rather than as separate arguments. This is, I hope, the last piece of our response to a bug report from Alan Jackson. Back-patch to v11 where this code came in. Discussion: https://postgr.es/m/FAD28A83-AC73-489E-A058-2681FA31D648@tvsquared.com
2019-05-17Fix regression test outputsMichael Paquier
75445c1 has caused various failures in tests across the tree after updating some error messages, so fix the newly-expected output. Author: Michael Paquier Reviewed-by: Tom Lane Discussion: https://postgr.es/m/8332.1558048838@sss.pgh.pa.us
2019-05-16More message style fixesAlvaro Herrera
Discussion: https://postgr.es/m/20190515183005.GA26486@alvherre.pgsql
2019-05-16Remove extra nbtree half-dead internal page check.Peter Geoghegan
It's not safe for nbtree VACUUM to attempt to delete a target page whose right sibling is already half-dead, since that would fail the cross-check when VACUUM attempts to re-find a downlink to the right sibling in the parent page. Logic to prevent this from happening was added by commit 8da31837803, which addressed a bug in the overhaul of page deletion that went into PostgreSQL 9.4 (commit efada2b8e92). VACUUM was made to check the right sibling page, and back off when it happened to be half-dead already. However, it is only truly necessary to do the right sibling check on the leaf level, since that transitively determines if the deletion target's parent's right sibling page is itself undergoing deletion. Remove the internal page level check, and add a comment explaining why the leaf level check alone suffices. The extra check is also unnecessary due to the fact that internal pages that are marked half-dead are generally considered corrupt. Commit efada2b8e92 established the principle that there should never be half-dead internal pages (internal pages pending deletion are possible, but that status is never directly represented in the internal page). VACUUM will complain about corruption when it encounters half-dead internal pages, so VACUUM is bound to raise an error one way or another when an nbtree index has a half-dead internal page (contrib/amcheck will also report that the page is corrupt). It's possible that a pg_upgrade'd 9.3 database will still have half-dead internal pages, so it may seem like there is an argument for leaving the check in place to reliably get a cleaner error message that advises the user to REINDEX. However, leaf pages are also deleted in the first phase of deletion prior to PostgreSQL 9.4, so I believe we won't even attempt to re-find the parent page anyway (we won't have the fully deleted leaf page as the right sibling of our target page, so we won't even try to find a downlink for it). Discussion: https://postgr.es/m/CAH2-Wzm_ntmqJjWLRyKzimFmFvk+BnVAvUpaA4s1h9Ja58woaQ@mail.gmail.com
2019-05-16Fix bogus logic for combining range-partitioned columns during pruning.Tom Lane
gen_prune_steps_from_opexps's notion of how to do this was overly complicated and underly correct. Per discussion of a report from Alan Jackson (though this fixes only one aspect of that problem). Back-patch to v11 where this code came in. Amit Langote Discussion: https://postgr.es/m/FAD28A83-AC73-489E-A058-2681FA31D648@tvsquared.com
2019-05-16Fix partition pruning to treat stable comparison operators properly.Tom Lane
Cross-type comparison operators in a btree or hash opclass might be only stable not immutable (this is true of timestamp vs. timestamptz for example). partprune.c ignored this possibility and would perform plan-time pruning with them anyway, possibly leading to wrong answers if the environment changed between planning and execution. To fix, teach gen_partprune_steps() to do things differently when creating plan-time pruning steps vs. run-time pruning steps. analyze_partkey_exprs() also needs an extra check, which is rather annoying but now is not the time to restructure things enough to avoid that. While at it, simplify the logic for the plan-time case a little by insisting that the comparison value be a Const and nothing else. This relies on the assumption that eval_const_expressions will have reduced any immutable expression to a Const; which is not quite 100% true, but certainly any case that comes up often enough to be interesting should have simplification logic there. Also improve a bunch of inadequate/obsolete/wrong comments. Per discussion of a report from Alan Jackson (though this fixes only one aspect of that problem). Back-patch to v11 where this code came in. David Rowley, with some further hacking by me Discussion: https://postgr.es/m/FAD28A83-AC73-489E-A058-2681FA31D648@tvsquared.com
2019-05-15Remove obsolete nbtree insertion comment.Peter Geoghegan
Remove a Berkeley-era comment above _bt_insertonpg() that admonishes the reader to grok Lehman and Yao's paper before making any changes. This made a certain amount of sense back when _bt_insertonpg() was responsible for most of the things that are now spread across _bt_insertonpg(), _bt_findinsertloc(), _bt_insert_parent(), and _bt_split(), but it doesn't work like that anymore. I believe that this comment alludes to the need to "couple" or "crab" buffer locks as we ascend the tree as page splits cascade upwards. The nbtree README already explains this in detail, which seems sufficient. Besides, the changes to page splits made by commit 40dae7ec537 altered the exact details of how buffer locks are retained during splits; Lehman and Yao's original algorithm seems to release the lock on the left child page/buffer slightly earlier than _bt_insertonpg()/_bt_insert_parent() can.