summaryrefslogtreecommitdiff
path: root/src/include
AgeCommit message (Collapse)Author
2018-06-19Track new configure flags introduced for version 11 in pg_config.h.win32Michael Paquier
The following set of flags mainly matter when building Postgres code with MSVC and those have been forgotten with latest developments: - HAVE_LDAP_INITIALIZE, added by 35c0754f, and marked as disabled. ldap_initialize() is a non-standard extension that provides a way to use "ldaps" with OpenLDAP, but it is not supported on Windows, and instead the non-standard ldap_sslinit() is used if WIN32 is defined. Per input from Thomas Munro. - HAVE_X509_GET_SIGNATURE_NID, added by 054e8c6c, which is used by SCRAM's channel binding tls-server-end-point. Having this flag disabled would cause this channel binding type to be unsupported for Windows builds. - HAVE_SSL_CLEAR_OPTIONS, added recently as of a364dfa4 to disable SSL compression. - HAVE_ASN1_STRING_GET0_DATA, added by 5c6df67, which is used to track a new compatibility with OpenSSL 1.1.0. This was missing from pg_config.win32.h and is not enabled by default. HAVE_BIO_GET_DATA, HAVE_OPENSSL_INIT_SSL and HAVE_BIO_METH_NEW gain the same treatment. The second and third flags are enabled with this commit, which raises the bar of OpenSSL support to 1.0.2 on Windows as a minimum. As this is the LTS (long-time support) version of OpenSSL community and knowing that all recent installers referred by OpenSSL upstream don't have anymore 1.0.1 or older, we could live with that requirement. In order to allow the code to compile with OpenSSL 1.1.0, all the flags mentioned above need to be enabled in pg_config.h.win32. Author: Michael Paquier Reviewed-by: Andrew Dunstan Discussion: https://postgr.es/m/20180529211559.GF6632@paquier.xyz
2018-06-16Remove AELs from subxids correctly on standbySimon Riggs
Issues relate only to subtransactions that hold AccessExclusiveLocks when replayed on standby. Prior to PG10, aborting subtransactions that held an AccessExclusiveLock failed to release the lock until top level commit or abort. 49bff5300d527 fixed that. However, 49bff5300d527 also introduced a similar bug where subtransaction commit would fail to release an AccessExclusiveLock, leaving the lock to be removed sometimes early and sometimes late. This commit fixes that bug also. Backpatch to PG10 needed. Tested by observation. Note need for multi-node isolationtester to improve test coverage for this and other HS cases. Reported-by: Simon Riggs Author: Simon Riggs
2018-06-13Fix some ill-chosen names for globally-visible partition support functions.Tom Lane
"compute_hash_value" is particularly gratuitously generic, but IMO all of these ought to have names clearly related to partitioning.
2018-06-13Fix up run-time partition pruning's use of relcache's partition data.Tom Lane
The previous coding saved pointers into the partitioned table's relcache entry, but then closed the relcache entry, causing those pointers to nominally become dangling. Actual trouble would be seen in the field only if a relcache flush occurred mid-query, but that's hardly out of the question. While we could fix this by copying all the data in question at query start, it seems better to just hold the relcache entry open for the whole query. While at it, improve the handling of support-function lookups: do that once per query not once per pruning test. There's still something to be desired here, in that we fail to exploit the possibility of caching data across queries in the fn_extra fields of the relcache's FmgrInfo structs, which could happen if we just used those structs in-place rather than copying them. However, combining that with the possibility of per-query lookups of cross-type comparison functions seems to require changes in the APIs of a lot of the pruning support functions, so it's too invasive to consider as part of this patch. A win would ensue only for complex partition key data types (e.g. arrays), so it may not be worth the trouble. David Rowley and Tom Lane Discussion: https://postgr.es/m/17850.1528755844@sss.pgh.pa.us
2018-06-12Fix bugs in vacuum of shared rels, by keeping their relcache entries current.Andres Freund
When vacuum processes a relation it uses the corresponding relcache entry's relfrozenxid / relminmxid as a cutoff for when to remove tuples etc. Unfortunately for nailed relations (i.e. critical system catalogs) bugs could frequently lead to the corresponding relcache entry being stale. This set of bugs could cause actual data corruption as vacuum would potentially not remove the correct row versions, potentially reviving them at a later point. After 699bf7d05c some corruptions in this vein were prevented, but the additional error checks could also trigger spuriously. Examples of such errors are: ERROR: found xmin ... from before relfrozenxid ... and ERROR: found multixact ... from before relminmxid ... To be caused by this bug the errors have to occur on system catalog tables. The two bugs are: 1) Invalidations for nailed relations were ignored, based on the theory that the relcache entry for such tables doesn't change. Which is largely true, except for fields like relfrozenxid etc. This means that changes to relations vacuumed in other sessions weren't picked up by already existing sessions. Luckily autovacuum doesn't have particularly longrunning sessions. 2) For shared *and* nailed relations, the shared relcache init file was never invalidated while running. That means that for such tables (e.g. pg_authid, pg_database) it's not just already existing sessions that are affected, but even new connections are as well. That explains why the reports usually were about pg_authid et. al. To fix 1), revalidate the rd_rel portion of a relcache entry when invalid. This implies a bit of extra complexity to deal with bootstrapping, but it's not too bad. The fix for 2) is simpler, simply always remove both the shared and local init files. Author: Andres Freund Reviewed-By: Alvaro Herrera Discussion: https://postgr.es/m/20180525203736.crkbg36muzxrjj5e@alap3.anarazel.de https://postgr.es/m/CAMa1XUhKSJd98JW4o9StWPrfS=11bPgG+_GDMxe25TvUY4Sugg@mail.gmail.com https://postgr.es/m/CAKMFJucqbuoDRfxPDX39WhA3vJyxweRg_zDVXzncr6+5wOguWA@mail.gmail.com https://postgr.es/m/CAGewt-ujGpMLQ09gXcUFMZaZsGJC98VXHEFbF-tpPB0fB13K+A@mail.gmail.com Backpatch: 9.3-
2018-06-11Improve commentary about run-time partition pruning data structures.Tom Lane
No code changes except for a couple of new Asserts. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAKJS1f-6GODRNgEtdPxCnAPme2h2hTztB6LmtfdmcYAAOE0kQg@mail.gmail.com
2018-06-11Don't needlessly check the partition contraint twiceAlvaro Herrera
Starting with commit f0e44751d717, ExecConstraints was in charge of running the partition constraint; commit 19c47e7c8202 modified that so that caller could request to skip that checking depending on some conditions, but that commit and 15ce775faa42 together introduced a small bug there which caused ExecInsert to request skipping the constraint check but have this not be honored -- in effect doing the check twice. This could have been fixed in a very small patch, but on further analysis of the involved function and its callsites, it turns out to be simpler to give the responsibility of checking the partition constraint fully to the caller, and return ExecConstraints to its original (pre-partitioning) shape where it only checked tuple descriptor-related constraints. Each caller must do partition constraint checking on its own schedule, which is more convenient after commit 2f178441044 anyway. Reported-by: David Rowley Author: David Rowley, Álvaro Herrera Reviewed-by: Amit Langote, Amit Khandekar, Simon Riggs Discussion: https://postgr.es/m/CAKJS1f8w8+awsxgea8wt7_UX8qzOQ=Tm1LD+U1fHqBAkXxkW2w@mail.gmail.com
2018-06-10Assorted cosmetic cleanup of run-time-partition-pruning code.Tom Lane
Use "subplan" rather than "subnode" to refer to the child plans of a partitioning Append; this seems a bit more specific and hence clearer. Improve assorted comments. No non-cosmetic changes. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10Relocate partition pruning structs to a saner place.Tom Lane
These struct definitions were originally dropped into primnodes.h, which is a poor choice since that's mainly intended for primitive expression node types; these are not in that category. What they are is auxiliary info in Plan trees, so move them to plannodes.h. For consistency, also relocate some related code that was apparently placed with the aid of a dartboard. There's no interesting code changes in this commit, just reshuffling. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10Improve run-time partition pruning to handle any stable expression.Tom Lane
The initial coding of the run-time-pruning feature only coped with cases where the partition key(s) are compared to Params. That is a bit silly; we can allow it to work with any non-Var-containing stable expression, as long as we take special care with expressions containing PARAM_EXEC Params. The code is hardly any longer this way, and it's considerably clearer (IMO at least). Per gripe from Pavel Stehule. David Rowley, whacked around a bit by me Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10Fix and document lock handling for in-memory replication slot dataMichael Paquier
While debugging issues on HEAD for the new slot forwarding feature of Postgres 11, some monitoring of the code surrounding in-memory slot data has proved that the lock handling may cause inconsistent data to be read by read-only callers of slot functions, particularly pg_get_replication_slots() which fetches data for the system view pg_replication_slots, or modules looking directly at slot information. The code paths involved in those problems concern logical decoding initialization (down to 9.4) and WAL reservation for slots (new as of 10). A set of comments documenting all the lock handlings, particularly the dependency with LW locks for slots and the in_use flag as well as the internal mutex lock is added, based on a suggested by Simon Riggs. Some of the fixed code exists down to 9.4 where WAL decoding has been introduced, but as those race conditions are really unlikely going to happen as those concern code paths for slot and decoding creation, just fix the problem on HEAD. Author: Michael Paquier Discussion: https://postgr.es/m/20180528085747.GA27845@paquier.xyz
2018-06-08Teach SHOW ALL to honor pg_read_all_settings membershipAlvaro Herrera
Also, fix the pg_settings view to display source filename and line number when invoked by a pg_read_all_settings member. This addition by me (Álvaro). Also, fix wording of the comment in GetConfigOption regarding the restriction it implements, renaming the parameter for extra clarity. Noted by Michaël. These were all oversight in commit 25fff40798fc; backpatch to pg10, where that commit first appeared. Author: Laurenz Albe Reviewed-by: Michaël Paquier, Álvaro Herrera Discussion: https://postgr.es/m/1519917758.6586.8.camel@cybertec.at
2018-05-30Move _bt_upgrademetapage() into critical section.Teodor Sigaev
Any changes on page should be done in critical section, so move _bt_upgrademetapage into critical section. Improve comment. Found by Amit Kapila during post-commit review of 857f9c36. Author: Amit Kapila
2018-05-27Don't fall off the end of perl functionsAndrew Dunstan
This complies with the perlcritic policy Subroutines::RequireFinalReturn, which is a severity 4 policy. Since we only currently check at severity level 5, the policy is raised to that level until we move to level 4 or lower, so that any new infringements will be caught. A small cosmetic piece of tidying of the pgperlcritic script is included. Mike Blackwell Discussion: https://postgr.es/m/CAESHdJpfFm_9wQnQ3koY3c91FoRQsO-fh02za9R3OEMndOn84A@mail.gmail.com
2018-05-26Update a couple of long-obsolete comments in pg_type.h.Tom Lane
2018-05-23Remove configure's check for nonstandard "long long" printf modifiers.Tom Lane
We used to claim to support platforms using 'q' or 'I64' as the printf length modifier for long long int, by dint of replacing snprintf with our own code which uses the C99 standard 'll' modifier. But that is only adequate if we use INT64_MODIFIER only in snprintf-based calls, not directly with the platform's native printf or fprintf. Which hasn't been the case for years. We had not noticed, partially because of inadequate test coverage, and partially because the buildfarm is almost completely bare of machines that won't take 'll'. The last one seems to have been frogmouth, which was adjusted recently so that it will take 'll'. We might as well just give up on the pretense that anything else works, and save ourselves some configure cycles. Discussion: https://postgr.es/m/13103.1526749980@sss.pgh.pa.us Discussion: https://postgr.es/m/24769.1526772680@sss.pgh.pa.us
2018-05-23Fix incorrect ordering of operations in pg_resetwal and pg_rewind.Tom Lane
Commit c37b3d08c dropped its added GetDataDirectoryCreatePerm call into the wrong place in pg_resetwal.c, namely after the chdir to DataDir. That broke invocations using a relative path, as reported by Tushar Ahuja. We could have left it where it was and changed the argument to be ".", but that'd result in a rather confusing error message in event of a failure, so re-ordering seems like a better solution. Similarly reorder operations in pg_rewind.c. The issue there is that it doesn't seem like a good idea to do any actual operations before the not-root check (on Unix) or the restricted token acquisition (on Windows). I don't know that this is an actual bug, but I'm definitely not convinced that it isn't, either. Assorted other code review for c37b3d08c and da9b580d8: fix some misspelled or otherwise badly worded comments, put the #include for <sys/stat.h> where it actually belongs, etc. Discussion: https://postgr.es/m/aeb9c3a7-3c3f-a57f-1a18-c8d4fcdc2a1f@enterprisedb.com
2018-05-21Stamp 11beta1.REL_11_BETA1Tom Lane
2018-05-21Improve spelling of new FINALFUNC_MODIFY aggregate attribute.Tom Lane
I'd used SHARABLE as a value originally, but Peter Eisentraut points out that dictionaries agree that SHAREABLE is the preferred spelling. Run around and change that before it's too late. Discussion: https://postgr.es/m/d2e1afd4-659c-50d6-1b20-7cfd3675e909@2ndquadrant.com
2018-05-19Assorted minor cleanups for bootstrap-data Perl scripts.Tom Lane
FindDefinedSymbol was intended to take an array of possible include paths, but it never actually worked correctly for any but the first array element. Since there's no use-case for more than one path anyway, let's just simplify this code and its callers by redefining it as taking only one include path. Minor other code-beautification without functional effects, except that in one place we format the output as pgindent would do. John Naylor Discussion: https://postgr.es/m/CAJVSVGXM_n32hTTkircW4_K1LQFsJNb6xjs0pAP4QC0ZpyJfPQ@mail.gmail.com
2018-05-19Support platforms where strtoll/strtoull are spelled __strtoll/__strtoull.Tom Lane
Ancient HPUX, for one, does this. We hadn't noticed due to the lack of regression tests that required a working strtoll. (I was slightly tempted to remove the other historical spelling, strto[u]q, since it seems we have no buildfarm members testing that case. But I refrained.) Discussion: https://postgr.es/m/151935568942.1461.14623890240535309745@wrigleys.postgresql.org
2018-05-18Arrange to supply declarations for strtoll/strtoull if needed.Tom Lane
Buildfarm member dromedary is still unhappy about the recently-added ecpg "long long" tests. The reason turns out to be that it includes "-ansi" in its CFLAGS, and in their infinite wisdom Apple have decided to hide the declarations of strtoll/strtoull in C89-compliant builds. (I find it pretty curious that they hide those function declarations when you can nonetheless declare a "long long" variable, but anyway that is their behavior, both on dromedary's obsolete macOS version and the newest and shiniest.) As a result, gcc assumes these functions return "int", leading naturally to wrong results. (Looking at dromedary's past build results, it's evident that this problem also breaks pg_strtouint64() on 32-bit platforms; but we evidently have no regression tests that exercise that function with values above 32 bits.) To fix, supply declarations for these functions when the platform provides the functions but not the declarations, using the same type of mechanism as we use for some other similar cases. Discussion: https://postgr.es/m/151935568942.1461.14623890240535309745@wrigleys.postgresql.org
2018-05-18Recognize that MSVC can support strtoll() and strtoull().Tom Lane
This is needed for full support of "long long" variables in ecpg, but the previous patch for bug #15080 (commits 51057feaa et al) missed it. In MSVC versions where the functions don't exist under those names, we can nonetheless use _strtoi64() and _strtoui64(). Like the previous patch, back-patch all the way. Dang Minh Huong Discussion: https://postgr.es/m/151935568942.1461.14623890240535309745@wrigleys.postgresql.org
2018-05-16Detoast plpgsql variables if they might live across a transaction boundary.Tom Lane
Up to now, it's been safe for plpgsql to store TOAST pointers in its variables because the ActiveSnapshot for whatever query called the plpgsql function will surely protect such TOAST values from being vacuumed away, even if the owning table rows are committed dead. With the introduction of procedures, that assumption is no longer good in "non atomic" executions of plpgsql code. We adopt the slightly brute-force solution of detoasting all TOAST pointers at the time they are stored into variables, if we're in a non-atomic context, just in case the owning row goes away. Some care is needed to avoid long-term memory leaks, since plpgsql tends to run with CurrentMemoryContext pointing to its call-lifespan context, but we shouldn't assume that no memory is leaked by heap_tuple_fetch_attr. In plpgsql proper, we can do the detoasting work in the "eval_mcontext". Most of the code thrashing here is due to the need to add this capability to expandedrecord.c as well as plpgsql proper. In expandedrecord.c, we can't assume that the caller's context is short-lived, so make use of the short-term sub-context that was already invented for checking domain constraints. In view of this repurposing, it seems good to rename that variable and associated code from "domain_check_cxt" to "short_term_cxt". Peter Eisentraut and Tom Lane Discussion: https://postgr.es/m/5AC06865.9050005@anastigmatix.net
2018-05-09Make gen_partprune_steps staticAlvaro Herrera
There's no need to export this function, so don't. Michaël didn't actually write the patch, but we list him as first author because with a trivial one like this, intellectual authorship is as important (if not more) as bit shovelling. Author: Michaël Paquier, Amit Langote Discussion: https://postgr.es/m/c91299c4-199b-0f16-339b-a29d6d2a39ee@lab.ntt.co.jp
2018-05-05Fix scenario where streaming standby gets stuck at a continuation record.Heikki Linnakangas
If a continuation record is split so that its first half has already been removed from the master, and is only present in pg_wal, and there is a recycled WAL segment in the standby server that looks like it would contain the second half, recovery would get stuck. The code in XLogPageRead() incorrectly started streaming at the beginning of the WAL record, even if we had already read the first page. Backpatch to 9.4. In principle, older versions have the same problem, but without replication slots, there was no straightforward mechanism to prevent the master from recycling old WAL that was still needed by standby. Without such a mechanism, I think it's reasonable to assume that there's enough slack in how many old segments are kept around to not run into this, or you have a WAL archive. Reported by Jonathon Nelson. Analysis and patch by Kyotaro HORIGUCHI, with some extra comments by me. Discussion: https://www.postgresql.org/message-id/CACJqAM3xVz0JY1XFDKPP%2BJoJAjoGx%3DGNuOAshEDWCext7BFvCQ%40mail.gmail.com
2018-05-04Re-think predicate locking on GIN indexes.Teodor Sigaev
The principle behind the locking was not very well thought-out, and not documented. Add a section in the README to explain how it's supposed to work, and change the code so that it actually works that way. This fixes two bugs: 1. If fast update was turned on concurrently, subsequent inserts to the pending list would not conflict with predicate locks that were acquired earlier, on entry pages. The included 'predicate-gin-fastupdate' test demonstrates that. To fix, make all scans acquire a predicate lock on the metapage. That lock represents a scan of the pending list, whether or not there is a pending list at the moment. Forget about the optimization to skip locking/checking for locks, when fastupdate=off. 2. If a scan finds no match, it still needs to lock the entry page. The point of predicate locks is to lock the gabs between values, whether or not there is a match. The included 'predicate-gin-nomatch' test tests that case. In addition to those two bug fixes, this removes some unnecessary locking, following the principle laid out in the README. Because all items in a posting tree have the same key value, a lock on the posting tree root is enough to cover all the items. (With a very large posting tree, it would possibly be better to lock the posting tree leaf pages instead, so that a "skip scan" with a query like "A & B", you could avoid unnecessary conflict if a new tuple is inserted with A but !B. But let's keep this simple.) Also, some spelling fixes. Author: Heikki Linnakangas with some editorization by me Review: Andrey Borodin, Alexander Korotkov Discussion: https://www.postgresql.org/message-id/0b3ad2c2-2692-62a9-3a04-5724f2af9114@iki.fi
2018-05-03Rearrange makefile rules for running Gen_fmgrtab.pl.Tom Lane
Make these rules look more like the ones associated with genbki.pl, to wit: * Use a stamp file to record when we last ran the script, instead of relying on the timestamps of the individual output files. * Take the knowledge out of backend/Makefile and put it in utils/Makefile where it belongs. I moved down the handling of errcodes.h and probes.h too, although those continue to be built by separate processes. In itself, this is just much-needed cleanup with little practical effect. However, by decoupling these makefile rules from the timestamps of the generated header files, we open the door to not advancing those timestamps unnecessarily, which will be taken advantage of by the next commit. msvc/Solution.pm should be taught to do things similarly, but I'll leave that for another commit. Discussion: https://postgr.es/m/16925.1525376229@sss.pgh.pa.us
2018-05-03Fix SPI error cleanup and memory leakPeter Eisentraut
Since the SPI stack has been moved from TopTransactionContext to TopMemoryContext, setting _SPI_stack to NULL in AtEOXact_SPI() leaks memory. In fact, we don't need to do that anymore: We just leave the allocated stack around for the next SPI use. Also, refactor the SPI cleanup so that it is run both at transaction end and when returning to the main loop on an exception. The latter is necessary when a procedure calls a COMMIT or ROLLBACK command that itself causes an error.
2018-05-02Improve our method for probing the availability of ARM CRC instructions.Tom Lane
Instead of depending on glibc's getauxval() function, just try to execute the CRC code, and trap SIGILL if that happens. Thomas Munro Discussion: https://postgr.es/m/HE1PR0801MB1323D171938EABC04FFE7FA9E3110@HE1PR0801MB1323.eurprd08.prod.outlook.com
2018-05-02Fix some sloppiness in the new BufFileSize() and BufFileAppend() functions.Heikki Linnakangas
There were three related issues: * BufFileAppend() incorrectly reset the seek position on the 'source' file. As a result, if you had called BufFileRead() on the file before calling BufFileAppend(), it got confused, and subsequent calls would read/write at wrong position. * BufFileSize() did not work with files opened with BufFileOpenShared(). * FileGetSize() only worked on temporary files. To fix, change the way BufFileSize() works so that it works on shared files. Remove FileGetSize() altogether, as it's no longer needed. Remove buffilesize from TapeShare struct, as the leader process can simply call BufFileSize() to get the tape's size, there's no need to pass it through shared memory anymore. Discussion: https://www.postgresql.org/message-id/CAH2-WznEDYe_NZXxmnOfsoV54oFkTdMy7YLE2NPBLuttO96vTQ@mail.gmail.com
2018-05-02Change SIZEOF_BOOL to 1 for Windows.Tom Lane
For some reason it was previously defined as 0, which is silly. The only effect was to disable use of <stdbool.h>, which commit b2328bf62 intended to make possible. Thomas Munro Discussion: https://postgr.es/m/CAEepm%3D3%3DTDYEXUEcHpEx%2BTwc31wo7PA0oBAiNt6sWmq93MW02A%40mail.gmail.com
2018-05-01Fix some assorted compiler warnings on Windows.Tom Lane
Don't overflow the result type of constant expressions. Don't negate unsigned types. Define HAVE_STDBOOL_H for Visual C++ 2013 and later. Thomas Munro Reviewed-By: Michael Paquier and Tom Lane Discussion: https://postgr.es/m/CAEepm%3D3%3DTDYEXUEcHpEx%2BTwc31wo7PA0oBAiNt6sWmq93MW02A%40mail.gmail.com
2018-05-01Improve representation of 'moved partitions' indicator on deleted tuples.Andres Freund
Previously a tuple that has been moved to a different partition (see f16241bef7c), set the block number on the old tuple to an invalid value to indicate that fact. But the tuple offset was left untouched. That turned out to trigger a wal_consistency_checking failure as reported by Peter Geoghegan, as the offset wasn't always overwritten during WAL replay. Heikki observed that we're wasting valuable data by not putting information also in the offset. Thus set that to MovedPartitionsOffsetNumber when a tuple indicates it has moved. We continue to set the block number to MovedPartitionsBlockNumber, as that seems more likely to cause problems for code not updated to know about moved tuples. As t_ctid's offset number is now always set, this refinement also fixes the wal_consistency_checking issue. This technically is a minor disk format break, with previously created moved tuples not being recognized anymore. But since there not even has been a beta release since f16241bef7c... Reported-By: Peter Geoghegan Author: Heikki Linnakangas, Amul Sul Discussion: https://postgr.es/m/CAH2-Wzm9ty+1BX7-GMNJ=xPRg67oJTVeDNdA9LSyJJtMgRiCMA@mail.gmail.com
2018-04-29Cosmetic improvement: use BKI_DEFAULT and BKI_LOOKUP in pg_language.Tom Lane
The point of this is not really to remove redundancy in pg_language.dat; with only three entries, it's hardly worth it. Rather, it is to get to a point where there are exactly zero hard-coded numeric pg_proc OID references in the catalog .dat files. The lanvalidator column was the only remaining location of such references, and it seems like a good thing for future-proofing reasons to make it not be a special case. There are still a few places in the .dat files with numeric OID references to other catalogs, but after review I don't see any that seem worth changing at present. In each case there are just too few entries to make it worth the trouble to create lookup infrastructure. This doesn't change the emitted postgres.bki file, so no catversion bump.
2018-04-28In AtEOXact_Files, complain if any files remain unclosed at commit.Tom Lane
This change makes this module act more like most of our other low-level resource management modules. It's a caller error if something is not explicitly closed by the end of a successful transaction, so issue a WARNING about it. This would not actually have caught the file leak bug fixed in commit 231bcd080, because that was in a transaction-abort path; but it still seems like a good, and pretty cheap, cross-check. Discussion: https://postgr.es/m/152056616579.4966.583293218357089052@wrigleys.postgresql.org
2018-04-28Tweak reformat_dat_file.pl to make it more easily hand-invokable.Tom Lane
Use the same code we already applied in duplicate_oids and unused_oids to let this script find Catalog.pm without help. This removes the need to supply a -I switch in most cases. Also, mark the script executable, again to follow the precedent of duplicate_oids and unused_oids. Now you can just do "./reformat_dat_file.pl pg_proc.dat" if you want to reformat only one or a few .dat files rather than all. It'd be possible to remove the -I switches in the Makefile's convenience targets, but I chose to leave them: they don't hurt anything, and it's possible that in weird VPATH situations they might be of value.
2018-04-28Clarify handling of special-case values in bootstrap catalog data.Tom Lane
I (tgl) originally coded the special case for pg_proc.pronargs as though it were a kind of default value. It seems better though to treat computable columns as an independent concern: this makes the code clearer, and probably a bit faster too since we needn't do work inside the per-column loop. Improve related comments, as well, in the expectation that there might be more cases like this in future. John Naylor, some additional comment-hacking by me Discussion: https://postgr.es/m/CAJVSVGW-D7OobzU=dybVT2JqZAx-4X1yvBJdavBmqQL05Q6CLw@mail.gmail.com
2018-04-28Assorted minor doc/comment fixes.Tom Lane
Identify pg_replication_origin as a shared catalog in catalogs.sgml, using the same boilerplate wording used for most other shared catalogs (and tweak another place where someone had randomly deviated from that boilerplate). Make an example in mmgr/README more consistent with surrounding text. Update an obsolete cross-reference in a comment in storage/block.h. Zhuo Ql Discussion: https://postgr.es/m/44296255.1819230.1524889719001@mail.yahoo.com
2018-04-26C comment: add description of root_tuple_slotBruce Momjian
Reported-by: Amit Langote Discussion: https://postgr.es/m/d2e6674c-c5c6-fe89-1d0b-3534b9db0476@lab.ntt.co.jp
2018-04-26Post-feature-freeze pgindent run.Tom Lane
Discussion: https://postgr.es/m/15719.1523984266@sss.pgh.pa.us
2018-04-26Preliminary work for pgindent run.Tom Lane
Update typedefs.list from current buildfarm results. Adjust pgindent's typedef blacklist to block some more unfortunate typedef names that have snuck in since last time. Manually tweak a few places where I didn't like the initial results of pgindent'ing.
2018-04-26Fix duplicate_oids and unused_oids so user needn't cd to catalog dir.Tom Lane
Previously, you had to cd into src/include/catalog before running either of these scripts. That's a bit tedious, so let's make the scripts do it for you. In passing, improve the initial comments in both scripts. Also remove unused_oids' code to complain about duplicate oids. That was added in yesterday's commit 5602265f7, but on second thought we shouldn't be randomly redefining the script's behavior that way. John Naylor and Tom Lane Discussion: https://postgr.es/m/37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru
2018-04-25Convert unused_oids and duplicate_oids to use Catalog.pm infrastructure.Tom Lane
unused_oids was previously a shell script, which of course didn't work at all on Windows. Also, commit 372728b0d introduced some other portability problems, as complained of by Stas Kelvich. We can improve matters by converting it to Perl. While we're at it, let's future-proof both this script and duplicate_oids to use Catalog.pm rather than having a bunch of ad-hoc logic for parsing catalog headers and .dat files. These scripts are thereby a bit slower, which doesn't seem like a problem for typical manual use. It is a little annoying for buildfarm purposes, but we should be able to fix that case by having genbki.pl make the check instead of parsing the headers twice. (That's not done in this commit, though.) Stas Kelvich, adjusted a bit by me Discussion: https://postgr.es/m/37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru
2018-04-25Make Catalog.pm's representation of toast and index decls more abstract.Tom Lane
Instead of immediately constructing the string we need to emit into the .BKI file, preserve the items we extracted from the header file in a hash. This eases using the info for other purposes. John Naylor (with cosmetic adjustments by me) Discussion: https://postgr.es/m/37D774E4-FE1F-437E-B3D2-593F314B7505@postgrespro.ru
2018-04-25Reindent Perl files with perltidy version 20170521.Tom Lane
Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-24Initialize ExprStates once in run-time partition pruningAlvaro Herrera
Instead of doing ExecInitExpr every time a Param needs to be evaluated in run-time partition pruning, do it once during run-time pruning set-up and cache the exprstate in PartitionPruneContext, saving a lot of work. Author: David Rowley Reviewed-by: Amit Langote, Álvaro Herrera Discussion: https://postgr.es/m/CAKJS1f8-x+q-90QAPDu_okhQBV4DPEtPz8CJ=m0940GyT4DA4w@mail.gmail.com
2018-04-23Add GUC enable_partition_pruningAlvaro Herrera
This controls both plan-time and execution-time new-style partition pruning. While finer-grain control is possible (maybe using an enum GUC instead of boolean), there doesn't seem to be much need for that. This new parameter controls partition pruning for all queries: trivially, SELECT queries that affect partitioned tables are naturally under its control since they are using the new technology. However, while UPDATE/DELETE queries do not use the new code, we make the new GUC control their behavior also (stealing control from constraint_exclusion), because it is more natural, and it leads to a more natural transition to the future in which those queries will also use the new pruning code. Constraint exclusion still controls pruning for regular inheritance situations (those not involving partitioned tables). Author: David Rowley Review: Amit Langote, Ashutosh Bapat, Justin Pryzby, David G. Johnston Discussion: https://postgr.es/m/CAKJS1f_0HwsxJG9m+nzU+CizxSdGtfe6iF_ykPYBiYft302DCw@mail.gmail.com
2018-04-23Fix wrong validation of top-parent pointer during page deletion in Btree.Teodor Sigaev
After introducing usage of t_tid of inner or page high key for storing number of attributes of tuple, validation of tuple's ItemPointer with ItemPointerIsValid becomes incorrect, it's need to validate only blocknumber of ItemPointer. Missing this causes a incorrect page deletion, fix that. Test is added. BTW, current contrib/amcheck doesn't fail on index corrupted by this way. Also introduce BTreeTupleGetTopParent/BTreeTupleSetTopParent macroses to improve code readability and to avoid possible confusion with page high key: high key is used to store top-parent link for branch to remove. Bug found by Michael Paquier, but bug doesn't exist in previous versions because t_tid was set to P_HIKEY. Author: Teodor Sigaev Reviewer: Peter Geoghegan Discussion: https://www.postgresql.org/message-id/flat/20180419052436.GA16000%40paquier.xyz
2018-04-23Make PGJIT_* macros safer.Heikki Linnakangas
Author: David Rowley Discussion: https://www.postgresql.org/message-id/CAKJS1f8Ge2y0sDs6RQEJFH-vjb-bWhs86rCX4Fp4FZ+TmxtRkw@mail.gmail.com