summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2020-12-25pg_alterckey: adjust doc build and Win32 sleep/open build failsBruce Momjian
Fix for commit 62afb42a7f. Reported-by: Tom Lane Discussion: https://postgr.es/m/1252111.1608953815@sss.pgh.pa.us Backpatch-through: master
2020-12-25Fix function call typo in frontend Win32 code, commit 978f869b99Bruce Momjian
Reported-by: buildfarm member walleye Backpatch-through: master
2020-12-25Allow ssl_passphrase_command to prompt the terminalBruce Momjian
Previously the command could not access the terminal for a passphrase. Backpatch-through: master
2020-12-25Add pg_alterckey utility to change the cluster keyBruce Momjian
This can change the key that encrypts the data encryption keys used for cluster file encryption. Discussion: https://postgr.es/m/20201202213814.GG20285@momjian.us Backpatch-through: master
2020-12-25remove missing reference to crypto test from patch 978f869b99Bruce Momjian
Reported-by: Tom Lane Discussion: https://postgr.es/m/1205031.1608925990@sss.pgh.pa.us Backpatch-through: master
2020-12-25Really fix the dummy implementations in cipher.c.Tom Lane
945083b2f wasn't enough to silence compiler warnings.
2020-12-25fix no-return function call in cipher.c from commit 978f869b99Bruce Momjian
Reported-by: buildfarm member sifaka Backpatch-through: master
2020-12-25fixes docs and missing initdb help option for commit 978f869b99Bruce Momjian
Reported-by: Erik Rijkers Discussion: https://postgr.es/m/a27e7bb60fc4c4a1fe960f7b055ba822@xs4all.nl Backpatch-through: master
2020-12-25Invalidate acl.c caches when pg_authid changes.Noah Misch
This makes existing sessions reflect "ALTER ROLE ... [NO]INHERIT" as quickly as they have been reflecting "GRANT role_name". Back-patch to 9.5 (all supported versions). Reviewed by Nathan Bossart. Discussion: https://postgr.es/m/20201221095028.GB3777719@rfd.leadboat.com
2020-12-25remove uint128 requirement from patch 978f869b99 (CFE)Bruce Momjian
Used char[16] instead. Reported-by: buildfarm member florican Backpatch-through: master
2020-12-25Fix return value and const declaration from commit 978f869b99Bruce Momjian
This fixes the non-OpenSSL compile case. Reported-by: buildfarm member sifaka Backpatch-through: master
2020-12-25Add key management systemBruce Momjian
This adds a key management system that stores (currently) two data encryption keys of length 128, 192, or 256 bits. The data keys are AES256 encrypted using a key encryption key, and validated via GCM cipher mode. A command to obtain the key encryption key must be specified at initdb time, and will be run at every database server start. New parameters allow a file descriptor open to the terminal to be passed. pg_upgrade support has also been added. Discussion: https://postgr.es/m/CA+fd4k7q5o6Nc_AaX6BcYM9yqTbC6_pnH-6nSD=54Zp6NBQTCQ@mail.gmail.com Discussion: https://postgr.es/m/20201202213814.GG20285@momjian.us Author: Masahiko Sawada, me, Stephen Frost
2020-12-24Avoid time-of-day-dependent failure in log rotation test.Tom Lane
Buildfarm members pogona and petalura have shown a failure when pg_ctl/t/004_logrotate.pl starts just before local midnight. The default rotate-at-midnight behavior occurs just before the Perl script examines current_logfiles, so it figures that the rotation it's already requested has occurred ... but in reality, that rotation happens just after it looks, so the expected new log data goes into a different file than the one it's examining. In HEAD, src/test/kerberos/t/001_auth.pl has acquired similar code that evidently has a related failure mode. Besides being quite new, few buildfarm critters run that test, so it's unsurprising that we've not yet seen a failure there. Fix both cases by setting log_rotation_age = 0 so that no time-based rotation can occur. Also absorb 004_logrotate.pl's decision to set lc_messages = 'C' into the kerberos test, in hopes that it will work in non-English prevailing locales. Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=pogona&dt=2020-12-24%2022%3A10%3A04 Report: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=petalura&dt=2020-02-01%2022%3A20%3A04
2020-12-24revert removal of hex_decode() from ecpg from commit c3826f831eBruce Momjian
ecpglib on certain platforms can't handle the pg_log_fatal calls from libraries. This was reported by the buildfarm. It needs a refactoring and return value change if it is later removed. Backpatch-through: master
2020-12-24move hex_decode() to /common so it can be called from frontendBruce Momjian
This allows removal of a copy of hex_decode() from ecpg, and will be used by the soon-to-be added pg_alterckey command. Backpatch-through: master
2020-12-24Fix race condition between shutdown and unstarted background workers.Tom Lane
If a database shutdown (smart or fast) is commanded between the time some process decides to request a new background worker and the time that the postmaster can launch that worker, then nothing happens because the postmaster won't launch any bgworkers once it's exited PM_RUN state. This is fine ... unless the requesting process is waiting for that worker to finish (or even for it to start); in that case the requestor is stuck, and only manual intervention will get us to the point of being able to shut down. To fix, cancel pending requests for workers when the postmaster sends shutdown (SIGTERM) signals, and similarly cancel any new requests that arrive after that point. (We can optimize things slightly by only doing the cancellation for workers that have waiters.) To fit within the existing bgworker APIs, the "cancel" is made to look like the worker was started and immediately stopped, causing deregistration of the bgworker entry. Waiting processes would have to deal with premature worker exit anyway, so this should introduce no bugs that weren't there before. We do have a side effect that registration records for restartable bgworkers might disappear when theoretically they should have remained in place; but since we're shutting down, that shouldn't matter. Back-patch to v10. There might be value in putting this into 9.6 as well, but the management of bgworkers is a bit different there (notably see 8ff518699) and I'm not convinced it's worth the effort to validate the patch for that branch. Discussion: https://postgr.es/m/661570.1608673226@sss.pgh.pa.us
2020-12-24Improve client error messages for immediate-stop situations.Tom Lane
Up to now, if the DBA issued "pg_ctl stop -m immediate", the message sent to clients was the same as for a crash-and-restart situation. This is confusing, not least because the message claims that the database will soon be up again, something we have no business predicting. Improve things so that we can generate distinct messages for the two cases (and also recognize an ad-hoc SIGQUIT, should somebody try that). To do that, add a field to pmsignal.c's shared memory data structure that the postmaster sets just before broadcasting SIGQUIT to its children. No interlocking seems to be necessary; the intervening signal-sending and signal-receipt should sufficiently serialize accesses to the field. Hence, this isn't any riskier than the existing usages of pmsignal.c. We might in future extend this idea to improve other postmaster-to-children signal scenarios, although none of them currently seem to be as badly overloaded as SIGQUIT. Discussion: https://postgr.es/m/559291.1608587013@sss.pgh.pa.us
2020-12-24Fix typos and grammar in docs and commentsMichael Paquier
This fixes several areas of the documentation and some comments in matters of style, grammar, or even format. Author: Justin Pryzby Discussion: https://postgr.es/m/20201222041153.GK30237@telsasoft.com
2020-12-23Fix portability issues with parsing of recovery_target_xidMichael Paquier
The parsing of this parameter has been using strtoul(), which is not portable across platforms. On most Unix platforms, unsigned long has a size of 64 bits, while on Windows it is 32 bits. It is common in recovery scenarios to rely on the output of txid_current() or even the newer pg_current_xact_id() to get a transaction ID for setting up recovery_target_xid. The value returned by those functions includes the epoch in the computed result, which would cause strtoul() to fail where unsigned long has a size of 32 bits once the epoch is incremented. WAL records and 2PC data include only information about 32-bit XIDs and it is not possible to have XIDs across more than one epoch, so discarding the high bits from the transaction ID set has no impact on recovery. On the contrary, the use of strtoul() prevents a consistent behavior across platforms depending on the size of unsigned long. This commit changes the parsing of recovery_target_xid to use pg_strtouint64() instead, available down to 9.6. There is one TAP test stressing recovery with recovery_target_xid, where a tweak based on pg_reset{xlog,wal} is added to bump the XID epoch so as this change gets tested, as per an idea from Alexander Lakhin. Reported-by: Alexander Lakhin Discussion: https://postgr.es/m/16780-107fd0c0385b1035@postgresql.org Backpatch-through: 9.6
2020-12-22Increase timeout in 021_row_visibility.pl.Tom Lane
Commit 7b28913bc figured 30 seconds is long enough for anybody, but in contexts like valgrind runs, it isn't necessarily.
2020-12-22Improve find_em_expr_usable_for_sorting_rel commentTomas Vondra
Clarify the relationship between find_em_expr_usable_for_sorting_rel and prepare_sort_from_pathkeys, i.e. what restrictions need to be shared between those two places. Author: James Coleman Reviewed-by: Tomas Vondra Backpatch-through: 13 Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
2020-12-21Don't search for volatile expr in find_em_expr_usable_for_sorting_relTomas Vondra
While prepare_sort_from_pathkeys has to be concerned about matching up a volatile expression to the proper tlist entry, we don't need to do that in find_em_expr_usable_for_sorting_rel becausee such a sort will have to be postponed anyway. Author: James Coleman Reviewed-by: Tomas Vondra Backpatch-through: 13 Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com
2020-12-21Disallow SRFs when considering sorts below Gather MergeTomas Vondra
While we do allow SRFs in ORDER BY, scan/join processing should not consider such cases - such sorts should only happen via final Sort atop a ProjectSet. So make sure we don't try adding such sorts below Gather Merge, just like we do for expressions that are volatile and/or not parallel safe. Backpatch to PostgreSQL 13, where this code was introduced as part of the Incremental Sort patch. Author: James Coleman Reviewed-by: Tomas Vondra Backpatch-through: 13 Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com Discussion: https://postgr.es/m/295524.1606246314%40sss.pgh.pa.us
2020-12-21Remove "invalid concatenation of jsonb objects" error case.Tom Lane
The jsonb || jsonb operator arbitrarily rejected certain combinations of scalar and non-scalar inputs, while being willing to concatenate other combinations. This was of course quite undocumented. Rather than trying to document it, let's just remove the restriction, creating a uniform rule that unless we are handling an object-to-object concatenation, non-array inputs are converted to one-element arrays, resulting in an array-to-array concatenation. (This does not change the behavior for any case that didn't throw an error before.) Per complaint from Joel Jacobson. Back-patch to all supported branches. Discussion: https://postgr.es/m/163099.1608312033@sss.pgh.pa.us
2020-12-21Check parallel safety in generate_useful_gather_pathsTomas Vondra
Commit ebb7ae839d ensured we ignore pathkeys with volatile expressions when considering adding a sort below a Gather Merge. Turns out we need to care about parallel safety of the pathkeys too, otherwise we might try sorting e.g. on results of a correlated subquery (as demonstrated by a report from Luis Roberto). Initial investigation by Tom Lane, patch by James Coleman. Backpatch to 13, where the code was instroduced (as part of Incremental Sort). Reported-by: Luis Roberto Author: James Coleman Reviewed-by: Tomas Vondra Backpatch-through: 13 Discussion: https://postgr.es/m/622580997.37108180.1604080457319.JavaMail.zimbra%40siscobra.com.br Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
2020-12-21Consider unsorted paths in generate_useful_gather_pathsTomas Vondra
generate_useful_gather_paths used to skip unsorted paths (without any pathkeys), but that is unnecessary - the later code actually can handle such paths just fine by adding a Sort node. This is clearly a thinko, preventing construction of useful plans. Backpatch to 13, where Incremental Sort was introduced. Author: James Coleman Reviewed-by: Tomas Vondra Backpatch-through: 13 Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs=hC0mSksZC=H5M8LSchj5e5OxpTAg@mail.gmail.com
2020-12-21Fix compiler warning in multirange_constructor0()Alexander Korotkov
Discussion: https://postgr.es/m/X%2BBP8XE0UpIB6Yvh%40paquier.xyz Author: Michael Paquier
2020-12-21Refactor logic to check for ASCII-only characters in stringMichael Paquier
The same logic was present for collation commands, SASLprep and pgcrypto, so this removes some code. Author: Michael Paquier Reviewed-by: Stephen Frost, Heikki Linnakangas Discussion: https://postgr.es/m/X9womIn6rne6Gud2@paquier.xyz
2020-12-21Fix typalign in rangetypes statisticsAlexander Korotkov
6df7a9698b introduces multirange types, whose typanalyze function shares infrastructure with range types typanalyze function. Since 6df7a9698b, information about type gathered by statistics is filled from typcache. But typalign is mistakenly always set to double. This commit fixes this oversight.
2020-12-20Avoid memcpy() with same source and destination in pgstat_recv_replslot.Tom Lane
Same type of issue as in commit 53d4f5fef and earlier fixes; also found by apparently-more-picky-than-the-buildfarm valgrind testing. This one is an oversight in commit 986816750. Since that's new in HEAD, no need for a back-patch.
2020-12-20Fix compiler warning introduced in 6df7a9698bAlexander Korotkov
2020-12-20Fixes for pg_dump.c regarding multirangesAlexander Korotkov
This commit fixes two wrong version number checks and one wrong check for null.
2020-12-20Multirange datatypesAlexander Korotkov
Multiranges are basically sorted arrays of non-overlapping ranges with set-theoretic operations defined over them. Since v14, each range type automatically gets a corresponding multirange datatype. There are both manual and automatic mechanisms for naming multirange types. Once can specify multirange type name using multirange_type_name attribute in CREATE TYPE.  Otherwise, a multirange type name is generated automatically. If the range type name contains "range" then we change that to "multirange". Otherwise, we add "_multirange" to the end. Implementation of multiranges comes with a space-efficient internal representation format, which evades extra paddings and duplicated storage of oids.  Altogether this format allows fetching a particular range by its index in O(n). Statistic gathering and selectivity estimation are implemented for multiranges. For this purpose, stored multirange is approximated as union range without gaps. This field will likely need improvements in the future. Catversion is bumped. Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0 Author: Paul Jungwirth, revised by me Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston Reviewed-by: Zhihong Yu, Alexander Korotkov
2020-12-19Remove now-useless ALWAYS_SUBDIRS entry in src/test/Makefile.Tom Lane
Commit 257836a75 added the "locale" subdirectory to SUBDIRS, but neglected to remove it from ALWAYS_SUBDIRS. This oversight had no functional effect because the filter-out function would remove it anyway. Still, it's confusing to readers to list a subdirectory in both places, especially because it makes the associated comment into a partial lie.
2020-12-19Update comment atop of ReorderBufferQueueMessage().Amit Kapila
The comments atop of this function describes behaviour in case of a transactional WAL message only, but it accepts both transactional and non-transactional WAL messages. Update the comments to describe behaviour in case of non-transactional WAL message as well. Ashutosh Bapat, rephrased by Amit Kapila Discussion: https://postgr.es/m/CAGEoWWTTzNzHOi8bj0wfAo1siGi-YEh6wqH1oaz4DrkTJ6HbTQ@mail.gmail.com
2020-12-18Add a couple of missed .gitignore entries.Tom Lane
Any subdirectory that's ignoring /output_iso/ should also ignore /tmp_check_iso/, which could be left behind by a failed pg_isolation_regress_check run. I think these have been wrong for awhile, but it doesn't seem important to fix in back branches.
2020-12-18Avoid memcpy() with same source and destination during relmapper init.Tom Lane
A narrow reading of the C standard says that memcpy(x,x,n) is undefined, although it's hard to envision an implementation that would really misbehave. However, analysis tools such as valgrind might whine about this; accordingly, let's band-aid relmapper.c to not do it. See also 5b630501e, d3f4e8a8a, ad7b48ea0, and other similar fixes. Apparently, none of those folk tried valgrinding initdb? This has been like this for long enough that I'm surprised it hasn't been reported before. Back-patch, just in case anybody wants to use a back branch on a platform that complains about this; we back-patched those earlier fixes too. Discussion: https://postgr.es/m/161790.1608310142@sss.pgh.pa.us
2020-12-17Revert "Get rid of the dedicated latch for signaling the startup process".Fujii Masao
Revert ac22929a26, as well as the followup fix 113d3591b8. Because it broke the assumption that the startup process waiting for the recovery conflict on buffer pin should be waken up only by buffer unpin or the timeout enabled in ResolveRecoveryConflictWithBufferPin(). It caused, for example, SIGHUP signal handler or walreceiver process to wake that startup process up unnecessarily frequently. Additionally, add the comments about why that dedicated latch that the reverted patch tried to get rid of should not be removed. Thanks to Kyotaro Horiguchi for the discussion. Author: Fujii Masao Discussion: https://postgr.es/m/d8c0c608-021b-3c73-fffd-3240829ee986@oss.nttdata.com
2020-12-15Fix varchar_2.out to match reality in cs_CZ locale.Tom Lane
Seems to be a copy-and-pasteo in c06d6aa4c. Per buildfarm.
2020-12-15Remove obsolete btrescan() comment.Peter Geoghegan
"Ordering stuff" refered to a _bt_first() call to _bt_orderkeys(). However, the _bt_orderkeys() function was renamed to _bt_preprocess_keys() by commit fa5c8a055a0. _bt_preprocess_keys() is directly referenced just after the removed comment already, which seems sufficient.
2020-12-15Remove useless variable storesAlvaro Herrera
Mistakenly introduced in 4cbe3ac3e867; bug repaired in 148e632c0541 but the stores were accidentally.
2020-12-15Error out when Gather Merge input is not sortedTomas Vondra
To build Gather Merge path, the input needs to be sufficiently sorted. Ensuring this is the responsibility of the code constructing the paths, but create_gather_merge_plan tried to handle unsorted paths by adding an explicit Sort. In light of the recent issues related to Incremental Sort, this is rather fragile. Some of the expressions may be volatile or parallel unsafe, in which case we can't add the Sort here. We could do more checks and add the Sort in at least some cases, but it seems cleaner to just error out and make it clear this is a bug in code constructing those paths. Author: James Coleman Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/CAAaqYe8cK3g5CfLC4w7bs%3DhC0mSksZC%3DH5M8LSchj5e5OxpTAg%40mail.gmail.com Discussion: https://postgr.es/m/CAJGNTeNaxpXgBVcRhJX%2B2vSbq%2BF2kJqGBcvompmpvXb7pq%2BoFA%40mail.gmail.com
2020-12-15Clean up ancient test stylePeter Eisentraut
Many older tests where written in a style like SELECT '' AS two, i.* FROM INT2_TBL where the first column indicated the number of expected result rows. This has gotten increasingly out of date, as the test data fixtures have expanded, so a lot of these were wrong and misleading. Moreover, this style isn't really necessary, since the psql output already shows the number of result rows. To clean this up, remove all those extra columns. Discussion: https://www.postgresql.org/message-id/flat/1a25312b-2686-380d-3c67-7a69094a999f%40enterprisedb.com
2020-12-15Improve hash_create()'s API for some added robustness.Tom Lane
Invent a new flag bit HASH_STRINGS to specify C-string hashing, which was formerly the default; and add assertions insisting that exactly one of the bits HASH_STRINGS, HASH_BLOBS, and HASH_FUNCTION be set. This is in hopes of preventing recurrences of the type of oversight fixed in commit a1b8aa1e4 (i.e., mistakenly omitting HASH_BLOBS). Also, when HASH_STRINGS is specified, insist that the keysize be more than 8 bytes. This is a heuristic, but it should catch accidental use of HASH_STRINGS for integer or pointer keys. (Nearly all existing use-cases set the keysize to NAMEDATALEN or more, so there's little reason to think this restriction should be problematic.) Tweak hash_create() to insist that the HASH_ELEM flag be set, and remove the defaults it had for keysize and entrysize. Since those defaults were undocumented and basically useless, no callers omitted HASH_ELEM anyway. Also, remove memset's zeroing the HASHCTL parameter struct from those callers that had one. This has never been really necessary, and while it wasn't a bad coding convention it was confusing that some callers did it and some did not. We might as well save a few cycles by standardizing on "not". Also improve the documentation for hash_create(). In passing, improve reinit.c's usage of a hash table by storing the key as a binary Oid rather than a string; and, since that's a temporary hash table, allocate it in CurrentMemoryContext for neatness. Discussion: https://postgr.es/m/590625.1607878171@sss.pgh.pa.us
2020-12-14Revert "Cannot use WL_SOCKET_WRITEABLE without WL_SOCKET_READABLE."Jeff Davis
This reverts commit 3a9e64aa0d96c8ffb6c682b082d0f72b1d373327. Commit 4bad60e3 fixed the root of the problem that 3a9e64aa worked around. This enables proper pipelining of commands after terminating replication, eliminating an undocumented limitation. Discussion: https://postgr.es/m/3d57bc29-4459-578b-79cb-7641baf53c57%40iki.fi Backpatch-through: 9.5
2020-12-14Improve some code around cryptohash functionsMichael Paquier
This adjusts some code related to recent changes for cryptohash functions: - Add a variable in md5.h to track down the size of a computed result, moved from pgcrypto. Note that pg_md5_hash() assumed a result of this size already. - Call explicit_bzero() on the hashed data when freeing the context for fallback implementations. For MD5, particularly, it would be annoying to leave some non-zeroed data around. - Clean up some code related to recent changes of uuid-ossp. .gitignore still included md5.c and a comment was incorrect. Discussion: https://postgr.es/m/X9HXKTgrvJvYO7Oh@paquier.xyz
2020-12-14Add some checkpoint/restartpoint status to ps displayMichael Paquier
This is done for end-of-recovery and shutdown checkpoints/restartpoints (end-of-recovery restartpoints don't exist) rather than all types of checkpoints, in cases where it may not be possible to rely on pg_stat_activity to get a status from the startup or checkpointer processes. For example, at the end of a crash recovery, this is useful to know if a checkpoint is running in the startup process, while previously the ps display may only show some information about "recovering" something, that can be confusing while a checkpoint runs. Author: Justin Pryzby Reviewed-by: Nathan Bossart, Kirk Jamison, Fujii Masao, Michael Paquier Discussion: https://postgr.es/m/20200818225238.GP17022@telsasoft.com
2020-12-12Use HASH_BLOBS for xidhash.Noah Misch
This caused BufFile errors on buildfarm member sungazer, and SIGSEGV was possible. Conditions for reaching those symptoms were more frequent on big-endian systems. Discussion: https://postgr.es/m/20201129214441.GA691200@rfd.leadboat.com
2020-12-12Correct behavior descriptions in comments, and correct a test name.Noah Misch
2020-12-12initdb: complete getopt_long alphabetizationBruce Momjian
Backpatch-through: 9.5
generated by cgit v1.2.3 (git 2.46.0) at 2025-08-31 18:28:00 +0000