summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2023-08-14Simplify determining logical replication worker types.Amit Kapila
We deduce a LogicalRepWorker's type from the values of several different fields ('relid' and 'leader_pid') whenever logic needs to know it. In fact, the logical replication worker type is already known at the time of launching the LogicalRepWorker and it never changes for the lifetime of that process. Instead of deducing the type, it is simpler to just store it one time, and access it directly thereafter. Author: Peter Smith Reviewed-by: Amit Kapila, Bharath Rupireddy Discussion: http://postgr.es/m/CAHut+PttPSuP0yoZ=9zLDXKqTJ=d0bhxwKaEaNcaym1XqcvDEg@mail.gmail.com
2023-08-12Fix off-by-one in XLogRecordMaxSize check.Noah Misch
pg_logical_emit_message(false, '_', repeat('x', 1069547465)) failed with self-contradictory message "WAL record would be 1069547520 bytes (of maximum 1069547520 bytes)". There's no particular benefit from allowing or denying one byte in either direction; XLogRecordMaxSize could rise a few megabytes without trouble. Hence, this is just for cleanliness. Back-patch to v16, where this check first appeared.
2023-08-12Show GIDs of two-phase commit commands as constants in pg_stat_statementsMichael Paquier
This relies on the "location" field added to TransactionStmt in 31de7e6, now applied to the "gid" field used by 2PC commands. These commands are now reported like: COMMIT PREPARED $1 PREPARE TRANSACTION $1 ROLLBACK PREPARED $1 Applying constants for these commands is a huge advantage for workloads that rely a lot on 2PC commands with different GIDs. Some tests are added to track the new behavior. Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/ZMhT9kNtJJsHw6jK@paquier.xyz
2023-08-11Fix code indentation violations introduced by recent commitMichael Paquier
The two culprit commits are 5765cfe and 5e0c761. Per buildfarm member koel for the first commit, while I have noticed the second one in passing.
2023-08-10Transform proconfig for faster execution.Jeff Davis
Store function config settings in lists to avoid the need to parse and allocate for each function execution. Speedup is modest but significant. Additionally, this change also seems cleaner and supports some other performance improvements under discussion. Discussion: https://postgr.es/m/04c8592dbd694e4114a3ed87139a7a04e4363030.camel@j-davis.com Reviewed-by: Nathan Bossart
2023-08-10Remove test from commit fa2e874946.Jeff Davis
The fix itself is fine, but the test revealed other problems related to parallel query that are not easily fixable. Remove the test for now to fix the buildfarm. Discussion: https://postgr.es/m/88825.1691665432@sss.pgh.pa.us Backpatch-through: 11
2023-08-10Fix erroneous -Werror=missing-braces on old GCCPeter Eisentraut
The buildfarm reports that this is an error on gcc (Debian 4.7.2-5) 4.7.2, 32-bit. The bug seems to be GCC bug 53119, which has obviously been fixed for years. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CT6HJ3U8068R.3A8SJMV02D9BC@gonk
2023-08-10Update Solution.pm for new LoongArch CRC symbolJohn Naylor
Oversight in 4d14ccd6a, per report from Amit Kapila and Michael Paquier. Discussion: https://postgr.es/m/CAA4eK1LsV3KuyUt8tzZDjPcUds1XfVVeW3Wpeju_59DtRV0%3DxQ%40mail.gmail.com
2023-08-10Document RelationGetIndexAttrBitmap betterAlvaro Herrera
Commit 19d8e2308bc5 changed the list of set-of-columns that can be returned by RelationGetIndexAttrBitmap, but didn't update its "documentation". That was pretty hard to read already, so rewrite to make it more comprehensible, adding the missing values while at it. Backpatch to 16, like that commit. Discussion: https://postgr.es/m/20230809091155.7c7f3gttjk3dj4ze@alvherre.pgsql Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
2023-08-10Use native CRC instructions on 64-bit LoongArchJohn Naylor
As with the Intel and Arm CRC instructions, compiler intrinsics for them must be supported by the compiler. In contrast, no runtime check is needed. Aligned memory access is faster, so use the Arm coding as a model. YANG Xudong Discussion: https://postgr.es/m/b522a0c5-e3b2-99cc-6387-58134fb88cbe%40ymatrix.cn
2023-08-09Recalculate search_path after ALTER ROLE.Jeff Davis
Renaming a role can affect the meaning of the special string $user, so must cause search_path to be recalculated. Discussion: https://postgr.es/m/186761d32c0255debbdf50b6310b581b9c973e6c.camel@j-davis.com Reviewed-by: Nathan Bossart, Michael Paquier Backpatch-through: 11
2023-08-09struct PQcommMethods: use C99 designated initializersAlvaro Herrera
As in 98afa68d9352, 2c860777656a, et al.
2023-08-09Fix last remaining uninitialized memory warningsPeter Eisentraut
gcc (version 13) fails to properly analyze the code due to the loop stop condition including `l != NULL`. Let's just help it out. Author: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/CT6HJ3U8068R.3A8SJMV02D9BC@gonk
2023-08-09Fix pg_dumpall with in-place tablespacesMichael Paquier
In-place tablespaces would be dumped with the path produced by pg_tablespace_location(), which is in this case a relative path built as pg_tblspc/OID, but this would fail to restore as such tablespaces need to use an empty string as location. In order to detect if an in-place tablespace is used, this commit checks if the path returned is relative and adapts the dump contents in consequence. Like the other changes related to in-place tablespaces, no backpatch is done as these are only intended for development purposes. Rui Zhao has fixed the code, while the test is from me. Author: Rui Zhao, Michael Paquier Discussion: https://postgr.es/m/80c80b4a-b87b-456f-bd46-1ae326601d79.xiyuan.zr@alibaba-inc.com
2023-08-08doc: Fix incorrect entries generated from wait_event_names.txtMichael Paquier
fa88928 has introduced wait_event_names.txt, and some of its entries had some documentation fields with more information than necessary. This commit brings back the description of all the wait events to be consistent with the older stable branches. Five descriptions were incorrect. Author: Bertrand Drouvot Discussion: https://postgr.es/m/e378989e-1899-643a-dec1-10f691a0a105@gmail.com
2023-08-07Reject substituting extension schemas or owners matching ["$'\].Noah Misch
Substituting such values in extension scripts facilitated SQL injection when @extowner@, @extschema@, or @extschema:...@ appeared inside a quoting construct (dollar quoting, '', or ""). No bundled extension was vulnerable. Vulnerable uses do appear in a documentation example and in non-bundled extensions. Hence, the attack prerequisite was an administrator having installed files of a vulnerable, trusted, non-bundled extension. Subject to that prerequisite, this enabled an attacker having database-level CREATE privilege to execute arbitrary code as the bootstrap superuser. By blocking this attack in the core server, there's no need to modify individual extensions. Back-patch to v11 (all supported versions). Reported by Micah Gate, Valerie Woolard, Tim Carey-Smith, and Christoph Berg. Security: CVE-2023-39417
2023-08-07Translation updatesPeter Eisentraut
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash: 97398d714ace69f0c919984e160f429b6fd2300e
2023-08-07Don't Memoize lateral joins with volatile join conditionsDavid Rowley
The use of Memoize was already disabled in normal joins when the join conditions had volatile functions per the code in match_opclause_to_indexcol(). Ordinarily, the parameterization for the inner side of a nested loop will be an Index Scan or at least eventually lead to an index scan (perhaps nested several joins deep). However, for lateral joins, that's not the case and seq scans can be parameterized too, so we can't rely on match_opclause_to_indexcol(). Here we explicitly check the parameterization for volatile functions and don't consider the generation of a Memoize path when such functions are present. Author: Richard Guo Discussion: https://postgr.es/m/CAMbWs49nHFnHbpepLsv_yF3qkpCS4BdB-v8HoJVv8_=Oat0u_w@mail.gmail.com Backpatch-through: 14, where Memoize was introduced
2023-08-07Fix RLS policy usage in MERGE.Dean Rasheed
If MERGE executes an UPDATE action on a table with row-level security, the code incorrectly applied the WITH CHECK clauses from the target table's INSERT policies to new rows, instead of the clauses from the table's UPDATE policies. In addition, it failed to check new rows against the target table's SELECT policies, if SELECT permissions were required (likely to always be the case). In addition, if MERGE executes a DO NOTHING action for matched rows, the code incorrectly applied the USING clauses from the target table's DELETE policies to existing target tuples. These policies were applied as checks that would throw an error, if they did not pass. Fix this, so that a MERGE UPDATE action applies the same RLS policies as a plain UPDATE query with a WHERE clause, and a DO NOTHING action does not apply any RLS checks (other than adding clauses from SELECT policies to the join). Back-patch to v15, where MERGE was introduced. Dean Rasheed, reviewed by Stephen Frost. Security: CVE-2023-39418
2023-08-07Improve const use in zlib-using codePeter Eisentraut
If we define ZLIB_CONST before including zlib.h, zlib augments some interfaces with const decorations. By doing that we can keep our own interfaces cleaner and can remove some unconstify calls. ZLIB_CONST was introduced in zlib 1.2.5.2 (17 Dec 2011). When compiling with older zlib releases, you might now get compiler warnings about discarding qualifiers. CentOS 6 has zlib 1.2.3, but in 8e278b6576, we removed support for the OpenSSL release in CentOS 6, so it seems ok to de-support the zlib release in CentOS 6 as well. Reviewed-by: Tristan Partin <tristan@neon.tech> Discussion: https://www.postgresql.org/message-id/flat/33462926-bb1e-7cc9-8d92-d86318e8ed1d%40eisentraut.org
2023-08-07Fix misleading comment in paraminfo_get_equal_hashopsDavid Rowley
The comment mistakenly claimed the code was checking PlaceHolderVars for volatile functions when the code was actually checking lateral vars. Update the comment to reflect the reality of the code. Author: Richard Guo Discussion: https://postgr.es/m/CAMbWs48HZGZOV85g0fx8z1qDx6NNKHexJPT2FCnKnZhxBWkd-A@mail.gmail.com
2023-08-06Tidy up join_search_one_level codeDavid Rowley
The code in join_search_one_level was a bit convoluted. With a casual glance, you might think that other_rels_list was being set to something other than joinrels[1] when level == 2, however, joinrels[level - 1] is joinrels[1] when level == 2, so nothing special needs to happen to set other_rels_list. Let's clean that up to avoid confusing anyone. In passing, we may as well modernize the loop in make_rels_by_clause_joins() and instead of passing in the ListCell to start looping from, let's just pass in the index where to start from and make use of for_each_from(). Ever since 1cff1b95a, Lists are arrays under the hood. lnext() and list_head() both seem a little too linked-list like. Author: Alex Hsieh, David Rowley, Richard Guo Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/CANWNU8x9P9aCXGF%3DaT-A_8mLTAT0LkcZ_ySYrGbcuHzMQw2-1g%40mail.gmail.com
2023-08-04Simplify some of the logical replication worker-type checks.Amit Kapila
Author: Peter Smith Reviewed-by: Hou Zhijie Discussion: http://postgr.es/m/CAHut+Pv-xkEpuPzbEJ=ZSi7Hp2RoGJf=VA-uDRxLi1KHSneFjg@mail.gmail.com
2023-08-04Attempt to stabilize new window agg regression testDavid Rowley
This test was recently added in 3900a02c9. It appears to be unstable in regards to the join order presumably due to the relations at either side of the join being equal in side. Here we add a qual to make one of them smaller so the planner is more likely to choose to hash the smaller of the two. Reported-by: Nathan Bossart, Tom Lane Discussion: https://www.postgr.es/m/20230803235403.GC1238296@nathanxps13
2023-08-04Minor adjustments to WindowAgg startup cost codeDavid Rowley
This is a follow-on of 3900a02c9 containing some changes which I forgot to commit locally before forming a patch with git format-patch. Discussion: https://postgr.es/m/CAApHDvrB0S5BMv+0-wTTqWFE-BJ0noWqTnDu9QQfjZ2VSpLv_g@mail.gmail.com
2023-08-04Account for startup rows when costing WindowAggsDavid Rowley
Here we adjust the costs for WindowAggs so that they properly take into account how much of their subnode they must read before outputting the first row. Without this, we always assumed that the startup cost for the WindowAgg was not much more expensive than the startup cost of its subnode, however, that's going to be completely wrong in many cases. The WindowAgg may have to read *all* of its subnode to output a single row with certain window bound options. Here we estimate how many rows we'll need to read from the WindowAgg's subnode and proportionally add more of the subnode's run costs onto the WindowAgg's startup costs according to how much of it we expect to have to read in order to produce the first WindowAgg row. The reason this is more important than we might have initially thought is that we may end up making use of a path from the lower planner that works well as a cheap startup plan when the query has a LIMIT clause, however, the WindowAgg might mean we need to read far more rows than what the LIMIT specifies. No backpatch on this so as not to cause plan changes in released versions. Bug: #17862 Reported-by: Tim Palmer Author: David Rowley Reviewed-by: Andy Fan Discussion: https://postgr.es/m/17862-1ab8f74b0f7b0611@postgresql.org Discussion: https://postgr.es/m/CAApHDvrB0S5BMv+0-wTTqWFE-BJ0noWqTnDu9QQfjZ2VSpLv_g@mail.gmail.com
2023-08-03Update comments on CustomPath struct.Etsuro Fujita
Commit e7cb7ee14 allowed custom scan providers to create CustomPath paths for join relations as well, but missed updating the comments. Back-patch to all supported branches. Discussion: https://postgr.es/m/CAPmGK15ODkN%2B%3DhkBCufj1HBW0x5OTb65Xuy7ryXchMdiCMpx_g%40mail.gmail.com
2023-08-03Refactor to split Apply and Tablesync Workers code.Amit Kapila
Both apply and tablesync workers were using ApplyWorkerMain() as entry point. As the name implies, ApplyWorkerMain() should be considered as the main function for apply workers. Tablesync worker's path was hidden and does not have enough in common to share the same main function with apply worker. Also, most of the code shared by both worker types is already combined in LogicalRepApplyLoop(). There is no need to combine the rest in ApplyWorkerMain() anymore. This patch introduces TablesyncWorkerMain() as a new entry point for tablesync workers. This aims to increase code readability and would help with future improvements like the reuse of tablesync workers in the initial synchronization. Author: Melih Mutlu based on suggestions by Melanie Plageman Reviewed-by: Peter Smith, Kuroda Hayato, Amit Kapila Discussion: http://postgr.es/m/CAGPVpCTq=rUDd4JUdaRc1XUWf4BrH2gdSNf3rtOMUGj9rPpfzQ@mail.gmail.com
2023-08-02Fix ReorderBufferCheckMemoryLimit() comment.Masahiko Sawada
Commit 7259736a6 updated the comment but it was not correct since ReorderBufferLargestStreamableTopTXN() returns only top-level transactions. Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CAD21AoA9XB7OR86BqvrCe2dMYX%2BZv3-BvVmjF%3DGY2z6jN-kqjg%40mail.gmail.com Backpatch-through: 14
2023-08-02Fix performance regression in pg_strtointNN_safe functionsDavid Rowley
Between 6fcda9aba and 1b6f632a3, the pg_strtoint functions became quite a bit slower in v16, despite efforts in 6b423ec67 to speed these up. Since the majority of cases for these functions will only contain base-10 digits, perhaps prefixed by a '-', it makes sense to have a special case for this and just fall back on the more complex version which processes hex, octal, binary and underscores if the fast path version fails to parse the string. While we're here, update the header comments for these functions to mention that hex, octal and binary formats along with underscore separators are now supported. Author: Andres Freund, David Rowley Reported-by: Masahiko Sawada Reviewed-by: Dean Rasheed, John Naylor Discussion: https://postgr.es/m/CAD21AoDvDmUQeJtZrau1ovnT_smN940%3DKp6mszNGK3bq9yRN6g%40mail.gmail.com Backpatch-through: 16, where 6fcda9aba and 1b6f632a3 were added
2023-08-01Fix pg_stat_io buffer reuse test instabilityAndres Freund
The stats regression test attempts to ensure that Buffer Access Strategy "reuses" are being counted in pg_stat_io by vacuuming a table which is larger than the size of the strategy ring. However, when shared buffers are in sufficiently high demand, another backend could evict one of the blocks in the strategy ring before the first backend has a chance to reuse the buffer. The backend using the strategy would then evict another shared buffer and add that buffer to the strategy ring. This counts as an eviction and not a reuse in pg_stat_io. Count both evictions and reuses in the test to ensure it does not fail incorrectly. Reported-by: Jeff Davis <pgsql@j-davis.com>, Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Alexander Lakhin <exclusion@gmail.com> Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAAKRu_bNG27AxG9TdPtwsL6wg8AWbVckjmTL2t1HF=miDQuNtw@mail.gmail.com
2023-08-01Add and use symbolic constants for tar header offsets and file types.Robert Haas
Because symbolic constants in a header file are better than magic constants embedded in the code. Patch by me, reviewed by Tom Lane, Dagfinn Ilmari Mannsåker, and Tristan Partin. Discussion: http://postgr.es/m/CA+TgmoZNbLwhmCrNtkJAvi8FLkwFdMeVU3myV2HQQpA5bvbRZg@mail.gmail.com
2023-08-02Fix overly strict Assert in jsonpath codeDavid Rowley
This was failing for queries which try to get the .type() of a jpiLikeRegex. For example: select jsonb_path_query('["string", "string"]', '($[0] like_regex ".{7}").type()'); Reported-by: Alexander Kozhemyakin Bug: #18035 Discussion: https://postgr.es/m/18035-64af5cdcb5adf2a9@postgresql.org Backpatch-through: 12, where SQL/JSON path was added.
2023-07-31Rename OverrideSearchPath to SearchPathMatcher.Noah Misch
The previous commit removed the "override" APIs. Surviving APIs facilitate plancache.c to snapshot search_path and test whether the current value equals a remembered snapshot. Aleksander Alekseev. Reported by Alexander Lakhin and Noah Misch. Discussion: https://postgr.es/m/8ffb4650-52c4-6a81-38fc-8f99be981130@gmail.com
2023-07-31Remove PushOverrideSearchPath() and PopOverrideSearchPath().Noah Misch
Since commit 681d9e4621aac0a9c71364b6f54f00f6d8c4337f, they have no in-tree calls. Any new calls would introduce security vulnerabilities like the one fixed in that commit. Alexander Lakhin, reviewed by Aleksander Alekseev. Discussion: https://postgr.es/m/8ffb4650-52c4-6a81-38fc-8f99be981130@gmail.com
2023-07-31Support custom wait events for wait event type "Extension"Michael Paquier
Two backend routines are added to allow extension to allocate and define custom wait events, all of these being allocated in the type "Extension": * WaitEventExtensionNew(), that allocates a wait event ID computed from a counter in shared memory. * WaitEventExtensionRegisterName(), to associate a custom string to the wait event ID allocated. Note that this includes an example of how to use this new facility in worker_spi with tests in TAP for various scenarios, and some documentation about how to use them. Any code in the tree that currently uses WAIT_EVENT_EXTENSION could switch to this new facility to define custom wait events. This is left as work for future patches. Author: Masahiro Ikeda Reviewed-by: Andres Freund, Michael Paquier, Tristan Partin, Bharath Rupireddy Discussion: https://postgr.es/m/b9f5411acda0cf15c8fbb767702ff43e@oss.nttdata.com
2023-07-31Bring some MSVC asserts in line with other platformsJohn Naylor
MSVC's _BitScan* functions return a boolean indicating whether any bits were set in the input, and we were previously asserting that they returned true, per our API. This is correct. However, other platforms simply assert that the input is non-zero, so do that to be more consistent. Noted while investigating a hypothesis from Ranier Vilela about undefined behavior, but this is not his proposed patch. Discussion: https://www.postgresql.org/message-id/CAEudQAoDhUZyKGJ1vbMGcgVUOcsixe-%3DjcVaDWarqkUg163D2w%40mail.gmail.com
2023-07-31Add WAIT_EVENT_{CLASS,ID}_MASK in wait_event.cMichael Paquier
These are useful to extract the class ID and the event ID associated to a single uint32 wait_event_info. Only two code paths use them now, but an upcoming patch will extend their use. Discussion: https://postgr.es/m/ZMcJ7F7nkGkIs8zP@paquier.xyz
2023-07-31Avoid memory leak in rmtree() when path cannot be openedMichael Paquier
An allocation done for the directory names to recurse into for their deletion was done before OPENDIR(), so, assuming that a failure happens, this could leak a bit of memory. Author: Ranier Vilela Reviewed-by: Daniel Gustafsson Discussion: https://postgr.es/m/CAEudQAoN3-2ZKBALThnEk_q2hu8En5A0WG9O+5siJTQKVZzoWQ@mail.gmail.com
2023-07-30Fix pg_rewind with in-place tablespaces when source is remoteMichael Paquier
libpq_source.c would consider any result returned by pg_tablespace_location() as a symlink, resulting in run-time errors like that: pg_rewind: error: file "pg_tblspc/NN" is of different type in source and target In-place tablespaces are directories located in pg_tblspc/, returned as relative paths instead of absolute paths, so rely on that to make the difference with a normal tablespace and an in-place one. If the path is relative, the tablespace is handled as a directory. If the path is absolute, consider it as a symlink. In-place tablespaces are only intended for development purposes, so like 363e8f9 no backpatch is done. A test is added in pg_rewind with an in-place tablespace and some data in it. Author: Rui Zhao, Michael Paquier Discussion: https://postgr.es/m/2b79d2a8-b2d5-4bd7-a15b-31e485100980.xiyuan.zr@alibaba-inc.com
2023-07-29worker_spi: Fix race condition in newly-added TAP testsMichael Paquier
The second portion of the tests had a race condition where it would be possible for the startup of the dynamic workers to fail, in the event where the static workers started before them with the library loading in shared_preload_libraries did not finish to create their respective schemas. The conflict is caused by the fact that the dynamic and static workers used the same IDs, overlapping each other, so, for now, switch the dynamic workers to use different IDs, leading to different schemas created. Reported-by: Andres Freund Discussion: https://postgr.es/m/20230728022332.egqzobhskmlf6ntr@awork3.anarazel.de
2023-07-28Harmonize password reuse in vacuumdb, clusterdb, and reindexdb.Nathan Bossart
Commits 83dec5a712 and ff402ae11b taught vacuumdb to reuse passwords instead of prompting repeatedly. However, the docs still warn about repeated prompts, and this improvement was not applied to clusterdb and reindexdb. This commit allows clusterdb and reindexdb to reuse passwords just like vacuumdb does, and it expunges the aforementioned warnings from the docs. Reviewed-by: Gurjeet Singh, Zhang Mingli Discussion: https://postgr.es/m/20230628045741.GA1813397%40nathanxps13
2023-07-28Disallow replacing joins with scans in problematic cases.Etsuro Fujita
Commit e7cb7ee14, which introduced the infrastructure for FDWs and custom scan providers to replace joins with scans, failed to add support handling of pseudoconstant quals assigned to replaced joins in createplan.c, leading to an incorrect plan without a gating Result node when postgres_fdw replaced a join with such a qual. To fix, we could add the support by 1) modifying the ForeignPath and CustomPath structs to store the list of RestrictInfo nodes to apply to the join, as in JoinPaths, if they represent foreign and custom scans replacing a join with a scan, and by 2) modifying create_scan_plan() in createplan.c to use that list in that case, instead of the baserestrictinfo list, to get pseudoconstant quals assigned to the join; but #1 would cause an ABI break. So fix by modifying the infrastructure to just disallow replacing joins with such quals. Back-patch to all supported branches. Reported by Nishant Sharma. Patch by me, reviewed by Nishant Sharma and Richard Guo. Discussion: https://postgr.es/m/CADrsxdbcN1vejBaf8a%2BQhrZY5PXL-04mCd4GDu6qm6FigDZd6Q%40mail.gmail.com
2023-07-27Eliminate fixed token-length limit in hba.c.Tom Lane
Historically, hba.c limited tokens in the authentication configuration files (pg_hba.conf and pg_ident.conf) to less than 256 bytes. We have seen a few reports of this limit causing problems; notably, for moderately-complex LDAP configurations. Let's get rid of the fixed limit by using a StringInfo instead of a fixed-size buffer. This actually takes less code than before, since we can get rid of a nontrivial error recovery stanza. It's doubtless a hair slower, but parsing the content of the HBA files should in no way be performance-critical. Although this is a pretty straightforward patch, it doesn't seem worth the risk to back-patch given the small number of complaints to date. In released branches, we'll just raise MAX_TOKEN to ameliorate the problem. Discussion: https://postgr.es/m/1588937.1690221208@sss.pgh.pa.us
2023-07-27worker_spi: Switch to TAP testsMichael Paquier
This commit moves worker_spi to use TAP tests. sql/worker_spi.sql is gone, replaced by an equivalent set of queries in a TAP script, without worker_spi loaded in shared_preload_libraries: - One query to launch a worker dynamically, relying now on "postgres" as the default database to connect to. - Two wait queries with poll_query_until(), one to wait for the worker schema to be initialized and a second to wait for a tuple processed by the worker. - Server reload to accelerate the main loop of the spawned worker. More coverage is added for workers registered when the library is loaded with shared_preload_libraries, while on it, checking that these are connecting to the database set in the GUC worker_spi.database. A local run of these test is showing that TAP is slightly faster than the original, while providing more coverage (3.7s vs 4.4s). There was also some discussions about keeping the SQL tests, but this would require initializing twice a cluster, increasing the runtime of the tests up to 5.6s here. These tests will be expanded more in an upcoming patch aimed at adding support for custom wait events for the Extension class, still under discussion, to check the new in-core APIs with and without a library set in shared_preload_libraries. Bharath has written the part where shared_preload_libraries is used, while I have migrated the existing SQL tests to TAP. Author: Bharath Rupireddy, Michael Paquier Reviewed-by: Masahiro Ikeda Discussion: https://postgr.es/m/CALj2ACWR9ncAiDF73unqdJF1dmsA2R0efGXX2624X+YVxcAVWg@mail.gmail.com
2023-07-27Fix performance problem with new COPY DEFAULT codeDavid Rowley
9f8377f7a added code to allow COPY FROM insert a column's default value when the input matches the DEFAULT string specified in the COPY command. Here we fix some inefficient code which needlessly palloc0'd an array to store if we should use the default value or input value for the given column. This array was being palloc0'd and pfree'd once per row. It's much more efficient to allocate this once and just reset the values once per row. Reported-by: Masahiko Sawada Author: Masahiko Sawada Discussion: https://postgr.es/m/CAD21AoDvDmUQeJtZrau1ovnT_smN940%3DKp6mszNGK3bq9yRN6g%40mail.gmail.com Backpatch-through: 16, where 9f8377f7a was introduced.
2023-07-27Add sanity asserts for index OID and attnums during cache initMichael Paquier
There was already a check on the relation OID, but not its index OID and the attributes that can be used during the syscache lookups. The two assertions added by this commit are cheap, and actually useful for developers to fasten the detection of incorrect data in a new entry added in the syscache list, as these assertions are triggered during the initial cache loading (initdb, or just backend startup), not requiring a syscache that uses the new entry. While on it, the relation OID check is switched to use OidIsValid(). Author: Aleksander Alekseev Reviewed-by: Dagfinn Ilmari Mannsåker, Zhang Mingli, Michael Paquier Discussion: https://postgr.es/m/CAJ7c6TOjUTJ0jxvWY6oJeP2-840OF8ch7qscZQsuVuotXTOS_g@mail.gmail.com
2023-07-27Show savepoint names as constants in pg_stat_statementsMichael Paquier
In pg_stat_statements, savepoint names now show up as constants with a parameter symbol, using as base query string the one added as a new entry to the PGSS hash table, leading to: RELEASE $1 ROLLBACK TO $1 SAVEPOINT $1 Applying constants to these query parts is a huge advantage for workloads that generate randomly savepoint points, like ORMs (Django is at the origin of this patch). The ODBC driver is a second layer that likes a lot savepoints, though it does not use a random naming pattern. A "location" field is added to TransactionStmt, now set only for savepoints. The savepoint name is ignored by the query jumbling. The location can be extended to other query patterns, if required, like 2PC commands. Some tests are added to pg_stat_statements for all the query patterns supported by the parser. ROLLBACK, ROLLBACK TO SAVEPOINT and ROLLBACK TRANSACTION TO SAVEPOINT have the same Node representation, so all these are equivalents. The same happens for RELEASE and RELEASE SAVEPOINT. Author: Greg Sabino Mullane Discussion: https://postgr.es/m/CAKAnmm+2s9PA4OaumwMJReWHk8qvJ_-g1WqxDRDAN1BSUfxyTw@mail.gmail.com
2023-07-26Adjust extra lines generated by psql to be valid SQL comments.Nathan Bossart
psql's --echo-hidden, --log-file, and --single-step options generate extra lines to clearly separate queries from other output. Presently, these extra lines are not valid SQL comments, which makes them a hazard for anyone trying to copy/paste the decorated queries into a client or query editor. This commit replaces the starting and ending asterisks in these extra lines with forward slashes so that they are valid SQL comments that can be copy/pasted without incident. Author: Kirk Wolak Reviewed-by: Pavel Stehule, Laurenz Albe, Tom Lane, Alvaro Herrera, Andrey Borodin Discussion: https://postgr.es/m/CACLU5mTFJRJYtbvmZ26txGgmXWQo0hkGhH2o3hEquUPmSbGtBw%40mail.gmail.com
2023-07-26Add more SQL/JSON constructor functionsAmit Langote
This Patch introduces three SQL standard JSON functions: JSON() JSON_SCALAR() JSON_SERIALIZE() JSON() produces json values from text, bytea, json or jsonb values, and has facilitites for handling duplicate keys. JSON_SCALAR() produces a json value from any scalar sql value, including json and jsonb. JSON_SERIALIZE() produces text or bytea from input which containis or represents json or jsonb; For the most part these functions don't add any significant new capabilities, but they will be of use to users wanting standard compliant JSON handling. Catversion bumped as this changes ruleutils.c. Author: Nikita Glukhov <n.gluhov@postgrespro.ru> Author: Teodor Sigaev <teodor@sigaev.ru> Author: Oleg Bartunov <obartunov@gmail.com> Author: Alexander Korotkov <aekorotkov@gmail.com> Author: Andrew Dunstan <andrew@dunslane.net> Author: Amit Langote <amitlangote09@gmail.com> Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby, Álvaro Herrera, Peter Eisentraut Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com