summaryrefslogtreecommitdiff
path: root/doc/src
AgeCommit message (Collapse)Author
2017-04-03doc: Change xref style to number onlyPeter Eisentraut
Change the style of links generated by xrefs to section number only, as it was with DSSSL, instead of number and title, as is the default of the XSLT stylesheets. Our documentation is mostly written expecting the old style, so keep that for the time being, per discussion.
2017-04-02Doc: clarify behavior of OT_WHOLE_LINE and OT_FILEPIPE psql slash commands.Tom Lane
This is another bit of ancient behavior that was documented poorly (in a couple of cases) or not at all (in several others). Discussion: https://postgr.es/m/9b4ea968-753f-4b5f-b46c-d7d3bf7c8f90@manitou-mail.org
2017-04-02Document psql's behavior of recalling the previously executed query.Tom Lane
Various psql slash commands that normally act on the current query buffer will automatically recall and re-use the most recently executed SQL command instead, if the current query buffer is empty. Although this behavior is ancient (dating apparently to commit 77a472993), it was documented nowhere in the psql reference page. For that matter, we'd never bothered to define the concept of "current query buffer" explicitly. Fix that. Do some wordsmithing on relevant command descriptions to improve clarity and consistency. Discussion: https://postgr.es/m/9b4ea968-753f-4b5f-b46c-d7d3bf7c8f90@manitou-mail.org
2017-04-01Allow psql variable substitution to occur in backtick command strings.Tom Lane
Previously, text between backquotes in a psql metacommand's arguments was always passed to the shell literally. That considerably hobbles the usefulness of the feature for scripting, so we'd foreseen for a long time that we'd someday want to allow substitution of psql variables into the shell command. IMO the addition of \if metacommands has brought us to that point, since \if can greatly benefit from some sort of client-side expression evaluation capability, and psql itself is not going to grow any such thing in time for v10. Hence, this patch. It allows :VARIABLE to be replaced by the exact contents of the named variable, while :'VARIABLE' is replaced by the variable's contents suitably quoted to become a single shell-command argument. (The quoting rules for that are different from those for SQL literals, so this is a bit of an abuse of the :'VARIABLE' notation, but I doubt anyone will be confused.) As with other situations in psql, no substitution occurs if the word following a colon is not a known variable name. That limits the risk of compatibility problems for existing psql scripts; but the risk isn't zero, so this needs to be called out in the v10 release notes. Discussion: https://postgr.es/m/9561.1490895211@sss.pgh.pa.us
2017-04-01BRIN de-summarizationAlvaro Herrera
When the BRIN summary tuple for a page range becomes too "wide" for the values actually stored in the table (because the tuples that were present originally are no longer present due to updates or deletes), it can be useful to remove the outdated summary tuple, so that a future summarization can install a tighter summary. This commit introduces a SQL-callable interface to do so. Author: Álvaro Herrera Reviewed-by: Eiji Seki Discussion: https://postgr.es/m/20170228045643.n2ri74ara4fhhfxf@alvherre.pgsql
2017-04-01BRIN auto-summarizationAlvaro Herrera
Previously, only VACUUM would cause a page range to get initially summarized by BRIN indexes, which for some use cases takes too much time since the inserts occur. To avoid the delay, have brininsert request a summarization run for the previous range as soon as the first tuple is inserted into the first page of the next range. Autovacuum is in charge of processing these requests, after doing all the regular vacuuming/ analyzing work on tables. This doesn't impose any new tasks on autovacuum, because autovacuum was already in charge of doing summarizations. The only actual effect is to change the timing, i.e. that it occurs earlier. For this reason, we don't go any great lengths to record these requests very robustly; if they are lost because of a server crash or restart, they will happen at a later time anyway. Most of the new code here is in autovacuum, which can now be told about "work items" to process. This can be used for other things such as GIN pending list cleaning, perhaps visibility map bit setting, both of which are currently invoked during vacuum, but do not really depend on vacuum taking place. The requests are at the page range level, a granularity for which we did not have SQL-level access; we only had index-level summarization requests via brin_summarize_new_values(). It seems reasonable to add SQL-level access to range-level summarization too, so add a function brin_summarize_range() to do that. Authors: Álvaro Herrera, based on sketch from Simon Riggs. Reviewed-by: Thomas Munro. Discussion: https://postgr.es/m/20170301045823.vneqdqkmsd4as4ds@alvherre.pgsql
2017-03-31Add transition table support to plpgsql.Kevin Grittner
Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and suggestions from many others
2017-03-31Add infrastructure to support EphemeralNamedRelation references.Kevin Grittner
A QueryEnvironment concept is added, which allows new types of objects to be passed into queries from parsing on through execution. At this point, the only thing implemented is a collection of EphemeralNamedRelation objects -- relations which can be referenced by name in queries, but do not exist in the catalogs. The only type of ENR implemented is NamedTuplestore, but provision is made to add more types fairly easily. An ENR can carry its own TupleDesc or reference a relation in the catalogs by relid. Although these features can be used without SPI, convenience functions are added to SPI so that ENRs can easily be used by code run through SPI. The initial use of all this is going to be transition tables in AFTER triggers, but that will be added to each PL as a separate commit. An incidental effect of this patch is to produce a more informative error message if an attempt is made to modify the contents of a CTE from a referencing DML statement. No tests previously covered that possibility, so one is added. Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and suggestions from many others
2017-03-31Don't use bgw_main even to specify in-core bgworker entrypoints.Robert Haas
On EXEC_BACKEND builds, this can fail if ASLR is in use. Backpatch to 9.5. On master, completely remove the bgw_main field completely, since there is no situation in which it is safe for an EXEC_BACKEND build. On 9.6 and 9.5, leave the field intact to avoid breaking things for third-party code that doesn't care about working under EXEC_BACKEND. Prior to 9.5, there are no in-core bgworker entrypoints. Petr Jelinek, reviewed by me. Discussion: http://postgr.es/m/09d8ad33-4287-a09b-a77f-77f8761adb5e@2ndquadrant.com
2017-03-31For foreign keys, check REFERENCES privilege only on the referenced table.Tom Lane
We were requiring that the user have REFERENCES permission on both the referenced and referencing tables --- but this doesn't seem to have any support in the SQL standard, which says only that you need REFERENCES permission on the referenced table. And ALTER TABLE ADD FOREIGN KEY has already checked that you own the referencing table, so the check could only fail if a table owner has revoked his own REFERENCES permission. Moreover, the symmetric interpretation of this permission is unintuitive and confusing, as per complaint from Paul Jungwirth. So let's drop the referencing-side check. In passing, do a bit of wordsmithing on the GRANT reference page so that all the privilege types are described in similar fashion. Discussion: https://postgr.es/m/8940.1490906755@sss.pgh.pa.us
2017-03-31Improve documentation for table partitioning.Robert Haas
Emphasize the new declarative partitioning more, and compare and contrast it more clearly with inheritance-based partitioning. Amit Langote, reviewed and somewhat revised by me Discussion: http://postgr.es/m/a6f99cdb-21e7-1d65-1381-91f2cfa156e2@lab.ntt.co.jp
2017-03-31Revert "Allow ON CONFLICT .. DO NOTHING on a partitioned table."Robert Haas
This reverts commit 8355a011a0124bdf7ccbada206a967d427039553, which turns out to have been a misguided effort. We can't really support this in a partitioning hierarchy after all for exactly the reasons stated in the documentation removed by that commit. It's still possible to use ON CONFLICT .. DO NOTHING (or for that matter ON CONFLICT .. DO UPDATE) on individual partitions if desired, but but to allow this on a partitioned table implies that we have some way of evaluating uniqueness across the whole partitioning hierarchy, which is false. Shinoda Noriyoshi noticed that the old code was crashing (which we could fix, though not in a nice way) and Amit Langote realized that this was indicative of a fundamental problem with the commit being reverted here. Discussion: http://postgr.es/m/ff3dc21d-7204-c09c-50ac-cf11a8c45c81@lab.ntt.co.jp
2017-03-31Don't allocate storage for partitioned tables.Robert Haas
Also, don't allow setting reloptions on them, since that would have no effect given the lack of storage. The patch does this by introducing a new reloption kind for which there are currently no reloptions -- we might have some in the future -- so it adjusts parseRelOptions to handle that case correctly. Bumped catversion. System catalogs that contained reloptions for partitioned tables are no longer valid; plus, there are now fewer physical files on disk, which is not technically a catalog change but still a good reason to re-initdb. Amit Langote, reviewed by Maksim Milyutin and Kyotaro Horiguchi and revised a bit by me. Discussion: http://postgr.es/m/20170331.173326.212311140.horiguchi.kyotaro@lab.ntt.co.jp
2017-03-31postgres_fdw: Teach IMPORT FOREIGN SCHEMA about partitioning.Robert Haas
Don't import partitions. Do import partitioned tables which are not themselves partitions. Report by Stephen Frost. Design and patch by Michael Paquier, reviewed by Amit Langote. Documentation revised by me. Discussion: http://postgr.es/m/20170309141531.GD9812@tamriel.snowman.net
2017-03-31Full Text Search support for json and jsonbAndrew Dunstan
The new functions are ts_headline() and to_tsvector. Dmitry Dolgov, edited and documented by me.
2017-03-30Fix broken markup.Tom Lane
Per buildfarm.
2017-03-30Default monitoring rolesSimon Riggs
Three nologin roles with non-overlapping privs are created by default * pg_read_all_settings - read all GUCs. * pg_read_all_stats - pg_stat_*, pg_database_size(), pg_tablespace_size() * pg_stat_scan_tables - may lock/scan tables Top level role - pg_monitor includes all of the above by default, plus others Author: Dave Page Reviewed-by: Stephen Frost, Robert Haas, Peter Eisentraut, Simon Riggs
2017-03-30Support \if ... \elif ... \else ... \endif in psql scripting.Tom Lane
This patch adds nestable conditional blocks to psql. The control structure feature per se is complete, but the boolean expressions understood by \if and \elif are pretty primitive; basically, after variable substitution and backtick expansion, the result has to be "true" or "false" or one of the other standard spellings of a boolean value. But that's enough for many purposes, since you can always do the heavy lifting on the server side; and we can extend it later. Along the way, pay down some of the technical debt that had built up around psql/command.c: * Refactor exec_command() into a function per command, instead of being a 1500-line monstrosity. This makes the file noticeably longer because of repetitive function header/trailer overhead, but it seems much more readable. * Teach psql_get_variable() and psqlscanslash.l to suppress variable substitution and backtick expansion on the basis of the conditional stack state, thereby allowing removal of the OT_NO_EVAL kluge. * Fix the no-doubt-once-expedient hack of sometimes silently substituting mainloop.c's previous_buf for query_buf when calling HandleSlashCmds. (It's a bit remarkable that commands like \r worked at all with that.) Recall of a previous query is now done explicitly in the slash commands where that should happen. Corey Huinker, reviewed by Fabien Coelho, further hacking by me Discussion: https://postgr.es/m/CADkLM=c94OSRTnat=LX0ivNq4pxDNeoomFfYvBKM5N_xfmLtAA@mail.gmail.com
2017-03-30Try to fix xml docs build broken in 5ded4bd.Andres Freund
Apparently the sgml to xml conversion treats non-closed <para>s differently than jade does.
2017-03-31Simplify the example of VACUUM in documentation.Fujii Masao
Previously a detailed activity report by VACUUM VERBOSE ANALYZE was described as an example of VACUUM in docs. But it had been obsolete for a long time. For example, commit feb4f44d296b88b7f0723f4a4f3945a371276e0b updated the content of that activity report in 2003, but we had forgotten to update the example. So basically we need to update the example. But since no one cared about the details of VACUUM output and complained about that mistake for such long time, per discussion on hackers, we decided to get rid of the detailed activity report from the example and simplify it. Back-patch to all supported versions. Reported by Masahiko Sawada, patch by me. Discussion: https://postgr.es/m/CAD21AoAGA2pB3p-CWmTkxBsbkZS1bcDGBLcYVcvcDxspG_XAfA@mail.gmail.com
2017-03-30Remove support for version-0 calling conventions.Andres Freund
The V0 convention is failure prone because we've so far assumed that a function is V0 if PG_FUNCTION_INFO_V1 is missing, leading to crashes if a function was coded against the V1 interface. V0 doesn't allow proper NULL, SRF and toast handling. V0 doesn't offer features that V1 doesn't. Thus remove V0 support and obsolete fmgr README contents relating to it. Author: Andres Freund, with contributions by Peter Eisentraut & Craig Ringer Reviewed-By: Peter Eisentraut, Craig Ringer Discussion: https://postgr.es/m/20161208213441.k3mbno4twhg2qf7g@alap3.anarazel.de
2017-03-28doc: Mention --enable-tap-tests in regression test chapterPeter Eisentraut
Reported-by: Jeff Janes <jeff.janes@gmail.com>
2017-03-28Altering default privileges on schemasTeodor Sigaev
Extend ALTER DEFAULT PRIVILEGES command to schemas. Author: Matheus Oliveira Reviewed-by: Petr Jelínek, Ashutosh Sharma https://commitfest.postgresql.org/13/887/
2017-03-28Cleanup slots during drop databaseSimon Riggs
Automatically drop all logical replication slots associated with a database when the database is dropped. Previously we threw an ERROR if a slot existed. Now we throw ERROR only if a slot is active in the database being dropped. Craig Ringer
2017-03-27Show ignored constants as "$N" rather than "?" in pg_stat_statements.Tom Lane
The trouble with the original choice here is that "?" is a valid (and indeed used) operator name, so that you could end up with ambiguous statement texts like "SELECT ? ? ?". With this patch, you instead see "SELECT $1 ? $2", which seems significantly more readable. The numbers used for this purpose begin after the last actual $N parameter in the particular query. The conflict with external parameters has its own potential for confusion of course, but it was agreed to be an improvement over the previous behavior. Lukas Fittl Discussion: https://postgr.es/m/CAP53PkxeaCuwYmF-A4J5z2-qk5fYFo5_NH3gpXGJJBxv1DMwEw@mail.gmail.com
2017-03-27doc: Improve rendering of notes/cautions using XSL-FOPeter Eisentraut
Center title and put a border around it, like the output that the DSSSL version gave.
2017-03-27doc: Fix oldhtml/old PDF build againPeter Eisentraut
Commit e259e1f748c7a6d67e307a90d6c27b8ab8b90df8 was faulty and created some broken output. This one fixes it better.
2017-03-27PL/Python: Add cursor and execute methods to plan objectPeter Eisentraut
Instead of plan = plpy.prepare(...) res = plpy.execute(plan, ...) you can now write plan = plpy.prepare(...) res = plan.execute(...) or even res = plpy.prepare(...).execute(...) and similarly for the cursor() method. This is more in object oriented style, and makes the hybrid nature of the existing execute() function less confusing. Reviewed-by: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2017-03-27Improve documentation of how NOT NULL works with partitioning.Robert Haas
Amit Langote Discussion: http://postgr.es/m/a6f99cdb-21e7-1d65-1381-91f2cfa156e2@lab.ntt.co.jp
2017-03-27Tidy up the CREATE TABLE documentation for partitioning.Robert Haas
Remove some <note> tags that make this too "loud". Fix some typos. Amit Langote, with a few minor corrections by me Discussion: http://postgr.es/m/a6f99cdb-21e7-1d65-1381-91f2cfa156e2@lab.ntt.co.jp
2017-03-27Allow ON CONFLICT .. DO NOTHING on a partitioned table.Robert Haas
ON CONFLICT .. DO UPDATE still doesn't work, for lack of a way of enforcing uniqueness across partitions, but we can still allow this case. Amit Langote, per discussion with Peter Geoghegan. Additional wordsmithing by me. Discussion: http://postgr.es/m/CAA-aLv7Z4uygtq-Q5CvDi9Y=VZxUyEnuWjL=EwCfOof=L04hgg@mail.gmail.com
2017-03-27Change default of log_directory to 'log'Peter Eisentraut
The previous default 'pg_log' might have indicated by its "pg_" prefix that it is an internal system directory. The new default is more in line with the typical naming of directories with user-facing log files. Together with the renaming of pg_clog and pg_xlog, this should clear up that difference. Author: Andreas Karlsson <andreas@proxel.se>
2017-03-27Clean up Perl code according to perlcriticPeter Eisentraut
Fix all perlcritic warnings of severity level 5, except in src/backend/utils/Gen_dummy_probes.pl, which is automatically generated. Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2017-03-26Show more processes in pg_stat_activity.Robert Haas
Previously, auxiliary processes and background workers not connected to a database (such as the logical replication launcher) weren't shown. Include them, so that we can see the associated wait state information. Add a new column to identify the processes type, so that people can filter them out easily using SQL if they wish. Before this patch was written, there was discussion about whether we should expose this information in a separate view, so as to avoid contaminating pg_stat_activity with things people might not want to see. But putting everything in pg_stat_activity was a more popular choice, so that's what the patch does. Kuntal Ghosh, reviewed by Amit Langote and Michael Paquier. Some revisions and bug fixes by me. Discussion: http://postgr.es/m/CA+TgmoYES5nhkEGw9nZXU8_FhA8XEm8NTm3-SO+3ML1B81Hkww@mail.gmail.com
2017-03-26doc: Fix oldhtml/old PDF buildPeter Eisentraut
"xref to REFSECT1 unsupported" with the DSSSL stylesheets. Reported-by: Devrim Gündüz <devrim@gunduz.org>
2017-03-26doc: Clean up bibliography rendering for XSLTPeter Eisentraut
In the DSSSL stylesheets, we had an extensive customization of the bibliography rendering. Since the bibliography isn't that used much, it doesn't seem worth doing an elaborate porting of that to XSLT. So this just moves some things around, removes some unused things, and does some minimal XSLT stylesheet customizations to make things look clean.
2017-03-25Report catalog_xmin separately in hot_standby_feedbackSimon Riggs
If the upstream walsender is using a physical replication slot, store the catalog_xmin in the slot's catalog_xmin field. If the upstream doesn't use a slot and has only a PGPROC entry behaviour doesn't change, as we store the combined xmin and catalog_xmin in the PGPROC entry. Author: Craig Ringer
2017-03-25Remove ICU tests from default runPeter Eisentraut
These tests require the test database to be in UTF8 encoding. Until there is a better solution, take them out of the default test set and treat them like the existing collate.linux.utf8 test, meaning it has to be selected manually.
2017-03-24Add COMMENT and SECURITY LABEL support for publications and subscriptionsPeter Eisentraut
2017-03-25Make VACUUM VERBOSE report the number of skipped frozen pages.Fujii Masao
Previously manual VACUUM did not report the number of skipped frozen pages even when VERBOSE option is specified. But this information is helpful to monitor the VACUUM activity, and also autovacuum reports that number in the log file when the condition of log_autovacuum_min_duration is met. This commit changes VACUUM VERBOSE so that it reports the number of frozen pages that it skips. Author: Masahiko Sawada Reviewed-by: Yugo Nagata and Jim Nasby Discussion: http://postgr.es/m/CAD21AoDZQKCxo0L39Mrq08cONNkXQKXuh=2DP1Q8ebmt35SoaA@mail.gmail.com
2017-03-24Implement multivariate n-distinct coefficientsAlvaro Herrera
Add support for explicitly declared statistic objects (CREATE STATISTICS), allowing collection of statistics on more complex combinations that individual table columns. Companion commands DROP STATISTICS and ALTER STATISTICS ... OWNER TO / SET SCHEMA / RENAME are added too. All this DDL has been designed so that more statistic types can be added later on, such as multivariate most-common-values and multivariate histograms between columns of a single table, leaving room for permitting columns on multiple tables, too, as well as expressions. This commit only adds support for collection of n-distinct coefficient on user-specified sets of columns in a single table. This is useful to estimate number of distinct groups in GROUP BY and DISTINCT clauses; estimation errors there can cause over-allocation of memory in hashed aggregates, for instance, so it's a worthwhile problem to solve. A new special pseudo-type pg_ndistinct is used. (num-distinct estimation was deemed sufficiently useful by itself that this is worthwhile even if no further statistic types are added immediately; so much so that another version of essentially the same functionality was submitted by Kyotaro Horiguchi: https://postgr.es/m/20150828.173334.114731693.horiguchi.kyotaro@lab.ntt.co.jp though this commit does not use that code.) Author: Tomas Vondra. Some code rework by Álvaro. Reviewed-by: Dean Rasheed, David Rowley, Kyotaro Horiguchi, Jeff Janes, Ideriha Takeshi Discussion: https://postgr.es/m/543AFA15.4080608@fuzzy.cz https://postgr.es/m/20170320190220.ixlaueanxegqd5gr@alvherre.pgsql
2017-03-24Add a txid_status function.Robert Haas
If your connection to the database server is lost while a COMMIT is in progress, it may be difficult to figure out whether the COMMIT was successful or not. This function will tell you, provided that you don't wait too long to ask. It may be useful in other situations, too. Craig Ringer, reviewed by Simon Riggs and by me Discussion: http://postgr.es/m/CAMsr+YHQiWNEi0daCTboS40T+V5s_+dst3PYv_8v2wNVH+Xx4g@mail.gmail.com
2017-03-24Allow SCRAM authentication, when pg_hba.conf says 'md5'.Heikki Linnakangas
If a user has a SCRAM verifier in pg_authid.rolpassword, there's no reason we cannot attempt to perform SCRAM authentication instead of MD5. The worst that can happen is that the client doesn't support SCRAM, and the authentication will fail. But previously, it would fail for sure, because we would not even try. SCRAM is strictly more secure than MD5, so there's no harm in trying it. This allows for a more graceful transition from MD5 passwords to SCRAM, as user passwords can be changed to SCRAM verifiers incrementally, without changing pg_hba.conf. Refactor the code in auth.c to support that better. Notably, we now have to look up the user's pg_authid entry before sending the password challenge, also when performing MD5 authentication. Also simplify the concept of a "doomed" authentication. Previously, if a user had a password, but it had expired, we still performed SCRAM authentication (but always returned error at the end) using the salt and iteration count from the expired password. Now we construct a fake salt, like we do when the user doesn't have a password or doesn't exist at all. That simplifies get_role_password(), and we can don't need to distinguish the "user has expired password", and "user does not exist" cases in auth.c. On second thoughts, also rename uaSASL to uaSCRAM. It refers to the mechanism specified in pg_hba.conf, and while we use SASL for SCRAM authentication at the protocol level, the mechanism should be called SCRAM, not SASL. As a comparison, we have uaLDAP, even though it looks like the plain 'password' authentication at the protocol level. Discussion: https://www.postgresql.org/message-id/6425.1489506016@sss.pgh.pa.us Reviewed-by: Michael Paquier
2017-03-23doc: add missing closing 'sect3' tag for ICU patchBruce Momjian
2017-03-23ICU supportPeter Eisentraut
Add a column collprovider to pg_collation that determines which library provides the collation data. The existing choices are default and libc, and this adds an icu choice, which uses the ICU4C library. The pg_locale_t type is changed to a union that contains the provider-specific locale handles. Users of locale information are changed to look into that struct for the appropriate handle to use. Also add a collversion column that records the version of the collation when it is created, and check at run time whether it is still the same. This detects potentially incompatible library upgrades that can corrupt indexes and other structures. This is currently only supported by ICU-provided collations. initdb initializes the default collation set as before from the `locale -a` output but also adds all available ICU locales with a "-x-icu" appended. Currently, ICU-provided collations can only be explicitly named collations. The global database locales are still always libc-provided. ICU support is enabled by configure --with-icu. Reviewed-by: Thomas Munro <thomas.munro@enterprisedb.com> Reviewed-by: Andreas Karlsson <andreas@proxel.se>
2017-03-23Track the oldest XID that can be safely looked up in CLOG.Robert Haas
This provides infrastructure for looking up arbitrary, user-supplied XIDs without a risk of scary-looking failures from within the clog module. Normally, the oldest XID that can be safely looked up in CLOG is the same as the oldest XID that can reused without causing wraparound, and the latter is already tracked. However, while truncation is in progress, the values are different, so we must keep track of them separately. Craig Ringer, reviewed by Simon Riggs and by me. Discussion: http://postgr.es/m/CAMsr+YHQiWNEi0daCTboS40T+V5s_+dst3PYv_8v2wNVH+Xx4g@mail.gmail.com
2017-03-23Remove createlang and droplangPeter Eisentraut
They have been deprecated since PostgreSQL 9.1. Reviewed-by: Magnus Hagander <magnus@hagander.net> Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2017-03-23Replication lag tracking for walsendersSimon Riggs
Adds write_lag, flush_lag and replay_lag cols to pg_stat_replication. Implements a lag tracker module that reports the lag times based upon measurements of the time taken for recent WAL to be written, flushed and replayed and for the sender to hear about it. These times represent the commit lag that was (or would have been) introduced by each synchronous commit level, if the remote server was configured as a synchronous standby. For an asynchronous standby, the replay_lag column approximates the delay before recent transactions became visible to queries. If the standby server has entirely caught up with the sending server and there is no more WAL activity, the most recently measured lag times will continue to be displayed for a short time and then show NULL. Physical replication lag tracking is automatic. Logical replication tracking is possible but is the responsibility of the logical decoding plugin. Tracking is a private module operating within each walsender individually, with values reported to shared memory. Module not used outside of walsender. Design and code is good enough now to commit - kudos to the author. In many ways a difficult topic, with important and subtle behaviour so this shoudl be expected to generate discussion and multiple open items: Test now! Author: Thomas Munro, following designs by Fujii Masao and Simon Riggs Review: Simon Riggs, Ian Barwick and Craig Ringer
2017-03-23Logical replication support for initial data copyPeter Eisentraut
Add functionality for a new subscription to copy the initial data in the tables and then sync with the ongoing apply process. For the copying, add a new internal COPY option to have the COPY source data provided by a callback function. The initial data copy works on the subscriber by receiving COPY data from the publisher and then providing it locally into a COPY that writes to the destination table. A WAL receiver can now execute full SQL commands. This is used here to obtain information about tables and publications. Several new options were added to CREATE and ALTER SUBSCRIPTION to control whether and when initial table syncing happens. Change pg_dump option --no-create-subscription-slots to --no-subscription-connect and use the new CREATE SUBSCRIPTION ... NOCONNECT option for that. Author: Petr Jelinek <petr.jelinek@2ndquadrant.com> Tested-by: Erik Rijkers <er@xs4all.nl>
2017-03-22Expose waitforarchive option through pg_stop_backup()Stephen Frost
Internally, we have supported the option to either wait for all of the WAL associated with a backup to be archived, or to return immediately. This option is useful to users of pg_stop_backup() as well, when they are reading the stop backup record position and checking that the WAL they need has been archived independently. This patch adds an additional, optional, argument to pg_stop_backup() which allows the user to indicate if they wish to wait for the WAL to be archived or not. The default matches current behavior, which is to wait. Author: David Steele, with some minor changes, doc updates by me. Reviewed by: Takayuki Tsunakawa, Fujii Masao Discussion: https://postgr.es/m/758e3fd1-45b4-5e28-75cd-e9e7f93a4c02@pgmasters.net