From 5dc92b844e680c54a7ecd68de0ba53c949c3d605 Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Fri, 29 Mar 2019 08:25:20 +0100 Subject: REINDEX CONCURRENTLY This adds the CONCURRENTLY option to the REINDEX command. A REINDEX CONCURRENTLY on a specific index creates a new index (like CREATE INDEX CONCURRENTLY), then renames the old index away and the new index in place and adjusts the dependencies, and then drops the old index (like DROP INDEX CONCURRENTLY). The REINDEX command also has the capability to run its other variants (TABLE, DATABASE) with the CONCURRENTLY option (but not SYSTEM). The reindexdb command gets the --concurrently option. Author: Michael Paquier, Andreas Karlsson, Peter Eisentraut Reviewed-by: Andres Freund, Fujii Masao, Jim Nasby, Sergei Kornilov Discussion: https://www.postgresql.org/message-id/flat/60052986-956b-4478-45ed-8bd119e9b9cf%402ndquadrant.com#74948a1044c56c5e817a5050f554ddee --- doc/src/sgml/mvcc.sgml | 1 + doc/src/sgml/ref/create_index.sgml | 1 + doc/src/sgml/ref/reindex.sgml | 190 ++++++++++++++++++++++++++++++++++++- doc/src/sgml/ref/reindexdb.sgml | 10 ++ 4 files changed, 200 insertions(+), 2 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml index bedd9a008d3..9b7ef8bf095 100644 --- a/doc/src/sgml/mvcc.sgml +++ b/doc/src/sgml/mvcc.sgml @@ -926,6 +926,7 @@ ERROR: could not serialize access due to read/write dependencies among transact Acquired by VACUUM (without ), ANALYZE, CREATE INDEX CONCURRENTLY, + REINDEX CONCURRENTLY, CREATE STATISTICS, and certain ALTER INDEX and ALTER TABLE variants (for full details see and + diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml index 47cef987d48..ccabb330cbf 100644 --- a/doc/src/sgml/ref/reindex.sgml +++ b/doc/src/sgml/ref/reindex.sgml @@ -21,7 +21,7 @@ PostgreSQL documentation -REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } name +REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } [ CONCURRENTLY ] name @@ -68,7 +68,7 @@ REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } CONCURRENTLY option failed, leaving an invalid index. Such indexes are useless but it can be convenient to use REINDEX to rebuild them. Note that - REINDEX will not perform a concurrent build. To build the + REINDEX will not perform a concurrent build on an invalid index. To build the index without interfering with production you should drop the index and reissue the CREATE INDEX CONCURRENTLY command. @@ -151,6 +151,21 @@ REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } + + CONCURRENTLY + + + When this option is used, PostgreSQL will rebuild the + index without taking any locks that prevent concurrent inserts, + updates, or deletes on the table; whereas a standard reindex build + locks out writes (but not reads) on the table until it's done. + There are several caveats to be aware of when using this option + — see . + + + + VERBOSE @@ -241,6 +256,159 @@ REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } + + Rebuilding Indexes Concurrently + + + index + rebuilding concurrently + + + + Rebuilding an index can interfere with regular operation of a database. + Normally PostgreSQL locks the table whose index is rebuilt + against writes and performs the entire index build with a single scan of the + table. Other transactions can still read the table, but if they try to + insert, update, or delete rows in the table they will block until the + index rebuild is finished. This could have a severe effect if the system is + a live production database. Very large tables can take many hours to be + indexed, and even for smaller tables, an index rebuild can lock out writers + for periods that are unacceptably long for a production system. + + + + PostgreSQL supports rebuilding indexes with minimum locking + of writes. This method is invoked by specifying the + CONCURRENTLY option of REINDEX. When this option + is used, PostgreSQL must perform two scans of the table + for each index that needs to be rebuild and in addition it must wait for + all existing transactions that could potentially use the index to + terminate. This method requires more total work than a standard index + rebuild and takes significantly longer to complete as it needs to wait + for unfinished transactions that might modify the index. However, since + it allows normal operations to continue while the index is rebuilt, this + method is useful for rebuilding indexes in a production environment. Of + course, the extra CPU, memory and I/O load imposed by the index rebuild + may slow down other operations. + + + + The following steps occur in a concurrent reindex. Each step is run in a + separate transaction. If there are multiple indexes to be rebuilt, then + each step loops through all the indexes before moving to the next step. + + + + + A new temporary index definition is added into the catalog + pg_index. This definition will be used to replace + the old index. A SHARE UPDATE EXCLUSIVE lock at + session level is taken on the indexes being reindexed as well as its + associated table to prevent any schema modification while processing. + + + + + + A first pass to build the index is done for each new index. Once the + index is built, its flag pg_index.indisready is + switched to true to make ready for inserts, making it + visible to other sessions once the transaction that performed the build + is finished. This step is done in a separate transaction for each + index. + + + + + + Then a second pass is performed to add tuples that were added while the + first pass build was running. This step is also done in a separate + transaction for each index. + + + + + + All the constraints that refer to the index are changed to refer to the + new index definition, and the names of the indexes are changed. At + this point pg_index.indisvalid is switched to + true for the new index and to false for + the old, and a cache invalidation is done so as all the sessions that + referenced the old index are invalidated. + + + + + + The old indexes have pg_index.indisready switched to + false to prevent any new tuple insertions, after waiting + for running queries that might reference the old index to complete. + + + + + + The old indexes are dropped. The SHARE UPDATE + EXCLUSIVE session locks for the indexes and the table ar + released. + + + + + + + If a problem arises while rebuilding the indexes, such as a + uniqueness violation in a unique index, the REINDEX + command will fail but leave behind an invalid new index on top + of the existing one. This index will be ignored for querying purposes + because it might be incomplete; however it will still consume update + overhead. The psql \d command will report + such an index as INVALID: + + +postgres=# \d tab + Table "public.tab" + Column | Type | Modifiers +--------+---------+----------- + col | integer | +Indexes: + "idx" btree (col) + "idx_ccnew" btree (col) INVALID + + + The recommended recovery method in such cases is to drop the invalid index + and try again to perform REINDEX CONCURRENTLY. The + concurrent index created during the processing has a name ending in the + suffix ccnew, or ccold if it is an + old index definition which we failed to drop. Invalid indexes can be + dropped using DROP INDEX, including invalid toast + indexes. + + + + Regular index builds permit other regular index builds on the same table + to occur in parallel, but only one concurrent index build can occur on a + table at a time. In both cases, no other types of schema modification on + the table are allowed meanwhile. Another difference is that a regular + REINDEX TABLE or REINDEX INDEX + command can be performed within a transaction block, but REINDEX + CONCURRENTLY cannot. + + + + REINDEX SYSTEM does not support + CONCURRENTLY since system catalogs cannot be reindexed + concurrently. + + + + Furthermore, indexes for exclusion constraints cannot be reindexed + concurrently. If such an index is named directly in this command, an + error is raised. If a table or database with exclusion constraint indexes + is reindexed concurrently, those indexes will be skipped. (It is possible + to reindex such indexes without the concurrently option.) + + @@ -272,6 +440,14 @@ $ psql broken_db ... broken_db=> REINDEX DATABASE broken_db; broken_db=> \q + + + + Rebuild a table while authorizing read and write operations on involved + relations when performed: + + +REINDEX TABLE CONCURRENTLY my_broken_table; @@ -282,4 +458,14 @@ broken_db=> \q There is no REINDEX command in the SQL standard. + + + See Also + + + + + + + diff --git a/doc/src/sgml/ref/reindexdb.sgml b/doc/src/sgml/ref/reindexdb.sgml index 1273dad8072..cdfac3fe4f9 100644 --- a/doc/src/sgml/ref/reindexdb.sgml +++ b/doc/src/sgml/ref/reindexdb.sgml @@ -118,6 +118,16 @@ PostgreSQL documentation + + + + + Use the CONCURRENTLY option. See for further information. + + + + -- cgit v1.2.3