From b4570d33aa045df330bb325ba8a2cbf02266a555 Mon Sep 17 00:00:00 2001 From: Tom Lane Date: Mon, 16 Mar 2020 21:05:28 -0400 Subject: Avoid holding a directory FD open across assorted SRF calls. This extends the fixes made in commit 085b6b667 to other SRFs with the same bug, namely pg_logdir_ls(), pgrowlocks(), pg_timezone_names(), pg_ls_dir(), and pg_tablespace_databases(). Also adjust various comments and documentation to warn against expecting to clean up resources during a ValuePerCall SRF's final call. Back-patch to all supported branches, since these functions were all born broken. Justin Pryzby, with cosmetic tweaks by me Discussion: https://postgr.es/m/20200308173103.GC1357@telsasoft.com --- doc/src/sgml/xfunc.sgml | 96 +++++++++++++++++++++++++++++++------------------ 1 file changed, 61 insertions(+), 35 deletions(-) (limited to 'doc/src') diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml index a91f8871191..6350f92f866 100644 --- a/doc/src/sgml/xfunc.sgml +++ b/doc/src/sgml/xfunc.sgml @@ -2812,22 +2812,50 @@ HeapTupleGetDatum(HeapTuple tuple) Returning Sets - There is also a special API that provides support for returning - sets (multiple rows) from a C-language function. A set-returning - function must follow the version-1 calling conventions. Also, - source files must include funcapi.h, as - above. - - - - A set-returning function (SRF) is called - once for each item it returns. The SRF must - therefore save enough state to remember what it was doing and - return the next item on each call. - The structure FuncCallContext is provided to help - control this process. Within a function, fcinfo->flinfo->fn_extra - is used to hold a pointer to FuncCallContext - across calls. + C-language functions have two options for returning sets (multiple + rows). In one method, called ValuePerCall + mode, a set-returning function is called repeatedly (passing the same + arguments each time) and it returns one new row on each call, until + it has no more rows to return and signals that by returning NULL. + The set-returning function (SRF) must therefore + save enough state across calls to remember what it was doing and + return the correct next item on each call. + In the other method, called Materialize mode, + a SRF fills and returns a tuplestore object containing its + entire result; then only one call occurs for the whole result, and + no inter-call state is needed. + + + + When using ValuePerCall mode, it is important to remember that the + query is not guaranteed to be run to completion; that is, due to + options such as LIMIT, the executor might stop + making calls to the set-returning function before all rows have been + fetched. This means it is not safe to perform cleanup activities in + the last call, because that might not ever happen. It's recommended + to use Materialize mode for functions that need access to external + resources, such as file descriptors. + + + + The remainder of this section documents a set of helper macros that + are commonly used (though not required to be used) for SRFs using + ValuePerCall mode. Additional details about Materialize mode can be + found in src/backend/utils/fmgr/README. Also, + the contrib modules in + the PostgreSQL source distribution contain + many examples of SRFs using both ValuePerCall and Materialize mode. + + + + To use the ValuePerCall support macros described here, + include funcapi.h. These macros work with a + structure FuncCallContext that contains the + state that needs to be saved across calls. Within the calling + SRF, fcinfo->flinfo->fn_extra is used to + hold a pointer to FuncCallContext across + calls. The macros automatically fill that field on first use, + and expect to find the same pointer there on subsequent uses. typedef struct FuncCallContext { @@ -2892,29 +2920,26 @@ typedef struct FuncCallContext - An SRF uses several functions and macros that - automatically manipulate the FuncCallContext - structure (and expect to find it via fn_extra). Use: + The macros to be used by an SRF using this + infrastructure are: SRF_IS_FIRSTCALL() - to determine if your function is being called for the first or a - subsequent time. On the first call (only) use: + Use this to determine if your function is being called for the first or a + subsequent time. On the first call (only), call: SRF_FIRSTCALL_INIT() to initialize the FuncCallContext. On every function call, - including the first, use: + including the first, call: SRF_PERCALL_SETUP() - to properly set up for using the FuncCallContext - and clearing any previously returned data left over from the - previous pass. + to set up for using the FuncCallContext. - If your function has data to return, use: + If your function has data to return in the current call, use: SRF_RETURN_NEXT(funcctx, result) @@ -2938,7 +2963,14 @@ SRF_RETURN_DONE(funcctx) multi_call_memory_ctx is a suitable location for any data that needs to survive until the SRF is finished running. In most cases, this means that you should switch into - multi_call_memory_ctx while doing the first-call setup. + multi_call_memory_ctx while doing the + first-call setup. + Use funcctx->user_fctx to hold a pointer to + any such cross-call data structures. + (Data you allocate + in multi_call_memory_ctx will go away + automatically when the query ends, so it is not necessary to free + that data manually, either.) @@ -2995,8 +3027,8 @@ my_set_returning_function(PG_FUNCTION_ARGS) } else { - /* Here we are done returning items and just need to clean up: */ - user code + /* Here we are done returning items, so just report that fact. */ + /* (Resist the temptation to put cleanup code here.) */ SRF_RETURN_DONE(funcctx); } } @@ -3118,12 +3150,6 @@ CREATE OR REPLACE FUNCTION retcomposite(IN integer, IN integer, Notice that in this method the output type of the function is formally an anonymous record type. - - - The directory contrib/tablefunc - module in the source distribution contains more examples of - set-returning functions. - -- cgit v1.2.3