diff options
| author | Tomas Vondra <tomas.vondra@postgresql.org> | 2019-10-01 14:13:44 +0200 |
|---|---|---|
| committer | Tomas Vondra <tomas.vondra@postgresql.org> | 2019-10-01 14:28:28 +0200 |
| commit | 11a078cf87ffb611d19c7dec6df68b41084ad9c9 (patch) | |
| tree | cdb79b0578ea99a1c090d2704d4d5f0aa9266dc9 /src/include | |
| parent | 002962dc7293043126561b0d0df79d6c76251804 (diff) | |
Optimize partial TOAST decompression
Commit 4d0e994eed added support for partial TOAST decompression, so the
decompression is interrupted after producing the requested prefix. For
prefix and slices near the beginning of the entry, this may saves a lot
of decompression work.
That however only deals with decompression - the whole compressed entry
was still fetched and re-assembled, even though the compression used
only a small fraction of it. This commit improves that by computing how
much compressed data may be needed to decompress the requested prefix,
and then fetches only the necessary part.
We always need to fetch a bit more compressed data than the requested
(uncompressed) prefix, because the prefix may not be compressible at all
and pglz itself adds a bit of overhead. That means this optimization is
most effective when the requested prefix is much smaller than the whole
compressed entry.
Author: Binguo Bao
Reviewed-by: Andrey Borodin, Tomas Vondra, Paul Ramsey
Discussion: https://www.postgresql.org/message-id/flat/CAL-OGkthU9Gs7TZchf5OWaL-Gsi=hXqufTxKv9qpNG73d5na_g@mail.gmail.com
Diffstat (limited to 'src/include')
| -rw-r--r-- | src/include/access/toast_internals.h | 1 | ||||
| -rw-r--r-- | src/include/common/pg_lzcompress.h | 2 |
2 files changed, 3 insertions, 0 deletions
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h index 494b07a4b11..b8703d8c94e 100644 --- a/src/include/access/toast_internals.h +++ b/src/include/access/toast_internals.h @@ -31,6 +31,7 @@ typedef struct toast_compress_header */ #define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header)) #define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize) +#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE(ptr) - TOAST_COMPRESS_HDRSZ) #define TOAST_COMPRESS_RAWDATA(ptr) \ (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ) #define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \ diff --git a/src/include/common/pg_lzcompress.h b/src/include/common/pg_lzcompress.h index 555576436cc..3e53fbe97bd 100644 --- a/src/include/common/pg_lzcompress.h +++ b/src/include/common/pg_lzcompress.h @@ -87,5 +87,7 @@ extern int32 pglz_compress(const char *source, int32 slen, char *dest, const PGLZ_Strategy *strategy); extern int32 pglz_decompress(const char *source, int32 slen, char *dest, int32 rawsize, bool check_complete); +extern int32 pglz_maximum_compressed_size(int32 rawsize, + int32 total_compressed_size); #endif /* _PG_LZCOMPRESS_H_ */ |