Fix failure of btree_gin indexscans with "char" type and </<= operators.

As a result of confusion about whether the "char" type is signed or
unsigned, scans for index searches like "col < 'x'" or "col <= 'x'"
would start at the middle of the index not the left end, thus missing
many or all of the entries they should find.  Fortunately, this
is not a symptom of index corruption.  It's only the search logic
that is broken, and we can fix it without unpleasant side-effects.

Per report from Jason Kim.  This has been wrong since btree_gin's
beginning, so back-patch to all supported branches.

Discussion: https://postgr.es/m/20210810001649.htnltbh7c63re42p@jasonk.me

2 files changed, 10 insertions, 3 deletions

diff --git a/contrib/btree_gin/btree_gin.c b/contrib/btree_gin/btree_gin.c
index 2ecf7a2d87c..c7b217288f2 100644
--- a/contrib/btree_gin/btree_gin.c
+++ b/contrib/btree_gin/btree_gin.c
@@ -358,7 +358,7 @@ GIN_SUPPORT(bpchar, true, leftmostvalue_text, bpcharcmp)
 static Datum
 leftmostvalue_char(void)
 {
-	return CharGetDatum(SCHAR_MIN);
+	return CharGetDatum(0);
 }
 
 GIN_SUPPORT(char, false, leftmostvalue_char, btcharcmp)
diff --git a/contrib/btree_gin/expected/char.out b/contrib/btree_gin/expected/char.out
index 09e0315de0a..6563546d3ce 100644
--- a/contrib/btree_gin/expected/char.out
+++ b/contrib/btree_gin/expected/char.out
@@ -7,12 +7,19 @@ CREATE INDEX idx_char ON test_char USING gin (i);
 SELECT * FROM test_char WHERE i<'d'::"char" ORDER BY i;
  i 
 ---
-(0 rows)
+ a
+ b
+ c
+(3 rows)
 
 SELECT * FROM test_char WHERE i<='d'::"char" ORDER BY i;
  i 
 ---
-(0 rows)
+ a
+ b
+ c
+ d
+(4 rows)
 
 SELECT * FROM test_char WHERE i='d'::"char" ORDER BY i;
  i