Avoid some zero-divide hazards in the planner.

Although I think on all modern machines floating division by zero
results in Infinity not SIGFPE, we still don't want infinities
running around in the planner's costing estimates; too much risk
of that leading to insane behavior.

grouping_planner() failed to consider the possibility that final_rel
might be known dummy and hence have zero rowcount.  (I wonder if it
would be better to set a rows estimate of 1 for dummy relations?
But at least in the back branches, changing this convention seems
like a bad idea, so I'll leave that for another day.)

Make certain that get_variable_numdistinct() produces a nonzero result.
The case that can be shown to be broken is with stadistinct < 0.0 and
small ntuples; we did not prevent the result from rounding to zero.
For good luck I applied clamp_row_est() to all the nonconstant return
values.

In ExecChooseHashTableSize(), Assert that we compute positive nbuckets
and nbatch.  I know of no reason to think this isn't the case, but it
seems like a good safety check.

Per reports from Piotr Stefaniak.  Back-patch to all active branches.

2 files changed, 6 insertions, 3 deletions

diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 16d3af32ed0..6647640b385 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -356,7 +356,7 @@ query_planner(PlannerInfo *root, List *tlist,
 		 * can be divided by the number of tuples.
 		 */
 		if (tuple_fraction >= 1.0)
-			tuple_fraction /= final_rel->rows;
+			tuple_fraction /= clamp_row_est(final_rel->rows);
 	}
 
 	/*
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 0bb8d20d8e6..300fbcec163 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1295,11 +1295,14 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
 		/*
 		 * Extract rowcount and width estimates for possible use in grouping
 		 * decisions.  Beware here of the possibility that
-		 * cheapest_path->parent is NULL (ie, there is no FROM clause).
+		 * cheapest_path->parent is NULL (ie, there is no FROM clause).  Also,
+		 * if the final rel has been proven dummy, its rows estimate will be
+		 * zero; clamp it to one to avoid zero-divide in subsequent
+		 * calculations.
 		 */
 		if (cheapest_path->parent)
 		{
-			path_rows = cheapest_path->parent->rows;
+			path_rows = clamp_row_est(cheapest_path->parent->rows);
 			path_width = cheapest_path->parent->width;
 		}
 		else