Fix subquery pullup to wrap a PlaceHolderVar around the entire RowExpr

that's generated for a whole-row Var referencing the subquery, when the
subquery is in the nullable side of an outer join.  The previous coding
instead put PlaceHolderVars around the elements of the RowExpr.  The effect
was that when the outer join made the subquery outputs go to null, the
whole-row Var produced ROW(NULL,NULL,...) rather than just NULL.  There
are arguments afoot about whether those things ought to be semantically
indistinguishable, but for the moment they are not entirely so, and the
planner needs to take care that its machinations preserve the difference.
Per bug #5025.

Making this feasible required refactoring ResolveNew() to allow more caller
control over what is substituted for a Var.  I chose to make ResolveNew()
a wrapper around a new general-purpose function replace_rte_variables().
I also fixed the ancient bogosity that ResolveNew might fail to set
a query's hasSubLinks field after inserting a SubLink in it.  Although
all current callers make sure that happens anyway, we've had bugs of that
sort before, and it seemed like a good time to install a proper solution.

Back-patch to 8.4.  The problem can be demonstrated clear back to 8.0,
but the fix would be too invasive in earlier branches; not to mention
that people may be depending on the subtly-incorrect behavior.  The
8.4 series is new enough that fixing this probably won't cause complaints,
but it might in older branches.  Also, 8.4 shows the incorrect behavior
in more cases than older branches do, because it is able to flatten
subqueries in more cases.

1 files changed, 26 insertions, 2 deletions

diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index 51993f3a78b..8247028f21b 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -7,7 +7,7 @@
  * Portions Copyright (c) 1996-2009, PostgreSQL Global Development Group
  * Portions Copyright (c) 1994, Regents of the University of California
  *
- * $PostgreSQL: pgsql/src/include/rewrite/rewriteManip.h,v 1.50 2009/06/11 14:49:12 momjian Exp $
+ * $PostgreSQL: pgsql/src/include/rewrite/rewriteManip.h,v 1.50.2.1 2009/09/02 17:52:33 tgl Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -17,6 +17,21 @@
 #include "nodes/parsenodes.h"
 
 
+typedef struct replace_rte_variables_context replace_rte_variables_context;
+
+typedef Node * (*replace_rte_variables_callback) (Var *var,
+									replace_rte_variables_context *context);
+
+struct replace_rte_variables_context
+{
+	replace_rte_variables_callback callback;	/* callback function */
+	void	   *callback_arg;		/* context data for callback function */
+	int			target_varno;		/* RTE index to search for */
+	int			sublevels_up;		/* (current) nesting depth */
+	bool		inserted_sublink;	/* have we inserted a SubLink? */
+};
+
+
 extern void OffsetVarNodes(Node *node, int offset, int sublevels_up);
 extern void ChangeVarNodes(Node *node, int old_varno, int new_varno,
 			   int sublevels_up);
@@ -42,8 +57,17 @@ extern bool checkExprHasAggs(Node *node);
 extern bool checkExprHasWindowFuncs(Node *node);
 extern bool checkExprHasSubLink(Node *node);
 
+extern Node *replace_rte_variables(Node *node,
+					  int target_varno, int sublevels_up,
+					  replace_rte_variables_callback callback,
+					  void *callback_arg,
+					  bool *outer_hasSubLinks);
+extern Node *replace_rte_variables_mutator(Node *node,
+							  replace_rte_variables_context *context);
+
 extern Node *ResolveNew(Node *node, int target_varno, int sublevels_up,
 		   RangeTblEntry *target_rte,
-		   List *targetlist, int event, int update_varno);
+		   List *targetlist, int event, int update_varno,
+		   bool *outer_hasSubLinks);
 
 #endif   /* REWRITEMANIP_H */