The canonical config file is in src/backend/gpopt/.clang-format (instead
of under the non-existent src/backend/gporca), I've created one (instead
of two) symlink, for GPOPT headers. Care has been taken to repoint the
symlink to the canonical config under gpopt, instead of gpopt as it is
under HEAD.

This is spiritually a cherry-pick of commit 2f7dd76c.
(cherry picked from commit 2f7dd76c)

Orca uses this property for cardinality estimation of joins.
For example, a join predicate foo join bar on foo.a = upper(bar.b)
will have a cardinality estimate similar to foo join bar on foo.a = bar.b.

Other functions, like foo join bar on foo.a = substring(bar.b, 1, 1)
won't be treated that way, since they are more likely to have a greater
effect on join cardinalities.

Since this is specific to ORCA, we use logic in the translator to determine
whether a function or operator is NDV-preserving. Right now, we consider
a very limited set of operators, we may add more at a later time.

This commit enables the MCVs for text related types such as varchar, name etc
to be passed to ORCA so that it can estimate the cardinalities for columns
containing text related types. Prior to this commit, ORCA would estimate the
cardinality to be 40% of the tuples which would cause mis-estimation for
certain queries.
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>

Whenever there are queries involving window functions, ORCA requires the query
to be in a specific form in order to optimize it. Specifically, the query
should only contain window functions and the columns referenced by the window
clause. The algebrizer is responsible for normalizing the query to a specific
form. However the algebrizer did not handle the case when one of the columns
selected was a subquery.

For e.g

select (select b from w3 where a = w1.a) as one,
       row_number() over(partition by w1.a) as two
       from w2, w1;

In this case, the algebrizer would simply pull the subquery to the top level
without fixing the vars for the outer references.This caused the algebrizer to
crash.

This commit fixes the issue by recursing into the subquery and fixing up the
vars.

The corresponding ORCA PR is https://github.com/greenplum-db/gporca/pull/554.

Change the check when translating an ORCA query to a plan.
The old check prohibited ArrayCmp on a btree index. The new
check is similar, except that it allows an ArrayCmp on a
btree index when it is done in a bitmap index probe.

Updated ICG result files and added a new test case.

CMemoryPoolPalloc previously used headers and logic that were only
needed in CMemoryPoolTracker. For each allocation, a fairly large header
was added, which caused memory intensive operations in ORCA to use large
amounts of memory. Now, we only store the size of array allocations in a
header if needed. Otherwise, no header information is needed/stored on
the ORCA side. This reduces the memory utilization for some queries by
30%+. For TPC-DS Q72 on my laptop, peak memory utilization went from 1.1GB to
720MB. This header accounted for ~20MB of the 720MB peak usage in Q72.

This functionality can be enabled with the
`optimizer_use_gpdb_allocators` GUC.

Corresponding ORCA commit: https://github.com/greenplum-db/gporca/commit/d828eed "Simplify CMemoryPool to reduce unnecessary headers and logic".
Co-authored-by: NChris Hajas <chajas@pivotal.io>
Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>

We introduce a new type of memory pool and memory pool manager:
CMemoryPoolPalloc and CMemoryPoolPallocManager

The motivation for this PR is to improve memory allocation/deallocation
performance when using GPDB allocators. Additionally, we would like to
use the GPDB memory allocators by default (change the default for
optimizer_use_gpdb_allocators to on), to prevent ORCA from crashing when
we run out of memory (OOM). However, with the current way of doing
things, doing so would add around 10 % performance overhead to ORCA.

CMemoryPoolPallocManager overrides the default CMemoryPoolManager in
ORCA, and instead creates a CMemoryPoolPalloc memory pool instead of a
CMemoryPoolTracker. In CMemoryPoolPalloc, we now call MemoryContextAlloc
and pfree instead of gp_malloc and gp_free, and we don’t do any memory
accounting.

So where does the performance improvement come from? Previously, we
would (essentially) pass in gp_malloc and gp_free to an underlying
allocation structure (which has been removed on the ORCA side). However,
we would add additional headers and overhead to maintain a list of all
of these allocations. When tearing down the memory pool, we would
iterate through the list of allocations and explicitly free each one. So
we would end up doing overhead on the ORCA side, AND the GPDB side.
However, the overhead on both sides was quite expensive!

If you want to compare against the previous implementation, see the
Allocate and Teardown functions in CMemoryPoolTracker.

With this PR, we improve optimization time by ~15% on average and up to
30-40% on some queries which are memory intensive.

This PR does remove memory accounting in ORCA. This was only enabled
when the optimizer_use_gpdb_allocators GUC was set. By setting
`optimizer_use_gpdb_allocators`, we still capture the memory used when
optimizing a query in ORCA, without the overhead of the memory
accounting framework.

Additionally, Add a top level ORCA context where new contexts are created

The OptimizerMemoryContext is initialized in InitPostgres(). For each
memory pool in ORCA, a new memory context is created in
OptimizerMemoryContext.
Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>
Co-authored-by: NChris Hajas <chajas@pivotal.io>

- The corresponding ORCA PR is : https://github.com/greenplum-db/gporca/pull/533

- Change GUC value OPTIMIZER_UNEXPECTED_FAIL so that we log only unexpected failures.
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>

- Fix "missing prototype" warnings
- Fix "generalized initializer lists are a C++ extension" warning

funcs.cpp:43:1: warning: no previous prototype for function 'DisableXform' [-Wmissing-prototypes]
DisableXform(PG_FUNCTION_ARGS)
^
funcs.cpp:76:1: warning: no previous prototype for function 'EnableXform' [-Wmissing-prototypes]
EnableXform(PG_FUNCTION_ARGS)
^
funcs.cpp:109:1: warning: no previous prototype for function 'LibraryVersion' [-Wmissing-prototypes]
LibraryVersion()
^
funcs.cpp:123:1: warning: no previous prototype for function 'OptVersion' [-Wmissing-prototypes]
OptVersion()
^
4 warnings generated.

CTranslatorDXLToScalar.cpp:730:9: warning: generalized initializer lists are a C++11 extension [-Wc++11-extensions]
        return { .oid_type = inner_type_oid, .type_modifier = type_modifier};

Previously we would not pass the statistics for UUID columns to ORCA. This
would cause cardinality mis-estimation and hence would cause ORCA to pick a bad
plan. This patch fixes the issue by passing in the statistics for UUID columns.

This commit corresponds to the ORCA commit "Implement Full Merge Join"

In GPDB 5, merge join is disabled, but the following changes were made
to continue allowing compilation of GPDB 5 with ORCA.

1. Translator changes for Merge Join.
2. Add IsOpMergeJoinable() and GetMergeJoinOpFamilies()  wrappers.

* Remove reference to deleted header file
* Bump ORCA version to 3.56 for PR 503
* Bump ORCA version to 3.57 for PR 510

The IMemoryPool interface was removed in ORCA to remove an unnecessary
abstraction layer and avoid costly casting.

Corresponding ORCA commit: e64a2b42

Bumps ORCA version to 3.46.0
Authored-by: NChris Hajas <chajas@pivotal.io>

ORCA's algebrizer must first normalize GROUP BYs in a query to a form
usable in ORCA. It must flatten expressions in the project list to
contain only aggregates and grouping columns
For example:

ORGINAL QUERY:
  SELECT * from r where r.a > (SELECT max(c) + min(d)
                               FROM t where r.b = t.e)
NEW QUERY:
  SELECT * from r where r.a > (SELECT x1+x2 as x3
  FROM (SELECT max(c) as x2, min(d) as x2
        FROM t where r.b = t.e) t2)

However this process did not support subqueries in the target list that
may contain outer references, sometimes in other (nested) subqueries. It
also did not support CTEs. All these would produce a normalization error
and fall back.

This commit fixes that by supporting subqueries & CTEs.

It also includes some refactors in related areas:

- Rename IncrLevelsUpInVar to IncrLevelsUpIfOuterRef, to capture it's
  implementation.
- Remove SContextHavingQualMutator, after realizing that it has almost
  the same members as SContextGrpbyPlMutator.
- Use MakeVarInDerivedTable & RunGroupByProjListMutator in
  RunGroupByProjListMutator to reduce code drift.
- Merge RunGroupByProjListMutator & RunHavingQualMutator (see below)

RunGroupByProjListMutator() was implemented at a later date than the
RunHavingQualMutator, and did not handle subqueries and ctes correctly.
After understanding its purpose, I think the functionality of both the
above methods should be exactly the same, since they're trying to
achieve the same goal.

So, this commit just merges the two functions together into a new
function - RunExtractAggregatesMutator. In this process, I also
discovered a bug in the old RunHavingQualMutator, that is now fixed:

  create table fooh1 (a int, b int, c int);
  insert into fooh1 select i%4, i%3, i from generate_series(1,20) i;
  select f1.a + 1 from fooh1 f1 group by f1.a+1 having sum(f1.a+1) + 1 > 20;

Finally, the earlier code deduplicated AGGREFs when possible for HAVING
clauses, but not for GROUP BY target lists when moving them into the
derived query. Not only is that inconsistent, but may also give
incorrect results in case of volatile functions. The executor already
handles de-duplicating AGGREFs in right circumstances, so doing this in
the algebrizer doesn't provide much of a benefit.

Prior to this patch, the MCVs for text columns was not being passed to ORCA.
Hence the cardinality estimation for predicates involving text was inaccurate
and led to sub optimal plans being picked. This patch allows the MCVs to be
passed in to ORCA so that it can now estimate the cardinality using MCVs equal
and not equal operators for text columns.
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>

Via https://github.com/greenplum-db/gporca/pull/400, ORCA will optimize
DML queries by enforcing a gather on segment instead of master, whenever
possible.

Previous to this commit, ORCA always picked the first segment to gather
on while translating the DXL-GatherMotion node to GPDB motion node.

This commit uses GPDB's hash function to select the segment to gather
on, in a round-robin fashion starting with a random segment index. This
will ensure that concurrent DML queries issued via a same session, will
be gathered on different segments to distribute the workload.
Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

I was getting a compiler warning from it:

CTranslatorUtils.cpp: In static member function ‘static gpos::BOOL gpdxl::CTranslatorUtils::IsGroupingColumn(const SortGroupClause*, List*)’:
CTranslatorUtils.cpp:2079:42: warning: self-comparison always evaluates to true [-Wtautological-compare]
   if (sort_group_clause->tleSortGroupRef == sort_group_clause->tleSortGroupRef)
       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

But since the function doesn't seem to be used anywhere, let's just remove
it, instead of trying to fix it.

Commit d334b016 changed the name of this argument, but got the name wrong
in this check. Because it was in a GPOS_BLOCK(), it would only compile,
and throw the erorr, with an assertion-enabled ORCA build.

As part of moving away from Hungarian notation in the GPORCA codebase,
the integration points between GPORCA and GPDB in the translator have
been renamed to the new convention used in GPORCA. The libraries
currently updated to the new notation in GPORCA are Naucrates and GPOS.
The new naming convention is a custom version of common C++ naming
conventions. The style guide for this convention can be found in the
GPORCA repository.

Also bump ORCA version to 2.69.0
Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>
Co-authored-by: NMelanie Plageman <mplageman@pivotal.io>
Co-authored-by: NEkta Khanna <ekhanna@pivotal.io>
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io&gt;&lt;Paste>
Co-authored-by: NDhanashree Kashid <dkashid@pivotal.io>
Co-authored-by: NOmer Arap <oarap@pivotal.io>

When there is no stats available for any table, ORCA was treating it as an
empty table while planning. On the other hand planner is utilizing a guc
`gp_enable_relsize_collection` to obtain the estimated size of the table, but
no other statistics. This commit enables ORCA to have the same behavior as
planner when the guc is set.
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>

Fixes greenplum-db/gporca#358

ORCA has historically ignored type modifiers from databases that support
them, noticeably Postgres and Greenplum. This has led to surprises in a
few cases:

1. The output description over the wire (for Postgres protocol) will
lose the type modifier information, which often meant length. This
surprises code that expects a non-default type modifier, e.g. a JDBC
driver.

2. The executor in some cases -- notably DML -- expects a precise type
modifier. Because ORCA always erases the type modifiers and presents a
default, the executor is forced to find that information elsewhere.

After this commit, ORCA will be aware of type modifiers in table
columns, scalar identifiers, constants, and length-coercion casts.
Signed-off-by: NShreedhar Hardikar <shardikar@pivotal.io>
(cherry picked from commit 2d907526)

In scenarios where pg_statistic contains wrong statistic entry for an
attribute, or when the statistics on a particular attribute are broken,
for e.g the type of elements stored in stavalues<1/2/3> is different
than the actual attribute type or when there are holes in the attribute
numbers due to adding/dropping columns; following two APIs fail because
they relied on the attribute type sent by the caller:

- get_attstatsslot() : Extracts the contents (numbers/frequency array and
values array) of the requested statistic slot (MCV, HISTOGRAM etc). If the
attribute is pass-by-reference or if the attribute is of toastable type
(varlena types)then it returns a copy allocated with palloc()
- free_attstatsslot() : Frees any palloc'd data by get_attstatsslot()

This problem was fixed in upstream 8.3
(8c21b4e9) for get_attstatsslot(),
wherein the actual element type of the array will be used for
deconstructing it rather that using caller passed OID.
free_attstatsslot() still depends on the type oid sent by caller.

However the issue still exists for free_attstatsslot() where it crashes while
freeing the array. The crash happened because the caller sent type OID was of
type TEXT meaning this a varlena type and hence free_attstatsslot() attempted
to free the datum; however due to the broken slot the datums extracted from
values array were of fixed length type such as int. We considered the int value
as memory address and crashed while freeing it.

This commit brings in a following fix from upstream 10 which redesigns
get_attstatsslot()/free_attstatsslot() such than they robust to scenarios like
these.

commit 9aab83fc
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Sat May 13 15:14:39 2017 -0400

    Redesign get_attstatsslot()/free_attstatsslot() for more safety and speed.

    The mess cleaned up in commit da075960 is clear evidence that it's a
    bug hazard to expect the caller of get_attstatsslot()/free_attstatsslot()
    to provide the correct type OID for the array elements in the slot.
    Moreover, we weren't even getting any performance benefit from that,
    since get_attstatsslot() was extracting the real type OID from the array
    anyway.  So we ought to get rid of that requirement; indeed, it would
    make more sense for get_attstatsslot() to pass back the type OID it found,
    in case the caller isn't sure what to expect, which is likely in binary-
    compatible-operator cases.

    Another problem with the current implementation is that if the stats array
    element type is pass-by-reference, we incur a palloc/memcpy/pfree cycle
    for each element.  That seemed acceptable when the code was written because
    we were targeting O(10) array sizes --- but these days, stats arrays are
    almost always bigger than that, sometimes much bigger.  We can save a
    significant number of cycles by doing one palloc/memcpy/pfree of the whole
    array.  Indeed, in the now-probably-common case where the array is toasted,
    that happens anyway so this method is basically free.  (Note: although the
    catcache code will inline any out-of-line toasted values, it doesn't
    decompress them.  At the other end of the size range, it doesn't expand
    short-header datums either.  In either case, DatumGetArrayTypeP would have
    to make a copy.  We do end up using an extra array copy step if the element
    type is pass-by-value and the array length is neither small enough for a
    short header nor large enough to have suffered compression.  But that
    seems like a very acceptable price for winning in pass-by-ref cases.)

    Hence, redesign to take these insights into account.  While at it,
    convert to an API in which we fill a struct rather than passing a bunch
    of pointers to individual output arguments.  That will make it less
    painful if we ever want further expansion of what get_attstatsslot can
    pass back.

    It's certainly arguable that this is new development and not something to
    push post-feature-freeze.  However, I view it as primarily bug-proofing
    and therefore something that's better to have sooner not later.  Since
    we aren't quite at beta phase yet, let's put it in.

    Discussion: https://postgr.es/m/16364.1494520862@sss.pgh.pa.us

Most of the changes are same as the upstream commit with following additional
changes:
- Relcache translator changes in ORCA.
- Added a test that simulates the crash due to broken stats
- get_attstatsslot() contains an extra check for empty slot array which existed
in master but is not there in upstream.
Signed-off-by: NAbhijit Subramanya <asubramanya@pivotal.io>
(cherry picked from commit ae06d7b0)

Instead of assuming that casts are always binary coercible (and hence that we
could get away with just dropping them), translate casts in ORCA plans into
either a RelabelType or a FuncExpr.
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>

As pointed out by Heikki, maintaining another variable to match one in
the database system will be error-prone and cumbersome, especially while
merging with upstream. This commit initializes ORCA with a pointer to a
GPDB function that returns true when QueryCancelPending or
ProcDiePending is set. This way we no longer have to micro-manage
setting and re-setting some internal ORCA variable, or touch signal
handlers.

This commit also reverts commit 0dfd0ebc "Support optimization interrupts
in ORCA" and reuses tests already pushed by 916f460f and 0dfd0ebc.

To support that, this commit adds 2 new ORCA APIs:
- SignalInterruptGPOPT(), which notifies ORCA that an abort is requested
  (must be called from the signal handler)
- ResetInterruptsGPOPT(), which resets ORCA's state to before the
  interruption, so that the next query can run normally (needs to be
  called only on the QD)

Also check for interrupts right after ORCA returns.

Before this commit all memory allocations made by ORCA/GPOS were a
blackbox to GPDB. However the ground work had been in place to allow
GPDB's Memory Accounting Framework to track memory consumption by ORCA.
This commit introduces two new functions
Ext_OptimizerAlloc and Ext_OptimizerFree which
pass through their parameters to gp_malloc and gp_free and do some bookeeping
against the Optimizer Memory Account. This introduces very little
overhead to the GPOS memory management framework.
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>

GPOS raises exception with different severity level, but
they were being logged to GPDB logs at LOG severity level.
This disabled users to not turn off logging for GPOS exceptions, unless
GPDB log setting was changed higher than LOG severity level.

This is the initial commit which introduces the functionality. If an
exception is created without the GPDB severity level, it will default to
LOG severity level in GPDB.
Signed-off-by: NJemish Patel <jpatel@pivotal.io>

GPORCA should not spend time extracting column statistics that are not
needed for cardinality estimation. This commit eliminates this overhead
of requesting and generating the statistics for columns that are not
used in cardinality estimation unnecessarily.

E.g:
`CREATE TABLE foo (a int, b int, c int);`

For table foo, the query below only needs for stats for column `a` which
is the distribution column and column `c` which is the column used in
where clause.
`select * from foo where c=2;`

However, prior to that commit, the column statistics for column `b` is
also calculated and passed for the cardinality estimation. The only
information needed by the optimizer is the `width` of column `b`. For
this tiny information, we transfer every stats information for that
column.

This commit and its counterpart commit in GPORCA ensures that the column
width information is passed and extracted in the `dxl:Relation` metadata
information.

Preliminary results for short running queries provides up to 65x
performance improvement.
Signed-off-by: NJemish Patel <jpatel@pivotal.io>

Currently ORCA does not support index scan on leaf partitions. It only supports
index scan if we query the root table. This commit along with the corresponding
ORCA changes adds a support for using indexes when leaf partitions are queried
directly.

When a root table that has indexes (either homogenous/complete or
heterogenous/partial) is queried; the Relcache Translator sends index
information to ORCA.  This enables ORCA to generate an alternative plan with
Dynamic Index Scan on all partitions (in case of homogenous index) or a plan
with partial scan i.e. Dynamic Table Scan on leaf partitions that  don’t have
indexes + Dynamic Index Scan on leaf partitions with indexes (in case of
heterogeneous index).

This is a two step process in Relcache Translator as described below:

Step 1 - Get list of all index oids

`CTranslatorRelcacheToDXL::PdrgpmdidRelIndexes()` performs this step and it
only retrieves indexes on root and regular tables; for leaf partitions it bails
out.

Now for root, list of index oids is nothing but index oids on its leaf
partitions. For instance:

```
CREATE TABLE foo ( a int, b int, c int, d int) DISTRIBUTED by (a) PARTITION
BY RANGE(b) (PARTITION p1 START (1) END (10) INCLUSIVE, PARTITION p2 START (11)
END (20) INCLUSIVE);

CREATE INDEX complete_c on foo USING btree (c); CREATE INDEX partial_d on
foo_1_prt_p2 using btree(d);
```
The index list will look like = { complete_c_1_prt_p1, partial_d }

For a complete index, the index oid of the first leaf partitions is retrieved.
If there are partial indexes, all the partial index oids are retrieved.

Step 2 - Construct Index Metadata object

`CTranslatorRelcacheToDXL::Pmdindex()` performs this step.

For each index oid retrieved in Step #1 above; construct an Index Metadata
object (CMDIndexGPDB) to be stored in metadata cache such that ORCA can get all
the information about the index.
Along with all other information about the index, `CMDIndexGPDB` also contains
a flag `fPartial` which denotes if the given index is homogenous (if yes, ORCA
will apply it to all partitions selected by partition selector) or heterogenous
(if yes, the index will be applied to only appropriate partitions).
The process is as follows:
```
	Foreach oid in index oid list :
		Get index relation (rel)
		If rel is a leaf partition :
			Get the root rel of the leaf partition
			Get all	the indexes on the root (this will be same list as step #1)
			Determine if the current index oid is homogenous or heterogenous
			Construct CMDIndexGPDB based appropriately (with fPartial, part constraint,
			defaultlevels info)
		Else:
			Construct a normal CMDIndexGPDB object.
```

Now for leaf partitions, there is no notion of homogenous or heterogenous
indexes since a leaf partition is like a regular table. Hence in `Pmdindex()`
we should not got for checking if index is complete or not.

Additionally, If a given index is homogenous or heterogenous needs to be
decided from the perspective of relation we are querying(such as root or a
leaf).

Hence the right place of `fPartial` flag is in the relation metadata object
(CMDRelationGPDB) and not the independent Index metadata object (CMDIndexGPDB).
This commit makes following changes to support index scan on leaf partitions
along with partial scans :

Relcache Translator:

In Step1, retrieve the index information on the leaf partition and create a
list of CMDIndexInfo object which contain the index oid and `fPartial` flag.
Step 1 is the place where we know what relation we are querying which enable us
to determine whether or not the index is homogenous from the context of the
relation.

The relation metadata tag will look like following after this change:

Before:
```
	<dxl:Indexes>
		<dxl:Index Mdid="0.17159874.1.0"/>
		<dxl:Index Mdid="0.17159920.1.0"/>
	</dxl:Indexes>
```

After:
```
	<dxl:IndexInfoList>
		<dxl:IndexInfo Mdid="0.17159874.1.0" IsPartial="true"/>
		<dxl:IndexInfo Mdid="0.17159920.1.0" IsPartial="false"/>
	</dxl:IndexInfoList>

```

A new class `CMDIndexInfo` has been created in ORCA which contains index mdid
and `fPartial` flag.  For external tables, normal tables and leaf partitions;
the `fPartial` flag will always be false.

Hence at the end, relcache translator will provide list of indexes defined on
leaf partitions when they are queried directly with `fPartial` being false
always. And when root table is queried; the `fPartial` will be set
appropriately based on the completeness of the index.  ORCA will refer to
Relation Metadata for fPartial information and not to the indepedent Index
Metadata Object.

[Ref ##120303669]

(cherry picked from commit dae6849f)

While gathering metadata information about a partitioned relation, the
relcache translator in GPDB constructs and sends partition
constraints in `CMDPartConstraintGPDB` object.
CMDPartConstraintGPDB contains following:
1. m_pdrgpulDefaultParts = List of partition levels that have default
                           partitions
2. m_fUnbounded          = indicate if the constraint is unbounded
3. m_pdxln               = Part Constraint expression

When you have more partitioning levels, the part constraint expression
can grow large and we end up spending significant amount of time in
translating this expression to DXL format.
For instance, for the following query, relcache translator spends
`18895.444000 ms` in fetching the metadata information on debug build:
```
DROP TABLE IF EXISTS date_parts;
CREATE TABLE DATE_PARTS (id int, year int, month int, day int, region text)
DISTRIBUTED BY (id) PARTITION BY RANGE (year) SUBPARTITION BY LIST
(month) SUBPARTITION TEMPLATE ( SUBPARTITION Q1 VALUES (1, 2, 3),
SUBPARTITION Q2 VALUES (4 ,5 ,6), SUBPARTITION Q3 VALUES (7, 8, 9),
SUBPARTITION Q4 VALUES (10, 11, 12), DEFAULT SUBPARTITION other_months )
SUBPARTITION BY RANGE(day) SUBPARTITION TEMPLATE ( START (1) END (31)
EVERY (10), DEFAULT SUBPARTITION other_days) ( START (2002) END (2012)
EVERY (1), DEFAULT PARTITION outlying_years );

INSERT INTO date_parts SELECT i, EXTRACT(year FROM dt), EXTRACT(month
FROM dt), EXTRACT(day FROM dt), NULL FROM (SELECT i, '2002-01-01'::DATE
+ i * INTERVAL '1 day' day AS dt FROM GENERATE_SERIES(1, 3650) AS i) AS
t;

EXPLAIN SELECT * FROM date_parts WHERE month BETWEEN 1 AND 4;
```

At ORCA end, however, we do not use this part constraints expression
unless the relation has indices built on it.  This is evident from
CTranslatorDXLToExpr.cpp + L565 (`CTranslatorDXLToExpr::PexprLogicalGet`).
Hence we do not need to send the PartConstraint expression from GPDB
Recache Translator since:
A. It will not be consumed if there are no indices
B. The DXL translation is expensive.

This commit fixes the Relcache Translator to send the Partition Constraint
Expression to ORCA only in the cases listed below :

	IsPartTable     Index 	DefaultParts   ShouldSendPartConstraint
	NO		-	-	       -
	YES		YES	YES	       YES
	YES		NO 	NO	       NO
	YES		NO 	YES            YES (but only default levels info)

After fix the metadata fetch and translation time is reduced to `87.828000ms`.

We also need changes in ORCA ParseHandler since we will be sending the part
constraint expression in some cases only.

[Ref #149769559]
Signed-off-by: NOmer Arap <oarap@pivotal.io>
(cherry picked from commit 752e06f6)

This is potentially a tiny bit faster, if the coercion can be performed
just once at parse/plan time, rather than on every row.

This fixes some of the bogus error checks and inconsistencies in handling
the ROWS expressions. For example, before, if you passed a string constant
as the ROWS expression, you got an error, but if you passed a more
complicated expression, that returned a string, the string was cast to an
integer at runtime. And those casts evaded the plan-time checks for
negative values.

Also, move the checks for negative ROWS/RANGE value from the parser to the
beginning of execution, even in the cases where the value is a constant, or
a stable expression that only needs to be executed once. We were missing
the checks in ORCA, so this fixes the behavior with ORCA for such queries.

This allows removing all the code in CTranslatorDXLToPlStmt that tracked
the parent of each call.

I found the plan node IDs awkward, when I was hacking on
CTranslatorDXLToPlStmt. I tried to make a change where a function would
construct a child Plan node first, and a Result node on top of that, but
only if necessary, depending on the kind of child plan. The parent plan
node IDs made it impossible to construct a part of Plan tree like that, in
a bottom-up fashion, because you always had to pass the parent's ID when
constructing a child node. Now that is possible.

Fix Relcache Translator to send CoercePath info

Currently ORCA crashes while executing following query:
```
CREATE TABLE FOO(a integer NOT NULL, b double precision[]);
SELECT b FROM foo
UNION ALL
SELECT ARRAY[90, 90] as Cont_features;
```

In the query, we are appending an integer array (ARRAY[90, 90]) to a double
precision array (foo.b) and hence we need to apply a cast on ARRAY[90, 90] to
generate ARRAY[90, 90]::double precision[].
In gpdb5 there is not direct function available that can cast array of any type
to array of any other type.
So in relcache to dxl translator we look into the array elements and get their type
and try to find a cast function for them.  For this query, source type is 23 i.e.
integer and destination type is 701 i.e. double precision and we try to find if
we have a conversion function for 23 -> 701. Since that is available we send
that function to ORCA as follows:
```
<dxl:MDCast Mdid="3.1007.1.0;1022.1.0"
Name="float8" BinaryCoercible="false" SourceTypeId="0.1007.1.0"
DestinationTypeId="0.1022.1.0" CastFuncId="0.316.1.0"/>
```
Here we are misinforming ORCA by specifying that function with id 316 is available
to convert type 1007 i.e. integer array to 1022 i.e. double precision array.
However Function id 316 is simple int4 to float8 conversion function and it CAN NOT
convert an array of int4 to array of double precision. ORCA generates a plan
using this function but executor crashes while executing this function because
this function can not handle arrays.

This commit fixes this issue by passing a ArrayCoercePath info to ORCA.
In Relcache Translator, The appropriate cast function is retrieved in `gpdb::FCastFunc()`
which relies on `find_coercion_pathway()` to provide the cast function oid given the src
and dest types.

`find_coercion_pathway()` does not just determines the cast function to be used but
also determines the coercion path; however we ignored the coercision path
and generate a simple Cast Metadata Object.

With this commit, we now pass the pathtype to relcache translator and
generate ArrayCoerceCast Metadata object depending on the coercion path.

In ORCA, when the dxl is translated to expression, we check the path type along with
the cast function and generate `CScalarArrayCoerceExpr` if the path type is
array coerce path; otherwise we generate simple `CScalaraCast`.

Please check the corresponding ORCA PR.

Bump ORCA version to 2.40
Signed-off-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>

In ORCA, we donot process interrupts during planning stage, however
if there are elog/ereport (which further calls errfinish) statements to
print additional messages we prematurely exit out the planning stage
without cleaning up the memory pools leading to inconsistent memory pool
state. This results in crashes for the subsequent queries.

This commit fixes the issue by handling interrupts while
printing messages using elog/ereport in ORCA.
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

This commit introduces a new operator for ValuesScan, earlier we
generated `UNION ALL` for cases where VALUES lists passed are all
constants, but now a new Operator CLogicalConstTable with an array of
const tuples will be generated

Once the plan is generated by ORCA, it will be translated to valuesscan
node in GPDB.

This enhancement helps significantly in improving the total run time for the queries
involving values scan in ORCA with const values.
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

* Remove PartOidExpr, it's not used in GPDB.

The target lists of DML nodes that ORCA generates includes a column for the
target partition OID. It can then be referenced by PartOidExprs. ORCA uses
these to allow sorting the tuples by partition, before inserting them to the
underlying table. That feature is used by HAWQ, where grouping tuples that
go to the same output partition is cheaper.

Since commit adfad608, which removed the gp_parquet_insert_sort GUC, we
don't do that in GPDB, however. GPDB can hold multiple result relations open
at the same time, so there is no performance benefit to grouping the tuples
first (or at least not enough benefit to counterbalance the cost of a sort).

So remove the now unused support for PartOidExpr in the executor.

* Bump ORCA version to 2.37
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

* Removed acceptedLeaf
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

Instead we should maintain NDVRemain and NullFreq to do Cardinality
Estimation.

Adding function to check if we need to create stats bucket in DXL

Function `FCreateStatsBucket` returns true if column data type is
not a text/varchar/char/bpchar type.
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

Bump orca version to 2.28.0
Signed-off-by: NJemish Patel <jpatel@pivotal.io>

ORCA can do some optimizations - partition pruning at least - if it can
"see" into the elements of an array in a ScalarArrayOpExpr. For example, if
you have a qual like "column IN (1, 2, 3)", and the table is partitioned on
column, it can eliminate partitions that don't hold those values. The
IN-clause is converted into an ScalarArrayOpExpr, so that is really
equivalent to "column = ANY <array>"

However, ORCA doesn't know how to extract elements from an array-typed
Const, so it can only do that if the array in the ScalarArrayOpExpr is
an ArrayExpr. Normally, eval_const_expressions() simplifies any ArrayExprs
into Const, if all the elements are constants, but we had disabled that
when ORCA was used, to keep the ArrayExprs visible to it.

There are a couple of reasons why that was not a very good solution. First,
while we refrain from converting an ArrayExpr to an array Const, it doesn't
help if the argument was an array Const to begin with. The "x IN (1,2,3)"
construct is converted to an ArrayExpr by the parser, but we would miss the
opportunity if it's written as "x = ANY ('{1,2,3}'::int[])" instead.
Secondly, by not simplifying the ArrayExpr, we miss the opportunity to
simplify the expression further. For example, if you have a qual like
"1 IN (1,2)", we can evaluate that completely at plan time to 'true', but
we would not do that with ORCA because the ArrayExpr was not simplified.

To be able to also optimize those cases, and to slighty reduce our diff
vs upstream in clauses.c, always simplify ArrayExprs to Consts, when
possible. To compensate, so that ORCA still sees ArrayExprs rather than
array Consts (in those cases where it matters), when a ScalarArrayOpExpr
is handed over to ORCA, we check if the argument array is a Const, and
convert it (back) to an ArrayExpr if it is.
Signed-off-by: NJemish Patel <jpatel@pivotal.io>