The canonical config file is in src/backend/gpopt/.clang-format (instead
of under the non-existent src/backend/gporca), I've created one (instead
of two) symlink, for GPOPT headers. Care has been taken to repoint the
symlink to the canonical config under gpopt, instead of gpopt as it is
under HEAD.

This is spiritually a cherry-pick of commit 2f7dd76c.
(cherry picked from commit 2f7dd76c)

We introduce a new type of memory pool and memory pool manager:
CMemoryPoolPalloc and CMemoryPoolPallocManager

The motivation for this PR is to improve memory allocation/deallocation
performance when using GPDB allocators. Additionally, we would like to
use the GPDB memory allocators by default (change the default for
optimizer_use_gpdb_allocators to on), to prevent ORCA from crashing when
we run out of memory (OOM). However, with the current way of doing
things, doing so would add around 10 % performance overhead to ORCA.

CMemoryPoolPallocManager overrides the default CMemoryPoolManager in
ORCA, and instead creates a CMemoryPoolPalloc memory pool instead of a
CMemoryPoolTracker. In CMemoryPoolPalloc, we now call MemoryContextAlloc
and pfree instead of gp_malloc and gp_free, and we don’t do any memory
accounting.

So where does the performance improvement come from? Previously, we
would (essentially) pass in gp_malloc and gp_free to an underlying
allocation structure (which has been removed on the ORCA side). However,
we would add additional headers and overhead to maintain a list of all
of these allocations. When tearing down the memory pool, we would
iterate through the list of allocations and explicitly free each one. So
we would end up doing overhead on the ORCA side, AND the GPDB side.
However, the overhead on both sides was quite expensive!

If you want to compare against the previous implementation, see the
Allocate and Teardown functions in CMemoryPoolTracker.

With this PR, we improve optimization time by ~15% on average and up to
30-40% on some queries which are memory intensive.

This PR does remove memory accounting in ORCA. This was only enabled
when the optimizer_use_gpdb_allocators GUC was set. By setting
`optimizer_use_gpdb_allocators`, we still capture the memory used when
optimizing a query in ORCA, without the overhead of the memory
accounting framework.

Additionally, Add a top level ORCA context where new contexts are created

The OptimizerMemoryContext is initialized in InitPostgres(). For each
memory pool in ORCA, a new memory context is created in
OptimizerMemoryContext.
Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>
Co-authored-by: NChris Hajas <chajas@pivotal.io>

As part of moving away from Hungarian notation in the GPORCA codebase,
the integration points between GPORCA and GPDB in the translator have
been renamed to the new convention used in GPORCA. The libraries
currently updated to the new notation in GPORCA are Naucrates and GPOS.
The new naming convention is a custom version of common C++ naming
conventions. The style guide for this convention can be found in the
GPORCA repository.

Also bump ORCA version to 2.69.0
Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>
Co-authored-by: NMelanie Plageman <mplageman@pivotal.io>
Co-authored-by: NEkta Khanna <ekhanna@pivotal.io>
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io&gt;&lt;Paste>
Co-authored-by: NDhanashree Kashid <dkashid@pivotal.io>
Co-authored-by: NOmer Arap <oarap@pivotal.io>

As pointed out by Heikki, maintaining another variable to match one in
the database system will be error-prone and cumbersome, especially while
merging with upstream. This commit initializes ORCA with a pointer to a
GPDB function that returns true when QueryCancelPending or
ProcDiePending is set. This way we no longer have to micro-manage
setting and re-setting some internal ORCA variable, or touch signal
handlers.

This commit also reverts commit 0dfd0ebc "Support optimization interrupts
in ORCA" and reuses tests already pushed by 916f460f and 0dfd0ebc.

To support that, this commit adds 2 new ORCA APIs:
- SignalInterruptGPOPT(), which notifies ORCA that an abort is requested
  (must be called from the signal handler)
- ResetInterruptsGPOPT(), which resets ORCA's state to before the
  interruption, so that the next query can run normally (needs to be
  called only on the QD)

Also check for interrupts right after ORCA returns.

Before this commit all memory allocations made by ORCA/GPOS were a
blackbox to GPDB. However the ground work had been in place to allow
GPDB's Memory Accounting Framework to track memory consumption by ORCA.
This commit introduces two new functions
Ext_OptimizerAlloc and Ext_OptimizerFree which
pass through their parameters to gp_malloc and gp_free and do some bookeeping
against the Optimizer Memory Account. This introduces very little
overhead to the GPOS memory management framework.
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>

There are other functions in the same file that reference the same
functions, so we don't need this to hint the linker anymore.

When a "null value in column <col> violates not-null constraint" error was
thrown from the ORCA translator code, it was propagated to the rest of the
system with ERRCODE_INTERNAL_ERROR. That's not nice, if an application
wants to e.g. catch errors based on the error code.

The reason to do this right now was that my previous commit that moved
some ORCA translator code in the source files made the regression tests
to fail. That's because an error handling code adds the source file name
and line number to the error message, like (CTranslatorUtils.cpp:2627),
and we had some such line numbers memorized in the regression tests.
When the code moved, the tests broke. That's obviously too fragile, and
using a proper error code fixes that by preventing the line number from
being printed. That also removed the last difference between the ORCA
and non-ORCA output from the 'insert' regression test.

We could probably do the same for other errors thrown from ORCA, but I'll
leave that for another day.

gporca has a set of banned API calls which needs to be allowed with the
ALLOW_xxx macro in order for gpopt to compile. But it should be the
library caller(GPDB/Orca)'s resposibility to take care of the function call.

see discussions on greenplum-db/gpdb#1136
and https://groups.google.com/a/greenplum.org/forum/#!topic/gpdb-dev/Mcw6JPav6h4

- In ORCA, due to the way exception handled previously we do warning first
and then later print error referring that message. In this commit, we
enhanced the exception handling so we just print a single error message.
- Also, we removed 'PQO unable to generate a plan' or 'Aborting PQO plan
generation' message and make the error message as close as the planner.
- Updated error message with filename and line number from where the exception
is raised.

libgpos has a set of banned API calls which needs to be allowed with
the ALLOW_xxx macro in order for gpopt to compile (and thus run).
The changes to ereport() brought a need for allowing abort() since
it now invokes abort when building with --enable-cassert.

This is a temporary fix awaiting the removal of the banning of
function calls entirely. Pushed even though the CI pipeline failed
to provide a clean run (for seeminly unrelated reasons) due to the
absence of this blocking other efforts.

Any GPDB exception happens in while ORCA generating plan will abort
query.
Signed-off-by: NFoyzur Rahman <frahman@gmail.com>

These were initialized by constructors earlier. To pass any parameters
for initializing GPOPT or any of its dependendencies, we need to do that
explicitly.

This commit eliminates the global new/delete overrides that were causing
compatibility problems (the Allocators.(h/cpp/inl) files have been
completely removed). The GPOS `New()` macro is retained and works the
same way, but has been renamed `GPOS_NEW()` to avoid confusion and
possible name collisions. `GPOS_NEW()` works only for allocating
singleton objects. For allocating arrays, `GPOS_NEW_ARRAY()` is
provided. Because we no longer override the global delete,
objects/arrays allocated by `GPOS_NEW()` and `GPOS_NEW_ARRAY()` must now
be deleted by the new functions `GPOS_DELETE()` and
`GPOS_DELETE_ARRAY()` respectively. All code in GPOS has been
retrofitted for these changes, but Orca and other code that depends on
GPOS should also be changed.

Note that `GPOS_NEW()` and `GPOS_NEW_ARRAY()` should both be
exception-safe and not leak memory when a constructor throws.

Closes #166