These utilities were only used by gpseginstall, which has already been
removed.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>
(cherry picked from commit 90aa22d5)

* fix error -- missing ']'

* remove unnecessary space before brace

Also bump ORCA version to v3.44.0.

The 9.4 merge introduced a new way to represent TVF RTEs in GPDB - using
RangeTblFunctions (instead of FuncExpr used earlier). This commit
correctly populates RangeTblFunctions::funcparams when translation an
ORCA generated plan.
This param list is needed when TVF contains an outer reference in its
arguments. If it is left empty, ExecReScanFunctionScan() ignores all
chgParams, and rewinds instead of resetting its tuplestores. This will
produce wrong results because the FunctionScan will continue to return
the same result (as of its first execution) for every Rescan, even when
there are outer param changes.

See attached test for an example.

* docs - add cfg/using info about pxf jdbc named queries

* some edits requested by david

* address some comments from alex that are simple edits

* add month to sample subquery

* add explanatory info to the steps in the example

* that reference

* remove appropriate

* more comments from alex

* add note about aggregating over subquery aggregate

* remove the note i just added

For queries containing joins of tables with many columns, algebrization
in ORCA can be slow due to hash table collisions. For example, a 10-way
join of tables with 1500 cols each will generate a hash table with 80k+
colrefs. By minimizing these collisions, the total optimization time was reduced for
some queries by ~20%, and the algebrization time was reduced by 50%.

We're planning to make hashmaps grow as they reach some load
factor. However, this is a hot spot that we'd like to fix sooner.
Authored-by: NChris Hajas <chajas@pivotal.io>

Previously, gpconfig would not allow setting a GUC to a value of empty string
(that is, "gpconfig -c guc_name -v ''" or the like) because it conflated passing
an empty string with not setting the flag at all, so this commit fixes that and
ensures that empty-string values print nicely.
Authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
(cherry-picked from commit 941ef0d2)

ORCA's algebrizer must first normalize GROUP BYs in a query to a form
usable in ORCA. It must flatten expressions in the project list to
contain only aggregates and grouping columns
For example:

ORGINAL QUERY:
  SELECT * from r where r.a > (SELECT max(c) + min(d)
                               FROM t where r.b = t.e)
NEW QUERY:
  SELECT * from r where r.a > (SELECT x1+x2 as x3
  FROM (SELECT max(c) as x2, min(d) as x2
        FROM t where r.b = t.e) t2)

However this process did not support subqueries in the target list that
may contain outer references, sometimes in other (nested) subqueries. It
also did not support CTEs. All these would produce a normalization error
and fall back.

This commit fixes that by supporting subqueries & CTEs.

It also includes some refactors in related areas:

- Rename IncrLevelsUpInVar to IncrLevelsUpIfOuterRef, to capture it's
  implementation.
- Remove SContextHavingQualMutator, after realizing that it has almost
  the same members as SContextGrpbyPlMutator.
- Use MakeVarInDerivedTable & RunGroupByProjListMutator in
  RunGroupByProjListMutator to reduce code drift.
- Merge RunGroupByProjListMutator & RunHavingQualMutator (see below)

RunGroupByProjListMutator() was implemented at a later date than the
RunHavingQualMutator, and did not handle subqueries and ctes correctly.
After understanding its purpose, I think the functionality of both the
above methods should be exactly the same, since they're trying to
achieve the same goal.

So, this commit just merges the two functions together into a new
function - RunExtractAggregatesMutator. In this process, I also
discovered a bug in the old RunHavingQualMutator, that is now fixed:

  create table fooh1 (a int, b int, c int);
  insert into fooh1 select i%4, i%3, i from generate_series(1,20) i;
  select f1.a + 1 from fooh1 f1 group by f1.a+1 having sum(f1.a+1) + 1 > 20;

Finally, the earlier code deduplicated AGGREFs when possible for HAVING
clauses, but not for GROUP BY target lists when moving them into the
derived query. Not only is that inconsistent, but may also give
incorrect results in case of volatile functions. The executor already
handles de-duplicating AGGREFs in right circumstances, so doing this in
the algebrizer doesn't provide much of a benefit.

* docs - add content for new pxf read file as row feature

* some of the edits requested by david

* use echo to create third file

* remove JSON from titles

Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NMark Sliva <msliva@pivotal.io>
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>

Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NMark Sliva <msliva@pivotal.io>
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit 38834844)

(cherry picked from commit 1cf60c46)

Previously, gpconfig used bash and Perl to manipulate the master and
segment postgresql.conf files.  This made it hard to understand the
transformation on string valued gucs.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NMark Sliva <msliva@pivotal.io>
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit b6137171)

* docs - add module landing page for gp_sparse_vector

* edits requested by david

* docs - revisit hstore module doc contents

* edits requested by david

* docs - add pxf server cfg topic, rework other cfg topics, misc edits

* some of the edits requested by francisco

* add Hive back in to list of JDBC SQL dbs

* sneaking in a misc unrelated formatting fix

* edits requested by david

The curl slist API properly handle NULLs so we can be less verbose
and skip the check before passing to the slist cleanup function.
Reviewed-by: NFrancisco Guerrero <aguerrero@pivotal.io>

The header callback was storing the response in the context, but as
it was never used we might as well save the memory and just return
the required return of the number of bytes we would've saved should
we have allocated.
Reviewed-by: NFrancisco Guerrero <aguerrero@pivotal.io>

The only time the internal buffer cleanup code was called was just
before freeing the entire context, so individually zeroing out the
members is pointless. Remove the function entirely and inline the
buffer freeing into the context cleanup codepath.

For zeroing the error buffer, it's only called right after allocating
the error buffer with palloc0() in the first place so the memory will
always be zeroed out when reaching here.
Reviewed-by: NFrancisco Guerrero <aguerrero@pivotal.io>

If strlen(addr) is zero then based on how get_dest_address() works
addr will be NULL, and pfree() on NULL is not permitted.  Also, we
know that addr will either be a non-empty string or NULL, so we can
just as well test for addr being NULL and avoid a strlen() call.

Fix by only pfreeing when addr is set. (this is in an elog(ERROR..)
context so freeing isn't terribly interesting but it also doesn't
hurt so I'm keeping the current codepath.)
Reviewed-by: NFrancisco Guerrero <aguerrero@pivotal.io>

The libchurl abstraction layer has many internal helper functions
which weren't marked static and thus exported. Fix by marking all
as static.
Reviewed-by: NFrancisco Guerrero <aguerrero@pivotal.io>

The default libcurl on centos7 uses NSS SSL library instead of OPENSSL.
GPDB with that libcurl will report "Unknown cipher in list: AES128-SHA"
on connecting gpfdist SSL server.
Actually, we don't have to set cipher list in GPDB. It may choose stronger
cipher on negotiation. So remove setting cipher list.

Patch back to 6X_STABLE

GPDB uses libcurl-gnutls.so.4 as default libcurl on ubuntu. gpfdist SSL
connection reports error (56 - Failure when receiving data from the peer)
on handling POST message. We find it shutdown socket directly without sending
close_notify to client. So we call SSL_shutdown() before socket shutdown.

But SSL_accept() will return error with above patch on Centos, because
GPDB curl reuses SSL session ID in the second client hello, but server considers
that session is shutdown, so it won't accept that session ID.
The solution is disabling SSL session ID cache by setting curl option.
Then both Centos and ubuntu work well.

Enable gpfdist SSL test case.

Patch back to 6X_STABLE

When determine waiting relation in global dead lock detector,
we should also check whether the waiter's lockmode conflict with
holder's holdmask.
Co-authored-by: NNing Yu <nyu@pivotal.io>
Reviewed-by: NZhenghua Lyu <zlv@pivotal.io>

`create rule <...> do instead select * from t for update`
will dispatch a query with lockingclause node. Add
deserialization method for it to make things correct.

(cherry-picked from 256d8380)
Co-authored-by: NGoutam Tadi <gtadi@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>

* Source for the segment recovery flowchart

* Rename gprecovery-flow-chart.graffle to recovermatrix.graffle to match png filename

* docs - add info about pxf jdbc statement properties

* misc edits requested by david

The utilities reference page contained (unlinked) mentions of long
since deprecated utilities, and since they aren't shipping it's time
to remove them:

	gpsizecalc - removed in June 2010
	gpskew - removed in October 2009
	gprebuildsystem - removed in January 2010
	gpchecknet - removed in April 2010
	gpcheckos - removed in July 2016

Reviewed-by: Lisa Owen
Reviewed-by: David Yozie

gpdetective was removed in 4.3.5, so it seems about time to also
remove the documentation for it (which was unreachable due to the
app being marked deprecated).

Reviewed-by: Lisa Owen
Reviewed-by: David Yozie

* docs - add pageinspect module landing page

* remove peer scope

* cannot inspect ao or external relations

The "combine" function for the int4 sum/avg aggregate functions is
backported to Greenplum 6.0 in commit 313cef6e (from
postgres/postgres@11c8669c0cc) but we made the inevitable omission of
setting the "strictness" of int4_avg_combine to false. This by itself is
harmless as the actual function body *does* guard against NULL input,
but it also prohibits a whole host of optimizations when the executor
and planner can detect NULL input early on and short-circuit the
execution. Oops.

This patch flips the `proisstrict` flag back to true for
int4_avg_combine.

Backpatch to 6X_STABLE.

(cherry picked from commit c334bedc)