Adds a script to format and check formatting.

This is intended for both local developer use and for CI.

This depends on GNU parallel. One-time install:

macOS: brew install parallel clang-format
Debian: apt install parallel clang-format-10

To format all ORCA / GPOPT code:

$ src/tools/fmt fmt

To check for formatting conformance:

$ src/tools/fmt chk

To modify the configuration, you'll need two steps:
1. Edit clang-format.intent.yml
2. Generate the expanded configuration file:

$ src/tools/fmt gen

This commit also adds a formatting README, To document some of the
rationale behind tooling choice. Also mention the new `README.format.md`
from both the style guide and ORCA's main README.

src/backend/gporca/README.format.md

+# GPOPT and ORCA code formatting
+
+### Tools
+
+1. We are using [clang-format](https://clang.llvm.org/docs/ClangFormat.html).
+
+[clang-format-style-options]: https://clang.llvm.org/docs/ClangFormatStyleOptions.html
+[clang-format-style-options.10]: https://releases.llvm.org/10.0.0/tools/clang/docs/ClangFormatStyleOptions.html
+[clang-format.10]: https://releases.llvm.org/10.0.0/tools/clang/docs/ClangFormat.html
+1. We use the current stable release, with an eye to good new options coming from [the next release][clang-format-style-options].
+   As of writing we're using [release 10][clang-format.10],
+   the configuration options are [documented here][clang-format-style-options.10].
+
+1. As mentioned in the [style guide](StyleGuide.md), the format style is based on Google C++ style guide,
+   but mixed in with Postgres formatting rules (break before braces, 4-space tabbing, etc)
+
+1. We should use an explicit, complete (locked down) specification for the [.clang-format](.clang-format) file.
+
+1. But our intent is better expressed as [well organized, commented yaml](clang-format.intent.yaml).
+   We use a [simple script](../../../src/tools/fmt) to generate the complete config file from the intent file. For example, on my Linux laptop, I run:
+
+   ```shell
+   CLANG_FORMAT=clang-format-10 src/tools/fmt gen
+   ```
+
+   If the correct version of `clang-format` is installed as `clang-format` (as is the case in macOS), you can omit the environment variable override.
+
+1. To check for formatting conformance, one can run
+
+   ```shell
+   src/tools/fmt chk
+   ```
+
+   It will succeed quietly (with return code 0) or point out the first few places that need to be formatted.
+
+1. To wholesale format all of ORCA and GPOPT
+
+   ```shell
+   src/tools/fmt fmt
+   ```
+
+   On my laptop this takes about 2.5 seconds.
+
+1. Of course, when you're writing code, 2.5 seconds is an eternity.
+   Here's a non-exhaustive list of editor / IDE plugins that provide an
+   integrated formatting experience.
+   The in-editor formatting typically takes around 50ms.
+   Depending on your plugin / editor,
+   the formatting either happens automatically as you type, on save,
+   or when you invoke it explicitly.
+
+   [int.emacs]: https://clang.llvm.org/docs/ClangFormat.html#emacs-integration
+   * LLVM's official [Emacs integration][int.emacs] provides the _`clang-format-region`_ macro.
+
+   [int.vim]: https://clang.llvm.org/docs/ClangFormat.html#vim-integration
+   * [Vim integration][int.vim] as documented by the official LLVM project
+
+   [int.altvim]: https://github.com/rhysd/vim-clang-format
+   * A community [VIM plugin][int.altvim] that looks promising and claims to be better than the official VIM integration
+
+   [int.clion]: https://www.jetbrains.com/help/clion/clangformat-as-alternative-formatter.html
+   * [Clion][int.clion] detects `.clang-format` automatically and switches to using it for the built-in "Reformat Code" and "Reformat File...".
+
+   [int.vscode]: https://code.visualstudio.com/docs/cpp/cpp-ide#_code-formatting
+   * Similar to CLion, [Visual Studio Code has it built in][int.vscode].
+
+   [int.xcode8]: https://github.com/mapbox/XcodeClangFormat
+   * [Plugin for Xcode 8+][int.xcode8]
+
+### Annotating history
+1. It's usually desirable to skip past formatting commits when doing a `git blame`. Fortunately, git blame has two options that are extremely helpful:
+    1. `-w` ignore whitespace
+    1. `--ignore-rev` / `--ignore-revs-file` accepts commits to skip while annotating
+
+  When combined, `-w --ignore-revs-file` produces a perfectly clean annotation, unencumbered by a noisy history. This also gives us a peace of mind in changing our formats more frequently without worrying about hindering the `git blame` experience, rather than "do it once, set it in stone, and forget about it".
+
+### Reformatting in-flight patches
+ORCA is actively developed, that means when the "big bang" formatting change is merged,
+there will inevitably be a number of patches that were started well before the formatting change,
+and these patches will now need to conform to the new format.
+The following steps are used to convert in-flight branches to the new format.
+
+1. Get the format script
+
+   ```sh
+   git restore --source master -- src/tools/fmt
+   ```
+
+1. Get the format configuration files
+
+   ```sh
+   git restore --source master -- src/include/gpopt/.clang-format src/backend/gpopt/.clang-format src/backend/gporca/.clang-format
+   ```
+
+1. Use `git filter-branch` to rewrite the branch history
+
+   ```sh
+   git filter-branch --tree-filter 'src/tools/fmt fmt' master..
+   ```
+
+1. Now that we have reformatted the history, rebase on top of master
+
+   ```sh
+   git rebase master
+   ```

src/backend/gporca/README.md

@@ -226,5 +226,17 @@ ninja -v -C build
 
 We accept contributions via [Github Pull requests](https://help.github.com/articles/using-pull-requests) only.
 
+
+ORCA has a [style guide](StyleGuilde.md), try to follow the existing style in your contribution to be consistent.
+
+[clang-format]: https://clang.llvm.org/docs/ClangFormat.html
+A set of [clang-format]-based rules are enforced in CI. Your editor or IDE may automatically support it. When in doubt, check formatting locally before submitting your PR:
+
+```
+CLANG_FORMAT=clang-format src/tools/fmt chk
+```
+
+For more information, head over to the [formatting README](README.format.md).
+
 We follow GPDB's comprehensive contribution policy. Please refer to it [here](https://github.com/greenplum-db/gpdb#contributing) for details.
 

src/tools/fmt

+#!/usr/bin/env bash
+
+# src/tools/fmt
+
+set -e -u -o pipefail
+
+# This script formats (or checks) C++ code with clang-format in the following
+# locations:
+#
+# src/backend/gpopt
+# src/include/gpopt
+# src/backend/gporca
+#
+# To generate the expanded configuration file for a change in intent:
+# $ fmt gen
+#
+# To format all ORCA / GPOPT code:
+# $ fmt fmt
+#
+# To check for formatting conformance:
+# $ fmt chk
+#
+# If the name of your clang-format executable isn't "clang-format", you can
+# override it in the CLANG_FORMAT environment variable.
+#
+# For more about ORCA's format, see src/backend/gporca/README.format.md
+
+
+# we use mapfile from Bash 4.0, but it's sufficiently likely that *someone* on
+# a macOS system would be using Bash 3.2 that it'd be unkind not to guard
+# against it.
+if ((BASH_VERSINFO[0] < 4)); then false; fi
+
+readonly INTENT=src/backend/gporca/clang-format.intent.yaml
+: "${CLANG_FORMAT:=clang-format}"
+
+gen() {
+	local DIR
+	DIR=$(mktemp -d -t orca-clang-format.XXX)
+	cp "${INTENT}" "${DIR}/.clang-format"
+
+	local -a CLANG_FORMAT_CONFIG_FILES
+	# do not squish the assignment into the declaration, as that will swallow
+	# errors in the subshell
+	local _find_result
+	_find_result=$(find src/backend/gporca -name .clang-format)
+	mapfile -t CLANG_FORMAT_CONFIG_FILES <<<$_find_result
+
+	(
+		set -e
+		cd "${DIR}"
+		"${CLANG_FORMAT}" -style=file -dump-config >clang-format.yaml
+	)
+
+	for cf in "${CLANG_FORMAT_CONFIG_FILES[@]}"; do
+		cp -v "${DIR}/clang-format.yaml" "${cf}"
+	done
+	rm -r "${DIR}"
+}
+
+# NUL-delimited list of files to format
+files_to_format() {
+	local -a CLANG_FORMAT_CONFIG_FILES
+	# do not squish the assignment into the declaration, as that will swallow
+	# errors in the subshell
+	local _find_result
+	_find_result=$(find src/backend/gporca src/backend/gpopt src/include/gpopt -name .clang-format)
+	mapfile -t CLANG_FORMAT_CONFIG_FILES <<<$_find_result
+
+	local -a CF_DIRS PATTERNS
+	CF_DIRS=("${CLANG_FORMAT_CONFIG_FILES[@]%/*}")
+	PATTERNS=("${CF_DIRS[@]/%/\/*.cpp}" "${CF_DIRS[@]/%/\/*.h}")
+	git ls-files -z "${PATTERNS[@]}"
+}
+
+# This depends on GNU parallel (https://www.gnu.org/software/parallel/). To install:
+#   macOS: brew install parallel
+#   Debian-derivatives: apt install parallel
+fmt() {
+	files_to_format | parallel -n64 -P +0 -0 \
+		"${CLANG_FORMAT}" -i
+}
+
+chk() {
+	# --tty: enables colorful diagnostics
+	# --halt: fail as soon as we encounter the first formatting diagnostic
+	files_to_format | parallel -n64 -P +0 -0 \
+		--halt now,fail=1 --tty \
+		"${CLANG_FORMAT}" -dry-run -ferror-limit 1 -Werror
+}
+
+usage() {
+	printf >&2 "Usage: [CLANG_FORMAT=clang-format] %s (gen|fmt|chk)\n" "$0"
+
+	return 1
+}
+
+_main() {
+	local cmd
+	cmd=${1:-}
+	local TOPLEVEL
+	TOPLEVEL=$(git rev-parse --show-toplevel)
+	cd "${TOPLEVEL}"
+	case "${cmd}" in
+	gen)
+		gen
+		;;
+	fmt)
+		fmt
+		;;
+	chk)
+		chk
+		;;
+	*)
+		usage
+		;;
+	esac
+}
+
+_main "$@"