- 29 9月, 2020 1 次提交
-
-
由 Jesse Zhang 提交于
The canonical config file is in src/backend/gpopt/.clang-format (instead of under the non-existent src/backend/gporca), I've created one (instead of two) symlink, for GPOPT headers. Care has been taken to repoint the symlink to the canonical config under gpopt, instead of gpopt as it is under HEAD. This is spiritually a cherry-pick of commit 2f7dd76c. (cherry picked from commit 2f7dd76c)
-
- 19 10月, 2019 1 次提交
-
-
由 Chris Hajas 提交于
CMemoryPoolPalloc previously used headers and logic that were only needed in CMemoryPoolTracker. For each allocation, a fairly large header was added, which caused memory intensive operations in ORCA to use large amounts of memory. Now, we only store the size of array allocations in a header if needed. Otherwise, no header information is needed/stored on the ORCA side. This reduces the memory utilization for some queries by 30%+. For TPC-DS Q72 on my laptop, peak memory utilization went from 1.1GB to 720MB. This header accounted for ~20MB of the 720MB peak usage in Q72. This functionality can be enabled with the `optimizer_use_gpdb_allocators` GUC. Corresponding ORCA commit: https://github.com/greenplum-db/gporca/commit/d828eed "Simplify CMemoryPool to reduce unnecessary headers and logic". Co-authored-by: NChris Hajas <chajas@pivotal.io> Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>
-
- 05 10月, 2019 1 次提交
-
-
由 Chris Hajas 提交于
We introduce a new type of memory pool and memory pool manager: CMemoryPoolPalloc and CMemoryPoolPallocManager The motivation for this PR is to improve memory allocation/deallocation performance when using GPDB allocators. Additionally, we would like to use the GPDB memory allocators by default (change the default for optimizer_use_gpdb_allocators to on), to prevent ORCA from crashing when we run out of memory (OOM). However, with the current way of doing things, doing so would add around 10 % performance overhead to ORCA. CMemoryPoolPallocManager overrides the default CMemoryPoolManager in ORCA, and instead creates a CMemoryPoolPalloc memory pool instead of a CMemoryPoolTracker. In CMemoryPoolPalloc, we now call MemoryContextAlloc and pfree instead of gp_malloc and gp_free, and we don’t do any memory accounting. So where does the performance improvement come from? Previously, we would (essentially) pass in gp_malloc and gp_free to an underlying allocation structure (which has been removed on the ORCA side). However, we would add additional headers and overhead to maintain a list of all of these allocations. When tearing down the memory pool, we would iterate through the list of allocations and explicitly free each one. So we would end up doing overhead on the ORCA side, AND the GPDB side. However, the overhead on both sides was quite expensive! If you want to compare against the previous implementation, see the Allocate and Teardown functions in CMemoryPoolTracker. With this PR, we improve optimization time by ~15% on average and up to 30-40% on some queries which are memory intensive. This PR does remove memory accounting in ORCA. This was only enabled when the optimizer_use_gpdb_allocators GUC was set. By setting `optimizer_use_gpdb_allocators`, we still capture the memory used when optimizing a query in ORCA, without the overhead of the memory accounting framework. Additionally, Add a top level ORCA context where new contexts are created The OptimizerMemoryContext is initialized in InitPostgres(). For each memory pool in ORCA, a new memory context is created in OptimizerMemoryContext. Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io> Co-authored-by: NChris Hajas <chajas@pivotal.io>
-