1. 29 9月, 2020 1 次提交
    • J
      Format ORCA and GPOPT. · 219fe0c4
      Jesse Zhang 提交于
      The canonical config file is in src/backend/gpopt/.clang-format (instead
      of under the non-existent src/backend/gporca), I've created one (instead
      of two) symlink, for GPOPT headers. Care has been taken to repoint the
      symlink to the canonical config under gpopt, instead of gpopt as it is
      under HEAD.
      
      This is spiritually a cherry-pick of commit 2f7dd76c.
      (cherry picked from commit 2f7dd76c)
      219fe0c4
  2. 19 10月, 2019 1 次提交
    • C
      Bump ORCA version to 3.78.0, Optimize CMemoryPoolPalloc in ORCA (#8867) · 7c725c4a
      Chris Hajas 提交于
      CMemoryPoolPalloc previously used headers and logic that were only
      needed in CMemoryPoolTracker. For each allocation, a fairly large header
      was added, which caused memory intensive operations in ORCA to use large
      amounts of memory. Now, we only store the size of array allocations in a
      header if needed. Otherwise, no header information is needed/stored on
      the ORCA side. This reduces the memory utilization for some queries by
      30%+. For TPC-DS Q72 on my laptop, peak memory utilization went from 1.1GB to
      720MB. This header accounted for ~20MB of the 720MB peak usage in Q72.
      
      This functionality can be enabled with the
      `optimizer_use_gpdb_allocators` GUC.
      
      Corresponding ORCA commit: https://github.com/greenplum-db/gporca/commit/d828eed "Simplify CMemoryPool to reduce unnecessary headers and logic".
      Co-authored-by: NChris Hajas <chajas@pivotal.io>
      Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>
      7c725c4a
  3. 05 10月, 2019 1 次提交
    • C
      Bump ORCA version to 3.74.0, Introduce PallocMemoryPool for use in GPORCA (#8747) · a3266308
      Chris Hajas 提交于
      We introduce a new type of memory pool and memory pool manager:
      CMemoryPoolPalloc and CMemoryPoolPallocManager
      
      The motivation for this PR is to improve memory allocation/deallocation
      performance when using GPDB allocators. Additionally, we would like to
      use the GPDB memory allocators by default (change the default for
      optimizer_use_gpdb_allocators to on), to prevent ORCA from crashing when
      we run out of memory (OOM). However, with the current way of doing
      things, doing so would add around 10 % performance overhead to ORCA.
      
      CMemoryPoolPallocManager overrides the default CMemoryPoolManager in
      ORCA, and instead creates a CMemoryPoolPalloc memory pool instead of a
      CMemoryPoolTracker. In CMemoryPoolPalloc, we now call MemoryContextAlloc
      and pfree instead of gp_malloc and gp_free, and we don’t do any memory
      accounting.
      
      So where does the performance improvement come from? Previously, we
      would (essentially) pass in gp_malloc and gp_free to an underlying
      allocation structure (which has been removed on the ORCA side). However,
      we would add additional headers and overhead to maintain a list of all
      of these allocations. When tearing down the memory pool, we would
      iterate through the list of allocations and explicitly free each one. So
      we would end up doing overhead on the ORCA side, AND the GPDB side.
      However, the overhead on both sides was quite expensive!
      
      If you want to compare against the previous implementation, see the
      Allocate and Teardown functions in CMemoryPoolTracker.
      
      With this PR, we improve optimization time by ~15% on average and up to
      30-40% on some queries which are memory intensive.
      
      This PR does remove memory accounting in ORCA. This was only enabled
      when the optimizer_use_gpdb_allocators GUC was set. By setting
      `optimizer_use_gpdb_allocators`, we still capture the memory used when
      optimizing a query in ORCA, without the overhead of the memory
      accounting framework.
      
      Additionally, Add a top level ORCA context where new contexts are created
      
      The OptimizerMemoryContext is initialized in InitPostgres(). For each
      memory pool in ORCA, a new memory context is created in
      OptimizerMemoryContext.
      Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>
      Co-authored-by: NChris Hajas <chajas@pivotal.io>
      a3266308