提交 fa65063d 编写于 作者: J Jesse Zhang 提交者: Melanie Plageman

Remove codgen wholesale (#4301)

This patch removes codegen wholesale from Greenplum.

In addition to reverting the commits involving codegen, we also removed
miscellaneous references to the feature and GUC.

The following commits from 5.0.0 were reverted (topologically ordered):

f38c9064 Support avg aggregate function in codegen
87dcae4c Capture and print error messages from llvm::verifyFunction
65137540 libgpcodegen assert() overrides use GPDB Assert()
81d378b4 GenerateExecVariableList calls regular slot_getattr when the code generation of the latter fails
05a28211 Update Google Test source path used by codegen
22a35fcc Call ereport when code generating float operators
79517271 Support overflow checks for doubles.
b5373a1e Fix codegen unittest link problem
7a1a98c9 Print filename and lineno in codegened CreateElog
58eda293 Fix wrong virtual tuple check in codegened slot_getattr
bc6faa08 Set llvm_isNull_ptr to false when the result of codegened expression evaluation is not null
8bbbd63f Enhance codegened advance_aggregates with support for null attributes
e1fd6072 Abort code generation of expression evaluation trees with unsupported ExprState types
509460ee Support null attributes in codegen expression evaluation framework
739d978d Move enrollment of codegened ExecVariableList to ExecInitNode
c05528d1 Fix CHECK_SYMBOL_DEFINED macro in CMakeLists.txt
12cfd7bd Support offset calculation when there is null in the tuple and codegen is enabled
40a2631e Use slot_getattr wrapper function for regular version.(#1194)
613e9fbb Revert "Fix codegen issue by moving slot_getattr to heaptuple.c similar to"
ee03f799 Fix cpplint error on advance aggregate
2a65b0aa Fix slot function nullptr issue in expr eval.
3107fc0e Fix for !(expr1 & expr2) and hasnull logic in slot_getatrr codegen.
c940c7f6 Fix codegen issue by moving slot_getattr to heaptuple.c similar to Postgres.
c8125736 Introduce guc to enable/disable all generator.
a4f39507 Ensure that codegen unittests pass with GCC 6.2.0 (#1177)
682d0b28 Allow overriding assert() functionality in libgpcodegen in DEBUG mode
3209258a Organize codegen source files and unit tests
d2ba88a9 Fix codegen unittests using DatumCastGenerator
020b71a5 Generate code for COUNT aggregate function
0eeec886 Fixing codegen related bugs in InitializeSupportedFunction
41055352 Rewrite PGGenericFuncGenerator to support variable number of types
e5318b6b Add AdvanceAggregatesCodegenEnroll mock function
87521715 Codegen advance_aggregates for SUM transition function
2043d95b Use string messages in codegened slot_getattr fallback block
f160fa5a Add new GUC to control Codegen Optimization level.
697ffc1a Fix cpplint errors
c5e3aed4 Support Virtual Tuples and Memtuples in SlotGetAttrCodegen
5996aaa7 Keep a reference to the CodegenManager in code generators
6833b3c2 Remove unused header and just include what you use in codegen
ab1eda87 Allow setting codegen guc to ON only if code generation is supported by the build
dcd40712 Use PGFuncGeneratorInfo to codegen pg functions
83869d1c Replace dynamic_cast with dyn_cast for llvm objects
23007017 Decide what to generate for ExecEvalExpr based on PlanState
387d8ce8 Add EXPLAIN CODEGEN to print generated IR to the client
c4a9bd27 Introduce Datum to cpp cast, cpp type to Datum cast  and normal cast.(#944)
66158dfd Record external function names for useful debugging
adab9120 Support variable length attributes in SlotGetAttrCodegen.
335e09aa Proclaim that the codegen'ed version of generated functions are called in debug build
50fd9140 Fix cpplint errors
88a04412 Use ExprTreeGeneratorInfo for expression tree generation.
3b4af3bb Split code generation of ExecVariableList from slot_getattr
8cd9ed9f Support <= operator for date/timestamp data types and some minor refactors.
e4dccf46 Implement InlineFunction to force inline at call site
71170942 Mock postmaster.o for codegen_framework_unittest.t
09f00581 Codegen expressions that contain plus, minus and mul operators on float8 data types
d7fb2f6d Fix codegen unittests on Linux and various compiler warnings while building codegen.
45f2aa96 Fix test and update coding style This closes #874
1b26fbfc Enrolled targetlist for scan and aggregate
ebd1d014 Enhance codegen framework to support arbitrary expr tree
ec626ce6 Generate and enroll naive ExecEvalExpr in Scan's quals
1186001e Revert "Create naive code-generated version of ExecQual"
6f928a65 Replace RegisterExternalFunction with GetOrRegisterExternalFunction using an unordered_map in codegen_utils
6ae0085b Move ElogWrapper to GpCodegenUtils.
d3f80b45 Add verifyfunction for generated llvm function.
7bcf094a Fix codegen compiler error with 8.3 merge
aae0ad3d Create naive code-generated version of ExecQual
dce266ad Minor code quality fixes in src/backend/codegen
d281f340 Support null attributes in code generated ExecVariableList
82fd418e Address a number of cpplint errors in codegen_utils_unittest.cc
887aa48d Add check for CodegenUtils::GetType for bool arrays
bb9b92c6 Enhance Base Codegen to do clean up when generation fails
b9ef5e3f Fix build error for Annotated types
a5cfefd9 Add support for array types in codegen_utils.
2b883384 Fix static_assert call
7b75d9ea This commit  generates code for code path: ExecVariableList > slot_getattr > _slot_getsomeattrs > slot_deform_tuple. This code path is executed during scan in simple select queries that do not have a where clause (e.g., select bar from foo;).
6d0a06e8 Fix CodeGen typos and CodeGeneratorManagerCreate function signature in gpcodegen_mock.c
4916a606 Add support for registering vararg external functions with codegen utils.
ae4a7754 Integrate codegen framework and make simple external call to slot deform tuple. This closes #649
ee5fb851 Renaming code_generator to codegen_utils and CodeGenerator to CodegenUtils. This closes #648
88e9baba Adding GPDB code generation utils
Signed-off-by: NJesse Zhang <sbjesse@gmail.com>
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>
上级 dab48b17
......@@ -157,22 +157,6 @@ Currently, GPDPB is built with PXF by default (--enable-pxf is on).
In order to build GPDB without pxf, simply invoke `./configure` with additional option `--disable-pxf`.
PXF requires curl, so `--enable-pxf` is not compatible with the `--without-libcurl` option.
### Building GPDB with code generation enabled
To build GPDB with code generation (codegen) enabled, you will need cmake 2.8 or higher
and a recent version of llvm and clang (include headers and developer libraries). Codegen utils
is currently developed against the LLVM 3.7.X release series. You can find more details about the codegen feature,
including details about obtaining the prerequisites, building and testing GPDB with codegen in the [Codegen README](src/backend/codegen).
In short, you can change the `configure` with additional option
`--enable-codegen`, optionally giving the path to llvm and clang libraries on
your system.
```
# Configure build environment to install at /usr/local/gpdb
# Enable CODEGEN
./configure --with-perl --with-python --with-libxml --enable-codegen --prefix=/usr/local/gpdb --with-codegen-prefix="/path/to/llvm;/path/to/clang"
```
### Building GPDB with gpperfmon enabled
gpperfmon tracks a variety of queries, statistics, system properties, and metrics.
......
......@@ -65,7 +65,6 @@ If a script file is not referenced in any of the directories it is considered ab
There are some exceptions to this rule.
Please do not create any more exceptions, and remove these as the occasion arises:
* `cpplint.py` is being used by codegen
* `package_tarball.bash` is being used by
[a gporca pipeline](https://github.com/greenplum-db/gporca/blob/master/concourse/pipeline.yml)
......
......@@ -9,10 +9,6 @@ import sys
from builds.GpBuild import GpBuild
CODEGEN_MODE = 'codegen'
ORCA_CODEGEN_DEFAULT_MODE = "orca_codegen"
ORCA_MODE = 'orca'
PLANNER_MODE = 'planner'
INSTALL_DIR = "/usr/local/gpdb"
DEPENDENCY_INSTALL_DIR = "/usr/local"
......
import os
import subprocess
import sys
from GpdbBuildBase import GpdbBuildBase
class GporcacodegenBuild(GpdbBuildBase):
def __init__(self):
pass
def configure(self):
return subprocess.call(' '.join(["./configure",
"--enable-codegen",
"--enable-mapreduce",
"--with-perl",
"--with-libxml",
"--with-python",
"--disable-gpfdist",
"--prefix=/usr/local/gpdb"]),
cwd="gpdb_src", shell=True)
def icg(self):
status = subprocess.call(
"printf '\nLD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib\nexport \
LD_LIBRARY_PATH' >> /usr/local/gpdb/greenplum_path.sh", shell=True)
if status:
return status
status = subprocess.call([
"runuser gpadmin -c \"source /usr/local/gpdb/greenplum_path.sh \
&& make create-demo-cluster\""], cwd="gpdb_src/gpAux/gpdemo", shell=True)
if status:
return status
return subprocess.call([
"runuser gpadmin -c \"source /usr/local/gpdb/greenplum_path.sh \
&& source gpAux/gpdemo/gpdemo-env.sh && PGOPTIONS='-c optimizer=on -c codegen=on' \
make installcheck-good\""], cwd="gpdb_src", shell=True)
......@@ -243,28 +243,6 @@ AC_DEFUN([PGAC_CHECK_STRIP],
])# PGAC_CHECK_STRIP
# GPAC_PATH_CMAKE
# ---------------
# Check for the 'cmake' program which is required for compiling
# Greenplum with Code Generation
AC_DEFUN([GPAC_PATH_CMAKE],
[
if test -z "$CMAKE"; then
AC_PATH_PROGS(CMAKE, cmake)
fi
if test -n "$CMAKE"; then
gpac_cmake_version=`$CMAKE --version 2>/dev/null | sed q`
if test -z "$gpac_cmake_version"; then
AC_MSG_ERROR([cmake is required for codegen, unable to identify version])
fi
AC_MSG_NOTICE([using $gpac_cmake_version])
else
AC_MSG_ERROR([cmake is required for codegen, unable to find binary])
fi
]) # GPAC_PATH_CMAKE
# GPAC_PATH_APR_1_CONFIG
# ----------------------
# Check for apr-1-config, used by gpfdist
......
......@@ -743,8 +743,6 @@ enable_mapreduce
enable_netbackup
enable_ddboost
enable_snmp
CMAKE
enable_codegen
enable_orca
autodepend
TAS
......@@ -864,8 +862,6 @@ enable_cassert
enable_debugntuplestore
enable_testutils
enable_orca
enable_codegen
with_codegen_prefix
enable_snmp
enable_ddboost
enable_netbackup
......@@ -1552,7 +1548,6 @@ Optional Features:
enable debug_ntuplestore (for debugging)
--enable-testutils enable testing utilities
--disable-orca disable ORCA optimizer
--enable-codegen enable code generation
--enable-snmp enable snmp for MIB and alerts via TRAP/INFORM
--enable-ddboost enable DD Boost support
--enable-netbackup enable NetBackup support
......@@ -1581,9 +1576,6 @@ Optional Packages:
--with-wal-segsize=SEGSIZE
set WAL segment size in MB [16]
--with-CC=CMD set compiler (deprecated)
--with-codegen-prefix="PATH;PATH"
semicolon separated prefix search path for LLVM and
Clang
--with-tcl build Tcl modules (PL/Tcl)
--with-tclconfig=DIR tclConfig.sh is in DIR
--with-perl build Perl modules (PL/Perl)
......@@ -6287,149 +6279,6 @@ fi
$as_echo "checking whether to build with ORCA... $enable_orca" >&6; }
#
# Enable code generation
#
pgac_args="$pgac_args enable_codegen"
# Check whether --enable-codegen was given.
if test "${enable_codegen+set}" = set; then :
enableval=$enable_codegen;
case $enableval in
yes)
$as_echo "#define USE_CODEGEN 1" >>confdefs.h
;;
no)
:
;;
*)
as_fn_error $? "no argument expected for --enable-codegen option" "$LINENO" 5
;;
esac
else
enable_codegen=no
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: checking whether to build with codegen... $enable_codegen" >&5
$as_echo "checking whether to build with codegen... $enable_codegen" >&6; }
pgac_args="$pgac_args with_codegen_prefix"
# Check whether --with-codegen-prefix was given.
if test "${with_codegen_prefix+set}" = set; then :
withval=$with_codegen_prefix;
case $withval in
yes)
as_fn_error $? "argument required for --with-codegen-prefix option" "$LINENO" 5
;;
no)
as_fn_error $? "argument required for --with-codegen-prefix option" "$LINENO" 5
;;
*)
if test "${enable_codegen}" = no; then :
{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: --with-codegen-prefix used without --enable-codegen" >&5
$as_echo "$as_me: WARNING: --with-codegen-prefix used without --enable-codegen" >&2;}
fi
;;
esac
fi
if test "$enable_codegen" = yes; then :
# then
if test -z "$CMAKE"; then
for ac_prog in cmake
do
# Extract the first word of "$ac_prog", so it can be a program name with args.
set dummy $ac_prog; ac_word=$2
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
$as_echo_n "checking for $ac_word... " >&6; }
if ${ac_cv_path_CMAKE+:} false; then :
$as_echo_n "(cached) " >&6
else
case $CMAKE in
[\\/]* | ?:[\\/]*)
ac_cv_path_CMAKE="$CMAKE" # Let the user override the test with a path.
;;
*)
as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
for as_dir in $PATH
do
IFS=$as_save_IFS
test -z "$as_dir" && as_dir=.
for ac_exec_ext in '' $ac_executable_extensions; do
if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
ac_cv_path_CMAKE="$as_dir/$ac_word$ac_exec_ext"
$as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5
break 2
fi
done
done
IFS=$as_save_IFS
;;
esac
fi
CMAKE=$ac_cv_path_CMAKE
if test -n "$CMAKE"; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $CMAKE" >&5
$as_echo "$CMAKE" >&6; }
else
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
$as_echo "no" >&6; }
fi
test -n "$CMAKE" && break
done
fi
if test -n "$CMAKE"; then
gpac_cmake_version=`$CMAKE --version 2>/dev/null | sed q`
if test -z "$gpac_cmake_version"; then
as_fn_error $? "cmake is required for codegen, unable to identify version" "$LINENO" 5
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: using $gpac_cmake_version" >&5
$as_echo "$as_me: using $gpac_cmake_version" >&6;}
else
as_fn_error $? "cmake is required for codegen, unable to find binary" "$LINENO" 5
fi
cd src/backend/codegen
if test "${prefix}" = NONE; then :
prefix=${ac_default_prefix}
fi
if test "${enable_debug}" = yes; then :
build_type="Debug"
else
build_type="Release"
fi
CMAKE_CMD="${CMAKE} -DCMAKE_PREFIX_PATH="${with_codegen_prefix}" -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_BUILD_TYPE=${build_type} ."
echo "Executing cmake command : ${CMAKE_CMD}"
if ${CMAKE_CMD}; then :
else
as_fn_error $? "'error while configuring codegen.'" "$LINENO" 5
fi
cd -
fi # fi
#
# --enable-snmp enables snmp mib and trap/inform
#
......
......@@ -778,34 +778,6 @@ PGAC_ARG_BOOL(enable, orca, yes, [disable ORCA optimizer],
AC_MSG_RESULT([checking whether to build with ORCA... $enable_orca])
AC_SUBST(enable_orca)
#
# Enable code generation
#
PGAC_ARG_BOOL(enable, codegen, no, [enable code generation],
[AC_DEFINE([USE_CODEGEN], 1,
[Define to 1 to enable code generation. (--enable-codegen)])])
AC_MSG_RESULT([checking whether to build with codegen... $enable_codegen])
AC_SUBST(enable_codegen)
PGAC_ARG_REQ(with, codegen-prefix,
[["PATH;PATH"]], [semicolon separated prefix search path for LLVM and Clang],
[AS_IF([test "${enable_codegen}" = no],
[AC_MSG_WARN([--with-codegen-prefix used without --enable-codegen])])])
AS_IF([test "$enable_codegen" = yes],
[ # then
GPAC_PATH_CMAKE
cd src/backend/codegen
AS_IF([test "${prefix}" = NONE], [prefix=${ac_default_prefix}])
AS_IF([test "${enable_debug}" = yes], [build_type="Debug"], [build_type="Release"])
CMAKE_CMD="${CMAKE} -DCMAKE_PREFIX_PATH="${with_codegen_prefix}" -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_BUILD_TYPE=${build_type} ."
echo "Executing cmake command : ${CMAKE_CMD}"
AS_IF([${CMAKE_CMD}], [], [AC_MSG_ERROR(['error while configuring codegen.'])])
cd -
]) # fi
#
# --enable-snmp enables snmp mib and trap/inform
#
......
......@@ -35,7 +35,7 @@ export MPP_ARCH=$($(BLD_ARCH)_MPP_ARCH)
endif
# take over the gcc version we use
BLD_CC=$(shell which gcc)
BLD_CC=gcc
hpux_ia64_CC=gcc
osx106_x86_CC=gcc
......
......@@ -89,7 +89,7 @@ _main() {
$HADOOPCMD fs -rm -f -r /mapred/*
# gphdfs_regress_schedule
PGOPTIONS="-c optimizer=off -c codegen=off -c gp_hadoop_home=${HADOOP_HOME} -c gp_hadoop_target_version=${GP_HADOOP_TARGET_VERSION}" ${PGREGRESS} --psqldir=$GPHOME/bin/ --init-file=$CURDIR/gphdfs_init_file --schedule=$CURDIR/gphdfs_regress_schedule --inputdir=$CURDIR/source_replaced --outputdir=.
PGOPTIONS="-c optimizer=off -c gp_hadoop_home=${HADOOP_HOME} -c gp_hadoop_target_version=${GP_HADOOP_TARGET_VERSION}" ${PGREGRESS} --psqldir=$GPHOME/bin/ --init-file=$CURDIR/gphdfs_init_file --schedule=$CURDIR/gphdfs_regress_schedule --inputdir=$CURDIR/source_replaced --outputdir=.
}
_main "$@"
......@@ -185,7 +185,6 @@ enable_coverage = @enable_coverage@
enable_thread_safety = @enable_thread_safety@
enable_largefile = @enable_largefile@
enable_orca = @enable_orca@
enable_codegen = @enable_codegen@
enable_gpfdist = @enable_gpfdist@
enable_pxf = @enable_pxf@
enable_gphdfs = @enable_gphdfs@
......
......@@ -29,10 +29,6 @@ ifeq ($(enable_orca),yes)
SUBDIRS += gpopt
endif
ifeq ($(enable_codegen),yes)
SUBDIRS += codegen
endif
include $(srcdir)/common.mk
# As of 1/2010:
......@@ -59,11 +55,6 @@ LIBS := $(filter-out -lpgport, $(LIBS)) $(LDAP_LIBS_BE)
# The backend doesn't need everything that's in LIBS, however
LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
# adding codegen libraries
ifeq ($(enable_codegen),yes)
LIBS := $(LIBS) -L$(top_builddir)/src/backend/codegen -lgpcodegen
endif
# Greenplum uses threads in the backend
LIBS := $(LIBS) -lpthread
......@@ -212,9 +203,6 @@ endif
ifeq ($(enable_orca), yes)
$(MAKE) -C gpopt $@ INSTLOC=$(DESTDIR)$(libdir)
endif
ifeq ($(enable_codegen), yes)
$(MAKE) -C codegen $@ DESTDIR=$(DESTDIR)
endif
install-bin: postgres $(POSTGRES_IMP) installdirs
$(INSTALL_PROGRAM) postgres$(X) '$(DESTDIR)$(bindir)/postgres$(X)'
......
......@@ -1129,10 +1129,7 @@ heap_deformtuple(HeapTuple tuple,
* re-computing information about previously extracted attributes.
* slot->tts_nvalid is the number of attributes already extracted.
*/
#ifndef USE_CODEGEN
static
#endif
void
static void
slot_deform_tuple(TupleTableSlot *slot, int natts)
{
HeapTuple tuple = TupGetHeapTuple(slot);
......@@ -1264,6 +1261,7 @@ _slot_getsomeattrs(TupleTableSlot *slot, int attnum)
slot_deform_tuple(slot, attno);
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
* rest as null
......
# Compiled Object files
*.slo
*.lo
*.o
*.obj
# Precompiled Headers
*.gch
*.pch
# Compiled Dynamic libraries
*.so
*.dylib
*.dll
# Fortran module files
*.mod
# Compiled Static libraries
*.lai
*.la
*.a
*.lib
# Executables
*.exe
*.out
*.app
*.t
# Mac Finder cache
.DS_Store
# Editor backup files
*~
cscope.*
cctree.*
# CMake files
CMakeFiles/
CMakeCache.txt
CTestTestfile.cmake
cmake_install.cmake
install_manifest.txt
Makefile
Testing
# Copyright 2015-2016 Pivotal Software, Inc.
#
# CMakeLists.txt
# Cmake configuration for building GPDB codegen module.
#
cmake_minimum_required(VERSION 2.8.12)
project(gpcodegen C CXX)
set(CMAKE_BUILD_FILES_DIRECTORY build)
set(CMAKE_BUILD_DIRECTORY build)
get_filename_component(TOP_SRC_DIR "../../.." ABSOLUTE)
set(MOCK_DIR ${TOP_SRC_DIR}/src/test/unit/mock)
# Options. Turn on with 'cmake -Dvar_name=ON'
option(build_examples "Build examples also" OFF)
# Look for flags to enable C++11 support.
include(CheckCXXCompilerFlag)
CHECK_CXX_COMPILER_FLAG("-std=c++11" COMPILER_HAS_STD_CXX11)
if (COMPILER_HAS_STD_CXX11)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
else()
CHECK_CXX_COMPILER_FLAG("-std=c++0x" COMPILER_HAS_STD_CXX0X)
if (COMPILER_HAS_STD_CXX0X)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x")
endif()
endif()
# Turn on all warnings.
CHECK_CXX_COMPILER_FLAG("-Wall" COMPILER_HAS_WALL)
if (COMPILER_HAS_WALL)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall")
endif()
#CHECK_CXX_COMPILER_FLAG("-pedantic" COMPILER_HAS_PEDANTIC)
#if (COMPILER_HAS_PEDANTIC)
# set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pedantic")
#endif()
#CHECK_CXX_COMPILER_FLAG("-fno-enforce-eh-specs" COMPILER_HAS_NO_ENFORCE_EH_SPECS)
#if (COMPILER_HAS_NO_ENFORCE_EH_SPECS)
# set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-enforce-eh-specs")
#endif()
# Suppress warnings about C99 extensions that should be supported in C++11 mode.
CHECK_CXX_COMPILER_FLAG("-Wno-c99-extensions" COMPILER_HAS_WNO_C99_EXTENSIONS)
if (COMPILER_HAS_WNO_C99_EXTENSIONS)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-c99-extensions")
endif()
# Suppress warnings about C99 extensions that should be supported in C++11 mode.
CHECK_CXX_COMPILER_FLAG("-Wno-conversion-null" COMPILER_HAS_WNO_C99_EXTENSIONS)
if (COMPILER_HAS_WNO_C99_EXTENSIONS)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-conversion-null")
endif()
# Suppress warnings about deprecated keyword register
CHECK_CXX_COMPILER_FLAG("-Wno-deprecated-register" COMPILER_HAS_WNO_C99_EXTENSIONS)
if (COMPILER_HAS_WNO_C99_EXTENSIONS)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-deprecated-register")
endif()
macro (ADD_DEBUG_COMPILE_DEFINITION SYMBOL)
if (CMAKE_MAJOR_VERSION GREATER 2)
cmake_policy(SET CMP0043 NEW)
set_property(
DIRECTORY
APPEND PROPERTY COMPILE_DEFINITIONS $<$<CONFIG:Debug>:${SYMBOL}>
)
else()
set_property(
DIRECTORY
APPEND PROPERTY COMPILE_DEFINITIONS_DEBUG ${SYMBOL}
)
endif()
endmacro()
# Turn on the CODEGEN_DEBUG flag if this is a debug build.
ADD_DEBUG_COMPILE_DEFINITION(CODEGEN_DEBUG)
# Check for POSIX I/O syscalls needed by TemporaryFile.
include(CheckCXXSymbolExists)
CHECK_CXX_SYMBOL_EXISTS(mkstemp "stdlib.h" HAVE_POSIX_MKSTEMP)
CHECK_CXX_SYMBOL_EXISTS(write "unistd.h" HAVE_POSIX_WRITE)
CHECK_CXX_SYMBOL_EXISTS(fsync "unistd.h" HAVE_POSIX_FSYNC)
if (HAVE_POSIX_MKSTEMP AND HAVE_POSIX_WRITE AND HAVE_POSIX_FSYNC)
set(codegen_tmpfile_sources utils/temporary_file.cc)
set_property(DIRECTORY
APPEND PROPERTY COMPILE_DEFINITIONS CODEGEN_HAVE_TEMPORARY_FILE)
else()
message(WARNING "Missing required POSIX I/O syscalls for temporary files. "
"Line-by-line DEBUG information for generated code will not "
"be available.")
endif()
# Include our include paths.
include_directories(${TOP_SRC_DIR}/src/include)
include_directories(include)
# Pull in LLVM libraries.
find_package(LLVM REQUIRED CONFIG)
message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}")
message(STATUS "Using LLVMConfig.cmake in: ${LLVM_DIR}")
include_directories(${LLVM_INCLUDE_DIRS})
add_definitions(${LLVM_DEFINITIONS})
# Disable RTTI (C++ run-time type information) if LLVM was built without it.
if (NOT LLVM_ENABLE_RTTI)
CHECK_CXX_COMPILER_FLAG("-fno-rtti" COMPILER_HAS_FNO_RTTI)
if (COMPILER_HAS_FNO_RTTI)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")
else()
message(WARNING "LLVM was built without RTTI (run-time type information) "
"support, but compiler does not support -fno-rtti flag to "
"also build gpcodegen without RTTI support. You may see "
"linking errors about undefined references to typeinfo for "
"various LLVM classes.")
endif()
endif()
set(CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/cmake")
# Some distros (Fedora, maybe others?) package LLVM as a single monolithic
# library instead of a shared library.
option(MONOLITHIC_LLVM_LIBRARY
"Look for a single monolithic LLVM library instead of modular libraries"
OFF)
if (MONOLITHIC_LLVM_LIBRARY)
find_package(LLVMMonolithic REQUIRED)
endif()
# Pull in Clang libraries using our custom CMake module.
find_package(Clang REQUIRED)
include_directories(${CLANG_INCLUDE_DIRS})
# Core codegen library.
# This builds all the codegen-related C++ sources into one library
add_library(gpcodegen SHARED
utils/clang_compiler.cc
utils/codegen_utils.cc
utils/gp_codegen_utils.cc
utils/gp_assert.cc
codegen_interface.cc
codegen_manager.cc
const_expr_tree_generator.cc
exec_variable_list_codegen.cc
slot_getattr_codegen.cc
exec_eval_expr_codegen.cc
expr_tree_generator.cc
op_expr_tree_generator.cc
pg_date_func_generator.cc
pg_numeric_func_generator.cc
var_expr_tree_generator.cc
advance_aggregates_codegen.cc
${codegen_tmpfile_sources})
if(APPLE)
set(WL_START_GROUP "")
set(WL_END_GROUP "")
else()
set(WL_START_GROUP "-Wl,--start-group")
set(WL_END_GROUP "-Wl,--end-group")
endif()
# By default, the Darwin linker throws an error if there are any undefined
# references in a dynamic library. Instead, it should wait till it's loaded
# by the postgres binary.
if(APPLE)
set(WL_UNDEFINED_DYNLOOKUP "-Wl,-undefined -Wl,dynamic_lookup")
else()
set(WL_UNDEFINED_DYNLOOKUP "")
endif()
target_link_libraries(gpcodegen ${WL_START_GROUP} ${CLANG_LIBRARIES} ${WL_END_GROUP} ${WL_UNDEFINED_DYNLOOKUP})
if (MONOLITHIC_LLVM_LIBRARY)
target_link_libraries(gpcodegen ${LLVM_MONOLITHIC_LIBRARIES})
else()
# Here we link against the LLVM libraries that we use directly, as well as
# those that are needed by the Clang libraries that we use (e.g. objcarcopts,
# which the Clang frontend requires even though we do not compile any
# objective-C). The llvm_map_components_to_libnames() function also takes care
# of pulling in any transitive linking dependencies for the libraries we
# specify.
llvm_map_components_to_libnames(codegen_llvm_libs
analysis bitwriter core executionengine ipo
irreader linker mc mcjit native objcarcopts
option passes support target)
target_link_libraries(gpcodegen ${WL_START_GROUP} ${codegen_llvm_libs} ${WL_END_GROUP})
endif()
# This macro checks to see if the given C symbol is defined in the given LIBRARY. A library with
# an appropriate name is searched for in the LIBPATH. VARIABLE is set to true if the symbol is
# found defined as a type in (T in the output of nm) in the library, or set false otherwise.
macro(CHECK_SYMBOL_DEFINED LIBRARY CSYMBOL LIBPATH VARIABLE)
find_library(LIBLOCATION ${LIBRARY} ${LIBPATH})
if(LIBLOCATION STREQUAL "LIBLOCATION-NOTFOUND")
message(FATAL_ERROR "${LIBRARY} not found in ${LIBPATH}.")
endif()
find_program(NM_BIN "nm")
execute_process(COMMAND ${NM_BIN} ${LIBLOCATION}
COMMAND grep "T" # Symbol is defined
COMMAND grep ${CSYMBOL}
RESULT_VARIABLE RETURN_CODE
OUTPUT_QUIET)
if(${RETURN_CODE} EQUAL 0) # One or more lines were selected
set(${VARIABLE} true)
elseif(${RETURN_CODE} EQUAL 1) # No lines were selected.
set(${VARIABLE} false)
else()
message(FATAL_ERROR "Attempted to determine it ${CSYMBOL} is defined in ${LIBLOCATION} "
"but the execution failed. Return code = ${RETURN_CODE}")
endif()
endmacro()
if(APPLE)
set(ASSERT_FUNCTION_TO_OVERRIDE "__assert_rtn")
elseif(UNIX)
set(ASSERT_FUNCTION_TO_OVERRIDE "__assert_fail")
endif()
if(CMAKE_BUILD_TYPE STREQUAL Debug)
# Since we want to override assert handling in GPDB, we need to make sure
# that LLVM and Clang libraries we depend on haven't done that already
# If they do, we simply report a WARNING, and skip assert overriding in GPDB.
CHECK_SYMBOL_DEFINED(LLVMSupport ${ASSERT_FUNCTION_TO_OVERRIDE} ${LLVM_LIBRARY_DIRS} LLVM_ASSERT_REDEFINED)
if (LLVM_ASSERT_REDEFINED)
message(WARNING
"Found ${ASSERT_FUNCTION_TO_OVERRIDE} redefined in LLVM libraries. "
"Disabling GPDB codegen assert handling! "
"To enable this, rebuild LLVM libraries with -DLLVM_ENABLE_CRASH_OVERRIDES=off.")
else()
ADD_DEBUG_COMPILE_DEFINITION(CODEGEN_GPDB_ASSERT_HANDLING)
endif()
endif()
get_filename_component(full_install_name_dir "${CMAKE_INSTALL_PREFIX}/lib" ABSOLUTE)
set_target_properties(
gpcodegen PROPERTIES
INSTALL_NAME_DIR ${full_install_name_dir}
MACOSX_RPATH ON)
# Integrate with GPDB build system.
# Here we compile the GPDB wrappers and link it with the gpcodegen shared
# library to create a binary SUBSYS.o as expected by GPDB make system. We
# invoke the linker with -nostdlib since we don't really want to create a full
# executable.
add_executable(SUBSYS.o codegen_wrapper.cc)
set_target_properties(SUBSYS.o
PROPERTIES
LINK_FLAGS "-Wl,-r -nostdlib")
target_link_libraries(
gpcodegen
)
# Integrate with GPDB build system
# GPDB unit tests use unittest-check instead of test, so we add an alias target
# that calls the ctests we registered above.
add_custom_target(check
COMMAND ${CMAKE_CTEST_COMMAND})
add_custom_target(unittest-check
COMMAND ${CMAKE_COMMAND} .
COMMAND ${CMAKE_MAKE_PROGRAM} check
)
# Googletest framework for tests.
SET(GTEST_DIR ../../../gpAux/extensions/googletest/googletest)
add_subdirectory(${GTEST_DIR} ${CMAKE_BINARY_DIR}/gtest EXCLUDE_FROM_ALL)
enable_testing()
set(TEST_LIB_INC_DIRECTORIES
${TOP_SRC_DIR}/src/test/unit/cmockery
${GTEST_DIR}/include)
function(prepend_path var prefix)
SET(listVar "")
FOREACH(f ${ARGN})
LIST(APPEND listVar "${prefix}/${f}")
ENDFOREACH(f)
SET(${var} "${listVar}" PARENT_SCOPE)
endfunction(prepend_path)
# Usage add_cmock_gtest ${TEST_NAME} ${TEST_SOURCES} ${MOCK_DIR}/hello_mock.o ${MOCK_DIR}/world_mock.o)
function(add_cmockery_gtest TEST_NAME TEST_SOURCES)
set(FILES_TO_LINK ${OBJFILES})
foreach(MOCK_OBJ_NAME ${ARGN})
string(REPLACE "_mock" "" REAL_OBJ_NAME ${MOCK_OBJ_NAME})
string(REPLACE "${MOCK_DIR}" "${TOP_SRC_DIR}/src" REAL_OBJ_NAME ${REAL_OBJ_NAME})
list(REMOVE_ITEM FILES_TO_LINK ${REAL_OBJ_NAME})
if(EXISTS ${MOCK_OBJ_NAME})
list(APPEND FILES_TO_LINK ${MOCK_OBJ_NAME})
else()
string(REPLACE ".o" ".c" REAL_SRC_NAME ${REAL_OBJ_NAME})
string(REPLACE ".o" ".c" MOCK_SRC_NAME ${MOCK_OBJ_NAME})
get_filename_component(REAL_SRC_ABS_PATH ${REAL_SRC_NAME} ABSOLUTE)
execute_process(COMMAND python mocker.py ${REAL_SRC_ABS_PATH}
WORKING_DIRECTORY ${MOCK_DIR})
list(APPEND FILES_TO_LINK ${MOCK_SRC_NAME})
endif()
endforeach()
add_executable(${TEST_NAME} EXCLUDE_FROM_ALL
${TEST_SOURCES}
codegen_wrapper.cc
${FILES_TO_LINK}
${MOCK_OBJS}
${CMOCKERY_OBJS}
)
target_include_directories(${TEST_NAME} PUBLIC ${TEST_LIB_INC_DIRECTORIES})
# Bring these from $ENV{LIBS}
target_link_libraries(${TEST_NAME} "-ldl -lnetsnmp -lpam -lxml2 -lpgport -lbz2 -lrt -lssl -lcrypto -lkrb5 -lcom_err -lgssapi_krb5 -lz -lldap -lreadline -lcrypt -lm -lcurl -L${CMAKE_INSTALL_PREFIX}/lib -L../../port -lpgport_srv" gpcodegen gtest)
add_test(${TEST_NAME} ${TEST_NAME})
add_dependencies(check ${TEST_NAME})
endfunction(add_cmockery_gtest)
# Get the list for all real objects from objfiles.txt for CMockery integration
set(TXT_OBJFILE ${TOP_SRC_DIR}/src/backend/objfiles.txt)
if(EXISTS ${TXT_OBJFILE})
file(GLOB_RECURSE GPOPT_OBJS RELATIVE ${TOP_SRC_DIR} ${TOP_SRC_DIR}/src/backend/gpopt/*.o)
set(EXCL_OBJS
src/backend/main/main.o
${GPOPT_OBJS})
file(READ ${TXT_OBJFILE} OBJFILES)
string(REPLACE "\n" ";" OBJFILES "${OBJFILES}")
string(REPLACE " " ";" OBJFILES "${OBJFILES}")
set(OBJFILES ${OBJFILES})
foreach(F ${OBJFILES})
get_filename_component(_F ${F} NAME)
if ("${_F}" STREQUAL "objfiles.txt")
list(REMOVE_ITEM OBJFILES ${F})
endif()
endforeach()
foreach(OBJ ${EXCL_OBJS})
list(REMOVE_ITEM OBJFILES ${OBJ})
endforeach()
prepend_path(OBJFILES ${TOP_SRC_DIR} ${OBJFILES})
set(MOCK_OBJS "")
foreach(OBJ
${TOP_SRC_DIR}/src/test/unit/mock/main_mock.o
${TOP_SRC_DIR}/src/test/unit/mock/gpopt_mock.o
)
if(EXISTS ${OBJ})
set(MOCK_OBJS ${OBJ} ${MOCK_OBJS})
endif()
endforeach()
set(CMOCKERY_DIR ${TOP_SRC_DIR}/src/test/unit/cmockery)
set(CMOCKERY_OBJS ${CMOCKERY_DIR}/cmockery.o)
endif()
# Add CMockery tests
# All tests must be linked with postgres, even if they don't need the cmockery
# framework since libgpcodegen may now have references to postgres symbols.
#
# Usage add_cmock_gtest ${TEST_NAME} ${TEST_SOURCES} ${MOCK_DIR}/hello_mock.o ${MOCK_DIR}/world_mock.o)
if(EXISTS ${TXT_OBJFILE})
add_cmockery_gtest(codegen_framework_unittest.t
tests/codegen_framework_unittest.cc
)
add_cmockery_gtest(codegen_pg_func_generator_unittest.t
tests/codegen_pg_func_generator_unittest.cc
)
add_cmockery_gtest(clang_compiler_unittest.t
tests/clang_compiler_unittest.cc
)
add_cmockery_gtest(instance_method_wrappers_unittest.t
tests/instance_method_wrappers_unittest.cc
)
add_cmockery_gtest(codegen_utils_unittest.t
tests/codegen_utils_unittest.cc
)
add_cmockery_gtest(gp_codegen_utils_unittest.t
tests/gp_codegen_utils_unittest.cc
)
endif()
# Examples
if (build_examples)
add_subdirectory(example)
endif()
# Installation
install(TARGETS gpcodegen DESTINATION lib)
# Clean up
set(CMAKE_FILES Testing CMakeCache.txt CTestTestfile.cmake cmake_install.cmake install_manifest.txt Makefile)
set_directory_properties(PROPERTIES ADDITIONAL_MAKE_CLEAN_FILES "${CMAKE_FILES}")
# GPDB codegen utils (src/backend/codegen/utils)
Support library for integrating LLVM code-generation in GPDB.
This augments the core functionality provided by the LLVM C++ libraries for
program construction, optimization, and execution with additional features to
help runtime-generated code integrate smoothly with statically compiled code.
Key features include:
* A `CodegenUtils` class that encapsulates LLVM modules, optimization passes,
and execution engines, managing the whole lifetime of generated code.
* Fully automatic mapping between C++ types and LLVM IR types (including
function types of any arity) using templates.
* A type-safe and fully automatic foreign function interface that allows any
statically compiled function or external function pointer to be called by
generated code, and vice versa, to allow statically compiled code to get
pointers to JIT-compiled functions that can be called.
* A family of wrapper function templates that allow instance methods of C++
classes to be easily used with the foreign-function interface without the need
to manually write bespoke wrapper code.
* A frontend based on Clang for compiling in-memory C/C++ source code snippets
for code generation, including foreign-function support that has feature
parity with the IR version.
* Optional generation of debugging information so that runtime-generated and JIT
compiled C++ code can be interactively debugged by GDB just like statically
compiled code.
# Contents
1. [Building Codegen Utils](#building-codegen-utils)
2. [Coding Guidelines](#coding-guidelines)
3. [Debugging Generated Code](#debugging-generated-code)
4. [Learning Resources](#learning-resources)
5. [Codegen GPDB Function](#codegen-gpdb-function)
## Building Codegen Utils
### Prerequisites
To build codegen utils, you will need cmake 2.8 or higher and a recent version of
llvm and clang (including headers and developer libraries) (codegen utils is
currently developed against the LLVM 3.7.X release series). Here's how to
acquire these dependencies for various OSes.
#### Gentoo Linux
Gentoo has the latest stable LLVM in portage. It's easy to install like any
other package:
```
sudo emerge --sync
sudo emerge -n cmake llvm clang
```
#### Debian or Ubuntu Linux
##### Latest OS Versions
Debian 9 "Stretch" and Ubuntu 15.10 "Wily" and above provide packages for the
necessary versions of cmake, LLVM, and dependencies that can be installed as
follows:
```
sudo apt-get update
sudo apt-get install -y build-essential zlib1g-dev libedit-dev cmake llvm-3.7 llvm-3.7-dev clang-3.7 libclang-3.7-dev
```
##### Older Releases
Older versions of Debian and Ubuntu do not package the latest LLVM version
required, but you may be able to install it from the
[APT packages provided by the LLVM project](http://llvm.org/apt/). This should
work with Debian 8 "Jessie" or Ubuntu 14.04 "Trusty" and later.
First, install some necessary prerequisite development pacakges.
```
sudo apt-get update
sudo apt-get install -y build-essential zlib1g-dev libedit-dev cmake wget
```
Next, edit `/etc/apt/sources.list` and add these two lines (example is for Debian
Jessie, [use the appropriate directories for your OS listed here](http://llvm.org/apt/)).
```
deb http://llvm.org/apt/jessie/ llvm-toolchain-jessie-3.7 main
deb-src http://llvm.org/apt/jessie/ llvm-toolchain-jessie-3.7 main
```
Now, add the GPG key for the LLVM repo to your trusted keys, rerun
`apt-get update`, and install the LLVM packages like so:
```
wget -O - http://llvm.org/apt/llvm-snapshot.gpg.key|sudo apt-key add -
sudo apt-get update
sudo apt-get install -y llvm-3.7 llvm-3.7-dev clang-3.7 libclang-3.7-dev
```
#### FreeBSD
FreeBSD 10.2-STABLE comes with LLVM and Clang 3.4.1 installed by default, but
it also provides the more recent versions that we need in the package manager.
They can be installed like this:
```
sudo pkg update
sudo pkg install -y cmake llvm37 clang37
```
When building codegen, you will need to point cmake at the more recent LLVM
like so:
```
./configure --enable-codegen --with-codegen-prefix=/usr/local/llvm37
```
#### Mac OS X
Installing LLVM and clang libraries is easiest if you use an add-on package
manager like [Homebrew](http://brew.sh/). Do not worry about conflicts with
Apple's development tools and distribution of LLVM, Homebrew will install the
side-by-side and NOT override the defaults.
If you do not already have homebrew, you can install it with this shell snippet:
```
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
```
As of this writing, the "stable" version of LLVM in Homebrew is 3.6. To install
the newer LLVM 3.7 libraries, do the following:
```
brew tap homebrew/versions
brew install --with-clang llvm37
```
Unfortunately, the cmake configuration module for LLVM that is installed by
brew is broken. Fortunately, we can fix it. Edit
`/usr/local/opt/llvm37/lib/llvm-3.7/share/llvm/cmake/LLVMConfig.cmake` and fix
the path detection logic at the front of the file (before
`set(LLVM_VERSION_MAJOR 3)`) to look like this:
```
# Compute the CMake directory from the LLVMConfig.cmake file location.
get_filename_component(_LLVM_CMAKE_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
# Compute the installation prefix from the LLVMConfig.cmake file location.
get_filename_component(LLVM_INSTALL_PREFIX "${CMAKE_CURRENT_LIST_FILE}" PATH)
get_filename_component(LLVM_INSTALL_PREFIX "${LLVM_INSTALL_PREFIX}" PATH)
get_filename_component(LLVM_INSTALL_PREFIX "${LLVM_INSTALL_PREFIX}" PATH)
get_filename_component(LLVM_INSTALL_PREFIX "${LLVM_INSTALL_PREFIX}" PATH)
set(_LLVM_LIBRARY_DIR "${LLVM_INSTALL_PREFIX}/lib")
```
Because the version of LLVM installed by brew is not installed in one of the
default locations, you will need to point cmake at it when building codegen
like so:
```
./configure --enable-codegen --with-codegen-prefix=/usr/local/opt/llvm37/lib/llvm-3.7
```
### Testing
You can run unit tests as follows
```
make -C src/backend/codegen unittest-check
```
## Coding Guidelines
This module is written using the
[Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html).
We have a few additional style rules that aren't covered by the guide:
* Publicly visible classes (and their public methods) should have doxygen
comments explaining them. Private methods/members and internal helper classes
don't need full doxygen comments, but some ordinary comments to clarify and
improve readability are often a good idea.
* Sigils indicating a pointer (`*`) or reference (`&` or `&&` for
rvalue-reference) type should be written next to the name of the type, not the
name of a variable or function (e.g. write `int* my_pointer;` not
`int *my_pointer;`.
## Debugging Generated Code
### Viewing Generated Code
LLVM Modules have a `dump()` method that prints out the complete IR source for
a module to stderr. Similarly, when generating C++ source, an LLVM Twine's
complete text can be printed to stderr by the `dump()` method. Finally, if
errors are encountered during processing of C++ source by
`ClangCompiler::CompileCppSource()` clang error messages will be logged to
stderr.
### Validating Generated Code
LLVM provides some functions in the header `llvm/IR/Verifier.h` that can be used
to verify that modules (`verifyModule()`) and individual functions
(`verifyFunction()`) are well-formed before they are compiled. When compiling
C++ source code, the return value of `ClangCompiler::CompileCppSource()`
indicates whether compilation was successful.
### Interactive Debugging with GDB
GDB can be used to debug generated C++ source code just like statically
compiled C++ source. Calling `ClangCompiler::CompileCppSource()` with the option
`debug = true` causes DWARF debugging information to be included with the
compiled code and dumps the source code to a temporary file, enabling the usual
GDB features. Some caveats apply:
1. Debugging for JITed code only works on platforms that use the
[ELF](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) binary
format. This includes Linux and all the major BSD flavors, but does NOT
include Mac OS X (which uses Mach-O).
2. LLVM's MCJIT execution engine uses special hooks to expose symbols to the
debugger. These hooks are only supported by GDB 7.0+ (and appear to be only
partially supported by LLDB, so debugging with GDB is recommended).
When running an executable under GDB, you can set a breakpoint on a generated
function that hasn't been compiled yet. GDB will alert you that the function is
not defined and ask if you want to make the breakpoint pending on a future
shared library load (say yes).
Currently full debugging symbols are only generated when compiling from C++
source. Work is in progress to expose debugging information for generated IR.
## Learning Resources
We have collected some documentation and resources to learn more about using
LLVM generally and about some specific programming techniques that are used in
codegen. When we refer to "LLVM IR" below, that means LLVM intermediate
representation, which is the internal language used to build up programs in
LLVM. IR is assembly-like, but it is strongly-typed, uses static single
assignment, and has the illusion of unlimited registers.
### LLVM Resources
* [LLVM Tutorial](http://llvm.org/docs/tutorial/) A guided excercise in
creating an interpreter for a "toy" language using LLVM. Covers the basics
of creating values, expressions, control flow, and functions in LLVM IR using
the C++ IRBuilder interface.
* [LLVM Language Reference Manual](http://llvm.org/docs/LangRef.html) A
detailed reference about the LLVM IR language. Especially useful are the
sections labelled "Type System", "Instruction Reference", and "Intrinsic
Functions".
* [LLVM Programmer's Manual](http://llvm.org/docs/ProgrammersManual.html) A
guide to working with the LLVM C++ code base. This describes some of the
general-purpose APIs and data structures that are used internally by LLVM,
and also provides a useful guide to the most important parts of the C++ API
for LLVM IR objects in the section labelled "The Core LLVM Class Hierarchy
Reference".
* [LLVM Doxygen](http://llvm.org/doxygen/) A more or less complete code
reference for LLVM. Contains automatically-generated documentation for almost
every publicly visible class and method in the LLVM source.
### C++ Resources
* [Template Specialization](http://www.cprogramming.com/tutorial/template_specialization.html)
A brief introduction to template specialization and partial specialization,
which can be used to make a special version of a class template for a
specific type or category of types.
* [SFINAE](https://en.wikipedia.org/wiki/Substitution_failure_is_not_an_error)
A wikipedia article on the SFINAE (substitution failure is not an error)
idiom in C++. SFINAE helps resolve multiple different overloads or partial
specializations that might apply when instantiating a template down to just
one by eliminating versions of a template that can't possibly "fit".
* [C++ type_traits library](http://en.cppreference.com/w/cpp/header/type_traits)
A reference guide to the `type_traits` header introduced to the standard
library in C++11. `type_traits` provides many useful templates to help with
template metaprogramming. An especially important one is
[std::enable_if](http://en.cppreference.com/w/cpp/types/enable_if), which
is used to selectively enable a partial template specialization depending on
a compile-time boolean constant.
* [C++ Variadic Templates Guide](http://eli.thegreenplace.net/2014/variadic-templates-in-c/)
A nice guided tour of variadic templates in C++11 and how to use them to make
variadic code that is both type-safe and fast.
## Codegen GPDB Function
In order to facilitate codegen GPDB function, we introduced the following classes:
* `CodegenInterface` - Interface for all code generators.
* `BaseCodegen` - Inherits CodegenInterface and provides common implementation for all
generators that derive from it.
* `CodegenManager` - Manages all CodegenInterface.
More details for above classes are available in doxygen document or in respective source code.
Below are the necessary steps to codegen a target function *F* (e.g. `slot_deform_tuple`) which is
access through an operator.
1. Create a CodegenManager instance in the corresponding operator.
2. Create a new generator class *GC* (E.g. `SlotDeformTupleCodegen`) that derives from BaseCodegen and
implements a function that generartes IR instructions / C++ code for *F*.
3. Store *GC* and a function pointer to *F* in an appropriate struct. For instance the proper struct
for `slot_deform_tuple` is `TupleTableSlot`.
4. Enroll *GC* with the manager for current operator during `ExecInit`. Enrollment
process makes sure that the function pointer initally points to the regular version of *F*.
5. Replace the actual function call in GPDB with a call to the above function pointer.
After `ExecInit`, manager uses `CodegenInterface` to generate the runtime code. On successful generation,
manager swaps the function pointer (see Step 3) to point to the generated version of *F*. Note that, *GC*
stores the target function *F* along with a reference to a function pointer to *F* (see Step 3). This
mechanism allows manager to fallback on the regular version of *F* when code generation fails.
\ No newline at end of file
# Copyright 2015-2016 Pivotal Software, Inc.
#
# FindClang.cmake
# Module to find necessary Clang libraries.
# Some distros (Fedora, maybe others?) package clang as a single monolithic
# library instead of a shared library.
option(MONOLITHIC_CLANG_LIBRARY
"Look for a single monolithic clang library instead of modular libraries"
OFF)
find_path(CLANG_INCLUDE_DIR clang/Tooling/Tooling.h
HINTS ${LLVM_INCLUDE_DIRS})
set(CLANG_INCLUDE_DIRS ${CLANG_INCLUDE_DIR})
if (MONOLITHIC_CLANG_LIBRARY)
find_library(CLANG_LIBRARY
NAMES clang libclang
HINTS ${LLVM_LIBRARY_DIRS})
set(CLANG_LIBVARS CLANG_LIBRARY)
set(CLANG_LIBRARIES ${CLANG_LIBRARY})
else()
set(CLANG_COMPONENTS
CodeGen Frontend Tooling AST Basic Lex Driver Edit Parse Sema
Serialization ASTMatchers Rewrite ToolingCore Analysis)
foreach(CLANG_COMPONENT ${CLANG_COMPONENTS})
find_library(CLANG_${CLANG_COMPONENT}_LIBRARY
NAMES clang${CLANG_COMPONENT} libclang${CLANG_COMPONENT}
HINTS ${LLVM_LIBRARY_DIRS})
set(CLANG_LIBVARS ${CLANG_LIBVARS} CLANG_${CLANG_COMPONENT}_LIBRARY)
set(CLANG_LIBRARIES ${CLANG_LIBRARIES} ${CLANG_${CLANG_COMPONENT}_LIBRARY})
endforeach()
endif()
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Clang DEFAULT_MSG
${CLANG_LIBVARS}
CLANG_INCLUDE_DIR)
mark_as_advanced(CLANG_INCLUDE_DIR
${CLANG_LIBVARS})
# Copyright 2015-2016 Pivotal Software, Inc.
#
# FindLLVMMonolithic.cmake
# Module to find a single monolithic LLVM library (LLVM is packaged this way on
# Fedora, for example).
find_library(LLVM_MONOLITHIC_LIBRARY
NAMES llvm libllvm LLVM libLLVM llvm-3.7 libllvm-3.7 LLVM-3.7 libLLVM-3.7
HINTS ${LLVM_LIBRARY_DIRS} /usr/lib/llvm /usr/lib64/llvm /usr/lib32/llvm)
set(LLVM_MONOLITHIC_LIBRARIES ${LLVM_MONOLITHIC_LIBRARY})
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(LLVM-monolithic DEFAULT_MSG
LLVM_MONOLITHIC_LIBRARY)
mark_as_advanced(LLVM_MONOLITHIC_LIBRARY)
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// codegen_interface.cc
//
// @doc:
// Implementation of codegen interface's static function
//
//---------------------------------------------------------------------------
#include "codegen/codegen_interface.h"
#include <string>
using gpcodegen::CodegenInterface;
// Initalization of unique counter
unsigned CodegenInterface::unique_counter_ = 0;
std::string CodegenInterface::GenerateUniqueName(
const std::string& orig_func_name) {
return orig_func_name + std::to_string(unique_counter_++);
}
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// codegen_manager.cpp
//
// @doc:
// Implementation of a code generator manager
//
//---------------------------------------------------------------------------
#include <assert.h>
#include <iosfwd>
#include <memory>
#include <string>
#include <vector>
#include "llvm/Support/raw_ostream.h"
#include "codegen/codegen_interface.h"
#include "codegen/codegen_manager.h"
#include "codegen/codegen_wrapper.h"
#include "codegen/utils/codegen_utils.h"
#include "codegen/utils/gp_codegen_utils.h"
extern "C" {
#include "postgres.h" // NOLINT(build/include)
#include "utils/guc.h"
}
using gpcodegen::CodegenManager;
CodegenManager::CodegenManager(const std::string& module_name) {
module_name_ = module_name;
codegen_utils_.reset(new gpcodegen::GpCodegenUtils(module_name));
}
bool CodegenManager::EnrollCodeGenerator(
CodegenFuncLifespan funcLifespan, CodegenInterface* generator) {
// Only CodegenFuncLifespan_Parameter_Invariant is supported as of now
assert(funcLifespan == CodegenFuncLifespan_Parameter_Invariant);
assert(nullptr != generator);
enrolled_code_generators_.emplace_back(generator);
return true;
}
unsigned int CodegenManager::GenerateCode() {
// First, allow all code generators to initialize their dependencies
for (size_t i = 0; i < enrolled_code_generators_.size(); ++i) {
// NB: This list is still volatile at this time, as more generators may be
// enrolled as we iterate to initialize dependencies.
enrolled_code_generators_[i]->InitDependencies();
}
// Then ask them to generate code
unsigned int success_count = 0;
for (std::unique_ptr<CodegenInterface>& generator :
enrolled_code_generators_) {
success_count += generator->GenerateCode(codegen_utils_.get());
}
return success_count;
}
unsigned int CodegenManager::PrepareGeneratedFunctions() {
unsigned int success_count = 0;
// If no generator registered, just return with success count as 0
if (enrolled_code_generators_.empty()) {
return success_count;
}
STATIC_ASSERT_OPTIMIZATION_LEVEL(kNone,
CODEGEN_OPTIMIZATION_LEVEL_NONE);
STATIC_ASSERT_OPTIMIZATION_LEVEL(kLess,
CODEGEN_OPTIMIZATION_LEVEL_LESS);
STATIC_ASSERT_OPTIMIZATION_LEVEL(kDefault,
CODEGEN_OPTIMIZATION_LEVEL_DEFAULT);
STATIC_ASSERT_OPTIMIZATION_LEVEL(kAggressive,
CODEGEN_OPTIMIZATION_LEVEL_AGGRESSIVE);
// Call GpCodegenUtils to compile entire module
bool compilation_status = codegen_utils_->PrepareForExecution(
gpcodegen::GpCodegenUtils::OptimizationLevel(codegen_optimization_level),
true);
if (!compilation_status) {
return success_count;
}
// On successful compilation, go through all generator and swap
// the pointer so compiled function get called
gpcodegen::GpCodegenUtils* codegen_utils = codegen_utils_.get();
for (std::unique_ptr<CodegenInterface>& generator :
enrolled_code_generators_) {
success_count += generator->SetToGenerated(codegen_utils);
}
return success_count;
}
void CodegenManager::NotifyParameterChange() {
// no support for parameter change yet
assert(false);
}
bool CodegenManager::InvalidateGeneratedFunctions() {
// no support for invalidation of generated function
assert(false);
return false;
}
const std::string& CodegenManager::GetExplainString() {
return explain_string_;
}
void CodegenManager::AccumulateExplainString() {
explain_string_.clear();
// This is called only when EXPLAIN CODEGEN. Because we don't want to compile
// at this time, we need to call CodegenUtils::Optimize to "optimize" LLVM IR.
codegen_utils_->Optimize(gpcodegen::CodegenUtils::OptimizationLevel(
codegen_optimization_level),
gpcodegen::CodegenUtils::SizeLevel::kNormal,
false);
llvm::raw_string_ostream out(explain_string_);
codegen_utils_->PrintUnderlyingModules(out);
}
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// codegen_wrapper.cc
//
// @doc:
// C wrappers for initialization of code generator.
//
//---------------------------------------------------------------------------
#include "codegen/codegen_wrapper.h"
#include <assert.h>
#include <string>
#include <type_traits>
#include "codegen/codegen_config.h"
#include "codegen/base_codegen.h"
#include "codegen/codegen_manager.h"
#include "codegen/exec_eval_expr_codegen.h"
#include "codegen/exec_variable_list_codegen.h"
#include "codegen/expr_tree_generator.h"
#include "codegen/utils/gp_codegen_utils.h"
#include "codegen/advance_aggregates_codegen.h"
extern "C" {
#include "lib/stringinfo.h"
#include "postgres.h" // NOLINT(build/include)
}
using gpcodegen::CodegenManager;
using gpcodegen::BaseCodegen;
using gpcodegen::ExecVariableListCodegen;
using gpcodegen::ExecEvalExprCodegen;
using gpcodegen::AdvanceAggregatesCodegen;
// Current code generator manager that oversees all code generators
static void* ActiveCodeGeneratorManager = nullptr;
// Perform global set-up tasks for code generation. Returns 0 on
// success, nonzero on error.
unsigned int InitCodegen() {
return gpcodegen::GpCodegenUtils::InitializeGlobal();
}
void* CodeGeneratorManagerCreate(const char* module_name) {
if (!codegen) {
return nullptr;
}
return new CodegenManager(module_name);
}
unsigned int CodeGeneratorManagerGenerateCode(void* manager) {
if (!codegen) {
return 0;
}
return static_cast<CodegenManager*>(manager)->GenerateCode();
}
unsigned int CodeGeneratorManagerPrepareGeneratedFunctions(void* manager) {
if (!codegen) {
return 0;
}
return static_cast<CodegenManager*>(manager)->PrepareGeneratedFunctions();
}
unsigned int CodeGeneratorManagerNotifyParameterChange(void* manager) {
// parameter change notification is not supported yet
assert(false);
return 0;
}
void CodeGeneratorManagerAccumulateExplainString(void* manager) {
if (!codegen) {
return;
}
assert(nullptr != manager);
static_cast<CodegenManager*>(manager)->AccumulateExplainString();
}
char* CodeGeneratorManagerGetExplainString(void* manager) {
if (!codegen) {
return nullptr;
}
StringInfo return_string = makeStringInfo();
appendStringInfoString(
return_string,
static_cast<CodegenManager*>(manager)->GetExplainString().c_str());
return return_string->data;
}
void CodeGeneratorManagerDestroy(void* manager) {
delete (static_cast<CodegenManager*>(manager));
}
void* GetActiveCodeGeneratorManager() {
return ActiveCodeGeneratorManager;
}
void SetActiveCodeGeneratorManager(void* manager) {
ActiveCodeGeneratorManager = manager;
}
Datum
slot_getattr_regular(TupleTableSlot *slot, int attnum, bool *isnull) {
return slot_getattr(slot, attnum, isnull);
}
int
att_align_nominal_regular(int cur_offset, char attalign) {
return att_align_nominal(cur_offset, attalign);
}
void SET_VARSIZE_regular(void* ptr, size_t len) {
SET_VARSIZE(ptr, len);
}
uint32
VARSIZE_regular(void* ptr) {
return VARSIZE(ptr);
}
void* ExecVariableListCodegenEnroll(
ExecVariableListFn regular_func_ptr,
ExecVariableListFn* ptr_to_chosen_func_ptr,
ProjectionInfo* proj_info,
TupleTableSlot* slot) {
CodegenManager* manager = static_cast<CodegenManager*>(
GetActiveCodeGeneratorManager());
ExecVariableListCodegen* generator =
CodegenManager::CreateAndEnrollGenerator<ExecVariableListCodegen>(
manager,
regular_func_ptr,
ptr_to_chosen_func_ptr,
proj_info,
slot);
return generator;
}
void* ExecEvalExprCodegenEnroll(
ExecEvalExprFn regular_func_ptr,
ExecEvalExprFn* ptr_to_chosen_func_ptr,
ExprState *exprstate,
ExprContext *econtext,
PlanState* plan_state) {
CodegenManager* manager = static_cast<CodegenManager*>(
GetActiveCodeGeneratorManager());
ExecEvalExprCodegen* generator =
CodegenManager::CreateAndEnrollGenerator<ExecEvalExprCodegen>(
manager,
regular_func_ptr,
ptr_to_chosen_func_ptr,
exprstate,
econtext,
plan_state);
return generator;
}
void* AdvanceAggregatesCodegenEnroll(
AdvanceAggregatesFn regular_func_ptr,
AdvanceAggregatesFn* ptr_to_chosen_func_ptr,
AggState *aggstate) {
CodegenManager* manager = static_cast<CodegenManager*>(
GetActiveCodeGeneratorManager());
AdvanceAggregatesCodegen* generator =
CodegenManager::CreateAndEnrollGenerator<AdvanceAggregatesCodegen>(
manager,
regular_func_ptr,
ptr_to_chosen_func_ptr,
aggstate);
return generator;
}
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// const_expr_tree_generator.cc
//
// @doc:
// Object that generate code for const expression.
//
//---------------------------------------------------------------------------
#include <assert.h>
#include <memory>
#include "codegen/const_expr_tree_generator.h"
#include "codegen/expr_tree_generator.h"
#include "codegen/utils/gp_codegen_utils.h"
#include "llvm/IR/Constant.h"
extern "C" {
#include "postgres.h" // NOLINT(build/include)
#include "nodes/execnodes.h"
#include "nodes/nodes.h"
#include "nodes/primnodes.h"
}
namespace llvm {
class Value;
} // namespace llvm
using gpcodegen::ConstExprTreeGenerator;
using gpcodegen::ExprTreeGenerator;
bool ConstExprTreeGenerator::VerifyAndCreateExprTree(
const ExprState* expr_state,
ExprTreeGeneratorInfo* gen_info,
std::unique_ptr<ExprTreeGenerator>* expr_tree) {
assert(nullptr != expr_state &&
nullptr != expr_state->expr &&
T_Const == nodeTag(expr_state->expr) &&
nullptr != expr_tree);
expr_tree->reset(new ConstExprTreeGenerator(expr_state));
return true;
}
ConstExprTreeGenerator::ConstExprTreeGenerator(const ExprState* expr_state) :
ExprTreeGenerator(expr_state, ExprTreeNodeType::kConst) {
}
bool ConstExprTreeGenerator::GenerateCode(GpCodegenUtils* codegen_utils,
const ExprTreeGeneratorInfo& gen_info,
llvm::Value** llvm_out_value,
llvm::Value* const llvm_isnull_ptr) {
assert(nullptr != llvm_out_value);
assert(nullptr != llvm_isnull_ptr);
auto irb = codegen_utils->ir_builder();
Const* const_expr = reinterpret_cast<Const*>(expr_state()->expr);
// const_expr->constvalue is a datum
*llvm_out_value = codegen_utils->GetConstant(const_expr->constvalue);
// *isNull = con->constisnull;
irb->CreateStore(
codegen_utils->GetConstant<bool>(const_expr->constisnull),
llvm_isnull_ptr);
return true;
}
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// exec_eval_expr_codegen.cc
//
// @doc:
// Generates code for ExecEvalExpr function.
//
//---------------------------------------------------------------------------
#include <assert.h>
#include <stddef.h>
#include <cstdint>
#include <memory>
#include <string>
#include "codegen/base_codegen.h"
#include "codegen/codegen_wrapper.h"
#include "codegen/exec_eval_expr_codegen.h"
#include "codegen/expr_tree_generator.h"
#include "codegen/op_expr_tree_generator.h"
#include "codegen/slot_getattr_codegen.h"
#include "codegen/utils/gp_codegen_utils.h"
#include "codegen/utils/utility.h"
#include "llvm/IR/Argument.h"
#include "llvm/IR/Constant.h"
#include "llvm/IR/IRBuilder.h"
extern "C" {
#include "postgres.h" // NOLINT(build/include)
#include "nodes/execnodes.h"
#include "utils/elog.h"
#include "executor/tuptable.h"
#include "nodes/nodes.h"
}
namespace llvm {
class BasicBlock;
class Function;
class Value;
} // namespace llvm
using gpcodegen::ExecEvalExprCodegen;
using gpcodegen::SlotGetAttrCodegen;
constexpr char ExecEvalExprCodegen::kExecEvalExprPrefix[];
ExecEvalExprCodegen::ExecEvalExprCodegen(
CodegenManager* manager,
ExecEvalExprFn regular_func_ptr,
ExecEvalExprFn* ptr_to_regular_func_ptr,
ExprState *exprstate,
ExprContext *econtext,
PlanState* plan_state)
: BaseCodegen(manager,
kExecEvalExprPrefix,
regular_func_ptr, ptr_to_regular_func_ptr),
exprstate_(exprstate),
plan_state_(plan_state),
gen_info_(econtext, nullptr, nullptr, nullptr, 0),
slot_getattr_codegen_(nullptr),
expr_tree_generator_(nullptr) {
}
bool ExecEvalExprCodegen::InitDependencies() {
OpExprTreeGenerator::InitializeSupportedFunction();
ExprTreeGenerator::VerifyAndCreateExprTree(
exprstate_, &gen_info_, &expr_tree_generator_);
// Prepare dependent slot_getattr() generation
PrepareSlotGetAttr();
return true;
}
void ExecEvalExprCodegen::PrepareSlotGetAttr() {
TupleTableSlot* slot = nullptr;
assert(nullptr != plan_state_);
switch (nodeTag(plan_state_)) {
case T_SeqScanState:
case T_TableScanState:
// Generate dependent slot_getattr() implementation for the given slot
if (gen_info_.max_attr > 0) {
slot = reinterpret_cast<ScanState*>(plan_state_)
->ss_ScanTupleSlot;
assert(nullptr != slot);
}
break;
case T_AggState:
// For now, we assume that tuples for the Aggs are already going to be
// deformed in which case, we can avoid generating and calling the
// generated slot_getattr(). This may not be true always, but calling the
// regular slot_getattr() will still preserve correctness.
break;
default:
elog(DEBUG1,
"Attempting to generate ExecEvalExpr for an unsupported operator!");
}
if (nullptr != slot) {
slot_getattr_codegen_ = SlotGetAttrCodegen::GetCodegenInstance(
manager(), slot, gen_info_.max_attr);
}
}
bool ExecEvalExprCodegen::GenerateExecEvalExpr(
gpcodegen::GpCodegenUtils* codegen_utils) {
assert(NULL != codegen_utils);
if (nullptr == exprstate_ ||
nullptr == exprstate_->expr ||
nullptr == gen_info_.econtext) {
return false;
}
if (nullptr == expr_tree_generator_.get()) {
return false;
}
// If slot_getattr_codegen_ is not set or generation fails
// we revert to use the external slot_getattr()
if (nullptr == slot_getattr_codegen_ ||
false == slot_getattr_codegen_->GenerateCode(codegen_utils)) {
gen_info_.llvm_slot_getattr_func =
codegen_utils->GetOrRegisterExternalFunction(slot_getattr_regular,
"slot_getattr_regular");
} else {
gen_info_.llvm_slot_getattr_func =
slot_getattr_codegen_->GetGeneratedFunction();
assert(nullptr != gen_info_.llvm_slot_getattr_func);
}
llvm::Function* exec_eval_expr_func = CreateFunction<ExecEvalExprFn>(
codegen_utils, GetUniqueFuncName());
// Function arguments to ExecVariableList
llvm::Value* llvm_isnull_arg = ArgumentByPosition(exec_eval_expr_func, 2);
// BasicBlock of function entry.
llvm::BasicBlock* llvm_entry_block = codegen_utils->CreateBasicBlock(
"entry", exec_eval_expr_func);
llvm::BasicBlock* llvm_error_block = codegen_utils->CreateBasicBlock(
"error_block", exec_eval_expr_func);
gen_info_.llvm_main_func = exec_eval_expr_func;
gen_info_.llvm_error_block = llvm_error_block;
auto irb = codegen_utils->ir_builder();
irb->SetInsertPoint(llvm_entry_block);
#ifdef CODEGEN_DEBUG
EXPAND_CREATE_ELOG(codegen_utils,
DEBUG1,
"Codegen'ed expression evaluation called!");
#endif
// Generate code from expression tree generator
llvm::Value* value = nullptr;
bool is_generated = expr_tree_generator_->GenerateCode(codegen_utils,
gen_info_,
&value,
llvm_isnull_arg);
if (!is_generated ||
nullptr == value) {
return false;
}
llvm::Value* llvm_ret_value = codegen_utils->CreateCppTypeToDatumCast(value);
irb->CreateRet(llvm_ret_value);
irb->SetInsertPoint(llvm_error_block);
irb->CreateRet(codegen_utils->GetConstant<int64_t>(0));
return true;
}
bool ExecEvalExprCodegen::GenerateCodeInternal(GpCodegenUtils* codegen_utils) {
bool isGenerated = GenerateExecEvalExpr(codegen_utils);
if (isGenerated) {
elog(DEBUG1, "ExecEvalExpr was generated successfully!");
return true;
} else {
elog(DEBUG1, "ExecEvalExpr generation failed!");
return false;
}
}
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// codegen_utils.cpp
//
// @doc:
// Contains different code generators
//
//---------------------------------------------------------------------------
#include <assert.h>
#include <stddef.h>
#include <algorithm>
#include <string>
#include <cstdint>
#include "codegen/base_codegen.h"
#include "codegen/codegen_wrapper.h"
#include "codegen/exec_variable_list_codegen.h"
#include "codegen/slot_getattr_codegen.h"
#include "codegen/utils/gp_codegen_utils.h"
#include "codegen/utils/utility.h"
#include "llvm/IR/Argument.h"
#include "llvm/IR/Constant.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Instructions.h"
extern "C" {
#include "postgres.h" // NOLINT(build/include)
#include "executor/tuptable.h"
#include "nodes/execnodes.h"
#include "utils/elog.h"
#include "access/tupdesc.h"
#include "nodes/pg_list.h"
}
namespace llvm {
class BasicBlock;
class Function;
class Value;
} // namespace llvm
using gpcodegen::ExecVariableListCodegen;
using gpcodegen::SlotGetAttrCodegen;
constexpr char ExecVariableListCodegen::kExecVariableListPrefix[];
ExecVariableListCodegen::ExecVariableListCodegen(
CodegenManager* manager,
ExecVariableListFn regular_func_ptr,
ExecVariableListFn* ptr_to_regular_func_ptr,
ProjectionInfo* proj_info,
TupleTableSlot* slot)
: BaseCodegen(manager,
kExecVariableListPrefix,
regular_func_ptr,
ptr_to_regular_func_ptr),
proj_info_(proj_info),
slot_(slot),
max_attr_(0),
slot_getattr_codegen_(nullptr) {
}
bool ExecVariableListCodegen::InitDependencies() {
assert(nullptr != proj_info_);
assert(nullptr != proj_info_->pi_targetlist);
// Find the largest attribute index in projInfo->pi_targetlist
max_attr_ = *std::max_element(
proj_info_->pi_varNumbers,
proj_info_->pi_varNumbers + list_length(proj_info_->pi_targetlist));
slot_getattr_codegen_ = SlotGetAttrCodegen::GetCodegenInstance(
manager(), slot_, max_attr_);
return true;
}
bool ExecVariableListCodegen::GenerateExecVariableList(
gpcodegen::GpCodegenUtils* codegen_utils) {
assert(nullptr != codegen_utils);
static_assert(sizeof(Datum) == sizeof(int64_t),
"sizeof(Datum) doesn't match sizeof(int64)");
if ( nullptr == proj_info_->pi_varSlotOffsets ) {
elog(DEBUG1,
"Cannot codegen ExecVariableList because varSlotOffsets are null");
return false;
}
// Only do codegen if all the elements in the target list are on the same
// tuple slot This is an assumption for scan nodes, but we fall back when
// joins are involved.
for (int i = list_length(proj_info_->pi_targetlist) - 1; i > 0; i--) {
if (proj_info_->pi_varSlotOffsets[i] !=
proj_info_->pi_varSlotOffsets[i-1]) {
elog(DEBUG1,
"Cannot codegen ExecVariableList because multiple slots to deform.");
return false;
}
}
// System attribute
if (max_attr_ <= 0) {
elog(DEBUG1, "Cannot generate code for ExecVariableList"
"because max_attr is negative (i.e., system attribute).");
return false;
} else if (max_attr_ > slot_->tts_tupleDescriptor->natts) {
elog(DEBUG1, "Cannot generate code for ExecVariableList"
"because max_attr is greater than natts.");
return false;
}
llvm::Function* exec_variable_list_func = CreateFunction<ExecVariableListFn>(
codegen_utils, GetUniqueFuncName());
auto irb = codegen_utils->ir_builder();
// BasicBlocks
llvm::BasicBlock* entry_block = codegen_utils->CreateBasicBlock(
"entry", exec_variable_list_func);
// BasicBlock for checking correct slot
llvm::BasicBlock* slot_check_block = codegen_utils->CreateBasicBlock(
"slot_check", exec_variable_list_func);
// BasicBlock for main.
llvm::BasicBlock* main_block = codegen_utils->CreateBasicBlock(
"main", exec_variable_list_func);
// BasicBlock for final computations.
llvm::BasicBlock* final_block = codegen_utils->CreateBasicBlock(
"final", exec_variable_list_func);
llvm::BasicBlock* fallback_block = codegen_utils->CreateBasicBlock(
"fallback", exec_variable_list_func);
// Generation-time constants
llvm::Value* llvm_max_attr = codegen_utils->GetConstant(max_attr_);
llvm::Value* llvm_slot = codegen_utils->GetConstant(slot_);
// Function arguments to ExecVariableList
llvm::Value* llvm_projInfo_arg = ArgumentByPosition(exec_variable_list_func,
0);
llvm::Value* llvm_values_arg = ArgumentByPosition(exec_variable_list_func, 1);
llvm::Value* llvm_isnull_arg = ArgumentByPosition(exec_variable_list_func, 2);
// Generate slot_getattr for attributes all the way to max_attr
llvm::Function* slot_getattr_func = nullptr;
// If slot_getattr_codegen_ is not set or generation fails
// we revert to use the external slot_getattr()
if (nullptr == slot_getattr_codegen_ ||
false == slot_getattr_codegen_->GenerateCode(codegen_utils)) {
slot_getattr_func = codegen_utils->GetOrRegisterExternalFunction(
slot_getattr_regular, "slot_getattr_regular");
} else {
slot_getattr_func = slot_getattr_codegen_->GetGeneratedFunction();
assert(nullptr != slot_getattr_func);
}
// In case the above generation failed, no point in continuing since that was
// the most crucial part of ExecVariableList code generation.
if (nullptr == slot_getattr_func) {
elog(DEBUG1, "Cannot generate code for ExecVariableList "
"because slot_getattr generation failed!");
return false;
}
// Entry block
// -----------
irb->SetInsertPoint(entry_block);
#ifdef CODEGEN_DEBUG
EXPAND_CREATE_ELOG(codegen_utils,
DEBUG1,
"Codegen'ed ExecVariableList called!");
#endif
irb->CreateBr(slot_check_block);
// Slot check block
// ----------------
irb->SetInsertPoint(slot_check_block);
llvm::Value* llvm_econtext =
irb->CreateLoad(codegen_utils->GetPointerToMember(
llvm_projInfo_arg, &ProjectionInfo::pi_exprContext));
// We want to fall back when ExecVariableList is called with a slot that's
// different from the one we generated the function (eg HashJoin). We also
// assume only 1 slot and that the slot is in a scan node ie from
// exprContext->ecxt_scantuple or varOffset = 0
llvm::Value* llvm_slot_arg =
irb->CreateLoad(codegen_utils->GetPointerToMember(
llvm_econtext, &ExprContext::ecxt_scantuple));
irb->CreateCondBr(
irb->CreateICmpEQ(llvm_slot, llvm_slot_arg),
main_block /* true */,
fallback_block /* false */);
// Main block
// ----------
irb->SetInsertPoint(main_block);
// Allocate a dummy int so that slot_getattr can write isnull out
llvm::Value* llvm_dummy_isnull =
irb->CreateAlloca(codegen_utils->GetType<bool>());
irb->CreateCall(slot_getattr_func, {
llvm_slot,
llvm_max_attr,
llvm_dummy_isnull
});
irb->CreateBr(final_block);
// Final Block
// -----------
irb->SetInsertPoint(final_block);
llvm::Value* llvm_slot_PRIVATE_tts_isnull /* bool* */ =
irb->CreateLoad(codegen_utils->GetPointerToMember(
llvm_slot, &TupleTableSlot::PRIVATE_tts_isnull));
llvm::Value* llvm_slot_PRIVATE_tts_values /* Datum* */ =
irb->CreateLoad(codegen_utils->GetPointerToMember(
llvm_slot, &TupleTableSlot::PRIVATE_tts_values));
// This code from ExecVariableList copies the contents of the isnull & values
// arrays in the slot to output variable from the function parameters to
// ExecVariableList
int *varNumbers = proj_info_->pi_varNumbers;
for (int i = list_length(proj_info_->pi_targetlist) - 1; i >= 0; i--) {
// *isnull = slot->PRIVATE_tts_isnull[attnum-1];
llvm::Value* llvm_isnull_from_slot_val =
irb->CreateLoad(irb->CreateInBoundsGEP(
llvm_slot_PRIVATE_tts_isnull,
{codegen_utils->GetConstant(varNumbers[i] - 1)}));
llvm::Value* llvm_isnull_ptr =
irb->CreateInBoundsGEP(llvm_isnull_arg,
{codegen_utils->GetConstant(i)});
irb->CreateStore(llvm_isnull_from_slot_val, llvm_isnull_ptr);
// values[i] = slot_getattr(varSlot, varNumber+1, &(isnull[i]));
llvm::Value* llvm_value_from_slot_val =
irb->CreateLoad(irb->CreateInBoundsGEP(
llvm_slot_PRIVATE_tts_values,
{codegen_utils->GetConstant(varNumbers[i] - 1)}));
llvm::Value* llvm_values_ptr =
irb->CreateInBoundsGEP(llvm_values_arg,
{codegen_utils->GetConstant(i)});
irb->CreateStore(llvm_value_from_slot_val, llvm_values_ptr);
}
irb->CreateRetVoid();
// Fall back Block
// ---------------
irb->SetInsertPoint(fallback_block);
EXPAND_CREATE_ELOG(codegen_utils,
DEBUG1,
"Falling back to regular ExecVariableList");
codegen_utils->CreateFallback<ExecVariableListFn>(
codegen_utils->GetOrRegisterExternalFunction(ExecVariableList,
"ExecVariableList"),
exec_variable_list_func);
return true;
}
bool ExecVariableListCodegen::GenerateCodeInternal(
GpCodegenUtils* codegen_utils) {
bool isGenerated = GenerateExecVariableList(codegen_utils);
if (isGenerated) {
elog(DEBUG1, "ExecVariableList was generated successfully!");
return true;
} else {
elog(DEBUG1, "ExecVariableList generation failed!");
return false;
}
}
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// expr_tree_generator.cc
//
// @doc:
// Base class for expression tree to generate code
//
//---------------------------------------------------------------------------
#include <cassert>
#include <memory>
#include "codegen/const_expr_tree_generator.h"
#include "codegen/expr_tree_generator.h"
#include "codegen/op_expr_tree_generator.h"
#include "codegen/var_expr_tree_generator.h"
extern "C" {
#include "postgres.h" // NOLINT(build/include)
#include "nodes/execnodes.h"
#include "utils/elog.h"
#include "nodes/nodes.h"
}
using gpcodegen::ExprTreeGenerator;
bool ExprTreeGenerator::VerifyAndCreateExprTree(
const ExprState* expr_state,
ExprTreeGeneratorInfo* gen_info,
std::unique_ptr<ExprTreeGenerator>* expr_tree) {
assert(nullptr != expr_state &&
nullptr != expr_state->expr &&
nullptr != expr_tree);
if (!(IsA(expr_state, FuncExprState) ||
IsA(expr_state, ExprState))) {
elog(DEBUG1, "Input expression state type (%d) is not supported",
expr_state->type);
return false;
}
expr_tree->reset(nullptr);
bool supported_expr_tree = false;
switch (nodeTag(expr_state->expr)) {
case T_OpExpr: {
supported_expr_tree = OpExprTreeGenerator::VerifyAndCreateExprTree(
expr_state, gen_info, expr_tree);
break;
}
case T_Var: {
supported_expr_tree = VarExprTreeGenerator::VerifyAndCreateExprTree(
expr_state, gen_info, expr_tree);
break;
}
case T_Const: {
supported_expr_tree = ConstExprTreeGenerator::VerifyAndCreateExprTree(
expr_state, gen_info, expr_tree);
break;
}
default : {
supported_expr_tree = false;
elog(DEBUG1, "Unsupported expression tree %d found",
nodeTag(expr_state->expr));
}
}
assert((!supported_expr_tree && nullptr == expr_tree->get()) ||
(supported_expr_tree && nullptr != expr_tree->get()));
return supported_expr_tree;
}
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// advance_aggregates_codegen.h
//
// @doc:
// Headers for AdvanceAggregates codegen.
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_ADVANCEAGGREGATES_CODEGEN_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_ADVANCEAGGREGATES_CODEGEN_H_
#include "codegen/base_codegen.h"
#include "codegen/codegen_wrapper.h"
#include "codegen/pg_func_generator_interface.h"
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
class AdvanceAggregatesCodegen: public BaseCodegen<AdvanceAggregatesFn> {
public:
/**
* @brief Constructor
*
* @param regular_func_ptr Regular version of the target function.
* @param ptr_to_chosen_func_ptr Reference to the function pointer that the
* caller will call.
* @param aggstate The AggState to use for generating code.
*
* @note The ptr_to_chosen_func_ptr can refer to either the generated
* function or the corresponding regular version.
*
**/
explicit AdvanceAggregatesCodegen(
CodegenManager* manager,
AdvanceAggregatesFn regular_func_ptr,
AdvanceAggregatesFn* ptr_to_regular_func_ptr,
AggState *aggstate);
virtual ~AdvanceAggregatesCodegen() = default;
protected:
/**
* @brief Generate code for advance_aggregates.
*
* @param codegen_utils
*
* @return true on successful generation; false otherwise.
*
* This implementation does not support percentile and ordered aggregates.
*
* If at execution time, we see any of the above types of attributes,
* we fall backs to the regular function.
*
*/
bool GenerateCodeInternal(gpcodegen::GpCodegenUtils* codegen_utils) final;
private:
AggState *aggstate_;
static constexpr char kAdvanceAggregatesPrefix[] = "AdvanceAggregates";
/**
* @brief Generates runtime code that implements advance_aggregates.
*
* @param codegen_utils Utility to ease the code generation process.
* @return true on successful generation.
**/
bool GenerateAdvanceAggregates(gpcodegen::GpCodegenUtils* codegen_utils);
/**
* @brief Generates runtime code that implements advance_transition_function.
* It is called by GenerateAdvanceAggregates to generate code for a given
* aggregate function.
*
* @param codegen_utils Utility to ease the code generation process.
* @param llvm_pergroup_arg LLVM pointer to pergroup argument of regular
* advance_aggregates
* @param aggno ith aggregate function
* @param advance_aggregates_func LLVM function pointer to the code generated
* advance_aggregate function
* @param error_block LLVM block for treating errors
* @param llvm_arg argument of aggregate function
*
* @return true on successful generation; false otherwise.
*/
bool GenerateAdvanceTransitionFunction(
gpcodegen::GpCodegenUtils* codegen_utils,
llvm::Value* llvm_pergroup_arg,
int aggno,
gpcodegen::PGFuncGeneratorInfo* pg_func_info,
llvm::Value* llvm_mem_manager_arg);
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_ADVANCEAGGREGATES_CODEGEN_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// base_codegen.h
//
// @doc:
// Base class for all the code generators with common implementation
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_BASE_CODEGEN_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_BASE_CODEGEN_H_
extern "C" {
#include <utils/elog.h>
}
#include <string>
#include <vector>
#include "codegen/utils/gp_codegen_utils.h"
#include "codegen/codegen_config.h"
#include "codegen/codegen_interface.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/Verifier.h"
#include "llvm/Support/raw_ostream.h"
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
// Forward declaration
class CodegenManager;
/**
* @brief Base code generator with common implementation that other
* code generators can use.
*
* @tparam FuncPtrType Function type for regular version of target functions
* or generated functions.
**/
template<class FuncPtrType>
class BaseCodegen: public CodegenInterface {
public:
/**
* @brief Destroys the code generator and reverts back to using regular
* version of the target function.
**/
virtual ~BaseCodegen() {
SetToRegular(regular_func_ptr_, ptr_to_chosen_func_ptr_);
}
bool InitDependencies() override {
return true;
}
bool GenerateCode(gpcodegen::GpCodegenUtils* codegen_utils) final {
bool valid_generated_functions = true;
valid_generated_functions &= GenerateCodeInternal(codegen_utils);
// Do this check only if it enabled by guc
if (codegen_validate_functions && valid_generated_functions) {
for (llvm::Function* function : uncompiled_generated_functions_) {
assert(nullptr != function);
std::string func_name = function->getName();
std::string error_message = "";
llvm::raw_string_ostream out(error_message);
// Verify function returns true if there are errors.
valid_generated_functions &= !llvm::verifyFunction(*function, &out);
out.flush();
if (!valid_generated_functions) {
elog(WARNING, "Broken function found %s: %s",
func_name.c_str(), error_message.c_str());
break;
}
}
}
if (!valid_generated_functions) {
// If failed to generate, or have invalid functions, make sure we
// do clean up by erasing all the llvm functions.
for (llvm::Function* function : uncompiled_generated_functions_) {
assert(nullptr != function);
function->eraseFromParent();
}
}
is_generated_ = valid_generated_functions;
// We don't need to keep these pointers any more
std::vector<llvm::Function*>().swap(uncompiled_generated_functions_);
return is_generated_;
}
bool SetToRegular() final {
assert(nullptr != regular_func_ptr_);
SetToRegular(regular_func_ptr_, ptr_to_chosen_func_ptr_);
return true;
}
bool SetToGenerated(gpcodegen::GpCodegenUtils* codegen_utils) final {
if (false == IsGenerated()) {
assert(*ptr_to_chosen_func_ptr_ == regular_func_ptr_);
return false;
}
FuncPtrType compiled_func_ptr = codegen_utils->GetFunctionPointer<
FuncPtrType>(GetUniqueFuncName());
if (nullptr != compiled_func_ptr) {
*ptr_to_chosen_func_ptr_ = compiled_func_ptr;
return true;
}
return false;
}
void Reset() final {
SetToRegular();
}
const std::string& GetOrigFuncName() const final {
return orig_func_name_;
}
const std::string& GetUniqueFuncName() const final {
return unique_func_name_;
}
bool IsGenerated() const final {
return is_generated_;
}
/**
* @return Regular version of the target function.
*
**/
FuncPtrType GetRegularFuncPointer() {
return regular_func_ptr_;
}
/**
* @brief Sets up the caller to use the corresponding regular version of the
* target function.
*
* @param regular_func_ptr Regular version of the target function.
* @param ptr_to_chosen_func_ptr Reference to caller.
*
* @return true on setting to regular version.
**/
static bool SetToRegular(FuncPtrType regular_func_ptr,
FuncPtrType* ptr_to_chosen_func_ptr) {
assert(nullptr != ptr_to_chosen_func_ptr);
assert(nullptr != regular_func_ptr);
*ptr_to_chosen_func_ptr = regular_func_ptr;
return true;
}
protected:
/**
* @brief Constructor
*
* @param manager The manager in which this is enrolled.
* @param orig_func_name Original function name.
* @param regular_func_ptr Regular version of the target function.
* @param ptr_to_chosen_func_ptr Reference to the function pointer that the caller will call.
*
* @note The ptr_to_chosen_func_ptr can refer to either the generated function or the
* corresponding regular version.
*
**/
explicit BaseCodegen(gpcodegen::CodegenManager* manager,
const std::string& orig_func_name,
FuncPtrType regular_func_ptr,
FuncPtrType* ptr_to_chosen_func_ptr)
: manager_(manager),
orig_func_name_(orig_func_name),
unique_func_name_(CodegenInterface::GenerateUniqueName(orig_func_name)),
regular_func_ptr_(regular_func_ptr),
ptr_to_chosen_func_ptr_(ptr_to_chosen_func_ptr),
is_generated_(false) {
// Initialize the caller to use regular version of target function.
SetToRegular(regular_func_ptr, ptr_to_chosen_func_ptr);
}
gpcodegen::CodegenManager* manager() const {
return manager_;
}
/**
* @brief Generates specialized code at run time.
*
* @note A template method design pattern to be overridden by the sub-class to implement
* the actual code generation.
*
* @note This is being called from GenerateCode and derived class will implement actual
* code generation
*
* @param codegen_utils Utility to ease the code generation process.
* @return true on successful generation.
**/
virtual bool GenerateCodeInternal(
gpcodegen::GpCodegenUtils* codegen_utils) = 0;
/**
* @brief Create llvm Function for given type and store the function pointer
* in vector
*
* @note If generation fails, this class takes responsibility to clean up all
* the functions it created
*
* @tparam FunctionType Type of the function to create
*
* @param codegen_utils Utility to ease the code generation process.
* @param function_name Name of the function to create
* @return llvm::Function pointer
**/
public:
template <typename FunctionType>
llvm::Function* CreateFunction(gpcodegen::GpCodegenUtils* codegen_utils,
const std::string& function_name) {
assert(nullptr != codegen_utils);
llvm::Function* function = codegen_utils->CreateFunction<FunctionType>(
function_name);
assert(nullptr != function);
uncompiled_generated_functions_.push_back(function);
return function;
}
private:
gpcodegen::CodegenManager* manager_;
std::string orig_func_name_;
std::string unique_func_name_;
FuncPtrType regular_func_ptr_;
FuncPtrType* ptr_to_chosen_func_ptr_;
bool is_generated_;
// To track uncompiled llvm functions it creates and erase from
// llvm module on failed generations.
std::vector<llvm::Function*> uncompiled_generated_functions_;
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_BASE_CODEGEN_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// codegen_config.h
//
// @doc:
// Configure Class to handle gucs in gpdb
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_CODEGEN_CONFIG_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_CODEGEN_CONFIG_H_
extern "C" {
// Variables that are defined in guc
extern bool init_codegen;
extern bool codegen;
extern bool codegen_validate_functions;
extern bool codegen_exec_variable_list;
extern bool codegen_slot_getattr;
extern bool codegen_exec_eval_expr;
extern bool codegen_advance_aggregate;
// TODO(shardikar): Retire this GUC after performing experiments to find the
// tradeoff of codegen-ing slot_getattr() (potentially by measuring the
// difference in the number of instructions) when one of the first few
// attributes is varlen.
extern int codegen_varlen_tolerance;
}
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
// Forward declaration
class ExecVariableListCodegen;
class SlotGetAttrCodegen;
class ExecEvalExprCodegen;
class AdvanceAggregatesCodegen;
class CodegenConfig {
public:
/**
* @brief Template function to check if generator enabled.
*
* @tparam ClassType Type of Code Generator class
*
* @return true if respective generator is enabled with gpdb guc.
**/
template <class ClassType>
inline static bool IsGeneratorEnabled();
};
template<>
inline bool CodegenConfig::IsGeneratorEnabled<ExecVariableListCodegen>() {
return codegen_exec_variable_list;
}
template<>
inline bool CodegenConfig::IsGeneratorEnabled<SlotGetAttrCodegen>() {
return codegen_slot_getattr;
}
template<>
inline bool CodegenConfig::IsGeneratorEnabled<ExecEvalExprCodegen>() {
return codegen_exec_eval_expr;
}
template<>
inline bool CodegenConfig::IsGeneratorEnabled<AdvanceAggregatesCodegen>() {
return codegen_advance_aggregate;
}
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_CODEGEN_CONFIG_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// codegen_interface.h
//
// @doc:
// Interface for all code generator
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_CODEGEN_INTERFACE_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_CODEGEN_INTERFACE_H_
#include <string>
#include <vector>
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
// Forward declaration
class GpCodegenUtils;
/**
* @brief Interface for all code generators.
**/
class CodegenInterface {
public:
virtual ~CodegenInterface() = default;
/**
* @brief Hook to request for and initialize any dependencies
*
* @return true on success
*/
virtual bool InitDependencies() = 0;
/**
* @brief Generates specialized code at run time.
*
*
* @param codegen_utils Utility to ease the code generation process.
* @return true on successful generation.
**/
virtual bool GenerateCode(gpcodegen::GpCodegenUtils* codegen_utils) = 0;
/**
* @brief Sets up the caller to use the corresponding regular version of the
* target function.
*
*
* @return true on setting to regular version.
**/
virtual bool SetToRegular() = 0;
/**
* @brief Sets up the caller to use the generated function instead of the
* regular version.
*
* @param codegen_utils Facilitates in obtaining the function pointer from
* the compiled module.
* @return true on successfully setting to generated functions
**/
virtual bool SetToGenerated(gpcodegen::GpCodegenUtils* codegen_utils) = 0;
/**
* @brief Resets the state of the generator, including reverting back to
* the regular version of the function.
*
**/
virtual void Reset() = 0;
/**
*
* @return Original function name.
*
**/
virtual const std::string& GetOrigFuncName() const = 0;
/**
* @return Unique function name of the generated function to avoid name collision.
*
**/
virtual const std::string& GetUniqueFuncName() const = 0;
/**
* @return true if the code generation is successful.
*
**/
virtual bool IsGenerated() const = 0;
protected:
/**
* @brief Utility function to construct a unique function name from the
* original function name by appending a numeric suffix.
*
* @param orig_func_name Function name that needs to be made unique.
* @return Unique string for given input string.
*
**/
static std::string GenerateUniqueName(const std::string& orig_func_name);
private:
// Unique counter for all instances of Codegen Interface.
static unsigned unique_counter_;
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_CODEGEN_INTERFACE_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// codegen_manager.h
//
// @doc:
// Object that manage all CodegenInterface and GpCodegenUtils
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_CODEGEN_MANAGER_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_CODEGEN_MANAGER_H_
#include <memory>
#include <vector>
#include <string>
#include "codegen/utils/macros.h"
#include "codegen/codegen_config.h"
#include "codegen/codegen_interface.h"
#include "codegen/base_codegen.h"
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
// Forward declaration of GpCodegenUtils to manage llvm module
class GpCodegenUtils;
// Forward declaration of a CodegenInterface that will be managed by manager
class CodegenInterface;
/**
* @brief Object that manages all code gen.
**/
class CodegenManager {
public:
/**
* @brief Constructor.
*
* @param module_name A human-readable name for the module that this
* CodegenManager will manage.
**/
explicit CodegenManager(const std::string& module_name);
~CodegenManager() = default;
/**
* @brief Template function to facilitate enroll for any type of
* CodegenInterface that CodegenManager wants to keep track of.
*
* @tparam ClassType Type of Code Generator class that derives from
* CodegenInterface.
* @tparam FuncType Type of the function pointer that CodegenManager swaps.
* @tparam Args Variable argument that ClassType will take in its constructor
*
* @param manager Current Codegen Manager
* @param regular_func_ptr Regular version of the target function.
* @param ptr_to_chosen_func_ptr Pointer to the function pointer that the
* caller will call.
* @param args Variable length argument for ClassType
*
* This function creates a new code generator object of type ClassType using
* the passed-in args, and enrolls it in the given codegen manager.
*
* It does not create a generator when codegen or manager is unset, or the
* code generator ClassType is disabled (with the appropriate GUC).
* It always initializes the given double function pointer
* (ptr_to_chosen_func_ptr) to the regular_func_ptr.
*
* This transfers the ownership of the code generator to the manager.
*
* @return Pointer to ClassType
**/
template <typename ClassType, typename FuncType, typename ...Args>
static ClassType* CreateAndEnrollGenerator(
CodegenManager* manager,
FuncType regular_func_ptr,
FuncType* ptr_to_chosen_func_ptr,
Args&&... args) { // NOLINT(build/c++11)
assert(nullptr != regular_func_ptr);
assert(nullptr != ptr_to_chosen_func_ptr);
bool can_enroll =
// manager may be NULL if ExecInitNode/ExecProcNode weren't previously
// called. This happens e.g during gpinitsystem.
(nullptr != manager) &&
codegen && // if codegen guc is false
// if generator is disabled
CodegenConfig::IsGeneratorEnabled<ClassType>();
if (!can_enroll) {
gpcodegen::BaseCodegen<FuncType>::SetToRegular(
regular_func_ptr, ptr_to_chosen_func_ptr);
return nullptr;
}
ClassType* generator = new ClassType(
manager,
regular_func_ptr,
ptr_to_chosen_func_ptr,
std::forward<Args>(args)...);
bool is_enrolled = manager->EnrollCodeGenerator(
CodegenFuncLifespan_Parameter_Invariant, generator);
assert(is_enrolled);
return generator;
}
/**
* @brief Enroll a code generator with manager
*
* @note Manager manages the memory of enrolled generator.
*
* @param funcLifespan Life span of the enrolling generator. Based on life span,
* corresponding GpCodegenUtils will be used for code generation
* @param generator Generator that needs to be enrolled with manager.
* @return true on successful enrollment.
**/
bool EnrollCodeGenerator(CodegenFuncLifespan funcLifespan,
CodegenInterface* generator);
/**
* @brief Request all enrolled generators to generate code.
*
* @return The number of enrolled codegen that successfully generated code.
**/
unsigned int GenerateCode();
/**
* @brief Compile all the generated functions. On success,
* a pointer to the generated method becomes available to the caller.
*
* @return The number of enrolled codegen that successully generated code
* and 0 on failure
**/
unsigned int PrepareGeneratedFunctions();
/**
* @brief Notifies the manager of a parameter change.
*
* @note This is called during a ReScan or other parameter change process.
* Upon receiving this notification the manager may invalidate all the
* generated code that depend on parameters.
*
**/
void NotifyParameterChange();
/**
* @brief Invalidate all generated functions.
*
* @return true if successfully invalidated.
**/
bool InvalidateGeneratedFunctions();
/**
* @return Number of enrolled generators.
**/
size_t GetEnrollmentCount() {
return enrolled_code_generators_.size();
}
/*
* @brief Accumulate the explain string with a dump of all the underlying LLVM
* modules
*/
void AccumulateExplainString();
/*
* @brief Return the previous accumulated explain string
*/
const std::string& GetExplainString();
private:
// GpCodegenUtils provides a facade to LLVM subsystem.
std::unique_ptr<gpcodegen::GpCodegenUtils> codegen_utils_;
std::string module_name_;
// List of all enrolled code generators.
std::vector<std::unique_ptr<CodegenInterface>> enrolled_code_generators_;
// Holds the dumped IR of all underlying modules for EXPLAIN CODEGEN queries
std::string explain_string_;
DISALLOW_COPY_AND_ASSIGN(CodegenManager);
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_CODEGEN_MANAGER_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// const_expr_tree_generator.h
//
// @doc:
// Object that generate code for const expression.
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_CONST_EXPR_TREE_GENERATOR_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_CONST_EXPR_TREE_GENERATOR_H_
#include "codegen/expr_tree_generator.h"
#include "llvm/IR/Value.h"
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
/**
* @brief Object that generate code for const expression.
**/
class ConstExprTreeGenerator : public ExprTreeGenerator {
public:
static bool VerifyAndCreateExprTree(
const ExprState* expr_state,
ExprTreeGeneratorInfo* gen_info,
std::unique_ptr<ExprTreeGenerator>* expr_tree);
bool GenerateCode(gpcodegen::GpCodegenUtils* codegen_utils,
const ExprTreeGeneratorInfo& gen_info,
llvm::Value** llvm_out_value,
llvm::Value* const llvm_isnull_ptr) final;
protected:
/**
* @brief Constructor.
*
* @param expr_state Expression state from epxression tree
**/
explicit ConstExprTreeGenerator(const ExprState* expr_state);
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_CONST_EXPR_TREE_GENERATOR_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// exec_eval_expr_codegen.h
//
// @doc:
// Headers for ExecEvalExpr codegen.
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_EXECEVALEXPR_CODEGEN_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_EXECEVALEXPR_CODEGEN_H_
#include "codegen/base_codegen.h"
#include "codegen/codegen_wrapper.h"
#include "codegen/expr_tree_generator.h"
#include "codegen/slot_getattr_codegen.h"
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
class ExecEvalExprCodegen: public BaseCodegen<ExecEvalExprFn> {
public:
/**
* @brief Constructor
*
* @param regular_func_ptr Regular version of the target function.
* @param ptr_to_chosen_func_ptr Reference to the function pointer that the
* caller will call.
* @param exprstate The ExprState to use for generating code.
* @param econtext The ExprContext to use for generating code.
* @param slot The slot to use for generating code.
*
* @note The ptr_to_chosen_func_ptr can refer to either the generated
* function or the corresponding regular version.
*
**/
explicit ExecEvalExprCodegen(CodegenManager* manager,
ExecEvalExprFn regular_func_ptr,
ExecEvalExprFn* ptr_to_regular_func_ptr,
ExprState *exprstate,
ExprContext *econtext,
PlanState* plan_state);
virtual ~ExecEvalExprCodegen() = default;
bool InitDependencies() override;
protected:
/**
* @brief Generate code for expression evaluation.
*
* @param codegen_utils
*
* @return true on successful generation; false otherwise.
*
* @note Walks down expression tree and create respective ExprTreeGenerator
* to generate code.
*
* This implementation does not support:
* (1) Null attributes
* (2) Variable length attributes
*
* If at execution time, we see any of the above types of attributes,
* we fall backs to the regular function.
*
*/
bool GenerateCodeInternal(gpcodegen::GpCodegenUtils* codegen_utils) final;
private:
ExprState *exprstate_;
PlanState* plan_state_;
ExprTreeGeneratorInfo gen_info_;
SlotGetAttrCodegen* slot_getattr_codegen_;
std::unique_ptr<ExprTreeGenerator> expr_tree_generator_;
static constexpr char kExecEvalExprPrefix[] = "ExecEvalExpr";
/**
* @brief Generates runtime code that implements expression evaluation.
*
* @param codegen_utils Utility to ease the code generation process.
* @return true on successful generation.
**/
bool GenerateExecEvalExpr(gpcodegen::GpCodegenUtils* codegen_utils);
/**
* @brief Prepare generation of dependent slot_getattr() if necessary
* @return true on successful generation.
**/
void PrepareSlotGetAttr();
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_EXECEVALEXPR_CODEGEN_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// exec_variable_list.h
//
// @doc:
// Headers for slot_deform_tuple codegen
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_EXECVARIABLELIST_CODEGEN_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_EXECVARIABLELIST_CODEGEN_H_
#include "codegen/codegen_wrapper.h"
#include "codegen/slot_getattr_codegen.h"
#include "codegen/base_codegen.h"
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
class ExecVariableListCodegen: public BaseCodegen<ExecVariableListFn> {
public:
/**
* @brief Constructor
*
* @param regular_func_ptr Regular version of the target function.
* @param ptr_to_chosen_func_ptr Reference to the function pointer that the caller will call.
* @param slot The slot to use for generating code.
*
* @note The ptr_to_chosen_func_ptr can refer to either the generated function or the
* corresponding regular version.
*
**/
explicit ExecVariableListCodegen(CodegenManager* manager,
ExecVariableListFn regular_func_ptr,
ExecVariableListFn* ptr_to_regular_func_ptr,
ProjectionInfo* proj_info,
TupleTableSlot* slot);
virtual ~ExecVariableListCodegen() = default;
bool InitDependencies() override;
protected:
/**
* @brief Generate code for the code path ExecVariableList > slot_getattr >
* _slot_getsomeattrs > slot_deform_tuple.
*
* @param codegen_utils
*
* @return true on successful generation; false otherwise.
*
* @note Generate code for code path ExecVariableList > slot_getattr >
* _slot_getsomeattrs > slot_deform_tuple. The regular implementation of
* ExecvariableList, for each attribute <i>A</i> in the target list, retrieves
* the slot that <i>A</> comes from and calls slot_getattr. slot_get_attr() will
* eventually call slot_deform_tuple (through _slot_getsomeattrs), which fetches
* all yet unread attributes of the slot until the given attribute.
*
* This function generates the code for the case that all the
* attributes in target list use the same slot (created during a
* scan i.e, ecxt_scantuple). Moreover, instead of looping over the target
* list one at a time, this approach uses slot_getattr only once, with the
* largest attribute index from the target list.
*
* If code generation is not possible for any reason (e.g., attributes
* use different slots), then the function returns false and the codegen manager
* will manage the clean up.
*
* This implementation does not support:
* (1) Null attributes
* (2) Variable length attributes
* (3) Attributes passed by reference
*
* If at execution time, we see any of the above types of attributes,
* we fall backs to the regular function.
*/
bool GenerateCodeInternal(gpcodegen::GpCodegenUtils* codegen_utils) final;
private:
ProjectionInfo* proj_info_;
TupleTableSlot* slot_;
int max_attr_;
SlotGetAttrCodegen* slot_getattr_codegen_;
static constexpr char kExecVariableListPrefix[] = "ExecVariableList";
/**
* @brief Generates runtime code that implements ExecVariableList.
*
* @param codegen_utils Utility to ease the code generation process.
* @return true on successful generation.
**/
bool GenerateExecVariableList(gpcodegen::GpCodegenUtils* codegen_utils);
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_EXECVARIABLELIST_CODEGEN_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// expr_tree_generator.h
//
// @doc:
// Base class for expression tree to generate code
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_EXPR_TREE_GENERATOR_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_EXPR_TREE_GENERATOR_H_
#include <string>
#include <vector>
#include <memory>
#include <utility>
#include <unordered_map>
#include "codegen/utils/gp_codegen_utils.h"
#include "codegen/codegen_wrapper.h"
#include "llvm/IR/Value.h"
typedef struct ExprContext ExprContext;
typedef struct ExprState ExprState;
typedef struct Expr Expr;
typedef struct OpExpr OpExpr;
typedef struct Var Var;
typedef struct Const Const;
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
enum class ExprTreeNodeType {
kConst = 0,
kVar = 1,
kOperator = 2
};
/**
* @brief Object that holds the information needed for creating the ExprTree
* for the code generation of expressions in ExecQual, ExecEvalExpr etc.
*
* @note Members of this structures may be initialized at different stages (see
* ExecEvalExprCodegen::GenerateExecEvalExpr). Some are available immediately at
* the time of code generation, others need to be calculated by traversing the
* expression tree by the ExprTreeGenerator::VerifyAndCreateExprTree pass.
* However, all members must be initialized before the
* ExecEvalExprCodegen::GenerateCode pass when they are all required.
*
**/
struct ExprTreeGeneratorInfo {
// Members that are immediately available at operator init.
ExprContext* econtext;
// Convenience members for code generation
llvm::Function* llvm_main_func;
llvm::BasicBlock* llvm_error_block;
// Members that will be updated by the
// ExprTreeGenerator::VerifyAndCreateExprTree pass
// Keep track of the max_attr needed for the expression. Needed by
// the code generation logic for slot_getattr()
int16_t max_attr;
// Records the generated slot_getattr() if generation was successful; a
// pointer to the external function otherwise.
llvm::Function* llvm_slot_getattr_func;
ExprTreeGeneratorInfo(
ExprContext* econtext,
llvm::Function* llvm_main_func,
llvm::BasicBlock* llvm_error_block,
llvm::Function* llvm_slot_getattr_func,
int16_t max_attr) :
econtext(econtext),
llvm_main_func(llvm_main_func),
llvm_error_block(llvm_error_block),
max_attr(max_attr),
llvm_slot_getattr_func(llvm_slot_getattr_func) {
}
};
class ExprTreeGenerator {
public:
virtual ~ExprTreeGenerator() = default;
/**
* @brief Verify if we support the given expression tree, and create an
* instance of ExprTreeGenerator if supported.
*
* @param expr_state Expression state from expression tree.
* @param gen_info Information needed for generating the expression tree.
* @param expr_tree Hold the new instance of expression tree class.
*
* @return true when it can codegen otherwise it return false.
**/
static bool VerifyAndCreateExprTree(
const ExprState* expr_state,
ExprTreeGeneratorInfo* gen_info,
std::unique_ptr<ExprTreeGenerator>* expr_tree);
/**
* @brief Generate the code for given expression.
*
* @param codegen_utils Utility to easy code generation.
* @param gen_info Information needed for generating the expression
* tree.
* @param llvm_out_value Store the expression results
* @param llvm_isnull_ptr Set to true if current expr is null
*
* @return true when it generated successfully otherwise it return false.
**/
virtual bool GenerateCode(gpcodegen::GpCodegenUtils* codegen_utils,
const ExprTreeGeneratorInfo& gen_info,
llvm::Value** value,
llvm::Value* const llvm_isnull_ptr) = 0;
/**
* @return Expression state
**/
const ExprState* expr_state() { return expr_state_; }
protected:
/**
* @brief Constructor.
*
* @param expr_state Expression state
* @param node_type Type of the ExprTreeGenerator
**/
ExprTreeGenerator(const ExprState* expr_state,
ExprTreeNodeType node_type) :
expr_state_(expr_state) {}
private:
const ExprState* expr_state_;
DISALLOW_COPY_AND_ASSIGN(ExprTreeGenerator);
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_EXPR_TREE_GENERATOR_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// op_expr_tree_generator.h
//
// @doc:
// Object that generate code for operator expression.
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_OP_EXPR_TREE_GENERATOR_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_OP_EXPR_TREE_GENERATOR_H_
#include <vector>
#include "codegen/expr_tree_generator.h"
#include "codegen/pg_func_generator_interface.h"
#include "codegen/pg_func_generator.h"
#include "llvm/IR/Value.h"
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
using CodeGenFuncMap = std::unordered_map<unsigned int,
std::unique_ptr<gpcodegen::PGFuncGeneratorInterface>>;
/**
* @brief Object that generate code for operator expression.
**/
class OpExprTreeGenerator : public ExprTreeGenerator {
public:
/**
* @brief Initialize PG operator function that we support for code generation.
**/
static void InitializeSupportedFunction();
static bool VerifyAndCreateExprTree(
const ExprState* expr_state,
ExprTreeGeneratorInfo* gen_info,
std::unique_ptr<ExprTreeGenerator>* expr_tree);
/**
* @brief Checks if a built-in function can be code generated.
*
* @param oid The oid of the function.
* @return If function is supported, then returns a pointer to the
* PGFuncGeneratorInterface of the function; nullptr otherwise.
**/
static gpcodegen::PGFuncGeneratorInterface* GetPGFuncGenerator(
unsigned int oid);
bool GenerateCode(gpcodegen::GpCodegenUtils* codegen_utils,
const ExprTreeGeneratorInfo& gen_info,
llvm::Value** llvm_out_value,
llvm::Value* const llvm_isnull_ptr) final;
protected:
/**
* @brief Constructor.
*
* @param expr_state Expression state
* @param arguments Arguments to operator as list of ExprTreeGenerator
**/
OpExprTreeGenerator(
const ExprState* expr_state,
std::vector<
std::unique_ptr<
ExprTreeGenerator>>&& arguments); // NOLINT(build/c++11)
private:
std::vector<std::unique_ptr<ExprTreeGenerator>> arguments_;
// Map of supported function with respective generator to generate code
static CodeGenFuncMap supported_function_;
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_OP_EXPR_TREE_GENERATOR_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// pg_arith_func_generator.h
//
// @doc:
// Class with Static member function to generate code for +, - and * operator
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_PG_ARITH_FUNC_GENERATOR_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_PG_ARITH_FUNC_GENERATOR_H_
#include <string>
#include <vector>
#include <memory>
#include "codegen/utils/gp_codegen_utils.h"
#include "codegen/pg_func_generator_interface.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Value.h"
extern "C" {
#include "utils/elog.h"
}
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
namespace gpcodegen_ArithOp_detail {
// ArithOpOverFlowErrorMsg has various template specializations to
// handle error message for different C++ types. The specialized versions
// have a static method OverFlowErrMsg() that returns an overflow error message
// based on CppType as const char*
template <typename CppType, typename Enable = void>
class ArithOpOverFlowErrorMsg {};
template <typename IntType>
class ArithOpOverFlowErrorMsg<
IntType,
typename std::enable_if<std::is_integral<IntType>::value>::type> {
public:
static const char* OverFlowErrMsg() { return "integer out of range"; }
};
template <>
class ArithOpOverFlowErrorMsg<
float> {
public:
static const char* OverFlowErrMsg() { return "float4 out of range"; }
};
template <>
class ArithOpOverFlowErrorMsg<
double> {
public:
static const char* OverFlowErrMsg() { return "value out of range: overflow"; }
};
} // namespace gpcodegen_ArithOp_detail
using gpcodegen_ArithOp_detail::ArithOpOverFlowErrorMsg;
/**
* @brief Class with Static member function to generate code for +, - and *
* operator
*
* @tparam rtype Return type of function
* @tparam Arg0 First argument's type
* @tparam Arg1 Second argument's type
**/
template <typename rtype, typename Arg0, typename Arg1>
class PGArithFuncGenerator {
template <typename CppType>
using CGArithOpTemplateFunc = llvm::Value* (GpCodegenUtils::*)(
llvm::Value* arg0, llvm::Value* arg1);
using CGArithOpFunc = CGArithOpTemplateFunc<rtype>;
public:
/**
* @brief Create LLVM Mul instruction with check for overflow
*
* @param codegen_utils Utility to easy code generation.
* @param llvm_main_func Current function for which we are generating code
* @param llvm_error_block Basic Block to jump when error happens
* @param llvm_args Vector of llvm arguments for the function
* @param llvm_out_value Store the results of function
*
* @return true if generation was successful otherwise return false
*
* @note If there is overflow, it will do elog::ERROR and then jump to given
* error block.
**/
static bool MulWithOverflow(gpcodegen::GpCodegenUtils* codegen_utils,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value) {
return ArithOpWithOverflow(
codegen_utils,
&gpcodegen::GpCodegenUtils::CreateMulOverflow<rtype>,
ArithOpOverFlowErrorMsg<rtype>::OverFlowErrMsg(),
pg_func_info,
llvm_out_value);
}
/**
* @brief Create LLVM Add instruction with check for overflow
*
* @param codegen_utils Utility to easy code generation.
* @param llvm_main_func Current function for which we are generating code
* @param llvm_error_block Basic Block to jump when error happens
* @param llvm_args Vector of llvm arguments for the function
* @param llvm_out_value Store the results of function
*
* @return true if generation was successful otherwise return false
*
* @note If there is overflow, it will do elog::ERROR and then jump to given
* error block.
**/
static bool AddWithOverflow(gpcodegen::GpCodegenUtils* codegen_utils,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value) {
return ArithOpWithOverflow(
codegen_utils,
&gpcodegen::GpCodegenUtils::CreateAddOverflow<rtype>,
ArithOpOverFlowErrorMsg<rtype>::OverFlowErrMsg(),
pg_func_info,
llvm_out_value);
}
/**
* @brief Create LLVM instructions to check if arguments are NULL.
*
* @param codegen_utils Utility for easy code generation.
* @param pg_func_info Details for pgfunc generation
* @param llvm_isnull_ptr Records if result is NULL
* @param llvm_out_value_ptr Store location for the result
* @param llvm_is_set_ptr Pointer to flag that shows if a value has been
* assigned to the contents of llvm_out_value_ptr
*
* @return true if generation was successful otherwise return false
*
* @note It is used only if built-in function is not strict.
* This function implements the first part of int4_sum built-in
* function that checks for NULL arguments.
**/
static bool CreateArgumentNullChecks(gpcodegen::GpCodegenUtils* codegen_utils,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value* llvm_out_value_ptr,
llvm::Value* llvm_is_set_ptr,
llvm::Value* const llvm_isnull_ptr) {
assert(nullptr != codegen_utils);
assert(nullptr != llvm_out_value_ptr);
assert(nullptr != llvm_is_set_ptr);
assert(pg_func_info.llvm_args.size() ==
pg_func_info.llvm_args_isNull.size());
auto irb = codegen_utils->ir_builder();
// Entry block that shows that clearly shows the beginning of CheckNull
llvm::BasicBlock* entry_block = codegen_utils->CreateBasicBlock(
"CheckNull_entry_block", pg_func_info.llvm_main_func);
// Block that includes instructions in case that arg0 is null
llvm::BasicBlock* arg0_is_null_block = codegen_utils->
CreateBasicBlock("CheckNull_arg0_is_null_block",
pg_func_info.llvm_main_func);
// Block that includes instructions in case that arg0 is not null
llvm::BasicBlock* arg0_is_not_null_block = codegen_utils->
CreateBasicBlock("CheckNull_arg0_is_not_null_block",
pg_func_info.llvm_main_func);
// Block that includes instructions in case that arg1 is null
llvm::BasicBlock* arg1_is_null_block = codegen_utils->
CreateBasicBlock("CheckNull_arg1_is_null_block",
pg_func_info.llvm_main_func);
// Block that includes instructions in case that arg1 is not null
llvm::BasicBlock* arg1_is_not_null_block = codegen_utils->
CreateBasicBlock("CheckNull_arg1_is_not_null_block",
pg_func_info.llvm_main_func);
// Block that includes instructions in case that there is a null argument
llvm::BasicBlock* return_null_block = codegen_utils->CreateBasicBlock(
"CheckNull_return_null_block", pg_func_info.llvm_main_func);
// All CheckNull blocks create branch to continue_block (final block)
llvm::BasicBlock* continue_block = codegen_utils->CreateBasicBlock(
"CheckNull_continue_block", pg_func_info.llvm_main_func);
irb->CreateBr(entry_block);
// entry_block
// -----------
// Generate code that examines if arg0 is null
irb->SetInsertPoint(entry_block);
// if (PG_ARGISNULL(0))
irb->CreateCondBr(pg_func_info.llvm_args_isNull[0],
arg0_is_null_block /* true */,
arg0_is_not_null_block /* false */);
// arg0_is_null_block
// ------------------
// arg0 is NULL, so examine if arg1 is NULL too.
irb->SetInsertPoint(arg0_is_null_block);
// if (PG_ARGISNULL(1))
irb->CreateCondBr(pg_func_info.llvm_args_isNull[1],
return_null_block /* true */,
arg1_is_not_null_block /* false */);
// return_null_block
// -----------------
// Generate code for PG_RETURN_NULL()
irb->SetInsertPoint(return_null_block);
// fcinfo->isnull = true;
irb->CreateStore(codegen_utils->GetConstant<bool>(true),
llvm_isnull_ptr);
// similar to: return (Datum) 0
irb->CreateStore(codegen_utils->GetConstant<rtype>(0),
llvm_out_value_ptr);
// set the content of llvm_is_set_ptr to true, so that we do not need to
// execute the code of non-strict built-in function
irb->CreateStore(codegen_utils->GetConstant<bool>(true),
llvm_is_set_ptr);
irb->CreateBr(continue_block);
// arg1_is_not_null_block
// ----------------------
// In this case, arg0 is NULL and arg1 is not NULL.
irb->SetInsertPoint(arg1_is_not_null_block);
// val = (int64) PG_GETARG_INT32(1)
// PG_RETURN_INT64(val)
irb->CreateStore(codegen_utils->CreateCast<rtype, Arg1>(
pg_func_info.llvm_args[1]), llvm_out_value_ptr);
// set the content of llvm_is_set_ptr to true, so that we do not need to
// execute the code of non-strict built-in function
irb->CreateStore(codegen_utils->GetConstant<bool>(true),
llvm_is_set_ptr);
irb->CreateStore(codegen_utils->GetConstant<bool>(false),
llvm_isnull_ptr);
irb->CreateBr(continue_block);
// arg0_is_not_null_block
// ----------------------
// arg0 is not null, examine if arg1 is null
// If both arguments are not NULL, then just continue and execute the
// generated code of the built-in function
irb->SetInsertPoint(arg0_is_not_null_block);
// if (PG_ARGISNULL(1))
irb->CreateCondBr(pg_func_info.llvm_args_isNull[1],
arg1_is_null_block,
continue_block);
// arg1_is_null_block
// ------------------
// arg0 is not NULL, but arg1 is NULL
// Generate code for PG_RETURN_INT64(val), where val = arg0
irb->SetInsertPoint(arg1_is_null_block);
irb->CreateStore(codegen_utils->CreateCast<rtype, Arg0>(
pg_func_info.llvm_args[0]),
llvm_out_value_ptr);
// set the content of llvm_is_set_ptr to true, so that we do not need to
// execute the code of non-strict built-in function
irb->CreateStore(codegen_utils->GetConstant<bool>(true),
llvm_is_set_ptr);
irb->CreateStore(codegen_utils->GetConstant<bool>(false),
llvm_isnull_ptr);
irb->CreateBr(continue_block);
// continue_block
// --------------
// Continue with the rest of the generated code
irb->SetInsertPoint(continue_block);
return true;
}
/**
* @brief Create LLVM Sub instruction with check for overflow
*
* @param codegen_utils Utility to easy code generation.
* @param llvm_main_func Current function for which we are generating code
* @param llvm_error_block Basic Block to jump when error happens
* @param llvm_args Vector of llvm arguments for the function
* @param llvm_out_value Store the results of function
*
* @return true if generation was successful otherwise return false
*
* @note If there is overflow, it will do elog::ERROR and then jump to given
* error block.
**/
static bool SubWithOverflow(gpcodegen::GpCodegenUtils* codegen_utils,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value) {
return ArithOpWithOverflow(
codegen_utils,
&gpcodegen::GpCodegenUtils::CreateSubOverflow<rtype>,
ArithOpOverFlowErrorMsg<rtype>::OverFlowErrMsg(),
pg_func_info,
llvm_out_value);
}
static bool ArithOpWithOverflow(gpcodegen::GpCodegenUtils* codegen_utils,
CGArithOpFunc codegen_mem_funcptr,
const char* error_msg,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value);
};
/**
* @brief Class with Static member function to generate code for unary
* operator such as ++, -- etc.
*
* @tparam rtype Return type of function
* @tparam Arg Argument's type
**/
template <typename rtype, typename Arg>
class PGArithUnaryFuncGenerator {
template <typename CppType>
using CGArithOpTemplateFunc = llvm::Value* (GpCodegenUtils::*)(
llvm::Value* arg);
using CGArithOpFunc = CGArithOpTemplateFunc<rtype>;
public:
/**
* @brief Increase the value of a variable with check for overflow
*
* @param codegen_utils Utility for easy code generation.
* @param pg_func_info Details for pgfunc generation
* @param llvm_out_value Store location for the result
*
* @return true if generation was successful otherwise return false
*
* @note If there is overflow, it will do elog::ERROR and then jump to given
* error block.
**/
static bool IncWithOverflow(gpcodegen::GpCodegenUtils* codegen_utils,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value) {
return PGArithUnaryFuncGenerator<rtype, Arg>::ArithOpWithOverflow(
codegen_utils,
&gpcodegen::GpCodegenUtils::CreateIncOverflow<rtype>,
ArithOpOverFlowErrorMsg<rtype>::OverFlowErrMsg(),
pg_func_info,
llvm_out_value);
}
static bool ArithOpWithOverflow(gpcodegen::GpCodegenUtils* codegen_utils,
CGArithOpFunc codegen_mem_funcptr,
const char* error_msg,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value);
};
/**
* @brief Utility function for generating code for overflow checking. This
* includes an overflow block and non-overflow block, check and conditional
* branch instructions.
*
* @param codegen_utils Utility for easy code generation.
* @param pg_func_info Details for pgfunc generation
* @param llvm_arith_output Result of the arithmetic operation
* @param error_msg Error message to use when overflow occurs
* @param llvm_out_value Store location for the result
*
* @return true if generation was successful otherwise return false
*
* @note Only integral types are currently supported.
**/
template<typename rtype>
static bool CreateOverflowCheckLogic(
gpcodegen::GpCodegenUtils* codegen_utils,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value* llvm_arith_output,
const char* error_msg,
llvm::Value** llvm_out_value) {
assert(nullptr != codegen_utils);
assert(nullptr != llvm_arith_output);
assert(nullptr != error_msg);
llvm::IRBuilder<>* irb = codegen_utils->ir_builder();
llvm::BasicBlock* llvm_non_overflow_block = codegen_utils->CreateBasicBlock(
"arith_non_overflow_block", pg_func_info.llvm_main_func);
llvm::BasicBlock* llvm_overflow_block = codegen_utils->CreateBasicBlock(
"arith_overflow_block", pg_func_info.llvm_main_func);
*llvm_out_value = irb->CreateExtractValue(llvm_arith_output, 0);
llvm::Value* llvm_overflow_flag =
irb->CreateExtractValue(llvm_arith_output, 1);
irb->CreateCondBr(llvm_overflow_flag, llvm_overflow_block,
llvm_non_overflow_block);
irb->SetInsertPoint(llvm_overflow_block);
EXPAND_CREATE_EREPORT(codegen_utils,
ERROR,
ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE,
error_msg);
irb->CreateBr(pg_func_info.llvm_error_block);
irb->SetInsertPoint(llvm_non_overflow_block);
return true;
}
template <typename rtype, typename Arg>
bool PGArithUnaryFuncGenerator<rtype, Arg>::ArithOpWithOverflow(
gpcodegen::GpCodegenUtils* codegen_utils,
CGArithOpFunc codegen_mem_funcptr,
const char* error_msg,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value) {
assert(nullptr != codegen_utils);
assert(nullptr != llvm_out_value);
assert(nullptr != codegen_mem_funcptr);
assert(nullptr != error_msg);
// Assumed caller checked vector size and nullptr for codegen_utils
llvm::Value* casted_arg =
codegen_utils->CreateCast<rtype, Arg>(pg_func_info.llvm_args[0]);
llvm::Value* llvm_arith_output = (codegen_utils->*codegen_mem_funcptr)(
casted_arg);
return CreateOverflowCheckLogic<rtype>(codegen_utils,
pg_func_info,
llvm_arith_output,
error_msg,
llvm_out_value);
}
template <typename rtype, typename Arg0, typename Arg1>
bool PGArithFuncGenerator<rtype, Arg0, Arg1>::ArithOpWithOverflow(
gpcodegen::GpCodegenUtils* codegen_utils,
CGArithOpFunc codegen_mem_funcptr,
const char* error_msg,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value) {
assert(nullptr != codegen_utils);
assert(nullptr != llvm_out_value);
assert(nullptr != codegen_mem_funcptr);
assert(nullptr != error_msg);
// Assumed caller checked vector size and nullptr for codegen_utils
llvm::Value* casted_arg0 =
codegen_utils->CreateCast<rtype, Arg0>(pg_func_info.llvm_args[0]);
llvm::Value* casted_arg1 =
codegen_utils->CreateCast<rtype, Arg1>(pg_func_info.llvm_args[1]);
llvm::Value* llvm_arith_output =
(codegen_utils->*codegen_mem_funcptr)(casted_arg0, casted_arg1);
return CreateOverflowCheckLogic<rtype>(codegen_utils,
pg_func_info,
llvm_arith_output,
error_msg,
llvm_out_value);
}
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_PG_ARITH_FUNC_GENERATOR_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright (C) 2016 Pivotal Software, Inc.
//
// @filename:
// pg_date_func_generator.h
//
// @doc:
// Base class for date functions to generate code
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_PG_DATE_FUNC_GENERATOR_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_PG_DATE_FUNC_GENERATOR_H_
#include <memory>
#include <string>
#include <vector>
#include "codegen/base_codegen.h"
#include "codegen/pg_func_generator_interface.h"
#include "codegen/utils/codegen_utils.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Value.h"
namespace llvm {
class Value;
} // namespace llvm
namespace gpcodegen {
/** \addtogroup gpcodegen
* @{
*/
class GpCodegenUtils;
struct PGFuncGeneratorInfo;
/**
* @brief Class with Static member function to generate code for date
* operators.
**/
class PGDateFuncGenerator {
public:
/**
* @brief Create instructions for date_le_timestamp function
*
* @param codegen_utils Utility to easy code generation.
* @param llvm_main_func Current function for which we are generating code
* @param llvm_error_block Basic Block to jump when error happens
* @param llvm_args Vector of llvm arguments for the function
* @param llvm_out_value Store the results of function
*
* @return true if generation was successful otherwise return false
*
**/
static bool DateLETimestamp(
gpcodegen::GpCodegenUtils* codegen_utils,
const PGFuncGeneratorInfo& pg_func_info,
llvm::Value** llvm_out_value);
private:
/**
* @brief Internal routines for promoting date to timestamp and timestamp
* with time zone (see date2timestamp).
*
* @param codegen_utils Utility to easy code generation.
* @param llvm_main_func Current function for which we are generating code
* @param llvm_arg llvm value for the date
* @param llvm_error_block Basic Block to jump when error happens
*
* @note If there is overflow, it will do elog::ERROR and then jump to given
* error block.
*/
static llvm::Value* GenerateDate2Timestamp(
GpCodegenUtils* codegen_utils,
const PGFuncGeneratorInfo& pg_func_info);
};
/** @} */
} // namespace gpcodegen
#endif // GPCODEGEN_PG_ARITH_FUNC_GENERATOR_H_
//---------------------------------------------------------------------------
// Greenplum Database
// Copyright 2016 Pivotal Software, Inc.
//
// @filename:
// macros.h
//
// @doc:
// Utility macros
//
// @test:
//
//
//---------------------------------------------------------------------------
#ifndef GPCODEGEN_MACROS_H_ // NOLINT(build/header_guard)
#define GPCODEGEN_MACROS_H_
#define DISALLOW_COPY_AND_ASSIGN(classname) \
classname(const classname& orig) = delete; \
classname& operator=(const classname &orig) = delete
#endif // GPCODEGEN_MACROS_H_
// EOF
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
# Copyright 2015-2016 Pivotal Software, Inc.
#
# CMakeLists.txt
# Cmake configuration for building GPDB codegen examples.
#
cmake_minimum_required(VERSION 2.8.12)
# Example 1: Materialize Tuple
add_executable(materialize_tuple_ex materialize_tuple_ex.cc)
target_link_libraries(materialize_tuple_ex gpcodegen)
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册