README.md 5.9 KB
Newer Older
1 2 3 4
[![Build Status](https://travis-ci.org/greenplum-db/gpdb.svg?branch=master)](https://travis-ci.org/greenplum-db/gpdb)

----------------------------------------------------------------------

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
![Greenplum](/gpAux/releng/images/logo-greenplum.png)

The Greenplum Database (GPDB) is an advanced, fully featured, open
source data warehouse. It provides powerful and rapid analytics on
petabyte scale data volumes. Uniquely geared toward big data
analytics, Greenplum Database is powered by the world’s most advanced
cost-based query optimizer delivering high analytical query
performance on large data volumes.

The Greenplum project is released under the [Apache 2
license](http://www.apache.org/licenses/LICENSE-2.0). We want to thank
all our current community contributors and are really interested in
all new potential contributions. For the Greenplum Database community
no contribution is too small, we encourage all types of contributions.

## Overview

A Greenplum cluster consists of a __master__ server, and multiple
__segment__ servers. All user data resides in the segments, the master
contains only metadata. The master server, and all the segments, share
the same schema.

Users always connect to the master server, which divides up the query
into fragments that are executed in the segments, sends the fragments
to the segments, and collects the results.

## Requirements

* From the GPDB doc set, review [Configuring Your Systems and
  Installing
  Greenplum](http://gpdb.docs.pivotal.io/4360/prep_os-overview.html#topic1)
  and perform appropriate updates to your system for GPDB use.

* **gpMgmt** utilities - command line tools for managing the cluster.

  You will need to add the following Python modules (2.7 & 2.6 are
  supported) into your installation

  * psi
44
  * lockfile (>= 0.9.1)
45 46 47 48
  * paramiko
  * setuptools
  * epydoc

49 50 51
  If necessary, upgrade modules using "pip install --upgrade".
  pip should be at least version 7.x.x.

52

53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76
## Code layout

The directory layout of the repository follows the same general layout
as upstream PostgreSQL. There are changes compared to PostgreSQL
throughout the codebase, but a few larger additions worth noting:

* __gpMgmt/__

  Contains Greenplum-specific command-line tools for managing the
  cluster. Scripts like gpinit, gpstart, gpstop live here. They are
  mostly written in Python.

* __gpAux/__

  Contains Greenplum-specific extensions such as gpfdist and
  gpmapreduce.  Some additional directories are submodules and will be
  made available over time.

* __doc/__

  In PostgreSQL, the user manual lives here. In Greenplum, the user
  manual is distributed separately (see http://gpdb.docs.pivotal.io),
  and only the reference pages used to build man pages are here.

77 78 79 80 81
* __ci/__

  Contains configuration files for the GPDB continuous integration system.
 

82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111
* __src/backend/cdb/__

  Contains larger Greenplum-specific backend modules. For example,
  communication between segments, turning plans into parallelizable
  plans, mirroring, distributed transaction and snapshot management,
  etc. __cdb__ stands for __Cluster Database__ - it was a workname used in
  the early days. That name is no longer used, but the __cdb__ prefix
  remains.

* __src/backend/gpopt/__

  Contains the so-called __translator__ library, for using the ORCA
  optimizer with Greenplum. The translator library is written in C++
  code, and contains glue code for translating plans and queries
  between the DXL format used by ORCA, and the PostgreSQL internal
  representation. This goes unused, unless building with
  _--enable-orca_.

* __src/backend/gp_libpq_fe/__

  A slightly modified copy of libpq. The master node uses this to
  connect to segments, and to send fragments of a query plan to
  segments for execution. It is linked directly into the backend, it
  is not a shared library like libpq.

* __src/backend/fts/__

  FTS is a process that runs in the master node, and periodically
  polls the segments to maintain the status of each segment.

X
Xin Zhang 已提交
112
## Build GPDB with Planner
113 114

```
X
Xin Zhang 已提交
115 116 117
# Clean environment
make distclean

X
Xin Zhang 已提交
118
# Configure build environment to install at /usr/local/gpdb
X
Xin Zhang 已提交
119
./configure --prefix=/usr/local/gpdb
120

X
Xin Zhang 已提交
121
# Compile and install
122 123 124
make
make install

X
Xin Zhang 已提交
125 126
# Bring in greenplum environment into your running shell
source /usr/local/gpdb/greenplum_path.sh
127

X
Xin Zhang 已提交
128 129
# Start demo cluster (gpdemo-env.sh is created which contain
# __PGPORT__ and __MASTER_DATA_DIRECTORY__ values)
130 131 132 133 134
cd gpAux/gpdemo
make cluster
source gpdemo-env.sh
```

X
Xin Zhang 已提交
135 136
## Build GPDB with GPORCA

X
Xin Zhang 已提交
137
Only need to change the `configure` with additional option `--enable-orca`.
X
Xin Zhang 已提交
138 139
```
# Configure build environment to install at /usr/local/gpdb
X
Xin Zhang 已提交
140
# Enable GPORCA
141 142 143
# Build with perl module (PL/Perl)
# Build with python module (PL/Python)
# Build with XML support
X
Xin Zhang 已提交
144 145 146
./configure --enable-orca --with-perl --with-python --with-libxml --prefix=/usr/local/gpdb
```

X
Xin Zhang 已提交
147 148 149 150 151 152
Once build and started, run `psql` and check the GPOPT (e.g. GPORCA) version:

```
select gp_opt_version();
```

X
Xin Zhang 已提交
153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177
## Regression tests

* The default regression tests

```
make installcheck-good
```

* optional extra/heavier regression tests

```
make installcheck-bugbuster
```

* The PostgreSQL __check__ target does not work. Setting up a
  Greenplum cluster is more complicated than a single-node PostgreSQL
  installation, and no-one's done the work to have __make check__
  create a cluster. Create a cluster manually or use gpAux/gpdemo/
  (example below) and run __make installcheck-good__ against
  that. Patches are welcome!

* The PostgreSQL __installcheck__ target does not work either, because
  some tests are known to fail with Greenplum. The
  __installcheck-good__ schedule excludes those tests.

178 179 180 181 182 183 184 185 186 187 188 189 190 191
## Glossary

* __QD__

  Query Dispatcher. A synonym for the master server.

* __QE__

  Query Executor. A synonym for a segment server.

## Documentation

For Greenplum Database documentation, please check online docs:
http://gpdb.docs.pivotal.io
J
Jignesh Patel 已提交
192 193

There is also a Vagrant-based quickstart guide for developers in `vagrant/README.md`.