README.md 3.4 KB
Newer Older
W
Wu Tao 已提交
1 2 3 4 5 6
[github-release]: https://github.com/XiaoMi/pegasus/releases
[PacificA]: https://www.microsoft.com/en-us/research/publication/pacifica-replication-in-log-based-distributed-storage-systems/
[pegasus-rocksdb]: https://github.com/xiaomi/pegasus-rocksdb
[facebook-rocksdb]: https://github.com/facebook/rocksdb
[hbase]: https://hbase.apache.org/
[website]: https://pegasus-kv.github.io
Q
qinzuoyan 已提交
7

W
Wu Tao 已提交
8
![pegasus-logo](docs/media-img/pegasus-logo.png)
Q
qinzuoyan 已提交
9

W
Wu Tao 已提交
10 11 12
[![Build Status](https://travis-ci.org/XiaoMi/pegasus.svg?branch=master)](https://travis-ci.org/XiaoMi/pegasus)
[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)
[![Releases](https://img.shields.io/github/release/xiaomi/pegasus.svg)][github-release]
Q
qinzuoyan 已提交
13

W
Wu Tao 已提交
14 15
**Note**: The `master` branch may be in an *unstable or even broken state* during development.
Please use [releases][github-release] instead of the `master` branch in order to get stable binaries.
Q
qinzuoyan 已提交
16

W
Wu Tao 已提交
17
Pegasus is a distributed key-value storage system which is designed to be:
Q
qinzuoyan 已提交
18

W
Wu Tao 已提交
19 20 21 22
- **horizontally scalable** distributed using hash-based partitioning
- **strongly consistent**: ensured by [PacificA][PacificA] consensus protocol
- **high-performance**: using [RocksDB][pegasus-rocksdb] as underlying storage engine
- **simple**: well-defined, easy-to-use APIs
Q
qinzuoyan 已提交
23

W
Wu Tao 已提交
24 25 26
Pegasus has been widely-used in XiaoMi and serves millions of requests per second.
It is a mature, active project. We hope to build a diverse developer and user
community and attract contributions from more people.
Q
qinzuoyan 已提交
27

W
Wu Tao 已提交
28
## Background
Q
qinzuoyan 已提交
29

W
Wu Tao 已提交
30 31 32
HBase was recognized as the only large-scale KV store solution in XiaoMi
until Pegasus came out in 2015 to solve the problem of high latency
of HBase because of its Java GC and RPC overhead of the underlying distributed filesystem.
Q
qinzuoyan 已提交
33

W
Wu Tao 已提交
34 35 36 37
Pegasus targets to fill the gap between Redis and HBase. As the former
is in-memory, low latency, but does not provide a strong-consistency guarantee.
And unlike the latter, Pegasus is entirely written in C++ and its write-path
relies merely on the local filesystem.
Q
qinzuoyan 已提交
38

W
Wu Tao 已提交
39 40 41
Apart from the performance requirements, we also need a storage system
to ensure multiple-level data safety and support fast data migration
between data centers, automatic load balancing, and online partition split.
Q
qinzuoyan 已提交
42

W
Wu Tao 已提交
43 44 45
After investigating the existing storage systems in the open source world,
we could hardly find a suitable solution to satisfy all the requirements.
So the journey of Pegasus begins.
Q
qinzuoyan 已提交
46

W
Wu Tao 已提交
47
## To start using Pegasus
Q
qinzuoyan 已提交
48

W
Wu Tao 已提交
49
See our documentation on [Pegasus Website][website].
Q
qinzuoyan 已提交
50

Q
qinzuoyan 已提交
51 52
## Related Projects

Q
QinZuoyan 已提交
53
Submodules:
W
Wu Tao 已提交
54 55 56

- [rDSN](https://github.com/xiaomi/rdsn)
- [RocksDB](https://github.com/xiaomi/pegasus-rocksdb)
Q
QinZuoyan 已提交
57

Q
qinzuoyan 已提交
58
Client libs:
W
Wu Tao 已提交
59 60 61 62 63 64

- [Java client](https://github.com/xiaomi/pegasus-java-client)
- [Python Client](https://github.com/xiaomi/pegasus-python-client)
- [Go Client](https://github.com/xiaomi/pegasus-go-client)
- [Node.js Client](https://github.com/xiaomi/pegasus-nodejs-client)
- [Scala Client](https://github.com/xiaomi/pegasus-scala-client)
Q
qinzuoyan 已提交
65 66

Test tools:
W
Wu Tao 已提交
67 68 69

- [Java YCSB](https://github.com/xiaomi/pegasus-YCSB)
- [Go YCSB](https://github.com/xiaomi/pegasus-YCSB-go)
Q
QinZuoyan 已提交
70 71

Data import/export tools:
Q
qinzuoyan 已提交
72

W
Wu Tao 已提交
73
- [DataX](https://github.com/xiaomi/pegasus-datax)
Q
qinzuoyan 已提交
74

W
Wu Tao 已提交
75
## Contact
Q
qinzuoyan 已提交
76

W
Wu Tao 已提交
77 78
- Gitter: <https://gitter.im/XiaoMi/Pegasus>
- Issues: <https://github.com/XiaoMi/pegasus/issues>
W
WeijieSun 已提交
79

Q
qinzuoyan 已提交
80 81
## License

W
Wu Tao 已提交
82 83
Copyright 2015-now Xiaomi, Inc. Licensed under the Apache License, Version 2.0:
<http://www.apache.org/licenses/LICENSE-2.0>