加入CODE CHINA

· 不限速    · 不限空间    · 不限人数    · 私仓免费

免费加入
README.md

FlinkX

License

English | 中文

Communication

  • We are recruiting Big data platform development engineers.If you want more information about the position, please add WeChat ID [ysqwhiletrue] or email your resume to sishu@dtstack.com.

  • We use DingTalk to communicate,You can search the group number [30537511] or scan the QR code below to join the communication group

Introduction

FlinkX is a data synchronization tool based on Flink. FlinkX can collect static data, such as MySQL, HDFS, etc, as well as real-time changing data, such as MySQL binlog, Kafka, etc. FlinkX currently includes the following features:

  • Most plugins support concurrent reading and writing of data, which can greatly improve the speed of reading and writing;

  • Some plug-ins support the function of failure recovery, which can restore tasks from the failed location and save running time; Failure Recovery

  • The Reader plugin for relational databases supports interval polling. It can continuously collect changing data; Interval Polling

  • Some databases support opening Kerberos security authentication; Kerberos

  • Limit the reading speed of Reader plugins and reduce the impact on business databases;

  • Save the dirty data when writing data;

  • Limit the maximum number of dirty data;

  • Multiple running modes: Local,Standalone,Yarn Session,Yarn Per;

The following databases are currently supported:

Database Type Reader Writer
Batch Synchronization MySQL doc doc
Oracle doc doc
SqlServer doc doc
PostgreSQL doc doc
DB2 doc doc
GBase doc doc
ClickHouse doc doc
PolarDB doc doc
SAP Hana doc doc
Teradata doc doc
Phoenix doc doc
达梦 doc doc
Greenplum doc doc
KingBase doc doc
Cassandra doc doc
ODPS doc doc
HBase doc doc
MongoDB doc doc
Kudu doc doc
ElasticSearch doc doc
FTP doc doc
HDFS doc doc
Carbondata doc doc
Stream doc doc
Redis doc
Hive doc
Stream Synchronization Kafka doc doc
EMQX doc doc
RestApi doc
MySQL Binlog doc
MongoDB Oplog doc
PostgreSQL WAL doc

Quick Start

Please click Quick Start

General Configuration

Please click General Configuration

Statistics Metric

Please click Statistics Metric

Kerberos

Please click Kerberos

Questions

Please click Questions

How to contribute FlinkX

Please click Contribution

License

FlinkX is under the Apache 2.0 license. See the LICENSE file for details.

项目简介

🚀 Github 镜像仓库 🚀

源项目地址

https://github.com/dtstack/flinkx

发行版本 22

1.11.0

全部发行版

贡献者 28

全部贡献者

开发语言

  • Java 99.8 %
  • Shell 0.2 %
  • Batchfile 0.0 %