描述

ltdts_mysql 用于将MySQL / MariaDB中的数据同步到LightDB。 在运行ltdts_mysql.jar之前，您应该首先完成以下准备工作。

Zookeeper

您可以从以下网址下载：https://dlcdn.apache.org/zookeeper/zookeeper-3.8.0/apache-zookeeper-3.8.0-bin.tar.gz

# 解压缩包
tar -zxvf apache-zookeeper-3.8.0-bin.tar.gz
# 创建一个新目录以存储zookeeper数据
mkdir zk-data
cd conf/
# 复制文件
cp zoo_sample.cfg zoo.cfg
# 编辑文件
vim zoo.cfg
dataDir=/data/lightdb/zm/apache-zookeeper-3.8.0-bin/zk-data

cd apache-zookeeper-3.8.0-bin/bin
# 启动zookeeper服务器
./zkServer.sh start

Kafka

您可以从以下网址下载Kafka：https://downloads.apache.org/kafka/3.3.1/kafka_2.12-3.3.1.tgz

您可以从以下网址下载Debezium for mysql：https://repo1.maven.org/maven2/io/debezium/debezium-connector-mysql/1.9.7.Final/debezium-connector-mysql-1.9.7.Final-plugin.tar.gz

# 解压Kafka包
tar -zxvf kafka_2.12-3.3.1.tgz
# 解压Debezium包
unzip debezium-debezium-connector-mysql-1.9.7.zip

# 创建一个新目录以存储kafka数据
mkdir kafka-data

# 创建一个新目录以存储kafka插件
mkdir kafka-plugin

# 将Debezium目录移动到kafka-plugin下
mv debezium-debezium-connector-mysql-1.9.7 kafka-plugin/

# 然后kafka目录如下所示
--kafka_2.12-3.3.1
   --config
   --bin
   --kafka-data
   --kafka-plugin
      --debezium-debezium-connector-mysql-1.9.7


# 编辑kafka的server.properties文件
vim server.properties
# 将以下值更改为
log.dirs=/data/lightdb/zm/kafka_2.12-3.3.1/kafka-data
# 将以下值更改为，使用您的地址
listeners=PLAINTEXT://Your IP:9092


# 编辑kafka的connect-distributed.properties文件
vim connect-distributed.properties
# 将以下值更改为，使用您的地址
bootstrap.servers=Your IP:9092
listeners=HTTP://Your IP:8083
# 将以下值更改为
plugin.path=/data/lightdb/zm/kafka_2.12-3.3.1/kafka-plugin


# 启动kafka
cd kafka_2.12-3.3.1/bin
./kafka-server-start.sh -daemon ../config/server.properties

# 启动kafka connect
./connect-distributed.sh -daemon ../config/connect-distributed.properties

Debezium

# 首先检查Kafka Connect是否可用，使用您的IP地址
curl -H "Accept:application/json" IP:8083/

# 列出所有Kafka连接器
curl -H "Accept:application/json" IP:8083/connectors

# 部署MySQL连接器
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" IP:8083/connectors/ -d '{
    "name": "test-connector",
    "config": {
        "connector.class": "io.debezium.connector.mysql.MySqlConnector",
        "tasks.max": "1",
        "database.hostname": "mysql host,example:127.0.0.1",
        "database.port": "3306",
        "database.user": "mysql user",
        "database.password": "mysql password",
        "database.server.id": "184077",
        "database.server.name": "mysql database name,example:acm",
        "database.include.list": "mysql database name,example:acm",
        "database.history.kafka.bootstrap.servers": "your kafka IP and port,example:127.0.0.1:9092",
        "database.history.kafka.topic": "schema-changes.acm",
        "include.schema.changes": "true",
        "decimal.handling.mode": "string",
        "include.schema.comments": "false"
    }
}'

# 删除MySQL连接器
curl -v -X DELETE IP:8083/connectors/test-connector

# 检查连接器的工作状态
curl -i IP:8083/connectors/test-connector/status

# 查看连接器的配置
curl -i IP:8083/connectors/test-connector/config

Options

必须指定以下至少一个选项来选择一个操作：

See Also