Seatunnel部署

基础环境

Seatunnel version 3.5.6

JDK

• version >= 1.8.0 下载地址
• 环境变量

➜  vim .zshrc
JAVA_HOME=/usr/local/jdk-17.0.12
export PATH=$JAVA_HOME/bin:$PATH
CLASSPATH=$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$CLASSPATH:.
export CLASSPATH

➜  ~ java -version
java version "17.0.12" 2024-07-16 LTS
Java(TM) SE Runtime Environment (build 17.0.12+8-LTS-286)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.12+8-LTS-286, mixed mode, sharing)

Maven

• 下载地址

• 环境变量

➜  ~ vim ~/.zshrc
M2_HOME=/usr/local/apache-maven-3.9.9
export PATH=$M2_HOME/bin:$PATH

➜  ~ mvn -version
Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937)
Maven home: /usr/local/apache-maven-3.9.9
Java version: 17.0.12, vendor: Oracle Corporation, runtime: /usr/local/jdk-17.0.12
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "5.14.0-480.el9.aarch64", arch: "aarch64", family: "unix"

• 配置代理

 <proxies>
    <!-- proxy
     | Specification for one proxy, to be used in connecting to the network.
     | -->
    <proxy>
      <id>http</id>
      <active>true</active>
      <protocol>http</protocol>
      <username></username>
      <password></password>
      <host>192.168.10.2</host>
      <port>8889</port>
      <nonProxyHosts>local.net|some.host.com</nonProxyHosts>
    </proxy>
    <proxy>
      <id>https</id>
      <active>true</active>
      <protocol>https</protocol>
      <username></username>
      <password></password>
      <host>192.168.10.2</host>
      <port>8889</port>
      <nonProxyHosts>local.net|some.host.com</nonProxyHosts>
    </proxy>
  </proxies>

Seatunnel 安装

安装插件

• 下载

export version="2.3.5"
wget "https://archive.apache.org/dist/seatunnel/${version}/apache-seatunnel-${version}-bin.tar.gz"
tar -xzvf "apache-seatunnel-${version}-bin.tar.gz"

• 配置环境变量

如果 SeaTunnel Engine 和 Seatunnel Web 必须配置

export SEATUNNEL_HOME=/root/apache-seatunnel-2.3.6
export PATH=$PATH:$SEATUNNEL_HOME/bin

安装插件

# 修改插件脚本,替换使用本地mvn
➜  apache-seatunnel-2.3.6 vim bin/install-plugin.sh
:%s/${SEATUNNEL_HOME}\/mvnw/mvn/g
# 编辑 插件文件
➜  apache-seatunnel-2.3.6 vim config/plugin_config
# 安装插件
➜  apache-seatunnel-2.3.6 sh bin/install-plugin.sh 2.3.6
# 创建插件文件夹
➜  apache-seatunnel-2.3.6 mkdir connectors/{flink,flink-sql,seatunnel,spark}
# 测试安装状态
➜  apache-seatunnel-2.3.6 ./bin/seatunnel.sh --config ./config/v2.batch.config.template -e local

# 打印如下信息表示运行成功
***********************************************
           Job Statistic Information
***********************************************
Start Time                : 2024-08-19 20:47:46
End Time                  : 2024-08-19 20:47:47
Total Time(s)             :                   1
Total Read Count          :                  32
Total Write Count         :                  32
Total Failed Count        :                   0
***********************************************

部署 SeaTunnel Engine

官方文档

• 配置环境变量

export SEATUNNEL_HOME=/root/apache-seatunnel-2.3.6
export PATH=$PATH:$SEATUNNEL_HOME/bin

• 修改配置文件
• 创建日子文件夹 mkdir -p $SEATUNNEL_HOME/logs
• 启动 ./bin/seatunnel-cluster.sh -d
• 查看日志 tail -f $SEATUNNEL_HOME/logs/seatunnel-engine-server.log