clickhouse集群部署保姆级教程-EW帮帮网

ClickHouse安装

版本要求

23.8及之后的版本

硬件要求

三台机器

建议配置

磁盘 ssd 500G
内存 32g
cpu 16c

最低配置

磁盘机械硬盘 50G
内存 4g
cpu 4c

容量规划

一亿条数据大约使用1TB磁盘容量

参考官方容量推荐

安装包准备

zookeeper安装

zookeeper需要java启动，8版本以上，三台机器都需要执行

解压安装：

yum install -y java

tar -zxvf apache-zookeeper-3.5.9-bin.tar.gz -C /app

配置环境变量：

 vi /etc/profile

添加我们需要的配置信息：

ZOOKEEPER_HOME 为你安装的zookeeper目录

export ZOOKEEPER_HOME=/app/apache-zookeeper-3.9.3-bin
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH

检测生效

source /etc/profile
echo $ZOOKEEPER_HOME

配置zookeeper

创建data目录

mkdir /app/apache-zookeeper-3.9.3-bin/data

修改配置

cd /app/apache-zookeeper-3.9.3-bin/conf
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/app/apache-zookeeper-3.9.3-bin/data
clientPort=2181

server.1=ip1:2888:3888
server.2=ip2:2888:3888
server.3=ip3:2888:3888

为每台机器配置节点id

在机器一上执行：

echo 1 >/app/apache-zookeeper-3.9.3-bin/data/myid

在机器二上执行：

echo 2 >/app/apache-zookeeper-3.9.3-bin/data/myid

在机器三上执行：

echo 3 >/app/apache-zookeeper-3.9.3-bin/data/myid

启动zookeeper

cd /app/apache-zookeeper-3.9.3-bin/bin
sh zkServer.sh start

验证

启动成功后可以随便选择一个节点尝试连接另一个节点验证：

sh zkCli.sh -server ip:2181

clickhouse安装

检查cpu指令集是否支持

执行grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"，返回SSE 4.2 supported
表示检查通过

数据盘挂载

2.1.挂载目录

:::info
clickhouse数据库数据存储目录建议使用挂载目录

挂载目录需要客户提供并完成挂载

示例挂载命令：mount /dev/sdb1 /app

:::

2.2.创建数据目录

mkdir -p /app/clickhouse/data
mkdir -p /app/clickhouse/tmp

2.3.授予目录权限

chown -R clickhouse:clickhouse /app/clickhouse
chmod 755 /app/clickhouse

下载安装包

http://192.168.0.107/clickhouse/rpm

![](https://i-blog.csdnimg.cn/img_convert/422adfdebe8ec469fc4dcd422cf8d1cb.png下载最新安装包（推荐最新版本）

arm版本使用：

clickhouse-common-static-xxx.aarch64.rpm
clickhouse-server-xxx.aarch64.rpm
clickhouse-client-xxx.aarch64.rpm

安装

rpm -ivh clickhouse-common-static-*.rpm clickhouse-server-*.rpm clickhouse-client-*.rpm

# 有提示输入账号的直接回车跳过，后续统一处理

clickhouse配置

默认配置

添加配置/etc/clickhouse-server/config.xml

vim config.xml

<listen_host>0.0.0.0</listen_host>
<timezone>Asia/Shanghai</timezone>

<level>information</level>

修改数据目录

vim /etc/clickhouse-server/config.xml
路径改为挂载路径即可

6.1 用户管理配置

修改default用户密码为econage123，全部权限，只允许本地登录
添加用户user，密码为econage123，读写权限，bpi服务使用
添加管理用户admin，密码为econage123，读写权限，ddl权限
添加只读用户read，密码为econage123

添加/etc/clickhouse-server/users.d/users.xml，自行修改相关密码

<?xml version="1.0"?>
<clickhouse>
    <!-- See also the files in users.d directory where the settings can be overridden. -->

    <!-- Profiles of settings. -->
    <profiles>
        <!-- Default settings. -->
        <default>
            <max_bytes_before_external_group_by>268435456</max_bytes_before_external_group_by>

            <max_bytes_before_external_sort>268435456</max_bytes_before_external_sort>

            <max_download_threads>1</max_download_threads>

            <input_format_parallel_parsing>0</input_format_parallel_parsing>

            <output_format_parallel_formatting>0</output_format_parallel_formatting>

            <optimize_trivial_insert_select>1</optimize_trivial_insert_select>

        </default>

        <!-- Profile that allows only read queries. -->
        <read>
            <readonly>1</readonly>

            <max_bytes_before_external_group_by>268435456</max_bytes_before_external_group_by>

            <max_bytes_before_external_sort>268435456</max_bytes_before_external_sort>

            <max_download_threads>1</max_download_threads>

            <input_format_parallel_parsing>0</input_format_parallel_parsing>

            <output_format_parallel_formatting>0</output_format_parallel_formatting>

            <optimize_trivial_insert_select>1</optimize_trivial_insert_select>

        </read>

        <write>
            <readonly>0</readonly>

            <allow_ddl>0</allow_ddl>

            <max_bytes_before_external_group_by>268435456</max_bytes_before_external_group_by>

            <max_bytes_before_external_sort>268435456</max_bytes_before_external_sort>

            <max_download_threads>1</max_download_threads>

            <input_format_parallel_parsing>0</input_format_parallel_parsing>

            <output_format_parallel_formatting>0</output_format_parallel_formatting>

            <optimize_trivial_insert_select>1</optimize_trivial_insert_select>

        </write>

    </profiles>

    <!-- Users and ACL. -->
    <users>
        <read>
            <password>econage123</password>

            <networks>
                <ip>::/0</ip>

            </networks>

            <profile>read</profile>

            <quota>default</quota>

        </read>

        <admin>
            <password>econage123</password>

            <networks>
                <ip>::/0</ip>

            </networks>

            <profile>default</profile>

            <quota>default</quota>

        </admin>

        <user>
            <password>econage123</password>

            <networks>
                <ip>::/0</ip>

            </networks>

            <profile>write</profile>

            <quota>default</quota>

        </user>

        <!-- If user name was not specified, 'default' user is used. -->
        <default>
            <password>econage123</password>

            <networks>
                <ip>::1</ip>

            </networks>

            <!-- Settings profile for user. -->
            <profile>default</profile>

            <!-- Quota for user. -->
            <quota>default</quota>

            <!-- User can create other users and grant rights to them. -->
            <access_management>1</access_management>

        </default>

    </users>

    <!-- Quotas. -->
    <quotas>
        <!-- Name of quota. -->
        <default>
            <!-- Limits for time interval. You could specify many intervals with different limits. -->
            <interval>
                <!-- Length of interval. -->
                <duration>3600</duration>

                <!-- No limits. Just calculate resource usage for time interval. -->
                <queries>0</queries>

                <errors>0</errors>

                <result_rows>0</result_rows>

                <read_rows>0</read_rows>

                <execution_time>0</execution_time>

            </interval>

        </default>

    </quotas>

</clickhouse>

启动服务

systemctl start clickhouse-server
或者
clickhouse start

校验服务

curl localhost:8123
#返回'Ok.'则服务启动正常
clickhouse-client --user=user --password=econage123 --host=192.168.7.18 --query="select version()"
#返回版本号则用户创建正常

clickhouse-client --user=admin --password=econage123 --host=192.168.7.18 --query="select version()"
#返回版本号则用户创建正常

clickhosue集群版本配置

三台机器都需要执行上述clickouse安装步骤

添加zookeeper配置

vim /etc/clickhouse-server/config.xml

host是zookeeper的三个地址

port默认2181

修改配置

<zookeeper>
        <node index="1">
            <host>192.168.7.141</host>
            <port>2181</port>
        </node>
        <node index="2">
            <host>192.168.7.142</host>
            <port>2181</port>
        </node>
        <node index="3">
            <host>192.168.7.143</host>
            <port>2181</port>
        </node>
    </zookeeper>

配置分片

host是clickhouse的三个地址

port默认9000

                <replica>
                    <host>192.168.7.141</host>
                    <port>9000</port>
                </replica>
                <replica>
                    <host>192.168.7.142</host>
                    <port>9000</port>
                </replica>
                <replica>
                    <host>192.168.7.143</host>
                    <port>9000</port>
                </replica>

配置macros

第一台机器

    <macros>
        <shard>01</shard>
        <replica>01</replica>
    </macros>

第二台机器

    <macros>
        <shard>01</shard>
        <replica>02</replica>
    </macros>

第三台机器

    <macros>
        <shard>01</shard>
        <replica>03</replica>
    </macros>

启动服务

systemctl start clickhouse-server
或者
clickhouse start

验证集群同步

登录任意一台机器

clickhouse-client --user admin --password econage123

select * from system.clusters;

集群名称default

创建 ReplicatedMergeTree 测试表：

任选一台机器，创建一个 ReplicatedMergeTree 引擎的测试表，测试 ZooKeeper 同步功能

CREATE TABLE test_ck ON CLUSTER default (EventDate DateTime, Number UInt32, id UInt32 )
 ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/test_ck', '{replica}') PARTITION BY toYYYYMM(EventDate) 
 ORDER BY (Number, EventDate, intHash32(id)) SAMPLE BY intHash32(id);

在其他机器节点show tables查看表结构是否同步成功;

创建 Distributed 引擎测试表：
创建一个分布式测试表测试数据分片是否正常。因为已经配置了zookeeper，所以创建表的DDL语句也会同步到其他节点上。

CREATE TABLE dis_test ON CLUSTER default  AS test_ck
ENGINE = Distributed(default, default, test_ck, rand())

show table

文件清单

clickhouse-common-static-*.rpm
clickhouse-server-*.rpm
clickhouse-client-*.rpm
/etc/clickhouse-server/users.d/users.xml
/etc/clickhouse-server/config.d/config.xml

clickhouse目录说明

数据文件目录 /var/lib/clickhouse/
日志文件目录 /var/log/clickhouse-server/

可视化管理工具

dbeaver
http://localhost:8123/play
更多参考 https://clickhouse.com/docs/zh/interfaces/third-party/gui

clickhouse集群部署保姆级教程

ClickHouse安装

版本要求

硬件要求

三台机器

建议配置

最低配置

容量规划

安装包准备

zookeeper安装

解压安装：

配置环境变量：

配置zookeeper

为每台机器配置节点id

启动zookeeper

验证

clickhouse安装

检查cpu指令集是否支持

数据盘挂载

2.1.挂载目录

2.2.创建数据目录

2.3.授予目录权限

下载安装包

安装

clickhouse配置

默认配置

修改挂载目录（可选，无挂载目录可不用）

6.1 用户管理配置

启动服务

校验服务

clickhosue集群版本配置

添加zookeeper配置

配置分片

配置macros

启动服务

验证集群同步

文件清单

clickhouse目录说明

可视化管理工具

网站公告

今日签到

热门文章

最新发布