docker-deploy-hbase

docker部署hbase

系统为 centos7

docker 安装的版本为 20.10.9

一,docker安装

卸载系统仓库自带的docker

1
2
3
4
5
6
7
8
sudo yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine

有三种方式来安装docker

  1. 更新官方仓库后yum安装
  2. 手动下载文件安装
  3. 脚本文件安装

目前准备使用第三种方式来安装

1
curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh

二,docker-compose来实现单节点容器的编排

1
2
3
4
curl -L https://get.daocloud.io/docker/compose/releases/download/1.25.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

/usr/local/bin/docker-compose -f docker-compose-distributed-local.yml up -d

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
resourcemanager  8088 

hbase-master 16000

hbase-regionserver 16020

zookeeper 2888 2181 3888

historyserver 8188

namenode 50070

datanode 50075

nodemanager 8042

三,使用docker stack来实现

1.创建swarm的网络保证各个节点都能平等的被编排

docker swarm init 初始化一个swarm 的网络

docker swarm join-token worker 显示 加入的token

docker swarm join-token manager 显示加入的token

目前使用下面加入manager,因为worker节点不能显示集群的细节

在其他服务器上输入 下面类似的内容加入节点

docker swarm join –token SWMTKN-1-0w6kjxx0j4epupf6r844dxpzoyxikrxqqgrqg319vqfg0y1gw7-31q237m76pyii2g0pw1c8nmtu 192.168.31.136:2377

2.docker node ls 查看目前的网络

image-20211214152312286

docker network create –driver overlay –scope swarm hbase

docker stack deploy –compose-file docker-compose-v3.yml hdp

docker stack ls

docker stack down hadoop

docker network create –driver overlay –attachable –subnet 10.11.0.0/24 sg-hadoop

Creating service hdp_nodemanager
Creating service hdp_hbase-master

/opt/hbase-1.2.6/bin/hbase-daemon.sh restart regionserver

echo -e “hbase-region2\nhbase-region” > /etc/hbase/regionservers

docker service update –with-registry-auth

需要追加 –with-registry-auth

docker exec -t 70f261259630 netstat -an | grep LISTEN | grep 50070

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
version: '3'

services:
namenode:
image: bde2020/hadoop-namenode:2.0.0-hadoop3.2.1-java8
networks:
- hbase
volumes:
- namenode:/hadoop/dfs/name
environment:
- CLUSTER_NAME=test
env_file:
- ./hadoop.env
deploy:
mode: global
# replicas: 1
restart_policy:
condition: on-failure
# placement:
# constraints:
# - node.hostname == akswnc4.aksw.uni-leipzig.de
labels:
traefik.docker.network: hbase
traefik.port: 9000
ports:
- 9000:9000
# - 50070:50070

datanode:
image: bde2020/hadoop-datanode:2.0.0-hadoop3.2.1-java8
networks:
- hbase
volumes:
- datanode:/hadoop/dfs/data
env_file:
- ./hadoop.env
environment:
SERVICE_PRECONDITION: "namenode:9000"
deploy:
mode: global
restart_policy:
condition: on-failure
labels:
traefik.docker.network: hbase
traefik.port: 9867
ports:
- 9867:9867

resourcemanager:
image: bde2020/hadoop-resourcemanager:2.0.0-hadoop3.2.1-java8
networks:
- hbase
environment:
SERVICE_PRECONDITION: "namenode:9000 datanode:9867"
env_file:
- ./hadoop.env
deploy:
mode: global
# replicas: 1
restart_policy:
condition: on-failure
# placement:
# constraints:
# - node.hostname == akswnc4.aksw.uni-leipzig.de
labels:
traefik.docker.network: hbase
traefik.port: 8088
healthcheck:
disable: true
ports:
- 8088:8088


nodemanager:
image: bde2020/hadoop-nodemanager:2.0.0-hadoop3.2.1-java8
networks:
- hbase
environment:
SERVICE_PRECONDITION: "namenode:9000 datanode:9867 resourcemanager:8088"
env_file:
- ./hadoop.env
deploy:
mode: global
restart_policy:
condition: on-failure
labels:
traefik.docker.network: hbase
traefik.port: 8042
ports:
- 8042:8042


historyserver:
image: bde2020/hadoop-historyserver:2.0.0-hadoop3.2.1-java8
networks:
- hbase
volumes:
- hadoop_historyserver:/hadoop/yarn/timeline
environment:
SERVICE_PRECONDITION: "namenode:9000 datanode:9867 resourcemanager:8088"
env_file:
- ./hadoop.env
deploy:
mode: global
# replicas: 1
# placement:
# constraints:
# - node.hostname == akswnc4.aksw.uni-leipzig.de
labels:
traefik.docker.network: hbase
traefik.port: 8188
ports:
- 8188:8188
zookeeper:
image: zookeeper:3.4.10
networks:
- hbase
environment:
ZOO_MY_ID: 1
ZOO_SERVERS: server.1=0.0.0.0:2888:3888
ports:
- 2181:2181

hbase-master:
image: bde2020/hbase-master:1.0.0-hbase1.2.6
hostname: hbase-master
networks:
- hbase
env_file:
- ./hbase-distributed-local.env
environment:
SERVICE_PRECONDITION: "namenode:9000 datanode:9867 zookeeper:2181"
ports:
- 16000:16000
- 16010:16010

hbase-region:
image: bde2020/hbase-regionserver:1.0.0-hbase1.2.6
hostname: hbase-regionserver
networks:
- hbase
env_file:
- ./hbase-distributed-local.env
environment:
HBASE_CONF_hbase_regionserver_hostname: hbase-region
SERVICE_PRECONDITION: "namenode:9000 datanode:9867 zookeepe:2181 hbase-master:16010"
ports:
- 16020:16020
- 16030:16030

visualizer:
image: dockersamples/visualizer:stable
ports:
- "8080:8080"
stop_grace_period: 1m30s
deploy:
mode: global
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
deploy:
placement:
constraints: [node.role == manager]

volumes:
datanode:
namenode:
hadoop_historyserver:

networks:
hbase:
external:
name: hbase

将自动创建service

3.使用docker stack ls来查看创建stack

image-20211214152334572

4.使用docker service ls来查看service列表

image-20211214152347155

5.使用yml中visualizer组件来显示部署的情况

image-20211214152401033

组件 节点 默认端口 配置 用途说明
HDFS DateNode 50010 dfs.datanode.address datanode服务端口,用于数据传输
HDFS DateNode 50075 dfs.datanode.http.address http服务的端口
HDFS DateNode 50475 dfs.datanode.https.address http服务的端口
HDFS DateNode 50020 dfs.datanode.ipc.address ipc服务的端口
HDFS NameNode 50070 dfs.namenode.http-address http服务的端口
HDFS NameNode 50470 dfs.namenode.https-address https服务的端口
HDFS NameNode 8020 fs.defaultFS 接收Client连接的RPC端口,用于获取文件系统metadata信息。
HDFS journalnode 8485 dfs.journalnode.rpc-address RPC服务
HDFS journalnode 8480 dfs.journalnode.http-address HTTP服务
HDFS ZKFC 8019 dfs.ha.zkfc.port ZooKeeper FailoverController,用于NN HA
YARN ResourceManage 8032 yarn.resourcemanager.address RM的applications manager(ASM)端口
YARN ResourceManage 8030 yarn.resourcemanager.scheduler.address scheduler组件的IPC端口
YARN ResourceManage 8031 yarn.resourcemanager.resource-tracker.address IPC
YARN ResourceManage 8033 yarn.resourcemanager.admin.address IPC
YARN ResourceManage 8088 yarn.resourcemanager.webapp.address http服务端口
YARN NodeManager 8040 yarn.nodemanager.localizer.address localizer IPC
YARN NodeManager 8042 yarn.nodemanager.webapp.address http服务端口
YARN NodeManager 8041 yarn.nodemanager.address NM中container manager的端口
YARN JobHistory Server 10020 mapreduce.jobhistory.address IPC
YARN JobHistory Server 19888 mapreduce.jobhistory.webapp.address http服务端口
HBase Master 60000 hbase.master.port IPC (主节点(HBase 主节点和任何备份的 HBase 主节点)
HBase Master 60010 hbase.master.info.port http服务端口 (主节点(HBase 主节点和备份 HBase 主节点如果有))
HBase RegionServer 60020 hbase.regionserver.port IPC( 所有从节点)
HBase RegionServer 60030 hbase.regionserver.info.port http服务端口
HBase HQuorumPeer 2181 hbase.zookeeper.property.clientPort HBase-managed ZK mode,使用独立的ZooKeeper集群则不会启用该端口。
HBase HQuorumPeer 2888 hbase.zookeeper.peerport HBase-managed ZK mode,使用独立的ZooKeeper集群则不会启用该端口。
HBase HQuorumPeer 3888 hbase.zookeeper.leaderport HBase-managed ZK mode,使用独立的ZooKeeper集群则不会启用该端口。
Hive Metastore 9085 /etc/default/hive-metastore中export PORT=来更新默认端口
Hive HiveServer 10000 /etc/hive/conf/hive-env.sh中export HIVE_SERVER2_THRIFT_PORT=来更新默认端口
ZooKeeper Server 2181 /etc/zookeeper/conf/zoo.cfg中clientPort= 来自 ZooKeeper 的 config 的属性zoo.cfg。客户端将连接的端口。
ZooKeeper Server 2888 /etc/zookeeper/conf/zoo.cfg中server.x=[hostname]:nnnnn[:nnnnn],标蓝部分 follower用来连接到leader,只在leader上监听该端口
ZooKeeper Server 3888 /etc/zookeeper/conf/zoo.cfg中server.x=[hostname]:nnnnn[:nnnnn],标蓝部分 ZooKeeper 对等点用于相互通信的端口。有关更多信息,请参见此处

十,遇到的坑

1.在hadoop配置的需要主要

配置url类似于 host:port 这种时 host 不能报考_ 等字符,否则会报 contain a valid host:port authority 包含 _

2.使用docker stack 初始化的环境

所有的服务会有个stack名称的前缀,且不能修改

在集群中加入容器中后可以通过 服务名称就是 STACK_NAME 来调用也可以通过在yml中定义的services下面的节点名称来调用这个需要注意

初始化后出现所有的service会加前缀导致域名不能和访问

3.哈哈哈

1
File /hbase/.tmp/hbase.version could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and 1 node(s) are excluded in this operation

4.配置出错导致服务绑定对应的ip

1
java.io.IOException: Problem binding to hbase-region/10.0.4.107:16040 : Cannot assign requested address. To switch ports use the 'hbase.regionserver.port' configuration property.