【ELK】【Linux】单机搭建【zookeeper】&【kafka】
xixuefeng
ELK, ZooKeeper
2018-01-03 18:54:15
1,274 次浏览
kafka, zookeeper
【ELK】【Linux】单机搭建【zookeeper】&【kafka】已关闭评论
1:环境
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
## Linux [root@coll ~]# cat /etc/redhat-release CentOS Linux release 7.3.1611 (Core) [root@coll ~]# ## zookeeper [root@coll ~]# cd /soft/ [root@coll soft]# [root@coll soft]# ls zookeeper-3.5.3-beta.tar.gz zookeeper-3.5.3-beta.tar.gz [root@coll soft]# ## kafka [root@coll soft]# ls kafka_2.12-1.0.0.tgz kafka_2.12-1.0.0.tgz [root@coll soft]# |
2:部署zookeeper
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
## 创建目录存放快照数据及日志 [root@coll ~]# mkdir -p /data/zookeeper/zkdata [root@coll ~]# mkdir -p /data/zookeeper/zkdatalog ## 解tar包 [root@coll ~]# cd /soft [root@coll soft]# tar xvf zookeeper-3.5.3-beta.tar.gz -C /usr/local/ ## 修改配置文件 [root@coll soft]# cd /usr/local/zookeeper-3.5.3-beta/conf [root@coll conf]# ls configuration.xsl log4j.properties zoo_sample.cfg [root@coll conf]# [root@coll conf]# cp zoo_sample.cfg zoo.cfg [root@coll conf]# [root@coll conf]# ls configuration.xsl log4j.properties zoo.cfg zoo_sample.cfg [root@coll conf]# [root@coll conf]# vi zoo.cfg # 修改dataDir参数路径 dataDir=/data/zookeeper/zkdata # 增加dataLogDir,其实这个参数配不配置都行,如果不配置,日志会默认放在dataDir指定的目录下 dataLogDir=/data/zookeeper/zkdatalog ## 保存退出 ## 启动zookeeper,默认会找conf目录下的zoo.cfg配置文件 [root@coll zookeeper-3.5.3-beta]# pwd /usr/local/zookeeper-3.5.3-beta [root@coll zookeeper-3.5.3-beta]# ./bin/zkServer.sh start ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper-3.5.3-beta/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@coll zookeeper-3.5.3-beta]# ## 查看节点运行状态 [root@coll zookeeper-3.5.3-beta]# ./bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper-3.5.3-beta/bin/../conf/zoo.cfg Client port found: 2181. Client address: localhost. Mode: standalone [root@coll zookeeper-3.5.3-beta]# ## 使用jps命令可看到zookeeper服务的QuormPeerMain进程存在。 [root@coll zookeeper-3.5.3-beta]# jps 10201 QuorumPeerMain 9660 Main 10270 Jps [root@coll zookeeper-3.5.3-beta]# |
3:部署kafka
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
## 创建存放日志目录 [root@coll ~]# mkdir -p /data/kafka-logs [root@coll ~]# ## 修改配置文件 [root@coll ~]# cd /usr/local/kafka_2.12-1.0.0/config/ [root@coll config]# [root@coll config]# ls |grep server.properties server.properties [root@coll config]# [root@coll config]# vi server.properties # 日志目录,默认是在/tmp目录下,我们改为我们指定的目录,并将权限改为777 log.dirs=/data/kafka-logs # zookeeper的IP及端口,因为部署在一台机器上,如下默认值即可 zookeeper.connect=localhost:2181 ## 由于测试环境内存非常小,只有512M,所以启动时会提示如下错误 [root@coll kafka_2.12-1.0.0]# ./bin/kafka-server-start.sh config/server.properties Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c0000000, 1073741824, 0) failed; error='Cannot allocate memory' (errno=12) ## 可以通过编辑调整JVM堆大小来启动 export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M" -Xms 参数指定的最小堆大小。 -Xmx 最大堆大小。 ## 启动kafka [root@coll kafka_2.12-1.0.0]# export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M" [root@coll kafka_2.12-1.0.0]# ./bin/kafka-server-start.sh config/server.properties |