运维开发网

storm环境搭建

运维开发网 https://www.qedev.com 2020-03-25 17:24 出处:网络 作者:运维开发网整理
线上kafka版本 0.8.2.0 线上zookeeper版本:zookeeper-3.4.5 1.配置三台服务的host 127.0.0.1 storm01 127.0.0.2 storm02 127.0.0.3 storm03 开通防火墙 #zookeeper,storm -A INPUT -s 127.0.0.1/32 -p tcp -m multiport --dports 22:6553

线上kafka版本 0.8.2.0

线上zookeeper版本:zookeeper-3.4.5

1.配置三台服务的host

127.0.0.1 storm01

127.0.0.2 storm02

127.0.0.3 storm03

开通防火墙

#zookeeper,storm

-A INPUT -s 127.0.0.1/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT

-A INPUT -s 127.0.0.2/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT

-A INPUT -s 127.0.0.3/32 -p tcp -m multiport --dports 22:65535 -j ACCEPT

2.配置jdk

wget rpm包url

配置环境变量

export JAVA_HOME=/home/storm/jdk1.8.0_111

export ZOOKEEPER_HOME=/home/storm/zookeeper-3.4.10

export STORM_HOME=/home/storm/apache-storm-1.1.0

PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin:$PATH:$HOME/bin

3.配置zookeeper

tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/home/storm/zookeeper-3.4.10/data
dataLogDir=/home/storm/zookeeper-3.4.10/datalog
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

zookeeper.log.maxbackupindex=7

server.0=storm01:2888:3888
server.1=storm02:2888:3888
server.2=storm03:2888:3888

在dataDir下新建一个myid文件,写上对应的id

启动zookeeper

zkServer.sh start /home/storm/zookeeper-3.4.10/conf/zoo.cfg

查看zookeeper状态

zkServer.sh status

 

4.配置storm

storm.zookeeper.servers:
     - "storm01"
     - "storm02"
     - "storm03"
# 
nimbus.seeds: ["storm01"]
storm.local.dir: "/home/storm/apache-storm-1.1.0/data"
#slot数量一般以每个服务器的CPU线程数来计算。
supervisor.slots.ports:
     - 6700
     - 6701
     - 6702
     - 6703
     - 6704
     - 6705

 

同步配置

scp storm.yaml [email protected]:/home/storm/apache-storm-1.1.0/conf/

scp storm.yaml [email protected]:/home/storm/apache-storm-1.1.0/conf/

scp zoo.cfg [email protected]:/home/storm/zookeeper-3.4.10/conf

scp zoo.cfg [email protected]:/home/storm/zookeeper-3.4.10/conf

 

启动storm

storm nimbus >/dev/null 2>&1 &

storm ui >/dev/null 2>&1 &

storm supervisor >/dev/null 2>&1 &

 

错误1:

[[email protected] gavin]$storm nimbus

Need python version > 2.6

storm启动需要python2.6以上版本,storm中启动脚本python路径指的不正确,如果/usr/bin/python2.6不存在,$PYTHON也不存在,则使用PYTHON=/usr/bin/python这个

在第二个if后面手动指定PYTHON=/usr/local/Python-2.7.10/bin/python

0

精彩评论

暂无评论...
验证码 换一张
取 消