一、前期准备
1.1 静态ip,请查看虚拟机安装不放呢
1.2 hostname 以及 hosts文件修改
cat /etc/hostname 不同的机器设置不同的名字 cat /etc/hosts
192.168.0.110 kyle1192.168.0.111 kyle2192.168.0.112 kyle3
1.3 jdk安装(请见 )
1.4 SSH免密码登陆
cd ~/.ssh/ # 如果不存在,执行一下 ssh localhost 就好了# 生成密钥ssh-keygen -t rsa (一路回车)Generating public/private rsa key pair.Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa.Your public key has been saved in /root/.ssh/id_rsa.pub.The key fingerprint is:SHA256:cvlMVKUdaY41l+IiSS2qAO9el9xGg/5MGZT554uhE0o root@kyle1The key's randomart image is:+---[RSA 2048]----+| + ..o..|| . * o +=o.|| o = = o=oo || o o O o.o. || . . = S * + || . o E X . . || . . o * = o . || . . = . . || . |+----[SHA256]-----+# 导入公钥cat id_rsa.pub >> authorized_keys # 另外两台机器上执行完毕之后,将另外两台机器的id_rsa.pub 粘贴到 第一台机器的 authorized_keys 里# 将authorized_keys分发到另外两台机器上scp authorized_keys root@kyle2:.ssh/authorized_keys # 测试ssh kyle3
1.5 关闭防火墙
systemctl stop firewalld.servicesystemctl disable firewalld.service
二、下载
wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.8.5/hadoop-2.8.5.tar.gz
tar -zxvf hadoop-2.8.5.tar.gz -C /usr/local/
三、hadoop master节点配置
配置hadoop的配置文件core-site.xml,hdfs-site.xml,mapred-site.xml,yarn-site.xml,slaves(都在/usr/local/hadoop-2.8.5/etc/hadoop )文件夹下
3.1 core-site.xml
fs.default.name hdfs://kyle1:9000 hadoop.tmp.dir file:/usr/local/hadoop-2.8.5/tmp
3.2 hdfs-site.xml
dfs.replication 2 dfs.namenode.name.dir file:/usr/local/hadoop-2.8.5/dfs/name dfs.datanode.data.dir file:/usr/local/hadoop-2.8.5/dfs/data dfs.namenode.secondary.http-address kyle1:9001 dfs.webhdfs.enabled true
3.3 mapred-site.xml
cp mapred-site.xml.template mapred-site.xmlvim mapred-site.xmlmapreduce.framework.name yarn
3.4 yarn-site.xml
NOTE: 第一个hostname是配置yarn主节点的域名,需要根据自己的情况配置
yarn.resourcemanager.hostname kyle1 yarn.nodemanager.aux-services mapreduce_shuffle
3.5 slaves
kyle2kyle3
3.6 hadoop-env.sh (/usr/local/hadoop-2.8.5/etc/hadoop)
export JAVA_HOME=/usr/local/java/jdk1.8.0_191
四、其他机器配置
scp -r hadoop-2.8.5 root@kyle2:/usr/local/scp -r hadoop-2.8.5 root@kyle3:/usr/local/
五、配置环境变量
## hadoopexport HADOOP_HOME=/usr/local/hadoop-2.8.5export PATH=$HADOOP_HOME/sbin:$PATH
六、格式化节点。
bin/hdfs namenode -format
七、启动
# 启动hdfssbin/start-dfs.sh# 启动yarnsbin/start-yarn.sh
八、查看进程
[root@kyle1 hadoop-2.8.5]# jps6634 SecondaryNameNode2251 QuorumPeerMain6478 NameNode
九、网页查看
http://kyle1:50070 (hdfs地址)
http://kyle1:8088 (yarn地址)