centos6.5 安装hadoop1.2.1的教程详解【亲测版】
本篇只简单介绍安装步骤
1.角色分配
10.11.84.4web-crawler--1.novalocalmaster/slave 10.11.84.5web-crawler--2.novalocalslave 10.11.84.6web-crawler--3.novalocalslave
2.安装目录/usr/local/hadoop（HADOOP_HOME）
3,创建grouphadoop
groupaddhadoop
4.创建用户hadoop
useradd-d/home/hadoop-ghadoophadoop chown-Rhadoop:hadoop/usr/local/hadoop
5.每台机器安装jdk（安装在相同的目录）
1)查看java版本，可以看到jdk为openjdk，最好更换为sunjdk
[hadoop@hadoop2~]$java-version openjdkversion"1.8.0_131" OpenJDKRuntimeEnvironment(build1.8.0_131-b12) OpenJDK64-BitServerVM(build25.131-b12,mixedmode) [hadoop@hadoop2~]$rpm-qa|grepjava java-1.8.0-openjdk-headless-1.8.0.131-11.b12.el7.x86_64 javapackages-tools-3.4.1-11.el7.noarch tzdata-java-2017b-1.el7.noarch java-1.7.0-openjdk-headless-1.7.0.141-2.6.10.5.el7.x86_64 java-1.7.0-openjdk-1.7.0.141-2.6.10.5.el7.x86_64 java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64 python-javapackages-3.4.1-11.el7.noarch
2）卸载openjdk
rpm-qa|grepjava|xargsyum-yremove
3）安装sunjdk
/usr/local目录下解压jdk-8u181-linux-i586.tar.gz并命名为jdk
配置环境变量如下
vim/etc/profile 末尾添加如下内容
exportJAVA_HOME=/usr/local/jdk
exportJRE_HOME=${JAVA_HOME}/jre
exportPATH=${JAVA_HOME}/bin:${PATH}
exportCLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:${CLASSPATH}
4）查看jdk是否安装成功
[root@hadoop2local]#java-version javaversion"1.8.0_151" Java(TM)SERuntimeEnvironment(build1.8.0_151-b12) JavaHotSpot(TM)64-BitServerVM(build25.151-b12,mixedmode)
6.master到slavessh免密码登录
在每台机器hadoop用户下执行如下命令
[hadoop@hadoop1~]$ssh-keygen-trsa-P''#一路直接回车 发送master机器~/.ssh/id_rsa.pub到所有slave节点 Scpid_rsa.pubhadoop@10.11.84.5:~ Scpid_rsa.pubhadoop@10.11.84.6:~ 让后在每台slave节点hadoop用户下执行如下命令 Cd~/.ssh Cat../id_rsa.pub>>authorized_keys Chmod644authorized_keys
7.关闭所有机器防火墙、selinux
1）centos7以下
防火墙
查看防火墙状态
serviceiptablesstatus
临时关闭防火墙
serviceiptablesstop
永久关闭防火墙
chkconfigiptablesoff
selinux
setenforce0#临时关闭
永久关闭需要修改如下文件
vim/etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled
2)centos7
firewall-cmd--reload#重启firewall
systemctlstopfirewalld.service#停止firewall
systemctldisablefirewalld.service#禁止firewall开机启动
firewall-cmd--state#查看默认防火墙状态（关闭后显示notrunning，开启后显示running）
selinux同上
8.修改hadoop配置文件
Core-site.xmlhadoop-env.sh添加java_home exportJAVA_HOME=/usr/local/jdk hdfs-site.xml hadoop.tmp.dir /home/hadoop/tmp Abaseforothertemporarydirectories. fs.default.name hdfs://web-crawler--1.novalocal:9000 mapred-site.xml dfs.replication 2 dfs.name.dir /home/hadoop/name dfs.data.dir /home/hadoop/data mapred.job.tracker http://web-crawler--1.novalocal:9001 ;;;mapred.system.dir /home/hadoop/mapred/system mapred.local.dir /home/hadoop/mapred/local
5)masters
web-crawler--1.novalocal
6)slaves
web-crawler--1.novalocal web-crawler--2.novalocal web-crawler--3.novalocal
9.修改环境变量，添加HADOOP_HOME和JAVA_HOME
Vim/etc/profile(root)
exportANT_HOME=/usr/local/ant
exportPATH=${PATH}:${ANT_HOME}/bin
exportHADOOP_HOME=/usr/local/hadoop-1.2.1
exportJAVA_HOME=/usr/local/jdk
exportPATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH
exportCLASSPATH=$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH
exportPATH=/usr/local/lib/phantomjs/bin:$PATH
exportANT_HOME=/usr/local/ant
exportPATH=${PATH}:${ANT_HOME}/bin
exportPATH=$PATH:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin
10.启动hadoop
1）在master上formatnamenode
hadoopnamenode-format
2)启动hadoop
sh/user/local/hadoop/bin/start-all.sh
11.进行简单测试
Hadoopfs-mkdir-p/user/hadoop/private/in Hadoopfs-put/user/local/hadoop/bin/start-all.sh/user/hadoop/private/in Cd/usr/local/hadoop Hadoopjarhadoop-examples-1.2.1.jarwordcount/user/hadoop/private/in/user/hadoop/private/out
总结
以上所述是小编给大家介绍的centos6.5安装hadoop1.2.1的教程详解，希望对大家有所帮助，如果大家有任何疑问请给我留言，小编会及时回复大家的。在此也非常感谢大家对毛票票网站的支持！