hadoop2单机搭建

Hadoop2.2.0 环境配置说明（多节点配置在最后） 1.关闭防火墙 #chkconfigiptablesoff 2.检查状态 #chkconfig –list|grepiptables 全 off 即可 3.将 hadoop-2.2.0.tar.gz 文件复制到 /usr/local 目录下 4.解压 #tar –zxvfhadoop-2.2.0.tar.gz 5.改名 #mv Ha

Hadoop2.2.0环境配置说明（多节点配置在最后）

1. 关闭防火墙# chkconfig iptables off

2. 检查状态#chkconfig –list|grep iptables 全off即可

3. 将hadoop-2.2.0.tar.gz文件复制到/usr/local目录下

4. 解压# tar –zxvf hadoop-2.2.0.tar.gz

5. 改名 # mv Hadoop-2.2.0 hadoop2.2

6. 修改环境变量 # vim /etc/profile

7. 添加 export HADOOP_HOME=/usr/local/hadoop2.2

export HADOOP_MAPRED_HOME=$HADOOP_HOME

export HADOOP_COMMON_HOME=$HADOOP_HOME

export HADOOP_HDFS_HOME=$HADOOP_HOME

export YARN_HOME=$HADOOP_HOME

export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop

PATH下添加:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

8. 重载# source /etc/profile

9. 目录切换到 # cd /usr/local/hadoop2.2/etc/Hadoop

10. 在如下文件中添加对应内容

11. hadoop-env.sh 第27行修改为

export JAVA_HOME=/usr/local/jdk1.6

12. yarn-env.sh 第23行修改为

export JAVA_HOME=/usr/local/jdk1.6

13. 将mapred-site.xml.template 复制为mapred-site.xml

# cp mapred-site.xml.template mapred-site.xml

14. mapred-site.xml中第20行(configuration)添加

mapreduce.framework.name

yarn

15. yarn-site.xml第18行(configuration)添加

yarn.resourcemanager.hostname

localhost

hostanem of RM

yarn.resourcemanager.resource-tracker.address

localhost:5274

host is the hostname of the resource manager and

port is the port on which the NodeManagers contact the Resource Manager.

yarn.resourcemanager.scheduler.address

localhost:5273

host is the hostname of the resourcemanager and port is the port

on which the Applications in the cluster talk to the Resource Manager.

yarn.resourcemanager.scheduler.class

org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler

In case you do not want to use the default scheduler

yarn.resourcemanager.address

localhost:5271

the host is the hostname of the ResourceManager and the port is the port on which the clients can talk to the Resource Manager.

yarn.nodemanager.local-dirs

the local directories used by the nodemanager

yarn.nodemanager.address

localhost:5272

the nodemanagers bind to this port

yarn.nodemanager.resource.memory-mb

10240

the amount of memory on the NodeManager in GB

yarn.nodemanager.remote-app-log-dir

/app-logs

directory on hdfs where the application logs are moved to

yarn.nodemanager.log-dirs

/usr/log

the directories used by Nodemanagers as log directories

yarn.nodemanager.aux-services

mapreduce_shuffle

shuffle service that needs to be set for Map Reduce to run

16. core-site.xml第20行(configuration)添加

hadoop.tmp.dir

/usr/local/hadoop2.2/tmp

fs.defaultFS

hdfs://localhost:9000

true

17. hdfs-site.xml第20行(configuration)添加

dfs.namenode.name.dir

file:///dfs/name

true

dfs.datanode.data.dir

file:///dfs/data

true

dfs.replication

1

dfs.permissions.enabled

false

hadoop-env.sh里面# export JAVA_HOME=/usr/local/jdk

18. 设置ssh

# ssh-keygen –t rsa 一路回车默认值

进行查看 (应有id_rsa和id_rsa.pub一对密钥文件)

# cd ~/.ssh

# ls

复制出公钥

# cp id_rsa.pub authorized_keys

查看# ls 应有三个文件了

确认过程

# ssh localhost (输入yes)

# exit

# ssh localhost

19. 格式化

# hadoop namenode –format

20. 启动

#start-dfs.sh

#start-yarn.sh

21. 查看 # jps 应有6个

22. 用自带浏览器，能打开http://localhost:50070/ 和http://localhost:8088/cluster即可

（多机环境配置）

23. DNS配置（建议修改之后重启虚拟机）

24. 将主机名换为hadoop2

# vim /etc/sysconfig/network

第2行，localhost.localdomain改为hadoop2

25. 修改地址映射

# vim /etc/hosts

第三行添加192.168.100.11 hadoop2 （地址不固定）