官方文档地址/common/docs/r1.0.3/
/
下载到:jdk-6u26-linux-x64.bin andhadoop-1.0.3.tar.gz
它有三种模式:
Local (Standalone) Mode #本地节点
Pseudo-Distributed Mode #伪分布式
Fully-Distributed Mode#全分布式
首先用单节点做一种伪分布式的架构
chmod+xjdk-6u26-linux-x64.bin./jdk-6u26-linux-x64.binmvjdk1.6.0_26//usr/local/jdkvim.bash_profilePATH=$PATH:$HOME/bin:/usr/local/jdk/binsource.bash_profile
useraddyejkpasswd yejkcd /home/yejkvim.bash_profilePATH=$PATH:$HOME/bin:/usr/local/jdk/binsource.bash_profilecphadoop-1.0.3.tar.gz/home/yejk/su-yejktarzxfhadoop-1.0.3.tar.gzcdhadoop-1.0.3修改一些配置文件
vimconf/hadoop-env.sh#Thejavaimplementationtouse.Required.exportJAVA_HOME=/usr/local/jdk
vimconf/core-site.xml:<configuration><property><name>fs.default.name</name><value>hdfs://localhost:9000</value></property></configuration>vimconf/hdfs-site.xml:<configuration><property><name>dfs.replication</name><value>1</value></property></configuration>vimconf/mapred-site.xml:<configuration><property><name>mapred.job.tracker</name><value>localhost:9001</value></property></configuration>
建立ssh无密码访问
ssh-keygen#一路回车ssh-copy-id-i~/.ssh/id_rsa.publocalhost
格式化一个新的dfs文件系统:
bin/hadoopnamenode-format***************2/06/0307:04:49INFOcommon.Storage:Storagedirectory/tmp/hadoop-yejk/dfs/namehasbeensuccessfullyformatted.*****************
启动hadoop:
bin/start-all.sh
NameNode : http://localhost:50070/
JobTracker :http://localhost:50030/
在文件系统中新建一个目录
bin/hadoopfs-mkdirtest
将conf文件中的数据复制上传到刚刚建立的文件夹中:
bin/hadoopfs-putconftest
[yejk@server95hadoop-1.0.3]$bin/hadoopfs-duFound1items54816hdfs://localhost:9000/user/yejk/test[yejk@server95hadoop-1.0.3]$bin/hadoopfs-lsFound1itemsdrwxr-xr-x-yejksupergroup0-06-0307:19/user/yejk/test
用自带的一个程序进行测试
bin/hadoopjarhadoop-examples-1.0.3.jargreptest/*output'dfs[a-z.]+'
意为使用这个java程序从上传到dfs里的test文件夹里的所有数据中搜索以dfs开头的关键字并统计排序,并把结果保存在output中
产看结果:
[yejk@server95hadoop-1.0.3]$bin/hadoopfs-catoutput/*2dfs.replication2dfs.server.namenode.2dfsadmincat:Filedoesnotexist:/user/yejk/output/_logs
或者可以:
bin/hadoopfs-getoutputoutput[yejk@server95output]$catpart-000002dfs.replication2dfs.server.namenode.2dfsadmin