hadoopd的secondarynamenode的使用

本文参考:http://www.cnblogs.com/ggjucheng/archive/2012/04/18/2454693.html,本文使用的是伪分布式的hadoop集群,版本为1.0.4,主要的配置文件如下:

core-site.xml:

<!-- Put site-specific property overrides in this file. -->
<configuration>
	<property>
		<name>hadoop.tmp.dir</name>
		<value>/home/hadoop/fs/tmp</value>
	</property>
	<property>
		<name>fs.default.name</name>
		<value>hdfs://hadoop:9000</value>
	</property>
	<property>
		<name>fs.checkpoint.period</name>
		<value>300</value>
	</property>
	<property>
		<name>fs.checkpoint.size</name>
		<value>67108864</value>
	</property>
	<property>
		<name>fs.checkpoint.dir</name>
		<value>/home/hadoop/checkpoint/namesecodary</value>
	</property>
</configuration>

hdfs-site.xml:

<!-- Put site-specific property overrides in this file. -->
<configuration>
	<property>
		<name>dfs.replicatioin</name>
		<value>1</value>
	</property>
	<property>
		<name>dfs.data.dir</name>
		<value>/home/hadoop/fs/data</value>
	</property>
	<property>
		<name>dfs.name.dir</name>
		<value>/home/hadoop/fs/name</value>
	</property>
	<property>
		<name>dfs.back.http.address</name>
		<value>hadoop:50070</value>
	</property>
	<property>
		<name>dfs.permissions</name>
		<value>false</value>
	</property>
</configuration>

mapred-site.xml:

<!-- Put site-specific property overrides in this file. -->
<configuration>
	<property>
		<name>mapred.job.tracker</name>
		<value>hadoop:9001</value>
	</property>
</configuration>

1.首先输入jps,查看hadoop进程,如下:

hadoop@hadoop:~/hadoop-1.0.4/bin$ jps
16018 TaskTracker
15646 DataNode
17250 NameNode
15870 JobTracker
17296 Jps
15790 SecondaryNameNode

2.kill掉namenode的进程,继续查看hadoop进程,如下:

hadoop@hadoop:~/hadoop-1.0.4/bin$ kill 17250
hadoop@hadoop:~/hadoop-1.0.4/bin$ jps
16018 TaskTracker
15646 DataNode
17328 Jps
15870 JobTracker
15790 SecondaryNameNode

3. 删除dfs.name.dir的目录的全部内容;

4. 这里没有按照参考的博客那样把secondarynamenode的fs.checkpoint.dir的内容拷贝到namenode的dfs.name.dir下面,而是直接使用./hadoop namenode -importCheckPoint命令,这个命令应该已经含有了拷贝的命令了,出现下面的logs信息:

hadoop@hadoop:~/hadoop-1.0.4/bin$ ./hadoop namenode -importCheckPoint
13/05/28 15:37:14 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = hadoop/192.168.128.138
STARTUP_MSG:   args = [-importCheckPoint]
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
13/05/28 15:37:14 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
13/05/28 15:37:14 INFO impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
13/05/28 15:37:14 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
13/05/28 15:37:14 INFO impl.MetricsSystemImpl: NameNode metrics system started
13/05/28 15:37:14 INFO impl.MetricsSourceAdapter: MBean for source ugi registered.
13/05/28 15:37:14 WARN impl.MetricsSystemImpl: Source name ugi already exists!
13/05/28 15:37:14 INFO impl.MetricsSourceAdapter: MBean for source jvm registered.
13/05/28 15:37:14 INFO impl.MetricsSourceAdapter: MBean for source NameNode registered.
13/05/28 15:37:14 INFO util.GSet: VM type       = 32-bit
13/05/28 15:37:14 INFO util.GSet: 2% max memory = 19.33375 MB
13/05/28 15:37:14 INFO util.GSet: capacity      = 2^22 = 4194304 entries
13/05/28 15:37:14 INFO util.GSet: recommended=4194304, actual=4194304
13/05/28 15:37:14 INFO namenode.FSNamesystem: fsOwner=hadoop
13/05/28 15:37:14 INFO namenode.FSNamesystem: supergroup=supergroup
13/05/28 15:37:14 INFO namenode.FSNamesystem: isPermissionEnabled=false
13/05/28 15:37:14 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
13/05/28 15:37:14 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
13/05/28 15:37:15 INFO namenode.FSNamesystem: Registered FSNamesystemStateMBean and NameNodeMXBean
13/05/28 15:37:15 INFO namenode.NameNode: Caching file names occuring more than 10 times 
13/05/28 15:37:15 INFO common.Storage: Storage directory /home/hadoop/fs/name is not formatted.
13/05/28 15:37:15 INFO common.Storage: Formatting ...
13/05/28 15:37:15 INFO common.Storage: Number of files = 8
13/05/28 15:37:15 INFO common.Storage: Number of files under construction = 0
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 loaded in 0 seconds.
13/05/28 15:37:15 INFO common.Storage: Edits file /home/hadoop/checkpoint/namesecodary/current/edits of size 4 edits # 0 loaded in 0 seconds.
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 saved in 0 seconds.
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 saved in 0 seconds.
13/05/28 15:37:15 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 saved in 0 seconds.
13/05/28 15:37:15 INFO common.Storage: Image file of size 755 saved in 0 seconds.
13/05/28 15:37:15 INFO namenode.NameCache: initialized with 0 entries 0 lookups
13/05/28 15:37:15 INFO namenode.FSNamesystem: Finished loading FSImage in 345 msecs
13/05/28 15:37:15 INFO hdfs.StateChange: STATE* Safe mode ON. 
The ratio of reported blocks 0.0000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
13/05/28 15:37:15 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
13/05/28 15:37:15 INFO impl.MetricsSourceAdapter: MBean for source FSNamesystemMetrics registered.
13/05/28 15:37:15 INFO impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort9000 registered.
13/05/28 15:37:15 INFO impl.MetricsSourceAdapter: MBean for source RpcActivityForPort9000 registered.
13/05/28 15:37:15 INFO ipc.Server: Starting SocketReader
13/05/28 15:37:15 INFO namenode.NameNode: Namenode up at: hadoop/192.168.128.138:9000
13/05/28 15:37:35 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/05/28 15:37:35 INFO http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
13/05/28 15:37:35 INFO http.HttpServer: dfs.webhdfs.enabled = false
13/05/28 15:37:35 INFO http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50070
13/05/28 15:37:35 INFO http.HttpServer: listener.getLocalPort() returned 50070 webServer.getConnectors()[0].getLocalPort() returned 50070
13/05/28 15:37:35 INFO http.HttpServer: Jetty bound to port 50070
13/05/28 15:37:35 INFO mortbay.log: jetty-6.1.26
13/05/28 15:37:35 INFO mortbay.log: Started [email protected]:50070
13/05/28 15:37:35 INFO namenode.NameNode: Web-server up at: 0.0.0.0:50070
13/05/28 15:37:35 INFO ipc.Server: IPC Server Responder: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server listener on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 0 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 1 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 2 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 3 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 4 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 5 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 6 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 7 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 8 on 9000: starting
13/05/28 15:37:35 INFO ipc.Server: IPC Server handler 9 on 9000: starting
13/05/28 15:37:35 INFO hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.128.138:50010 storage DS-2106362203-192.168.128.138-50010-1368229536640
13/05/28 15:37:35 INFO net.NetworkTopology: Adding a new node: /default-rack/192.168.128.138:50010
13/05/28 15:37:35 INFO hdfs.StateChange: STATE* Safe mode extension entered. 
The ratio of reported blocks 1.0000 has reached the threshold 0.9990. Safe mode will be turned off automatically in 29 seconds.
13/05/28 15:37:35 INFO hdfs.StateChange: *BLOCK* NameSystem.processReport: from 192.168.128.138:50010, blocks: 1, processing time: 10 msecs
13/05/28 15:37:56 INFO hdfs.StateChange: STATE* Safe mode ON. 
The ratio of reported blocks 1.0000 has reached the threshold 0.9990. Safe mode will be turned off automatically in 9 seconds.
13/05/28 15:38:06 INFO namenode.FSNamesystem: Total number of blocks = 1
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of invalid blocks = 0
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of under-replicated blocks = 1
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of  over-replicated blocks = 0
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* Safe mode termination scan for invalid, over- and under-replicated blocks completed in 56 msec
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* Leaving safe mode after 51 secs.
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* Safe mode is OFF.
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* Network topology has 1 racks and 1 datanodes
13/05/28 15:38:06 INFO hdfs.StateChange: STATE* UnderReplicatedBlocks has 1 blocks
13/05/28 15:38:06 INFO namenode.FSNamesystem: ReplicateQueue QueueProcessingStatistics: First cycle completed 0 blocks in 1 msec
13/05/28 15:38:06 INFO namenode.FSNamesystem: ReplicateQueue QueueProcessingStatistics: Queue flush completed 0 blocks in 1 msec processing time, 1 msec clock time, 1 cycles
13/05/28 15:38:06 INFO namenode.FSNamesystem: InvalidateQueue QueueProcessingStatistics: First cycle completed 0 blocks in 0 msec
13/05/28 15:38:06 INFO namenode.FSNamesystem: InvalidateQueue QueueProcessingStatistics: Queue flush completed 0 blocks in 0 msec processing time, 0 msec clock time, 1 cycles
13/05/28 15:38:06 INFO namenode.FSNamesystem: Roll Edit Log from 192.168.128.138
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 
13/05/28 15:38:06 INFO namenode.FSNamesystem: Roll FSImage from 192.168.128.138
13/05/28 15:38:06 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 1 SyncTimes(ms): 5 

5.在kill掉namenode后,50070界面就不能访问了,但是使用了4的命令后,这个界面又可以访问了。但是使用CTRL+c后50070页面又不可以访问了,这时使用命令./hadoop-daemon.sh start namenode 后稍等片刻就又可以访问50070界面了。

你可能感兴趣的:(secondary)