wind520

hive加载大文件(3G)

1：上传文件到hadoop

[jifeng@jifeng01 ~]$ ls -l
总计 3835464
-rw-r--r--  1 jifeng jifeng    7118627 2012-02-01 access_log.txt
drwxrwxr-x  6 jifeng jifeng       4096 04-30 11:14 apache-ant-1.9.4
-rw-r--r--  1 jifeng jifeng    5564593 08-05 16:59 apache-ant-1.9.4-bin.tar.gz
drwxr-xr-x  2 jifeng jifeng       4096 2014-02-26 Desktop
-rw-r--r--  1 jifeng jifeng 3415284428 09-11 16:51 firewallb-week2-origin.sql

[jifeng@jifeng01 ~]$ hadoop fs -mkdir /user/jifeng/fire
Warning: $HADOOP_HOME is deprecated.

[jifeng@jifeng01 ~]$ hadoop fs -put /home/jifeng/firewallb-week2-origin.sql /user/jifeng/fire/
Warning: $HADOOP_HOME is deprecated.

2：hive总创建外部表

hive> CREATE EXTERNAL TABLE firewall (
    >   idauto INT,
    >   dt TIMESTAMP,
    >   dtstr STRING,
    >   timeSeconds INT,
    >   priority STRING,
    >   operation STRING,
    >   messageCode STRING,
    >   protocol STRING,
    >   srcIp STRING,
    >   destIp STRING,
    >   srcIPNumber TINYINT,
    >   destIPNumber TINYINT,
    >   srcPort INT,
    >   destPort INT,
    >   destService STRING,
    >   direction STRING,
    >   flags STRING,
    >   command STRING,
    >   slot1min INT,
    >   slot5min INT,
    >   slot30min INT,
    >   slot60min INT 
    > ) row format delimited fields terminated by '\t';
OK
Time taken: 0.405 seconds

CREATE EXTERNAL TABLE firewall (  
       idauto INT,  
       dt TIMESTAMP,  
       dtstr STRING,  
       timeSeconds INT,  
       priority STRING,  
       operation STRING,  
       messageCode STRING,  
       protocol STRING,  
       srcIp STRING,  
       destIp STRING,  
       srcIPNumber bigINT,  
       destIPNumber bigINT,  
       srcPort INT,  
       destPort INT,  
       destService STRING,  
       direction STRING,  
       flags STRING,  
       command STRING,  
       slot1min INT,  
       slot5min INT,  
       slot30min INT,  
       slot60min INT   
     ) row format delimited fields terminated by '\t';

3：加载hdfs文件

hive> load data inpath '/user/jifeng/fire/firewallb-week2-origin.sql' into table firewall; 
Loading data to table default.firewall
Table default.firewall stats: [num_partitions: 0, num_files: 1, num_rows: 0, total_size: 3415284428, raw_data_size: 0]
OK
Time taken: 0.574 seconds

4：统计IP出现次数

hive> select srcIp,count(*) as num  from firewall  group by srcIp order by  num desc ; 
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 4
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201409072150_0030, Tracking URL = http://jifeng01:50030/jobdetails.jsp?jobid=job_201409072150_0030
Kill Command = /home/jifeng/hadoop/hadoop-1.2.1/libexec/../bin/hadoop job  -kill job_201409072150_0030
Hadoop job information for Stage-1: number of mappers: 13; number of reducers: 4
2014-09-11 18:00:17,094 Stage-1 map = 0%,  reduce = 0%
2014-09-11 18:00:28,139 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 3.22 sec
2014-09-11 18:00:29,145 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 3.22 sec
2014-09-11 18:00:30,148 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 3.22 sec
2014-09-11 18:00:31,158 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 3.22 sec
2014-09-11 18:00:32,162 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 3.22 sec
2014-09-11 18:00:33,165 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 3.22 sec
2014-09-11 18:00:34,168 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 3.22 sec
2014-09-11 18:00:35,171 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 3.22 sec
2014-09-11 18:00:36,176 Stage-1 map = 8%,  reduce = 1%, Cumulative CPU 3.22 sec
2014-09-11 18:00:37,179 Stage-1 map = 10%,  reduce = 1%, Cumulative CPU 3.22 sec
2014-09-11 18:00:38,182 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:39,185 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:40,190 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:41,193 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:42,197 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:43,201 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:44,203 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:45,209 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:46,212 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:47,215 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:48,218 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:49,221 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:50,225 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:51,229 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:52,233 Stage-1 map = 10%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:53,236 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:54,240 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:55,243 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:56,247 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:57,251 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 3.22 sec
2014-09-11 18:00:58,256 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:00:59,260 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:00,265 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:01,271 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:02,275 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:03,279 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:04,286 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:05,289 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:06,292 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:07,295 Stage-1 map = 13%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:08,298 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:09,302 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:10,306 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:11,308 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:12,312 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:13,317 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:14,321 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:15,324 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:16,327 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:17,331 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:18,335 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:19,340 Stage-1 map = 15%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:20,343 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:21,348 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:22,351 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:23,354 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:24,359 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 12.05 sec
2014-09-11 18:01:25,363 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 17.21 sec
2014-09-11 18:01:26,368 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 17.21 sec
2014-09-11 18:01:27,371 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 17.21 sec
2014-09-11 18:01:28,374 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 17.21 sec
2014-09-11 18:01:29,378 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 17.21 sec
2014-09-11 18:01:30,381 Stage-1 map = 17%,  reduce = 2%, Cumulative CPU 17.21 sec
2014-09-11 18:01:31,383 Stage-1 map = 17%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:32,388 Stage-1 map = 17%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:33,391 Stage-1 map = 19%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:34,394 Stage-1 map = 19%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:35,398 Stage-1 map = 21%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:36,401 Stage-1 map = 21%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:37,405 Stage-1 map = 21%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:38,408 Stage-1 map = 21%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:39,410 Stage-1 map = 21%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:40,413 Stage-1 map = 21%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:41,416 Stage-1 map = 23%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:42,419 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:43,422 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:44,424 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:45,429 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:46,432 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:47,436 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:48,438 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:49,442 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:50,446 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:51,448 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:52,453 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:53,456 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:54,460 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:55,462 Stage-1 map = 25%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:56,465 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:57,468 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 17.21 sec
2014-09-11 18:01:58,472 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:01:59,476 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:00,479 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:01,481 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:02,485 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:03,487 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:04,490 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:05,492 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:06,495 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:07,498 Stage-1 map = 27%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:08,502 Stage-1 map = 29%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:09,506 Stage-1 map = 29%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:10,510 Stage-1 map = 29%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:11,513 Stage-1 map = 29%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:12,517 Stage-1 map = 33%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:13,519 Stage-1 map = 33%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:14,522 Stage-1 map = 33%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:15,525 Stage-1 map = 33%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:16,529 Stage-1 map = 33%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:17,531 Stage-1 map = 33%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:18,534 Stage-1 map = 35%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:19,537 Stage-1 map = 35%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:20,540 Stage-1 map = 35%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:21,543 Stage-1 map = 37%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:22,548 Stage-1 map = 37%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:23,553 Stage-1 map = 37%,  reduce = 3%, Cumulative CPU 25.15 sec
2014-09-11 18:02:24,556 Stage-1 map = 37%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:25,559 Stage-1 map = 37%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:26,562 Stage-1 map = 37%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:27,565 Stage-1 map = 37%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:28,568 Stage-1 map = 37%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:29,571 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:30,573 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:31,576 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:32,579 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:33,583 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:34,585 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:35,588 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:36,591 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:37,594 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:38,596 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:39,600 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:40,602 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:41,605 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:42,607 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:43,610 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:44,613 Stage-1 map = 38%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:45,615 Stage-1 map = 40%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:46,619 Stage-1 map = 40%,  reduce = 3%, Cumulative CPU 28.38 sec
2014-09-11 18:02:47,622 Stage-1 map = 42%,  reduce = 3%, Cumulative CPU 29.53 sec
2014-09-11 18:02:48,626 Stage-1 map = 42%,  reduce = 3%, Cumulative CPU 29.53 sec
2014-09-11 18:02:49,629 Stage-1 map = 42%,  reduce = 3%, Cumulative CPU 29.53 sec
2014-09-11 18:02:50,631 Stage-1 map = 42%,  reduce = 3%, Cumulative CPU 29.53 sec
2014-09-11 18:02:51,633 Stage-1 map = 44%,  reduce = 4%, Cumulative CPU 29.53 sec
2014-09-11 18:02:52,637 Stage-1 map = 44%,  reduce = 4%, Cumulative CPU 29.53 sec
2014-09-11 18:02:53,641 Stage-1 map = 46%,  reduce = 4%, Cumulative CPU 30.81 sec
2014-09-11 18:02:54,643 Stage-1 map = 48%,  reduce = 4%, Cumulative CPU 33.52 sec
2014-09-11 18:02:55,646 Stage-1 map = 48%,  reduce = 4%, Cumulative CPU 33.52 sec
2014-09-11 18:02:56,650 Stage-1 map = 48%,  reduce = 4%, Cumulative CPU 33.52 sec
2014-09-11 18:02:57,653 Stage-1 map = 48%,  reduce = 5%, Cumulative CPU 33.52 sec
2014-09-11 18:02:58,656 Stage-1 map = 48%,  reduce = 5%, Cumulative CPU 36.95 sec
2014-09-11 18:02:59,659 Stage-1 map = 48%,  reduce = 6%, Cumulative CPU 36.95 sec
2014-09-11 18:03:00,663 Stage-1 map = 50%,  reduce = 7%, Cumulative CPU 37.23 sec
2014-09-11 18:03:01,667 Stage-1 map = 50%,  reduce = 7%, Cumulative CPU 37.23 sec
2014-09-11 18:03:02,671 Stage-1 map = 50%,  reduce = 7%, Cumulative CPU 37.23 sec
2014-09-11 18:03:03,674 Stage-1 map = 50%,  reduce = 11%, Cumulative CPU 37.23 sec
2014-09-11 18:03:04,678 Stage-1 map = 50%,  reduce = 11%, Cumulative CPU 37.23 sec
2014-09-11 18:03:05,681 Stage-1 map = 50%,  reduce = 13%, Cumulative CPU 37.23 sec
2014-09-11 18:03:06,683 Stage-1 map = 50%,  reduce = 13%, Cumulative CPU 37.23 sec
2014-09-11 18:03:07,688 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 38.38 sec
2014-09-11 18:03:08,692 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 38.38 sec
2014-09-11 18:03:09,695 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 38.38 sec
2014-09-11 18:03:10,700 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 38.38 sec
2014-09-11 18:03:11,703 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 38.38 sec
2014-09-11 18:03:12,709 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 38.38 sec
2014-09-11 18:03:13,712 Stage-1 map = 54%,  reduce = 13%, Cumulative CPU 38.38 sec
2014-09-11 18:03:14,714 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:15,717 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:16,719 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:17,722 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:18,725 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:19,728 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:20,730 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:21,735 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:22,739 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:23,741 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 38.38 sec
2014-09-11 18:03:24,744 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:25,746 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:26,748 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:27,752 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:28,754 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:29,758 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:30,761 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:31,764 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:32,767 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:33,774 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:34,777 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:35,780 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:36,782 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:37,785 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:38,788 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:39,790 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:40,793 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:41,796 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 41.66 sec
2014-09-11 18:03:42,800 Stage-1 map = 60%,  reduce = 15%, Cumulative CPU 42.04 sec
2014-09-11 18:03:43,804 Stage-1 map = 63%,  reduce = 15%, Cumulative CPU 42.04 sec
2014-09-11 18:03:44,808 Stage-1 map = 63%,  reduce = 15%, Cumulative CPU 42.04 sec
2014-09-11 18:03:45,811 Stage-1 map = 63%,  reduce = 15%, Cumulative CPU 42.04 sec
2014-09-11 18:03:46,815 Stage-1 map = 65%,  reduce = 15%, Cumulative CPU 42.04 sec
2014-09-11 18:03:47,818 Stage-1 map = 65%,  reduce = 15%, Cumulative CPU 42.04 sec
2014-09-11 18:03:48,821 Stage-1 map = 65%,  reduce = 16%, Cumulative CPU 42.04 sec
2014-09-11 18:03:49,823 Stage-1 map = 65%,  reduce = 16%, Cumulative CPU 42.04 sec
2014-09-11 18:03:50,826 Stage-1 map = 65%,  reduce = 17%, Cumulative CPU 42.04 sec
2014-09-11 18:03:51,829 Stage-1 map = 65%,  reduce = 18%, Cumulative CPU 42.04 sec
2014-09-11 18:03:52,831 Stage-1 map = 65%,  reduce = 18%, Cumulative CPU 42.04 sec
2014-09-11 18:03:53,834 Stage-1 map = 65%,  reduce = 18%, Cumulative CPU 42.04 sec
2014-09-11 18:03:54,837 Stage-1 map = 65%,  reduce = 18%, Cumulative CPU 42.04 sec
2014-09-11 18:03:55,840 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 42.04 sec
2014-09-11 18:03:56,843 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 42.04 sec
2014-09-11 18:03:57,846 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 42.04 sec
2014-09-11 18:03:58,850 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:03:59,854 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:00,859 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:01,863 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:02,867 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:03,870 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:04,872 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:05,877 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:06,881 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:07,884 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:08,888 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:09,891 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:10,894 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:11,897 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:12,899 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:13,903 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:14,908 Stage-1 map = 71%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:15,911 Stage-1 map = 71%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:16,915 Stage-1 map = 71%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:17,917 Stage-1 map = 71%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:18,920 Stage-1 map = 71%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:19,923 Stage-1 map = 73%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:20,925 Stage-1 map = 73%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:21,928 Stage-1 map = 75%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:22,931 Stage-1 map = 75%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:23,934 Stage-1 map = 75%,  reduce = 18%, Cumulative CPU 52.31 sec
2014-09-11 18:04:24,937 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:25,940 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:26,943 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:27,948 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:28,952 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:29,955 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:30,957 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:31,960 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:32,964 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:33,967 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:34,970 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:35,972 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:36,976 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:37,978 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:38,982 Stage-1 map = 77%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:39,985 Stage-1 map = 79%,  reduce = 18%, Cumulative CPU 56.59 sec
2014-09-11 18:04:40,988 Stage-1 map = 81%,  reduce = 18%, Cumulative CPU 57.94 sec
2014-09-11 18:04:41,995 Stage-1 map = 81%,  reduce = 18%, Cumulative CPU 57.94 sec
2014-09-11 18:04:42,998 Stage-1 map = 81%,  reduce = 18%, Cumulative CPU 57.94 sec
2014-09-11 18:04:44,000 Stage-1 map = 81%,  reduce = 19%, Cumulative CPU 57.94 sec
2014-09-11 18:04:45,003 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:46,005 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:47,010 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:48,014 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:49,017 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:50,019 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:51,022 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:52,025 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:53,028 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:54,030 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:55,033 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:56,036 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:57,039 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:58,043 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 57.94 sec
2014-09-11 18:04:59,047 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:00,050 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:01,052 Stage-1 map = 85%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:02,055 Stage-1 map = 85%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:03,058 Stage-1 map = 85%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:04,062 Stage-1 map = 85%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:05,065 Stage-1 map = 85%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:06,069 Stage-1 map = 85%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:07,073 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:08,075 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:09,079 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:10,082 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:11,085 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:12,088 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:13,090 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:14,092 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:15,095 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:16,099 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:17,102 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:18,105 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:19,108 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:20,110 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:21,114 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:22,117 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:23,119 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:24,122 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 64.35 sec
2014-09-11 18:05:25,124 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 67.35 sec
2014-09-11 18:05:26,127 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 67.35 sec
2014-09-11 18:05:27,130 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 67.35 sec
2014-09-11 18:05:28,133 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 67.35 sec
2014-09-11 18:05:29,137 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 67.35 sec
2014-09-11 18:05:30,140 Stage-1 map = 91%,  reduce = 21%, Cumulative CPU 67.35 sec
2014-09-11 18:05:31,142 Stage-1 map = 92%,  reduce = 21%, Cumulative CPU 68.52 sec
2014-09-11 18:05:32,146 Stage-1 map = 92%,  reduce = 21%, Cumulative CPU 68.52 sec
2014-09-11 18:05:33,149 Stage-1 map = 92%,  reduce = 21%, Cumulative CPU 68.52 sec
2014-09-11 18:05:34,152 Stage-1 map = 92%,  reduce = 21%, Cumulative CPU 68.52 sec
2014-09-11 18:05:35,157 Stage-1 map = 92%,  reduce = 21%, Cumulative CPU 68.52 sec
2014-09-11 18:05:36,160 Stage-1 map = 94%,  reduce = 21%, Cumulative CPU 68.94 sec
2014-09-11 18:05:37,164 Stage-1 map = 94%,  reduce = 21%, Cumulative CPU 68.94 sec
2014-09-11 18:05:38,167 Stage-1 map = 94%,  reduce = 22%, Cumulative CPU 68.94 sec
2014-09-11 18:05:39,170 Stage-1 map = 96%,  reduce = 23%, Cumulative CPU 70.0 sec
2014-09-11 18:05:40,172 Stage-1 map = 96%,  reduce = 23%, Cumulative CPU 70.0 sec
2014-09-11 18:05:41,175 Stage-1 map = 96%,  reduce = 23%, Cumulative CPU 70.0 sec
2014-09-11 18:05:42,178 Stage-1 map = 96%,  reduce = 23%, Cumulative CPU 70.0 sec
2014-09-11 18:05:43,181 Stage-1 map = 96%,  reduce = 23%, Cumulative CPU 70.0 sec
2014-09-11 18:05:44,185 Stage-1 map = 98%,  reduce = 26%, Cumulative CPU 70.73 sec
2014-09-11 18:05:45,188 Stage-1 map = 98%,  reduce = 29%, Cumulative CPU 70.73 sec
2014-09-11 18:05:46,190 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 71.43 sec
2014-09-11 18:05:47,193 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 71.43 sec
2014-09-11 18:05:48,196 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 71.43 sec
2014-09-11 18:05:49,199 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 71.43 sec
2014-09-11 18:05:50,203 Stage-1 map = 100%,  reduce = 65%, Cumulative CPU 72.67 sec
2014-09-11 18:05:51,209 Stage-1 map = 100%,  reduce = 65%, Cumulative CPU 73.35 sec
2014-09-11 18:05:52,212 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 73.35 sec
2014-09-11 18:05:53,214 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 73.35 sec
2014-09-11 18:05:54,217 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 74.94 sec
2014-09-11 18:05:55,221 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 74.94 sec
2014-09-11 18:05:56,225 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 74.94 sec
2014-09-11 18:05:57,229 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 74.94 sec
MapReduce Total cumulative CPU time: 1 minutes 14 seconds 940 msec
Ended Job = job_201409072150_0030
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201409072150_0031, Tracking URL = http://jifeng01:50030/jobdetails.jsp?jobid=job_201409072150_0031
Kill Command = /home/jifeng/hadoop/hadoop-1.2.1/libexec/../bin/hadoop job  -kill job_201409072150_0031
Hadoop job information for Stage-2: number of mappers: 3; number of reducers: 1
2014-09-11 18:05:59,481 Stage-2 map = 0%,  reduce = 0%
2014-09-11 18:06:01,490 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 0.36 sec
2014-09-11 18:06:02,494 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 0.36 sec
2014-09-11 18:06:03,498 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 0.36 sec
2014-09-11 18:06:04,500 Stage-2 map = 67%,  reduce = 0%, Cumulative CPU 0.71 sec
2014-09-11 18:06:05,503 Stage-2 map = 67%,  reduce = 0%, Cumulative CPU 0.71 sec
2014-09-11 18:06:06,506 Stage-2 map = 67%,  reduce = 0%, Cumulative CPU 0.71 sec
2014-09-11 18:06:07,509 Stage-2 map = 67%,  reduce = 0%, Cumulative CPU 0.71 sec
2014-09-11 18:06:08,512 Stage-2 map = 67%,  reduce = 22%, Cumulative CPU 0.71 sec
2014-09-11 18:06:09,515 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 1.19 sec
2014-09-11 18:06:10,518 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 1.19 sec
2014-09-11 18:06:11,520 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 1.19 sec
2014-09-11 18:06:12,524 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 1.19 sec
2014-09-11 18:06:13,527 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 1.19 sec
2014-09-11 18:06:14,531 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 1.81 sec
2014-09-11 18:06:15,534 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 1.81 sec
2014-09-11 18:06:16,538 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 1.81 sec
MapReduce Total cumulative CPU time: 1 seconds 810 msec
Ended Job = job_201409072150_0031
MapReduce Jobs Launched: 
Job 0: Map: 13  Reduce: 4   Cumulative CPU: 74.94 sec   HDFS Read: 3415495752 HDFS Write: 45773 SUCCESS
Job 1: Map: 3  Reduce: 1   Cumulative CPU: 1.81 sec   HDFS Read: 47452 HDFS Write: 23372 SUCCESS
Total MapReduce CPU Time Spent: 1 minutes 16 seconds 750 msec
OK
10.12.15.152    3875019
10.13.77.49     1954742
10.6.6.7        1769857
10.138.235.111  1630920
10.12.14.15     756475
10.17.15.10     720936
172.10.0.4      482568
10.78.100.150   300530
10.247.58.182   292384
10.38.217.48    282588
172.30.0.4      232850
10.10.11.102    205579
10.170.32.110   202735
10.170.32.181   200539
10.138.214.18   197864
10.247.106.27   193912
10.156.165.120  183929
10.15.7.85      172422
10.0.0.42       165063
10.200.20.2     164519
10.70.68.127    161735
172.20.0.15     145924
10.250.178.101  140053
172.20.0.4      122768
172.30.1.223    51329
172.20.1.81     51328
172.10.2.66     51304
172.20.1.47     51298
172.30.1.218    51236

Time taken: 366.904 seconds, Fetched: 1389 row(s)

hive> desc firewall;
OK
idauto                  int                     None                
dt                      timestamp               None                
dtstr                   string                  None                
timeseconds             int                     None                
priority                string                  None                
operation               string                  None                
messagecode             string                  None                
protocol                string                  None                
srcip                   string                  None                
destip                  string                  None                
srcipnumber             tinyint                 None                
destipnumber            tinyint                 None                
srcport                 int                     None                
destport                int                     None                
destservice             string                  None                
direction               string                  None                
flags                   string                  None                
command                 string                  None                
slot1min                int                     None                
slot5min                int                     None                
slot30min               int                     None                
slot60min               int                     None                
Time taken: 0.106 seconds, Fetched: 22 row(s)
hive> select * from firewall order by dt desc limit 10;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201409072150_0032, Tracking URL = http://jifeng01:50030/jobdetails.jsp?jobid=job_201409072150_0032
Kill Command = /home/jifeng/hadoop/hadoop-1.2.1/libexec/../bin/hadoop job  -kill job_201409072150_0032
Hadoop job information for Stage-1: number of mappers: 13; number of reducers: 1
2014-09-11 18:53:36,120 Stage-1 map = 0%,  reduce = 0%
2014-09-11 18:54:03,210 Stage-1 map = 8%,  reduce = 0%
2014-09-11 18:54:05,217 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 19.33 sec
2014-09-11 18:54:06,221 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 19.33 sec
2014-09-11 18:54:07,264 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:08,267 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:09,270 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:10,272 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:11,277 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:12,282 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:13,286 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:14,289 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:15,294 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:16,299 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:17,301 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:18,304 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:19,307 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:20,312 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:21,315 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:22,320 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:23,325 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:24,329 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:25,331 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:26,335 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:27,337 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:28,341 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:29,344 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:30,444 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 40.04 sec
2014-09-11 18:54:31,448 Stage-1 map = 20%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:32,451 Stage-1 map = 24%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:33,455 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:34,458 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:35,461 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:36,464 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:37,469 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:38,473 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:39,476 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:40,480 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:41,482 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:42,486 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 69.96 sec
2014-09-11 18:54:43,490 Stage-1 map = 30%,  reduce = 0%, Cumulative CPU 75.79 sec
2014-09-11 18:54:44,493 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:45,496 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:46,501 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:47,506 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:48,509 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:49,513 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:50,517 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:51,521 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:52,525 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:53,530 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:54,533 Stage-1 map = 38%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:55,536 Stage-1 map = 38%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:56,539 Stage-1 map = 38%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:57,543 Stage-1 map = 38%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:58,546 Stage-1 map = 38%,  reduce = 0%, Cumulative CPU 81.82 sec
2014-09-11 18:54:59,551 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 104.83 sec
2014-09-11 18:55:00,559 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 104.83 sec
2014-09-11 18:55:01,563 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 104.83 sec
2014-09-11 18:55:02,568 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 104.83 sec
2014-09-11 18:55:03,572 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 104.83 sec
2014-09-11 18:55:04,578 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 104.83 sec
2014-09-11 18:55:05,582 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 148.55 sec
2014-09-11 18:55:06,586 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 148.55 sec
2014-09-11 18:55:07,591 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 171.88 sec
2014-09-11 18:55:08,594 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 171.88 sec
2014-09-11 18:55:09,598 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 171.88 sec
2014-09-11 18:55:10,601 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 171.88 sec
2014-09-11 18:55:11,605 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 171.88 sec
2014-09-11 18:55:12,608 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 171.88 sec
2014-09-11 18:55:13,610 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 172.61 sec
2014-09-11 18:55:14,617 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 172.61 sec
2014-09-11 18:55:15,620 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 172.61 sec
2014-09-11 18:55:16,625 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 172.61 sec
2014-09-11 18:55:17,627 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 172.61 sec
2014-09-11 18:55:18,632 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 172.61 sec
2014-09-11 18:55:19,636 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:20,639 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:21,643 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:22,646 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:23,652 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:24,656 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:25,660 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:26,664 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:27,667 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:28,671 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 173.86 sec
2014-09-11 18:55:29,675 Stage-1 map = 50%,  reduce = 13%, Cumulative CPU 173.86 sec
2014-09-11 18:55:30,678 Stage-1 map = 50%,  reduce = 13%, Cumulative CPU 173.86 sec
2014-09-11 18:55:31,684 Stage-1 map = 50%,  reduce = 13%, Cumulative CPU 194.76 sec
2014-09-11 18:55:32,689 Stage-1 map = 50%,  reduce = 13%, Cumulative CPU 194.76 sec
2014-09-11 18:55:33,693 Stage-1 map = 50%,  reduce = 13%, Cumulative CPU 194.76 sec
2014-09-11 18:55:34,696 Stage-1 map = 50%,  reduce = 13%, Cumulative CPU 194.76 sec
2014-09-11 18:55:35,700 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 201.47 sec
2014-09-11 18:55:36,703 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 201.47 sec
2014-09-11 18:55:37,707 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 201.47 sec
2014-09-11 18:55:38,851 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 201.47 sec
2014-09-11 18:55:39,854 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 201.47 sec
2014-09-11 18:55:40,859 Stage-1 map = 52%,  reduce = 13%, Cumulative CPU 201.47 sec
2014-09-11 18:55:41,862 Stage-1 map = 52%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:42,866 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:43,870 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:44,873 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:45,878 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:46,884 Stage-1 map = 54%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:47,888 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:48,894 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:49,898 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:50,902 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:51,905 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:52,908 Stage-1 map = 56%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:53,912 Stage-1 map = 58%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:54,915 Stage-1 map = 58%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:55,919 Stage-1 map = 60%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:56,923 Stage-1 map = 60%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:57,927 Stage-1 map = 60%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:58,929 Stage-1 map = 60%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:55:59,933 Stage-1 map = 62%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:56:00,937 Stage-1 map = 62%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:56:01,939 Stage-1 map = 62%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:56:02,943 Stage-1 map = 62%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:56:03,947 Stage-1 map = 62%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:56:04,952 Stage-1 map = 62%,  reduce = 15%, Cumulative CPU 201.47 sec
2014-09-11 18:56:05,955 Stage-1 map = 65%,  reduce = 15%, Cumulative CPU 233.97 sec
2014-09-11 18:56:06,958 Stage-1 map = 65%,  reduce = 15%, Cumulative CPU 255.02 sec
2014-09-11 18:56:07,960 Stage-1 map = 65%,  reduce = 15%, Cumulative CPU 255.02 sec
2014-09-11 18:56:08,965 Stage-1 map = 65%,  reduce = 15%, Cumulative CPU 255.02 sec
2014-09-11 18:56:09,969 Stage-1 map = 65%,  reduce = 15%, Cumulative CPU 255.02 sec
2014-09-11 18:56:10,973 Stage-1 map = 65%,  reduce = 15%, Cumulative CPU 255.02 sec
2014-09-11 18:56:11,976 Stage-1 map = 65%,  reduce = 18%, Cumulative CPU 255.02 sec
2014-09-11 18:56:12,979 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 255.02 sec
2014-09-11 18:56:13,984 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 255.02 sec
2014-09-11 18:56:14,987 Stage-1 map = 67%,  reduce = 18%, Cumulative CPU 255.02 sec
2014-09-11 18:56:15,991 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 255.02 sec
2014-09-11 18:56:16,995 Stage-1 map = 69%,  reduce = 18%, Cumulative CPU 255.02 sec
2014-09-11 18:56:18,000 Stage-1 map = 71%,  reduce = 18%, Cumulative CPU 267.57 sec
2014-09-11 18:56:19,003 Stage-1 map = 71%,  reduce = 18%, Cumulative CPU 267.57 sec
2014-09-11 18:56:20,007 Stage-1 map = 71%,  reduce = 18%, Cumulative CPU 267.57 sec
2014-09-11 18:56:21,011 Stage-1 map = 71%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:22,014 Stage-1 map = 71%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:23,022 Stage-1 map = 71%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:24,025 Stage-1 map = 71%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:25,028 Stage-1 map = 71%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:26,033 Stage-1 map = 71%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:27,037 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:28,041 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:29,047 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:30,050 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:31,053 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 267.57 sec
2014-09-11 18:56:32,056 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:33,060 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:34,064 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:35,067 Stage-1 map = 73%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:36,071 Stage-1 map = 75%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:37,074 Stage-1 map = 75%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:38,077 Stage-1 map = 77%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:39,080 Stage-1 map = 77%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:40,084 Stage-1 map = 79%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:41,087 Stage-1 map = 79%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:42,092 Stage-1 map = 79%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:43,095 Stage-1 map = 79%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:44,099 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:45,102 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:46,105 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:47,109 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:48,113 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:49,117 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:50,120 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:51,124 Stage-1 map = 81%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:52,129 Stage-1 map = 83%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:53,136 Stage-1 map = 85%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:54,139 Stage-1 map = 87%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:55,144 Stage-1 map = 87%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:56,149 Stage-1 map = 87%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:57,152 Stage-1 map = 87%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:58,155 Stage-1 map = 87%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:56:59,164 Stage-1 map = 87%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:57:00,169 Stage-1 map = 89%,  reduce = 21%, Cumulative CPU 272.61 sec
2014-09-11 18:57:01,171 Stage-1 map = 90%,  reduce = 21%, Cumulative CPU 288.97 sec
2014-09-11 18:57:02,175 Stage-1 map = 90%,  reduce = 21%, Cumulative CPU 288.97 sec
2014-09-11 18:57:03,178 Stage-1 map = 90%,  reduce = 21%, Cumulative CPU 288.97 sec
2014-09-11 18:57:04,181 Stage-1 map = 90%,  reduce = 21%, Cumulative CPU 288.97 sec
2014-09-11 18:57:05,184 Stage-1 map = 90%,  reduce = 21%, Cumulative CPU 288.97 sec
2014-09-11 18:57:06,188 Stage-1 map = 90%,  reduce = 23%, Cumulative CPU 323.8 sec
2014-09-11 18:57:07,192 Stage-1 map = 90%,  reduce = 23%, Cumulative CPU 345.22 sec
2014-09-11 18:57:08,195 Stage-1 map = 92%,  reduce = 23%, Cumulative CPU 345.22 sec
2014-09-11 18:57:09,200 Stage-1 map = 92%,  reduce = 23%, Cumulative CPU 345.22 sec
2014-09-11 18:57:10,204 Stage-1 map = 92%,  reduce = 23%, Cumulative CPU 345.22 sec
2014-09-11 18:57:11,209 Stage-1 map = 92%,  reduce = 23%, Cumulative CPU 345.22 sec
2014-09-11 18:57:12,213 Stage-1 map = 94%,  reduce = 23%, Cumulative CPU 349.49 sec
2014-09-11 18:57:13,217 Stage-1 map = 94%,  reduce = 23%, Cumulative CPU 349.49 sec
2014-09-11 18:57:14,220 Stage-1 map = 94%,  reduce = 23%, Cumulative CPU 349.49 sec
2014-09-11 18:57:15,224 Stage-1 map = 94%,  reduce = 23%, Cumulative CPU 349.49 sec
2014-09-11 18:57:16,227 Stage-1 map = 98%,  reduce = 23%, Cumulative CPU 372.21 sec
2014-09-11 18:57:17,230 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 378.56 sec
2014-09-11 18:57:18,233 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 378.56 sec
2014-09-11 18:57:19,236 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 378.56 sec
2014-09-11 18:57:20,240 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 378.56 sec
2014-09-11 18:57:21,245 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 378.56 sec
2014-09-11 18:57:22,248 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 378.56 sec
2014-09-11 18:57:23,251 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 378.56 sec
2014-09-11 18:57:24,255 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 380.17 sec
2014-09-11 18:57:25,258 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 380.17 sec
2014-09-11 18:57:26,261 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 380.17 sec
2014-09-11 18:57:27,264 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 380.17 sec
MapReduce Total cumulative CPU time: 6 minutes 20 seconds 170 msec
Ended Job = job_201409072150_0032
MapReduce Jobs Launched: 
Job 0: Map: 13  Reduce: 1   Cumulative CPU: 380.17 sec   HDFS Read: 3415495752 HDFS Write: 1935 SUCCESS
Total MapReduce CPU Time Spent: 6 minutes 20 seconds 170 msec
OK
16600911        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.13.77.4910.0.3.2        NULL    NULL    58200   1993    1993_tcp        (empty) (empty) (empty) 22767000        4553400 758900  379450
16600930        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.13.77.4910.0.3.2        NULL    NULL    58200   13817   13817_tcp       (empty) (empty) (empty) 22767000        4553400 758900  379450
16600920        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Info    Teardown        ASA-6-302014    TCP     10.6.6.7    172.10.0.2      NULL    NULL    42706   3389    3389_tcp        inbound SYN Timeout     (empty) 22767000        4553400 758900      379450
16600913        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.13.77.4910.0.3.2        NULL    NULL    58199   32179   32179_tcp       (empty) (empty) (empty) 22767000        4553400 758900  379450
16600912        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.6.6.710.0.2.2        NULL    NULL    42465   556     remotefs        (empty) (empty) (empty) 22767000        4553400 758900  379450
16600918        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.13.77.4910.0.3.2        NULL    NULL    58199   25087   25087_tcp       (empty) (empty) (empty) 22767000        4553400 758900  379450
16600919        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.6.6.710.0.2.2        NULL    NULL    42465   38300   38300_tcp       (empty) (empty) (empty) 22767000        4553400 758900  379450
16600931        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.6.6.710.0.2.2        NULL    NULL    42464   33004   33004_tcp       (empty) (empty) (empty) 22767000        4553400 758900  379450
16600915        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.13.77.4910.0.3.2        NULL    NULL    58200   32179   32179_tcp       (empty) (empty) (empty) 22767000        4553400 758900  379450
16600916        2013-04-15 10:00:00     15/Apr/2013 10:00:00    1366020000      Warning Deny    ASA-4-106023    TCP     10.6.6.710.0.2.2        NULL    NULL    42465   52372   52372_tcp       (empty) (empty) (empty) 22767000        4553400 758900  379450
Time taken: 233.592 seconds, Fetched: 10 row(s)
hive>

你可能感兴趣的:(hadoop,hive)

nosql数据库技术与应用知识点皆过客，揽星河 NoSQL nosql 数据库大数据数据分析数据结构非关系型数据库
Nosql知识回顾大数据处理流程数据采集(flume、爬虫、传感器)数据存储(本门课程NoSQL所处的阶段)Hdfs、MongoDB、HBase等数据清洗(入仓)Hive等数据处理、分析(Spark、Flink等)数据可视化数据挖掘、机器学习应用(Python、SparkMLlib等)大数据时代存储的挑战(三高)高并发(同一时间很多人访问)高扩展(要求随时根据需求扩展存储)高效率(要求读写速度快)
浅谈MapReduce Android路上的人 Hadoop 分布式计算 mapreduce 分布式框架 hadoop
从今天开始，本人将会开始对另一项技术的学习，就是当下炙手可热的Hadoop分布式就算技术。目前国内外的诸多公司因为业务发展的需要，都纷纷用了此平台。国内的比如BAT啦，国外的在这方面走的更加的前面，就不一一列举了。但是Hadoop作为Apache的一个开源项目，在下面有非常多的子项目，比如HDFS，HBase,Hive，Pig,等等，要先彻底学习整个Hadoop，仅仅凭借一个的力量，是远远不够的。
Hadoop 傲雪凌霜，松柏长青后端大数据 hadoop 大数据分布式
ApacheHadoop是一个开源的分布式计算框架，主要用于处理海量数据集。它具有高度的可扩展性、容错性和高效的分布式存储与计算能力。Hadoop核心由四个主要模块组成，分别是HDFS（分布式文件系统）、MapReduce（分布式计算框架）、YARN（资源管理）和HadoopCommon（公共工具和库）。1.HDFS（HadoopDistributedFileSystem）HDFS是Hadoop生
Hadoop架构 henan程序媛 hadoop 大数据分布式
一、案列分析1.1案例概述现在已经进入了大数据(BigData)时代，数以万计用户的互联网服务时时刻刻都在产生大量的交互，要处理的数据量实在是太大了，以传统的数据库技术等其他手段根本无法应对数据处理的实时性、有效性的需求。HDFS顺应时代出现，在解决大数据存储和计算方面有很多的优势。1.2案列前置知识点1.什么是大数据大数据是指无法在一定时间范围内用常规软件工具进行捕捉、管理和处理的大量数据集合，
分享一个基于python的电子书数据采集与可视化分析 hadoop电子书数据分析与推荐系统 spark大数据毕设项目（源码、调试、LW、开题、PPT) 计算机源码社 Python项目大数据大数据 python hadoop 计算机毕业设计选题计算机毕业设计源码数据分析 spark毕设
作者：计算机源码社个人简介：本人八年开发经验，擅长Java、Python、PHP、.NET、Node.js、Android、微信小程序、爬虫、大数据、机器学习等，大家有这一块的问题可以一起交流！学习资料、程序开发、技术解答、文档报告如需要源码，可以扫取文章下方二维码联系咨询Java项目微信小程序项目Android项目Python项目PHP项目ASP.NET项目Node.js项目选题推荐项目实战|p
Presto【基础 01】简介+架构+数据源+数据模型 2401_84254343 程序员架构
一个Catalog包含Schema和Connector。例如，配置JMX的Catalog，通过JXMConnector访问JXM信息。当执行一条SQL语句时，可以同时运行在多个Catalog。Presto处理table时，是通过表的完全限定（fully-qualified）名来找到Catalog。例如，一个表的权限定名是hive.test_data.test，则test是表名，test_data是
hbase介绍 CrazyL- 云计算+大数据 hbase
hbase是一个分布式的、多版本的、面向列的开源数据库hbase利用hadoophdfs作为其文件存储系统，提供高可靠性、高性能、列存储、可伸缩、实时读写、适用于非结构化数据存储的数据库系统hbase利用hadoopmapreduce来处理hbase、中的海量数据hbase利用zookeeper作为分布式系统服务特点：数据量大：一个表可以有上亿行，上百万列（列多时，插入变慢）面向列：面向列（族）的
大数据毕业设计hadoop+spark+hive知识图谱租房数据分析可视化大屏租房推荐系统 58同城租房爬虫房源推荐系统房价预测系统计算机毕业设计机器学习深度学习人工智能 2401_84572577 程序员大数据 hadoop 人工智能
做了那么多年开发，自学了很多门编程语言，我很明白学习资源对于学一门新语言的重要性，这些年也收藏了不少的Python干货，对我来说这些东西确实已经用不到了，但对于准备自学Python的人来说，或许它就是一个宝藏，可以给你省去很多的时间和精力。别在网上瞎学了，我最近也做了一些资源的更新，只要你是我的粉丝，这期福利你都可拿走。我先来介绍一下这些东西怎么用，文末抱走。（1）Python所有方向的学习路线（
Spark集群的三种模式 MelodyYN #Spark spark hadoop big data
文章目录1、Spark的由来1.1Hadoop的发展1.2MapReduce与Spark对比2、Spark内置模块3、Spark运行模式3.1Standalone模式部署配置历史服务器配置高可用运行模式3.2Yarn模式安装部署配置历史服务器运行模式4、WordCount案例1、Spark的由来定义：Hadoop主要解决，海量数据的存储和海量数据的分析计算。Spark是一种基于内存的快速、通用、可
月度总结 | 2022年03月 | 考研与就业的抉择 | 确定未来走大数据开发路线「已注销」个人总结 hadoop
一、时间线梳理3月3日，寻找到同专业的就业伙伴3月5日，着手准备Java八股文，决定先走Java后端路线3月8月，申请到了校图书馆的考研专座，决定暂时放弃就业，先准备考研，买了数学和408的资料书3月9日-3月13日，因疫情原因，宿舍区暂封，这段时间在准备考研，发现内容特别多3月13日-3月19日，大部分时间在刷Hadoop、Zookeeper、Kafka的视频，同时在准备实习的项目3月20日，退
HBase介绍 mingyu1016 数据库
概述HBase是一个分布式的、面向列的开源数据库,源于google的一篇论文《bigtable：一个结构化数据的分布式存储系统》。HBase是GoogleBigtable的开源实现，它利用HadoopHDFS作为其文件存储系统，利用HadoopMapReduce来处理HBase中的海量数据，利用Zookeeper作为协同服务。HBase的表结构HBase以表的形式存储数据。表有行和列组成。列划分为
大数据之flink与hive 星辰_mya 大数据 flink hive
其实吧我不太想写flink，因为线上经验确实不多，这也是我需要补的地方，没有条件创造条件，先来一篇吧flink：高性能低延迟流批一体的分布式计算框架基于事件时间对实时数据精准处理快速响应支持批处理，高效离线分析和数据挖掘数据仓库的引擎丰富数据源/接收器，集成多种数据存储格式和源，比较常见就是咱们今天的主题hive了checkpoint恢复机制，故障恢复快速恢复计算任务分布式弹性扩展，据业务灵活增加
Java中的大数据处理框架对比分析省赚客app开发者 java 开发语言
Java中的大数据处理框架对比分析大家好，我是微赚淘客系统3.0的小编，是个冬天不穿秋裤，天冷也要风度的程序猿！今天，我们将深入探讨Java中常用的大数据处理框架，并对它们进行对比分析。大数据处理框架是现代数据驱动应用的核心，它们帮助企业处理和分析海量数据，以提取有价值的信息。本文将重点介绍ApacheHadoop、ApacheSpark、ApacheFlink和ApacheStorm这四种流行的
hive血缘关系之输入表与目标表的解析 zxfBdd hive 大数据治理大数据
接了一个新需求：需要做数据仓库的血缘关系。正所谓兵来将挡水来土掩，那咱就动手吧。血缘关系是数据治理的一块，其实有专门的第三方数据治理框架，但考虑到目前的线上环境已经趋于稳定，引入新的框架无疑是劳民伤财，伤筋动骨，所以就想以最小的代价把这个事情给做了。目前我们考虑做的血缘关系呢只是做输入表和输出表，最后会形成一张表与表之间的链路图。这个东西的好处就是有助于仓库人员梳理业务，后面可能还会做字段之间的血
Hadoop windows intelij 跑 MR WordCount piziyang12138
一、软件环境我使用的软件版本如下:IntellijIdea2017.1Maven3.3.9Hadoop分布式环境二、创建maven工程打开Idea,file->new->Project,左侧面板选择maven工程。(如果只跑MapReduce创建java工程即可，不用勾选Creatfromarchetype，如果想创建web工程或者使用骨架可以勾选)image.png设置GroupId和Artif
初级练习[3]:Hive SQL子查询应用大数据深度洞察 Hive hive sql hadoop 数据仓库大数据数据库
目录环境准备看如下链接子查询查询所有课程成绩均小于60分的学生的学号、姓名查询没有学全所有课的学生的学号、姓名解释：没有学全所有课，也就是该学生选修的课程数<总的课程数。查询出只选修了三门课程的全部学生的学号和姓名环境准备看如下链接环境准备https://blog.csdn.net/qq_45115959/article/details/142057624?spm=1001.2014.3001.5
Linux下载压缩包：tar.gz、zip、tar.bz2格式全攻略 promise524 Linux linux 运维服务器后端 bash shell
在Linux中，下载各种格式的压缩包（如.tar.gz、.zip、.tar.bz2等）通常使用命令行工具如wget和curl。1.使用wget下载压缩包wget是Linux中最常用的文件下载工具，支持HTTP、HTTPS、FTP等协议，可以直接从命令行下载文件。基本命令：wget[URL]下载.tar.gz文件wgethttps://test.com/archive.tar.gz此命令将从指定的U
Anaconda版本和Python版本对应关系纬领网络 python anaconda3
官网下载地址：https://repo.anaconda.com/archive/下载地址：https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/anaconda3版本基础python版本Anaconda3-2024.06-1Python3.12.4Anaconda3-2024.02-1Python3.11.7Anaconda3-2023.09
Hadoop学习第三课（HDFS架构--读、写流程）小小程序员呀~ 数据库 hadoop 架构 big data
1.块概念举例1：一桶水1000ml，瓶子的规格100ml=>需要10个瓶子装完一桶水1010ml，瓶子的规格100ml=>需要11个瓶子装完一桶水1010ml，瓶子的规格200ml=>需要6个瓶子装完块的大小规格，只要是需要存储，哪怕一点点，也是要占用一个块的块大小的参数：dfs.blocksize官方默认的大小为128M官网：https://hadoop.apache.org/docs/r3.
hadoop启动HDFS命令 m0_67401228 java 搜索引擎 linux 后端
启动命令：/hadoop/sbin/start-dfs.sh停止命令：/hadoop/sbin/stop-dfs.sh
R语言包AMORE安装报错问题以及RStudio与Rtools环境配置卡卡_R-Python R语言数据分析与可视化 r语言开发语言
在使用R语言进行AMORE安装时会遇到报错，这时候需要采用解决办法：'''AMORE包安装，需要离线官网下载安装包：Indexof/src/contrib/Archive/AMORE(r-project.org)https://cran.r-project.org/src/contrib/Archive/AMORE/一、出现的问题最近开始学习R语言，安装了最新版的R4.4.1和RStudio，但安
中级练习[3]：Hive SQL用户行为与商品销售数据分析大数据深度洞察 Hive hive 数据仓库大数据 sql
目录1.用户累计消费金额及VIP等级查询1.1题目需求1.2代码实现2.首次下单后第二天连续下单的用户比率查询2.1题目需求2.2代码实现3.每个商品销售首年的年份、销售数量和销售金额统计3.1题目需求3.2代码实现1.用户累计消费金额及VIP等级查询1.1题目需求从订单信息表(order_info)中统计每个用户截止其每个下单日期的累积消费金额，以及每个用户在其每个下单日期的VIP等级。VIP等
Python基础知识进阶之正则表达式_头歌python正则表达式进阶前端陈萨龙程序员 python 学习面试
最后硬核资料：关注即可领取PPT模板、简历模板、行业经典书籍PDF。技术互助：技术群大佬指点迷津，你的问题可能不是问题，求资源在群里喊一声。面试题库：由技术群里的小伙伴们共同投稿，热乎的大厂面试真题，持续更新中。知识体系：含编程语言、算法、大数据生态圈组件（Mysql、Hive、Spark、Flink）、数据仓库、Python、前端等等。网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是
【计算机毕设-大数据方向】基于Hadoop的电商交易数据分析可视化系统的设计与实现程序员-石头山大数据实战案例大数据 hadoop 毕业设计毕设
博主介绍：✌全平台粉丝5W+,高级大厂开发程序员，博客之星、掘金/知乎/华为云/阿里云等平台优质作者。【源码获取】关注并且私信我【联系方式】最下边感兴趣的可以先收藏起来，同学门有不懂的毕设选题，项目以及论文编写等相关问题都可以和学长沟通，希望帮助更多同学解决问题前言随着电子商务行业的迅猛发展，电商平台积累了海量的数据资源，这些数据不仅包括用户的基本信息、购物记录，还包括用户的浏览行为、评价反馈等多
分布式离线计算—Spark—基础介绍测试开发abbey 人工智能—大数据
原文作者：饥渴的小苹果原文地址：【Spark】Spark基础教程目录Spark特点Spark相对于Hadoop的优势Spark生态系统Spark基本概念Spark结构设计Spark各种概念之间的关系Executor的优点Spark运行基本流程Spark运行架构的特点Spark的部署模式Spark三种部署方式Hadoop和Spark的统一部署摘要：Spark是基于内存计算的大数据并行计算框架Spar
spark常用命令我是浣熊的微笑 spark
查看报错日志：yarnlogsapplicationIDspark2-submit--masteryarn--classcom.hik.ReadHdfstest-1.0-SNAPSHOT.jar进入$SPARK_HOME目录，输入bin/spark-submit--help可以得到该命令的使用帮助。hadoop@wyy:/app/hadoop/spark100$bin/spark-submit--
spark启动命令学不会又听不懂 spark 大数据分布式
hadoop启动：cd/root/toolssstart-dfs.sh，只需在hadoop01上启动stop-dfs.sh日志查看：cat/root/toolss/hadoop/logs/hadoop-root-datanode-hadoop03.outzookeeper启动：cd/root/toolss/zookeeperbin/zkServer.shstart，三台都要启动bin/zkServ
编程常用命令总结 Yellow0523 Linux BigData 大数据
编程命令大全1.软件环境变量的配置JavaScalaSparkHadoopHive2.大数据软件常用命令Spark基本命令Spark-SQL命令Hive命令HDFS命令YARN命令Zookeeper命令kafka命令Hibench命令MySQL命令3.Linux常用命令Git命令conda命令pip命令查看Linux系统的详细信息查看Linux系统架构(X86还是ARM，两种方法都可)端口号命令L
博客园怎么了？ YYH1992
新年好，给大家拜个早年！今年来到安徽过年，无聊中，不知不觉中又来到博客园了（忠实粉丝哦），却发现一件奇怪的事情，请看截图难道博客园被挂马了？抑或其它问题？如果真有问题，还请dudu抓紧时间修正，免得影响我们园子的声誉！我要下线了，出去买回家的车票了，只能年后回家了。。。转载于:https://www.cnblogs.com/HollisYao/archive/2008/02/06/1065351.
Hadoop常见面试题整理及解答叶青舟 Linux hdfs 大数据 hadoop linux
Hadoop常见面试题整理及解答一、基础知识篇：1.把数据仓库从传统关系型数据库转到hadoop有什么优势？答：（1）关系型数据库成本高，且存储空间有限。而Hadoop使用较为廉价的机器存储数据，且Hadoop可以将大量机器构建成一个集群，并在集群中使用HDFS文件系统统一管理数据，极大的提高了数据的存储及处理能力。（2）关系型数据库仅支持标准结构化数据格式，Hadoop不仅支持标准结构化数据格式
项目中枚举与注解的结合使用飞翔的马甲 java enum annotation
前言：版本兼容，一直是迭代开发头疼的事，最近新版本加上了支持新题型，如果新创建一份问卷包含了新题型，那旧版本客户端就不支持，如果新创建的问卷不包含新题型，那么新旧客户端都支持。这里面我们通过给问卷类型枚举增加自定义注解的方式完成。顺便巩固下枚举与注解。一、枚举 1.在创建枚举类的时候，该类已继承java.lang.Enum类，所以自定义枚举类无法继承别的类，但可以实现接口。
【Scala十七】Scala核心十一：下划线_的用法 bit1129 scala
下划线_在Scala中广泛应用，_的基本含义是作为占位符使用。_在使用时是出问题非常多的地方，本文将不断完善_的使用场景以及所表达的含义 1. 在高阶函数中使用 scala> val list = List(-3,8,7,9) list: List[Int] = List(-3, 8, 7, 9) scala> list.filter(_ > 7) r
web缓存基础：术语、http报头和缓存策略 dalan_123 Web
对于很多人来说，去访问某一个站点，若是该站点能够提供智能化的内容缓存来提高用户体验，那么最终该站点的访问者将络绎不绝。缓存或者对之前的请求临时存储，是http协议实现中最核心的内容分发策略之一。分发路径中的组件均可以缓存内容来加速后续的请求，这是受控于对该内容所声明的缓存策略。接下来将讨web内容缓存策略的基本概念，具体包括如如何选择缓存策略以保证互联网范围内的缓存能够正确处理的您的内容，并谈论下
crontab 问题周凡杨 linux crontab unix
一： 0481-079 Reached a symbol that is not expected. 背景： */5 * * * * /usr/IBMIHS/rsync.sh
让tomcat支持2级域名共享session g21121 session
tomcat默认情况下是不支持2级域名共享session的，所有有些情况下登陆后从主域名跳转到子域名会发生链接session不相同的情况，但是只需修改几处配置就可以了。打开tomcat下conf下context.xml文件找到Context标签,修改为如下内容如果你的域名是www.test.com <Context sessionCookiePath="/path&q
web报表工具FineReport常用函数的用法总结（数学和三角函数）老A不折腾 Web finereport 总结
ABS ABS(number):返回指定数字的绝对值。绝对值是指没有正负符号的数值。 Number:需要求出绝对值的任意实数。示例: ABS(-1.5)等于1.5。 ABS(0)等于0。 ABS(2.5)等于2.5。 ACOS ACOS(number):返回指定数值的反余弦值。反余弦值为一个角度，返回角度以弧度形式表示。 Number:需要返回角
linux 启动java进程 sh文件墙头上一根草 linux shell jar
#!/bin/bash #初始化服务器的进程PId变量 user_pid=0; robot_pid=0; loadlort_pid=0; gateway_pid=0; ######### #检查相关服务器是否启动成功 #说明： #使用JDK自带的JPS命令及grep命令组合，准确查找pid #jps 加 l 参数，表示显示java的完整包路径 #使用awk，分割出pid
我的spring学习笔记5-如何使用ApplicationContext替换BeanFactory aijuans Spring 3 系列
如何使用ApplicationContext替换BeanFactory？ package onlyfun.caterpillar.device; import org.springframework.beans.factory.BeanFactory; import org.springframework.beans.factory.xml.XmlBeanFactory; import
Linux 内存使用方法详细解析 annan211 linux 内存 Linux内存解析
来源 http://blog.jobbole.com/45748/ 我是一名程序员，那么我在这里以一个程序员的角度来讲解Linux内存的使用。一提到内存管理，我们头脑中闪出的两个概念，就是虚拟内存，与物理内存。这两个概念主要来自于linux内核的支持。 Linux在内存管理上份为两级，一级是线性区，类似于00c73000-00c88000，对应于虚拟内存，它实际上不占用
数据库的单表查询常用命令及使用方法(-) 百合不是茶 oracle 函数单表查询
创建数据库; --建表 create table bloguser(username varchar2(20),userage number(10),usersex char(2)); 创建bloguser表,里面有三个字段 &nbs
多线程基础知识 bijian1013 java 多线程 thread java多线程
一．进程和线程进程就是一个在内存中独立运行的程序，有自己的地址空间。如正在运行的写字板程序就是一个进程。 “多任务”：指操作系统能同时运行多个进程（程序）。如WINDOWS系统可以同时运行写字板程序、画图程序、WORD、Eclipse等。线程：是进程内部单一的一个顺序控制流。线程和进程 a. 每个进程都有独立的
fastjson简单使用实例 bijian1013 fastjson
一.简介阿里巴巴fastjson是一个Java语言编写的高性能功能完善的JSON库。它采用一种“假定有序快速匹配”的算法，把JSON Parse的性能提升到极致，是目前Java语言中最快的JSON库；包括“序列化”和“反序列化”两部分，它具备如下特征：
【RPC框架Burlap】Spring集成Burlap bit1129 spring
Burlap和Hessian同属于codehaus的RPC调用框架，但是Burlap已经几年不更新，所以Spring在4.0里已经将Burlap的支持置为Deprecated,所以在选择RPC框架时，不应该考虑Burlap了。这篇文章还是记录下Burlap的用法吧，主要是复制粘贴了Hessian与Spring集成一文，【RPC框架Hessian四】Hessian与Spring集成
【Mahout一】基于Mahout 命令参数含义 bit1129 Mahout
1. mahout seqdirectory $ mahout seqdirectory --input (-i) input Path to job input directory(原始文本文件). --output (-o) output The directory pathna
linux使用flock文件锁解决脚本重复执行问题 ronin47 linux lock　重复执行
linux的crontab命令，可以定时执行操作，最小周期是每分钟执行一次。关于crontab实现每秒执行可参考我之前的文章《linux crontab 实现每秒执行》现在有个问题，如果设定了任务每分钟执行一次，但有可能一分钟内任务并没有执行完成，这时系统会再执行任务。导致两个相同的任务在执行。例如： <? // test .php
java-74-数组中有一个数字出现的次数超过了数组长度的一半，找出这个数字 bylijinnan java
public class OcuppyMoreThanHalf { /** * Q74 数组中有一个数字出现的次数超过了数组长度的一半，找出这个数字 * two solutions: * 1.O(n) * see <beauty of coding>--每次删除两个不同的数字，不改变数组的特性 * 2.O(nlogn) * 排序。中间
linux 系统相关命令 candiio linux
系统参数 cat /proc/cpuinfo cpu相关参数 cat /proc/meminfo 内存相关参数 cat /proc/loadavg 负载情况性能参数 1）top M：按内存使用排序 P：按CPU占用排序 1：显示各CPU的使用情况 k：kill进程 o：更多排序规则回车：刷新数据 2）ulimit ulimit -a：显示本用户的系统限制参
[经营与资产]保持独立性和稳定性对于软件开发的重要意义 comsci 软件开发
一个软件的架构从诞生到成熟，中间要经过很多次的修正和改造如果在这个过程中，外界的其它行业的资本不断的介入这种软件架构的升级过程中那么软件开发者原有的设计思想和开发路线
在CentOS5.5上编译OpenJDK6 Cwind linux OpenJDK
几番周折终于在自己的CentOS5.5上编译成功了OpenJDK6，将编译过程和遇到的问题作一简要记录，备查。 0. OpenJDK介绍 OpenJDK是Sun（现Oracle）公司发布的基于GPL许可的Java平台的实现。其优点： 1、它的核心代码与同时期Sun（-> Oracle）的产品版基本上是一样的，血统纯正，不用担心性能问题，也基本上没什么兼容性问题；（代码上最主要的差异是
java乱码问题 dashuaifu java乱码问题 js中文乱码
swfupload上传文件参数值为中文传递到后台接收中文乱码在js中用setPostParams（{"tag" : encodeURI( document.getElementByIdx_x("filetag").value，"utf-8")}）; 然后在servlet中String t
cygwin很多命令显示command not found的解决办法 dcj3sjt126com cygwin
cygwin很多命令显示command not found的解决办法修改cygwin.BAT文件如下 @echo off D: set CYGWIN=tty notitle glob set PATH=%PATH%;d:\cygwin\bin;d:\cygwin\sbin;d:\cygwin\usr\bin;d:\cygwin\usr\sbin;d:\cygwin\us
[介绍]从 Yii 1.1 升级 dcj3sjt126com PHP yii2
2.0 版框架是完全重写的，在 1.1 和 2.0 两个版本之间存在相当多差异。因此从 1.1 版升级并不像小版本间的跨越那么简单，通过本指南你将会了解两个版本间主要的不同之处。如果你之前没有用过 Yii 1.1，可以跳过本章，直接从"入门篇"开始读起。请注意，Yii 2.0 引入了很多本章并没有涉及到的新功能。强烈建议你通读整部权威指南来了解所有新特性。这样有可能会发
Linux SSH免登录配置总结 eksliang ssh-keygen Linux SSH免登录认证 Linux SSH互信
转载请出自出处：http://eksliang.iteye.com/blog/2187265 一、原理我们使用ssh-keygen在ServerA上生成私钥跟公钥，将生成的公钥拷贝到远程机器ServerB上后,就可以使用ssh命令无需密码登录到另外一台机器ServerB上。生成公钥与私钥有两种加密方式，第一种是
手势滑动销毁Activity gundumw100 android
老是效仿ios，做android的真悲催！有需求：需要手势滑动销毁一个Activity 怎么办尼？自己写？不用~，网上先问一下百度。结果： http://blog.csdn.net/xiaanming/article/details/20934541 首先将你需要的Activity继承SwipeBackActivity，它会在你的布局根目录新增一层SwipeBackLay
JavaScript变换表格边框颜色 ini JavaScript html Web html5 css
效果查看：http://hovertree.com/texiao/js/2.htm代码如下，保存到HTML文件也可以查看效果： <html> <head> <meta charset="utf-8"> <title>表格边框变换颜色代码-何问起</title> </head> <body&
Kafka Rest : Confluent kane_xie kafka REST confluent
最近拿到一个kafka rest的需求，但kafka暂时还没有提供rest api（应该是有在开发中，毕竟rest这么火），上网搜了一下，找到一个Confluent Platform，本文简单介绍一下安装。这里插一句，给大家推荐一个九尾搜索，原名叫谷粉SOSO，不想fanqiang谷歌的可以用这个。以前在外企用谷歌用习惯了，出来之后用度娘搜技术问题，那匹配度简直感人。环境声明：Ubu
Calender不是单例 men4661273 单例 Calender
在我们使用Calender的时候，使用过Calendar.getInstance()来获取一个日期类的对象，这种方式跟单例的获取方式一样，那么它到底是不是单例呢，如果是单例的话，一个对象修改内容之后，另外一个线程中的数据不久乱套了吗？从试验以及源码中可以得出，Calendar不是单例。测试： Calendar c1 =
线程内存和主内存之间联系 qifeifei java thread
1， java多线程共享主内存中变量的时候，一共会经过几个阶段， lock:将主内存中的变量锁定，为一个线程所独占。 unclock:将lock加的锁定解除，此时其它的线程可以有机会访问此变量。 read:将主内存中的变量值读到工作内存当中。 load:将read读取的值保存到工作内存中的变量副本中。
schedule和scheduleAtFixedRate tangqi609567707 java timer schedule
原文地址：http://blog.csdn.net/weidan1121/article/details/527307 import java.util.Timer;import java.util.TimerTask;import java.util.Date; /** * @author vincent */public class TimerTest {
erlang 部署 wudixiaotie erlang
1.如果在启动节点的时候报这个错： {"init terminating in do_boot",{'cannot load',elf_format,get_files}} 则需要在reltool.config中加入 {app, hipe, [{incl_cond, exclude}]}, 2.当generate时，遇到： ERROR