wind520

mahout贝叶斯分类器测试样例

MAHOUT 0.9版本的贝叶斯分类器测试样例测试非常简单，执行下面这条语句，然后选择第2项就OK了。

$MAHOUT_HOME/examples/bin/classify-20newsgroups.sh

[jifeng@jifeng01 hadoop]$ $MAHOUT_HOME/examples/bin/classify-20newsgroups.sh
Please select a number to choose the corresponding task to run
1. cnaivebayes
2. naivebayes
3. sgd
4. clean -- cleans up the work area in /tmp/mahout-work-jifeng
Enter your choice :

选择第二项,等待运行完成：

Enter your choice : 2
ok. You chose 2 and we'll use naivebayes
creating work directory at /tmp/mahout-work-jifeng
Downloading 20news-bydate
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.7M  100 13.7M    0     0  96135      0  0:02:30  0:02:30 --:--:--  103k
Extracting...
+ echo 'Preparing 20newsgroups data'
Preparing 20newsgroups data
+ rm -rf /tmp/mahout-work-jifeng/20news-all
+ mkdir /tmp/mahout-work-jifeng/20news-all
+ cp -R /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/alt.atheism /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/comp.graphics /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/comp.os.ms-windows.misc /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/comp.sys.ibm.pc.hardware /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/comp.sys.mac.hardware /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/comp.windows.x /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/misc.forsale /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/rec.autos /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/rec.motorcycles /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/rec.sport.baseball /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/rec.sport.hockey /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/sci.crypt /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/sci.electronics /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/sci.med /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/sci.space /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/soc.religion.christian /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/talk.politics.guns /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/talk.politics.mideast /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/talk.politics.misc /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-test/talk.religion.misc /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/alt.atheism /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/comp.graphics /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/comp.os.ms-windows.misc /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/comp.sys.ibm.pc.hardware /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/comp.sys.mac.hardware /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/comp.windows.x /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/misc.forsale /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/rec.autos /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/rec.motorcycles /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/rec.sport.baseball /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/rec.sport.hockey /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/sci.crypt /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/sci.electronics /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/sci.med /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/sci.space /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/soc.religion.christian /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/talk.politics.guns /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/talk.politics.mideast /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/talk.politics.misc /tmp/mahout-work-jifeng/20news-bydate/20news-bydate-train/talk.religion.misc /tmp/mahout-work-jifeng/20news-all
+ '[' /home/jifeng/hadoop/hadoop-1.2.1 '!=' '' ']'
+ '[' '' == '' ']'
+ echo 'Copying 20newsgroups data to HDFS'
Copying 20newsgroups data to HDFS
+ set +e
+ /home/jifeng/hadoop/hadoop-1.2.1/bin/hadoop dfs -rmr /tmp/mahout-work-jifeng/20news-all
Warning: $HADOOP_HOME is deprecated.

rmr: cannot remove /tmp/mahout-work-jifeng/20news-all: No such file or directory.
+ set -e
+ /home/jifeng/hadoop/hadoop-1.2.1/bin/hadoop dfs -put /tmp/mahout-work-jifeng/20news-all /tmp/mahout-work-jifeng/20news-all
Warning: $HADOOP_HOME is deprecated.

+ echo 'Creating sequence files from 20newsgroups data'
Creating sequence files from 20newsgroups data
+ ./bin/mahout seqdirectory -i /tmp/mahout-work-jifeng/20news-all -o /tmp/mahout-work-jifeng/20news-seq -ow
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/jifeng/hadoop/hadoop-1.2.1/bin/hadoop and HADOOP_CONF_DIR=/home/jifeng/hadoop/hadoop-1.2.1/conf
MAHOUT-JOB: /home/jifeng/hadoop/mahout-distribution-0.9/mahout-examples-0.9-job.jar
Warning: $HADOOP_HOME is deprecated.

14/09/20 15:35:58 WARN driver.MahoutDriver: No seqdirectory.props found on classpath, will use command-line arguments only
14/09/20 15:35:59 INFO common.AbstractJob: Command line arguments: {--charset=[UTF-8], --chunkSize=[64], --endPhase=[2147483647], --fileFilterClass=[org.apache.mahout.text.PrefixAdditionFilter], --input=[/tmp/mahout-work-jifeng/20news-all], --keyPrefix=[], --method=[mapreduce], --output=[/tmp/mahout-work-jifeng/20news-seq], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
14/09/20 15:36:05 INFO input.FileInputFormat: Total input paths to process : 18846
14/09/20 15:36:06 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/09/20 15:36:06 WARN snappy.LoadSnappy: Snappy native library not loaded
14/09/20 15:36:20 INFO mapred.JobClient: Running job: job_201409201505_0001
14/09/20 15:36:21 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:36:34 INFO mapred.JobClient:  map 10% reduce 0%
14/09/20 15:36:37 INFO mapred.JobClient:  map 17% reduce 0%
14/09/20 15:36:41 INFO mapred.JobClient:  map 20% reduce 0%
14/09/20 15:36:44 INFO mapred.JobClient:  map 24% reduce 0%
14/09/20 15:36:47 INFO mapred.JobClient:  map 29% reduce 0%
14/09/20 15:36:50 INFO mapred.JobClient:  map 33% reduce 0%
14/09/20 15:36:53 INFO mapred.JobClient:  map 38% reduce 0%
14/09/20 15:36:56 INFO mapred.JobClient:  map 43% reduce 0%
14/09/20 15:36:59 INFO mapred.JobClient:  map 49% reduce 0%
14/09/20 15:37:01 INFO mapred.JobClient:  map 55% reduce 0%
14/09/20 15:37:04 INFO mapred.JobClient:  map 59% reduce 0%
14/09/20 15:37:07 INFO mapred.JobClient:  map 66% reduce 0%
14/09/20 15:37:10 INFO mapred.JobClient:  map 73% reduce 0%
14/09/20 15:37:13 INFO mapred.JobClient:  map 81% reduce 0%
14/09/20 15:37:16 INFO mapred.JobClient:  map 92% reduce 0%
14/09/20 15:37:20 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:37:20 INFO mapred.JobClient: Job complete: job_201409201505_0001
14/09/20 15:37:20 INFO mapred.JobClient: Counters: 18
14/09/20 15:37:20 INFO mapred.JobClient:   Job Counters 
14/09/20 15:37:20 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=56433
14/09/20 15:37:20 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:37:20 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:37:20 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:37:20 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
14/09/20 15:37:20 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:37:20 INFO mapred.JobClient:     Bytes Written=19202391
14/09/20 15:37:20 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:37:20 INFO mapred.JobClient:     HDFS_BYTES_READ=37622181
14/09/20 15:37:20 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=59535
14/09/20 15:37:20 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=19202391
14/09/20 15:37:20 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:37:20 INFO mapred.JobClient:     Bytes Read=0
14/09/20 15:37:20 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:37:20 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:37:20 INFO mapred.JobClient:     Physical memory (bytes) snapshot=84213760
14/09/20 15:37:20 INFO mapred.JobClient:     Spilled Records=0
14/09/20 15:37:20 INFO mapred.JobClient:     CPU time spent (ms)=21400
14/09/20 15:37:20 INFO mapred.JobClient:     Total committed heap usage (bytes)=57679872
14/09/20 15:37:20 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=349954048
14/09/20 15:37:20 INFO mapred.JobClient:     Map output records=18846
14/09/20 15:37:20 INFO mapred.JobClient:     SPLIT_RAW_BYTES=1767178
14/09/20 15:37:20 INFO driver.MahoutDriver: Program took 81580 ms (Minutes: 1.3596666666666666)
+ echo 'Converting sequence files to vectors'
Converting sequence files to vectors
+ ./bin/mahout seq2sparse -i /tmp/mahout-work-jifeng/20news-seq -o /tmp/mahout-work-jifeng/20news-vectors -lnorm -nv -wt tfidf
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/jifeng/hadoop/hadoop-1.2.1/bin/hadoop and HADOOP_CONF_DIR=/home/jifeng/hadoop/hadoop-1.2.1/conf
MAHOUT-JOB: /home/jifeng/hadoop/mahout-distribution-0.9/mahout-examples-0.9-job.jar
Warning: $HADOOP_HOME is deprecated.

14/09/20 15:37:23 WARN driver.MahoutDriver: No seq2sparse.props found on classpath, will use command-line arguments only
14/09/20 15:37:23 INFO vectorizer.SparseVectorsFromSequenceFiles: Maximum n-gram size is: 1
14/09/20 15:37:23 INFO vectorizer.SparseVectorsFromSequenceFiles: Minimum LLR value: 1.0
14/09/20 15:37:23 INFO vectorizer.SparseVectorsFromSequenceFiles: Number of reduce tasks: 1
14/09/20 15:37:23 INFO vectorizer.SparseVectorsFromSequenceFiles: Tokenizing documents in /tmp/mahout-work-jifeng/20news-seq
14/09/20 15:37:27 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:37:28 INFO mapred.JobClient: Running job: job_201409201505_0002
14/09/20 15:37:29 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:37:48 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:37:49 INFO mapred.JobClient: Job complete: job_201409201505_0002
14/09/20 15:37:49 INFO mapred.JobClient: Counters: 19
14/09/20 15:37:49 INFO mapred.JobClient:   Job Counters 
14/09/20 15:37:49 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=16049
14/09/20 15:37:49 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:37:49 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:37:49 INFO mapred.JobClient:     Rack-local map tasks=1
14/09/20 15:37:49 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:37:49 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
14/09/20 15:37:49 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:37:49 INFO mapred.JobClient:     Bytes Written=27503580
14/09/20 15:37:49 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:37:49 INFO mapred.JobClient:     HDFS_BYTES_READ=19202523
14/09/20 15:37:49 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=57471
14/09/20 15:37:49 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=27503580
14/09/20 15:37:49 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:37:49 INFO mapred.JobClient:     Bytes Read=19202391
14/09/20 15:37:49 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:37:49 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:37:49 INFO mapred.JobClient:     Physical memory (bytes) snapshot=90681344
14/09/20 15:37:49 INFO mapred.JobClient:     Spilled Records=0
14/09/20 15:37:49 INFO mapred.JobClient:     CPU time spent (ms)=5370
14/09/20 15:37:49 INFO mapred.JobClient:     Total committed heap usage (bytes)=31916032
14/09/20 15:37:49 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=519794688
14/09/20 15:37:49 INFO mapred.JobClient:     Map output records=18846
14/09/20 15:37:49 INFO mapred.JobClient:     SPLIT_RAW_BYTES=132
14/09/20 15:37:49 INFO vectorizer.SparseVectorsFromSequenceFiles: Creating Term Frequency Vectors
14/09/20 15:37:49 INFO vectorizer.DictionaryVectorizer: Creating dictionary from /tmp/mahout-work-jifeng/20news-vectors/tokenized-documents and saving at /tmp/mahout-work-jifeng/20news-vectors/wordcount
14/09/20 15:37:52 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:37:52 INFO mapred.JobClient: Running job: job_201409201505_0003
14/09/20 15:37:53 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:38:08 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:38:15 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:38:16 INFO mapred.JobClient:  map 100% reduce 100%
14/09/20 15:38:17 INFO mapred.JobClient: Job complete: job_201409201505_0003
14/09/20 15:38:17 INFO mapred.JobClient: Counters: 29
14/09/20 15:38:17 INFO mapred.JobClient:   Job Counters 
14/09/20 15:38:17 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:38:17 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=8755
14/09/20 15:38:17 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:38:17 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:38:17 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:38:17 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:38:17 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=8294
14/09/20 15:38:17 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:38:17 INFO mapred.JobClient:     Bytes Written=2315037
14/09/20 15:38:17 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:38:17 INFO mapred.JobClient:     FILE_BYTES_READ=11857906
14/09/20 15:38:17 INFO mapred.JobClient:     HDFS_BYTES_READ=27503736
14/09/20 15:38:17 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=15512128
14/09/20 15:38:17 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=2315037
14/09/20 15:38:17 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:38:17 INFO mapred.JobClient:     Bytes Read=27503580
14/09/20 15:38:17 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:38:17 INFO mapred.JobClient:     Map output materialized bytes=3538084
14/09/20 15:38:17 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:38:17 INFO mapred.JobClient:     Reduce shuffle bytes=3538084
14/09/20 15:38:17 INFO mapred.JobClient:     Spilled Records=849345
14/09/20 15:38:17 INFO mapred.JobClient:     Map output bytes=39462740
14/09/20 15:38:17 INFO mapred.JobClient:     Total committed heap usage (bytes)=219021312
14/09/20 15:38:17 INFO mapred.JobClient:     CPU time spent (ms)=6420
14/09/20 15:38:17 INFO mapred.JobClient:     Combine input records=3026242
14/09/20 15:38:17 INFO mapred.JobClient:     SPLIT_RAW_BYTES=156
14/09/20 15:38:17 INFO mapred.JobClient:     Reduce input records=192904
14/09/20 15:38:17 INFO mapred.JobClient:     Reduce input groups=192904
14/09/20 15:38:17 INFO mapred.JobClient:     Combine output records=554873
14/09/20 15:38:17 INFO mapred.JobClient:     Physical memory (bytes) snapshot=243875840
14/09/20 15:38:17 INFO mapred.JobClient:     Reduce output records=93563
14/09/20 15:38:17 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=699461632
14/09/20 15:38:17 INFO mapred.JobClient:     Map output records=2664273
14/09/20 15:38:21 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:38:21 INFO mapred.JobClient: Running job: job_201409201505_0004
14/09/20 15:38:22 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:38:32 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:38:39 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:38:42 INFO mapred.JobClient:  map 100% reduce 91%
14/09/20 15:38:44 INFO mapred.JobClient:  map 100% reduce 100%
14/09/20 15:38:44 INFO mapred.JobClient: Job complete: job_201409201505_0004
14/09/20 15:38:44 INFO mapred.JobClient: Counters: 29
14/09/20 15:38:44 INFO mapred.JobClient:   Job Counters 
14/09/20 15:38:44 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:38:44 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=5169
14/09/20 15:38:44 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:38:44 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:38:44 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:38:44 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:38:44 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=11769
14/09/20 15:38:44 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:38:44 INFO mapred.JobClient:     Bytes Written=29314118
14/09/20 15:38:44 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:38:44 INFO mapred.JobClient:     FILE_BYTES_READ=29226519
14/09/20 15:38:44 INFO mapred.JobClient:     HDFS_BYTES_READ=27503736
14/09/20 15:38:44 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=54668830
14/09/20 15:38:44 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=29314118
14/09/20 15:38:44 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:38:44 INFO mapred.JobClient:     Bytes Read=27503580
14/09/20 15:38:44 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:38:44 INFO mapred.JobClient:     Map output materialized bytes=27274291
14/09/20 15:38:44 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:38:44 INFO mapred.JobClient:     Reduce shuffle bytes=27274291
14/09/20 15:38:44 INFO mapred.JobClient:     Spilled Records=37692
14/09/20 15:38:44 INFO mapred.JobClient:     Map output bytes=27199343
14/09/20 15:38:44 INFO mapred.JobClient:     Total committed heap usage (bytes)=258617344
14/09/20 15:38:44 INFO mapred.JobClient:     CPU time spent (ms)=5930
14/09/20 15:38:44 INFO mapred.JobClient:     Combine input records=0
14/09/20 15:38:44 INFO mapred.JobClient:     SPLIT_RAW_BYTES=156
14/09/20 15:38:44 INFO mapred.JobClient:     Reduce input records=18846
14/09/20 15:38:44 INFO mapred.JobClient:     Reduce input groups=18846
14/09/20 15:38:44 INFO mapred.JobClient:     Combine output records=0
14/09/20 15:38:44 INFO mapred.JobClient:     Physical memory (bytes) snapshot=285229056
14/09/20 15:38:44 INFO mapred.JobClient:     Reduce output records=18846
14/09/20 15:38:44 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=699805696
14/09/20 15:38:44 INFO mapred.JobClient:     Map output records=18846
14/09/20 15:38:46 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:38:46 INFO mapred.JobClient: Running job: job_201409201505_0005
14/09/20 15:38:47 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:38:58 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:39:05 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:39:07 INFO mapred.JobClient: Job complete: job_201409201505_0005
14/09/20 15:39:07 INFO mapred.JobClient: Counters: 29
14/09/20 15:39:07 INFO mapred.JobClient:   Job Counters 
14/09/20 15:39:07 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:39:07 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=5851
14/09/20 15:39:07 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:39:07 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:39:07 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:39:07 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:39:07 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=9887
14/09/20 15:39:07 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:39:07 INFO mapred.JobClient:     Bytes Written=29314118
14/09/20 15:39:07 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:39:07 INFO mapred.JobClient:     FILE_BYTES_READ=29059398
14/09/20 15:39:07 INFO mapred.JobClient:     HDFS_BYTES_READ=29314272
14/09/20 15:39:07 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=58235856
14/09/20 15:39:07 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=29314118
14/09/20 15:39:07 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:39:07 INFO mapred.JobClient:     Bytes Read=29314118
14/09/20 15:39:07 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:39:07 INFO mapred.JobClient:     Map output materialized bytes=29059398
14/09/20 15:39:07 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:39:07 INFO mapred.JobClient:     Reduce shuffle bytes=29059398
14/09/20 15:39:07 INFO mapred.JobClient:     Spilled Records=37692
14/09/20 15:39:07 INFO mapred.JobClient:     Map output bytes=28984080
14/09/20 15:39:07 INFO mapred.JobClient:     Total committed heap usage (bytes)=161316864
14/09/20 15:39:07 INFO mapred.JobClient:     CPU time spent (ms)=3580
14/09/20 15:39:07 INFO mapred.JobClient:     Combine input records=0
14/09/20 15:39:07 INFO mapred.JobClient:     SPLIT_RAW_BYTES=154
14/09/20 15:39:07 INFO mapred.JobClient:     Reduce input records=18846
14/09/20 15:39:07 INFO mapred.JobClient:     Reduce input groups=18846
14/09/20 15:39:07 INFO mapred.JobClient:     Combine output records=0
14/09/20 15:39:07 INFO mapred.JobClient:     Physical memory (bytes) snapshot=213598208
14/09/20 15:39:07 INFO mapred.JobClient:     Reduce output records=18846
14/09/20 15:39:07 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=698466304
14/09/20 15:39:07 INFO mapred.JobClient:     Map output records=18846
14/09/20 15:39:07 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jifeng/20news-vectors/partial-vectors-0
14/09/20 15:39:07 INFO vectorizer.SparseVectorsFromSequenceFiles: Calculating IDF
14/09/20 15:39:09 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:39:09 INFO mapred.JobClient: Running job: job_201409201505_0006
14/09/20 15:39:10 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:39:23 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:39:30 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:39:31 INFO mapred.JobClient:  map 100% reduce 100%
14/09/20 15:39:31 INFO mapred.JobClient: Job complete: job_201409201505_0006
14/09/20 15:39:31 INFO mapred.JobClient: Counters: 29
14/09/20 15:39:31 INFO mapred.JobClient:   Job Counters 
14/09/20 15:39:31 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:39:31 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=10231
14/09/20 15:39:31 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:39:31 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:39:31 INFO mapred.JobClient:     Rack-local map tasks=1
14/09/20 15:39:31 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:39:31 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=8205
14/09/20 15:39:31 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:39:31 INFO mapred.JobClient:     Bytes Written=1890073
14/09/20 15:39:31 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:39:31 INFO mapred.JobClient:     FILE_BYTES_READ=4880830
14/09/20 15:39:31 INFO mapred.JobClient:     HDFS_BYTES_READ=29314273
14/09/20 15:39:31 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=6306594
14/09/20 15:39:31 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=1890073
14/09/20 15:39:31 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:39:31 INFO mapred.JobClient:     Bytes Read=29314118
14/09/20 15:39:31 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:39:31 INFO mapred.JobClient:     Map output materialized bytes=1309902
14/09/20 15:39:31 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:39:31 INFO mapred.JobClient:     Reduce shuffle bytes=1309902
14/09/20 15:39:31 INFO mapred.JobClient:     Spilled Records=442190
14/09/20 15:39:31 INFO mapred.JobClient:     Map output bytes=31005336
14/09/20 15:39:31 INFO mapred.JobClient:     Total committed heap usage (bytes)=132190208
14/09/20 15:39:31 INFO mapred.JobClient:     CPU time spent (ms)=5200
14/09/20 15:39:31 INFO mapred.JobClient:     Combine input records=2838840
14/09/20 15:39:31 INFO mapred.JobClient:     SPLIT_RAW_BYTES=155
14/09/20 15:39:31 INFO mapred.JobClient:     Reduce input records=93564
14/09/20 15:39:31 INFO mapred.JobClient:     Reduce input groups=93564
14/09/20 15:39:31 INFO mapred.JobClient:     Combine output records=348626
14/09/20 15:39:31 INFO mapred.JobClient:     Physical memory (bytes) snapshot=186679296
14/09/20 15:39:31 INFO mapred.JobClient:     Reduce output records=93564
14/09/20 15:39:31 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=699461632
14/09/20 15:39:31 INFO mapred.JobClient:     Map output records=2583778
14/09/20 15:39:31 INFO vectorizer.SparseVectorsFromSequenceFiles: Pruning
14/09/20 15:39:35 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:39:35 INFO mapred.JobClient: Running job: job_201409201505_0007
14/09/20 15:39:36 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:39:49 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:39:57 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:40:00 INFO mapred.JobClient:  map 100% reduce 67%
14/09/20 15:40:02 INFO mapred.JobClient:  map 100% reduce 100%
14/09/20 15:40:04 INFO mapred.JobClient: Job complete: job_201409201505_0007
14/09/20 15:40:04 INFO mapred.JobClient: Counters: 29
14/09/20 15:40:04 INFO mapred.JobClient:   Job Counters 
14/09/20 15:40:04 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:40:04 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=11081
14/09/20 15:40:04 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:40:04 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:40:04 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:40:04 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:40:04 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=13644
14/09/20 15:40:04 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:40:04 INFO mapred.JobClient:     Bytes Written=28689283
14/09/20 15:40:04 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:40:04 INFO mapred.JobClient:     FILE_BYTES_READ=9646422
14/09/20 15:40:04 INFO mapred.JobClient:     HDFS_BYTES_READ=29314273
14/09/20 15:40:04 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=15601678
14/09/20 15:40:04 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=28689283
14/09/20 15:40:04 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:40:04 INFO mapred.JobClient:     Bytes Read=29314118
14/09/20 15:40:04 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:40:04 INFO mapred.JobClient:     Map output materialized bytes=7741585
14/09/20 15:40:04 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:40:04 INFO mapred.JobClient:     Reduce shuffle bytes=7741585
14/09/20 15:40:04 INFO mapred.JobClient:     Spilled Records=37692
14/09/20 15:40:04 INFO mapred.JobClient:     Map output bytes=28984080
14/09/20 15:40:04 INFO mapred.JobClient:     Total committed heap usage (bytes)=192774144
14/09/20 15:40:04 INFO mapred.JobClient:     CPU time spent (ms)=8050
14/09/20 15:40:04 INFO mapred.JobClient:     Combine input records=0
14/09/20 15:40:04 INFO mapred.JobClient:     SPLIT_RAW_BYTES=155
14/09/20 15:40:04 INFO mapred.JobClient:     Reduce input records=18846
14/09/20 15:40:04 INFO mapred.JobClient:     Reduce input groups=18846
14/09/20 15:40:04 INFO mapred.JobClient:     Combine output records=0
14/09/20 15:40:04 INFO mapred.JobClient:     Physical memory (bytes) snapshot=306360320
14/09/20 15:40:04 INFO mapred.JobClient:     Reduce output records=18846
14/09/20 15:40:04 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1039331328
14/09/20 15:40:04 INFO mapred.JobClient:     Map output records=18846
14/09/20 15:40:05 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:40:05 INFO mapred.JobClient: Running job: job_201409201505_0008
14/09/20 15:40:06 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:40:15 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:40:22 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:40:25 INFO mapred.JobClient:  map 100% reduce 100%
14/09/20 15:40:25 INFO mapred.JobClient: Job complete: job_201409201505_0008
14/09/20 15:40:25 INFO mapred.JobClient: Counters: 29
14/09/20 15:40:25 INFO mapred.JobClient:   Job Counters 
14/09/20 15:40:25 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:40:25 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=7062
14/09/20 15:40:25 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:40:25 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:40:25 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:40:25 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:40:25 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=9478
14/09/20 15:40:25 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:40:25 INFO mapred.JobClient:     Bytes Written=28689283
14/09/20 15:40:25 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:40:25 INFO mapred.JobClient:     FILE_BYTES_READ=28437750
14/09/20 15:40:25 INFO mapred.JobClient:     HDFS_BYTES_READ=28689448
14/09/20 15:40:25 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=56991474
14/09/20 15:40:25 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=28689283
14/09/20 15:40:25 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:40:25 INFO mapred.JobClient:     Bytes Read=28689283
14/09/20 15:40:25 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:40:25 INFO mapred.JobClient:     Map output materialized bytes=28437750
14/09/20 15:40:25 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:40:25 INFO mapred.JobClient:     Reduce shuffle bytes=28437750
14/09/20 15:40:25 INFO mapred.JobClient:     Spilled Records=37692
14/09/20 15:40:25 INFO mapred.JobClient:     Map output bytes=28362505
14/09/20 15:40:25 INFO mapred.JobClient:     Total committed heap usage (bytes)=160694272
14/09/20 15:40:25 INFO mapred.JobClient:     CPU time spent (ms)=3090
14/09/20 15:40:25 INFO mapred.JobClient:     Combine input records=0
14/09/20 15:40:25 INFO mapred.JobClient:     SPLIT_RAW_BYTES=165
14/09/20 15:40:25 INFO mapred.JobClient:     Reduce input records=18846
14/09/20 15:40:25 INFO mapred.JobClient:     Reduce input groups=18846
14/09/20 15:40:25 INFO mapred.JobClient:     Combine output records=0
14/09/20 15:40:25 INFO mapred.JobClient:     Physical memory (bytes) snapshot=212541440
14/09/20 15:40:25 INFO mapred.JobClient:     Reduce output records=18846
14/09/20 15:40:25 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=698466304
14/09/20 15:40:25 INFO mapred.JobClient:     Map output records=18846
14/09/20 15:40:25 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jifeng/20news-vectors/tf-vectors-partial
14/09/20 15:40:25 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jifeng/20news-vectors/tf-vectors-toprune
14/09/20 15:40:29 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:40:29 INFO mapred.JobClient: Running job: job_201409201505_0009
14/09/20 15:40:30 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:40:40 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:40:48 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:40:51 INFO mapred.JobClient:  map 100% reduce 89%
14/09/20 15:40:53 INFO mapred.JobClient:  map 100% reduce 100%
14/09/20 15:40:54 INFO mapred.JobClient: Job complete: job_201409201505_0009
14/09/20 15:40:54 INFO mapred.JobClient: Counters: 29
14/09/20 15:40:54 INFO mapred.JobClient:   Job Counters 
14/09/20 15:40:54 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:40:54 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=9479
14/09/20 15:40:54 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:40:54 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:40:54 INFO mapred.JobClient:     Rack-local map tasks=1
14/09/20 15:40:54 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:40:54 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=12699
14/09/20 15:40:54 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:40:54 INFO mapred.JobClient:     Bytes Written=28689283
14/09/20 15:40:54 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:40:54 INFO mapred.JobClient:     FILE_BYTES_READ=30342579
14/09/20 15:40:54 INFO mapred.JobClient:     HDFS_BYTES_READ=28689430
14/09/20 15:40:54 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=56995482
14/09/20 15:40:54 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=28689283
14/09/20 15:40:54 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:40:54 INFO mapred.JobClient:     Bytes Read=28689283
14/09/20 15:40:54 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:40:54 INFO mapred.JobClient:     Map output materialized bytes=28437750
14/09/20 15:40:54 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:40:54 INFO mapred.JobClient:     Reduce shuffle bytes=28437750
14/09/20 15:40:54 INFO mapred.JobClient:     Spilled Records=37692
14/09/20 15:40:54 INFO mapred.JobClient:     Map output bytes=28362505
14/09/20 15:40:54 INFO mapred.JobClient:     Total committed heap usage (bytes)=192151552
14/09/20 15:40:54 INFO mapred.JobClient:     CPU time spent (ms)=6690
14/09/20 15:40:54 INFO mapred.JobClient:     Combine input records=0
14/09/20 15:40:54 INFO mapred.JobClient:     SPLIT_RAW_BYTES=147
14/09/20 15:40:54 INFO mapred.JobClient:     Reduce input records=18846
14/09/20 15:40:54 INFO mapred.JobClient:     Reduce input groups=18846
14/09/20 15:40:54 INFO mapred.JobClient:     Combine output records=0
14/09/20 15:40:54 INFO mapred.JobClient:     Physical memory (bytes) snapshot=294711296
14/09/20 15:40:54 INFO mapred.JobClient:     Reduce output records=18846
14/09/20 15:40:54 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1024688128
14/09/20 15:40:54 INFO mapred.JobClient:     Map output records=18846
14/09/20 15:40:55 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:40:55 INFO mapred.JobClient: Running job: job_201409201505_0010
14/09/20 15:40:56 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:41:06 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:41:13 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:41:15 INFO mapred.JobClient: Job complete: job_201409201505_0010
14/09/20 15:41:15 INFO mapred.JobClient: Counters: 29
14/09/20 15:41:15 INFO mapred.JobClient:   Job Counters 
14/09/20 15:41:15 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:41:15 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=7371
14/09/20 15:41:15 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:41:15 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:41:15 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:41:15 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:41:15 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=9851
14/09/20 15:41:15 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:41:15 INFO mapred.JobClient:     Bytes Written=28689283
14/09/20 15:41:15 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:41:15 INFO mapred.JobClient:     FILE_BYTES_READ=28437750
14/09/20 15:41:15 INFO mapred.JobClient:     HDFS_BYTES_READ=28689437
14/09/20 15:41:15 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=56992548
14/09/20 15:41:15 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=28689283
14/09/20 15:41:15 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:41:15 INFO mapred.JobClient:     Bytes Read=28689283
14/09/20 15:41:15 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:41:15 INFO mapred.JobClient:     Map output materialized bytes=28437750
14/09/20 15:41:15 INFO mapred.JobClient:     Map input records=18846
14/09/20 15:41:15 INFO mapred.JobClient:     Reduce shuffle bytes=28437750
14/09/20 15:41:15 INFO mapred.JobClient:     Spilled Records=37692
14/09/20 15:41:15 INFO mapred.JobClient:     Map output bytes=28362505
14/09/20 15:41:15 INFO mapred.JobClient:     Total committed heap usage (bytes)=160694272
14/09/20 15:41:15 INFO mapred.JobClient:     CPU time spent (ms)=3450
14/09/20 15:41:15 INFO mapred.JobClient:     Combine input records=0
14/09/20 15:41:15 INFO mapred.JobClient:     SPLIT_RAW_BYTES=154
14/09/20 15:41:15 INFO mapred.JobClient:     Reduce input records=18846
14/09/20 15:41:15 INFO mapred.JobClient:     Reduce input groups=18846
14/09/20 15:41:15 INFO mapred.JobClient:     Combine output records=0
14/09/20 15:41:15 INFO mapred.JobClient:     Physical memory (bytes) snapshot=213041152
14/09/20 15:41:15 INFO mapred.JobClient:     Reduce output records=18846
14/09/20 15:41:15 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=700563456
14/09/20 15:41:15 INFO mapred.JobClient:     Map output records=18846
14/09/20 15:41:15 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jifeng/20news-vectors/partial-vectors-0
14/09/20 15:41:15 INFO driver.MahoutDriver: Program took 232258 ms (Minutes: 3.8709666666666664)
+ echo 'Creating training and holdout set with a random 80-20 split of the generated vector dataset'
Creating training and holdout set with a random 80-20 split of the generated vector dataset
+ ./bin/mahout split -i /tmp/mahout-work-jifeng/20news-vectors/tfidf-vectors --trainingOutput /tmp/mahout-work-jifeng/20news-train-vectors --testOutput /tmp/mahout-work-jifeng/20news-test-vectors --randomSelectionPct 40 --overwrite --sequenceFiles -xm sequential
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/jifeng/hadoop/hadoop-1.2.1/bin/hadoop and HADOOP_CONF_DIR=/home/jifeng/hadoop/hadoop-1.2.1/conf
MAHOUT-JOB: /home/jifeng/hadoop/mahout-distribution-0.9/mahout-examples-0.9-job.jar
Warning: $HADOOP_HOME is deprecated.

14/09/20 15:41:17 WARN driver.MahoutDriver: No split.props found on classpath, will use command-line arguments only
14/09/20 15:41:18 INFO common.AbstractJob: Command line arguments: {--endPhase=[2147483647], --input=[/tmp/mahout-work-jifeng/20news-vectors/tfidf-vectors], --method=[sequential], --overwrite=null, --randomSelectionPct=[40], --sequenceFiles=null, --startPhase=[0], --tempDir=[temp], --testOutput=[/tmp/mahout-work-jifeng/20news-test-vectors], --trainingOutput=[/tmp/mahout-work-jifeng/20news-train-vectors]}
14/09/20 15:41:19 INFO utils.SplitInput: part-r-00000 has 162419 lines
14/09/20 15:41:19 INFO utils.SplitInput: part-r-00000 test split size is 64968 based on random selection percentage 40
14/09/20 15:41:20 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/09/20 15:41:20 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
14/09/20 15:41:20 INFO compress.CodecPool: Got brand-new compressor
14/09/20 15:41:21 INFO compress.CodecPool: Got brand-new compressor
14/09/20 15:41:25 INFO utils.SplitInput: file: part-r-00000, input: 162419 train: 11198, test: 7648 starting at 0
14/09/20 15:41:25 INFO driver.MahoutDriver: Program took 7698 ms (Minutes: 0.1283)
+ echo 'Training Naive Bayes model'
Training Naive Bayes model
+ ./bin/mahout trainnb -i /tmp/mahout-work-jifeng/20news-train-vectors -el -o /tmp/mahout-work-jifeng/model -li /tmp/mahout-work-jifeng/labelindex -ow
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/jifeng/hadoop/hadoop-1.2.1/bin/hadoop and HADOOP_CONF_DIR=/home/jifeng/hadoop/hadoop-1.2.1/conf
MAHOUT-JOB: /home/jifeng/hadoop/mahout-distribution-0.9/mahout-examples-0.9-job.jar
Warning: $HADOOP_HOME is deprecated.

14/09/20 15:41:28 WARN driver.MahoutDriver: No trainnb.props found on classpath, will use command-line arguments only
14/09/20 15:41:28 INFO common.AbstractJob: Command line arguments: {--alphaI=[1.0], --endPhase=[2147483647], --extractLabels=null, --input=[/tmp/mahout-work-jifeng/20news-train-vectors], --labelIndex=[/tmp/mahout-work-jifeng/labelindex], --output=[/tmp/mahout-work-jifeng/model], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
14/09/20 15:41:28 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/09/20 15:41:28 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
14/09/20 15:41:28 INFO compress.CodecPool: Got brand-new decompressor
14/09/20 15:41:34 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:41:37 INFO mapred.JobClient: Running job: job_201409201505_0011
14/09/20 15:41:38 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:41:48 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:41:56 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:41:57 INFO mapred.JobClient:  map 100% reduce 100%
14/09/20 15:41:58 INFO mapred.JobClient: Job complete: job_201409201505_0011
14/09/20 15:41:58 INFO mapred.JobClient: Counters: 29
14/09/20 15:41:58 INFO mapred.JobClient:   Job Counters 
14/09/20 15:41:58 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:41:58 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=5959
14/09/20 15:41:58 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:41:58 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:41:58 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:41:58 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:41:58 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=8506
14/09/20 15:41:58 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:41:58 INFO mapred.JobClient:     Bytes Written=2736814
14/09/20 15:41:58 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:41:58 INFO mapred.JobClient:     FILE_BYTES_READ=1515825
14/09/20 15:41:58 INFO mapred.JobClient:     HDFS_BYTES_READ=12691767
14/09/20 15:41:58 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=3149124
14/09/20 15:41:58 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=2736814
14/09/20 15:41:58 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:41:58 INFO mapred.JobClient:     Bytes Read=12691625
14/09/20 15:41:58 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:41:58 INFO mapred.JobClient:     Map output materialized bytes=1515143
14/09/20 15:41:58 INFO mapred.JobClient:     Map input records=11198
14/09/20 15:41:58 INFO mapred.JobClient:     Reduce shuffle bytes=1515143
14/09/20 15:41:58 INFO mapred.JobClient:     Spilled Records=40
14/09/20 15:41:58 INFO mapred.JobClient:     Map output bytes=16617381
14/09/20 15:41:58 INFO mapred.JobClient:     Total committed heap usage (bytes)=219086848
14/09/20 15:41:58 INFO mapred.JobClient:     CPU time spent (ms)=2870
14/09/20 15:41:58 INFO mapred.JobClient:     Combine input records=11198
14/09/20 15:41:58 INFO mapred.JobClient:     SPLIT_RAW_BYTES=142
14/09/20 15:41:58 INFO mapred.JobClient:     Reduce input records=20
14/09/20 15:41:58 INFO mapred.JobClient:     Reduce input groups=20
14/09/20 15:41:58 INFO mapred.JobClient:     Combine output records=20
14/09/20 15:41:58 INFO mapred.JobClient:     Physical memory (bytes) snapshot=204357632
14/09/20 15:41:58 INFO mapred.JobClient:     Reduce output records=20
14/09/20 15:41:58 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=701775872
14/09/20 15:41:58 INFO mapred.JobClient:     Map output records=11198
14/09/20 15:41:58 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:41:58 INFO mapred.JobClient: Running job: job_201409201505_0012
14/09/20 15:41:59 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:42:06 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:42:13 INFO mapred.JobClient:  map 100% reduce 33%
14/09/20 15:42:14 INFO mapred.JobClient:  map 100% reduce 100%
14/09/20 15:42:14 INFO mapred.JobClient: Job complete: job_201409201505_0012
14/09/20 15:42:14 INFO mapred.JobClient: Counters: 29
14/09/20 15:42:14 INFO mapred.JobClient:   Job Counters 
14/09/20 15:42:14 INFO mapred.JobClient:     Launched reduce tasks=1
14/09/20 15:42:14 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=3188
14/09/20 15:42:14 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:42:14 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:42:14 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:42:14 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:42:14 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=8219
14/09/20 15:42:14 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:42:14 INFO mapred.JobClient:     Bytes Written=899207
14/09/20 15:42:14 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:42:14 INFO mapred.JobClient:     FILE_BYTES_READ=444260
14/09/20 15:42:14 INFO mapred.JobClient:     HDFS_BYTES_READ=2736948
14/09/20 15:42:14 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=1007696
14/09/20 15:42:14 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=899207
14/09/20 15:42:14 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:42:14 INFO mapred.JobClient:     Bytes Read=2736814
14/09/20 15:42:14 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:42:14 INFO mapred.JobClient:     Map output materialized bytes=444252
14/09/20 15:42:14 INFO mapred.JobClient:     Map input records=20
14/09/20 15:42:14 INFO mapred.JobClient:     Reduce shuffle bytes=444252
14/09/20 15:42:14 INFO mapred.JobClient:     Spilled Records=4
14/09/20 15:42:14 INFO mapred.JobClient:     Map output bytes=899081
14/09/20 15:42:14 INFO mapred.JobClient:     Total committed heap usage (bytes)=225873920
14/09/20 15:42:14 INFO mapred.JobClient:     CPU time spent (ms)=960
14/09/20 15:42:14 INFO mapred.JobClient:     Combine input records=2
14/09/20 15:42:14 INFO mapred.JobClient:     SPLIT_RAW_BYTES=134
14/09/20 15:42:14 INFO mapred.JobClient:     Reduce input records=2
14/09/20 15:42:14 INFO mapred.JobClient:     Reduce input groups=2
14/09/20 15:42:14 INFO mapred.JobClient:     Combine output records=2
14/09/20 15:42:14 INFO mapred.JobClient:     Physical memory (bytes) snapshot=224452608
14/09/20 15:42:14 INFO mapred.JobClient:     Reduce output records=2
14/09/20 15:42:14 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=701775872
14/09/20 15:42:14 INFO mapred.JobClient:     Map output records=2
14/09/20 15:42:15 INFO driver.MahoutDriver: Program took 47200 ms (Minutes: 0.7866666666666666)
+ echo 'Self testing on training set'
Self testing on training set
+ ./bin/mahout testnb -i /tmp/mahout-work-jifeng/20news-train-vectors -m /tmp/mahout-work-jifeng/model -l /tmp/mahout-work-jifeng/labelindex -ow -o /tmp/mahout-work-jifeng/20news-testing
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/jifeng/hadoop/hadoop-1.2.1/bin/hadoop and HADOOP_CONF_DIR=/home/jifeng/hadoop/hadoop-1.2.1/conf
MAHOUT-JOB: /home/jifeng/hadoop/mahout-distribution-0.9/mahout-examples-0.9-job.jar
Warning: $HADOOP_HOME is deprecated.

14/09/20 15:42:17 WARN driver.MahoutDriver: No testnb.props found on classpath, will use command-line arguments only
14/09/20 15:42:17 INFO common.AbstractJob: Command line arguments: {--endPhase=[2147483647], --input=[/tmp/mahout-work-jifeng/20news-train-vectors], --labelIndex=[/tmp/mahout-work-jifeng/labelindex], --model=[/tmp/mahout-work-jifeng/model], --output=[/tmp/mahout-work-jifeng/20news-testing], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
14/09/20 15:42:20 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:42:21 INFO mapred.JobClient: Running job: job_201409201505_0013
14/09/20 15:42:22 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:42:35 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:42:35 INFO mapred.JobClient: Job complete: job_201409201505_0013
14/09/20 15:42:35 INFO mapred.JobClient: Counters: 20
14/09/20 15:42:35 INFO mapred.JobClient:   Job Counters 
14/09/20 15:42:35 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=9283
14/09/20 15:42:35 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:42:35 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:42:35 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:42:35 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:42:35 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
14/09/20 15:42:35 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:42:35 INFO mapred.JobClient:     Bytes Written=2109460
14/09/20 15:42:35 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:42:35 INFO mapred.JobClient:     FILE_BYTES_READ=3663744
14/09/20 15:42:35 INFO mapred.JobClient:     HDFS_BYTES_READ=12691767
14/09/20 15:42:35 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=58876
14/09/20 15:42:35 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=2109460
14/09/20 15:42:35 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:42:35 INFO mapred.JobClient:     Bytes Read=12691625
14/09/20 15:42:35 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:42:35 INFO mapred.JobClient:     Map input records=11198
14/09/20 15:42:35 INFO mapred.JobClient:     Physical memory (bytes) snapshot=57454592
14/09/20 15:42:35 INFO mapred.JobClient:     Spilled Records=0
14/09/20 15:42:35 INFO mapred.JobClient:     CPU time spent (ms)=5310
14/09/20 15:42:35 INFO mapred.JobClient:     Total committed heap usage (bytes)=29954048
14/09/20 15:42:35 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=349515776
14/09/20 15:42:35 INFO mapred.JobClient:     Map output records=11198
14/09/20 15:42:35 INFO mapred.JobClient:     SPLIT_RAW_BYTES=142
14/09/20 15:42:36 INFO test.TestNaiveBayesDriver: Standard NB Results: 
=======================================================
Summary
-------------------------------------------------------
Correctly Classified Instances          :      11119       99.2945%
Incorrectly Classified Instances        :         79        0.7055%
Total Classified Instances              :      11198

=======================================================
Confusion Matrix
-------------------------------------------------------
a       b       c       d       e       f       g       h       i       j       k       l       m       n        o       p       q       r       s       t       <--Classified as
434     0       0       0       0       0       0       0       0       0       0       0       0       0        0       0       0       0       1       0        |  435         a     = alt.atheism
0       555     0       2       1       3       0       0       0       0       0       1       0       1        0       0       0       0       0       0        |  563         b     = comp.graphics
0       6       553     18      1       1       0       0       0       0       0       0       0       0        0       0       0       0       0       0        |  579         c     = comp.os.ms-windows.misc
0       0       0       564     1       0       3       0       0       0       0       0       0       0        0       0       0       0       0       0        |  568         d     = comp.sys.ibm.pc.hardware
0       0       1       0       573     0       0       0       0       0       0       0       1       0        0       0       0       0       0       0        |  575         e     = comp.sys.mac.hardware
0       1       1       0       0       585     0       0       0       0       0       0       0       0        0       0       0       0       0       0        |  587         f     = comp.windows.x
0       0       0       1       0       0       582     1       0       0       0       0       2       0        0       0       0       0       0       0        |  586         g     = misc.forsale
0       0       0       0       1       0       1       613     0       0       0       0       0       0        0       0       0       0       0       1        |  616         h     = rec.autos
0       0       0       0       0       0       1       0       603     0       0       0       0       0        0       0       0       0       0       0        |  604         i     = rec.motorcycles
0       0       0       0       0       0       0       0       0       595     0       1       0       0        0       0       0       0       0       0        |  596         j     = rec.sport.baseball
0       0       0       0       0       0       0       0       0       0       584     0       0       0        0       0       0       0       0       1        |  585         k     = rec.sport.hockey
0       0       0       0       0       0       0       0       0       0       0       583     1       0        0       0       0       0       0       1        |  585         l     = sci.crypt
0       0       0       2       0       0       0       0       0       0       0       0       584     0        0       0       0       0       0       0        |  586         m     = sci.electronics
0       1       0       0       0       0       0       0       0       0       0       0       1       570      0       0       0       0       0       0        |  572         n     = sci.med
0       0       0       0       0       0       0       0       0       0       0       0       0       1        617     0       0       0       0       0        |  618         o     = sci.space
0       0       0       0       0       0       0       0       0       0       0       0       0       0        0       592     1       0       0       0        |  593         p     = soc.religion.christian
0       0       0       0       0       0       0       0       0       0       0       0       0       0        0       0       565     0       0       0        |  565         q     = talk.politics.mideast
0       0       0       0       0       0       0       0       0       0       0       1       0       0        0       0       0       544     0       0        |  545         r     = talk.politics.guns
7       0       0       0       0       0       0       0       0       0       0       0       0       0        1       1       0       1       359     1        |  370         s     = talk.religion.misc
0       0       0       0       0       0       0       0       0       0       0       1       0       0        0       0       0       5       0       464      |  470         t     = talk.politics.misc

=======================================================
Statistics
-------------------------------------------------------
Kappa                                        0.987
Accuracy                                   99.2945%
Reliability                                94.5236%
Reliability (standard deviation)            0.2169

14/09/20 15:42:36 INFO driver.MahoutDriver: Program took 18730 ms (Minutes: 0.31216666666666665)
+ echo 'Testing on holdout set'
Testing on holdout set
+ ./bin/mahout testnb -i /tmp/mahout-work-jifeng/20news-test-vectors -m /tmp/mahout-work-jifeng/model -l /tmp/mahout-work-jifeng/labelindex -ow -o /tmp/mahout-work-jifeng/20news-testing
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/jifeng/hadoop/hadoop-1.2.1/bin/hadoop and HADOOP_CONF_DIR=/home/jifeng/hadoop/hadoop-1.2.1/conf
MAHOUT-JOB: /home/jifeng/hadoop/mahout-distribution-0.9/mahout-examples-0.9-job.jar
Warning: $HADOOP_HOME is deprecated.

14/09/20 15:42:39 WARN driver.MahoutDriver: No testnb.props found on classpath, will use command-line arguments only
14/09/20 15:42:39 INFO common.AbstractJob: Command line arguments: {--endPhase=[2147483647], --input=[/tmp/mahout-work-jifeng/20news-test-vectors], --labelIndex=[/tmp/mahout-work-jifeng/labelindex], --model=[/tmp/mahout-work-jifeng/model], --output=[/tmp/mahout-work-jifeng/20news-testing], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
14/09/20 15:42:39 INFO common.HadoopUtil: Deleting /tmp/mahout-work-jifeng/20news-testing
14/09/20 15:42:41 INFO input.FileInputFormat: Total input paths to process : 1
14/09/20 15:42:41 INFO mapred.JobClient: Running job: job_201409201505_0014
14/09/20 15:42:42 INFO mapred.JobClient:  map 0% reduce 0%
14/09/20 15:42:59 INFO mapred.JobClient:  map 100% reduce 0%
14/09/20 15:42:59 INFO mapred.JobClient: Job complete: job_201409201505_0014
14/09/20 15:42:59 INFO mapred.JobClient: Counters: 20
14/09/20 15:42:59 INFO mapred.JobClient:   Job Counters 
14/09/20 15:42:59 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=8035
14/09/20 15:42:59 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/09/20 15:42:59 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/09/20 15:42:59 INFO mapred.JobClient:     Launched map tasks=1
14/09/20 15:42:59 INFO mapred.JobClient:     Data-local map tasks=1
14/09/20 15:42:59 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
14/09/20 15:42:59 INFO mapred.JobClient:   File Output Format Counters 
14/09/20 15:42:59 INFO mapred.JobClient:     Bytes Written=1440968
14/09/20 15:42:59 INFO mapred.JobClient:   FileSystemCounters
14/09/20 15:42:59 INFO mapred.JobClient:     FILE_BYTES_READ=3663744
14/09/20 15:42:59 INFO mapred.JobClient:     HDFS_BYTES_READ=8662748
14/09/20 15:42:59 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=58876
14/09/20 15:42:59 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=1440968
14/09/20 15:42:59 INFO mapred.JobClient:   File Input Format Counters 
14/09/20 15:42:59 INFO mapred.JobClient:     Bytes Read=8662607
14/09/20 15:42:59 INFO mapred.JobClient:   Map-Reduce Framework
14/09/20 15:42:59 INFO mapred.JobClient:     Map input records=7648
14/09/20 15:42:59 INFO mapred.JobClient:     Physical memory (bytes) snapshot=58269696
14/09/20 15:42:59 INFO mapred.JobClient:     Spilled Records=0
14/09/20 15:42:59 INFO mapred.JobClient:     CPU time spent (ms)=4120
14/09/20 15:42:59 INFO mapred.JobClient:     Total committed heap usage (bytes)=31711232
14/09/20 15:42:59 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=349515776
14/09/20 15:42:59 INFO mapred.JobClient:     Map output records=7648
14/09/20 15:42:59 INFO mapred.JobClient:     SPLIT_RAW_BYTES=141
14/09/20 15:43:00 INFO test.TestNaiveBayesDriver: Standard NB Results: 
=======================================================
Summary
-------------------------------------------------------
Correctly Classified Instances          :       6913       90.3896%
Incorrectly Classified Instances        :        735        9.6104%
Total Classified Instances              :       7648

=======================================================
Confusion Matrix
-------------------------------------------------------
a       b       c       d       e       f       g       h       i       j       k       l       m       n        o       p       q       r       s       t       <--Classified as
327     0       0       0       1       0       0       0       0       1       0       0       0       1        0       9       0       2       20      3        |  364         a     = alt.atheism
0       350     2       22      7       10      8       0       0       1       0       1       2       1        5       0       0       0       1       0        |  410         b     = comp.graphics
0       25      254     73      17      20      8       0       0       0       0       1       4       2        0       0       0       0       0       2        |  406         c     = comp.os.ms-windows.misc
1       4       2       375     18      3       4       1       0       0       0       0       5       0        0       0       0       0       1       0        |  414         d     = comp.sys.ibm.pc.hardware
0       4       3       16      355     0       5       0       0       0       0       1       4       0        0       0       0       0       0       0        |  388         e     = comp.sys.mac.hardware
0       28      0       7       8       348     2       0       1       1       0       5       0       0        1       0       0       0       0       0        |  401         f     = comp.windows.x
1       5       1       15      9       0       330     11      4       0       3       0       5       2        1       0       0       0       1       1        |  389         g     = misc.forsale
0       1       0       0       1       1       7       357     3       0       0       0       1       0        2       0       0       1       0       0        |  374         h     = rec.autos
0       0       0       0       0       0       0       5       386     0       0       0       0       1        0       0       0       0       0       0        |  392         i     = rec.motorcycles
0       0       0       1       2       0       1       0       1       389     3       0       1       0        0       0       0       0       0       0        |  398         j     = rec.sport.baseball
0       0       0       1       0       0       0       0       0       5       405     0       2       0        0       1       0       0       0       0        |  414         k     = rec.sport.hockey
1       2       1       0       3       2       1       0       0       0       0       386     1       1        0       0       0       4       1       3        |  406         l     = sci.crypt
0       3       0       14      7       1       10      6       0       0       1       2       350     1        1       0       0       1       0       1        |  398         m     = sci.electronics
1       1       1       1       2       0       6       3       0       2       1       2       2       389      2       0       1       3       1       0        |  418         n     = sci.med
0       3       0       0       4       2       1       0       0       0       0       1       3       1        346     0       0       2       3       3        |  369         o     = sci.space
2       3       0       1       0       0       0       0       0       1       0       0       1       4        0       382     0       1       8       1        |  404         p     = soc.religion.christian
1       1       0       0       0       0       0       0       0       0       1       0       0       0        0       3       367     0       0       2        |  375         q     = talk.politics.mideast
0       0       1       0       0       0       0       2       2       0       0       2       0       0        1       0       0       347     1       9        |  365         r     = talk.politics.guns
24      1       0       0       0       1       1       0       0       0       1       0       0       0        0       17      3       7       196     7        |  258         s     = talk.religion.misc
1       0       0       0       2       0       0       1       0       0       1       1       1       0        2       2       3       15      2       274      |  305         t     = talk.politics.misc

=======================================================
Statistics
-------------------------------------------------------
Kappa                                       0.8758
Accuracy                                   90.3896%
Reliability                                85.9085%
Reliability (standard deviation)            0.2138

14/09/20 15:43:00 INFO driver.MahoutDriver: Program took 21261 ms (Minutes: 0.35435)

你可能感兴趣的:(hadoop,Mahout)

浅谈MapReduce Android路上的人 Hadoop 分布式计算 mapreduce 分布式框架 hadoop
从今天开始，本人将会开始对另一项技术的学习，就是当下炙手可热的Hadoop分布式就算技术。目前国内外的诸多公司因为业务发展的需要，都纷纷用了此平台。国内的比如BAT啦，国外的在这方面走的更加的前面，就不一一列举了。但是Hadoop作为Apache的一个开源项目，在下面有非常多的子项目，比如HDFS，HBase,Hive，Pig,等等，要先彻底学习整个Hadoop，仅仅凭借一个的力量，是远远不够的。
Hadoop 傲雪凌霜，松柏长青后端大数据 hadoop 大数据分布式
ApacheHadoop是一个开源的分布式计算框架，主要用于处理海量数据集。它具有高度的可扩展性、容错性和高效的分布式存储与计算能力。Hadoop核心由四个主要模块组成，分别是HDFS（分布式文件系统）、MapReduce（分布式计算框架）、YARN（资源管理）和HadoopCommon（公共工具和库）。1.HDFS（HadoopDistributedFileSystem）HDFS是Hadoop生
Hadoop架构 henan程序媛 hadoop 大数据分布式
一、案列分析1.1案例概述现在已经进入了大数据(BigData)时代，数以万计用户的互联网服务时时刻刻都在产生大量的交互，要处理的数据量实在是太大了，以传统的数据库技术等其他手段根本无法应对数据处理的实时性、有效性的需求。HDFS顺应时代出现，在解决大数据存储和计算方面有很多的优势。1.2案列前置知识点1.什么是大数据大数据是指无法在一定时间范围内用常规软件工具进行捕捉、管理和处理的大量数据集合，
分享一个基于python的电子书数据采集与可视化分析 hadoop电子书数据分析与推荐系统 spark大数据毕设项目（源码、调试、LW、开题、PPT) 计算机源码社 Python项目大数据大数据 python hadoop 计算机毕业设计选题计算机毕业设计源码数据分析 spark毕设
作者：计算机源码社个人简介：本人八年开发经验，擅长Java、Python、PHP、.NET、Node.js、Android、微信小程序、爬虫、大数据、机器学习等，大家有这一块的问题可以一起交流！学习资料、程序开发、技术解答、文档报告如需要源码，可以扫取文章下方二维码联系咨询Java项目微信小程序项目Android项目Python项目PHP项目ASP.NET项目Node.js项目选题推荐项目实战|p
hbase介绍 CrazyL- 云计算+大数据 hbase
hbase是一个分布式的、多版本的、面向列的开源数据库hbase利用hadoophdfs作为其文件存储系统，提供高可靠性、高性能、列存储、可伸缩、实时读写、适用于非结构化数据存储的数据库系统hbase利用hadoopmapreduce来处理hbase、中的海量数据hbase利用zookeeper作为分布式系统服务特点：数据量大：一个表可以有上亿行，上百万列（列多时，插入变慢）面向列：面向列（族）的
大数据毕业设计hadoop+spark+hive知识图谱租房数据分析可视化大屏租房推荐系统 58同城租房爬虫房源推荐系统房价预测系统计算机毕业设计机器学习深度学习人工智能 2401_84572577 程序员大数据 hadoop 人工智能
做了那么多年开发，自学了很多门编程语言，我很明白学习资源对于学一门新语言的重要性，这些年也收藏了不少的Python干货，对我来说这些东西确实已经用不到了，但对于准备自学Python的人来说，或许它就是一个宝藏，可以给你省去很多的时间和精力。别在网上瞎学了，我最近也做了一些资源的更新，只要你是我的粉丝，这期福利你都可拿走。我先来介绍一下这些东西怎么用，文末抱走。（1）Python所有方向的学习路线（
Spark集群的三种模式 MelodyYN #Spark spark hadoop big data
文章目录1、Spark的由来1.1Hadoop的发展1.2MapReduce与Spark对比2、Spark内置模块3、Spark运行模式3.1Standalone模式部署配置历史服务器配置高可用运行模式3.2Yarn模式安装部署配置历史服务器运行模式4、WordCount案例1、Spark的由来定义：Hadoop主要解决，海量数据的存储和海量数据的分析计算。Spark是一种基于内存的快速、通用、可
月度总结 | 2022年03月 | 考研与就业的抉择 | 确定未来走大数据开发路线「已注销」个人总结 hadoop
一、时间线梳理3月3日，寻找到同专业的就业伙伴3月5日，着手准备Java八股文，决定先走Java后端路线3月8月，申请到了校图书馆的考研专座，决定暂时放弃就业，先准备考研，买了数学和408的资料书3月9日-3月13日，因疫情原因，宿舍区暂封，这段时间在准备考研，发现内容特别多3月13日-3月19日，大部分时间在刷Hadoop、Zookeeper、Kafka的视频，同时在准备实习的项目3月20日，退
HBase介绍 mingyu1016 数据库
概述HBase是一个分布式的、面向列的开源数据库,源于google的一篇论文《bigtable：一个结构化数据的分布式存储系统》。HBase是GoogleBigtable的开源实现，它利用HadoopHDFS作为其文件存储系统，利用HadoopMapReduce来处理HBase中的海量数据，利用Zookeeper作为协同服务。HBase的表结构HBase以表的形式存储数据。表有行和列组成。列划分为
Java中的大数据处理框架对比分析省赚客app开发者 java 开发语言
Java中的大数据处理框架对比分析大家好，我是微赚淘客系统3.0的小编，是个冬天不穿秋裤，天冷也要风度的程序猿！今天，我们将深入探讨Java中常用的大数据处理框架，并对它们进行对比分析。大数据处理框架是现代数据驱动应用的核心，它们帮助企业处理和分析海量数据，以提取有价值的信息。本文将重点介绍ApacheHadoop、ApacheSpark、ApacheFlink和ApacheStorm这四种流行的
Hadoop windows intelij 跑 MR WordCount piziyang12138
一、软件环境我使用的软件版本如下:IntellijIdea2017.1Maven3.3.9Hadoop分布式环境二、创建maven工程打开Idea,file->new->Project,左侧面板选择maven工程。(如果只跑MapReduce创建java工程即可，不用勾选Creatfromarchetype，如果想创建web工程或者使用骨架可以勾选)image.png设置GroupId和Artif
Hadoop学习第三课（HDFS架构--读、写流程）小小程序员呀~ 数据库 hadoop 架构 big data
1.块概念举例1：一桶水1000ml，瓶子的规格100ml=>需要10个瓶子装完一桶水1010ml，瓶子的规格100ml=>需要11个瓶子装完一桶水1010ml，瓶子的规格200ml=>需要6个瓶子装完块的大小规格，只要是需要存储，哪怕一点点，也是要占用一个块的块大小的参数：dfs.blocksize官方默认的大小为128M官网：https://hadoop.apache.org/docs/r3.
hadoop启动HDFS命令 m0_67401228 java 搜索引擎 linux 后端
启动命令：/hadoop/sbin/start-dfs.sh停止命令：/hadoop/sbin/stop-dfs.sh
【计算机毕设-大数据方向】基于Hadoop的电商交易数据分析可视化系统的设计与实现程序员-石头山大数据实战案例大数据 hadoop 毕业设计毕设
博主介绍：✌全平台粉丝5W+,高级大厂开发程序员，博客之星、掘金/知乎/华为云/阿里云等平台优质作者。【源码获取】关注并且私信我【联系方式】最下边感兴趣的可以先收藏起来，同学门有不懂的毕设选题，项目以及论文编写等相关问题都可以和学长沟通，希望帮助更多同学解决问题前言随着电子商务行业的迅猛发展，电商平台积累了海量的数据资源，这些数据不仅包括用户的基本信息、购物记录，还包括用户的浏览行为、评价反馈等多
分布式离线计算—Spark—基础介绍测试开发abbey 人工智能—大数据
原文作者：饥渴的小苹果原文地址：【Spark】Spark基础教程目录Spark特点Spark相对于Hadoop的优势Spark生态系统Spark基本概念Spark结构设计Spark各种概念之间的关系Executor的优点Spark运行基本流程Spark运行架构的特点Spark的部署模式Spark三种部署方式Hadoop和Spark的统一部署摘要：Spark是基于内存计算的大数据并行计算框架Spar
spark常用命令我是浣熊的微笑 spark
查看报错日志：yarnlogsapplicationIDspark2-submit--masteryarn--classcom.hik.ReadHdfstest-1.0-SNAPSHOT.jar进入$SPARK_HOME目录，输入bin/spark-submit--help可以得到该命令的使用帮助。hadoop@wyy:/app/hadoop/spark100$bin/spark-submit--
spark启动命令学不会又听不懂 spark 大数据分布式
hadoop启动：cd/root/toolssstart-dfs.sh，只需在hadoop01上启动stop-dfs.sh日志查看：cat/root/toolss/hadoop/logs/hadoop-root-datanode-hadoop03.outzookeeper启动：cd/root/toolss/zookeeperbin/zkServer.shstart，三台都要启动bin/zkServ
编程常用命令总结 Yellow0523 Linux BigData 大数据
编程命令大全1.软件环境变量的配置JavaScalaSparkHadoopHive2.大数据软件常用命令Spark基本命令Spark-SQL命令Hive命令HDFS命令YARN命令Zookeeper命令kafka命令Hibench命令MySQL命令3.Linux常用命令Git命令conda命令pip命令查看Linux系统的详细信息查看Linux系统架构(X86还是ARM，两种方法都可)端口号命令L
Hadoop常见面试题整理及解答叶青舟 Linux hdfs 大数据 hadoop linux
Hadoop常见面试题整理及解答一、基础知识篇：1.把数据仓库从传统关系型数据库转到hadoop有什么优势？答：（1）关系型数据库成本高，且存储空间有限。而Hadoop使用较为廉价的机器存储数据，且Hadoop可以将大量机器构建成一个集群，并在集群中使用HDFS文件系统统一管理数据，极大的提高了数据的存储及处理能力。（2）关系型数据库仅支持标准结构化数据格式，Hadoop不仅支持标准结构化数据格式
2025毕业设计指南：如何用Hadoop构建超市进货推荐系统？大数据分析助力精准采购计算机编程指导师 Java实战集 Python实战集大数据实战集课程设计 hadoop 数据分析 spring boot java 进货 python
✍✍计算机编程指导师⭐⭐个人介绍：自己非常喜欢研究技术问题！专业做Java、Python、小程序、安卓、大数据、爬虫、Golang、大屏等实战项目。⛽⛽实战项目：有源码或者技术上的问题欢迎在评论区一起讨论交流！⚡⚡Java实战|SpringBoot/SSMPython实战项目|Django微信小程序/安卓实战项目大数据实战项目⚡⚡文末获取源码文章目录⚡⚡文末获取源码基于hadoop的超市进货推荐系
Hadoop Common 之序列化机制小解猫君之上 #Apache Hadoop
1.JavaSerializable序列化该序列化通过ObjectInputStream的readObject实现序列化，ObjectOutputStream的writeObject实现反序列化。这不过此种序列化虽然跨病态兼容性强，但是因为存储过多的信息，但是传输效率比较低，所以hadoop弃用它。（序列化信息包括这个对象的类，类签名，类的所有静态，费静态成员的值，以及他们父类都要被写入）publ
深入理解hadoop(一)----Common的实现----Configuration maoxiao_jsd 深入理解----hadoop
属本人个人原创，转载请注明,希望对大家有帮助！！一,hadoop的配置管理a,hadoop通过独有的Configuration处理配置信息Configurationconf=newConfiguration();conf.addResource("core-default.xml");conf.addResource("core-site.xml");后者会覆盖前者中未final标记的相同配置项b
hadoop 0.22.0 部署笔记 weixin_33701564 大数据 java 运维
为什么80%的码农都做不了架构师？>>>因为需要使用hbase，所以开始对hbase进行学习。hbase是部署在hadoop平台上的NOSql数据库，因此在部署hbase之前需要先部署hadoop。环境：redhat5、hadoop-0.22.0.tar.gz、jdk-6u13-linux-i586.zipip192.168.1.128hostname：localhost.localdomain（
解决Windows环境下hadoop集群的运行_window运行hadoop,unknown hadoop01(4) 2401_84160087 大数据面试学习
网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是浅尝辄止，不再深入研究，那么很难做到真正的技术提升。需要这份系统化资料的朋友，可以戳这里获取一个人可以走的很快，但一群人才能走的更远！不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人，都欢迎加入我们的的圈子（技术交流、学习资源、职场吐槽、大厂内推、面试辅导），让我们一起学习成长！org.apache.hadoophadoop-com
解决Windows环境下hadoop集群的运行_window运行hadoop,unknown hadoop01(3) 2401_84160087 大数据面试学习
网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是浅尝辄止，不再深入研究，那么很难做到真正的技术提升。需要这份系统化资料的朋友，可以戳这里获取一个人可以走的很快，但一群人才能走的更远！不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人，都欢迎加入我们的的圈子（技术交流、学习资源、职场吐槽、大厂内推、面试辅导），让我们一起学习成长！xmlns:xsi="http://www.w3.or
深入解析HDFS：定义、架构、原理、应用场景及常用命令 CloudJourney hdfs 架构 hadoop
引言Hadoop分布式文件系统（HDFS，HadoopDistributedFileSystem）是Hadoop框架的核心组件之一，它提供了高可靠性、高可用性和高吞吐量的大规模数据存储和管理能力。本文将从HDFS的定义、架构、工作原理、应用场景以及常用命令等多个方面进行详细探讨，帮助读者全面深入地了解HDFS。1.HDFS的定义1.1什么是HDFSHDFS是Hadoop生态系统中的一个分布式文件系
Hadoop的搭建流程 lzhlizihang hadoop 大数据分布式
文章目录一、配置IP二、配置主机名三、配置主机映射四、关闭防火墙五、配置免密六、安装jdk1、第一步：2、第二步：3、第三步：4、第四步：5、第五步：七、安装hadoop1、上传2、解压3、重命名4、开始配置环境变量5、刷新配置文件6、验证hadoop命令是否可以识别八、全分布搭建7、修改配置文件core-site.xml8、修改配置文件hdfs-site.xml9、修改配置文件hadoop-en
hive搭建 -----内嵌模式和本地模式 lzhlizihang hive hadoop
文章目录一、内嵌模式（使用较少）1、上传、解压、重命名2、配置环境变量3、配置conf下的hive-env.sh4、修改conf下的hive-site.xml5、启动hadoop集群6、给hdfs创建文件夹7、修改hive-site.xml中的非法字符8、初始化元数据9、测试是否成功10、内嵌模式的缺点二、本地模式（最常用）1、检查mysql是否正常2、上传、解压、重命名3、配置环境变量4、修改c
Hadoop之mapreduce -- WrodCount案例以及各种概念 lzhlizihang hadoop mapreduce 大数据
文章目录一、MapReduce的优缺点二、MapReduce案例--WordCount1、导包2、Mapper方法3、Partitioner方法（自定义分区器）4、reducer方法5、driver（main方法）6、Writable（手机流量统计案例的实体类）三、关于片和块1、什么是片，什么是块？2、mapreduce启动多少个MapTask任务？四、MapReduce的原理五、Shuffle过
IAAS: IT公司去IOE-Alibaba系统构架解读 wishchin 心理学/职业 BigDataMini Spark PaaS
从Hadoop到自主研发，技术解读阿里去IOE后的系统架构原地址：......................云计算阿里飞天摘要：从IOE时代，到Hadoop与飞天并行，再到飞天单集群5000节点的实现，阿里一直摸索在技术衍变的前沿。这里，我们将从架构、性能、运维等多个方面深入了解阿里基础设施。【导读】互联网的普及，智能终端的增加，大数据时代悄然而至。在这个数据为王的时代，数十倍、数百倍的数据给各
HttpClient 4.3与4.3版本以下版本比较 spjich java httpclient
网上利用java发送http请求的代码很多，一搜一大把，有的利用的是java.net.*下的HttpURLConnection，有的用httpclient，而且发送的代码也分门别类。今天我们主要来说的是利用httpclient发送请求。 httpclient又可分为 httpclient3.x httpclient4.x到httpclient4.3以下 httpclient4.3
Essential Studio Enterprise Edition 2015 v1新功能体验 Axiba .net
概述：Essential Studio已全线升级至2015 v1版本了！新版本为JavaScript和ASP.NET MVC添加了新的文件资源管理器控件，还有其他一些控件功能升级，精彩不容错过，让我们一起来看看吧！ syncfusion公司是世界领先的Windows开发组件提供商，该公司正式对外发布Essential Studio Enterprise Edition 2015 v1版本。新版本
[宇宙与天文]微波背景辐射值与地球温度 comsci 背景
宇宙这个庞大,无边无际的空间是否存在某种确定的,变化的温度呢? 如果宇宙微波背景辐射值是表示宇宙空间温度的参数之一,那么测量这些数值,并观测周围的恒星能量输出值,我们是否获得地球的长期气候变化的情况呢? &nbs
lvs-server 男人50 server
#!/bin/bash # # LVS script for VS/DR # #./etc/rc.d/init.d/functions # VIP=10.10.6.252 RIP1=10.10.6.101 RIP2=10.10.6.13 PORT=80 case $1 in start) /sbin/ifconfig eth2:0 $VIP broadca
java的WebCollector爬虫框架 oloz 爬虫
WebCollector主页： https://github.com/CrawlScript/WebCollector 下载：webcollector-版本号-bin.zip将解压后文件夹中的所有jar包添加到工程既可。接下来看demo package org.spider.myspider; import cn.edu.hfut.dmic.webcollector.cra
jQuery append 与 after 的区别小猪猪08
1、after函数定义和用法： after() 方法在被选元素后插入指定的内容。语法： $(selector).after(content) 实例： <html> <head> <script type="text/javascript" src="/jquery/jquery.js"></scr
mysql知识充电香水浓 mysql
索引索引是在存储引擎中实现的，因此每种存储引擎的索引都不一定完全相同，并且每种存储引擎也不一定支持所有索引类型。根据存储引擎定义每个表的最大索引数和最大索引长度。所有存储引擎支持每个表至少16个索引，总索引长度至少为256字节。大多数存储引擎有更高的限制。MYSQL中索引的存储类型有两种：BTREE和HASH，具体和表的存储引擎相关； MYISAM和InnoDB存储引擎
我的架构经验系列文章索引 agevs 架构
下面是一些个人架构上的总结，本来想只在公司内部进行共享的，因此内容写的口语化一点，也没什么图示，所有内容没有查任何资料是脑子里面的东西吐出来的因此可能会不准确不全，希望抛砖引玉，大家互相讨论。要注意，我这些文章是一个总体的架构经验不针对具体的语言和平台，因此也不一定是适用所有的语言和平台的。（内容是前几天写的，现附上索引）前端架构 http://www.
Android so lib库远程http下载和动态注册 aijuans andorid
一、背景在开发Android应用程序的实现，有时候需要引入第三方so lib库，但第三方so库比较大，例如开源第三方播放组件ffmpeg库, 如果直接打包的apk包里面, 整个应用程序会大很多.经过查阅资料和实验，发现通过远程下载so文件，然后再动态注册so文件时可行的。主要需要解决下载so文件存放位置以及文件读写权限问题。二、主要
linux中svn配置出错 conf/svnserve.conf:12: Option expected 解决方法 baalwolf option
在客户端访问subversion版本库时出现这个错误： svnserve.conf:12: Option expected 为什么会出现这个错误呢，就是因为subversion读取配置文件svnserve.conf时，无法识别有前置空格的配置文件，如### This file controls the configuration of the svnserve daemon, if you##
MongoDB的连接池和连接管理 BigCat2013 mongodb
在关系型数据库中，我们总是需要关闭使用的数据库连接，不然大量的创建连接会导致资源的浪费甚至于数据库宕机。这篇文章主要想解释一下mongoDB的连接池以及连接管理机制，如果正对此有疑惑的朋友可以看一下。通常我们习惯于new 一个connection并且通常在finally语句中调用connection的close()方法将其关闭。正巧，mongoDB中当我们new一个Mongo的时候，会发现它也
AngularJS使用Socket.IO bijian1013 JavaScript AngularJS Socket.IO
目前，web应用普遍被要求是实时web应用，即服务端的数据更新之后，应用能立即更新。以前使用的技术（例如polling）存在一些局限性，而且有时我们需要在客户端打开一个socket，然后进行通信。 Socket.IO(http://socket.io/)是一个非常优秀的库，它可以帮你实
[Maven学习笔记四]Maven依赖特性 bit1129 maven
三个模块为了说明问题，以用户登陆小web应用为例。通常一个web应用分为三个模块，模型和数据持久化层user-core, 业务逻辑层user-service以及web展现层user-web， user-service依赖于user-core user-web依赖于user-core和user-service 依赖作用范围 Maven的dependency定义
【Akka一】Akka入门 bit1129 akka
什么是Akka Message-Driven Runtime is the Foundation to Reactive Applications In Akka, your business logic is driven through message-based communication patterns that are independent of physical locatio
zabbix_api之perl语言写法 ronin47 zabbix_api之perl
zabbix_api网上比较多的写法是python或curl。上次我用java－－http://bossr.iteye.com/blog/2195679，这次用perl。for example: #!/usr/bin/perl use 5.010 ; use strict ; use warnings ; use JSON :: RPC :: Client ; use
比优衣库跟牛掰的视频流出了，兄弟连Linux运维工程师课堂实录，更加刺激，更加实在！ brotherlamp linux运维工程师 linux运维工程师教程 linux运维工程师视频 linux运维工程师资料 linux运维工程师自学
比优衣库跟牛掰的视频流出了，兄弟连Linux运维工程师课堂实录，更加刺激，更加实在！ ----------------------------------------------------- 兄弟连Linux运维工程师课堂实录-计算机基础-1-课程体系介绍1 链接：http://pan.baidu.com/s/1i3GQtGL 密码：bl65 兄弟连Lin
bitmap求哈密顿距离-给定N（1<=N<=100000）个五维的点A(x1,x2,x3,x4,x5)，求两个点X(x1,x2,x3,x4,x5)和Y( bylijinnan java
import java.util.Random; /** * 题目： * 给定N（1<=N<=100000）个五维的点A(x1,x2,x3,x4,x5)，求两个点X(x1,x2,x3,x4,x5)和Y(y1,y2,y3,y4,y5)， * 使得他们的哈密顿距离（d=|x1-y1| + |x2-y2| + |x3-y3| + |x4-y4| + |x5-y5|）最大
map的三种遍历方法 chicony map
package com.test; import java.util.Collection; import java.util.HashMap; import java.util.Iterator; import java.util.Map; import java.util.Set; public class TestMap { public static v
Linux安装mysql的一些坑 chenchao051 linux
1、mysql不建议在root用户下运行 2、出现服务启动不了，111错误，注意要用chown来赋予权限，我在root用户下装的mysql，我就把usr/share/mysql/mysql.server复制到/etc/init.d/mysqld, (同时把my-huge.cnf复制/etc/my.cnf) chown -R cc /etc/init.d/mysql
Sublime Text 3 配置 daizj 配置 Sublime Text
Sublime Text 3 配置解释(默认){// 设置主题文件“color_scheme”: “Packages/Color Scheme – Default/Monokai.tmTheme”,// 设置字体和大小“font_face”: “Consolas”,“font_size”: 12,// 字体选项：no_bold不显示粗体字，no_italic不显示斜体字，no_antialias和
MySQL server has gone away 问题的解决方法 dcj3sjt126com SQL Server
MySQL server has gone away 问题解决方法，需要的朋友可以参考下。应用程序（比如PHP）长时间的执行批量的MYSQL语句。执行一个SQL，但SQL语句过大或者语句中含有BLOB或者longblob字段。比如，图片数据的处理。都容易引起MySQL server has gone away。今天遇到类似的情景，MySQL只是冷冷的说：MySQL server h
javascript/dom:固定居中效果 dcj3sjt126com JavaScript
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml&
使用 Spring 2.5 注释驱动的 IoC 功能 e200702084 spring bean 配置管理 IOC Office
使用 Spring 2.5 注释驱动的 IoC 功能 developerWorks 文档选项将打印机的版面设置成横向打印模式打印本页将此页作为电子邮件发送将此页作为电子邮件发送级别：初级陈雄华 ([email protected]), 技术总监, 宝宝淘网络科技有限公司 2008 年 2 月 28 日 &nb
MongoDB常用操作命令 geeksun mongodb
1. 基本操作 db.AddUser(username,password) 添加用户 db.auth(usrename,password) 设置数据库连接验证 db.cloneDataBase(fromhost)
php写守护进程（Daemon） hongtoushizi PHP
转载自： http://blog.csdn.net/tengzhaorong/article/details/9764655 守护进程（Daemon）是运行在后台的一种特殊进程。它独立于控制终端并且周期性地执行某种任务或等待处理某些发生的事件。守护进程是一种很有用的进程。php也可以实现守护进程的功能。 1、基本概念 &nbs
spring整合mybatis,关于注入Dao对象出错问题 jonsvien DAO spring bean mybatis prototype
今天在公司测试功能时发现一问题：先进行代码说明： 1，controller配置了Scope="prototype"（表明每一次请求都是原子型） @resource/@autowired service对象都可以（两种注解都可以）。 2，service 配置了Scope="prototype"（表明每一次请求都是原子型）
对象关系行为模式之标识映射 home198979 PHP 架构企业应用对象关系标识映射
HELLO!架构一、概念 identity Map:通过在映射中保存每个已经加载的对象，确保每个对象只加载一次，当要访问对象的时候，通过映射来查找它们。其实在数据源架构模式之数据映射器代码中有提及到标识映射，Mapper类的getFromMap方法就是实现标识映射的实现。二、为什么要使用标识映射？在数据源架构模式之数据映射器中 //c
Linux下hosts文件详解 pda158 linux
　1、主机名：　　无论在局域网还是INTERNET上，每台主机都有一个IP地址，是为了区分此台主机和彼台主机，也就是说IP地址就是主机的门牌号。　　公网：IP地址不方便记忆，所以又有了域名。域名只是在公网（INtERNET)中存在，每个域名都对应一个IP地址，但一个IP地址可有对应多个域名。　　局域网：每台机器都有一个主机名，用于主机与主机之间的便于区分，就可以为每台机器设置主机
nginx配置文件粗解 spjich java nginx
#运行用户#user nobody;#启动进程,通常设置成和cpu的数量相等worker_processes 2;#全局错误日志及PID文件#error_log logs/error.log;#error_log logs/error.log notice;#error_log logs/error.log inf
数学函数 w54653520 java
public class S { // 传入两个整数，进行比较，返回两个数中的最大值的方法。 public int get( int num1, int nu