mahout 运行Twenty Newsgroups Classification实例

按照mahout官网https://cwiki.apache.org/confluence/display/MAHOUT/Twenty+Newsgroups的说法，我只用运行一条命令就可以完成这个算法的调用了，如下：

mahout@ubuntu:~/mahout-d-0.7/examples/bin$ ./classify-20newsgroups.sh

但是，我首先运行就出错了，因为我不是root账户，所以先改下路径，打开classify-20newsgroups.sh，替换/tmp/mahout-work-为/home/mahout/mahout-work-，这样用户mahout就具有了操作权限，但是还是出错，提示curl 找不到命令，好吧，我没安装这个，sudo apt-get install curl，ok ，ubuntu还是方便呀。

然后再运行，结果运行到2/3时候还是出错，然后我查看详细信息，居然map输入的数据条数为0？啥意思？好吧，应该是本地文件操作和HDFS文件操作混淆了，其实在执行：

+ ./bin/mahout seqdirectory -i /home/mahout/mahout-work-mahout/20news-all -o /home/mahout/mahout-work-mahout/20news-seq

这一步前应该把本地的20news-all上传到HDFS文件系统上面，然后重新执行第一条命令即可，全部信息如下（太多了，不知道贴的完不？）：

mahout@ubuntu:~/mahout-d-0.7/examples/bin$ ./classify-20newsgroups.sh 
Please select a number to choose the corresponding task to run
1. cnaivebayes
2. naivebayes
3. sgd
4. clean -- cleans up the work area in /home/mahout/mahout-work-mahout
Enter your choice : 2
ok. You chose 2 and we'll use naivebayes
creating work directory at /home/mahout/mahout-work-mahout
+ echo 'Preparing 20newsgroups data'
Preparing 20newsgroups data
+ rm -rf /home/mahout/mahout-work-mahout/20news-all
+ mkdir /home/mahout/mahout-work-mahout/20news-all
+ cp -R /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/alt.atheism /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/comp.graphics /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/comp.os.ms-windows.misc /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/comp.sys.ibm.pc.hardware /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/comp.sys.mac.hardware /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/comp.windows.x /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/misc.forsale /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/rec.autos /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/rec.motorcycles /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/rec.sport.baseball /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/rec.sport.hockey /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/sci.crypt /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/sci.electronics /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/sci.med /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/sci.space /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/soc.religion.christian /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/talk.politics.guns /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/talk.politics.mideast /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/talk.politics.misc /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-test/talk.religion.misc /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/alt.atheism /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/comp.graphics /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/comp.os.ms-windows.misc /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/comp.sys.ibm.pc.hardware /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/comp.sys.mac.hardware /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/comp.windows.x /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/misc.forsale /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/rec.autos /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/rec.motorcycles /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/rec.sport.baseball /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/rec.sport.hockey /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/sci.crypt /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/sci.electronics /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/sci.med /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/sci.space /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/soc.religion.christian /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/talk.politics.guns /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/talk.politics.mideast /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/talk.politics.misc /home/mahout/mahout-work-mahout/20news-bydate/20news-bydate-train/talk.religion.misc /home/mahout/mahout-work-mahout/20news-all
+ echo 'Creating sequence files from 20newsgroups data'
Creating sequence files from 20newsgroups data
+ ./bin/mahout seqdirectory -i /home/mahout/mahout-work-mahout/20news-all -o /home/mahout/mahout-work-mahout/20news-seq
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/mahout/hadoop-1.0.4/bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/mahout/mahout-d-0.7/mahout-examples-0.7-job.jar
Warning: $HADOOP_HOME is deprecated.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/mahout-examples-0.7-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
13/08/26 23:38:49 INFO common.AbstractJob: Command line arguments: {--charset=[UTF-8], --chunkSize=[64], --endPhase=[2147483647], --fileFilterClass=[org.apache.mahout.text.PrefixAdditionFilter], --input=[/home/mahout/mahout-work-mahout/20news-all], --keyPrefix=[], --output=[/home/mahout/mahout-work-mahout/20news-seq], --startPhase=[0], --tempDir=[temp]}
13/08/26 23:42:57 INFO driver.MahoutDriver: Program took 248530 ms (Minutes: 4.142166666666666)
+ echo 'Converting sequence files to vectors'
Converting sequence files to vectors
+ ./bin/mahout seq2sparse -i /home/mahout/mahout-work-mahout/20news-seq -o /home/mahout/mahout-work-mahout/20news-vectors -lnorm -nv -wt tfidf
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/mahout/hadoop-1.0.4/bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/mahout/mahout-d-0.7/mahout-examples-0.7-job.jar
Warning: $HADOOP_HOME is deprecated.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/mahout-examples-0.7-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
13/08/26 23:43:13 INFO vectorizer.SparseVectorsFromSequenceFiles: Maximum n-gram size is: 1
13/08/26 23:43:13 INFO vectorizer.SparseVectorsFromSequenceFiles: Minimum LLR value: 1.0
13/08/26 23:43:13 INFO vectorizer.SparseVectorsFromSequenceFiles: Number of reduce tasks: 1
13/08/26 23:43:17 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:43:17 INFO mapred.JobClient: Running job: job_201308212334_0056
13/08/26 23:43:18 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:43:45 INFO mapred.JobClient:  map 78% reduce 0%
13/08/26 23:43:51 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:43:56 INFO mapred.JobClient: Job complete: job_201308212334_0056
13/08/26 23:43:56 INFO mapred.JobClient: Counters: 19
13/08/26 23:43:56 INFO mapred.JobClient:   Job Counters 
13/08/26 23:43:56 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=32883
13/08/26 23:43:56 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:43:56 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:43:56 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:43:56 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:43:56 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
13/08/26 23:43:56 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:43:56 INFO mapred.JobClient:     Bytes Written=27503580
13/08/26 23:43:56 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:43:56 INFO mapred.JobClient:     HDFS_BYTES_READ=36694022
13/08/26 23:43:56 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=21899
13/08/26 23:43:56 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=27503580
13/08/26 23:43:56 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:43:56 INFO mapred.JobClient:     Bytes Read=36693889
13/08/26 23:43:56 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:43:56 INFO mapred.JobClient:     Map input records=18846
13/08/26 23:43:56 INFO mapred.JobClient:     Physical memory (bytes) snapshot=75157504
13/08/26 23:43:56 INFO mapred.JobClient:     Spilled Records=0
13/08/26 23:43:56 INFO mapred.JobClient:     CPU time spent (ms)=5730
13/08/26 23:43:56 INFO mapred.JobClient:     Total committed heap usage (bytes)=15859712
13/08/26 23:43:56 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=974381056
13/08/26 23:43:56 INFO mapred.JobClient:     Map output records=18846
13/08/26 23:43:56 INFO mapred.JobClient:     SPLIT_RAW_BYTES=133
13/08/26 23:43:56 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:43:56 INFO mapred.JobClient: Running job: job_201308212334_0057
13/08/26 23:43:57 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:44:15 INFO mapred.JobClient:  map 3% reduce 0%
13/08/26 23:44:18 INFO mapred.JobClient:  map 23% reduce 0%
13/08/26 23:44:21 INFO mapred.JobClient:  map 60% reduce 0%
13/08/26 23:44:24 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:44:48 INFO mapred.JobClient:  map 100% reduce 100%
13/08/26 23:44:53 INFO mapred.JobClient: Job complete: job_201308212334_0057
13/08/26 23:44:53 INFO mapred.JobClient: Counters: 29
13/08/26 23:44:53 INFO mapred.JobClient:   Job Counters 
13/08/26 23:44:53 INFO mapred.JobClient:     Launched reduce tasks=1
13/08/26 23:44:53 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=31312
13/08/26 23:44:53 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:44:53 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:44:53 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:44:53 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:44:53 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=18422
13/08/26 23:44:53 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:44:53 INFO mapred.JobClient:     Bytes Written=2315037
13/08/26 23:44:53 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:44:53 INFO mapred.JobClient:     FILE_BYTES_READ=11857906
13/08/26 23:44:53 INFO mapred.JobClient:     HDFS_BYTES_READ=27503742
13/08/26 23:44:53 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=15440401
13/08/26 23:44:53 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=2315037
13/08/26 23:44:53 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:44:53 INFO mapred.JobClient:     Bytes Read=27503580
13/08/26 23:44:53 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:44:53 INFO mapred.JobClient:     Map output materialized bytes=3538084
13/08/26 23:44:53 INFO mapred.JobClient:     Map input records=18846
13/08/26 23:44:53 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/08/26 23:44:53 INFO mapred.JobClient:     Spilled Records=849345
13/08/26 23:44:53 INFO mapred.JobClient:     Map output bytes=39462740
13/08/26 23:44:53 INFO mapred.JobClient:     Total committed heap usage (bytes)=176033792
13/08/26 23:44:53 INFO mapred.JobClient:     CPU time spent (ms)=14080
13/08/26 23:44:53 INFO mapred.JobClient:     Combine input records=3026242
13/08/26 23:44:53 INFO mapred.JobClient:     SPLIT_RAW_BYTES=162
13/08/26 23:44:53 INFO mapred.JobClient:     Reduce input records=192904
13/08/26 23:44:53 INFO mapred.JobClient:     Reduce input groups=192904
13/08/26 23:44:53 INFO mapred.JobClient:     Combine output records=554873
13/08/26 23:44:53 INFO mapred.JobClient:     Physical memory (bytes) snapshot=283111424
13/08/26 23:44:53 INFO mapred.JobClient:     Reduce output records=93563
13/08/26 23:44:53 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1957584896
13/08/26 23:44:53 INFO mapred.JobClient:     Map output records=2664273
13/08/26 23:44:54 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:44:55 INFO mapred.JobClient: Running job: job_201308212334_0058
13/08/26 23:44:56 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:45:13 INFO mapred.JobClient:  map 94% reduce 0%
13/08/26 23:45:16 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:45:43 INFO mapred.JobClient:  map 100% reduce 100%
13/08/26 23:45:48 INFO mapred.JobClient: Job complete: job_201308212334_0058
13/08/26 23:45:48 INFO mapred.JobClient: Counters: 29
13/08/26 23:45:48 INFO mapred.JobClient:   Job Counters 
13/08/26 23:45:48 INFO mapred.JobClient:     Launched reduce tasks=1
13/08/26 23:45:48 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=21298
13/08/26 23:45:48 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:45:48 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:45:48 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:45:48 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:45:48 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=24763
13/08/26 23:45:48 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:45:48 INFO mapred.JobClient:     Bytes Written=29314118
13/08/26 23:45:48 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:45:48 INFO mapred.JobClient:     FILE_BYTES_READ=27274291
13/08/26 23:45:48 INFO mapred.JobClient:     HDFS_BYTES_READ=29440826
13/08/26 23:45:48 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=54595105
13/08/26 23:45:48 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=29314118
13/08/26 23:45:48 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:45:48 INFO mapred.JobClient:     Bytes Read=27503580
13/08/26 23:45:48 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:45:48 INFO mapred.JobClient:     Map output materialized bytes=27274291
13/08/26 23:45:48 INFO mapred.JobClient:     Map input records=18846
13/08/26 23:45:48 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/08/26 23:45:48 INFO mapred.JobClient:     Spilled Records=37692
13/08/26 23:45:48 INFO mapred.JobClient:     Map output bytes=27199343
13/08/26 23:45:48 INFO mapred.JobClient:     Total committed heap usage (bytes)=215695360
13/08/26 23:45:48 INFO mapred.JobClient:     CPU time spent (ms)=12980
13/08/26 23:45:48 INFO mapred.JobClient:     Combine input records=0
13/08/26 23:45:48 INFO mapred.JobClient:     SPLIT_RAW_BYTES=162
13/08/26 23:45:48 INFO mapred.JobClient:     Reduce input records=18846
13/08/26 23:45:48 INFO mapred.JobClient:     Reduce input groups=18846
13/08/26 23:45:48 INFO mapred.JobClient:     Combine output records=0
13/08/26 23:45:48 INFO mapred.JobClient:     Physical memory (bytes) snapshot=332349440
13/08/26 23:45:48 INFO mapred.JobClient:     Reduce output records=18846
13/08/26 23:45:48 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1957584896
13/08/26 23:45:48 INFO mapred.JobClient:     Map output records=18846
13/08/26 23:45:49 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:45:49 INFO mapred.JobClient: Running job: job_201308212334_0059
13/08/26 23:45:50 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:46:10 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:46:25 INFO mapred.JobClient:  map 100% reduce 92%
13/08/26 23:46:31 INFO mapred.JobClient:  map 100% reduce 100%
13/08/26 23:46:36 INFO mapred.JobClient: Job complete: job_201308212334_0059
13/08/26 23:46:36 INFO mapred.JobClient: Counters: 29
13/08/26 23:46:36 INFO mapred.JobClient:   Job Counters 
13/08/26 23:46:36 INFO mapred.JobClient:     Launched reduce tasks=1
13/08/26 23:46:36 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=18217
13/08/26 23:46:36 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:46:36 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:46:36 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:46:36 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:46:36 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=20981
13/08/26 23:46:36 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:46:36 INFO mapred.JobClient:     Bytes Written=29314118
13/08/26 23:46:36 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:46:36 INFO mapred.JobClient:     FILE_BYTES_READ=29059398
13/08/26 23:46:36 INFO mapred.JobClient:     HDFS_BYTES_READ=29314278
13/08/26 23:46:36 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=58163419
13/08/26 23:46:36 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=29314118
13/08/26 23:46:36 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:46:36 INFO mapred.JobClient:     Bytes Read=29314118
13/08/26 23:46:36 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:46:36 INFO mapred.JobClient:     Map output materialized bytes=29059398
13/08/26 23:46:36 INFO mapred.JobClient:     Map input records=18846
13/08/26 23:46:36 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/08/26 23:46:36 INFO mapred.JobClient:     Spilled Records=37692
13/08/26 23:46:36 INFO mapred.JobClient:     Map output bytes=28984080
13/08/26 23:46:36 INFO mapred.JobClient:     Total committed heap usage (bytes)=205225984
13/08/26 23:46:36 INFO mapred.JobClient:     CPU time spent (ms)=8650
13/08/26 23:46:36 INFO mapred.JobClient:     Combine input records=0
13/08/26 23:46:37 INFO mapred.JobClient:     SPLIT_RAW_BYTES=160
13/08/26 23:46:37 INFO mapred.JobClient:     Reduce input records=18846
13/08/26 23:46:37 INFO mapred.JobClient:     Reduce input groups=18846
13/08/26 23:46:37 INFO mapred.JobClient:     Combine output records=0
13/08/26 23:46:37 INFO mapred.JobClient:     Physical memory (bytes) snapshot=313606144
13/08/26 23:46:37 INFO mapred.JobClient:     Reduce output records=18846
13/08/26 23:46:37 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1957584896
13/08/26 23:46:37 INFO mapred.JobClient:     Map output records=18846
13/08/26 23:46:37 INFO common.HadoopUtil: Deleting /home/mahout/mahout-work-mahout/20news-vectors/partial-vectors-0
13/08/26 23:46:37 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:46:37 INFO mapred.JobClient: Running job: job_201308212334_0060
13/08/26 23:46:38 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:46:56 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:47:14 INFO mapred.JobClient:  map 100% reduce 100%
13/08/26 23:47:19 INFO mapred.JobClient: Job complete: job_201308212334_0060
13/08/26 23:47:19 INFO mapred.JobClient: Counters: 29
13/08/26 23:47:19 INFO mapred.JobClient:   Job Counters 
13/08/26 23:47:19 INFO mapred.JobClient:     Launched reduce tasks=1
13/08/26 23:47:19 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=21504
13/08/26 23:47:19 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:47:19 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:47:19 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:47:19 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:47:19 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=14273
13/08/26 23:47:19 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:47:19 INFO mapred.JobClient:     Bytes Written=1890073
13/08/26 23:47:19 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:47:19 INFO mapred.JobClient:     FILE_BYTES_READ=4880788
13/08/26 23:47:19 INFO mapred.JobClient:     HDFS_BYTES_READ=29314271
13/08/26 23:47:19 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=6235019
13/08/26 23:47:19 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=1890073
13/08/26 23:47:19 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:47:19 INFO mapred.JobClient:     Bytes Read=29314118
13/08/26 23:47:19 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:47:19 INFO mapred.JobClient:     Map output materialized bytes=1309902
13/08/26 23:47:19 INFO mapred.JobClient:     Map input records=18846
13/08/26 23:47:19 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/08/26 23:47:19 INFO mapred.JobClient:     Spilled Records=442187
13/08/26 23:47:19 INFO mapred.JobClient:     Map output bytes=31005336
13/08/26 23:47:19 INFO mapred.JobClient:     Total committed heap usage (bytes)=176033792
13/08/26 23:47:19 INFO mapred.JobClient:     CPU time spent (ms)=9210
13/08/26 23:47:19 INFO mapred.JobClient:     Combine input records=2838837
13/08/26 23:47:19 INFO mapred.JobClient:     SPLIT_RAW_BYTES=153
13/08/26 23:47:19 INFO mapred.JobClient:     Reduce input records=93564
13/08/26 23:47:19 INFO mapred.JobClient:     Reduce input groups=93564
13/08/26 23:47:19 INFO mapred.JobClient:     Combine output records=348623
13/08/26 23:47:19 INFO mapred.JobClient:     Physical memory (bytes) snapshot=284684288
13/08/26 23:47:19 INFO mapred.JobClient:     Reduce output records=93564
13/08/26 23:47:19 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1957584896
13/08/26 23:47:19 INFO mapred.JobClient:     Map output records=2583778
13/08/26 23:47:19 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:47:19 INFO mapred.JobClient: Running job: job_201308212334_0061
13/08/26 23:47:20 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:47:38 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:47:53 INFO mapred.JobClient:  map 100% reduce 67%
13/08/26 23:47:59 INFO mapred.JobClient:  map 100% reduce 100%
13/08/26 23:48:04 INFO mapred.JobClient: Job complete: job_201308212334_0061
13/08/26 23:48:04 INFO mapred.JobClient: Counters: 29
13/08/26 23:48:04 INFO mapred.JobClient:   Job Counters 
13/08/26 23:48:04 INFO mapred.JobClient:     Launched reduce tasks=1
13/08/26 23:48:04 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=18292
13/08/26 23:48:04 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:48:04 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:48:04 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:48:04 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:48:04 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=19293
13/08/26 23:48:04 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:48:04 INFO mapred.JobClient:     Bytes Written=28689283
13/08/26 23:48:04 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:48:04 INFO mapred.JobClient:     FILE_BYTES_READ=29059398
13/08/26 23:48:04 INFO mapred.JobClient:     HDFS_BYTES_READ=31204324
13/08/26 23:48:04 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=58165045
13/08/26 23:48:04 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=28689283
13/08/26 23:48:04 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:48:04 INFO mapred.JobClient:     Bytes Read=29314118
13/08/26 23:48:04 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:48:04 INFO mapred.JobClient:     Map output materialized bytes=29059398
13/08/26 23:48:04 INFO mapred.JobClient:     Map input records=18846
13/08/26 23:48:04 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/08/26 23:48:04 INFO mapred.JobClient:     Spilled Records=37692
13/08/26 23:48:04 INFO mapred.JobClient:     Map output bytes=28984080
13/08/26 23:48:04 INFO mapred.JobClient:     Total committed heap usage (bytes)=205225984
13/08/26 23:48:04 INFO mapred.JobClient:     CPU time spent (ms)=8770
13/08/26 23:48:04 INFO mapred.JobClient:     Combine input records=0
13/08/26 23:48:04 INFO mapred.JobClient:     SPLIT_RAW_BYTES=153
13/08/26 23:48:04 INFO mapred.JobClient:     Reduce input records=18846
13/08/26 23:48:04 INFO mapred.JobClient:     Reduce input groups=18846
13/08/26 23:48:04 INFO mapred.JobClient:     Combine output records=0
13/08/26 23:48:04 INFO mapred.JobClient:     Physical memory (bytes) snapshot=320401408
13/08/26 23:48:04 INFO mapred.JobClient:     Reduce output records=18846
13/08/26 23:48:04 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1957584896
13/08/26 23:48:04 INFO mapred.JobClient:     Map output records=18846
13/08/26 23:48:05 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:48:05 INFO mapred.JobClient: Running job: job_201308212334_0062
13/08/26 23:48:06 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:48:24 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:48:36 INFO mapred.JobClient:  map 100% reduce 33%
13/08/26 23:48:39 INFO mapred.JobClient:  map 100% reduce 86%
13/08/26 23:48:48 INFO mapred.JobClient:  map 100% reduce 100%
13/08/26 23:48:53 INFO mapred.JobClient: Job complete: job_201308212334_0062
13/08/26 23:48:53 INFO mapred.JobClient: Counters: 29
13/08/26 23:48:53 INFO mapred.JobClient:   Job Counters 
13/08/26 23:48:53 INFO mapred.JobClient:     Launched reduce tasks=1
13/08/26 23:48:53 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=18225
13/08/26 23:48:53 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:48:53 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:48:53 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:48:53 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:48:53 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=21045
13/08/26 23:48:53 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:48:53 INFO mapred.JobClient:     Bytes Written=28689283
13/08/26 23:48:53 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:48:53 INFO mapred.JobClient:     FILE_BYTES_READ=28437750
13/08/26 23:48:53 INFO mapred.JobClient:     HDFS_BYTES_READ=28689443
13/08/26 23:48:53 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=56920127
13/08/26 23:48:53 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=28689283
13/08/26 23:48:53 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:48:53 INFO mapred.JobClient:     Bytes Read=28689283
13/08/26 23:48:53 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:48:53 INFO mapred.JobClient:     Map output materialized bytes=28437750
13/08/26 23:48:53 INFO mapred.JobClient:     Map input records=18846
13/08/26 23:48:53 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/08/26 23:48:53 INFO mapred.JobClient:     Spilled Records=37692
13/08/26 23:48:53 INFO mapred.JobClient:     Map output bytes=28362505
13/08/26 23:48:53 INFO mapred.JobClient:     Total committed heap usage (bytes)=204603392
13/08/26 23:48:53 INFO mapred.JobClient:     CPU time spent (ms)=8340
13/08/26 23:48:53 INFO mapred.JobClient:     Combine input records=0
13/08/26 23:48:53 INFO mapred.JobClient:     SPLIT_RAW_BYTES=160
13/08/26 23:48:53 INFO mapred.JobClient:     Reduce input records=18846
13/08/26 23:48:53 INFO mapred.JobClient:     Reduce input groups=18846
13/08/26 23:48:53 INFO mapred.JobClient:     Combine output records=0
13/08/26 23:48:53 INFO mapred.JobClient:     Physical memory (bytes) snapshot=313868288
13/08/26 23:48:53 INFO mapred.JobClient:     Reduce output records=18846
13/08/26 23:48:53 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1957584896
13/08/26 23:48:53 INFO mapred.JobClient:     Map output records=18846
13/08/26 23:48:53 INFO common.HadoopUtil: Deleting /home/mahout/mahout-work-mahout/20news-vectors/partial-vectors-0
13/08/26 23:48:53 INFO driver.MahoutDriver: Program took 339621 ms (Minutes: 5.66035)
+ echo 'Creating training and holdout set with a random 80-20 split of the generated vector dataset'
Creating training and holdout set with a random 80-20 split of the generated vector dataset
+ ./bin/mahout split -i /home/mahout/mahout-work-mahout/20news-vectors/tfidf-vectors --trainingOutput /home/mahout/mahout-work-mahout/20news-train-vectors --testOutput /home/mahout/mahout-work-mahout/20news-test-vectors --randomSelectionPct 40 --overwrite --sequenceFiles -xm sequential
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/mahout/hadoop-1.0.4/bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/mahout/mahout-d-0.7/mahout-examples-0.7-job.jar
Warning: $HADOOP_HOME is deprecated.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/mahout-examples-0.7-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
13/08/26 23:49:06 WARN driver.MahoutDriver: No split.props found on classpath, will use command-line arguments only
13/08/26 23:49:07 INFO common.AbstractJob: Command line arguments: {--endPhase=[2147483647], --input=[/home/mahout/mahout-work-mahout/20news-vectors/tfidf-vectors], --method=[sequential], --overwrite=null, --randomSelectionPct=[40], --sequenceFiles=null, --startPhase=[0], --tempDir=[temp], --testOutput=[/home/mahout/mahout-work-mahout/20news-test-vectors], --trainingOutput=[/home/mahout/mahout-work-mahout/20news-train-vectors]}
13/08/26 23:49:11 INFO utils.SplitInput: part-r-00000 has 162419 lines
13/08/26 23:49:11 INFO utils.SplitInput: part-r-00000 test split size is 64968 based on random selection percentage 40
13/08/26 23:49:11 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/26 23:49:11 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
13/08/26 23:49:11 INFO compress.CodecPool: Got brand-new compressor
13/08/26 23:49:11 INFO compress.CodecPool: Got brand-new compressor
13/08/26 23:49:16 INFO utils.SplitInput: file: part-r-00000, input: 162419 train: 11321, test: 7525 starting at 0
13/08/26 23:49:16 INFO driver.MahoutDriver: Program took 9786 ms (Minutes: 0.1631)
+ echo 'Training Naive Bayes model'
Training Naive Bayes model
+ ./bin/mahout trainnb -i /home/mahout/mahout-work-mahout/20news-train-vectors -el -o /home/mahout/mahout-work-mahout/model -li /home/mahout/mahout-work-mahout/labelindex -ow
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/mahout/hadoop-1.0.4/bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/mahout/mahout-d-0.7/mahout-examples-0.7-job.jar
Warning: $HADOOP_HOME is deprecated.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/mahout-examples-0.7-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
13/08/26 23:49:22 WARN driver.MahoutDriver: No trainnb.props found on classpath, will use command-line arguments only
13/08/26 23:49:22 INFO common.AbstractJob: Command line arguments: {--alphaI=[1.0], --endPhase=[2147483647], --extractLabels=null, --input=[/home/mahout/mahout-work-mahout/20news-train-vectors], --labelIndex=[/home/mahout/mahout-work-mahout/labelindex], --output=[/home/mahout/mahout-work-mahout/model], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
13/08/26 23:49:23 INFO common.HadoopUtil: Deleting temp
13/08/26 23:49:23 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/26 23:49:23 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
13/08/26 23:49:23 INFO compress.CodecPool: Got brand-new decompressor
13/08/26 23:49:26 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:49:26 INFO mapred.JobClient: Running job: job_201308212334_0063
13/08/26 23:49:27 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:49:49 INFO mapred.JobClient:  map 43% reduce 0%
13/08/26 23:49:52 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:50:13 INFO mapred.JobClient:  map 100% reduce 100%
13/08/26 23:50:18 INFO mapred.JobClient: Job complete: job_201308212334_0063
13/08/26 23:50:18 INFO mapred.JobClient: Counters: 29
13/08/26 23:50:18 INFO mapred.JobClient:   Job Counters 
13/08/26 23:50:18 INFO mapred.JobClient:     Launched reduce tasks=1
13/08/26 23:50:18 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=22816
13/08/26 23:50:18 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:50:18 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:50:18 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:50:18 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:50:18 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=20680
13/08/26 23:50:18 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:50:18 INFO mapred.JobClient:     Bytes Written=2718605
13/08/26 23:50:18 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:50:18 INFO mapred.JobClient:     FILE_BYTES_READ=1404371
13/08/26 23:50:18 INFO mapred.JobClient:     HDFS_BYTES_READ=12669237
13/08/26 23:50:18 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=2854477
13/08/26 23:50:18 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=2718605
13/08/26 23:50:18 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:50:18 INFO mapred.JobClient:     Bytes Read=12668431
13/08/26 23:50:18 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:50:18 INFO mapred.JobClient:     Map output materialized bytes=1404363
13/08/26 23:50:18 INFO mapred.JobClient:     Map input records=11321
13/08/26 23:50:18 INFO mapred.JobClient:     Reduce shuffle bytes=1404363
13/08/26 23:50:18 INFO mapred.JobClient:     Spilled Records=40
13/08/26 23:50:18 INFO mapred.JobClient:     Map output bytes=16682576
13/08/26 23:50:18 INFO mapred.JobClient:     Total committed heap usage (bytes)=176164864
13/08/26 23:50:18 INFO mapred.JobClient:     CPU time spent (ms)=8190
13/08/26 23:50:18 INFO mapred.JobClient:     Combine input records=11321
13/08/26 23:50:18 INFO mapred.JobClient:     SPLIT_RAW_BYTES=148
13/08/26 23:50:18 INFO mapred.JobClient:     Reduce input records=20
13/08/26 23:50:18 INFO mapred.JobClient:     Reduce input groups=20
13/08/26 23:50:18 INFO mapred.JobClient:     Combine output records=20
13/08/26 23:50:18 INFO mapred.JobClient:     Physical memory (bytes) snapshot=294400000
13/08/26 23:50:18 INFO mapred.JobClient:     Reduce output records=20
13/08/26 23:50:18 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1961967616
13/08/26 23:50:18 INFO mapred.JobClient:     Map output records=11321
13/08/26 23:50:18 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:50:18 INFO mapred.JobClient: Running job: job_201308212334_0064
13/08/26 23:50:19 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:50:40 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:51:01 INFO mapred.JobClient:  map 100% reduce 100%
13/08/26 23:51:06 INFO mapred.JobClient: Job complete: job_201308212334_0064
13/08/26 23:51:06 INFO mapred.JobClient: Counters: 29
13/08/26 23:51:06 INFO mapred.JobClient:   Job Counters 
13/08/26 23:51:06 INFO mapred.JobClient:     Launched reduce tasks=1
13/08/26 23:51:06 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=24609
13/08/26 23:51:06 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:51:06 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:51:06 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:51:06 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:51:06 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=15258
13/08/26 23:51:06 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:51:06 INFO mapred.JobClient:     Bytes Written=893560
13/08/26 23:51:06 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:51:06 INFO mapred.JobClient:     FILE_BYTES_READ=362674
13/08/26 23:51:06 INFO mapred.JobClient:     HDFS_BYTES_READ=2718737
13/08/26 23:51:06 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=771195
13/08/26 23:51:06 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=893560
13/08/26 23:51:06 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:51:06 INFO mapred.JobClient:     Bytes Read=2718605
13/08/26 23:51:06 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:51:06 INFO mapred.JobClient:     Map output materialized bytes=362666
13/08/26 23:51:06 INFO mapred.JobClient:     Map input records=20
13/08/26 23:51:06 INFO mapred.JobClient:     Reduce shuffle bytes=362666
13/08/26 23:51:06 INFO mapred.JobClient:     Spilled Records=4
13/08/26 23:51:06 INFO mapred.JobClient:     Map output bytes=893434
13/08/26 23:51:06 INFO mapred.JobClient:     Total committed heap usage (bytes)=223264768
13/08/26 23:51:06 INFO mapred.JobClient:     CPU time spent (ms)=5370
13/08/26 23:51:06 INFO mapred.JobClient:     Combine input records=2
13/08/26 23:51:06 INFO mapred.JobClient:     SPLIT_RAW_BYTES=132
13/08/26 23:51:06 INFO mapred.JobClient:     Reduce input records=2
13/08/26 23:51:06 INFO mapred.JobClient:     Reduce input groups=2
13/08/26 23:51:06 INFO mapred.JobClient:     Combine output records=2
13/08/26 23:51:06 INFO mapred.JobClient:     Physical memory (bytes) snapshot=300597248
13/08/26 23:51:06 INFO mapred.JobClient:     Reduce output records=2
13/08/26 23:51:06 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=1961967616
13/08/26 23:51:06 INFO mapred.JobClient:     Map output records=2
13/08/26 23:51:07 INFO driver.MahoutDriver: Program took 104944 ms (Minutes: 1.7490666666666668)
+ echo 'Self testing on training set'
Self testing on training set
+ ./bin/mahout testnb -i /home/mahout/mahout-work-mahout/20news-train-vectors -m /home/mahout/mahout-work-mahout/model -l /home/mahout/mahout-work-mahout/labelindex -ow -o /home/mahout/mahout-work-mahout/20news-testing
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/mahout/hadoop-1.0.4/bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/mahout/mahout-d-0.7/mahout-examples-0.7-job.jar
Warning: $HADOOP_HOME is deprecated.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/mahout-examples-0.7-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
13/08/26 23:51:19 WARN driver.MahoutDriver: No testnb.props found on classpath, will use command-line arguments only
13/08/26 23:51:19 INFO common.AbstractJob: Command line arguments: {--endPhase=[2147483647], --input=[/home/mahout/mahout-work-mahout/20news-train-vectors], --labelIndex=[/home/mahout/mahout-work-mahout/labelindex], --model=[/home/mahout/mahout-work-mahout/model], --output=[/home/mahout/mahout-work-mahout/20news-testing], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
13/08/26 23:51:20 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:51:21 INFO mapred.JobClient: Running job: job_201308212334_0065
13/08/26 23:51:22 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:51:45 INFO mapred.JobClient:  map 51% reduce 0%
13/08/26 23:51:48 INFO mapred.JobClient:  map 89% reduce 0%
13/08/26 23:51:54 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:51:58 INFO mapred.JobClient: Job complete: job_201308212334_0065
13/08/26 23:51:58 INFO mapred.JobClient: Counters: 19
13/08/26 23:51:58 INFO mapred.JobClient:   Job Counters 
13/08/26 23:51:58 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=34216
13/08/26 23:51:58 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:51:58 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:51:58 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:51:58 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:51:58 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
13/08/26 23:51:58 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:51:58 INFO mapred.JobClient:     Bytes Written=2132486
13/08/26 23:51:58 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:51:58 INFO mapred.JobClient:     HDFS_BYTES_READ=16279896
13/08/26 23:51:58 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=22523
13/08/26 23:51:58 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=2132486
13/08/26 23:51:58 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:51:58 INFO mapred.JobClient:     Bytes Read=12668431
13/08/26 23:51:58 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:51:58 INFO mapred.JobClient:     Map input records=11321
13/08/26 23:51:58 INFO mapred.JobClient:     Physical memory (bytes) snapshot=87547904
13/08/26 23:51:58 INFO mapred.JobClient:     Spilled Records=0
13/08/26 23:51:58 INFO mapred.JobClient:     CPU time spent (ms)=9380
13/08/26 23:51:58 INFO mapred.JobClient:     Total committed heap usage (bytes)=28131328
13/08/26 23:51:58 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=976572416
13/08/26 23:51:58 INFO mapred.JobClient:     Map output records=11321
13/08/26 23:51:58 INFO mapred.JobClient:     SPLIT_RAW_BYTES=148
13/08/26 23:51:59 INFO test.TestNaiveBayesDriver: Standard NB Results: =======================================================
Summary
-------------------------------------------------------
Correctly Classified Instances          :      11256	   99.4258%
Incorrectly Classified Instances        :         65	    0.5742%
Total Classified Instances              :      11321

=======================================================
Confusion Matrix
-------------------------------------------------------
a    	b    	c    	d    	e    	f    	g    	h    	i    	j    	k   l    	m    	n    	o    	p    	q    	r    	s    	t    	<--Classified as
454  	0    	0    	1    	0    	0    	0    	0    	0    	0    	0   0    	0    	0    	0    	0    	0    	0    	3    	0    	 |  458     a     = alt.atheism
0    	588  	0    	3    	0    	2    	0    	0    	0    	0    	0   1    	0    	1    	0    	0    	0    	0    	0    	0    	 |  595     b     = comp.graphics
0    	3    	553  	7    	0    	0    	0    	0    	0    	0    	0   0    	0    	0    	0    	0    	0    	0    	0    	0    	 |  563     c     = comp.os.ms-windows.misc
0    	0    	0    	592  	1    	0    	2    	0    	0    	0    	0   0    	0    	0    	0    	0    	0    	0    	0    	0    	 |  595     d     = comp.sys.ibm.pc.hardware
0    	0    	0    	1    	593  	0    	0    	0    	0    	0    	0   0    	0    	0    	0    	0    	0    	0    	0    	0    	 |  594     e     = comp.sys.mac.hardware
0    	2    	0    	1    	0    	576  	1    	0    	0    	0    	0   0    	0    	0    	1    	0    	0    	0    	0    	0    	 |  581     f     = comp.windows.x
0    	1    	0    	0    	0    	0    	579  	0    	0    	0    	0   0    	1    	0    	0    	0    	0    	0    	0    	0    	 |  581     g     = misc.forsale
0    	0    	0    	0    	0    	0    	1    	594  	0    	0    	0   0    	1    	0    	0    	0    	0    	0    	0    	0    	 |  596     h     = rec.autos
0    	0    	0    	0    	0    	0    	1    	2    	591  	0    	0   0    	0    	0    	0    	0    	0    	0    	0    	0    	 |  594     i     = rec.motorcycles
0    	0    	0    	0    	0    	0    	0    	0    	0    	615  	1   0    	0    	0    	0    	0    	0    	0    	0    	0    	 |  616     j     = rec.sport.baseball
0    	0    	0    	0    	0    	0    	1    	0    	0    	1    	581 0    	0    	0    	0    	0    	0    	0    	0    	0    	 |  583     k     = rec.sport.hockey
0    	0    	1    	0    	0    	0    	0    	0    	0    	0    	0   627  	1    	0    	0    	0    	0    	1    	0    	0    	 |  630     l     = sci.crypt
0    	0    	0    	2    	0    	0    	1    	0    	0    	0    	0   0    	588  	0    	0    	0    	0    	0    	0    	0    	 |  591     m     = sci.electronics
0    	1    	0    	0    	0    	0    	0    	0    	0    	0    	0   0    	0    	586  	1    	0    	0    	0    	0    	0    	 |  588     n     = sci.med
0    	0    	0    	0    	0    	0    	0    	0    	0    	0    	0   0    	0    	0    	615  	0    	0    	0    	0    	0    	 |  615     o     = sci.space
0    	0    	0    	0    	0    	0    	0    	0    	0    	0    	0   0    	0    	0    	0    	619  	1    	0    	0    	0    	 |  620     p     = soc.religion.christian
1    	0    	0    	0    	0    	0    	0    	0    	0    	0    	0   0    	0    	0    	0    	1    	541  	0    	0    	0    	 |  543     q     = talk.politics.mideast
0    	0    	0    	0    	0    	0    	0    	0    	0    	0    	0   1    	0    	0    	0    	0    	0    	560  	0    	0    	 |  561     r     = talk.politics.guns
3    	0    	0    	0    	0    	0    	0    	0    	0    	0    	0   0    	0    	0    	0    	4    	0    	1    	351  	0    	 |  359     s     = talk.religion.misc
0    	0    	0    	0    	0    	0    	0    	0    	0    	0    	0   1    	0    	0    	0    	0    	0    	4    	0    	453  	 |  458     t     = talk.politics.misc


13/08/26 23:51:59 INFO driver.MahoutDriver: Program took 40214 ms (Minutes: 0.6702333333333333)
+ echo 'Testing on holdout set'
Testing on holdout set
+ ./bin/mahout testnb -i /home/mahout/mahout-work-mahout/20news-test-vectors -m /home/mahout/mahout-work-mahout/model -l /home/mahout/mahout-work-mahout/labelindex -ow -o /home/mahout/mahout-work-mahout/20news-testing
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /home/mahout/hadoop-1.0.4/bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/mahout/mahout-d-0.7/mahout-examples-0.7-job.jar
Warning: $HADOOP_HOME is deprecated.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/mahout-examples-0.7-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mahout/hadoop-1.0.4/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
13/08/26 23:52:09 WARN driver.MahoutDriver: No testnb.props found on classpath, will use command-line arguments only
13/08/26 23:52:09 INFO common.AbstractJob: Command line arguments: {--endPhase=[2147483647], --input=[/home/mahout/mahout-work-mahout/20news-test-vectors], --labelIndex=[/home/mahout/mahout-work-mahout/labelindex], --model=[/home/mahout/mahout-work-mahout/model], --output=[/home/mahout/mahout-work-mahout/20news-testing], --overwrite=null, --startPhase=[0], --tempDir=[temp]}
13/08/26 23:52:10 INFO common.HadoopUtil: Deleting /home/mahout/mahout-work-mahout/20news-testing
13/08/26 23:52:10 INFO input.FileInputFormat: Total input paths to process : 1
13/08/26 23:52:11 INFO mapred.JobClient: Running job: job_201308212334_0066
13/08/26 23:52:12 INFO mapred.JobClient:  map 0% reduce 0%
13/08/26 23:52:30 INFO mapred.JobClient:  map 85% reduce 0%
13/08/26 23:52:36 INFO mapred.JobClient:  map 100% reduce 0%
13/08/26 23:52:41 INFO mapred.JobClient: Job complete: job_201308212334_0066
13/08/26 23:52:41 INFO mapred.JobClient: Counters: 19
13/08/26 23:52:41 INFO mapred.JobClient:   Job Counters 
13/08/26 23:52:41 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=25113
13/08/26 23:52:41 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/08/26 23:52:41 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/08/26 23:52:41 INFO mapred.JobClient:     Launched map tasks=1
13/08/26 23:52:41 INFO mapred.JobClient:     Data-local map tasks=1
13/08/26 23:52:41 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
13/08/26 23:52:41 INFO mapred.JobClient:   File Output Format Counters 
13/08/26 23:52:41 INFO mapred.JobClient:     Bytes Written=1417942
13/08/26 23:52:41 INFO mapred.JobClient:   FileSystemCounters
13/08/26 23:52:41 INFO mapred.JobClient:     HDFS_BYTES_READ=12148944
13/08/26 23:52:41 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=22522
13/08/26 23:52:41 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=1417942
13/08/26 23:52:41 INFO mapred.JobClient:   File Input Format Counters 
13/08/26 23:52:41 INFO mapred.JobClient:     Bytes Read=8537480
13/08/26 23:52:41 INFO mapred.JobClient:   Map-Reduce Framework
13/08/26 23:52:41 INFO mapred.JobClient:     Map input records=7525
13/08/26 23:52:41 INFO mapred.JobClient:     Physical memory (bytes) snapshot=85057536
13/08/26 23:52:41 INFO mapred.JobClient:     Spilled Records=0
13/08/26 23:52:41 INFO mapred.JobClient:     CPU time spent (ms)=6630
13/08/26 23:52:41 INFO mapred.JobClient:     Total committed heap usage (bytes)=28155904
13/08/26 23:52:41 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=976572416
13/08/26 23:52:41 INFO mapred.JobClient:     Map output records=7525
13/08/26 23:52:41 INFO mapred.JobClient:     SPLIT_RAW_BYTES=147
13/08/26 23:52:42 INFO test.TestNaiveBayesDriver: Standard NB Results: =======================================================
Summary
-------------------------------------------------------
Correctly Classified Instances          :       6801	   90.3787%
Incorrectly Classified Instances        :        724	    9.6213%
Total Classified Instances              :       7525

=======================================================
Confusion Matrix
-------------------------------------------------------
a    	b    	c    	d    	e    	f    	g    	h    	i    	j    	k   l    	m    	n    	o    	p    	q    	r    	s    	t    	<--Classified as
318  	0    	0    	0    	1    	0    	0    	0    	1    	0    	0   0    	0    	0    	1    	4    	0    	0    	15   	1    	 |  341     a     = alt.atheism
1    	318  	7    	20   	4    	7    	7    	2    	0    	1    	0   1    	1    	2    	6    	0    	0    	0    	0    	1    	 |  378     b     = comp.graphics
0    	25   	277  	78   	12   	15   	5    	0    	0    	0    	0   2    	4    	0    	1    	0    	0    	0    	0    	3    	 |  422     c     = comp.os.ms-windows.misc
1    	4    	3    	336  	20   	3    	8    	0    	0    	0    	0   1    	11   	0    	0    	0    	0    	0    	0    	0    	 |  387     d     = comp.sys.ibm.pc.hardware
0    	3    	1    	6    	350  	1    	3    	0    	0    	0    	0   1    	3    	1    	0    	0    	0    	0    	0    	0    	 |  369     e     = comp.sys.mac.hardware
1    	20   	3    	6    	7    	365  	3    	0    	0    	0    	0   1    	0    	0    	0    	0    	1    	0    	0    	0    	 |  407     f     = comp.windows.x
0    	1    	1    	19   	8    	0    	329  	13   	1    	0    	0   2    	14   	0    	4    	0    	0    	1    	1    	0    	 |  394     g     = misc.forsale
0    	2    	1    	2    	3    	1    	10   	361  	8    	0    	0   0    	4    	0    	0    	0    	0    	1    	0    	1    	 |  394     h     = rec.autos
0    	0    	0    	1    	0    	0    	2    	3    	393  	1    	0   0    	0    	0    	0    	0    	0    	1    	0    	1    	 |  402     i     = rec.motorcycles
0    	0    	0    	1    	0    	0    	2    	3    	0    	360  	6   0    	2    	2    	1    	0    	0    	0    	0    	1    	 |  378     j     = rec.sport.baseball
0    	1    	0    	2    	1    	0    	0    	0    	2    	5    	401 0    	1    	0    	0    	1    	0    	0    	0    	2    	 |  416     k     = rec.sport.hockey
1    	1    	0    	1    	3    	2    	1    	1    	0    	0    	0   344  	1    	1    	2    	0    	1    	1    	0    	1    	 |  361     l     = sci.crypt
0    	5    	0    	15   	14   	0    	5    	1    	1    	0    	0   2    	348  	1    	1    	0    	0    	0    	0    	0    	 |  393     m     = sci.electronics
1    	2    	1    	1    	1    	0    	1    	0    	0    	1    	0   1    	4    	381  	5    	0    	0    	1    	1    	1    	 |  402     n     = sci.med
1    	4    	0    	0    	2    	0    	2    	1    	0    	0    	0   1    	2    	1    	356  	0    	0    	1    	0    	1    	 |  372     o     = sci.space
5    	0    	0    	1    	1    	0    	0    	1    	0    	0    	1   0    	0    	1    	0    	359  	3    	0    	4    	1    	 |  377     p     = soc.religion.christian
0    	0    	0    	0    	0    	0    	0    	0    	0    	1    	1   0    	1    	0    	1    	2    	389  	0    	0    	2    	 |  397     q     = talk.politics.mideast
0    	0    	1    	0    	1    	1    	0    	1    	0    	0    	0   2    	1    	1    	0    	0    	0    	335  	0    	6    	 |  349     r     = talk.politics.guns
29   	1    	0    	1    	0    	0    	1    	0    	0    	1    	0   0    	0    	0    	2    	24   	0    	8    	197  	5    	 |  269     s     = talk.religion.misc
2    	0    	0    	0    	2    	0    	0    	1    	0    	1    	1   1    	0    	1    	2    	0    	2    	17   	3    	284  	 |  317     t     = talk.politics.misc


13/08/26 23:52:42 INFO driver.MahoutDriver: Program took 32480 ms (Minutes: 0.5413333333333333)

在job信息可以看到全部的任务信息，如下：

然后对照每个job信息，查看相应的mapper和reducer就可以分析这个算法了。

分享，快乐，成长

转载请注明出处：http://blog.csdn.net/fansy1990

你可能感兴趣的:(Mahout)

传统推荐算法库使用--mahout初体验 Huterox 推荐算法算法机器学习
文章目录前言环境准备调用混合总结前言郑重声明：本博文做法仅限毕设糊弄老师使用，不建议生产环境使用！！！老项目缝缝补补又是三年，本来是打算直接重写写个社区然后给毕设使用的。但是怎么说呢，毕竟毕设的主角不是xx社区，这个社区是为我的编译器服务的，为了推广这个编译器，然后我才做了这个社区。然而不幸的是，开题答辩的时候，各位“专家”叫我以xx社区为主，听起来高级。于是没有办法，我只能强行做个社区，怎么做呢
基于音乐/电影/图书的协同过滤推荐算法代码实现 74b3a3e489d4
基于音乐/电影/图书的协同过滤推荐算法代码实现一、开发工具及使用技术MyEclipse10、jdk1.7、tomcat7、jsp、javascript、jquery、bootstrap、webuploader、layer、ssh、mysql、navicat、mahoutAPI等。二、开发过程1、本文主要介绍基于音乐的协同过滤推荐算法代码实现，电影、图书等推荐原理相同。2、本文使用的推荐算法有：基于
Hadoop 大数据技术原理与应用 kk8_ hadoop 大数据 hdfs
Hadoop大数据技术原理与应用大数据概述定义特征大量，多样，高速，价值研究意义应用场景医疗，金融，零售Hadoop概述历史优势扩容能力强，成本低，高效率，可靠性，高容错Hadoop生态分布式存储系统（HDFS）分布式计算框架（MapReduce）资源管理（YARN）数据迁移（Sqoop）数据挖掘算法库（Mahout）分布式数据库（HBase）分布式协调服务（Zookeeper）数据仓库（Hive
【大数据分析与挖掘技术】概述 Francek Chen 大数据技术基础数据分析数据挖掘 Mahout
目录一、数据挖掘简介（一）数据挖掘对象（二）数据挖掘流程（三）数据挖掘的分析方法（四）经典算法二、Mahout（一）Mahout简介（二）主要特性（三）Mahout安装与配置一、数据挖掘简介需要是发明之母。近年来，数据挖掘引起了信息产业界的极大关注，其主要原因是存在大量数据，可以广泛使用，并且迫切需要将这些数据转换成有用的信息和知识。获取的信息和知识可以广泛用于各种应用，包括商务管理，生产控制，市
【大数据分析与挖掘技术】Mahout推荐算法 Francek Chen 大数据技术基础数据分析人工智能数据挖掘 Mahout
目录一、推荐的定义与评估（一）推荐的定义（二）推荐的评估二、Mahout中的常见推荐算法（一）基于用户的推荐算法（二）基于物品的推荐算法（三）基于SVD的推荐算法（四）基于线性插值的推荐算法（五）基于聚类的推荐算法三、对GroupLens数据集进行推荐与评价（一）如何使用推荐器进行推荐（二）如何评估推荐器的好坏推荐是Mahout机器学习算法的主题之一，它极大地渗透到了人们日常生活的方方面面，比如，
计算机毕业设计之全网独家Spark租房爬虫数据分析与推荐系统租房大数据租房app 租房数据分析租房爬虫房源推荐系统房源数据分析房源可视化 haochengxu2022 数据分析爬虫推荐系统 spark 爬虫数据分析推荐系统
一、网站·登录与注册、注销·短信验证码修改密码·我的信息：身份证实名认证·租房业务流程（预约+看房+支付+完成+评价）、进度步骤条展示·支付宝沙箱支付·房屋浏览、中介信息查看·房屋推荐（基于mahout协同过滤算法）·房屋评价、点赞与收藏二、后端·统计主页、个人信息（带头像上传）、权限管理、用户管理、资讯管理、通知管理、日志管理、评论管理、轮播图管理、房屋管理、中介管理、订单管理。·中介权限可以登
推荐系统中协同过滤算法实现分析 weixin_33853794 人工智能 python 数据库
2019独角兽企业重金招聘Python工程师标准>>>原创博客，欢迎转载，转载请注明：http://my.oschina.net/BreathL/blog/62519最近研究Mahout比较多，特别是里面协同过滤算法；于是把协同过滤算法的这个实现思路与数据流程，总结了一下，以便以后对系统做优化时，有个清晰的思路，这样才能知道该如何优化且优化后数据亦能正确。推荐中的协同过滤算法简单说明下：首先，通过
大数据分析- 基于Hadoop/Mahout的大数据挖掘 shenmanli 大数据 hadoop 数据挖掘行业应用开发人员
随着互联网、移动互联网和物联网的发展，我们已经切实地迎来了一个大数据的时代。大数据是指无法在一定时间内用常规软件工具对其内容进行抓取、管理和处理的数据集合，对大数据的分析已经成为一个非常重要且紧迫的需求。目前对大数据的分析工具，首选的是Hadoop平台。Hadoop在可伸缩性、健壮性、计算性能和成本上具有无可替代的优势，事实上已成为当前互联网企业主流的大数据分析平台。一、培训对象1，系统架构师、系
“大数据分析挖掘-基于Hadoop/Mahout/Mllib的大数据挖掘（含Spark、Storm和Docker应用介绍）”培训 shenmanli 培训课程公开课企业培训大数据 hadoop spark
随着互联网、移动互联网和物联网的发展，我们已经切实地迎来了一个大数据的时代。大数据是指无法在一定时间内用常规软件工具对其内容进行抓取、管理和处理的数据集合，对大数据的分析已经成为一个非常重要且紧迫的需求。目前对大数据的分析工具，首选的是Hadoop/Yarn平台。Hadoop/Yarn在可伸缩性、健壮性、计算性能和成本上具有无可替代的优势，事实上已成为当前互联网企业主流的大数据分析平台。为解决广大
springboot集成mahout实现简单基于协同过滤算法的文章推荐算法程序个人练习生开源项目学习算法 spring boot 推荐算法
文章目录参考文章前言1.建表并且生成一些数据首先，建立一个用户文章操作表（user_article_operation）使用casewhen语句简单统计数据2.代码与测试只需要根据表生成相应实体类（注意要加一个value属性来存储分数）主要代码如下，其实就两个方法userArticleOperationMapper.getAllUserPreference()方法收集数据mapper文件如下测试算
java电影推荐系统_基于Mahout的电影推荐系统语文乌托邦 java电影推荐系统
1.Mahout简介ApacheMahout是ApacheSoftwareFoundation(ASF)旗下的一个开源项目，提供一些可扩展的机器学习领域经典算法的实现，旨在帮助开发人员更加方便快捷地创建智能应用程序。经典算法包括聚类、分类、协同过滤、进化编程等等，并且，在Mahout的最近版本中还加入了对ApacheHadoop的支持，使这些算法可以更高效的运行在云计算环境中。2.Taste简介T
mahout 源码解析之聚类--聚类迭代模型 theonlytank2011 数据挖掘 mahout源码 mahout源码解析
在前面讲聚类策略时，包org.apache.mahout.clustering.iterator里面还有几个类没有进行讲解，这次做下收尾工作。ClusterIterator利用ClusterClassifier和指定的迭代次数将样本进行聚类。其中有三个具体的函数。iterate主要对内存中的数据进行聚类，输入就为一个Vector类型的迭代器。publicClusterClassifieritera
理论学习--【Hadoop生态原理学习】 zenas_yuan Hadoop hadoop
一、Hadoop原理1.核心：HDFS(存储)、MapReduce(分析)解决大量数据存储与处理的问题离线分析：hive实现查询：hbaseBI分析：Mahout2.版本1.0mapreduce还进行资源调度2.0mapreduce=yarn（资源调度）+mapreduce（进行计算运行在yarn上），HDfs:nn,ha2.1.2yarn还支持strom、spark、。。选择考虑因素：是否开源、
推荐系统-基于物品协同过滤算法代码实现 Moutai码农大数据推荐系统算法推荐算法大数据 spark
1、简介当前Spark没有像mahout那样，严格区分基于物品的协同过滤推荐（ItemCF）和基于用户的协同过滤推荐（UserCF），只有基于模型的协同过滤推荐算法ALS（model-basedCF）。但ALS算法对于一些特定的问题(用户数量较小的场景，以及物品数量明显小于用户数量的场景)，效果并不理想，不像mahout提供了各种推荐算法选择。为了充分利用spark在速度上带来的提升同时为满足一些
java+jsp+mysql实现在线电影推荐系统movieCFWeb mahout实现基于用户的协同过滤推荐算法基于项目的协同过滤推荐算法 74b3a3e489d4
java+jsp+mysql实现在线电影推荐系统movieCFWeb一、项目简介http://localhost:8080/movieCFWeb/前台http://localhost:8080/movieCFWeb/admin后台自定义数据，mahout实现基于用户的协同过滤推荐算法前台包含用户注册、登录、搜索电影、分页、电影详情、评分、修改信息、评分列表、推荐电影等功能后台包括用户、电影、评分、
2.3 初探Hadoop世界 howard2005 数据清洗和预处理大数据离线分析 hadoop 大数据分布式
文章目录零、学习目标一、导入新课二、新课讲解（一）Hadoop的前世今生1、Google处理大数据三大技术2、Hadoop如何诞生3、Hadoop主要发展历程（二）Hadoop的优势1、扩容能力强2、成本低3、高效率4、可靠性5、高容错性（三）Hadoop的生态体系1、HDFS分布式文件系统2、MapReduce分布式计算框架3、Yarn资源管理框架4、Sqoop数据迁移工具5、Mahout数据挖
「大数据集群的搭建和使用」背景知识：大数据Hadoop生态圈介绍优秀的Athena在休息大数据集群的搭建和使用大数据 hadoop 分布式
目录一、Hadoop简介二、Hadoop的运行模式1.单机模式2.伪分布式模式3.完全分布式模式三、Hadoop生态圈组件1.HDFS2.MapReduce3.YARN4.Hive5.Pig6.HBase7.HCatalog8.Avro9.Thrift10.Drill11.Mahout12.Sqoop13.Flume14.Ambari15.Zookeeper四、Hadoop优缺点五、Hadoop学
【大数据】Hadoop 生态系统及其组件 G皮T #Hadoop hadoop big data 大数据 hdfs hive mapreduce yarn
Hadoop生态系统及其组件1.Hadoop生态系统的组成2.Hadoop生态系统简介2.1HDFS2.2MapReduce2.3YARN2.4Hive2.5Pig2.6HBase2.7HCatalog2.8Avro2.9Thrift2.10Drill2.11Mahout2.12Sqoop2.13Flume2.14Ambari2.15Zookeeper2.16Oozie1.Hadoop生态系统的组
26Hbase介绍及其数据模型和架构（hbase学习1）文茶君
Hbase介绍Hadoop生态系统spark已经替代mahouthbase简介：非关系型数据库知识面扩展cassandra、hbase、mongodb（文档型数据库）、rediscouchdb，文件存储数据库Neo4j非关系型图数据库HbaseHadoopDatabase，是一个高可靠性、高性能、面向列（面向列的KV数据库）、可伸缩（动态扩展机器。不需要停服务）、实时读写的分布式数据库利用Hado
Item-Based Recommendations with Hadoop liuyuan185442111 Old Hadoop hadoop 大数据分布式
Mahout在MapReduce上实现了Item-BasedCollaborativeFiltering，这里我尝试运行一下。安装Hadoop从下载Mahout并解压准备数据下载1MillionMovieLensDataset，解压得到ratings.dat，用sed‘s/:[0-9]{1,}):[0-9]{1})::[0-9]{1,}$/,\1,\2/’ratings.dat处理成需要的格式。运
【大数据毕设】基于Hadoop的音乐推荐系统论文(三) Maynor996 #课设＆毕设大数据课程设计 hadoop
博主介绍：✌全网粉丝6W+,csdn特邀作者、博客专家、大数据领域优质创作者,博客之星、掘金/华为云/阿里云/InfoQ等平台优质作者、专注于大数据技术领域和毕业项目实战✌文末获取项目联系摘要本文基于Hadoop技术，设计并实现了一个名为“酷酷音乐网站”的系统，用于音乐资源的存储、管理和推荐。该系统采用Hadoop生态系统中的组件，包括HDFS、MapReduce、HBase和Mahout等，实现
如何使用Java进行机器学习? 玥沐春风 java 机器学习开发语言
在Java中进行机器学习，可以使用各种开源机器学习库和框架来实现。以下是一些常用的Java机器学习库：Weka：Weka是一个非常流行的机器学习库，提供了大量的算法和工具，以及用于数据预处理、特征选择和可视化的功能。Deeplearning4j：Deeplearning4j是一个用于深度学习的开源库，支持多种神经网络模型和训练算法，可以用于图像分类、文本分析等任务。ApacheMahout：Apa
阿里云上部署java8和hadoop3.0、spark、hive及Mahout karwik 大数据
1.安装JDK1.8到oracle官网：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.htmllinux是64位的，安装jdk-8u131-linux-x64.tar.gz安装及配置参考http://blog.csdn.net/rchm8519/article/details/48721
【大数据】图解 Hadoop 生态系统及其组件 G皮T #Hadoop 大数据 hadoop 分布式 hdfs mapreduce yarn hive
图解Hadoop生态系统及其组件1.HDFS2.MapReduce3.YARN4.Hive5.Pig6.Mahout7.HBase8.Zookeeper9.Sqoop10.Flume11.Oozie12.Ambari13.Spark在了解Hadoop生态系统及其组件之前，我们首先了解一下Hadoop的三大组件，即HDFS、MapReduce、YARN，它们共同构成了Hadoop分布式计算框架的核心
斯皮尔曼相关性 —— Spearman Correlation ifnoelse 推荐算法 user cache action 存储
斯皮尔曼相关性可以理解为是排列后（Rank）用户喜好值之间的Pearson相关度。《MahoutinAction》中有这样的解释：假设对于每个用户，我们找到他最不喜欢的物品，重写他的评分值为“1”；然后找到下一个最不喜欢的物品，重写评分值为“2”，以此类推。然后我们对这些转换后的值求Pearson相关系数，这就是Spearman相关系数。斯皮尔曼相关度的计算舍弃了一些重要信息，即真实的评分值。但它
java+jsp+mysql实现个性化租车推荐系统carcfrs mahout实现基于用户、项目的协同过滤推荐算法 SSH（spring+struts+hibernate）开发框架 74b3a3e489d4
java+jsp+mysql实现个性化租车推荐系统carcfrs一、项目简介只有前台用户，没有管理员，功能是用户登录、注册、评论、评分、收藏、热点推荐、基于用户根据评分进行协同过滤推荐算法，数据爬虫爬取一嗨租车数据。二、项目展示
Mahout教程_编程入门自学教程_菜鸟教程-免费教程分享菜鸟一记笔记
教程简介Mahout是ApacheSoftwareFoundation（ASF）旗下的一个开源项目，提供一些可扩展的机器学习领域经典算法的实现，旨在帮助开发人员更加方便快捷地创建智能应用程序。Mahout包含许多实现，包括聚类、分类、推荐过滤、频繁子项挖掘。此外，通过使用ApacheHadoop库，Mahout可以有效地扩展到云中。Mahout教程-使用此入门教程，从简介，机器學習，环境，推荐，聚
SSH（Spring+Hibernate+Struts）开发框架开发购物商城推荐系统shop mahout实现基于用户、项目的协同过滤推荐算法个性化购物推荐系统 74b3a3e489d4
SSH（Spring+Hibernate+Struts）开发框架开发购物商城推荐系统shop项目简介1、前台：http://localhost:8080/ComputerRecom/后台：http://localhost:8080/ComputerRecom/admin/login.jsp用户名：admin密码：admin；2、推荐使用mahout接口实现基于用户、项目的协同过滤推荐算法，ssh开
大数据学习记录（hadoop hive flume azkaban sqoop）左上晨大数据 hadoop hive flume azkaban
大数据学习记录（hadoophiveflumeazkabansqoop）1.hadoop对海量数据进行分布式处理2.核心组件：HDFS（分布式文件系统）、YARN（运算资源调度系统）、MAPREDUCE（分布式运算编程框架）3.HIVE：基于大数据技术（文件系统+运算框架）的SQL数据仓库工具4.HBASE：基于HADOOP的分布式海量数据库5.Mahout：基于mapreduce/spark/f
构建智能电商推荐系统：大数据实战中的Kudu、Flink和Mahout应用【上进小菜猪大数据】上进小菜猪大数据专栏合集大数据 flink 人工智能
上进小菜猪，沈工大软件工程专业，爱好敲代码，持续输出干货。本文将介绍如何利用Kudu、Flink和Mahout这三种技术构建一个强大的大数据分析平台。我们将详细讨论这些技术的特点和优势，并提供代码示例，帮助读者了解如何在实际项目中应用它们。通过本文的指导，读者将能够掌握如何使用这些工具来处理大规模数据集，并进行智能分析。在当今的信息时代，大数据分析成为了各行各业中不可或缺的一环。为了有效地处理海量
java的(PO,VO,TO,BO,DAO,POJO) Cb123456 VO TO BO POJO DAO
转: http://www.cnblogs.com/yxnchinahlj/archive/2012/02/24/2366110.html ------------------------------------------------------------------- O/R Mapping 是 Object Relational Mapping（对象关系映
spring ioc原理（看完后大家可以自己写一个spring） aijuans spring
最近，买了本Spring入门书：spring In Action 。大致浏览了下感觉还不错。就是入门了点。Manning的书还是不错的，我虽然不像哪些只看Manning书的人那样专注于Manning,但怀着崇敬的心情和激情通览了一遍。又一次接受了IOC 、DI、AOP等Spring核心概念。先就IOC和DI谈一点我的看法。IO
MyEclipse 2014中Customize Persperctive设置无效的解决方法 Kai_Ge MyEclipse2014
高高兴兴下载个MyEclipse2014，发现工具条上多了个手机开发的按钮，心生不爽就想弄掉他！结果发现Customize Persperctive失效！！有说更新下就好了，可是国内Myeclipse访问不了，何谈更新... so~这里提供了更新后的一下jar包，给大家使用！ 1、将9个jar复制到myeclipse安装目录\plugins中 2、删除和这9个jar同包名但是版本号较
SpringMvc上传 120153216 springMVC
@RequestMapping(value = WebUrlConstant.UPLOADFILE) @ResponseBody public Map<String, Object> uploadFile(HttpServletRequest request,HttpServletResponse httpresponse) { try { //
Javascript----HTML DOM 事件何必如此 JavaScript html Web
HTML DOM 事件允许Javascript在HTML文档元素中注册不同事件处理程序。事件通常与函数结合使用，函数不会在事件发生前被执行！注：DOM：指明使用的 DOM 属性级别。 1.鼠标事件属性
动态绑定和删除onclick事件 357029540 JavaScript jquery
因为对JQUERY和JS的动态绑定事件的不熟悉，今天花了好久的时间才把动态绑定和删除onclick事件搞定!现在分享下我的过程。在我的查询页面，我将我的onclick事件绑定到了tr标签上同时传入当前行(this值)参数，这样可以在点击行上的任意地方时可以选中checkbox，但是在我的某一列上也有一个onclick事件是用于下载附件的，当
HttpClient|HttpClient请求详解 7454103 apache 应用服务器网络协议网络应用 Security
HttpClient 是 Apache Jakarta Common 下的子项目，可以用来提供高效的、最新的、功能丰富的支持 HTTP 协议的客户端编程工具包，并且它支持 HTTP 协议最新的版本和建议。本文首先介绍 HTTPClient，然后根据作者实际工作经验给出了一些常见问题的解决方法。HTTP 协议可能是现在 Internet 上使用得最多、最重要的协议了，越来越多的 Java 应用程序需
递归逐层统计树形结构数据 darkranger 数据结构
将集合递归获取树形结构: /** * * 递归获取数据 * @param alist:所有分类 * @param subjname:对应统计的项目名称 * @param pk:对应项目主键 * @param reportList: 最后统计的结果集 * @param count:项目级别 */ public void getReportVO(Arr
访问WEB-INF下使用frameset标签页面出错的原因 aijuans struts2
<frameset rows="61,*,24" cols="*" framespacing="0" frameborder="no" border="0">
MAVEN常用命令 avords
Maven库： http://repo2.maven.org/maven2/ Maven依赖查询： http://mvnrepository.com/ Maven常用命令： 1. 创建Maven的普通java项目： mvn archetype:create -DgroupId=packageName
PHP如果自带一个小型的web服务器就好了 houxinyou apache 应用服务器 Web PHP 脚本
最近单位用PHP做网站，感觉PHP挺好的，不过有一些地方不太习惯，比如，环境搭建。PHP本身就是一个网站后台脚本，但用PHP做程序时还要下载apache，配置起来也不太很方便，虽然有好多配置好的apache+php+mysq的环境，但用起来总是心里不太舒服，因为我要的只是一个开发环境，如果是真实的运行环境，下个apahe也无所谓，但只是一个开发环境，总有一种杀鸡用牛刀的感觉。如果php自己的程序中
NoSQL数据库之Redis数据库管理(list类型) bijian1013 redis 数据库 NoSQL
3.list类型及操作 List是一个链表结构，主要功能是push、pop、获取一个范围的所有值等等，操作key理解为链表的名字。Redis的list类型其实就是一个每个子元素都是string类型的双向链表。我们可以通过push、pop操作从链表的头部或者尾部添加删除元素，这样list既可以作为栈，又可以作为队列。 &nbs
谁在用Hadoop？ bingyingao hadoop 数据挖掘公司应用场景
Hadoop技术的应用已经十分广泛了，而我是最近才开始对它有所了解，它在大数据领域的出色表现也让我产生了兴趣。浏览了他的官网，其中有一个页面专门介绍目前世界上有哪些公司在用Hadoop，这些公司涵盖各行各业，不乏一些大公司如alibaba,ebay,amazon,google,facebook,adobe等，主要用于日志分析、数据挖掘、机器学习、构建索引、业务报表等场景,这更加激发了学习它的热情。
【Spark七十六】Spark计算结果存到MySQL bit1129 mysql
package spark.examples.db import java.sql.{PreparedStatement, Connection, DriverManager} import com.mysql.jdbc.Driver import org.apache.spark.{SparkContext, SparkConf} object SparkMySQLInteg
Scala: JVM上的函数编程 bookjovi scala erlang haskell
说Scala是JVM上的函数编程一点也不为过，Scala把面向对象和函数型编程这两种主流编程范式结合了起来，对于熟悉各种编程范式的人而言Scala并没有带来太多革新的编程思想，scala主要的有点在于Java庞大的package优势，这样也就弥补了JVM平台上函数型编程的缺失，MS家.net上已经有了F#，JVM怎么能不跟上呢？对本人而言
jar打成exe bro_feng java jar exe
今天要把jar包打成exe，jsmooth和exe4j都用了。遇见几个问题。记录一下。两个软件都很好使，网上都有图片教程，都挺不错。首先肯定是要用自己的jre的，不然不能通用，其次别忘了把需要的lib放到classPath中。困扰我很久的一个问题是，我自己打包成功后，在一个同事的没有装jdk的电脑上运行，就是不行，报错jvm.dll为无效的windows映像，如截图最后发现
读《研磨设计模式》-代码笔记-策略模式-Strategy bylijinnan java 设计模式
声明：本文只为方便我个人查阅和理解，详细的分析以及源代码请移步原作者的博客http://chjavach.iteye.com/ /* 策略模式定义了一系列的算法，并将每一个算法封装起来，而且使它们还可以相互替换。策略模式让算法独立于使用它的客户而独立变化简单理解： 1、将不同的策略提炼出一个共同接口。这是容易的，因为不同的策略，只是算法不同，需要传递的参数
cmd命令值cvfM命令 chenyu19891124 cmd
cmd命令还真是强大啊。今天发现jar -cvfM aa.rar @aaalist 就这行命令可以根据aaalist取出相应的文件例如：在d：\workspace\prpall\test.java 有这样一个文件，现在想要将这个文件打成一个包。运行如下命令即可比如在d：\wor
OpenJWeb(1.8) Java Web应用快速开发平台 comsci java 框架 Web 项目管理企业应用
OpenJWeb(1.8) Java Web应用快速开发平台的作者是我们技术联盟的成员，他最近推出了新版本的快速应用开发平台 OpenJWeb(1.8)，我帮他做做宣传 OpenJWeb快速开发平台以快速开发为核心，整合先进的java 开源框架，本着自主开发+应用集成相结合的原则，旨在为政府、企事业单位、软件公司等平台用户提供一个架构透
Python 报错：IndentationError: unexpected indent daizj python tab 空格缩进
IndentationError: unexpected indent 是缩进的问题，也有可能是tab和空格混用啦 Python开发者有意让违反了缩进规则的程序不能通过编译，以此来强制程序员养成良好的编程习惯。并且在Python语言里，缩进而非花括号或者某种关键字，被用于表示语句块的开始和退出。增加缩进表示语句块的开
HttpClient 超时设置 dongwei_6688 httpclient
HttpClient中的超时设置包含两个部分： 1. 建立连接超时，是指在httpclient客户端和服务器端建立连接过程中允许的最大等待时间 2. 读取数据超时，是指在建立连接后，等待读取服务器端的响应数据时允许的最大等待时间在HttpClient 4.x中如下设置： HttpClient httpclient = new DefaultHttpC
小鱼与波浪 dcj3sjt126com
一条小鱼游出水面看蓝天，偶然间遇到了波浪。　　小鱼便与波浪在海面上游戏，随着波浪上下起伏、汹涌前进。　　小鱼在波浪里兴奋得大叫：“你每天都过着这么刺激的生活吗？简直太棒了。”　　波浪说：“岂只每天过这样的生活，几乎每一刻都这么刺激！还有更刺激的，要有潮汐变化，或者狂风暴雨，那才是兴奋得心脏都会跳出来。”　　小鱼说：“真希望我也能变成一个波浪，每天随着风雨、潮汐流动，不知道有多么好！”　　很快，小鱼
Error Code: 1175 You are using safe update mode and you tried to update a table dcj3sjt126com mysql
快速高效用：SET SQL_SAFE_UPDATES = 0；下面的就不要看了！今日用MySQL Workbench进行数据库的管理更新时，执行一个更新的语句碰到以下错误提示： Error Code: 1175 You are using safe update mode and you tried to update a table without a WHERE that
枚举类型详细介绍及方法定义 gaomysion enum javaee
转发 http://developer.51cto.com/art/201107/275031.htm 枚举其实就是一种类型，跟int, char 这种差不多，就是定义变量时限制输入的，你只能够赋enum里面规定的值。建议大家可以看看，这两篇文章，《java枚举类型入门》和《C++的中的结构体和枚举》，供大家参考。枚举类型是JDK5.0的新特征。Sun引进了一个全新的关键字enum
Merge Sorted Array hcx2013 array
Given two sorted integer arrays nums1 and nums2, merge nums2 into nums1 as one sorted array. Note:You may assume that nums1 has enough space (size that is
Expression Language 3.0新特性 jinnianshilongnian el 3.0
Expression Language 3.0表达式语言规范最终版从2013-4-29发布到现在已经非常久的时间了；目前如Tomcat 8、Jetty 9、GlasshFish 4已经支持EL 3.0。新特性包括：如字符串拼接操作符、赋值、分号操作符、对象方法调用、Lambda表达式、静态字段/方法调用、构造器调用、Java8集合操作。目前Glassfish 4/Jetty实现最好，对大多数新特性
超越算法来看待个性化推荐 liyonghui160com 超越算法来看待个性化推荐
一提到个性化推荐，大家一般会想到协同过滤、文本相似等推荐算法，或是更高阶的模型推荐算法，百度的张栋说过，推荐40%取决于UI、30%取决于数据、20%取决于背景知识，虽然本人不是很认同这种比例，但推荐系统中，推荐算法起的作用起的作用是非常有限的。就像任何
写给Javascript初学者的小小建议 pda158 JavaScript
　　一般初学JavaScript的时候最头痛的就是浏览器兼容问题。在Firefox下面好好的代码放到IE就不能显示了，又或者是在IE能正常显示的代码在firefox又报错了。　　如果你正初学JavaScript并有着一样的处境的话建议你：初学JavaScript的时候无视DOM和BOM的兼容性，将更多的时间花在了解语言本身（ECMAScript）。只在特定浏览器编写代码（Chrome/Fi
Java 枚举 ShihLei java enum 枚举
注：文章内容大量借鉴使用网上的资料，可惜没有记录参考地址，只能再传对作者说声抱歉并表示感谢！一基础 1）语法枚举类型只能有私有构造器（这样做可以保证客户代码没有办法新建一个enum的实例）枚举实例必须最先定义 2）特性 &nb
Java SE 6 HotSpot虚拟机的垃圾回收机制 uuhorse java HotSpot GC 垃圾回收 VM
官方资料，关于Java SE 6 HotSpot虚拟机的garbage Collection，非常全，英文。 http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning &