Days before, I've submitted an application to participate in Apache Mahout and at this time, have got a reply from the guru of this project. It inspired us with lots of courages. We decided that if I were selected by ASF, we will integrate redpoll into Mahout which has the same end goals, same license with our project. We believe that there is a lot of synergies if we work together with ASF.
However, all of these are based on an
if, we will keep working during the time they making their decision. Our short-term goal is listed below,
April 19th, finish learning the coding style of hadoop and implmenting Naive Bayes classifier.
May 3rd, finish parallelizing EM clustering algorithm which can work together with Canopy.
May 24th, SVM classifier implementation deadline.
At present, We are also doing some preparations like learning something about data mining, thinking about how to parallize them and finding some large data sets we havenot.
BTW, through reading the source code of Mahout these days, I found that those two clustering algorithms can only deal with double values. IMHO, the text data format can be uniformed for most of data mining algorithms. If we have an infrastructure for parsing data types like numeric, nominal, date, etc and organizing them into some certain data structures, the Mahout will be more efficient and more practical by the means of supporting more data types.