Java版本的BloomFilter (布隆过滤器)

一般比较常见的应用是字符串去重..也就是采集网址去重.防止重复采集

Java代码  

BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream("D:\\Users\\caiqing\\workspace\\CQ\\library\\dictionary-utf8.TXT"),"UTF-8")) ;
        String str = null ;
        System.out.println("begin");
        long start = System.currentTimeMillis() ;
        while((str=br.readLine())!=null){
            if(bf.containsAndAdd(str)){
                System.out.println("containsAndAdd:"+str);
            }
        }

        br.close() ;

        br = new BufferedReader(new InputStreamReader(new FileInputStream("D:\\Users\\caiqing\\workspace\\CQ\\library\\dictionary-utf8.TXT"),"UTF-8")) ;
            System.out.println("begin-find");
            start = System.currentTimeMillis() ;
            while((str=br.readLine())!=null){
                if(!bf.contains(str)){
                    System.out.println("contains:"+str);
                }
            }

        System.out.println(System.currentTimeMillis()-start);
        br.close() ;