apache camel 用来复制网站页面

阅读更多

我需要将一些客户的网站从其它地方迁移到我的诗篇在线建站系统中去，由此开始了将近一个星期的与camel的接触，确实是一个宏大的程序。先看一张camel的router。apache camel 路由图


对应的java DSL：

from("jms:TOOL.CLONE.PAGE").beanRef("clonepage")
    										.split().body()
    										.choice()
    										.when().mvel("request.body[0] == 'img'").beanRef("clonepage", "processImg")
    										.when().mvel("request.body[0] == 'css'").beanRef("clonepage", "processCss").to("direct:cssimg")
    										.when().mvel("request.body[0] == 'script'").beanRef("clonepage", "processScript")
    										.otherwise().beanRef("clonepage", "processOthers");
    	from("direct:cssimg").split().body().beanRef("clonepage", "processImg");

流程说明：

从诗篇建站系统的后台发出一个复制页面的指令，将要复制的页面的参数包装后发送到activemq去。接下来就交给apache camel处理了，这是运行在另一个VM里面，和建站系统是解构的。camel从消息队列中获取消息，然后将要克隆的页面取下来，这是第一步，因为取下来的页面中包含许多img，css，javascript链接，这些链接在clonepage的默认处理方法中获取，然后交个下一个路由，当然css里面还包含有img图片，必须额外处理。



先看看克隆的效果吗？这是iteye的首页克隆: http://demo.m3958.com/iteye ，请注意没有任何手工修改，完全是程序自动克隆的。

这是guice配置：

    @Override
    protected void configure() {
        super.configure();

        // lets add in any RouteBuilder instances we want to use
        bind(MyRouteBuilder.class);
        bind(Printer.class);
    }

    /**
     * Lets configure the JMS component, parameterizing some properties from the
     * jndi.properties file
     */
    @Provides
    @JndiBind("jms")
    JmsComponent jms(@Named("activemq.brokerURL") String brokerUrl) {
        return JmsComponent.jmsComponent(new ActiveMQConnectionFactory(brokerUrl));
    }
    
    @Provides
    @JndiBind("myBean") 
    SomeBean someBean(Injector injector) {
        return injector.getInstance(SomeBean.class); 
    }
    
	private static final String PERSISTENCE_UNIT_NAME = "p-unit";
	public static EntityManagerFactory factory = Persistence.createEntityManagerFactory(PERSISTENCE_UNIT_NAME);
    
//	@Provides
//    EntityManager emp(){
//    	return MyModule.factory.createEntityManager();
//    }
	
    @Provides
    @JndiBind("hpb") 
    HttpProcessBean hpb(Injector injector) {
        return injector.getInstance(HttpProcessBean.class); 
    }
    
    @Provides
    @JndiBind("extrator")
    ArticleExtrator extrator(Injector injector){
    	return injector.getInstance(ArticleExtrator.class);
    }
    
    
    @Provides
    @JndiBind("clonepage")
    ClonePage clonepage(Injector injector){
    	return injector.getInstance(ClonePage.class);
    }

处理css的代码片段：

	public void processCss(@Header("sitetr") String siteThemeRoot, @MVEL("request.body[1]") final String pageUrl,@MVEL("request.body[2]") String saveFileName,Exchange exchange){
		ProducerTemplate template = context.createProducerTemplate();
		
		//in-out pattern.
    	Exchange ecc = template.send("http4://localhost",new Processor() {
			@Override
			public void process(Exchange exchange) throws Exception {
				exchange.getIn().setHeader(Exchange.HTTP_URI,"http4://" + pageUrl);
			}
		});
    	
    	Message out = ecc.getOut();
    	String c = out.getBody(String.class);
    	List relations = new ArrayList();
    	//url(../Images/lian.gif)
    	Pattern p = Pattern.compile("url\\s*?\\(['\"\\s]*(.*?)['\"\\s]*\\)");
    	Matcher m = p.matcher(c);
    	String[] hu = getHostnameUri(pageUrl);
    	while(m.find()){
    		String url = m.group(1);
    		relations.add(getOneItem(url, "img", null, hu[0], hu[1]));
    	}
    	exchange.getOut().setHeader("sitetr", siteThemeRoot);
    	exchange.getOut().setBody(relations);
    	template.sendBodyAndHeader("file://" + siteThemeRoot,c,Exchange.FILE_NAME,saveFileName);
	}