Solr: Data Import Handler (DIH) with mysql

The Handler has to be registered in the solrconfig.xml as follows.

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
    <lst name="defaults">
      <str name="config">/home/username/data-config.xml</str>
    </lst>
  </requestHandler>

he configuration is provided in two places:

• solrconfig.xml . The data config file location is added here
• The datasource also can be added here. Or it can be put directly into the data-config.xml
• data-config.xml
  • How to fetch data (queries,url etc)
  • What to read ( resultset columns, xml fields etc)
  • How to process (modify/add/remove fields)

 

 

Using query attribute for both full and delta import

can't work in solr4.7

 

 

schedule download http://pan.baidu.com/s/1c0in1By

 https://solr-dataimport-scheduler.googlecode.com/files/apache-solr-dataimportscheduler-1.1.jar

  http://blog.csdn.net/xintongfei/article/details/19171381

http://blog.csdn.net/ltr15036900300/article/details/23945515

 

 

date time type

http://www.niwozhi.net/demo_c82_i50518.html

Zulu time

http://www.timeanddate.com/library/abbreviations/timezones/military/z.html

 

deleted index that  docments deleted in mysql

http://stackoverflow.com/questions/12174115/deleting-unwanted-data-in-incremental-imports-when-using-solr-dih

http://solr.pl/en/2011/01/03/data-import-handler-%E2%80%93-removing-data-from-index/

 

http://blog.griddynamics.com/2013/09/solr-block-join-support.html

 

 

－－－－－－－－－－－－－－－－－－

import xml to solr index

<dataConfig>
 <dataSource type="FileDataSource" encoding="utf-8"  />
  <document>
        <entity rootEntity="true" name="question" processor="XPathEntityProcessor" stream="true" forEach="/docs/doc/"  url="/home/tomcat/test/questionTags.xml">
           <field column="id" xpath="/docs/doc/id" />
           <field column="tags" xpath="/docs/doc/tags" />
           <field column="content" xpath="/docs/doc/content" />
       </entity>

    </document>
</dataConfig>

 

 

 

org.apache.solr.common.SolrException; null:java.lang.StackOverflowError

 

 

 

 

 

 

 

 

 

References

http://wiki.apache.org/solr/DataImportHandler

http://www.yeeach.com/post/1150

 

http://www.cnblogs.com/chenying99/archive/2012/09/08/2676208.html

http://lucene.472066.n3.nabble.com/DIH-deleting-documents-high-performance-delta-imports-and-passing-parameters-td1388349.html

 

http://www.solrcn.com/books/#72-en