如何给Sphinx做增量索引更新

在利用 Sphinx 做搜索引擎的时候，一般他的索引建立构成有如下几个部分：

1. 固定不变的主索引
2. 增量索引重建
3. 索引数据合并

在实际操作中，需要需要为增量索引的建立创建辅助表，这样才可以记住最后建立索引的记录ID,做实际的增量部分的索引建立。

CREATE TABLE search_counter
(
   counterid INTEGER PRIMARY KEY NOT NULL,
   max_doc_id INTEGER NOT NULL
);

在主索引的数据源中作如下方式的取数据设置
    sql_query_pre            = SET NAMES utf8
    sql_query_pre            = SET SESSION query_cache_type=OFF
    sql_query_pre            = REPLACE INTO search_counter SELECT 1,MAX(pid) FROM cdb_posts  #创建主索引前更改标识位置
    sql_query                 =
                SELECT pid, fid,tid,authorid, dateline,subject, message
                FROM cdb_posts
                WHERE pid > $start AND pid <= $end

    sql_query_range          = SELECT 1, max_doc_id FROM search_counter WHERE counterid = 1
    sql_range_step            = 1000
    sql_ranged_throttle      = 1000
    sql_query_info             = SELECT * FROM cdb_posts WHERE pid=$id

在增量索引的数据源中作如下方式的取数据设置

    sql_query_pre            = SET NAMES utf8
    sql_query_pre            = SET SESSION query_cache_type=OFF
    sql_query                 =
                SELECT pid, fid,tid,authorid, dateline,
                subject, message
                FROM cdb_posts
                WHERE pid >
                 (SELECT max_doc_id FROM search_counter WHERE counterid=1)
                 #增量索引是id大于标识位置的部分

在建立好配置后首先对sphinx中配置的全部索引做初始化

/usr/local/sphinx/bin/indexer –config –all /usr/local/sphinx/etc/sphinx.conf

为创建2个shell脚本，一个用来创建主索引、一个用来创建增量索引（此步可以省略）

1.创建主索引脚本build_main_index.sh

#!/bin/sh

/usr/local/sphinx/bin/searchd –stop >> /var/log/sphinx/searchdlog

/usr/local/sphinx/bin/indexer discuz –config /usr/local/sphinx/etc/sphinx.conf >> /var/log/sphinx/mainindexlog

/usr/local/sphinx/bin/searchd >> /var/log/sphinx/searchdlog

2.创建增量索引脚本build_delta_index.sh

#!/bin/sh

/usr/local/sphinx/bin/searchd –stop >> /var/log/sphinx/searchdlog

/usr/local/sphinx/bin/indexer discuz_delta –config /usr/local/sphinx/etc/sphinx.conf >> /var/log/sphinx/deltaindexlog

#/usr/local/sphinx/bin/indexer –merge discuz discuz_delta –config /usr/local/sphinx/etc/sphinx.conf >> /var/log/sphinx/deltaindexlog

/usr/local/sphinx/bin/searchd >> /var/log/sphinx/searchdlog

在crontab 中添加定时脚本，按照自己期望的策略定期执行重建索引的操作。
比如可以每天凌晨2点执行主索引重建，其他每10分钟建立一次增量索引重建。

 

转载自  http://th9988.blog.163.com/blog/static/4888243220113207210871/