solr官网:https://lucene.apache.org/solr/
下载:https://lucene.apache.org/solr/mirrors-solr-latest-redir.html
文档:https://lucene.apache.org/solr/guide/
tar xzvf /data/upload/solr-7.7.1.tgz -C /usr/local/
mv /usr/local/solr-7.7.1/ /usr/local/solr
现在的solr由于已经集成了jetty,所以可以直接利用启动:
/usr/local/solr/bin/solr start -p 8080 -force
创建一个solr工作目录(/solr):
mkdir -p /solr/{webapp,home,logs}
在solr里面提供有管理控制台的工具,可以将这些代码拷贝到之前创建的目录之中:
cp -r /usr/local/solr/server/solr-webapp/webapp/* /solr/webapp/
拷贝依赖程序库
cp -r /usr/local/solr/server/lib/ext/* /solr/webapp/WEB-INF/lib/
cp -r /usr/local/solr/server/lib/metrics-*.* /solr/webapp/WEB-INF/lib
cp -r /usr/local/solr/dist/solr-dataimporthandler-* /solr/webapp/WEB-INF/lib/
solr中需要准备出一个home目录作为solr主目录,通过已有的目录复制
cp -r /usr/local/solr/server/solr/* /solr/home/
修改web.xml配置文件,取消安全配置:vim /solr/webapp/WEB-INF/web.xml
solr/home
/solr/home
java.lang.String
拷贝日志文件,放在/solr/webapp/WEB-INF/classes
mkdir /solr/webapp/WEB-INF/classes
cp -r /usr/local/solr/server/resources/log4j2.xml /solr/webapp/WEB-INF/classes/
而后需要在tomcat中设置JAVA_OPTS,bin/setclasspath.sh中加入
JAVA_OPTS="-Dsolr.log.dir=/solr/logs"
修改tomcat中server.xml配置虚拟目录
而后启动tomcat
/usr/local/tomcat/bin/catalina.sh start
把防火墙关了,或者开放端口。
地址:http://192.168.10.130:8080/solr/index.html
在solr_home目录下创建一个"solr-core/conf" 目录:
mkdir -p /solr/home/solr-core/conf
在整体的solrcore配置中需要一系列配置文件和依赖库:
cp -r /usr/local/solr/server/solr/configsets/_default/conf/* /solr/home/solr-core/conf/
cp -r /usr/local/solr/contrib/ /solr/home
cp -r /usr/local/solr/dist /solr/home
真正工作目录是“/solr/home/solr-core”目录,但是这里面需要依赖于之前拷贝的程序库文件,所以要手工修改solrconfig.xml配置文件,配置访问路径:vim /solr/home/solr-core/conf/solrconfig.xml
首先需要数据库连接
mkdir -p /solr/home/contrib/dataimporthandler/lib
cp /data/upload/mysql-connector-java-5.1.38.jar /solr/home/contrib/dataimporthandler/lib
cp /usr/local/solr/dist/solr-dataimporthandler-* /solr/home/contrib/dataimporthandler/lib
修改solrconfig.xml进行路径的配置:vim /solr/home/solr-core/conf/solrconfig.xml
创建一张表:
drop database if exists raoll_shop;
create database raoll_shop character set utf8;
use raoll_shop;
create table goods(
gid bigint auto_increment,
name varchar(50),
catalog varchar(50),
provider varchar(50),
price double,
note text,
photo varchar(50),
recdate datetime default now(),
isdelete int default 0,
constraint pk_gid primary key(gid)
)engine=innodb;
在solrconfig.xml配置文件里面还需要添加一个数据的导入文件配置路径(data-config.xml)
data-config.xml
创建data-config于solrconfig.xml同级,vim /solr/home/solr-core/conf/data-config.xml
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://47.101.195.95:3306/raoll_shop"
user="root"
password="******"/>
query="select gid,name,catalog,provider,price,note,photo,recdate,isdelete from goods"
deltaImportQuery="select gid,name,catalog,provider,price,note,photo,recdate,isdelete from goods where gid='${da
taimporter.delta.gid}'"
deltaQuery="select gid from goods where recdate > '${dataimporter.last_index_time}'"
deletedPkQuery="select gid from goods where isdelete=1">
红色的为增量索引配置。
修改managed-schema.xml配置文件:vim /solr/home/solr-core/conf/managed-schema
重启tomcat,在控制台导入数据、查询数据等。(数据查询、索引维护 百度)
需要定时重建索引(完全导入和增量索引)
ikanalyzer配置 git:https://github.com/magese/ik-analyzer-solr7(下载相应的文件和jar)
cp /data/upload/ik-analyzer-7.7.1.jar /solr/webapp/WEB-INF/lib/
cp /data/upload/resources/ /solr/webapp/WEB-INF/classes/
在manager-schema,添加ik分词器,vim /solr/home/solr-core/conf/managed-schema
重启tomcat。
org.apache.solr
solr-solrj
7.7.1
solr本身是基于http上的路径的配置操作,所以在进行访问的时候虽然使用的是solrj,但实际上它的内部采用的是httpclient实现的调用
package com.heyun.medicine.utils;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.impl.HttpSolrClient;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
import java.util.List;
import java.util.Map;
public class SolrJBasicQuery {
public static final String SOLR_HOST_URL = "http://192.168.10.130:8080/solr/raoll-core";
public static void main(String[] args) throws Exception {
HttpSolrClient solrClient = new HttpSolrClient.Builder(SOLR_HOST_URL).build();
SolrQuery query = new SolrQuery();
query.setStart(0);
query.setRows(5);
query.setQuery("goods_keywords:*22*");
query.setSort("solr_price", SolrQuery.ORDER.desc);
query.setHighlight(true);
query.addHighlightField("solr_name");
query.setHighlightSimplePre("");
query.setHighlightSimplePost("");
QueryResponse response = solrClient.query(query);
SolrDocumentList documents = response.getResults();
System.out.println("[数据行数]" + documents.getNumFound());
for (SolrDocument doc : documents) {
System.out.println("[返回信息]id = " + doc.get("id") + "、name = " + doc.get("solr_name") + "、catalog = " + doc.get("solr_catalog"));
}
System.out.println("--------------------------------");
Map>> map = response.getHighlighting();
for (SolrDocument doc : documents) {
Map> resultMap = map.get(doc.get("id"));
System.out.println(resultMap.get("solr_name"));
}
solrClient.close();
}
}
可与spring整合,spring也提供有spring-data-solr。
在整体的solrcloud架构之中,具体的操作形式如下:
①、solrcloud需要利用公共的zookeeper保持所有的solr主机的注册信息(将每一个core中的conf目录的内容进行公共存储);
②、一旦搭建了solrcloud集群,那么所有的数据的操作都将以collection为主,在一个collection下可以有若干个shard(分片),而后每一个分片上都会有core(每一个core都会存在有主从关系);
③、这些collection、shared、core的分片的信息都会自动的在zookeeper上进行存储;
④、对于整体的solr而言,需要提供统一的认证信息,这些信息也需要保持在zookeeper上;
⑤、solrcloud整体设计较为先进,采用了去中心化的设计思想,即:在整体的开发之中,solr客户端如果要想访问solr集群,需要将所有的主机全部定义在访问主机列表信息上;
⑥、如果要是更加完善则应该准备一个zookeeper集群。
三台服务器,192.168.10.130,192.168.10.107(复制130),192.168.10.115(复制130)。下面的操作三台服务器都需要配置。
tar xzvf /data/upload/zookeeper-3.4.10.tar.gz -C /usr/local/
mv /usr/local/zookeeper-3.4.10/ /usr/local/zookeeper
创建zookeeper数据保持目录:
mkdir -p /usr/data/zookeeper
拷贝模板文件:
cp /usr/local/zookeeper/conf/zoo_sample.cfg /usr/local/zookeeper/conf/zoo.cfg
修改文件:vim /usr/local/zookeeper/conf/zoo.cfg
dataDir=/usr/data/zookeeper
server.1=192.168.10.130:2888:3888
server.2=192.168.10.107:2888:3888
server.3=192.168.10.115:2888:3888
每一台机器把相应的编号发送的数据目录的myid中,130的:echo 1 >> /usr/data/zookeeper/myid
启动zookeeper:/usr/local/zookeeper/bin/zkServer.sh start ,启动前把相应的端口开放或者直接关闭防火墙
查看状态:/usr/local/zookeeper/bin/zkServer.sh status
修改solrcloud配置:vim /solr/home/solr.xml
${host:192.168.10.130}
${jetty.port:8080}
增加zookeeper配置:vim /usr/local/tomcat/bin/setclasspath.sh
JAVA_OPTS="-Dsolr.log.dir=/solr/logs -DzkHost=192.168.10.130:2181,192.168.10.107:2181,192.168.10.115:2181"
启动集群中所有的主机:
/usr/local/tomcat/bin/catalina.sh start
当服务启动之后可以正确的在zookeeper里面进行注册,登陆到zookeeper中:/usr/local/zookeeper/bin/zkCli.sh,可以看到以下节点
[zookeeper, overseer, aliases.json, live_nodes, collections, overseer_elect, security.json, clusterstate.json, autoscaling, autoscaling.json]
solrcloud里面核心的因素在于collection,那么可以发现此时可以直接提供有一个collection配置
/usr/local/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost 192.168.10.130:2181 -cmd upconfig -confdir /solr/home/solr-core/conf/ -confname raoll-conf
配置成功之后在zookeeper中 ls /configs 可以发现raoll-conf配置项。而后可以在控制台配置collection了。
在集群里面对于安全的配置需要上传到zookeeper中,在solrcloud之中安全配置需要有一个“security.json”文件进行存储。
文档:https://lucene.apache.org/solr/guide/7_7/authentication-and-authorization-plugins.html
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{"name":"security-edit",
"role":"admin"}],
"user-role":{"solr":"admin"}
}}
在/usr/local中创建security.json:vim /usr/local/security.json
{
"authentication":{
"blockUnknown":true,
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{"name":"security-edit",
"role":"admin"}],
"user-role":{"solr":"admin"}
}}
"blockUnknown":true 表示未认证的不给通过。
将security.json上传到zookeeper之中:
/usr/local/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost 192.168.10.130:2181 -cmd putfile /security.json /usr/local/security.json
添加用户:
curl --user solr:SolrRocks http://192.168.10.130:8080/solr/admin/authentication -H 'Content-type:application/json' -d '{"set-user":{"raoll":"hello"}}'
可以从控制台看到security.json的内容
为了安全应该删除原始的solr账号:
curl --user solr:SolrRocks http://192.168.10.130:8080/solr/admin/authentication -H 'Content-type:application/json' -d '{"delete-user": ["solr"]}'
package com.heyun.medicine.utils;
import org.apache.http.HttpException;
import org.apache.http.HttpHost;
import org.apache.http.HttpRequest;
import org.apache.http.HttpRequestInterceptor;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.AuthState;
import org.apache.http.auth.ContextAwareAuthScheme;
import org.apache.http.auth.Credentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.impl.auth.BasicScheme;
import org.apache.http.protocol.HttpContext;
import java.io.IOException;
public class AuthRequestInterceptor implements HttpRequestInterceptor {
private ContextAwareAuthScheme authScheme = new BasicScheme();
@Override
public void process(HttpRequest httpRequest, HttpContext httpContext) throws HttpException, IOException {
AuthState authState = (AuthState) httpContext.getAttribute(HttpClientContext.TARGET_AUTH_STATE);
if (authState != null && authState.getAuthScheme() == null) {
CredentialsProvider provider = (CredentialsProvider) httpContext.getAttribute(HttpClientContext.CREDS_PROVIDER);
HttpHost httpHost = (HttpHost) httpContext.getAttribute(HttpClientContext.HTTP_TARGET_HOST);
Credentials credentials = provider.getCredentials(new AuthScope(httpHost.getHostName(), httpHost.getPort()));
if (credentials == null) {
throw new HttpException("{" + httpHost.getHostName() + "}并没有HTTP认证处理支持");
}
httpRequest.addHeader(authScheme.authenticate(credentials, httpRequest, httpContext));
}
}
}
package com.heyun.medicine.utils;
import org.apache.http.client.HttpClient;
import org.apache.solr.client.solrj.impl.CloudSolrClient;
import org.apache.solr.client.solrj.impl.HttpClientUtil;
import org.apache.solr.common.params.ModifiableSolrParams;
import java.util.ArrayList;
import java.util.List;
public class SolrConnectionUtil {
private static final String USER_NAME = "raoll";
private static final String PASSWORD = "hello";
private static final int CONNECT_TIMEOUT = 6000;
private static final int SOCKET_TIMEOUT = 6000;
private static final int CONNECTION_MAX = 1000;
private static final int HOST_CONNECTION_MAX = 1000;
private SolrConnectionUtil() {
}
public static CloudSolrClient getClient() {
List solrHostList = new ArrayList<>();
solrHostList.add("http://192.168.10.130:8080/solr");
solrHostList.add("http://192.168.10.107:8080/solr");
solrHostList.add("http://192.168.10.115:8080/solr");
ModifiableSolrParams initParams = new ModifiableSolrParams();
initParams.set(HttpClientUtil.PROP_BASIC_AUTH_USER, USER_NAME);
initParams.set(HttpClientUtil.PROP_BASIC_AUTH_PASS, PASSWORD);
initParams.set(HttpClientUtil.PROP_ALLOW_COMPRESSION, true);
initParams.set(HttpClientUtil.PROP_FOLLOW_REDIRECTS, false);
initParams.set(HttpClientUtil.PROP_MAX_CONNECTIONS, CONNECTION_MAX);
initParams.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, HOST_CONNECTION_MAX);
HttpClientUtil.addRequestInterceptor(new AuthRequestInterceptor());
HttpClient httpClient = HttpClientUtil.createClient(initParams);
CloudSolrClient cloudSolrClient = new CloudSolrClient.Builder(solrHostList).withConnectionTimeout(CONNECT_TIMEOUT)
.withSocketTimeout(SOCKET_TIMEOUT).withHttpClient(httpClient).build();
cloudSolrClient.setDefaultCollection("raollcollection");
return cloudSolrClient;
}
}
package com.heyun.medicine.utils;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.impl.CloudSolrClient;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
import java.util.List;
import java.util.Map;
public class CloudSolrBasic {
public static void main(String[] args) throws Exception {
CloudSolrClient solrClient = SolrConnectionUtil.getClient();
SolrQuery query = new SolrQuery();
query.setStart(0);
query.setRows(5);
query.setQuery("goods_keywords:*小*");
query.setSort("solr_price", SolrQuery.ORDER.desc);
query.setHighlight(true);
query.addHighlightField("solr_name");
query.setHighlightSimplePre("");
query.setHighlightSimplePost("");
QueryResponse response = solrClient.query(query);
SolrDocumentList documents = response.getResults();
System.out.println("[数据行数]" + documents.getNumFound());
for (SolrDocument doc : documents) {
System.out.println("[返回信息]id = " + doc.get("id") + "、name = " + doc.get("solr_name") + "、catalog = " + doc.get("solr_catalog"));
}
System.out.println("--------------------------------");
Map>> map = response.getHighlighting();
for(SolrDocument doc:documents){
Map> resultMap = map.get(doc.get("id"));
System.out.println(resultMap.get("solr_name"));
}
}
}
可与spring整合,spring也提供有spring-data-solr。