Solr在5.0版本后,不再提供war包部署的方式,取而代之的是内置了jetty服务,
但是我们发现其中并没有内置任何安全性相关检查,任何人如果知道了我们的外网地址就能直接访问并修改其中的索引。经过查找可以使用jetty的方式来限制web访问。
solr/server/solr-webapp/WEB-INF/web.xml中增加以下字段:
- <security-constraint>
- <web-resource-collection>
- <web-resource-name>solrweb-resource-name>
- <url-pattern>/url-pattern>
- web-resource-collection>
- <auth-constraint>
- <role-name>solr_adminrole-name>
- <role-name>adminrole-name>
- auth-constraint>
-
- <login-config>
- <auth-method>BASICauth-method>
- <realm-name>Solr Adminrealm-name>
- login-config>
- security-constraint>
配置验证方式BASIC(用户名密码的方式)。
solr/server/etc/jetty.xml中增加Call标签:
- <Call name="addBean">
- <Arg>
- <New class="org.eclipse.jetty.security.HashLoginService">
- <Set name="name">Solr AdminSet>
- <Set name="config">
- /Users/mazhiqiang/develop/tools/solr-5.5.0/server/etc/realm.properties
- Set>
- <Set name="refreshInterval">0Set>
- New>
- Arg>
- Call>
config中指定密码文件的路径,可以在其中使用来共同组合路径,例如配置了环境变量的情况下,可以使用下面的方式:
- <Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.propertiesSet>
而指定的realm.properties就是相关密码文件了:
设置完成,重新启动solr即可,如果不输入用户名和密码,无法登陆成功:
注意该方法同样会影响HttpSolr连接以及SolrCloud连接,报出下面的错误:
- {"code":500,"codeMsg":"Error from server at http://xxx:8983/solr/brand: Expected mime type application/octet-stream but got text/html. \n\n\nError 401 Unauthorized\n\n
HTTP ERROR 401
\nProblem accessing /solr/brand/select. Reason:\n
Unauthorized
Powered by Jetty://
\n\n\n\n"}
我们可以使用简单验证的方式来测试一下:
- URL url = new URL("http://xxxx:8983/solr/");
- String encoding = Base64.encode("admin:xxxx".getBytes());
-
- HttpURLConnection connection = (HttpURLConnection) url.openConnection();
- connection.setRequestMethod("GET");
- connection.setDoOutput(true);
- connection.setRequestProperty ("Authorization", "Basic " + encoding);
- InputStream content = (InputStream)connection.getInputStream();
- BufferedReader in =
- new BufferedReader (new InputStreamReader(content));
- String line;
- while ((line = in.readLine()) != null) {
- System.out.println(line);
- }
如果加上credentials,是可以成功地将结果html打印出来,否则提示401(Unauthorized)错误。
此时使用SolrCloud的情况下,就不能用CloudSolrClient中内置的httpClient,而只能在外部声明并传入,顺带建立CredentialsProvider,用于Basic权限验证:
- String zkHost = PropertiesUtil.getProperty("zkhost", PROPERTY_FILE_NAME);
- String collection = PropertiesUtil.getProperty(collectionName.getConfigName(), PROPERTY_FILE_NAME);
- int zkClientTimeout = StringUtils
- .parseInt(PropertiesUtil.getProperty("zkClientTimeout", PROPERTY_FILE_NAME));
- int zkConnectTimeout = StringUtils
- .parseInt(PropertiesUtil.getProperty("zkConnectTimeout", PROPERTY_FILE_NAME));
-
- PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
-
- connManager.setMaxTotal(50);
-
- connManager.setDefaultMaxPerRoute(20);
-
-
- RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(2000).setSocketTimeout(2000)
- .setConnectionRequestTimeout(500).setCookieSpec(CookieSpecs.STANDARD)
- .build();
-
- CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
- credentialsProvider.setCredentials(new AuthScope(PropertiesUtil.getProperty("solrHost", PROPERTY_FILE_NAME),
- Integer.parseInt(PropertiesUtil.getProperty("solrPort", PROPERTY_FILE_NAME))),
- new UsernamePasswordCredentials(PropertiesUtil.getProperty("solrUserName", PROPERTY_FILE_NAME),
- PropertiesUtil.getProperty("solrPassword", PROPERTY_FILE_NAME)));
-
- CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(connManager).setDefaultRequestConfig(requestConfig)
- .setConnectionManagerShared(true).setDefaultCredentialsProvider(credentialsProvider).build();
-
- CloudSolrClient cloudSolrClient = new CloudSolrClient(zkHost, httpClient);
此时就可以像以前一样正常查询了,注意在已经添加Credentials情况下,如果此时服务器不需要验证,也不会出错。
对定时full/delta-import的影响
如果增加了安全校验,同样也会影响full-import, delta-import的执行,此时从solr.log日志中就可以拿到错误的信息:
- 2016-08-29 09:10:41.925 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Process started at .............. 29.08.2016 09:10:41 925
- 2016-08-29 09:10:41.948 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Full URL http:
- 2016-08-29 09:10:41.958 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Response message Unauthorized
- 2016-08-29 09:10:41.958 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Response code 401
- 2016-08-29 09:10:41.986 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Disconnected from server localhost
- 2016-08-29 09:10:41.986 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Process ended at ................ 29.08.2016 09:10:41 986
在full-import/delta-import的相关代码中,没有涉及到权限验证部分,直接使用http连接:
- protected void sendHttpPost(String completeUrl, String coreName) {
- DateFormat df = new SimpleDateFormat("dd.MM.yyyy HH:mm:ss SSS");
- Date startTime = new Date();
-
-
- String core = coreName == null ? "" : "[" + coreName + "] ";
-
- logger.info(core
- + " Process started at .............. "
- + df.format(startTime));
-
- try {
-
- URL url = new URL(completeUrl);
- HttpURLConnection conn = (HttpURLConnection) url.openConnection();
-
- conn.setRequestMethod("POST");
- conn.setRequestProperty("type", "submit");
- conn.setDoOutput(true);
-
-
- conn.connect();
-
- logger.info(core + " Full URL\t\t\t\t"
- + conn.getURL());
- logger.info(core + " Response message\t\t\t"
- + conn.getResponseMessage());
- logger.info(core + " Response code\t\t\t"
- + conn.getResponseCode());
-
-
- if (conn.getResponseCode() != 200) {
- reloadParams();
- }
-
- conn.disconnect();
- logger.info(core
- + " Disconnected from server\t\t"
- + server);
- Date endTime = new Date();
- logger.info(core
- + " Process ended at ................ "
- + df.format(endTime));
- } catch (MalformedURLException mue) {
- logger.error("Failed to assemble URL for HTTP POST", mue);
- } catch (IOException ioe) {
- logger.error(
- "Failed to connect to the specified URL while trying to send HTTP POST",
- ioe);
- } catch (Exception e) {
- logger.error("Failed to send HTTP POST", e);
- }
- }
这种方式同样需要增加权限验证, 在conn执行连接之前,要设置认证选项:
- String encoding = Base64.encode(“username:password".getBytes());
- conn.setRequestProperty("Authorization", "Basic " + encoding);
注意当前使用的仅仅是硬编码设置的用户名、密码,修改完成后,要将原有的jar包(apache-solr-dataimportscheduler-1.0.jar)进行重新编译并替换,重启solr服务即可。
使用内置的HttpSolrClient
即并不传入原来的,还有另外一种方法,可用于配置基本用户以及密码认证,在HttpClientConfigurer类中对DefaultHttpClient的参数进行配置时,使用了下面的属性配置:
- final String basicAuthUser = config
- .get(HttpClientUtil.PROP_BASIC_AUTH_USER);
- final String basicAuthPass = config
- .get(HttpClientUtil.PROP_BASIC_AUTH_PASS);
如果使用这种方式,那么在创建内置的HttpSolrClient时需要传入params,在params中加入基本认证即可。
- ModifiableSolrParams params = new ModifiableSolrParams();
- params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 128);
- params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 32);
- params.set(HttpClientUtil.PROP_FOLLOW_REDIRECTS, false);
- params.set(HttpClientUtil.PROP_BASIC_AUTH_USER, "admin");
- params.set(HttpClientUtil.PROP_BASIC_AUTH_PASS, "zhen.com");
- params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 1000);
- params.set(HttpClientUtil.PROP_ALLOW_COMPRESSION, true);
- params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 1000);
- CloseableHttpClient closeableHttpClient = HttpClientUtil.createClient(params);
-
- HttpSolrClient solrClient = new HttpSolrClient(solrurl, closeableHttpClient);
同样的方法也适用于CloudSolrClient。
原文链接:
http://brandnewuser.iteye.com/blog/2318027
Solr在5.0版本后,不再提供war包部署的方式,取而代之的是内置了jetty服务,
但是我们发现其中并没有内置任何安全性相关检查,任何人如果知道了我们的外网地址就能直接访问并修改其中的索引。经过查找可以使用jetty的方式来限制web访问。
solr/server/solr-webapp/WEB-INF/web.xml中增加以下字段:
- <security-constraint>
- <web-resource-collection>
- <web-resource-name>solrweb-resource-name>
- <url-pattern>/url-pattern>
- web-resource-collection>
- <auth-constraint>
- <role-name>solr_adminrole-name>
- <role-name>adminrole-name>
- auth-constraint>
-
- <login-config>
- <auth-method>BASICauth-method>
- <realm-name>Solr Adminrealm-name>
- login-config>
- security-constraint>
配置验证方式BASIC(用户名密码的方式)。
solr/server/etc/jetty.xml中增加Call标签:
- <Call name="addBean">
- <Arg>
- <New class="org.eclipse.jetty.security.HashLoginService">
- <Set name="name">Solr AdminSet>
- <Set name="config">
- /Users/mazhiqiang/develop/tools/solr-5.5.0/server/etc/realm.properties
- Set>
- <Set name="refreshInterval">0Set>
- New>
- Arg>
- Call>
config中指定密码文件的路径,可以在其中使用来共同组合路径,例如配置了环境变量的情况下,可以使用下面的方式:
- <Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.propertiesSet>
而指定的realm.properties就是相关密码文件了:
设置完成,重新启动solr即可,如果不输入用户名和密码,无法登陆成功:
注意该方法同样会影响HttpSolr连接以及SolrCloud连接,报出下面的错误:
- {"code":500,"codeMsg":"Error from server at http://xxx:8983/solr/brand: Expected mime type application/octet-stream but got text/html. \n\n\nError 401 Unauthorized\n\n
HTTP ERROR 401
\nProblem accessing /solr/brand/select. Reason:\n
Unauthorized
Powered by Jetty://
\n\n\n\n"}
我们可以使用简单验证的方式来测试一下:
- URL url = new URL("http://xxxx:8983/solr/");
- String encoding = Base64.encode("admin:xxxx".getBytes());
-
- HttpURLConnection connection = (HttpURLConnection) url.openConnection();
- connection.setRequestMethod("GET");
- connection.setDoOutput(true);
- connection.setRequestProperty ("Authorization", "Basic " + encoding);
- InputStream content = (InputStream)connection.getInputStream();
- BufferedReader in =
- new BufferedReader (new InputStreamReader(content));
- String line;
- while ((line = in.readLine()) != null) {
- System.out.println(line);
- }
如果加上credentials,是可以成功地将结果html打印出来,否则提示401(Unauthorized)错误。
此时使用SolrCloud的情况下,就不能用CloudSolrClient中内置的httpClient,而只能在外部声明并传入,顺带建立CredentialsProvider,用于Basic权限验证:
- String zkHost = PropertiesUtil.getProperty("zkhost", PROPERTY_FILE_NAME);
- String collection = PropertiesUtil.getProperty(collectionName.getConfigName(), PROPERTY_FILE_NAME);
- int zkClientTimeout = StringUtils
- .parseInt(PropertiesUtil.getProperty("zkClientTimeout", PROPERTY_FILE_NAME));
- int zkConnectTimeout = StringUtils
- .parseInt(PropertiesUtil.getProperty("zkConnectTimeout", PROPERTY_FILE_NAME));
-
- PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
-
- connManager.setMaxTotal(50);
-
- connManager.setDefaultMaxPerRoute(20);
-
-
- RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(2000).setSocketTimeout(2000)
- .setConnectionRequestTimeout(500).setCookieSpec(CookieSpecs.STANDARD)
- .build();
-
- CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
- credentialsProvider.setCredentials(new AuthScope(PropertiesUtil.getProperty("solrHost", PROPERTY_FILE_NAME),
- Integer.parseInt(PropertiesUtil.getProperty("solrPort", PROPERTY_FILE_NAME))),
- new UsernamePasswordCredentials(PropertiesUtil.getProperty("solrUserName", PROPERTY_FILE_NAME),
- PropertiesUtil.getProperty("solrPassword", PROPERTY_FILE_NAME)));
-
- CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(connManager).setDefaultRequestConfig(requestConfig)
- .setConnectionManagerShared(true).setDefaultCredentialsProvider(credentialsProvider).build();
-
- CloudSolrClient cloudSolrClient = new CloudSolrClient(zkHost, httpClient);
此时就可以像以前一样正常查询了,注意在已经添加Credentials情况下,如果此时服务器不需要验证,也不会出错。
对定时full/delta-import的影响
如果增加了安全校验,同样也会影响full-import, delta-import的执行,此时从solr.log日志中就可以拿到错误的信息:
- 2016-08-29 09:10:41.925 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Process started at .............. 29.08.2016 09:10:41 925
- 2016-08-29 09:10:41.948 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Full URL http:
- 2016-08-29 09:10:41.958 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Response message Unauthorized
- 2016-08-29 09:10:41.958 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Response code 401
- 2016-08-29 09:10:41.986 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Disconnected from server localhost
- 2016-08-29 09:10:41.986 INFO (Timer-0) [ ] o.a.s.h.d.s.BaseTimerTask [product] Process ended at ................ 29.08.2016 09:10:41 986
在full-import/delta-import的相关代码中,没有涉及到权限验证部分,直接使用http连接:
- protected void sendHttpPost(String completeUrl, String coreName) {
- DateFormat df = new SimpleDateFormat("dd.MM.yyyy HH:mm:ss SSS");
- Date startTime = new Date();
-
-
- String core = coreName == null ? "" : "[" + coreName + "] ";
-
- logger.info(core
- + " Process started at .............. "
- + df.format(startTime));
-
- try {
-
- URL url = new URL(completeUrl);
- HttpURLConnection conn = (HttpURLConnection) url.openConnection();
-
- conn.setRequestMethod("POST");
- conn.setRequestProperty("type", "submit");
- conn.setDoOutput(true);
-
-
- conn.connect();
-
- logger.info(core + " Full URL\t\t\t\t"
- + conn.getURL());
- logger.info(core + " Response message\t\t\t"
- + conn.getResponseMessage());
- logger.info(core + " Response code\t\t\t"
- + conn.getResponseCode());
-
-
- if (conn.getResponseCode() != 200) {
- reloadParams();
- }
-
- conn.disconnect();
- logger.info(core
- + " Disconnected from server\t\t"
- + server);
- Date endTime = new Date();
- logger.info(core
- + " Process ended at ................ "
- + df.format(endTime));
- } catch (MalformedURLException mue) {
- logger.error("Failed to assemble URL for HTTP POST", mue);
- } catch (IOException ioe) {
- logger.error(
- "Failed to connect to the specified URL while trying to send HTTP POST",
- ioe);
- } catch (Exception e) {
- logger.error("Failed to send HTTP POST", e);
- }
- }
这种方式同样需要增加权限验证, 在conn执行连接之前,要设置认证选项:
- String encoding = Base64.encode(“username:password".getBytes());
- conn.setRequestProperty("Authorization", "Basic " + encoding);
注意当前使用的仅仅是硬编码设置的用户名、密码,修改完成后,要将原有的jar包(apache-solr-dataimportscheduler-1.0.jar)进行重新编译并替换,重启solr服务即可。
使用内置的HttpSolrClient
即并不传入原来的,还有另外一种方法,可用于配置基本用户以及密码认证,在HttpClientConfigurer类中对DefaultHttpClient的参数进行配置时,使用了下面的属性配置:
- final String basicAuthUser = config
- .get(HttpClientUtil.PROP_BASIC_AUTH_USER);
- final String basicAuthPass = config
- .get(HttpClientUtil.PROP_BASIC_AUTH_PASS);
如果使用这种方式,那么在创建内置的HttpSolrClient时需要传入params,在params中加入基本认证即可。
- ModifiableSolrParams params = new ModifiableSolrParams();
- params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 128);
- params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 32);
- params.set(HttpClientUtil.PROP_FOLLOW_REDIRECTS, false);
- params.set(HttpClientUtil.PROP_BASIC_AUTH_USER, "admin");
- params.set(HttpClientUtil.PROP_BASIC_AUTH_PASS, "zhen.com");
- params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 1000);
- params.set(HttpClientUtil.PROP_ALLOW_COMPRESSION, true);
- params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 1000);
- CloseableHttpClient closeableHttpClient = HttpClientUtil.createClient(params);
-
- HttpSolrClient solrClient = new HttpSolrClient(solrurl, closeableHttpClient);
同样的方法也适用于CloudSolrClient。
原文链接: http://brandnewuser.iteye.com/blog/2318027