Elasticsearch Reference [5.2] » Setup Elasticsearch
- Downloading
- Installing
- Starting
- Configuring
JAVA_HOMEenvironment variable.
-server from jvm.optionsand if you are using any 32-bit JVM you should reconfigure the thread stack size from
-Xss1m to
-Xss320k.
Installing Elasticsearchedit
Elasticsearch is provided in the following package formats:
|
|
The Install Elasticsearch with |
|
|
The Install Elasticsearch with Debian Package |
|
|
The Install Elasticsearch with RPM |
|
|
An image is available for running Elasticsearch as a Docker container. It ships with X-Pack pre-installed and may be downloaded from the Elastic Docker Registry. Install Elasticsearch with Docker |
Configuration Management Toolsedit
We also provide the following configuration management tools to help with large deployments:
| Puppet |
puppet-elasticsearch |
| Chef |
cookbook-elasticsearch |
| Ansible |
ansible-elasticsearch |
Install Elasticsearch with .zip or .tar.gzedit
Elasticsearch is provided as a .zip and as a .tar.gz package. These packages can be used to install Elasticsearch on any system and are the easiest package format to use when trying out Elasticsearch.
The latest stable version of Elasticsearch can be found on the Download Elasticsearch page. Other versions can be found on the Past Releases page.
Elasticsearch requires Java 8 or later. Use the official Oracle distribution or an open-source distribution such as OpenJDK.
Download and install the .zip packageedit
The .zip archive for Elasticsearch v5.2.1 can be downloaded and installed as follows:
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.2.1.zip sha1sum elasticsearch-5.2.1.zipunzip elasticsearch-5.2.1.zip cd elasticsearch-5.2.1/
![]()
| |
Compare the SHA produced by |
| |
This directory is known as |
Download and install the .tar.gz packageedit
The .tar.gz archive for Elasticsearch v5.2.1 can be downloaded and installed as follows:
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.2.1.tar.gz sha1sum elasticsearch-5.2.1.tar.gztar -xzf elasticsearch-5.2.1.tar.gz cd elasticsearch-5.2.1/
![]()
| |
Compare the SHA produced by |
| |
This directory is known as |
Running Elasticsearch from the command lineedit
Elasticsearch can be started from the command line as follows:
./bin/elasticsearch
By default, Elasticsearch runs in the foreground, prints its logs to the standard output (stdout), and can be stopped by pressing Ctrl-C.
Checking that Elasticsearch is runningedit
You can test that your Elasticsearch node is running by sending an HTTP request to port 9200 on localhost:
GET /
which should give you a response something like this:
{"name":"Cp8oag6","cluster_name":"elasticsearch","cluster_uuid":"AT69_T_DTp-1qgIJlatQqA","version":{"number":"5.2.1","build_hash":"f27399d","build_date":"2016-03-30T09:51:41.449Z","build_snapshot":false,"lucene_version":"6.4.1"},"tagline":"You Know, for Search"}
Log printing to stdout can be disabled using the -q or --quiet option on the command line.
Running as a daemonedit
To run Elasticsearch as a daemon, specify -d on the command line, and record the process ID in a file using the -p option:
./bin/elasticsearch -d -p pid
Log messages can be found in the $ES_HOME/logs/ directory.
To shut down Elasticsearch, kill the process ID recorded in the pid file:
kill `cat pid`
The startup scripts provided in the RPM and Debian packages take care of starting and stopping the Elasticsearch process for you.
Configuring Elasticsearch on the command lineedit
Elasticsearch loads its configuration from the $ES_HOME/config/elasticsearch.yml file by default. The format of this config file is explained in Configuring Elasticsearch.
Any settings that can be specified in the config file can also be specified on the command line, using the -E syntax as follows:
./bin/elasticsearch -d -Ecluster.name=my_cluster -Enode.name=node_1
Typically, any cluster-wide settings (like cluster.name) should be added to the elasticsearch.yml config file, while any node-specific settings such as node.namecould be specified on the command line.
Directory layout of .zip and .tar.gz archivesedit
The .zip and .tar.gz packages are entirely self-contained. All files and directories are, by default, contained within $ES_HOME — the directory created when unpacking the archive.
This is very convenient because you don’t have to create any directories to start using Elasticsearch, and uninstalling Elasticsearch is as easy as removing the $ES_HOME directory. However, it is advisable to change the default locations of the config directory, the data directory, and the logs directory so that you do not delete important data later on.
| Type | Description | Default Location | Setting |
|---|---|---|---|
| home |
Elasticsearch home directory or |
Directory created by unpacking the archive |
|
| bin |
Binary scripts including |
|
|
| conf |
Configuration files including |
|
|
| data |
The location of the data files of each index / shard allocated on the node. Can hold multiple locations. |
|
|
| logs |
Log files location. |
|
|
| plugins |
Plugin files location. Each plugin will be contained in a subdirectory. |
|
|
| repo |
Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here. |
Not configured |
|
| script |
Location of script files. |
|
|
Next stepsedit
You now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, you will need to do some additional setup:
- Learn how to configure Elasticsearch.
- Configure important Elasticsearch settings.
- Configure important system settings.
Install Elasticsearch with Dockeredit
Elasticsearch is also available as a Docker image. The image is built with X-Pack.
Security noteedit
X-Pack is preinstalled in this image. Please take a few minutes to familiarize yourself with X-Pack Security and how to change default passwords. The default password for the elastic user is changeme.
X-Pack includes a trial license for 30 days. After that, you can obtain one of the available subscriptions or disable Security. The Basic license is free and includes the Monitoring extension.
Obtaining Elasticsearch for Docker is as simple as issuing a docker pull command against the Elastic Docker registry.
The Docker image can be retrieved with the following command:
docker pull docker.elastic.co/elasticsearch/elasticsearch:5.2.1
Running Elasticsearch from the command lineedit
Development modeedit
Elasticsearch can be quickly started for development or testing use with the following command:
docker run -p 9200:9200-e "http.host=0.0.0.0"-e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.2.1
Production modeedit
The vm_max_map_count kernel setting needs to be set to at least 262144 for production use. Depending on your platform:
-
Linux
The
vm_map_max_countsetting should be set permanently in /etc/sysctl.conf:$ grep vm.max_map_count /etc/sysctl.conf vm.max_map_count=262144
To apply the setting on a live system type:
sysctl -w vm.max_map_count=262144 -
OSX with Docker for Mac
The
vm_max_map_countsetting must be set within the xhyve virtual machine:$ screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
Log in with root and no password. Then configure the
sysctlsetting as you would for Linux:sysctl -w vm.max_map_count=262144
-
OSX with Docker Toolbox
The
vm_max_map_countsetting must be set via docker-machine:docker-machine ssh sudo sysctl -w vm.max_map_count=262144
The following example brings up a cluster comprising two Elasticsearch nodes. To bring up the cluster, use the docker-compose.yml and just type:
docker-compose up
docker-compose is not pre-installed with Docker on Linux. Instructions for installing it can be found on the docker-compose webpage.
The node elasticsearch1 listens on localhost:9200 while elasticsearch2 talks to elasticsearch1 over a Docker network.
This example also uses Docker named volumes, called esdata1 and esdata2 which will be created if not already present.
docker-compose.yml:
version: '2'services: elasticsearch1: image: docker.elastic.co/elasticsearch/elasticsearch:5.2.1 container_name: elasticsearch1 environment: - cluster.name=docker-cluster - bootstrap.memory_lock=true -"ES_JAVA_OPTS=-Xms512m -Xmx512m"ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 mem_limit: 1g cap_add: - IPC_LOCK volumes: - esdata1:/usr/share/elasticsearch/data ports: - 9200:9200 networks: - esnet elasticsearch2: image: docker.elastic.co/elasticsearch/elasticsearch:5.2.1 environment: - cluster.name=docker-cluster - bootstrap.memory_lock=true -"ES_JAVA_OPTS=-Xms512m -Xmx512m"-"discovery.zen.ping.unicast.hosts=elasticsearch1"ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 mem_limit: 1g cap_add: - IPC_LOCK volumes: - esdata2:/usr/share/elasticsearch/data networks: - esnet volumes: esdata1: driver: local esdata2: driver: local networks: esnet: driver: bridge
To stop the cluster, type docker-compose down. Data volumes will persist, so it’s possible to start the cluster again with the same data using docker-compose up. To destroy the cluster and the data volumes just type docker-compose down -v.
Inspect status of cluster:edit
curl -u elastic http://127.0.0.1:9200/_cat/health Enter host password for user 'elastic':147222592915:38:49 docker-cluster green 22420000-100.0%
Log messages go to the console and are handled by the configured Docker logging driver. By default you can access logs with docker logs.
Configuring Elasticsearch with Dockeredit
Elasticsearch loads its configuration from files under /usr/share/elasticsearch/config/. These configuration files are documented in Configuring Elasticsearch and Setting JVM options.
The image offers several methods for configuring Elasticsearch settings with the conventional approach being to provide customized files, i.e. elasticsearch.yml, but it’s also possible to use environment variables to set options:
A. Present the parameters via Docker environment variablesedit
For example, to define the cluster name with docker run you can pass -e "cluster.name=mynewclustername". Double quotes are required.
There is a difference between defining default settings and normal settings. The former are prefixed with default. and cannot override normal settings, if defined.
B. Bind-mounted configurationedit
Create your custom config file and mount this over the image’s corresponding file. For example, bind-mounting a custom_elasticsearch.yml with docker run can be accomplished with the parameter:
-v full_path_to/custom_elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
custom_elasticsearch.yml should be readable by uid:gid 1000:1000
C. Customized imageedit
In some environments, it may make more sense to prepare a custom image containing your configuration. A Dockerfile to achieve this may be as simple as:
FROM docker.elastic.co/elasticsearch/elasticsearch:5.2.1 ADD elasticsearch.yml /usr/share/elasticsearch/config/ USER root chown elasticsearch:elasticsearch config/elasticsearch.yml USER elasticsearch
You could then build and try the image with something like:
docker build --tag=elasticsearch-custom . docker run -ti -v /usr/share/elasticsearch/data elasticsearch-custom
D. Override the image’s default CMDedit
Options can be passed as command-line options to the Elasticsearch process by overriding the default command for the image. For example:
docker run <various parameters> bin/elasticsearch -Ecluster.name=mynewclustername
Notes for production use and defaultsedit
We have collected a number of best practices for production use.
Any Docker parameters mentioned below assume the use of docker run.
-
It is important to correctly set capabilities and ulimits via the Docker CLI. As seen earlier in the example docker-compose.yml, the following options are required:
--cap-add=IPC_LOCK --ulimit memlock=-1:-1 --ulimit nofile=65536:65536
-
Ensure
bootstrap.memory_lockis set totrueas explained in "Disable swapping".This can be achieved through any of the configuration methods, e.g. by setting the appropriate environments variable with
-e "bootstrap.memory_lock=true". - The image exposes TCP ports 9200 and 9300. For clusters it is recommended to randomize the published ports with
--publish-all, unless you are pinning one container per host. - Use the
ES_JAVA_OPTSenvironment variable to set heap size, e.g. to use 16GB use-e ES_JAVA_OPTS="-Xms16g -Xmx16g"withdocker run. It is also recommended to set a memory limit for the container. - Pin your deployments to a specific version of the Elasticsearch Docker image, e.g.
docker.elastic.co/elasticsearch/elasticsearch:5.2.1. -
Always use a volume bound on
/usr/share/elasticsearch/data, as shown in the production example, for the following reasons:- The data of your elasticsearch node won’t be lost if the container is killed
- Elasticsearch is I/O sensitive and the Docker storage driver is not ideal for fast I/O
- It allows the use of advanced Docker volume plugins
- If you are using the devicemapper storage driver (default on at least RedHat (rpm) based distributions) make sure you are not using the default
loop-lvmmode. Configure docker-engine to use direct-lvm instead. - Consider centralizing your logs by using a different logging driver. Also note that the default json-file logging driver is not ideally suited for production use.
Next stepsedit
You now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, you will need to do some additional setup:
- Learn how to configure Elasticsearch.
- Configure important Elasticsearch settings.
- Configure important system settings.
node.name and paths), or settings which a node requires in order to be able to join a cluster, such as
cluster.nameand
network.host.
elasticsearch.ymlfor configuring Elasticsearch, andlog4j2.propertiesfor configuring Elasticsearch logging.
- elasticsearch.yml，用于配置elasticsearch
- log4j2.properties， 用于配置Elasticsearch日志打印
$ES_HOME/config/. The Debian and RPM packages set the config directory location to
/etc/elasticsearch/.
path.conf setting, as follows:
./bin/elasticsearch -Epath.conf=/path/to/my/config/
path: data: /var/lib/elasticsearch logs: /var/log/elasticsearch
path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch
${...} notation within the configuration file will be replaced with the value of the environment variable, for instance:
node.name: ${HOSTNAME} network.host: ${ES_NETWORK_HOST}
${prompt.text} or
${prompt.secret} and start Elasticsearch in the foreground.
${prompt.secret}has echoing disabled so that the value entered will not be shown in your terminal;
${prompt.text}will allow you to see the value as you type it in. For example:
node: name: ${prompt.text}
Enter value for[node.name]:
${prompt.text} or
${prompt.secret} is used in the settings and the process is run as a service or in the background.
default. prefix. This will specify a value that will be used by default unless another value is specified in the config file.
For instance, if Elasticsearch is started as follows:
./bin/elasticsearch -Edefault.node.name=My_Node
node.name will be
My_Node, unless it is overwritten on the command line with
es.node.name or in the config file with
node.name.
${sys:es.logs} that can be referenced in the configuration file to determine the location of the log files; this will resolve to a prefix for the Elasticsearch log file at runtime.
path.logs) is
/var/log/elasticsearch and your cluster is named
production then
${sys:es.logs} will resolve to
/var/log/elasticsearch/production.
appender.rolling.type =RollingFileappender.rolling.name = rolling appender.rolling.fileName = ${sys:es.logs}.log
appender.rolling.layout.type =PatternLayout appender.rolling.layout.pattern =[%d{ISO8601}][%-5p][%-25c]%.10000m%n appender.rolling.filePattern = ${sys:es.logs}-%d{yyyy-MM-dd}.log
appender.rolling.policies.type =Policies appender.rolling.policies.time.type =TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval =1
appender.rolling.policies.time.modulate =true
![]()
| |
Configure the |
| |
Log to |
| |
Roll logs to |
| |
Using a time-based roll policy |
| |
Roll logs on a daily basis |
| |
Align rolls on the day boundary (as opposed to rolling every twenty-four hours) |
If you append .gz or .zip to appender.rolling.filePattern, then the logs will be compressed as they are rolled.
This will create a daily rolling deprecation log file in your log directory. Check this file regularly, especially when you intend to upgrade to a new major version.
The default logging configuration has set the roll policy for the deprecation logs to roll and compress after 1 GB, and to preserve a maximum of five log files (four rolled logs, and the active log).
config/log4j2.properties file by setting the deprecation log level to
error.
log4j2.properties and have the Elasticsearch config directory as an ancestor; this is useful for plugins that expose additional loggers. The logger section contains the java packages and their corresponding log level. The appender section contains the destinations for the logs. Extensive information on how to customize logging and all the supported appenders can be found on the Log4j documentation.
path.dataandpath.logscluster.namenode.namebootstrap.memory_locknetwork.hostdiscovery.zen.ping.unicast.hostsdiscovery.zen.minimum_master_nodes
path.data and
path.logs edit
.zip or
.tar.gz archives, the
data and
logs directories are sub-folders of
$ES_HOME. If these important folders are left in their default locations, there is a high risk of them being deleted while upgrading Elasticsearch to a new version.
path: logs: /var/log/elasticsearch data: /var/data/elasticsearch
data and
logs.
path.data settings can be set to multiple paths, in which case all paths will be used to store data (although the files belonging to a single shard will all be stored on the same data path):
path:
data:
- /mnt/elasticsearch_1
- /mnt/elasticsearch_2
- /mnt/elasticsearch_3
cluster.name edit
cluster.name with all the other nodes in the cluster. The default name is
elasticsearch, but you should change it to an appropriate name which describes the purpose of the cluster.
cluster.name: logging-prod
node.name edit
node.name: prod-data-2
node.name can also be set to the server’s HOSTNAME as follows:
node.name: ${HOSTNAME}
bootstrap.memory_lock edit
bootstrap.memory_lock setting to
true.
bootstrap.memory_lock for more details about how to set up memory locking correctly.
network.hostedit
127.0.0.1 and
[::1]. This is sufficient to run a single development node on a server.
$ES_HOME location on a single node. This can be useful for testing Elasticsearch’s ability to form clusters, but it is not a configuration recommended for production.
network.host:
network.host: 192.168.1.10
network.host setting also understands some special values such as
_local_,
_site_,
_global_ and modifiers like
:ip4 and
:ip6, details of which can be found in the section called “Special values for
network.host edit”.
network.host, Elasticsearch assumes that you are moving from development mode to production mode, and upgrades a number of system startup checks from warnings to exceptions. See the section called “Development mode vs production mode edit” for more information.
discovery.zen.ping.unicast.hostsedit
discovery.zen.ping.unicast.hosts: - 192.168.1.10:9300 - 192.168.1.11- seeds.mydomain.com
![]()
|
|
The port will default to |
| |
A hostname that resolves to multiple IP addresses will try all resolved addresses.
|
discovery.zen.minimum_master_nodes edit
discovery.zen.minimum_master_nodes setting so that each master-eligible node knows the
minimum number of master-eligible nodes that must be visible in order to form a cluster.
minimum_master_nodes edit”.
(master_eligible_nodes / 2) + 1
(3 / 2) + 1 or
2:
Bootstrap Checksedit
localhost for HTTP and transport (internal) communication. This is fine for downloading and playing with Elasticsearch, and everyday development but it’s useless for production systems. To form a cluster, Elasticsearch instances must be reachable via transport communication so they must bind transport to an external interface. Thus, we consider an Elasticsearch instance to be in development mode if it does not bind transport to an external interface (the default), and is otherwise in production mode if it does bind transport to an external interface. Note that HTTP can be configured independently of transport via
http.host and
transport.host; this can be useful for configuring a single instance to be reachable via HTTP for testing purposes without triggering production mode.
bootstrap.memory_lock is enabled, the JVM will lock the initial size of the heap on startup. If the initial heap size is not equal to the maximum heap size, after a resize it will not be the case that all of the JVM heap is locked in memory. To pass the heap size check, you must configure the heap size.
/proc/loadavg), or network sockets. Elasticsearch requires lots of file descriptors (e.g., every shard is composed of multiple segments and other files, plus connections to other nodes, etc.). This bootstrap check is enforced on OS X and Linux. To pass the file descriptor check, you might have to configure file descriptors.
mlockall (Unix) or virtual lock (Windows). This is done via the Elasticsearch setting
bootstrap.memory_lock. However, there are cases where this setting can be passed to Elasticsearch but Elasticsearch is not able to lock the heap (e.g., if the
elasticsearch user does not have
memlock unlimited). The memory lock check verifies that
if the
bootstrap.memory_lock setting is enabled, that the JVM was successfully able to lock the heap. To pass the memory lock check, you might have to configure
mlockall.
/etc/security/limits.conf using the
nprocsetting (note that you might have to increase the limits for the
root user too).
mmap to great effect to map portions of an index into the Elasticsearch address space. This keeps certain index data off the JVM heap but in memory for blazing fast access. For this to be effective, the Elasticsearch should have unlimited address space. The maximum size virtual memory check enforces that the Elasticsearch process has unlimited address space and is enforced only on Linux. To pass the maximum size virtual memory check, you must configure your system to allow the Elasticsearch process the ability to have unlimited address space. This can be done via
/etc/security/limits.conf using the
as setting to
unlimited (note that you might have to increase the limits for the
root user too).
mmap effectively, Elasticsearch also requires the ability to create many memory-mapped areas. The maximum map count check checks that the kernel allows a process to have at least 262,144 memory-mapped areas and is enforced on Linux only. To pass the maximum map count check, you must configure
vm.max_map_count via
sysctl to be at least
262144.
-XX:+UseSerialGC). Note that the default JVM configuration that ship with Elasticsearch configures Elasticsearch to use the CMS collector.
bootstrap.system_call_filter to
false.
OnError and
OnOutOfMemoryError enable executing arbitrary commands if the JVM encounters a fatal error (
OnError) or an
OutOfMemoryError(
OnOutOfMemoryError). However, by default, Elasticsearch system call filters (seccomp) are enabled and these filters prevent forking. Thus, using
OnError or
OnOutOfMemoryError and system call filters are incompatible. The
OnError and
OnOutOfMemoryError checks prevent Elasticsearch from starting if either of these JVM options are used and system call filters are enabled. This check is always enforced. To pass this check do not enable
OnError nor
OnOutOfMemoryError; instead, upgrade to Java 8u92 and use the JVM flag
ExitOnOutOfMemoryError. While this does not have the full capabilities of
OnError nor
OnOutOfMemoryError, arbitrary forking will not be supported with seccomp enabled.
G1GC checkedit
The following settings must be addressed before going to production:
- Set JVM heap size
- Disable swapping
- Increase file descriptors
- Ensure sufficient virtual memory
- Ensure sufficient threads
network.host, Elasticsearch assumes that you are moving to production and will upgrade the above warnings to exceptions. These exceptions will prevent your Elasticsearch node from starting. This is an important safety measure to ensure that you will not lose data because of a malconfigured server.
.zip or
.tar.gz packages, system settings can be configured:
- temporarily with
ulimit, or - permanently in
/etc/security/limits.conf.
ulimitedit
ulimit can be used to change resource limits on a temporary basis. Limits usually need to be set as
root before switching to the user that will run Elasticsearch. For example, to set the number of open file handles (
ulimit -n) to 65,536, you can do the following:
sudo suulimit -n 65536
su elasticsearch
![]()
| |
Become |
| |
Change the max number of open files. |
| |
Become the |
ulimit -a.
/etc/security/limits.confedit
/etc/security/limits.conf file. To set the maximum number of open files for the
elasticsearch user to 65,536, add the following line to the
limits.conf file:
elasticsearch - nofile 65536
elasticsearch user opens a new session.
Ubuntu and limits.conf
limits.conf file for processes started by
init.d. To enable the
limits.conf file, edit
/etc/pam.d/su and uncomment the following line:
# session required pam_limits.so
Sysconfig fileedit
| RPM |
|
| Debian |
|
systemd, system limits need to be specified via systemd.
Systemd configurationedit
When using the RPM or Debian packages on systems that use systemd, system limits must be specified via systemd.
The systemd service file (/usr/lib/systemd/system/elasticsearch.service) contains the limits that are applied by default.
To override these, add a file called/etc/systemd/system/elasticsearch.service.d/elasticsearch.conf and specify any changes in that file, such as:
[Service]LimitMEMLOCK=infinity
Setting JVM optionsedit
jvm.options configuration file. The default location of this file is
config/jvm.options (when installing from the tar or zip distributions) and
/etc/elasticsearch/jvm.options (when installing from the Debian or RPM packages). This file contains a line-delimited list of JVM arguments, which must begin with
-. You can add custom JVM flags to this file and check this configuration into your version control system.
ES_JAVA_OPTS environment variable. For instance:
export ES_JAVA_OPTS="$ES_JAVA_OPTS -Djava.io.tmpdir=/path/to/temp/dir"./bin/elasticsearch
ES_JAVA_OPTS can be specified in the system configuration file.
By default, Elasticsearch tells the JVM to use a heap with a minimum and maximum size of 2 GB. When moving to production, it is important to configure heap size to ensure that Elasticsearch has enough heap available.
- Set the minimum heap size (Xms) and maximum heap size (Xmx) to be equal to each other.通过Xms设置最小堆尺寸，通过Xmx设置最大堆尺寸，并设置这两个相等。
- The more heap available to Elasticsearch, the more memory it can use for caching. But note that too much heap can subject you to long garbage collection pauses.Elasticsearch使用的堆越多，可以用于缓存的内存越多。但是注意，太多的堆可能引起较长时间的垃圾收集等待。
- Set Xmx to no more than 50% of your physical RAM, to ensure that there is enough physical RAM left for kernel file system caches.设置Xmx不要超过物理RAM的50%，以保证有足够的物理RAM空间用于内核文件系统缓存。
-
Don’t set Xmx to above the cutoff that the JVM uses for compressed object pointers (compressed oops); the exact cutoff varies but is near 32 GB. You can verify that you are under the limit by looking for a line in the logs like the following: 不要设置Xmx高于JVM用于压缩对象指针（压缩的oops）的截断值。准确的阶段值接近32gb。你可以确认一下，日志中这一行是否在限制之下。
heap size [1.9gb], compressed ordinary object pointers [true]
-
Even better, try to stay below the threshold for zero-based compressed oops; the exact cutoff varies but 26 GB is safe on most systems, but can be as large as 30 GB on some systems. You can verify that you are under the limit by starting Elasticsearch with the JVM options
-XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsModeand looking for a line like the following: 更好的是，尝试保持低于zero－based压缩oops的限制值，准确的截断值比较大，但是26GB在大多数系统上比较安全，但是在某些系统上可以高达30GB。你可以确认一下，启动Elasticsearch时，通过JVM选项： -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode
heap address: 0x000000011be00000, size: 27648 MB, zero based Compressed Oops
showing that zero-based compressed oops are enabled instead of
heap address: 0x0000000118400000, size: 28672 MB, Compressed Oops with base: 0x00000001183ff000
-Xms2g-Xmx2g
![]()
| |
Set the minimum heap size to 2g. |
| |
Set the maximum heap size to 2g. |
Xms and
Xmx settings in the jvm.options file and setting these values via
ES_JAVA_OPTS:
ES_JAVA_OPTS="-Xms2g -Xmx2g"./bin/elasticsearchES_JAVA_OPTS="-Xms4000m -Xmx4000m"./bin/elasticsearch
![]()
| |
Set the minimum and maximum heap size to 2 GB. |
| |
Set the minimum and maximum heap size to 4000 MB. |
bootstrap.memory_lock edit
config/elasticsearch.yml file:
bootstrap.memory_lock: true
mlockall might cause the JVM or shell session to exit if it tries to allocate more memory than is available!
mlockall in the output from this request:
GET _nodes?filter_path=**.mlockall
mlockall is
false, then it means that the
mlockall request has failed. You will also see a line with more information in the logs with the words
Unable to lock JVM Memory.
.zip and .tar.gz
ulimit -l unlimited as root before starting Elasticsearch, or set
memlock to
unlimited in
/etc/security/limits.conf. 设置ulimit －l unlimited，使用root权限执行。在启动Elasticsearch之前执行。
MAX_LOCKED_MEMORY to
unlimited in the system configuration file (or see below for systems using
systemd).
systemd
LimitMEMLOCK to
infinity in the systemd configuration.
mlockall can fail is that the temporary directory (usually
/tmp) is mounted with the
noexec option. This can be solved by specifying a new temp directory using the
ES_JAVA_OPTS environment variable:
export ES_JAVA_OPTS="$ES_JAVA_OPTS -Djava.io.tmpdir=/path/to/temp/dir"./bin/elasticsearch
or setting this JVM flag in the jvm.options configuration file.
sudo swapoff -a. To disable it permanently, you will need to edit the
/etc/fstab file and comment out any lines that contain the word
swap.
On Windows, the equivalent can be achieved by disabling the paging file entirely via System Properties → Advanced → Performance → Advanced → Virtual memory.
swappiness edit
vm.swappiness is set to
1. This reduces the kernel’s tendency to swap and should not lead to swapping under normal circumstances, while still allowing the whole system to swap in emergency conditions.
.zip and
.tar.gz packages, set
ulimit -n 65536 as root before starting Elasticsearch, or set
nofile to
65536 in
/etc/security/limits.conf.
max_file_descriptors configured for each node using the
Nodes Stats API, with:
GET _nodes/stats/process?filter_path=**.max_file_descriptors
hybrid mmapfs / niofs directory by default to store its indices. The default operating system limits on mmap counts is likely to be too low, which may result in out of memory exceptions.
root:
sysctl -w vm.max_map_count=262144
vm.max_map_count setting in
/etc/sysctl.conf. To verify after rebooting, run
sysctl vm.max_map_count.
ulimit -u 2048 as root before starting Elasticsearch, or by setting
nproc to
2048 in
/etc/security/limits.conf.
Upgrading Elasticsearchedit
Before upgrading Elasticsearch:
- Consult the breaking changes docs.
- Use the Elasticsearch Migration Plugin to detect potential issues before upgrading.
- Test upgrades in a dev environment before upgrading your production cluster.
- Always back up your data before upgrading. You cannot roll back to an earlier version unless you have a backup of your data.
- If you are using custom plugins, check that a compatible version is available.
- 查看breaking changes文档
- 在升级前使用Elasticsearch迁移插件来检查潜在的条目
- 在开发环境测试升级
- 在升级前备份数据。你不可以回滚到之前的版本，除非你有数据备份
- 如果你使用自定义的插件，检查是否兼容升级版本
To determine whether a rolling upgrade is supported for your release, please consult this table:
| Upgrade From | Upgrade To | Supported Upgrade Type |
|---|---|---|
|
|
|
Reindex to upgrade |
|
|
|
Rolling upgrade (where |
|
|
|
Full cluster restart |
|
|
|
Full cluster restart |
|
|
|
Rolling upgrade (where |
Indices created in Elasticsearch 1.x or before
Elasticsearch is able to read indices created in the previous major version only. For instance, Elasticsearch 5.x can use indices created in Elasticsearch 2.x, but not those created in Elasticsearch 1.x or before.
This condition also applies to indices backed up with snapshot and restore. If an index was originally created in 1.x, it cannot be restored into a 5.x cluster even if the snapshot was made by a 2.x cluster.
Elasticsearch 5.x nodes will fail to start in the presence of too old indices.
-
Disable shard allocation 禁用shard分配
When you shut down a node, the allocation process will wait for one minute before starting to replicate the shards that were on that node to other nodes in the cluster, causing a lot of wasted I/O. This can be avoided by disabling allocation before shutting down a node: 当你停掉一个节点时，应用程序在开始备份其它节点上shards前会等待一分钟，会引起大量浪费的I／O。这个可以通过在停掉一个节点之前禁用分配来避免：
PUT _cluster/settings {"transient":{"cluster.routing.allocation.enable":"none"}}
COPY AS CURL VIEW IN CONSOLE -
Stop non-essential indexing and perform a synced flush (Optional) 禁用非必需索引化以及执行同步刷新（可选）
You may happily continue indexing during the upgrade. However, shard recovery will be much faster if you temporarily stop non-essential indexing and issue a synced-flush request: 你可以快乐的继续在升级期间进行索引。然而，shard恢复将会更快，如果临时停掉非必需的索引化以及执行同步刷新请求：
POST _flush/synced
COPY AS CURL VIEW IN CONSOLEA synced flush request is a “best effort” operation. It will fail if there are any pending indexing operations, but it is safe to reissue the request multiple times if necessary. 同步刷新请求是一个最有效的操作，如果有任何等待的索引化操作，它将会失败，但是如果必须的话，多次请求是安全的。
-
Stop and upgrade a single node 停止并升级单个节点
Shut down one of the nodes in the cluster before starting the upgrade. 在升级之前，停掉单个节点
When using the zip or tarball packages, theconfig,data,logsandpluginsdirectories are placed within the Elasticsearch home directory by default. 当使用zip或者 tarball格式的包时，config、data、logs、plugins目录都在Elasticsearch主目录中
It is a good idea to place these directories in a different location so that there is no chance of deleting them when upgrading Elasticsearch. These custom paths can be configured with thepath.conf,path.logs, andpath.datasettings, and usingES_JVM_OPTIONSto specify the location of thejvm.optionsfile.将这些目录迁移到不同目录下是个好主意，这样任何改变都不会删除它们。这些自定义的路径可以通过path.conf，path.logs还有path.data来设置，同时使用ES_JVM_OPTIONS指定jvm.options文件路径。
The Debian and RPM packages place these directories in the appropriate place for each operating
To upgrade using a Debian or RPM package:
- Use
rpmordpkgto install the new package. All files should be placed in their proper locations, and config files should not be overwritten.
To upgrade using a zip or compressed tarball:升级zip或者tar压缩包的格式：
- Extract the zip or tarball to a new directory, to be sure that you don’t overwrite the
configordatadirectories.解压zip或者tar包到新目录，确定你没有覆盖configs或者data目录 - Either copy the files in the
configdirectory from your old installation to your new installation, or set the environment variableES_JVM_OPTIONSto the location of thejvm.optionsfile and use the-E path.conf=option on the command line to point to an external config directory.不要拷贝旧版本的config目录的文件到新版本中，或者设置环境变量ES_JVM_OPTIONS为jvm.options文件，也不要使用－E path.conf=选项 指定一个外部配置目录。 - Either copy the files in the
datadirectory from your old installation to your new installation, or configure the location of the data directory in theconfig/elasticsearch.ymlfile, with thepath.datasetting. 既不要拷贝老版本数据目录到新版本数据目录，或者配置config/elasticesearch.yml的数据目录或者使用path.data配置。
- Use
-
Upgrade any plugins 升级任何插件
Elasticsearch plugins must be upgraded when upgrading a node. Use theelasticsearch-pluginscript to install the correct version of any plugins that you need.必须在升级节点之前升级Elasticsearch插件。使用elasticsearch-plugin脚本安装正确的版本。
-
Start the upgraded node 启动升级节点
Start the now upgraded node and confirm that it joins the cluster by checking the log file or by checking the output of this request: 启动升级节点并通过检查日志文件或者请求输出确认已经加入了集群：
GET _cat/nodes
COPY AS CURL VIEW IN CONSOLE -
Reenable shard allocation 重新开启shard分配
Once the node has joined the cluster, reenable shard allocation to start using the node:一旦节点加入集群，重新启动shard分配：
PUT _cluster/settings {"transient":{"cluster.routing.allocation.enable":"all"}}
COPY AS CURL VIEW IN CONSOLE -
Wait for the node to recover 等待节点数据恢复
You should wait for the cluster to finish shard allocation before upgrading the next node. You can check on progress with the_cat/healthrequest: 你可以等待集群完成数据shard分配之后再进行下一个节点升级。你可以通过_cat/health检查这个过程。
GET _cat/health
COPY AS CURL VIEW IN CONSOLEWait for the
statuscolumn to move fromyellowtogreen. Statusgreenmeans that all primary and replica shards have been allocated.等待status列从yellow变为green。status green意味着所有主要primary和备份shards意境分配了。During a rolling upgrade, primary shards assigned to a node with the higher version will never have their replicas assigned to a node with the lower version, because the newer version may have a different data format which is not understood by the older version.在滚动升级中，主shards永远不会将它们的备份分配到一个较低版本的节点上，因为较新版本又可能数据格式和旧版本不同。
If it is not possible to assign the replica shards to another node with the higher version — e.g. if there is only one node with the higher version in the cluster — then the replica shards will remain unassigned and the cluster health will remain statusyellow.如果不能将备份shards分配到另一个带有更高版本的节点上－例如，如果集群里只有一个更高版本的节点，那么备份shards将保持未分配状态，那么集群会保持yellow状态。
In this case, check that there are no initializing or relocating shards (the
initandrelocolumns) before proceding.这种情况下，在继续下面之前，需要检查是否有未初始化或者重定向的shards（init或者relo列）As soon as another node is upgraded, the replicas should be assigned and the cluster health will reach statusgreen.一旦其它节点升级了，备份应当分配，集群健康将达到green状态。
Shards that have not been sync-flushed may take some time to recover. The recovery status of individual shards can be monitored with the_cat/recoveryrequest: 没有 sync-flushed的shards需要一段时间恢复数据。个体数据恢复状态会在 _cat/recovery请求中提到。
GET _cat/recovery
COPY AS CURL VIEW IN CONSOLEIf you stopped indexing, then it is safe to resume indexing as soon as recovery has completed. 如果停止索引化，那么制药恢复完成那么重新进行索引化就是安全的。
-
Repeat
When the cluster is stable and the node has recovered, repeat the above steps for all remaining nodes. 当集群稳定了并且节点已经恢复了，重复上面的步骤升级其它节点。
Full cluster restart upgradeedit
Elasticsearch requires a full cluster restart when upgrading across major versions. Rolling upgrades are not supported across major versions. Consult this table to verify that a full cluster restart is required.
The process to perform an upgrade with a full cluster restart is as follows:
-
Disable shard allocation
When you shut down a node, the allocation process will immediately try to replicate the shards that were on that node to other nodes in the cluster, causing a lot of wasted I/O. This can be avoided by disabling allocation before shutting down a node:
PUT _cluster/settings {"persistent":{"cluster.routing.allocation.enable":"none"}}
COPY AS CURL VIEW IN CONSOLE -
Perform a synced flush
Shard recovery will be much faster if you stop indexing and issue a synced-flush request:
POST _flush/synced
COPY AS CURL VIEW IN CONSOLEA synced flush request is a “best effort” operation. It will fail if there are any pending indexing operations, but it is safe to reissue the request multiple times if necessary.
-
Shutdown and upgrade all nodes
Stop all Elasticsearch services on all nodes in the cluster. Each node can be upgraded following the same procedure described in [upgrade-node].
-
Upgrade any plugins
Elasticsearch plugins must be upgraded when upgrading a node. Use the
elasticsearch-pluginscript to install the correct version of any plugins that you need. -
Start the cluster
If you have dedicated master nodes — nodes with
node.masterset totrue(the default) andnode.dataset tofalse— then it is a good idea to start them first. Wait for them to form a cluster and to elect a master before proceeding with the data nodes. You can check progress by looking at the logs.As soon as the minimum number of master-eligible nodes have discovered each other, they will form a cluster and elect a master. From that point on, the
_cat/healthand_cat/nodesAPIs can be used to monitor nodes joining the cluster:GET _cat/health GET _cat/nodes
COPY AS CURL VIEW IN CONSOLEUse these APIs to check that all nodes have successfully joined the cluster.
-
Wait for yellow
As soon as each node has joined the cluster, it will start to recover any primary shards that are stored locally. Initially, the
_cat/healthrequest will report astatusofred, meaning that not all primary shards have been allocated.Once each node has recovered its local shards, the
statuswill becomeyellow, meaning all primary shards have been recovered, but not all replica shards are allocated. This is to be expected because allocation is still disabled. -
Reenable allocation
Delaying the allocation of replicas until all nodes have joined the cluster allows the master to allocate replicas to nodes which already have local shard copies. At this point, with all the nodes in the cluster, it is safe to reenable shard allocation:
PUT _cluster/settings {"persistent":{"cluster.routing.allocation.enable":"all"}}
COPY AS CURL VIEW IN CONSOLEThe cluster will now start allocating replica shards to all data nodes. At this point it is safe to resume indexing and searching, but your cluster will recover more quickly if you can delay indexing and searching until all shards have recovered.
You can monitor progress with the
_cat/healthand_cat/recoveryAPIs:GET _cat/health GET _cat/recovery
COPY AS CURL VIEW IN CONSOLEOnce thestatuscolumn in the_cat/healthoutput has reachedgreen, all primary and replica shards have been successfully allocated.
Stopping Elasticsearchedit
An orderly shutdown of Elasticsearch ensures that Elasticsearch has a chance to cleanup and close outstanding resources. For example, a node that is shutdown in an orderly fashion will remove itself from the cluster, sync translogs to disk, and perform other related cleanup activities. You can help ensure an orderly shutdown by properly stopping Elasticsearch.
If you’re running Elasticsearch as a service, you can stop Elasticsearch via the service management functionality provided by your installation.
If you’re running Elasticsearch directly, you can stop Elasticsearch by sending control-C if you’re running Elasticsearch in the console, or by sending SIGTERM to the Elasticsearch process on a POSIX system. You can obtain the PID to send the signal to via various tools (e.g., ps or jps):
$ jps | grep Elasticsearch14542Elasticsearch
From the Elasticsearch startup logs:
[2016-07-0712:26:18,908][INFO ][node ][I8hydUG] version[5.0.0-alpha4], pid[15399], build[3f5b994/2016-06-27T16:23:46.861Z], OS[Mac OS X/10.11.5/x86_64], JVM[OracleCorporation/JavaHotSpot(TM)64-BitServer VM/1.8.0_92/25.92-b14]
Or by specifying a location to write a PID file to on startup (-p <path>):
$ ./bin/elasticsearch -p /tmp/elasticsearch-pid -d $ cat /tmp/elasticsearch-pid && echo 15516 $ kill -SIGTERM 15516
Stopping on Fatal Errorsedit
During the life of the Elasticsearch virtual machine, certain fatal errors could arise that put the virtual machine in a questionable state. Such fatal errors include out of memory errors, internal errors in virtual machine, and serious I/O errors.
When Elasticsearch detects that the virtual machine has encountered such a fatal error Elasticsearch will attempt to log the error and then will halt the virtual machine. When Elasticsearch initiates such a shutdown, it does not go through an orderly shutdown as described above. The Elasticsearch process will also return with a special status code indicating the nature of the error.
| JVM internal error |
128 |
| Out of memory error |
127 |
| Stack overflow error |
126 |
| Unknown virtual machine error |
125 |
| Serious I/O error |
124 |
| Unknown fatal error |
1 |