如何使用Elastic Stack简化Docker容器日志分析
by Ravindu Fernando
通过拉文杜·费尔南多(Ravindu Fernando)
如何使用Elastic Stack简化Docker容器日志分析 (How to simplify Docker container log analysis with Elastic Stack)
Logging is an essential component within any application. Logs enable you to analyze and sneak a peak into what’s happening within your application code like a story. Software developers spend a large part of their day to day lives monitoring, troubleshooting and debugging applications, which can sometimes be a nightmare. Logging allows software developers to make this hectic process much easier and smoother.
日志记录是任何应用程序中必不可少的组件。 日志使您能够分析和潜入峰值(例如故事)的应用程序代码中发生的事情。 软件开发人员每天的大部分时间都用于监视，故障排除和调试应用程序，有时这可能是一场噩梦。 日志记录使软件开发人员可以使此繁忙的过程变得更加轻松和顺畅。
If you have containerized your application with a container platform like Docker, you may be familiar with docker logs which allows you to see the logs created within your application running inside your docker container. Why then think of Elastic Stack to analyze your logs? Well, there are mainly two burning problems here:
如果您已经使用Docker之类的容器平台对应用程序进行了容器化，那么您可能会熟悉Docker 日志 它使您可以查看在Docker容器中运行的应用程序中创建的日志。 为什么然后考虑使用Elastic Stack分析您的日志？ 好吧，这里主要存在两个亟待解决的问题：
- Imagine you have tens, hundreds, or even thousands of containers generating logs — SSH-ing in to all those servers and extracting logs won’t work well. 假设您有数十个，数百个甚至数千个生成日志的容器-SSH插入所有这些服务器，提取日志将无法正常工作。
- Also containers are immutable and ephemeral, which means they have a shorter life span. So once your containers are gone and replaced with new containers, all of your application logs related to old containers are gone. 另外，容器是不可变的且短暂的，这意味着它们的使用寿命较短。 因此，一旦容器消失并替换为新容器，与旧容器相关的所有应用程序日志都将消失。
So the ultimate solution for this is to create a centralized logging component for collecting all of your container logs into a single place. This is where Elastic Stacks comes in.
因此，最终的解决方案是创建一个集中式日志记录组件，以将所有容器日志收集到一个地方。 这是Elastic Stacks进来的地方。
Elastic Stack mainly consists of four major components:
Elastic Stack主要包含四个主要组件：
Beats is the new member which made the ELK Stack known as Elastic Stack. Beats are light weight log data shippers which can push logs to the ELK Stack. For this post I will be using Filebeats, a member of the Beats family, which offers a lightweight way to collect and forward and centralize logs and files.
Beats是使ELK Stack成为弹性堆栈的新成员。 Beats是重量轻的日志数据发送者，可以将日志推送到ELK堆栈。 在这篇文章中，我将使用Beats家族的成员Filebeats，它提供了一种轻便的方法来收集，转发和集中日志和文件。
Logstash is a component which aggregates, modifies, and transfers logs from multiple input locations into Elasticsearch.
Logstash是一个组件，可将日志从多个输入位置聚合，修改并将其传输到Elasticsearch中。
Elasticsearch is a distributed, JSON-based search and analytics engine that stores and indexes data (log entries in this case) in a scalable and manageable way.
Elasticsearch是 基于JSON的分布式搜索和分析引擎，以可扩展和可管理的方式存储和索引数据(在这种情况下为日志条目)。
Kibana is an enriched UI to analyze and easily access data in Elasticsearch.
Kibana是一个丰富的UI，用于分析和轻松访问Elasticsearch中的数据。
In this post, we will look into how to use the above mentioned components and implement a centralized log analyzer to collect and extract logs from Docker containers.
在本文中，我们将研究如何使用上述组件并实现一个集中式日志分析器，以从Docker容器中收集和提取日志。
For the purposes of this article, I have used two t2.small AWS EC2 instances, running Ubuntu 18.04 installed with Docker and Docker compose. Instance 1 is running a tomcat webapp and the instance 2 is running ELK stack (Elasticsearch, Logstash, Kibana).
出于本文的目的，我使用了两个t2.small AWS EC2实例，它们运行与Docker和Docker compose一起安装的Ubuntu 18.04。 实例1正在运行tomcat Webapp，实例2正在运行ELK堆栈(Elasticsearch，Logstash，Kibana)。
In Linux by default docker logs can be found in this location:/var/lib/docker/containers/
在Linux中，默认情况下可以在以下位置找到docker日志： /var/lib/docker/containers/
All docker logs will be collected via Filebeat running inside the host machine as a container. Filebeat will be installed on each docker host machine (we will be using a custom Filebeat docker file and systemd unit for this which will be explained in the Configuring Filebeat section.)
所有docker日志将通过在主机内部作为容器运行的Filebeat进行收集。 Filebeat将安装在每台docker主机上(为此，我们将使用自定义Filebeat docker文件和systemd单元，这将在“配置Filebeat”部分中进行说明。)
Our tomcat webapp will write logs to the above location by using the default docker logging driver. Filebeat will then extract logs from that location and push them towards Logstash.
我们的tomcat Webapp将使用默认的Docker日志记录驱动程序将日志写入上述位置。 然后，Filebeat将从该位置提取日志并将其推向Logstash。
Another important thing to note is that other than application generated logs, we also need metadata associated with the containers, such as container name, image, tags, host etc… This will allow us to specifically identify the exact host and container the logs are generating. These data can also be sent easily by Filebeat along with the application log entries.
另一个需要注意的重要事项是，除了应用程序生成的日志外，我们还需要与容器相关联的元数据，例如容器名称，图像，标签，主机等……这将使我们能够具体标识日志所生成的确切主机和容器。 。 这些数据还可以通过Filebeat与应用程序日志条目一起轻松发送。
By doing this kind of implementation the running containers don’t need to worry about the logging driver, how logs are collected and pushed. Filebeat will take care of those. This is often known as single responsibility principle.
通过执行这种实现，运行中的容器无需担心日志记录驱动程序，日志的收集和发送方式。 Filebeat将解决这些问题。 这通常被称为单一责任原则。
配置文件拍 (Configuring Filebeat)
For this section the filebeat.yml and Dockerfile were obtained from Bruno COSTE’s sample-filebeat-docker-logging github repo. Many thanks to his awesome work.
在本节中，从Bruno COSTE的sample-filebeat-docker-logging github存储库中获取filebeat.yml和Dockerfile。 非常感谢他的出色工作。
But since I have done several changes to filebeat.yml according to requirements of this article, I have hosted those with filebeat.service (systemd file) separately on my own repo. You can access the repo here.
但是，由于我已根据本文的要求对filebeat.yml进行了几处更改，因此我将那些具有filebeat.service(系统文件)的文件分别托管在自己的存储库中。 您可以在此处访问仓库。
As the initial step, you need to update your filebeat.yml file which contains the Filebeat configurations. Given below is a sample filebeat.yml file you can use. Note the line 21, the output.logstash field and the hosts field. I have configured it to the IP address of the server I’m running my ELK stack, but you can modify it if you are running Logstash on a separate server. By default Logstash is listening to Filebeat on port 5044.
作为第一步，您需要更新包含Filebeat配置的filebeat.yml文件。 下面提供的是您可以使用的示例filebeat.yml文件。 注意第21行，output.logstash字段和hosts字段。 我已经将其配置为正在运行ELK堆栈的服务器的IP地址，但是如果在单独的服务器上运行Logstash，则可以对其进行修改。 默认情况下，Logstash在端口5044上监听Filebeat。
To get to know more about Filebeat Docker configuration parameters, look here.
要了解有关Filebeat Docker配置参数的更多信息，请参见此处 。
After that you can create your own Filebeat Docker image by using the following Dockerfile.
之后，您可以使用以下Dockerfile创建自己的Filebeat Docker映像。
Once the image is built, you can push it in to your docker repository. Now since you have the capability to run Filebeat as a docker container, it’s just a matter of running the Filebeat container on your host instances running containers. Here is the docker run command.
构建映像后，您可以将其推入docker存储库。 现在，由于您具有将Filebeat作为docker容器运行的能力，因此只需在运行容器的主机实例上运行Filebeat容器即可。 这是docker run命令。
docker run -v '/var/lib/docker/containers:/usr/share/dockerlogs/data:ro' -v '/var/run/docker.sock:/var/run/docker.sock' --name filebeat ${YOUR_FILEBEAT_DOCKER_IMAGE}:latestIn the above Docker command, note the two bind mount parameters: /var/lib/docker/containers is the path where docker logs exist within the host machine, and it has been bound to /usr/share/dockerlogs/data path within Filebeat container with read only access. In the second bind mount argument, /var/run/docker.sock is bound into the Filebeat container’s Docker daemon. It is the unix socket the Docker daemon listens on by default and it can be used to communicate with the daemon from within a container. This allows our Filebeat container to obtain Docker metadata and enrich the container log entries along with the metadata and push it to ELK stack.
在上面的Docker命令中，请注意两个绑定安装参数：/ var / lib / docker / containers是Docker日志在主机中存在的路径，并且已绑定到Filebeat中的/ usr / share / dockerlogs / data路径具有只读访问权限的容器。 在第二个绑定安装参数中，/var/run/docker.sock被绑定到Filebeat容器的Docker守护程序中。 它是Docker守护程序默认监听的unix套接字，可用于在容器内与守护程序进行通信。 这使我们的Filebeat容器可以获得Docker元数据，并与元数据一起丰富容器日志条目，并将其推入ELK堆栈。
If you want to automate this process, I have written a Systemd Unit file for managing Filebeat as a service.
如果要自动执行此过程，我已经编写了一个Systemd Unit文件来将Filebeat作为服务进行管理。
配置ELK堆栈 (Configuring the ELK Stack)
For this I will be using my second EC2 instance, where I run the ELK stack. You can do this by simply installing Docker compose and checking out this awesome deviantony/docker-elk repo and just running docker-compose up -d
为此，我将使用第二个EC2实例，在该实例中运行ELK堆栈。 您可以通过简单地安装Docker compose并检查这个很棒的deviantony / docker-elk存储库并仅运行docker -compose up -d来完成此操作
Note that all your firewall rules allow inbound traffic into the Logstash, Elasticsearch and Kibana.
请注意，您所有的防火墙规则都允许入站流量进入Logstash，Elasticsearch和Kibana。
Before running the ELK stack you need to make sure your logstash.conf file is properly configured to listen to incoming beats logs on port 5044 and the logs are being properly added onto the elasticsearch host. Also you need to make sure to add an index parameter on to your Elasticsearch to identify the logs generated by Filbeat uniquely.
在运行ELK堆栈之前，您需要确保正确配置了logstash.conf文件以侦听端口5044上的传入节拍日志，并且已将这些日志正确添加到了Elasticsearch主机上。 另外，您还需要确保在Elasticsearch上添加索引参数，以唯一标识Filbeat生成的日志。
In your docker-elk repo you can find your logstash.conf file by following docker-elk/logstash/pipeline pathname. This is the configuration file for setting up Logstash configurations. You need to update it as follows:
在您的docker-elk存储库中，您可以通过遵循docker-elk / logstash / pipeline路径名找到您的logstash.conf文件。 这是用于设置Logstash配置的配置文件。 您需要进行如下更新：
Once you do it, you can access your Kibana dashboard on port 5601 by default as defined on the docker-compose.yml file on deviantony/docker-elk repo.
完成后，您可以默认在deviantony / docker -elk存储库上的docker -compose.yml文件中定义的端口5601上访问Kibana仪表板。
Under the management tab, you can create an index pattern for Filebeat logs. This has to be done before you can view the logs on Kibana dashboard.
在管理选项卡下，您可以为Filebeat日志创建索引模式。 必须先完成此操作，然后才能在Kibana仪表板上查看日志。
If your containers are pushing logs properly into Elasticsearch via Logstash, and you have successfully created the index pattern, you can go to the Discover tab on the Kibana dashboard and view your Docker container application logs along with Docker metadata under the filebeat* index pattern.
如果您的容器通过Logstash将日志正确地推送到Elasticsearch中，并且您已经成功创建了索引模式，则可以转到Kibana仪表板上的发现选项卡，并在filebeat *索引模式下查看Docker容器应用程序日志以及Docker元数据。
References
参考文献
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-getting-started.html
https://www.elastic.co/guide/zh-CN/beats/filebeat/current/filebeat-getting-started.html
https://medium.com/@bcoste/powerful-logging-with-docker-filebeat-and-elasticsearch-8ad021aecd87
https://medium.com/@bcoste/powerful-logging-with-docker-filebeat-and-elasticsearch-8ad021aecd87
https://www.elastic.co/guide/en/logstash/current/configuration.html
https://www.elastic.co/guide/zh-CN/logstash/current/configuration.html
https://medium.com/lucjuggery/about-var-run-docker-sock-3bfd276e12fd
https://medium.com/lucjuggery/about-var-run-docker-sock-3bfd276e12fd
翻译自: https://www.freecodecamp.org/news/docker-container-log-analysis-with-elastic-stack-53d5ec9e5953/