新一代云原生监控系统 Prometheus实践
李远君，Web和云计算开发人员。Java，Python，Golang爱好者。
个人网站：https://tomoncle.com
本文作者已加入Python中文社区专栏作者计划。
Prometheus 是源于 Google Borgmon 的一个开源监控系统，用 Golang 开发。被很多人称为下一代监控系统。 Prometheus 基本原理是通过 HTTP 协议周期性抓取被监控组件的状态，这样做的好处是任意组件只要提供 HTTP 接口就可以接入监控系统，不需要任何 SDK.(注意：prometheus使用UTC世界标准时间进行数据记录)
FAQ
为什么写这篇文章？
这几年一直在做云相关的工作，所以一直会有监控的业务场景，而我们常用的监控工具主要就是rometheus这款开源工具，当然Prometheus这个生态圈已经非常成熟，但是对于我们某些特定的场景
比如就获取虚拟机或容器上下线，这些我们一般是在PaaS 容器列表或IaaS 虚拟机列表中查看，他原生的UI还满足不了我们的需求，所以需要我们自己去通过Http接口查询，再举例，假如我们不用grafana这种组件显示,而是自定制自己UI监控页面，http接口查询更是必须实现的，基于此，我才有想写这篇文章。
关于prometheus?
关于prometheus， 这里我们不做太多的介绍，因为官方文档（https://prometheus.io/）是一个更好的选择，当然在下面我也会分享一些简单的配置。
prometheus-http-client
之前做云监控的时候，因为时间紧任务重，而且当时我们公司的监控引擎已经使用Java书写了一部分，所以我也就是直接在原来的基础上来开发了，但是越到最后，需求越多，越觉得麻烦，因为Java里面实现一个简单的http查询，还要转json（相比于Python实现而言复杂），还要实现对应的http查询接口等等，所以痛定思痛，我就自己闲余时间实现了一款基于Python的Prometheus Http API. 非常简单易用，便于拓展，目前支持 prometheus, node_exporter, mysqld_exporter, consul_exporter, memcached_exporter等等客户端查询，通过Python装饰器实现，如果你对监控没有兴趣，你也可以浏览该项目中装饰器的使用。 官方文档：https://tomoncle.github.io/prometheus-http-client/
环境
prometheus安装：
$ wget https://github.com/prometheus/prometheus/releases/download/v2.8.0-rc.0/prometheus-2.8.0-rc.0.linux-amd64.tar.gz
$ tar zxvf prometheus-2.8.0-rc.0.linux-amd64.tar.gz
默认配置(其他组件配置请看官方文档，安装了其他组件，即可查询对应的客户端):
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager).
external_labels:
monitor: 'codelab-monitor'
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
# - '127.0.0.1:9093'
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first.rules"
# - "second.rules"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=` to any timeseries scraped from this config.
- job_name: 'prometheus'
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ['localhost:9090']
更简单的扩展：使用包装器自动选择查询模式，不需要任何实现。(原理及实现见下方"核心实现")
@prom
def go_gc_duration_seconds(self, **kwargs):
pass
@prom
def go_gc_duration_seconds_count(self, **kwargs):
pass
自定义查询表达式：不需要任何实现。(原理及实现见下方"核心实现")
@relabel('100 - (avg by (instance, job) (irate(node_cpu{mode="idle"}[5m])) * 100)')
def node_cpu_rate(self, **kwargs):
pass
@relabel(
'(node_filesystem_size{} - node_filesystem_free{})'
' / (node_filesystem_size{} - node_filesystem_free{} + node_filesystem_avail{})'
' * 100')
def node_disk_rate(self, **kwargs):
pass
丰富的查询选择器：支持多标签查询。
参数:
graph: True/False , 默认 Falsestart: timestamp , 假如graph 为 True, 必须提供end: timestamp , 假如graph 为 True, 必须提供step: int (步长间距) , 默认 Nonetime: timestamp , 默认 time.time()filter: {'instance':'127.0.0.1', 'label':'go lang', …} , 默认 None
返回json数据:
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from prometheus_http_client import NodeExporter
>>> node_exporter = NodeExporter()
>>> node_exporter.node_cpu_rate(filter={'instance': '127.0.0.1:9100','mode':'user','job':'static_nodes'})
u'{"status":"success","data":{"resultType":"vector","result":[{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes"},"value":[1533779495.799,"70.7333333333338"]}]}}'
>>>
>>> node_exporter.node_cpu_rate(filter={'instance': '127.0.0.1:9100','job':'static_nodes'})
u'{"status":"success","data":{"resultType":"vector","result":[{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes"},"value":[1533779522.109,"2.2500000000051443"]}]}}'
>>>
安装
从源码安装：
$ git clone https://github.com/tomoncle/prometheus-http-client.git
$ cd prometheus-http-client && sudo python setup.py install
pip安装：
$ pip install prometheus-http-client
配置
默认 : http://localhost:9090
更新 : $ export PROMETHEUS_URL='http://192.168.1.2:9090'
认证 : $ export PROMETHEUS_HEAD='{"Cookie": "123456"}'
拓展
拓展
参考该目录的其他exporter, 继承 "Exporter" 类即可，不需要任何实现
示例：
核心实现(Python 装饰器)
prom : 使用该装饰器，自动解析函数名，并根据函数名称自动构建查询表达式，发起HTTP请求，返回json数据
def prom(func):
"""
execute metric request
:param func: exporter method
:return: metric monitor data, json
"""
def wrapper(*args, **kwargs):
graph = kwargs.pop('graph', False)
filters = kwargs.pop('filter', {})
prometheus = Prometheus()
kwargs['metric'] = func.__name__ + _build_params(filters)
function = prometheus.query_rang if graph else prometheus.query
try:
_ignore(args)
return function(**kwargs)
except AssertionError:
raise
except Exception as e:
raise PrometheusError(e)
return wrapper
relabel：该装饰器将显示的使用relabel参数值指定的表达式，这个是基于prom装饰器的灵活实现，供用户来自定义复杂表达式
def relabel(name):
"""
rename metric name
:param name:
:return: metric monitor data, json
"""
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
func.__name__ = name
filters = kwargs.pop('filter', {})
if '{mode="idle"}' in name and 'mode' not in filters.keys():
filters['mode'] = 'idle'
query = _build_params(filters)
if query:
func.__name__ = name.replace('{mode="idle"}', query).replace('{}', query)
return prom(func)(*args, **kwargs)
return wrapper
return decorator
check_params: 检测函数参数值
def check_params(params):
def decorator(func):
def wrapper(*args, **kwargs):
if isinstance(params, str):
if not kwargs.get(params, None):
raise AssertionError('parameter "{}" is necessary.'.format(params))
elif isinstance(params, list):
for param in params:
if not kwargs.get(param, None):
raise AssertionError('parameter "{}" is necessary.'.format(params))
else:
raise AssertionError('parameter "{}" is necessary.'.format(params))
return func(*args, **kwargs)
return wrapper
return decorator
查询使用示例
Prometheus 客户端
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from prometheus_http_client import Prometheus
>>> prometheus = Prometheus()
>>> prometheus.query(metric='irate(node_cpu{job="static_nodes"}[5m])')
u'{"status":"success","data":{"resultType":"vector","result":[{"metric":{"cpu":"cpu0","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"idle"},"value":[1533779660.16,"0.9340000000001358"]},{"metric":{"cpu":"cpu0","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"iowait"},"value":[1533779660.16,"0.003333333333334091"]},{"metric":{"cpu":"cpu0","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"irq"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu0","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"nice"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu0","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"softirq"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu0","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"steal"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu0","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"system"},"value":[1533779660.16,"0.016666666666666666"]},{"metric":{"cpu":"cpu0","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"user"},"value":[1533779660.16,"0.025333333333340608"]},{"metric":{"cpu":"cpu1","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"idle"},"value":[1533779660.16,"0.9373333333333297"]},{"metric":{"cpu":"cpu1","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"iowait"},"value":[1533779660.16,"0.025333333333333503"]},{"metric":{"cpu":"cpu1","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"irq"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu1","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"nice"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu1","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"softirq"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu1","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"steal"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu1","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"system"},"value":[1533779660.16,"0.0073333333333304536"]},{"metric":{"cpu":"cpu1","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"user"},"value":[1533779660.16,"0.02333333333332727"]},{"metric":{"cpu":"cpu2","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"idle"},"value":[1533779660.16,"0.9486666666666679"]},{"metric":{"cpu":"cpu2","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"iowait"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu2","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"irq"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu2","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"nice"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu2","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"softirq"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu2","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"steal"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu2","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"system"},"value":[1533779660.16,"0.009333333333332423"]},{"metric":{"cpu":"cpu2","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"user"},"value":[1533779660.16,"0.0319999999999709"]},{"metric":{"cpu":"cpu3","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"idle"},"value":[1533779660.16,"0.9540000000000267"]},{"metric":{"cpu":"cpu3","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"iowait"},"value":[1533779660.16,"0.0006666666666670077"]},{"metric":{"cpu":"cpu3","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"irq"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu3","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"nice"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu3","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"softirq"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu3","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"steal"},"value":[1533779660.16,"0"]},{"metric":{"cpu":"cpu3","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"system"},"value":[1533779660.16,"0.008666666666666363"]},{"metric":{"cpu":"cpu3","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","mode":"user"},"value":[1533779660.16,"0.03266666666665211"]}]}}'
>>>
>>> prometheus.query(metric='sum by (mode, instance,job) (irate(node_cpu{job="static_nodes"}[5m]))')
u'{"status":"success","data":{"resultType":"vector","result":[{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes","mode":"system"},"value":[1533779830.339,"0.038000000000010914"]},{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes","mode":"user"},"value":[1533779830.339,"0.09133333333332606"]},{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes","mode":"idle"},"value":[1533779830.339,"3.8"]},{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes","mode":"iowait"},"value":[1533779830.339,"0.035333333333332696"]},{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes","mode":"irq"},"value":[1533779830.339,"0"]},{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes","mode":"nice"},"value":[1533779830.339,"0"]},{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes","mode":"softirq"},"value":[1533779830.339,"0.0006666666666666524"]},{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes","mode":"steal"},"value":[1533779830.339,"0"]}]}}'
>>>公共的Exporter（https://prometheus.io/）标签
code :
from prometheus_http_client import Exporter
print(Exporter().go_gc_duration_seconds())data :
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"go_gc_duration_seconds","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","quantile":"0"},"value":[1533734495.25,"0.00002624"]},{"metric":{"__name__":"go_gc_duration_seconds","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","quantile":"0.25"},"value":[1533734495.25,"0.000048674"]},{"metric":{"__name__":"go_gc_duration_seconds","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","quantile":"0.5"},"value":[1533734495.25,"0.000064599"]},{"metric":{"__name__":"go_gc_duration_seconds","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","quantile":"0.75"},"value":[1533734495.25,"0.000084196"]},{"metric":{"__name__":"go_gc_duration_seconds","device_ID":"static_node","instance":"127.0.0.1:9100","job":"static_nodes","quantile":"1"},"value":[1533734495.25,"0.002391278"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9104","job":"mysql","mysql_version":"5.7","quantile":"0"},"value":[1533734495.25,"0.000028267"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9104","job":"mysql","mysql_version":"5.7","quantile":"0.25"},"value":[1533734495.25,"0.000040906"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9104","job":"mysql","mysql_version":"5.7","quantile":"0.5"},"value":[1533734495.25,"0.00005304"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9104","job":"mysql","mysql_version":"5.7","quantile":"0.75"},"value":[1533734495.25,"0.000093971"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9104","job":"mysql","mysql_version":"5.7","quantile":"1"},"value":[1533734495.25,"0.004254732"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9107","job":"consul","mysql_version":"5.7","quantile":"0"},"value":[1533734495.25,"0.00006202700000000001"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9107","job":"consul","mysql_version":"5.7","quantile":"0.25"},"value":[1533734495.25,"0.000198266"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9107","job":"consul","mysql_version":"5.7","quantile":"0.5"},"value":[1533734495.25,"0.00020824000000000003"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9107","job":"consul","mysql_version":"5.7","quantile":"0.75"},"value":[1533734495.25,"0.000271729"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9107","job":"consul","mysql_version":"5.7","quantile":"1"},"value":[1533734495.25,"0.001790822"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","quantile":"0"},"value":[1533734495.25,"0.000025387"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","quantile":"0.25"},"value":[1533734495.25,"0.000037497"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","quantile":"0.5"},"value":[1533734495.25,"0.000060197"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","quantile":"0.75"},"value":[1533734495.25,"0.000111523"]},{"metric":{"__name__":"go_gc_duration_seconds","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","quantile":"1"},"value":[1533734495.25,"0.000544621"]}]}}
node_exporter
code :
from prometheus_http_client import NodeExporter
node_exporter = NodeExporter()
# print(node_exporter.node_uname_info())
print(node_exporter.node_cpu_rate(filter={'instance': '127.0.0.1:9100'}))data :
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"instance":"127.0.0.1:9100","job":"static_nodes"},"value":[1533734848.099,"11.816666666676014"]}]}}
mysqld_exporter
code :
from prometheus_http_client import MysqlExporter
print(MysqlExporter().mysql_exporter_scrapes_total())data :
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"mysql_exporter_scrapes_total","instance":"127.0.0.1:9104","job":"mysql","mysql_version":"5.7"},"value":[1533735043.46,"345"]}]}}
consul_exporter
code :
from prometheus_http_client import ConsulExporter
print(ConsulExporter().consul_catalog_services())data :
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"consul_catalog_services","instance":"127.0.0.1:9107","job":"consul","mysql_version":"5.7"},"value":[1533735232.039,"1"]}]}}
memcached_exporter
code :
from prometheus_http_client import MemcachedExporter
print(MemcachedExporter().memcached_commands_total())data :
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"memcached_commands_total","command":"cas","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"badval"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"cas","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"hit"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"cas","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"miss"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"decr","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"hit"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"decr","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"miss"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"delete","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"hit"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"delete","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"miss"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"flush","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"hit"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"get","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"hit"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"get","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"miss"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"incr","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"hit"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"incr","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"miss"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"set","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"hit"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"touch","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"hit"},"value":[1533735286.809,"0"]},{"metric":{"__name__":"memcached_commands_total","command":"touch","instance":"127.0.0.1:9150","job":"memcached","mysql_version":"5.7","status":"miss"},"value":[1533735286.809,"0"]}]}}
更多使用见：https://github.com/tomoncle/prometheus-http-client ，当然您如果有时间，希望可以帮助我一起让这个项目变得更好。
zhi
支
chi
持
zuo
作
zhe
者
长按扫码鼓励作者
本文作者已加入Python中文社区专栏作者计划
推荐8个前沿技术领域公众号
用Python开发计时器程序
用Python生成抖音字符视频！
Python技术知识清单（数据科学）
用Pandas库实现MySQL数据库的读写
▼ 点击下方阅读原文
免费成为社区注册会员，会员可以享受更多权益