基于salt-stack实现的高可用负载均衡 salt-stack模块介绍认识
Salt-satck一键部署keepalived；haproxy服务：
| 角色 | server-id | 安装 |
|---|---|---|
| MASTER | Server1 | haproxy;keepalived（MASTER） |
| MINION | Server2 | httpd（REAL SERVER） |
| MINION | Server3 | nginx（REAL SERVER） |
| MINION | Server4 | haproxy;keepalived(BAUKUP) |
配置yum源：将server4添加到salt-stack集群中
[rhel-source]
name=Red Hat Enterprise Linux $releasever - $basearch - Source
baseurl=http://172.25.30.250/rhel6.5
enabled=1
gpgcheck=0
[salt]
name=salt-stack
baseurl=http://172.25.30.250/rhel6
enabled=1
gpgcheck=0
[LoadBalancer]
name=LoadBalancer
baseurl=http://172.25.30.250/rhel6.5/LoadBalancer
enabled=1
gpgcheck=01.keepalived源码安装
cd /srv/salt/
mkdir keepalived
mkdir pkgs #安装依赖包
mkdir files
cd keepalived/
vim install.sls
include:
- pkgs.make
kp-installed:
file.managed:
- name: /mnt/keepalived-2.0.6.tar.gz
- source: salt://keepalived/files/keepalived-2.0.6.tar.gz
cmd.run:
- name: cd /mnt && tar zxf keepalived-2.0.6.tar.gz && cd /mnt/keepalived-2.0.6 && ./configure --prefix=/usr/local/keepalived --with-init=SYSV &> /dev/null && make &> make install &> /dev/null
- creates: /usr/local/keepalived/
/etc/keepalived:
file.directory:
- mode: 755
/etc/sysconfig/keepalived:
file.symlink:
- target: /usr/local/keepalived/etc/sysconfig/keepalived
/sbin/keepalived:
file.symlink:
- target: /usr/local/keepalived/sbin/keepalived
cd pkgs
vim make.sls
make:
pkg.installed:
- pkgs:
- gcc
- pcre-devel
- openssl-devel
用pillar设置变量：
cd /srv/pillar/
make keepalived
cd /srv/pillar/keepalived
vim install.sls
~
{% if grains['fqdn']== 'server1' %}
state: MASTER #如果主机名称是server1;state:MASTER;权重100
vrid: 131
priority: 100
{% elif grains['fqdn']== 'server4'%} #如果主机是server4；state：BACKUP；权重50
state: BACKUP
vrid: 131
priority: 50
{% endif %}
cd /srv/pillar
vim top.sls
base:
'*':
- web.install
- keepalived.install
在files里设置vip
cd /srv/salt/keepalived/files/
vim keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
#vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state {{ STATE }} #状态调用pillar中设置的变量
interface eth0
virtual_router_id {{ VRID }} #虚拟id
priority {{ PRIORITY }} #权重
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.30.100 #VIP
}
}2.部署keepalived：
vim keepalived.servoce
include:
- keepalived.install #调用安装文件
/etc/keepalived/keepalived.conf:
file.managed:
- source: salt://keepalived/files/keepalived.conf
- template: jinja #jinja模块
- context:
STATE: {{ pillar['state'] }}
VRID: {{ pillar['vrid'] }}
PRIORITY: {{ pillar['priority'] }}
kp-service:
file.managed:
- name: /etc/init.d/keepalived
- source: salt://keepalived/files/keepalived
- mode: 755
service.running:
- name: keepalived
- reload: True
- watch:
- file: /etc/keepalived/keepalived.conf3.top 部署haproxy；keepalived高可用负载均衡平台：
上一节中我们部署的haproxy实现的均衡负载；现在我们在原来的基础上安装keepalived实现高可用
vim /srv/salt/top.sls
base:
'server1':
- haproxy.install
- keepalived.service
'server4':
- haproxy.install
- keepalived.service
'roles:apache':
- match: grain
- httpd.install
'roles:nginx':
- match: grain
- nginx.service推送部署：
salt ‘*’ state.highstate
推送测试：
我们这个服务当haproxy关掉并不能实现高可用因此我们编辑了一个监控脚本
4.实现haproxy高可用：
添加一个监控脚本：
cd /opt/
vim check_haproxy.sh #如果haproxy状态关闭就自动开启；如果返回值不等于0关闭keepalived；vip就会调转到另一个keepalived上
#!/bin/bash
/etc/init.d/haproxy status &> /dev/null || /etc/init.d/haproxy restart &> /dev/nnull
if [ $? -ne 0 ];then
/etc/init.d/keepalived stop &> /dev/null
fi
~
chmod +x check_haproxy.sh
vim /srv/salt/keepalived/files/keepalived.conf # 配置文件调用监控脚本
! Configuration File for keepalived
vrrp_script check_haproxy {
script "/opt/check_haproxy.sh"
interval 2
weight 2 #每当发现haproxy宕掉就减少master的权重直到比backup的权重低就会自动切换
}
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
#vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state {{ STATE }}
interface eth0
virtual_router_id {{ VRID }}
priority {{ PRIORITY }}
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.30.100
}
track_script {
check_haproxy
}
}
salt server4 state.sls keepalived.service #将keepalived.service推给server4
chmod +x check_haproxy.sh 测试：
server1:关闭/etc/init.d/haproxy stop
为了防止脚本将haproxy再打开我们可以取消haproxy脚本权限来测试效果测试完毕要将他的权限又赋予
/etc/init.d/salt-minion restart
Salt命令部署：
salt-cp '*' /etc/passwd /tmp/ #cp /etc/passwd到minion的/tmp下面
salt '*' cmd.run 'rm -f /tmp/passwd ' #cmd命令删除minion/tmp/passwd
salt server3 state.single pkg.installed httpd #模块安装单独给server3安装httpd
salt '*' cmd.run 'yum install httpd -y ' #命令安装5.安装数据库缓存数据
5.1 > minioin上安装数据库记录
External Job Cache - Minion-Side Returner
Salt-minion:
yum install -y MySQL-python.x86_64
vim /etc/salt/minion
/etc/init.d/salt-minion restart
yum insatll mysql-server
/etc/init.d/mysqld start
Mysql
mysql> grant all on salt.* to salt@'172.25.30.%' identified by 'westos';
mysql> drop database salt;
mysql < test.sql
salt 'server2' test.ping --return mysql
server2:
True
方法二：http://docs.saltstack.cn/topics/jobs/external_cache.html
5.2.在master安装
Master Job Cache - Master-Side Returner
yum install MySQL-python -y
vim /etc/salt/master
/etc/init.d/mysqld start
Server3:
/etc/init.d/salt-minion restart
6.封装模块：
有时候我们可以把很多函数封装在一个模块中这样在我们调用的过程中就可用调用模块中的函数；这样即方便还便于整理
mkdir /srv/salt/_modules
cd /srv/salt/_modules
vim my_disk.py
#!/usr/bin/env python
def df():
return __salt__['cmd.run']('df -h')
salt '*' saltutil.sync_modules #同步模块
当模块同步到minion中时会将封装文件推送到minion的缓存中和grains相似
执行封装模块命令
7.Syndic：
部署： server4——>top master—->master of master
Server4:
在master删除minion中的server4(salt-key -d server4)；server4把他之前的服务都关掉
/etc/init.d/salt-minion stop
chkconfig salt-minion off
/etc/init.d/haproxy stop
/etc/init.d/keepalived stop
yum install salt-master -y
vim /etc/salt/master #开启syndic端口
etc/init.d/salt-master start
Sever1:master----->syndic
yum install salt-syndic
vim /etc/salt/master
/etc/init.d/salt-master start
/etc/init.d/salt-master stop
/etc/init.d/salt-syndic startServer4:top-master:
当环境不允许我们安装salt我们可以通过salt-ssh来部署：
8.salt-ssh
Master端：
yum install -y salt-ssh
vim /etc/salt/roster #添加server3 因为是root所以不用赋予权限
vim /etc/salt/master #ssh和mysql有冲突所以我们需要注释mysql
/etc/init.d/salt-master stop
/etc/init.d/salt-master start
salt-ssh 'server3' test.ping关闭server3的minion测试：
cd /etc/pki/tls/private/
openssl genrsa 1024 > localhost.key #生成key在ocalhost.key文件中
cd ../certs/
make testcert #制作证书
vim /etc/salt/master.d/api.conf #api配置文件定义api端口加密文件和证书文件
rest_cherrypy:
port: 8000
ssl_crt: /etc/pki/tls/certs/localhost.crt
ssl_key: /etc/pki/tls/private/localhost.key
vim auth.conf #编辑认证配置文件
external_auth:
pam:
saltapi:
- '.*'
- '@wheel'
- '@runner'
- '@jobs'
useradd saltapi #创建认证用户和密码
passwd saltapi重启服务
[root@server1 master.d]# curl -sSk https://localhost:8000/login -H 'Accept: application/x-yaml' -d username=saltapi -d password=westos -d eauth=pam
return:
- eauth: pam
expire: 1534598878.4669149
perms:
- .*
- '@wheel'
- '@runner'
- '@jobs'
start: 1534555678.4669139
token: **4f0f24b3cea429878d66af4c1ca2d9cd630aef41**
user: saltapi
[root@server1 master.d]# curl -sSk https://localhost:8000 \
> -H 'Accept: application/x-yaml' \
> -H 'X-Auth-Token: **4f0f24b3cea429878d66af4c1ca2d9cd630aef41**' \
> -d client=local \
> -d tgt='*' \
> -d fun=test.ping
return:
- server1: true
server2: true
server3: true
vim saltapi.py #编辑api模块用html编写py文件调用方法：
# -*- coding: utf-8 -*-
import urllib2,urllib
import time
try:
import json
except ImportError:
import simplejson as json
class SaltAPI(object):
__token_id = ''
def __init__(self,url,username,password):
self.__url = url.rstrip('/')
self.__user = username
self.__password = password
def token_id(self):
''' user login and get token id '''
params = {'eauth': 'pam', 'username': self.__user, 'password': self.__password}
encode = urllib.urlencode(params)
obj = urllib.unquote(encode)
content = self.postRequest(obj,prefix='/login')
try:
self.__token_id = content['return'][0]['token']
except KeyError:
raise KeyError
def postRequest(self,obj,prefix='/'):
url = self.__url + prefix
headers = {'X-Auth-Token' : self.__token_id}
req = urllib2.Request(url, obj, headers)
opener = urllib2.urlopen(req)
content = json.loads(opener.read())
return content
def list_all_key(self):
params = {'client': 'wheel', 'fun': 'key.list_all'}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
minions = content['return'][0]['data']['return']['minions']
minions_pre = content['return'][0]['data']['return']['minions_pre']
return minions,minions_pre
def delete_key(self,node_name):
params = {'client': 'wheel', 'fun': 'key.delete', 'match': node_name}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
ret = content['return'][0]['data']['success']
return ret
def accept_key(self,node_name):
params = {'client': 'wheel', 'fun': 'key.accept', 'match': node_name}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
ret = content['return'][0]['data']['success']
return ret
def remote_noarg_execution(self,tgt,fun):
''' Execute commands without parameters '''
params = {'client': 'local', 'tgt': tgt, 'fun': fun}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
return content
def async_deploy(self,tgt,arg):
''' Asynchronously send a command to connected minions '''
params = {'client': 'local_async', 'tgt': tgt, 'fun': 'state.sls', 'arg': arg}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
jid = content['return'][0]['jid']
return jid
def target_deploy(self,tgt,arg):
''' Based on the node group forms deployment '''
params = {'client': 'local_async', 'tgt': tgt, 'fun': 'state.sls', 'arg': arg, 'expr_form': 'nodegroup'}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
content = self.postRequest(obj)
ret = content['return'][0][tgt]
return ret
def remote_execution(self,tgt,fun,arg):
''' Command execution with parameters '''
params = {'client': 'local', 'tgt': tgt, 'fun': fun, 'arg': arg}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
jid = content['return'][0]['jid']
return jid
def deploy(self,tgt,arg):
''' Module deployment '''
params = {'client': 'local', 'tgt': tgt, 'fun': 'state.sls', 'arg': arg}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
return content
def async_deploy(self,tgt,arg):
''' Asynchronously send a command to connected minions '''
params = {'client': 'local_async', 'tgt': tgt, 'fun': 'state.sls', 'arg': arg}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
jid = content['return'][0]['jid']
return jid
def target_deploy(self,tgt,arg):
''' Based on the node group forms deployment '''
params = {'client': 'local_async', 'tgt': tgt, 'fun': 'state.sls', 'arg': arg, 'expr_form': 'nodegroup'}
obj = urllib.urlencode(params)
self.token_id()
content = self.postRequest(obj)
jid = content['return'][0]['jid']
return jid
def main():
sapi = SaltAPI(url='https://**172.25.30.1:8000**',username='saltapi',password='westos')
#sapi.token_id()
print sapi.list_all_key()
#sapi.delete_key('test-01')
#sapi.accept_key('test-01')
sapi.deploy('**server3','nginx.service**')
#print sapi.remote_noarg_execution('test-01','grains.items')
if __name__ == '__main__':
main()
