Mysql-porxy 读写分离与Tcpdumo抓包工具
一、mysql-proxy概念：
MySQL Proxy 可以做Mysql数据库代理，基于mysql主从复制之上，能够实现读写分离、负载均衡等功能，即可以实现数据写入在主服务器，而数据查询则去往从服务器进行查询，从而大大降低主服务器的负载，对提升整个mysq集群能力有很大的作用。
下载地址：http://dev.mysql.com/downloads/mysql-proxy/
二、配置mysql主从集群，详情见http://zhangshijie.blog.51cto.com/806066/1606071
请确保主从同步运行正常、且时间一致，然后在继续下一步。
实验环境：
主服务器：192.168.10.204 MariaDB 10.0.10
从服务器：192.168.10.205 MariaDB 10.0.10
mysql-proxy服务器：192.168.10.206
mysql-proxy版本：0.8.3
1、mysql-proxy安装：
[root@node6 ~]# tar xvf /root/mysql-proxy-0.8.3-linux-glibc2.3-x86-64bit.tar.gz -C /usr/local/
[root@node6 local]# ln -sv mysql-proxy-0.8.3-linux-glibc2.3-x86-64bit mysql-proxy
2、添加mysql-proxy用户：
[root@node6 local]# useradd mysql-proxy
3、编辑开机脚本:
[root@node6 local]# vim /etc/rc.d/init.d/mysql-proxy
#!/bin/bash
#
# mysql-proxy This script starts and stops the mysql-proxy daemon
#
# chkconfig: - 78 30
# processname: mysql-proxy
# description: mysql-proxy is a proxy daemon for mysql
# Source function library.
. /etc/rc.d/init.d/functions
prog="/usr/local/mysql-proxy/bin/mysql-proxy"
# Source networking configuration.
if [ -f /etc/sysconfig/network ]; then
. /etc/sysconfig/network
fi
# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0
# Set default mysql-proxy configuration.
ADMIN_USER="admin"
ADMIN_PASSWD="admin"
ADMIN_LUA_SCRIPT="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua"
PROXY_OPTIONS="--daemon"
PROXY_PID=/var/run/mysql-proxy.pid
PROXY_USER="mysql-proxy"
# Source mysql-proxy configuration.
if [ -f /etc/sysconfig/mysql-proxy ]; then
. /etc/sysconfig/mysql-proxy
fi
RETVAL=0
start() {
echo -n $"Starting $prog: "
daemon $prog $PROXY_OPTIONS --pid-file=$PROXY_PID --proxy-address="$PROXY_ADDRESS" --user=$PROXY_USER --admin-username="$ADMIN_USER" --admin-lua-script="$ADMIN_LUA_SCRIPT" --admin-password="$ADMIN_PASSWORD"
RETVAL=$?
echo
if [ $RETVAL -eq 0 ]; then
touch /var/lock/subsys/mysql-proxy
fi
}
stop() {
echo -n $"Stopping $prog: "
killproc -p $PROXY_PID -d 3 $prog
RETVAL=$?
echo
if [ $RETVAL -eq 0 ]; then
rm -f /var/lock/subsys/mysql-proxy
rm -f $PROXY_PID
fi
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
condrestart|try-restart)
if status -p $PROXY_PIDFILE $prog >&/dev/null; then
stop
start
fi
;;
status)
status -p $PROXY_PID $prog
;;
*)
echo "Usage: $0 {start|stop|restart|reload|status|condrestart|try-restart}"
RETVAL=1
;;
esac
exit $RETVAL
4、为服务脚本提供配置文件/etc/sysconfig/mysql-proxy，内容如下所示：
# Options for mysql-proxy
ADMIN_USER="admin"
ADMIN_PASSWORD="admin"
ADMIN_ADDRESS=""
ADMIN_LUA_SCRIPT="/usr/local/mysql-proxy/share/doc/mysql-proxy/admin.lua"
PROXY_ADDRESS=""
PROXY_USER="mysql-proxy"
PROXY_OPTIONS="--daemon --log-level=info --log-use-syslog --plugins=proxy --plugins=admin --proxy-backend-addresses=192.168.10.204:3306 --proxy-read-only-backend-addresses=192.168.10.205:3306 --proxy-lua-script=/usr/local/mysql-proxy/share/doc/mysql-proxy/rw-splitting.lua"
5、 mysql-proxy的配置选项
mysql-proxy的配置选项大致可分为帮助选项、管理选项、代理选项及应用程序选项几类：
--help
--help-admin
--help-proxy
--help-all ―――― 以上四个选项均用于获取帮助信息；
--proxy-address=host:port ―――― 代理服务监听的地址和端口；
--admin-address=host:port ―――― 管理模块监听的地址和端口；
--proxy-backend-addresses=host:port ―――― 后端mysql服务器的地址和端口；
--proxy-read-only-backend-addresses=host:port ―――― 后端只读mysql服务器的地址和端口；
--proxy-lua-script=file_name ―――― 完成mysql代理功能的Lua脚本；
--daemon ―――― 以守护进程模式启动mysql-proxy；
--keepalive ―――― 在mysql-proxy崩溃时尝试重启之；
--log-file=/path/to/log_file_name ―――― 日志文件名称；
--log-level=level ―――― 日志级别；
--log-use-syslog ―――― 基于syslog记录日志；
--plugins=plugin,.. ―――― 在mysql-proxy启动时加载的插件；
--user=user_name ―――― 运行mysql-proxy进程的用户；
--defaults-file=/path/to/conf_file_name ―――― 默认使用的配置文件路径；其配置段使用[mysql-proxy]标识；
--proxy-skip-profiling ―――― 禁用profile；
--pid-file=/path/to/pid_file_name ―――― 进程文件名；
6、创建admin.lua文件，将其保存至/usr/local/mysql-proxy/share/doc/mysql-proxy/目录中
[root@node6 local]# vim /usr/local/mysql-proxy/share/doc/mysql-proxy/load-multi.lua
--[[ $%BEGINLICENSE%$
Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; version 2 of the
License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
02110-1301 USA
$%ENDLICENSE%$ --]]
---
--[[
(Not so) simple example of a run-time module loader for MySQL Proxy
Usage:
1. load this module in the Proxy.
2. From a client, run the command
PLOAD name_of_script
and you can use the features implemented in the new script
immediately.
3. Successive calls to PLOAD will load a new one.
4. Previous scripts will still be active, and the loader will
use all of them in sequence to see if one handles the
request
5. CAVEAT. If your script has a read_query function that
*always* returns a non null value (e.g. a logging feature)
then other modules, although loaded, are not used.
If you have any such modules, you must load them at the
end of the sequence.
6. to remove a module, use
UNLOAD module_name
IMPORTANT NOTICE:
the proxy will try to load the file in the directory where
the proxy (NOT THE CLIENT) was started.
To use modules not in such directory, you need to
specify an absolute path.
--]]
local VERSION = '0.1.3'
local DEBUG = os.getenv('DEBUG') or 0
DEBUG = DEBUG + 0
---
-- print_debug()
-- conditionally print a message
--
function proxy.global.print_debug (msg)
if DEBUG > 0 then
print(msg)
end
end
---
-- Handlers for MySQL Proxy hooks
-- Initially, they are void.
-- If the loaded module has implemented any
-- functions, they are associated with these handlers
--
if proxy.global.loaded_handlers == nil then
proxy.global.loaded_handlers = {
rq = {},
rqr = {},
dc = {},
cs = {},
rh = {},
ra = {},
rar = {},
}
end
---
-- list of functions loaded from user modules
--
if proxy.global.handled_function == nil then
proxy.global.handled_functions = {
rq = 'read_query',
rqr = 'read_query_result',
cs = 'connect_server',
rh = 'read_handshake',
ra = 'read_auth',
rar = 'read_auth_result',
dc = 'disconnect_client',
}
end
if proxy.global.handler_status == nil then
proxy.global.handler_status = {}
end
local funcs_to_create = {
cs = 0,
ra = 1,
rar = 1,
rh = 1,
dc = 0,
}
---
-- creates the hooks for Proxy handled functions
--
for id, has_param in pairs(funcs_to_create) do
local parameter = has_param == 1 and 'myarg' or ''
local fstr = string.format(
[[
function %s(%s)
if #proxy.global.loaded_handlers['%s'] then
for i, h in pairs(proxy.global.loaded_handlers['%s'])
do
if h then
proxy.global.print_debug ( 'handling "%s" using ' .. h.name)
local result = h.handler(%s)
if result then
return result
end
end
end
end
-- return
end
]],
proxy.global.handled_functions[id],
parameter,
id, id, proxy.global.handled_functions[id], parameter
)
proxy.global.print_debug ('creating function ' .. proxy.global.handled_functions[id])
assert(loadstring(fstr))()
end
---
-- an error string to describe the error occurring
--
local load_multi_error_str = ''
---
-- set_error()
--
-- sets the error string
--
-- @param msg the message to be assigned to the error string
--
function set_error(msg)
load_multi_error_str = msg
proxy.global.print_debug (msg)
return nil
end
---
-- file_exists()
--
-- checks if a file exists
--
-- @param fname the file name
--
function file_exists(fname)
local fh=io.open( fname, 'r')
if fh then
fh:close()
return true
else
return false
end
end
local module_counter = 0
---
-- get_module()
--
-- This function scans an existing lua file, and turns it into
-- a closure exporting two function handlers, one for
-- read_query() and one for read_query_result().
-- If the input script does not contain the above functions,
-- get_module fails.
--
-- @param module_name the name of the existing script
--
function get_module(module_name)
--
-- assumes success
--
load_multi_error_str = ''
--
-- the module is copied to a temporary file
-- on a given directory
--
module_counter = module_counter + 1
local new_module = 'tmpmodule' .. module_counter
local tmp_dir = '/tmp/'
local new_filename = tmp_dir .. new_module .. '.lua'
local source_script = module_name
if not source_script:match('.lua$') then
source_script = source_script .. '.lua'
end
--
-- if the new module does not exist
-- an error is returned
--
if not file_exists(source_script) then
set_error('file not found ' .. source_script)
return
end
--
-- if the module directory is not on the search path,
-- we need to add it
--
if not package.path:match(tmp_dir) then
package.path = tmp_dir .. '?.lua;' .. package.path
end
--
-- Make sure that the module is not loaded.
-- If we don't remove it from the list of loaded modules,
-- subsequent load attempts will silently fail
--
package.loaded[new_module] = nil
local ofh = io.open(new_filename, 'w')
--
-- Writing to the new package, starting with the
-- header and the wrapper function
--
ofh:write( string.format(
"module('%s', package.seeall)\n"
.. "function make_funcs()\n" , new_module)
)
local found_funcs = {}
--
-- Copying contents from the original script
-- to the new module, and checking for the existence
-- of the handler functions
--
for line in io.lines(source_script) do
ofh:write(line .. "\n")
for i,v in pairs(proxy.global.handled_functions) do
if line:match('^%s*function%s+' ..v .. '%s*%(') then
found_funcs[i] = v
break
end
end
end
--
-- closing the wrapper on the new module
--
local return_value = ''
for i,v in pairs(found_funcs) do
return_value = return_value .. i .. ' = ' .. v .. ', '
end
ofh:write(
'return { ' .. return_value .. '}\n' ..
'end\n'
)
ofh:close()
--
-- Final check. If the handlers were not found, the load fails
--
--
if (found_one == false ) then
set_error('script ' .. source_script ..
' does not contain a proxy handled function')
return
end
--
-- All set. The new module is loaded
-- with a new function that will return the handlers
--
local result = require(new_module)
os.remove(new_filename)
return result
end
---
-- simple_dataset()
--
-- returns a dataset made of a header and a message
--
-- @param header the column name
-- @param message the contents of the column
function proxy.global.simple_dataset (header, message)
proxy.response.type = proxy.MYSQLD_PACKET_OK
proxy.response.resultset = {
fields = {{type = proxy.MYSQL_TYPE_STRING, name = header}},
rows = { { message} }
}
return proxy.PROXY_SEND_RESULT
end
---
-- make_regexp_from_command()
--
-- creates a regular expression for fast scanning of the command
--
-- @param cmd the command to be converted to regexp
--
function proxy.global.make_regexp_from_command(cmd, options)
local regexp= '^%s*';
for ch in cmd:gmatch('(.)') do
regexp = regexp .. '[' .. ch:upper() .. ch:lower() .. ']'
end
if options and options.capture then
regexp = regexp .. '%s+(%S+)'
end
return regexp
end
--
-- The default command for loading a new module is PLOAD
-- You may change it through an environment variable
--
local proxy_load_command = os.getenv('PLOAD') or 'pload'
local proxy_unload_command = os.getenv('PUNLOAD') or 'punload'
local proxy_help_command = os.getenv('PLOAD_HELP') or 'pload_help'
local currently_using = 0
local pload_regexp = proxy.global.make_regexp_from_command(proxy_load_command, {capture = 1})
local punload_regexp = proxy.global.make_regexp_from_command(proxy_unload_command,{ capture = 1} )
local pload_help_regexp = proxy.global.make_regexp_from_command(proxy_help_command,{ capture = nil} )
local pload_help_dataset = {
{proxy_load_command .. ' module_name', 'loads a given module'},
{proxy_unload_command .. 'module_name', 'unloads and existing module'},
{proxy_help_command, 'shows this help'},
}
---
-- removes a module from the loaded list
--
function remove_module (module_name)
local found_module = false
local to_delete = { loaded = {}, status = {} }
for i,lmodule in pairs(proxy.global.handler_status) do
if i == module_name then
found_module = true
local counter = 0
for j,h in pairs(lmodule) do
-- proxy.global.print_debug('removing '.. module_name .. ' (' .. i ..') ' .. h.id .. ' -> ' .. h.ndx )
to_delete['loaded'][h.id] = h.ndx
counter = counter + 1
to_delete['status'][i] = counter
end
end
end
for i,v in pairs (to_delete['loaded']) do
table.remove(proxy.global.loaded_handlers[i], v)
end
for i,v in pairs (to_delete['status']) do
table.remove(proxy.global.handler_status[i], v)
end
if found_module == false then
return proxy.global.simple_dataset(module_name, 'NOT FOUND')
end
return proxy.global.simple_dataset(module_name, 'unloaded')
end
---
-- creates a dataset from a list of header names
-- and a list of rows
-- @param header a list of field names
-- @param dataset a list of row contents
function proxy.global.make_dataset (header, dataset)
proxy.response.type = proxy.MYSQLD_PACKET_OK
proxy.response.resultset = {
fields = {},
rows = {}
}
for i,v in pairs (header) do
table.insert(proxy.response.resultset.fields, {type = proxy.MYSQL_TYPE_STRING, name = v})
end
for i,v in pairs (dataset) do
table.insert(proxy.response.resultset.rows, v )
end
return proxy.PROXY_SEND_RESULT
end
--
-- This function is called at each query.
-- The request for loading a new script is handled here
--
function read_query (packet)
currently_using = 0
if packet:byte() ~= proxy.COM_QUERY then
return
end
local query = packet:sub(2)
-- Checks if a PLOAD command was issued.
-- A regular expresion check is faster than
-- doing a full tokenization. (Especially on large queries)
--
if (query:match(pload_help_regexp)) then
return proxy.global.make_dataset({'command','description'}, pload_help_dataset)
end
local unload_module = query:match(punload_regexp)
if (unload_module) then
return remove_module(unload_module)
end
local new_module = query:match(pload_regexp)
if (new_module) then
--[[
If a request for loading is received, then
we attempt to load the new module using the
get_module() function
--]]
local new_tablespace = get_module(new_module)
if (new_tablespace) then
local handlers = new_tablespace.make_funcs()
--
-- The new module must have at least handlers for read_query()
-- or disconnect_client. read_query_result() is optional.
-- The loading function returns nil if no handlers were found
--
proxy.global.print_debug('')
proxy.global.handler_status[new_module] = {}
for i,v in pairs( proxy.global.handled_functions) do
local handler_str = type(handlers[i])
proxy.global.print_debug (i .. ' ' .. handler_str )
if handlers[i] then
table.insert(proxy.global.loaded_handlers[i] ,
{ name = new_module, handler = handlers[i]} )
table.insert(proxy.global.handler_status[new_module],
{ func = proxy.global.handled_functions[i], id=i, ndx = #proxy.global.loaded_handlers[i] })
end
end
if handlers['rqr'] and not handlers['rq'] then
table.insert(proxy.global.loaded_handlers['rq'] , nil )
end
if handlers['rq'] and not handlers['rqr'] then
table.insert(proxy.global.loaded_handlers['rqr'] , nil )
end
--
-- Returns a confirmation that a new module was loaded
--
return proxy.global.simple_dataset('info', 'module "' .. new_module .. '" loaded' )
else
--
-- The load was not successful.
-- Inform the user
--
return proxy.global.simple_dataset('ERROR', 'load of "'
.. new_module .. '" failed ('
.. load_multi_error_str .. ')' )
end
end
--
-- If a handler was installed from a new module, it is called
-- now.
--
if #proxy.global.loaded_handlers['rq'] then
for i,rqh in pairs(proxy.global.loaded_handlers['rq'])
do
proxy.global.print_debug ( 'handling "read_query" using ' .. i .. ' -> ' .. rqh.name)
local result = rqh.handler(packet)
if (result) then
currently_using = i
return result
end
end
end
end
--
-- this function is called every time a result set is
-- returned after an injected query
--
function read_query_result(inj)
--
-- If the dynamically loaded module had an handler for read_query_result()
-- it is called now
--
local rqrh = proxy.global.loaded_handlers['rqr'][currently_using]
if rqrh then
proxy.global.print_debug ( 'handling "read_query_result" using ' .. currently_using .. ' -> ' .. rqrh.name)
local result = rqrh.handler(inj)
if result then
return result
end
end
end
7、添加到启动列表设置开机启动并启动服务：
[root@node6 local]# chkconfig --add mysql-proxy
[root@node6 local]# chkconfig mysql-proxy on
[root@node6 local]# service mysql-proxy start
三：验证：
1、端口验证：
2、管理功能验证：
主从服务器创建远程账号：
mysql> grant all on *.* to 'admin'@'192.168.%.%' identified by '123456';
mysql> grant all on *.* to 'admin'@'192.168.%.%' identified by '123456';
使用服务器账号在mysql-proxy服务器登陆：
[root@node6 local]# mysql -uadmin -padmin -h192.168.10.206 --port 4041
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.0.99-agent-admin
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> SELECT * FROM backends;
+-------------+---------------------+---------+------+------+-------------------+
| backend_ndx | address | state | type | uuid | connected_clients |
+-------------+---------------------+---------+------+------+-------------------+
| 1 | 192.168.10.204:3306 | unknown | rw | NULL | 0 |
| 2 | 192.168.10.205:3306 | unknown | ro | NULL | 0 |
+-------------+---------------------+---------+------+------+-------------------+
3、可以使用mysql-proxy连接从服务器查看mysql状态，也可以连接主服务器创建数据库和表以及插入数据：
[root@node6 local]# mysql -uadmin -padmin -h192.168.10.205 --port 3306
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 5.5.5-10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
4. 读写分离测试
[root@node6 local]# mysql -uadmin -padmin -h192.168.10.206 --port 4041
四：使用tcpdumo抓包工具抓包：
要想抓包必须要启用网卡的混杂模式
-i:任意接口
-n:不反解IP地址为主机名
-nn：不反解IP地址为主机名及端口号到服务器名
-X:将数据包转换为ASCII格式方便查看
-XX:将数据包转换为ASCII格式方便查看并显示数据帧信息
-v，-vv,-vvv 详细信息
-c 指定获取的数据包数量
-s:定义字节捕获单元长度（大小），默认即使此选项，不需要单独指定
-S:显示TCP/IP序列号
-e:获以太网首部信息
-A:将捕获的报文以ASCII显示
tcpdump的语法：
tcpdump [options] [Protocol] [Direction] [Host(s)] [Value] [Logical Operations] [Other expression]
Protocol(协议):
Values(取值): ether, fddi, ip, arp, rarp, decnet, lat, sca, moprc, mopdl, tcp and udp.
If no protocol is specified, all the protocols are used.
Direction(流向):
Values(取值): src, dst, src and dst, src or dst
If no source or destination is specified, the "src or dst" keywords are applied. (默认是src or dst)
For example, "host 10.2.2.2" is equivalent to "src or dst host 10.2.2.2".
Host(s)(主机):
Values(替代关键字): net, port, host, portrange.
If no host(s) is specified, the "host" keyword is used. 默认如果此段没有指定关键字，默认即host。
For example, "src 10.1.1.1" is equivalent to "src host 10.1.1.1".
Logical Operations:
(1) AND
and or &&
(2) OR
or or ||
(3) EXCEPT
not or !
测试ftp抓包:
[root@node4 ~]# tcpdump -i eth0 -nn -XX dst host 192.168.10.204 and tcp dst port 21
可以抓取到ftp明文的用户名密码，因此在公网一定不使用明文账户口令，尤其是mysql数据要使用ssl加密传输。
测试完成！！！！！！！！！！！！1