集群系列教程之:keepalived+lvs
前言：最近看群里很多人在问keepalived+lvs的架构怎么弄，出了各种各样的问题，为此特别放下了别的文档，先写一篇keepalived+lvs架构的文档，使那些有需求的人能够得以满足。但是此篇文档是架构文档，不是基础理论，但我想你能做这个架构，势必也了解了基础理论知识，更多的理论知识体系，请看下回分解。。。。
测试拓扑：
环境说明： 从上面的拓扑图，就可以看出本实验的环境信息，其中实线代表的是真实的物理连接，而虚线表示的是逻辑关系。hostname:test1.lxm.comIP: vip:eth0:0:10.0.10.200/24 //会不会有人看不懂含义啊？这个表示vip地址配置在eth0网卡的虚拟网卡eth0:0上，且ip地址是10.0.10.200/24 DIP:eth0:10.0.10.11/24 HAip:eth1:10.0.0.16/24 //所谓的HAip就是要用来传递心跳信息的网卡上的ip地址，在线上环境通常是要把数据网卡和心跳网卡分开的function: 用做keepalived+lvs的前端主机
hostname:test2.lxm.comIP: vip:eth0:0；10.0.10.200/24 DIP：eth0:10.0.10.12/24 HAip:eth1:10.0.10.18/24function:用做keepalived+lvs的前端主机
hostname:test3.lxm.comIP: vip:lo:0:10.0.10.200/32 RIP:eth0:10.0.10.13/24function:用作后端web
hostname:test4.lxm.comIP: vip:lo:0:10.0.10.200/32 RIP:eth0:10.0.10.14/24function:用作后端web
hostname:test5.lxm.comIP： vip:lo:0:10.0.10.200/32 RIP:eth0:10.0.10.15/24function: 用来做备用的web，什么 意思呢?就是当后端web全部故障的时候，keepalived可以设置将其请求转发到这台web上，给用户一个好的错误体验而已，体验内容随你自己喜欢。。
此外，这里的lvs使用的模型是DR模型，在拓扑上已经标注，其实从ip地址信息，也应该能看出来是DR模型。。。 初始化工作： 所有主机配置好网络环境，尤其要注意前端两台做keepalived+lvs的是两个网卡，其次关闭selinux和iptables，配置好yum源。另外一个重点是一定要所有主机同步好时间，这将是你集群能否成功的关键。。 架构部署： 关于架构部署的事情，我想说的是，不要一上来就乱搞一通，这个装装，那个装装，最后一测试，巴拉巴拉一大堆错误，看到错误信息，脑袋直接蒙了，不知道该怎么检查。。所以部署架构的时候一定要分层次。一个层次一个层次的去测试，直到最终完成。。 就比如这里谈到的keepalived+lvs的架构，下面分这几个层次部署：1.先部署后端三台web服务器，并测试web服务能够正常访问；2.在前端两台主机上分别部署lvs，整合lvs+web,分别测试两台lvs主机和后端web配合能正常提供服务；3.在前端两台主机上分别部署keepalived，整合keepalived+lvs；4.测试keepalived是否能够满足HA的需求；哈哈，看了上面几个层次，是否顿觉架构清晰了不少，当然不排除你比我有更好的思路哦。。。。。
1.部署realserver(后端web)上的web服务主机：test3.lxm.com[root@test3 /]# rpm -qa | grep httpd //这里以httpd为例，如果你需要，你也可以使用nginx等其他web软件[root@test3 /]# yum -y install httpd[root@test3 /]# cd /var/www/html/[root@test3 html]# ls[root@test3 html]# echo "<h1>this is test3.lxm.com</h1>" > index.html[root@test3 html]# vim /etc/httpd/conf/httpd.conf 将配置文件中的ServerName字段修改为：ServerName 0.0.0.0:80
[root@test3 html]# service httpd startStarting httpd: [ OK ][root@test3 html]# [root@test3 html]# ps aux | grep httpdroot 1660 0.0 1.4 175708 3660 ? Ss 11:56 0:00 /usr/sbin/httpdapache 1662 0.0 0.9 175708 2392 ? S 11:56 0:00 /usr/sbin/httpdapache 1663 0.0 0.9 175708 2392 ? S 11:56 0:00 /usr/sbin/httpdapache 1664 0.0 0.9 175708 2392 ? S 11:56 0:00 /usr/sbin/httpdapache 1665 0.0 0.9 175708 2392 ? S 11:56 0:00 /usr/sbin/httpdapache 1666 0.0 0.9 175708 2392 ? S 11:56 0:00 /usr/sbin/httpdapache 1667 0.0 0.9 175708 2392 ? S 11:56 0:00 /usr/sbin/httpdapache 1668 0.0 0.9 175708 2392 ? S 11:56 0:00 /usr/sbin/httpdapache 1669 0.0 0.9 175708 2392 ? S 11:56 0:00 /usr/sbin/httpdroot 1679 0.0 0.3 103244 848 pts/0 S+ 11:58 0:00 grep httpd[root@test3 html]# netstat -nultp | grep httpdtcp 0 0 :::80 :::* LISTEN 1660/httpd [root@test3 html]# links --dump
this is test3.lxm.com[root@test3 html]到此，第一台web已经搭建完成。。。。
主机：test4.lxm.com[root@test4 /]# rpm -qa | grep httpd[root@test4 /]# yum -y install httpd[root@test4 /]# cd /var/www/html/[root@test4 html]# ls[root@test4 html]# echo "<h1>this is test4.lxm.com</h1>" > index.html[root@test4 html]# vim /etc/httpd/conf/httpd.conf 将配置文件中的ServerName字段修改为：ServerName 0.0.0.0:80[root@test4 html]# service httpd startStarting httpd: [ OK ][root@test4 html]# ps aux | grep httpdroot 1672 0.0 1.5 175708 3668 ? Ss 12:01 0:00 /usr/sbin/httpdapache 1674 0.0 0.9 175708 2400 ? S 12:01 0:00 /usr/sbin/httpdapache 1675 0.0 0.9 175708 2400 ? S 12:01 0:00 /usr/sbin/httpdapache 1676 0.0 0.9 175708 2400 ? S 12:01 0:00 /usr/sbin/httpdapache 1677 0.0 0.9 175708 2400 ? S 12:01 0:00 /usr/sbin/httpdapache 1678 0.0 0.9 175708 2400 ? S 12:01 0:00 /usr/sbin/httpdapache 1679 0.0 0.9 175708 2400 ? S 12:01 0:00 /usr/sbin/httpdapache 1680 0.0 0.9 175708 2400 ? S 12:01 0:00 /usr/sbin/httpdapache 1681 0.0 0.9 175708 2400 ? S 12:01 0:00 /usr/sbin/httpdroot 1683 0.0 0.3 103244 848 pts/1 S+ 12:01 0:00 grep httpd[root@test4 html]# netstat -nultp | grep httpdtcp 0 0 :::80 :::* LISTEN 1672/httpd [root@test4 html]# links --dump
this is test4.lxm.com[root@test4 html]#到此，第二台web已经搭建测试完成。。。
注意：这里的每台web服务器页面内容都是不一样的，这是为了测试的时候效果明显，在生产环境中，加入负载均衡的web服务器上的web内容必须要保持一致。
2.部署备用web上的web服务主机：test5.lxm.com[root@test4 /]# rpm -qa | grep httpd[root@test4 /]# yum -y install httpd[root@test5 /]# cd /var/www/html/[root@test5 html]# ls[root@test5 html]# echo "<h1>this is test5.lxm.com</h1>" > index.html[root@test5 html]# vim /etc/httpd/conf/httpd.conf将配置文件中的ServerName字段修改为：ServerName 0.0.0.0:80[root@test5 html]# service httpd startStarting httpd: [ OK ][root@test5 html]# ps aux | grep httpdroot 1613 0.0 1.5 175708 3664 ? Ss 12:03 0:00 /usr/sbin/httpdapache 1615 0.0 0.9 175708 2396 ? S 12:03 0:00 /usr/sbin/httpdapache 1616 0.0 0.9 175708 2396 ? S 12:03 0:00 /usr/sbin/httpdapache 1617 0.0 0.9 175708 2396 ? S 12:03 0:00 /usr/sbin/httpdapache 1618 0.0 0.9 175708 2396 ? S 12:03 0:00 /usr/sbin/httpdapache 1619 0.0 0.9 175708 2396 ? S 12:03 0:00 /usr/sbin/httpdapache 1620 0.0 0.9 175708 2396 ? S 12:03 0:00 /usr/sbin/httpdapache 1621 0.0 0.9 175708 2396 ? S 12:03 0:00 /usr/sbin/httpdapache 1622 0.0 0.9 175708 2396 ? S 12:03 0:00 /usr/sbin/httpdroot 1624 0.0 0.3 103244 848 pts/0 S+ 12:03 0:00 grep httpd[root@test5 html]# netstat -nultp | grep httpdtcp 0 0 :::80 :::* LISTEN 1613/httpd [root@test5 html]# links --dump
hello world!!![root@test5 html]#到此，备份web也搭建完成了....
3.部署lvs环境1)安装配置两台lvs director环境主机:test1.lxm.com[root@test1 /]# grep -i 'ip_vs' /boot/config-2.6.32-431.el6.x86_64 //查看当前系统内核是否支持lvs的功能，默认情况下都已经将lvs的模块集成到内核了。。。CONFIG_IP_VS=mCONFIG_IP_VS_IPV6=y# CONFIG_IP_VS_DEBUG is not setCONFIG_IP_VS_TAB_BITS=12CONFIG_IP_VS_PROTO_TCP=yCONFIG_IP_VS_PROTO_UDP=yCONFIG_IP_VS_PROTO_AH_ESP=yCONFIG_IP_VS_PROTO_ESP=yCONFIG_IP_VS_PROTO_AH=yCONFIG_IP_VS_PROTO_SCTP=yCONFIG_IP_VS_RR=mCONFIG_IP_VS_WRR=mCONFIG_IP_VS_LC=mCONFIG_IP_VS_WLC=mCONFIG_IP_VS_LBLC=mCONFIG_IP_VS_LBLCR=mCONFIG_IP_VS_DH=mCONFIG_IP_VS_SH=mCONFIG_IP_VS_SED=mCONFIG_IP_VS_NQ=mCONFIG_IP_VS_FTP=mCONFIG_IP_VS_PE_SIP=m[root@test1 /]#
[root@test1 /]# rpm -qa | grep ipvsadm[root@test1 /]# yum -y install ipvsadmin //安装lvs用户空间的管理软件ipvsadm，[root@test1 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn[root@test1 /]#
如果能出现上面的信息，表示ipvsadm已经安装成功。。注：这个地方有人会产生怀疑，你怎么没还没启动就开始查看了呢？其实lvs是内核的一种功能，内核默认就支持了这种功能，ipvsadm本身就是用户空间的一种管理工具，启动停止ipvsadm无非就是刷新规则的过程。并不影响你使用ipvsadm来进行管理。不知道大家有没有注意到iptables，其实iptables也是这个特性。
配置VIP：[root@test1 /]# ifconfig eth0:0 10.0.10.200 netmask 255.255.255.0[root@test1 /]# ifconfig eth0:0eth0:0 Link encap:Ethernet HWaddr 08:00:27:ED:EF:33 inet addr:10.0.10.200 Bcast:10.0.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
从上面的信息可见，VIP配置成功。。。
配置LVS规则： 关于LVS的模型，在上面的环境说明中已经描述，使用DR模型,此外由于该技术文档的重点是测试keepalived的功能，所以这里对负载均衡的策略选择标准的rr(轮询)策略[root@test1 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn[root@test1 /]# ipvsadm -A -t 10.0.10.200:80 -s rr[root@test1 /]# ipvsadm -a -t 10.0.10.200:80 -r 10.0.10.13:80 -g[root@test1 /]# ipvsadm -a -t 10.0.10.200:80 -r 10.0.10.14:80 -g[root@test1 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0 [root@test1 /]# service ipvsadm save [root@test1 /]#
从上面的信息可见，lvs策略设置没有问题，但是有个注意点要说下，那就是lvs的持久性，如果你设置了持久连接，那么可能一段时间内访问的都是同一台服务器，所以在你测试的时候要特别注意。
到此，一台director上的配置就完成了。。。
主机2：test1.lxm.com[root@test2 /]# grep -i 'ip_vs' /boot/config-2.6.32-431.el6.x86_64 CONFIG_IP_VS=mCONFIG_IP_VS_IPV6=y# CONFIG_IP_VS_DEBUG is not setCONFIG_IP_VS_TAB_BITS=12CONFIG_IP_VS_PROTO_TCP=yCONFIG_IP_VS_PROTO_UDP=yCONFIG_IP_VS_PROTO_AH_ESP=yCONFIG_IP_VS_PROTO_ESP=yCONFIG_IP_VS_PROTO_AH=yCONFIG_IP_VS_PROTO_SCTP=yCONFIG_IP_VS_RR=mCONFIG_IP_VS_WRR=mCONFIG_IP_VS_LC=mCONFIG_IP_VS_WLC=mCONFIG_IP_VS_LBLC=mCONFIG_IP_VS_LBLCR=mCONFIG_IP_VS_DH=mCONFIG_IP_VS_SH=mCONFIG_IP_VS_SED=mCONFIG_IP_VS_NQ=mCONFIG_IP_VS_FTP=mCONFIG_IP_VS_PE_SIP=m[root@test2 /]# [root@test2 /]# rpm -qa | grep ipvsadm[root@test2 /]# yum -y install ipvsadmin[root@test2 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn[root@test2 /]#
配置VIP：[root@test2 /]# ifconfig eth0:0 10.0.10.200 netmask 255.255.255.0[root@test2 /]# ifconfig eth0:0eth0:0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.200 Bcast:10.0.10.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
配置LVS规则：[root@test2 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn[root@test2 /]# ipvsadm -A -t 10.0.10.200:80 -s rr[root@test2 /]# ipvsadm -a -t 10.0.10.200:80 -r 10.0.10.13:80 -g[root@test2 /]# ipvsadm -a -t 10.0.10.200:80 -r 10.0.10.14:80 -g[root@test2 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0 [root@test2 /]# service ipvsadm save[root@test2 /]#
到此，lvs的规则配置完成。。。。
2)配置后端realserver 后端两台realserver是：test3.lxm.com test4.lxm.com，其配置内容是一致的，以test3.lxm.com为例：主机；test3.lxm.com配置VIP：[root@test3 /]# ifconfig lolo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:39 errors:0 dropped:0 overruns:0 frame:0 TX packets:39 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3438 (3.3 KiB) TX bytes:3438 (3.3 KiB)
[root@test3 /]# ifconfig lo:0 10.0.10.200 netmask 255.255.255.255[root@test3 /]# ifconfig lo:0lo:0 Link encap:Local Loopback inet addr:10.0.10.200 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:16436 Metric:1
注意：这里为什么要配置在还回口上面呢？当然你愿意你也可以配置在网卡上，之所以选择还回口，是因为这是系统自带的，你网卡怎么坏，它都不会坏，除非你系统挂掉了。。 配置ARP规则：[root@test3 /]# sysctl -w net.ipv4.conf.all.arp_announce=2 >> /etc/sysctl.conf [root@test3 /]# sysctl -w net.ipv4.conf.all.arp_ignore=1 >> /etc/sysctl.conf[root@test3 /]# sysctl -pnet.ipv4.ip_forward = 0net.ipv4.conf.default.rp_filter = 1net.ipv4.conf.default.accept_source_route = 0kernel.sysrq = 0kernel.core_uses_pid = 1net.ipv4.tcp_syncookies = 1error: "net.bridge.bridge-nf-call-ip6tables" is an unknown keyerror: "net.bridge.bridge-nf-call-iptables" is an unknown keyerror: "net.bridge.bridge-nf-call-arptables" is an unknown keykernel.msgmnb = 65536kernel.msgmax = 65536kernel.shmmax = 68719476736kernel.shmall = 4294967296net.ipv4.conf.all.arp_announce = 2net.ipv4.conf.all.arp_ignore = 2net.ipv4.conf.all.arp_announce = 2net.ipv4.conf.all.arp_ignore = 1
检查：[root@test3 /]# cd /proc/sys/net/ipv4/conf/all/[root@test3 all]# lsaccept_local arp_announce bootp_relay forwarding promote_secondaries rp_filter src_valid_markaccept_redirects arp_filter disable_policy log_martians proxy_arp secure_redirects tagaccept_source_route arp_ignore disable_xfrm mc_forwarding proxy_arp_pvlan send_redirectsarp_accept arp_notify force_igmp_version medium_id route_localnet shared_media[root@test3 all]# cat arp_announce 2[root@test3 all]# cat arp_ignore 1
通过上面的设置，可见arp的内核参数，已经设置成功。。。
注:arp参数的含义：arp_ignore:定义接收到ARP请求时的响应级别； 0：默认行为，响应所有的地址ip和mac地址； 1：仅在请求的目标地址配置在请求到达的接口地址相匹配，才给予响应 在集群中选择1 arp_announce:定义将自己地址向外通告的通告级别； 0：将本地任何接口上的任何地址向外通告； 1：试图仅向目标网络通告与其网络匹配的地址； 2：仅向与本地接口上地址匹配的网络进行通告； 在集群中选择2
为什么要设置arp抑制呢？为了防止冲突，网络通信的的底层是通过mac地址通信，但是是通过arp协议来解析mac地址，如果后端服务器上不设置arp抑制，当有请求询问10.0.10.200的mac地址是什么啊，此时所以配置有10.0.10.200的主机都会响应请求，这不就乱套了啊。。 配置主机路由：[root@test3 /]# route add -host 10.0.10.200 dev lo:0[root@test3 /]# route -nKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface10.0.10.200 0.0.0.0 255.255.255.255 UH 0 0 0 lo10.0.10.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth00.0.0.0 10.0.10.254 0.0.0.0 UG 0 0 0 eth0[root@test3 /]#
有上面的信息可知，主机路由配置成功。。。。
注：为什么要设置主机路由呢？当一个客户端的请求发来时，目标地址是10.0.10.200，前端director收到请求后，会根据lvs规则转发到后端realserver服务器，后端realserver处理完请求后，就要发送响应包给客户端，由于本地配置有vip地址，所以会直接响应给客户端，但是此时响应的源地址是什么呢？因为客户端的目标地址是10.0.10.200，因此响应包的源地址必须是10.0.10.200，客户端才会接受。。这个时候主机路由就派上用场了，通过查看系统路由表发现10.0.10.200的目的地址通过lo:0口发出去，而此时lo:0口上的地址正式10.0.10.200，因此源地址就是10.0.10.200，这样到了客户端，就会被成功的接受下来。。
额，巴拉巴拉说了一堆，更详细的lvs知识，请看lvs专题吧。。。。累了。。。。
到此，一台realserver就配置完成了。。。。
注意；所有的realserver都要配置，而且配置一样，剩下的realserver的配置就不再巴拉巴拉了。。此外备用web也要跟realserver一样的配置。。
3)测试lvs架构 这里的测试要特别注意的是：一定要一台一台的测试lvs的架构，否则多台lvs director上都有VIP地址，都响应请求，那就完了。。你懂的。。而且在keepalived高可用中，也是一样，同一时间内只能有一台lvs director在线工作的。。
测试：test1.lxm.com(关闭test2.lxm.com这台director，只要取消VIP地址即可)为了让大家看测试效果，这里选用在备用的web上进行测试，你可以在浏览器中测试：[root@test1 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0
[root@test5 /]# links --dump
this is test3.lxm.com[root@test5 /]# links --dump this is test4.lxm.com[root@test5 /]# links --dump this is test3.lxm.com[root@test5 /]# links --dump this is test4.lxm.com[root@test5 /]# links --dump this is test3.lxm.com[root@test5 /]# links --dump this is test4.lxm.com[root@test5 /]#//到这里，会不会有人想，你这里都是文字，是不是你手写的啊。。额，你要这么想，你看着办。。。。
[root@test1 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 3 -> 10.0.10.14:80 Route 1 0 3 [root@test1 /]#
由上面的信息可见，我在客户端刷了6次，而director上查看，平均分配了。。。
关于test2.lxm.com的测试，这里不在展示，请你自行测试。。。。 最后，关闭director上的ipvsadm并不允许开机自启动： //这一点特别重要，在keepalived高可用中，lvs的规则是由keepalived来进行管理的。。以上所有的步骤只是为了验证系统运行LVS环境是否有问题。。。[root@test2 /]# ifconfig eth0:0 down[root@test2 /]# service ipvsadm stopipvsadm: Clearing the current IPVS table: [ OK ]ipvsadm: Unloading modules: [ OK ][root@test2 /]# chkconfig ipvsadm off[root@test2 /]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn[root@test2 /]# 上面的信息显示的是test2.lxm.com主机的，对于test1.lxm.com主机也是一样，要关闭ipvsadm，并取消VIP。。。 到此，lvs的架构就测试完成了。。。你还有什么问题吗？ 4.部署keepalived keepalived部署是和lvs的director在同一台主机上的,因此test1.lxm.com test2.lxm.com两台主机都要安装keepalvied软件。。。1)安装keepalived软件 keepalived的安装比较简单，因为其是一个轻量级的高可用工具，但是也有需要注意点的地方，就是其在编译安装的时候需要使用的内核源码的头文件。#yum -y install kernel-devel kernel-headers libnl-devel#cd /root/soft#tar -zxvf keepalived-1.2.7.tar.gz#cd keepalived-1.2.7#mkdir /usr/local/keepalived#./configure --prefix=/usr/local/keepalived --mandir=/usr/local/share/man --with-kernel-dir=/usr/src/kernels/2.6.32-279.el6.x86_64 configure完成后，会出现下面的信息：Keepalived configuration------------------------Keepalived version : 1.2.7Compiler : gccCompiler flags : -g -O2Extra Lib : -lpopt -lssl -lcrypto -lnlUse IPVS Framework : YesIPVS sync daemon support : YesIPVS use libnl : YesUse VRRP Framework : YesUse VRRP VMAC : YesSNMP support : NoUse Debug flags : No注：上面的信息就是当前keepalived所支持的功能，其中VRRP是keepalived的核心功能，这个是一定包含在内的，但是IPVS模块却是可选的，如果你要支持LVS，这里就必须为yes。。。
#make #make install#make clean
如果没什么错误，基本上keepalived编译安装算是完成了。。。
后续操作：#cd /usr/local/keepalived#cp -p etc/rc.d/init.d/keepalived /etc/rc.d/init.d#cp -p etc/sysconfig/keepalived /etc/sysconfig/
#vim /etc/profileexport PATH=$PATH:/usr/local/keepalived/sbin/keepalived
#vim /etc/rc.d/init.d/keepalived修改这个脚步文件，将可执行程序和配置文件改为正确的路径keepalivebin=${keepalivebin:-/usr/local/keepalived/sbin/keepalived}config=${config:-/usr/local/keepalived/etc/keepalived/keepalived.conf}
启动keepalived：[root@test2 init.d]# service keepalived startStarting keepalived: [ OK ][root@test2 init.d]# ps aux | grep keepalivedroot 3056 0.0 0.3 42140 976 ? Ss 15:24 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.confroot 3058 0.5 0.9 44376 2292 ? S 15:24 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.confroot 3059 0.2 0.6 44244 1628 ? S 15:24 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.conf由上面的信息可见，keepalived安装成功，并且能成功启动，但此时如果你查看日志:tail -f /var/log/messages 会发现不停报巴拉巴拉的错误，不要管他 ，暂时跟你没关系
到此，keepalived安装就算成功了。。。同理，安装另一台keepalived主机。。
2)配置文件keepalived.conf主机：test1.lxm.com:! Configuration File for keepalivedglobal_defs { notification_email {
} notification_email_from smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL}vrrp_sync_group VG_1 { group { VI_1 }}
vrrp_instance VI_1 { state BACKUP interface eth1 virtual_router_id 51 priority 100 advert_int 1 nopreempt authentication { auth_type PASS auth_pass keepalivepass } virtual_ipaddress { 10.0.10.200/24 dev eth0 label eth0:0 }}
virtual_server 10.0.10.200 80 { delay_loop 3 lb_algo rr lb_kind DR nat_mask 255.255.255.0# persistence_timeout 50 protocol TCP
real_server 10.0.10.13 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_port 80 connect_timeout 1 nb_get_retry 3 delay_before_retry 2 }} real_server 10.0.10.14 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_port 80 connect_timeout 1 nb_get_retry 3 delay_before_retry 2 }}}
主机；test2.lxm.com:! Configuration File for keepalivedglobal_defs { notification_email { } notification_email_from smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL}
vrrp_sync_group VG_1 { group { VI_1 }}
vrrp_instance VI_1 { state BACKUP interface eth1 virtual_router_id 51 priority 99 nopreempt advert_int 1 authentication { auth_type PASS auth_pass keepalivepass } virtual_ipaddress { 10.0.10.200/24 dev eth0 label eth0:0 }}
virtual_server 10.0.10.200 80 { delay_loop 3 lb_algo rr lb_kind DR nat_mask 255.255.255.0# persistence_timeout 50 protocol TCP
real_server 10.0.10.13 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_port 80 connect_timeout 1 nb_get_retry 3 delay_before_retry 2 }} real_server 10.0.10.14 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_port 80 connect_timeout 1 nb_get_retry 3 delay_before_retry 2 }}}
这边配置文件，我就不解释了，方便你直接贴。。。解释会专门写一篇博文。。。。3)测试keepalived是否能成功启动，并管理lvs和VIP资源[root@test1 keepalived]# ps aux | grep keepalivedroot 3696 0.0 0.3 42140 976 ? Ss 15:57 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.confroot 3698 0.0 0.9 46440 2336 ? S 15:57 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.confroot 3699 0.0 0.6 46316 1684 ? S 15:57 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.confroot 3701 0.0 0.3 103244 844 pts/0 S+ 16:00 0:00 grep keepalived[root@test1 keepalived]#
[root@test2 keepalived]# service keepalived startStarting keepalived: [ OK ][root@test2 keepalived]# ps aux | grep keepalivedroot 3220 0.0 0.3 42140 976 ? Ss 15:58 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.confroot 3222 0.0 0.9 44368 2296 ? S 15:58 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.confroot 3223 0.0 0.6 44244 1636 ? S 15:58 0:00 /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.confroot 3225 0.0 0.3 103244 844 pts/0 S+ 16:00 0:00 grep keepalived[root@test2 keepalived]#
以上信息显示，keepalived在两台主机上都已经启动了。。
注意；一定要注意的我的主机名的变化啊
查看日志：主机test1.lxm.com[root@test1 log]# tail -f /var/log/messages
Sep 2 15:57:58 test1 Keepalived[3695]: Starting Keepalived v1.2.7 (09/02,2014)Sep 2 15:57:58 test1 Keepalived[3696]: Starting Healthcheck child process, pid=3698Sep 2 15:57:58 test1 Keepalived[3696]: Starting VRRP child process, pid=3699Sep 2 15:57:58 test1 Keepalived_vrrp[3699]: Interface queue is emptySep 2 15:57:58 test1 Keepalived_vrrp[3699]: No such interface, eth1Sep 2 15:57:58 test1 Keepalived_vrrp[3699]: Netlink reflector reports IP 10.0.10.11 addedSep 2 15:57:58 test1 Keepalived_vrrp[3699]: Netlink reflector reports IP 10.0.0.16 added //这里提示检测到了系统的两个网卡的ip地址Sep 2 15:57:58 test1 Keepalived_vrrp[3699]: Netlink reflector reports IP fe80::a00:27ff:feed:ef33 addedSep 2 15:57:58 test1 Keepalived_vrrp[3699]: Netlink reflector reports IP fe80::a00:27ff:fe36:2415 addedSep 2 15:57:58 test1 Keepalived_vrrp[3699]: Registering Kernel netlink reflectorSep 2 15:57:58 test1 Keepalived_vrrp[3699]: Registering Kernel netlink command channelSep 2 15:57:58 test1 Keepalived_vrrp[3699]: Registering gratuitous ARP shared channelSep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Interface queue is emptySep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: No such interface, eth1Sep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Netlink reflector reports IP 10.0.10.11 addedSep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Netlink reflector reports IP 10.0.0.16 addedSep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Netlink reflector reports IP fe80::a00:27ff:feed:ef33 addedSep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Netlink reflector reports IP fe80::a00:27ff:fe36:2415 addedSep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Registering Kernel netlink reflectorSep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Registering Kernel netlink command channelSep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Opening file '/usr/local/keepalived/etc/keepalived/keepalived.conf'.Sep 2 15:57:58 test1 Keepalived_vrrp[3699]: Opening file '/usr/local/keepalived/etc/keepalived/keepalived.conf'.Sep 2 15:57:58 test1 Keepalived_vrrp[3699]: Truncating auth_pass to 8 charactersSep 2 15:57:58 test1 Keepalived_vrrp[3699]: Configuration is using : 65373 BytesSep 2 15:57:58 test1 Keepalived_vrrp[3699]: Using LinkWatch kernel netlink reflector...Sep 2 15:57:58 test1 Keepalived_vrrp[3699]: VRRP sockpool: [ifindex(3), proto(112), fd(10,11)]Sep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Configuration is using : 16462 BytesSep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Using LinkWatch kernel netlink reflector...Sep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Activating healthchecker for service [10.0.10.13]:80Sep 2 15:57:58 test1 Keepalived_healthcheckers[3698]: Activating healthchecker for service [10.0.10.14]:80 //这个地方显示对后端服务器做检测Sep 2 15:57:59 test1 Keepalived_vrrp[3699]: VRRP_Instance(VI_1) Transition to MASTER STATE Sep 2 15:58:00 test1 Keepalived_vrrp[3699]: VRRP_Instance(VI_1) Entering MASTER STATE //这里显示该主机被决策为masterSep 2 15:58:00 test1 Keepalived_vrrp[3699]: VRRP_Instance(VI_1) setting protocol VIPs.Sep 2 15:58:00 test1 Keepalived_vrrp[3699]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200Sep 2 15:58:00 test1 Keepalived_vrrp[3699]: VRRP_Group(VG_1) Syncing instances to MASTER state Sep 2 15:58:00 test1 Keepalived_healthcheckers[3698]: Netlink reflector reports IP 10.0.10.200 added //到这里显示，想对方主机同步master状态，并添加了VIPSep 2 15:58:05 test1 Keepalived_vrrp[3699]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200
主机:test2.lxm.com[root@test2 ~]# tail -f /var/log/messagesSep 2 15:58:03 test2 Keepalived[3219]: Starting Keepalived v1.2.7 (09/02,2014)Sep 2 15:58:03 test2 Keepalived[3220]: Starting Healthcheck child process, pid=3222Sep 2 15:58:03 test2 Keepalived[3220]: Starting VRRP child process, pid=3223Sep 2 15:58:03 test2 Keepalived_vrrp[3223]: Interface queue is emptySep 2 15:58:03 test2 Keepalived_vrrp[3223]: No such interface, eth1Sep 2 15:58:03 test2 Keepalived_vrrp[3223]: Netlink reflector reports IP 10.0.10.12 addedSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Netlink reflector reports IP 10.0.0.18 addedSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Netlink reflector reports IP fe80::a00:27ff:fe0d:26b8 addedSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Netlink reflector reports IP fe80::a00:27ff:fe35:e1f4 addedSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Registering Kernel netlink reflectorSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Registering Kernel netlink command channelSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Registering gratuitous ARP shared channelSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Opening file '/usr/local/keepalived/etc/keepalived/keepalived.conf'.Sep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Interface queue is emptySep 2 15:58:03 test2 Keepalived_vrrp[3223]: Truncating auth_pass to 8 charactersSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Configuration is using : 65388 BytesSep 2 15:58:03 test2 Keepalived_vrrp[3223]: Using LinkWatch kernel netlink reflector...Sep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: No such interface, eth1Sep 2 15:58:03 test2 Keepalived_vrrp[3223]: VRRP_Instance(VI_1) Entering BACKUP STATE //成为了backupSep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Netlink reflector reports IP 10.0.10.12 addedSep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Netlink reflector reports IP 10.0.0.18 addedSep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Netlink reflector reports IP fe80::a00:27ff:fe0d:26b8 addedSep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Netlink reflector reports IP fe80::a00:27ff:fe35:e1f4 addedSep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Registering Kernel netlink reflectorSep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Registering Kernel netlink command channelSep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Opening file '/usr/local/keepalived/etc/keepalived/keepalived.conf'.Sep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Configuration is using : 16477 BytesSep 2 15:58:03 test2 Keepalived_vrrp[3223]: VRRP sockpool: [ifindex(3), proto(112), fd(10,11)]Sep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Using LinkWatch kernel netlink reflector...Sep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Activating healthchecker for service [10.0.10.13]:80Sep 2 15:58:03 test2 Keepalived_healthcheckers[3222]: Activating healthchecker for service [10.0.10.14]:80
通过上面的信息可以看到，keepalived之间的心跳信息已经成功协商，并通过优先级的高低选出了主备。接下来就要验证是否能够驱动资源。在主设备应该能看到VIP和LVS的规则信息，而在从设备上应该只能看到LVS的规则信息，而没有VIPmaster:test1.lxm.com:[root@test1 keepalived]# ifconfig eth0:0eth0:0 Link encap:Ethernet HWaddr 08:00:27:ED:EF:33 inet addr:10.0.10.200 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
[root@test1 keepalived]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0 [root@test1 keepalived]#
backup:test2.lxm.com[root@test2 keepalived]# ifconfig eth0:0eth0:0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1[root@test2 keepalived]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0 [root@test2 keepalived]#
看到没，通过上面的信息显示，验证了之前的猜想。。。
上面的日志信息还显示，keepalived对后端的服务器发送了检测信息，这是因为在keepalived配置文件中，在lvs配置段，配置了检测功能，此时查看后端web服务器的访问日志，验证是否有检测信息：[root@test4 all]# cd /etc/httpd/logs/[root@test4 logs]# lsaccess_log error_log[root@test4 logs]# tail -f access_log 10.0.10.11 - - [02/Sep/2014:16:12:05 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.12 - - [02/Sep/2014:16:12:05 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.11 - - [02/Sep/2014:16:12:10 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.12 - - [02/Sep/2014:16:12:10 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.11 - - [02/Sep/2014:16:12:15 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.12 - - [02/Sep/2014:16:12:15 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.11 - - [02/Sep/2014:16:12:20 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.12 - - [02/Sep/2014:16:12:20 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.11 - - [02/Sep/2014:16:12:25 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.12 - - [02/Sep/2014:16:12:25 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.11 - - [02/Sep/2014:16:12:30 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"10.0.10.12 - - [02/Sep/2014:16:12:30 +0800] "GET / HTTP/1.0" 200 31 "-" "KeepAliveClient"通过上面的信息，验证了keepalived检测后端服务器是否在线的功能已经启用了。。你会发现这样的日志信息不停的在刷。。这个跟你设置的检测策略有关。。。
到此，关于keepalived的安装，配置，启动就成功了，初步说明keepalived+lvs架构已经搭建起来，至于效果怎么样，有待于后面的测试。。。
5.keepalived+lvs全面测试1)基于上面完成的环境，测试keepalived+lvs能够提供web访问[root@test5 /]# links --dump
this is test4.lxm.com[root@test5 /]# links --dump this is test3.lxm.com[root@test5 /]# links --dump this is test4.lxm.com[root@test5 /]# links --dump this is test3.lxm.com[root@test5 /]# links --dump this is test4.lxm.com[root@test5 /]# links --dump this is test3.lxm.com[root@test5 /]#[root@test1 keepalived]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 3 -> 10.0.10.14:80 Route 1 0 3 [root@test1 keepalived]#
由上面的测试信息可知，访问没有任何问题。。。因此，就之前搭建起来的环境是没有任何问题的。。。
2)测试keepalived自动切换 keepalived的切换主要分为三种；第一种：keepalived服务挂了 基于上面的环境，现在的master是test1.lxm.com，backup是test2.lxm.com，现在模拟test1.lxm.com上的keepalived服务挂了[root@test1 keepalived]# service keepalived stop //关闭了test1的keepalived服务，模拟keepalived挂了。。Stopping keepalived: [ OK ][root@test1 keepalived]#查看日志：Sep 2 16:27:26 test1 Keepalived[3696]: Stopping Keepalived v1.2.7 (09/02,2014)Sep 2 16:27:26 test1 Keepalived_vrrp[3699]: VRRP_Instance(VI_1) sending 0 prioritySep 2 16:27:26 test1 kernel: IPVS: __ip_vs_del_service: enterSep 2 16:27:26 test1 Keepalived_vrrp[3699]: VRRP_Instance(VI_1) removing protocol VIPs.
上面显示keepalived服务停止了，移除了VIP
查看资源：[root@test1 keepalived]# ifconfig eth0 Link encap:Ethernet HWaddr 08:00:27:ED:EF:33 inet addr:10.0.10.11 Bcast:10.0.10.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:feed:ef33/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:13896 errors:0 dropped:0 overruns:0 frame:0 TX packets:13783 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1768054 (1.6 MiB) TX bytes:1575730 (1.5 MiB)
eth1 Link encap:Ethernet HWaddr 08:00:27:36:24:15 inet addr:10.0.0.16 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe36:2415/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5 errors:0 dropped:0 overruns:0 frame:0 TX packets:2174 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:300 (300.0 b) TX bytes:117492 (114.7 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:348 errors:0 dropped:0 overruns:0 frame:0 TX packets:348 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:27544 (26.8 KiB) TX bytes:27544 (26.8 KiB)
[root@test1 keepalived]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn[root@test1 keepalived]#
从上面的信息，可见VIP没了，ipvsadm的规则也被清空了。。
此时查看test2.lxm.com主机：Sep 2 16:27:25 test2 Keepalived_vrrp[3223]: VRRP_Instance(VI_1) Transition to MASTER STATESep 2 16:27:25 test2 Keepalived_vrrp[3223]: VRRP_Group(VG_1) Syncing instances to MASTER stateSep 2 16:27:26 test2 Keepalived_vrrp[3223]: VRRP_Instance(VI_1) Entering MASTER STATESep 2 16:27:26 test2 Keepalived_vrrp[3223]: VRRP_Instance(VI_1) setting protocol VIPs.Sep 2 16:27:26 test2 Keepalived_vrrp[3223]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200Sep 2 16:27:26 test2 Keepalived_healthcheckers[3222]: Netlink reflector reports IP 10.0.10.200 addedSep 2 16:27:31 test2 Keepalived_vrrp[3223]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200
由上面的信息可知，原来的backup主机已经切换到master主机了
查看资源：[root@test2 keepalived]# ifconfig eth0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.12 Bcast:10.0.10.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe0d:26b8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10498 errors:0 dropped:0 overruns:0 frame:0 TX packets:11324 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1449942 (1.3 MiB) TX bytes:1171364 (1.1 MiB)
eth0:0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.200 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 08:00:27:35:E1:F4 inet addr:10.0.0.18 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe35:e1f4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2131 errors:0 dropped:0 overruns:0 frame:0 TX packets:239 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:127860 (124.8 KiB) TX bytes:13002 (12.6 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:87 errors:0 dropped:0 overruns:0 frame:0 TX packets:87 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:7300 (7.1 KiB) TX bytes:7300 (7.1 KiB) 看到没，多了个VIP。。。好，切换成功
此时在使用客户端访问测试：[root@test5 /]# dateTue Sep 2 16:31:37 CST 2014[root@test5 /]# links --dump
this is test4.lxm.com[root@test5 /]# links --dump this is test3.lxm.com[root@test5 /]# links --dump this is test4.lxm.com[root@test5 /]# links --dump this is test3.lxm.com[root@test5 /]# links --dump this is test4.lxm.com[root@test5 /]# links --dump this is test3.lxm.com[root@test5 /]#[root@test2 keepalived]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 3 -> 10.0.10.14:80 Route 1 0 3 通过上面的信息，访问没有任何影响，从这个测试看出了，keepalived在自身服务挂掉的情况下，可以正常切换。且也验证了，使用另一台keepalived+lvs主机，访问也是正常的。。
注意：这个时候可能有人有疑问了？如果我挂掉的keepalived主机重新上线了，会不会再次变为主设备，因为它的优先级高。我想说可以，只要你设置了抢占规则，但是在线上环境不建议设置抢占，就算时间很短也会有抖动。。默认情况下如果没有设置nopreempt，会根据优先级自动进行抢占的。。。而我上面的配置文件中设置了nopreempt，因此我这里是不会抢占的。
第二种：ipvsadm挂了
先看一下在没有做任何措施的情况下，停止ipvsadm服务是否会切换：基于此前的环境，这里的master是test2.lxm.com,因此在该主机上检测：[root@test2 keepalived]# service ipvsadm stopipvsadm: Clearing the current IPVS table: [ OK ]ipvsadm: Unloading modules: [ OK ]
查看日志：Sep 2 16:53:55 test2 kernel: IPVS: __ip_vs_del_service: enterSep 2 16:53:55 test2 kernel: IPVS: [rr] scheduler unregistered.Sep 2 16:53:55 test2 kernel: IPVS: ipvs unloaded.
从日志看出，就报了个IPVS 调度器为注册，ipvs模块卸载了或者未加载。。其他啥也没了。此时查看一下VIP信息：[root@test2 keepalived]# ifconfigeth0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.12 Bcast:10.0.10.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe0d:26b8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:13432 errors:0 dropped:0 overruns:0 frame:0 TX packets:14776 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1831388 (1.7 MiB) TX bytes:1499838 (1.4 MiB)
eth0:0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.200 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 08:00:27:35:E1:F4 inet addr:10.0.0.18 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe35:e1f4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2716 errors:0 dropped:0 overruns:0 frame:0 TX packets:1065 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:162960 (159.1 KiB) TX bytes:57606 (56.2 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:99 errors:0 dropped:0 overruns:0 frame:0 TX packets:99 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:8300 (8.1 KiB) TX bytes:8300 (8.1 KiB)
[root@test2 keepalived]#
从上面信息看出，VIP信息还在，由此可判断出，并没有切换。。。好，那该怎么办呢?
这个时候就需要keepalived一种特殊的功能：嵌套脚本
此时修改keepalived.conf配置文件，内容如下：! Configuration File for keepalivedglobal_defs { notification_email {
} notification_email_from smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL}vrrp_sync_group VG_1 { group { VI_1 }}
vrrp_script check_lvs { //这段内容就是添加的内容，用来导入外部的脚本 script "/usr/local/keepalived/etc/keepalived/lvs_check.sh" //该选项就是指定外部脚步的位置 interval 1 //没间隔一秒钟执行脚步一次 weight -10 //如果检测失败，则降低本主机keepalived的优先级 fall 1 // 检测失败一次就失败，生产中不建议这样，建议3次左右 rise 1 //检测一次成功，就表示成功。。。}
vrrp_instance VI_1 { state BACKUP interface eth1 virtual_router_id 51 priority 100 advert_int 1 nopreempt authentication { auth_type PASS auth_pass keepalivepass } virtual_ipaddress { 10.0.10.200/24 dev eth0 label eth0:0 } track_script { //这段内容也是添加的，必须要和vrrp_script联合使用，这表示调用vrrp_script定义的脚本。 check_lvs} }
virtual_server 10.0.10.200 80 { delay_loop 3 lb_algo rr lb_kind DR nat_mask 255.255.255.0# persistence_timeout 50 protocol TCP
real_server 10.0.10.13 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_port 80 connect_timeout 1 nb_get_retry 3 delay_before_retry 2 }} real_server 10.0.10.14 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_port 80 connect_timeout 1 nb_get_retry 3 delay_before_retry 2 }}}
注意：修改keepalived.conf配置文件，必须要相应的同步所有的keepalived主机，你不可能希望一台具有某个功能，另一台没有吧，但是要注意，主备的不同设置和相同设置创建/usr/local/keepalived/etc/keepalived/lvs_check.sh脚步，一个简单的脚步内容如下：#!/bin/bashfunction mailSend() { echo "ipvsadm service is down" | mail -s "ipvsadm service is down"
}num=`ipvsadm --list -n | grep 10.0.10.200| wc -l`[ $num -eq 0 ] && mailSend && exit 1 || exit 0
注：我这个脚步主要是帮助大家测试一下，调用外部脚本来检测服务，达到keepalived切换的目的。其次在该脚步中，当lvs的规则都没有时，此时就考虑ipvsadm挂了，需要切换。那有人就会问了，假如还有lvs规则，但是踢掉了几个规则，怎么办？那我想估计有两种情况，一是你的服务器太脆弱，别人随时能上去玩玩或者你自己踢了玩，二就是后端web服务器有些主机故障了，lvs踢掉了一些，但这个时候即使你切换了，还不是一样的效果哦。。
修改了配置文件，创建的脚本并赋予执行权限，重启服务，再次测试：注意：我这里还保持主为test2.lxm.com （亲，请不要再问我怎么保持了，如果你是根据我上面做的来，那么此时主默认就是test2.lxm.com,否则根据自己的情况实验即可)[root@test2 keepalived]# service ipvsadm stopipvsadm: Clearing the current IPVS table: [ OK ]ipvsadm: Unloading modules: [ OK ][root@test2 keepalived]# 查看test2.lxm.com的日志：Sep 5 13:52:21 test2 kernel: IPVS: __ip_vs_del_service: enterSep 5 13:52:21 test2 kernel: IPVS: [rr] scheduler unregistered.Sep 5 13:52:21 test2 kernel: IPVS: ipvs unloaded.发现日志信息中还是多了这么三行信息，其他没任何反应，怎么回事？按道理来说如果发生了切换，日志中肯定会报移除了虚拟ip地址，但是这里没有，因此可以猜测VIP还在。。(这里声明下，有人可能眼睛厉害，看到我日志的时间好像和前面不一样啊，哈，亲，因为实验不是一天测完的，本人有工作)
查看VIP地址：[root@test2 keepalived]# ifconfigeth0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.12 Bcast:10.0.10.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe0d:26b8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:13438 errors:0 dropped:0 overruns:0 frame:0 TX packets:15234 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1632335 (1.5 MiB) TX bytes:1576327 (1.5 MiB)
eth0:0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.200 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 08:00:27:35:E1:F4 inet addr:10.0.0.18 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe35:e1f4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:917 errors:0 dropped:0 overruns:0 frame:0 TX packets:2562 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:55020 (53.7 KiB) TX bytes:138084 (134.8 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:341 errors:0 dropped:0 overruns:0 frame:0 TX packets:341 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:24801 (24.2 KiB) TX bytes:24801 (24.2 KiB)
从上面的信息可见，VIP还在，因此并没有切换成功，根据日志的信息来看，没有报任何跟vrrp_script有关的信息，这是什么原因？傻眼了。。。其实这个错误是我故意展示的，这里是为了给你加深映像，如果不给你点出来，你可能感觉呀，以前实验好像成功啊，现在咋了？大部分人可能折腾几个小时，甚至更长都搞不明白。。。说实话我也是折腾了好久，查来查去无果，最后不得已扒日志，一条一条看，发现了这么一句话： Keepalived_vrrp[14961]: VRRP_Instance(VI_1) : ignoring tracked script with weights due to SYNC group 上面这句话的意思：由于sync group的原因，忽略了带有权值的跟踪脚本。。意思就是track_script这个设置根本没起作用。回过头来看看配置文件，果然有这么一段配置： vrrp_sync_group VG_1 {
group { VI_1 }
}
好，既然找到了原因，那么接下来，就要测试是否是这个原因，注释掉这段配置,然后再次启动keepalived:这个时候，查看日志，发现没有了那句话，而且出现了下面一句话：Sep 5 15:00:54 test2 Keepalived_vrrp[2074]: VRRP_Script(check_lvs) succeeded
由此说明，脚本调用已经成功了。
测试；[root@test2 keepalived]# service ipvsadm stopipvsadm: Clearing the current IPVS table: [ OK ]ipvsadm: Unloading modules: [ OK ][root@test2 keepalived]#
看日志：Sep 5 15:02:37 test2 kernel: IPVS: __ip_vs_del_service: enterSep 5 15:02:37 test2 kernel: IPVS: [rr] scheduler unregistered.Sep 5 15:02:37 test2 kernel: IPVS: ipvs unloaded.Sep 5 15:02:37 test2 kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)Sep 5 15:02:37 test2 kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)Sep 5 15:02:37 test2 kernel: IPVS: ipvs loaded.Sep 5 15:02:37 test2 Keepalived_vrrp[2074]: VRRP_Script(check_lvs) failed
从上面的信息看到，脚本已经检测到lvs的规则被刷掉，且返回的结果是失败，这就表示脚本成功返回了值1.但是还是没有移除虚拟ip的信息，这是为什么？
分析： 这个时候就要分析了，当前所在的主机是test2.lxm.com，本来是作为备用主机的，其优先级比较低，但是master发生了故障，切换到了该主机，因此该主机变成了master。当此前master主机，即test1.lxm.com恢复时，因为设置了nopreempt,因此不会抢占回去。那如果这个时候，test2主机的ipvsadm挂了，规则没了，但是keepalived的心跳还在，keepalived服务并没有挂掉，此时还是可以和tes1主机进行心跳沟通，这个时候发现test1是不抢占的机制，本来test2的优先级就比test1低，此时检测到故障再次降低优先级是一个效果，因此，此时并不会进行主备切换。这个时候，如果想发生错误进行切换，就要修改脚本test2.lxm.com上脚本的内容。修改脚本如下：
#!/bin/bashfunction mailSend() { echo "ipvsadm service is down" | mail -s "ipvsadm service is down"
}num=`ipvsadm --list -n | grep 10.0.10.200| wc -l`[ $num -eq 0 ] && mailSend && service keepalived stop || exit 0 //如果检测到失败，直接停止keepalived服务
测试；[root@test2 keepalived]# service ipvsadm stopipvsadm: Clearing the current IPVS table: [ OK ]ipvsadm: Unloading modules: [ OK ]看日志：Sep 5 15:14:58 test2 kernel: IPVS: __ip_vs_del_service: enterSep 5 15:14:58 test2 kernel: IPVS: [rr] scheduler unregistered.Sep 5 15:14:58 test2 kernel: IPVS: ipvs unloaded.Sep 5 15:14:58 test2 kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)Sep 5 15:14:58 test2 kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)Sep 5 15:14:58 test2 kernel: IPVS: ipvs loaded.Sep 5 15:14:58 test2 Keepalived[7365]: Stopping Keepalived v1.2.7 (09/05,2014)Sep 5 15:14:58 test2 Keepalived_vrrp[7368]: VRRP_Instance(VI_1) sending 0 prioritySep 5 15:14:58 test2 Keepalived_vrrp[7368]: VRRP_Instance(VI_1) removing protocol VIPs.Sep 5 15:14:58 test2 Keepalived_healthcheckers[7367]: Netlink reflector reports IP 10.0.10.200 removedSep 5 15:14:58 test2 Keepalived_healthcheckers[7367]: IPVS: No such destinationSep 5 15:14:58 test2 Keepalived_healthcheckers[7367]: IPVS: Service not definedSep 5 15:14:58 test2 Keepalived_healthcheckers[7367]: IPVS: No such service
从日志上看到，keepalived的停止了，VIP被移除了。。。
[root@test2 keepalived]# ifconfigeth0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.12 Bcast:10.0.10.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe0d:26b8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:25996 errors:0 dropped:0 overruns:0 frame:0 TX packets:29273 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3154978 (3.0 MiB) TX bytes:3116748 (2.9 MiB)
eth1 Link encap:Ethernet HWaddr 08:00:27:35:E1:F4 inet addr:10.0.0.18 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe35:e1f4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2282 errors:0 dropped:0 overruns:0 frame:0 TX packets:5657 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:136920 (133.7 KiB) TX bytes:305214 (298.0 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:3681 errors:0 dropped:0 overruns:0 frame:0 TX packets:3681 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:204929 (200.1 KiB) TX bytes:204929 (200.1 KiB)
这个时候，看到VIP资源没了，查看test1的信息，日志中你会看到切换成了master，VIP信息也添加完成。。。。到这里，有人肯定会想了？刚刚你在test2上的时候需要修改脚本才行，那test1是不是也要修改？这里的答案是不需要，因为test2是抢占的机制，一旦test1的优先级降低，test2就会立刻抢占过去：测试：test1上的脚本内容：#!/bin/bashfunction mailSend() { echo "ipvsadm service is down" | mail -s "ipvsadm service is down"
}num=`ipvsadm --list -n | grep 10.0.10.200| wc -l`[ $num -eq 0 ] && mailSend && exit 1 || exit 0 [root@test1 keepalived]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0 You have new mail in /var/spool/mail/root[root@test1 keepalived]# service ipvsadm stopipvsadm: Clearing the current IPVS table: [ OK ]ipvsadm: Unloading modules: [ OK ]
查看日志：
Sep 5 15:15:34 test1 kernel: IPVS: __ip_vs_del_service: enterSep 5 15:15:34 test1 kernel: IPVS: [rr] scheduler unregistered.Sep 5 15:15:34 test1 kernel: IPVS: ipvs unloaded.Sep 5 15:15:34 test1 kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)Sep 5 15:15:34 test1 kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)Sep 5 15:15:34 test1 kernel: IPVS: ipvs loaded.Sep 5 15:15:34 test1 Keepalived_vrrp[5788]: VRRP_Script(check_lvs) failedSep 5 15:15:36 test1 Keepalived_vrrp[5788]: VRRP_Instance(VI_1) Received higher prio advertSep 5 15:15:36 test1 Keepalived_vrrp[5788]: VRRP_Instance(VI_1) Entering BACKUP STATESep 5 15:15:36 test1 Keepalived_vrrp[5788]: VRRP_Instance(VI_1) removing protocol VIPs.Sep 5 15:15:36 test1 Keepalived_healthcheckers[5787]: Netlink reflector reports IP 10.0.10.200 removed
从上面的信息看到没，日志报的和刚刚test2不一样，当脚本检测到失败时，立即降低了自身了优先级，然后提示收到高优先级通告，然后转换到backup状态，移除了VIP。。 好，到这里，关于用脚本检测第三方服务成功与否实现VIP的切换就成功了。。。。 不过，这里还残留一个问题，估计你也想到了，就是vrrp_sync_group和vrrp_script同时使用的问题，从上面的讨论来看，好像是有冲突的概念。但是万一生产中，就是要同时使用怎么办呢？经过我的测试，如果你想在不注释vrrp_sync_group的情况下，使用vrrp_script的话，那就要修改track_script的内容如下：
track_script { check_lvs weight 0 } 从上面看出，就是要在脚本名后面 明确的加上weight 0 字段... 关于这个，自行测试吧。我测试是通过。。。 第三种：网卡通信故障 在keepalived中，还可以对网卡故障进行检测，一旦检测到对外通信的网卡发生了故障，就可以进行VIP的切换。在keepalived中有两种方式来检测网卡，一种是向上面一样使用外部脚本的的方式，另一种就是使用keepalived自身的track_interface检测
测试验证： 测试中的master还是test2.lxm.com(一直在这上面测的原因是其优先级低，如果优先级低的都能正常切换，那么优先级高的就没有问题)[root@test2 keepalived]#ifdown eth0 down 日志：Sep 5 15:38:36 test2 Keepalived_vrrp[14989]: VRRP_Instance(VI_1) Received higher prio advertSep 5 15:38:36 test2 Keepalived_vrrp[14989]: VRRP_Instance(VI_1) Entering BACKUP STATESep 5 15:38:36 test2 Keepalived_vrrp[14989]: VRRP_Instance(VI_1) removing protocol VIPs.Sep 5 15:38:36 test2 Keepalived_healthcheckers[14988]: Netlink reflector reports IP 10.0.10.200 removed
查看test1:Sep 5 15:35:55 test1 Keepalived_vrrp[19581]: VRRP_Instance(VI_1) Transition to MASTER STATESep 5 15:35:56 test1 Keepalived_vrrp[19581]: VRRP_Instance(VI_1) Entering MASTER STATESep 5 15:35:56 test1 Keepalived_vrrp[19581]: VRRP_Instance(VI_1) setting protocol VIPs.Sep 5 15:35:56 test1 Keepalived_vrrp[19581]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200Sep 5 15:35:56 test1 Keepalived_healthcheckers[19580]: Netlink reflector reports IP 10.0.10.200 added
可见test1变成了master，此时在将test2的网卡重新上线,发现test1的日志：Sep 5 15:38:34 test1 Keepalived_vrrp[19581]: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 5 15:38:34 test1 Keepalived_vrrp[19581]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200Sep 5 15:38:35 test1 Keepalived_vrrp[19581]: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 5 15:38:35 test1 Keepalived_vrrp[19581]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200
说明其心跳通信正常。。
再次测试，关掉test1的网卡，看test2的日志：
Sep 5 15:42:45 test2 Keepalived_vrrp[14989]: VRRP_Instance(VI_1) Transition to MASTER STATESep 5 15:42:46 test2 Keepalived_vrrp[14989]: VRRP_Instance(VI_1) Entering MASTER STATESep 5 15:42:46 test2 Keepalived_vrrp[14989]: VRRP_Instance(VI_1) setting protocol VIPs.Sep 5 15:42:46 test2 Keepalived_healthcheckers[14988]: Netlink reflector reports IP 10.0.10.200 addedSep 5 15:42:46 test2 Keepalived_vrrp[14989]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200Sep 5 15:42:51 test2 Keepalived_vrrp[14989]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.0.10.200
发现test2立即变成了master，看VIP
[root@test2 ~]# ifconfigeth0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.12 Bcast:10.0.10.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe0d:26b8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:29585 errors:0 dropped:0 overruns:0 frame:0 TX packets:33483 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3618179 (3.4 MiB) TX bytes:3520824 (3.3 MiB)
eth0:0 Link encap:Ethernet HWaddr 08:00:27:0D:26:B8 inet addr:10.0.10.200 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 08:00:27:35:E1:F4 inet addr:10.0.0.18 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe35:e1f4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2722 errors:0 dropped:0 overruns:0 frame:0 TX packets:7036 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:163320 (159.4 KiB) TX bytes:379776 (370.8 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:3691 errors:0 dropped:0 overruns:0 frame:0 TX packets:3691 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:205595 (200.7 KiB) TX bytes:205595 (200.7 KiB)
[root@test2 ~]#
发现VIP成功切换。。。。
注：这里就不详细贴日志，自己测试即可。。。
说明：对于上面2和3的测试看起来似乎绕了很大一圈，但是不绕这么一圈，你可能学不到什么，网上一大堆都是巴拉拉巴的都在优先级高的上测试，随便弄个例子就OK了。我这么做，就是让你知道这里面有这么个弯子，你自己想怎么用，自己看着办了。。。
第四种：后端web服务器全部故障时，使用sorry_server定向请求到其他备用服务器 对于这个功能，其实是可有可无，在生产环境中，肯定是每台服务器上的服务都是有所监控的，一旦发现错误就会理解处理，基本上不会发生后端web服务全部故障无法返回数据的情况。但是凡事不是绝对的，说不定奇葩了呢。。所以还是说一下这个功能。
在测试上面的功能，先测试一下，当后端web服务器有故障时，lvs是否会踢掉有故障的规则：测试：在master上查看lvs规则：[root@test2 ~]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0 [root@test2 ~]#
停止后端一台web服务器的httpd服务：[root@test4 ~]# service httpd stopStopping httpd: [ OK ][root@test4 ~]#
在查看lvs规则：[root@test2 ~]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 You have new mail in /var/spool/mail/root[root@test2 ~]#
看日志：Sep 5 15:55:44 test2 Keepalived_healthcheckers[19580]: Error connecting server [10.0.10.14]:80.Sep 5 15:55:44 test2 Keepalived_healthcheckers[19580]: Removing service [10.0.10.14]:80 from VS [10.0.10.200]:80Sep 5 15:55:44 test2 Keepalived_healthcheckers[19580]: Remote SMTP server [127.0.0.1]:25 connected.Sep 5 15:55:44 test2 Keepalived_healthcheckers[19580]: SMTP alert successfully sent.
可以看到，但后端web服务故障时，对应的规则成功被踢掉。。。
在启动httpd服务：[root@test4 ~]# service httpd startStarting httpd: [ OK ][root@test4 ~]#
看日志：
Sep 5 15:57:22 test2 Keepalived_healthcheckers[19580]: HTTP status code success to [10.0.10.14]:80 url(1).Sep 5 15:57:25 test2 Keepalived_healthcheckers[19580]: Remote Web server [10.0.10.14]:80 succeed on service.Sep 5 15:57:25 test2 Keepalived_healthcheckers[19580]: Adding service [10.0.10.14]:80 to VS [10.0.10.200]:80Sep 5 15:57:25 test2 Keepalived_healthcheckers[19580]: Remote SMTP server [127.0.0.1]:25 connected.Sep 5 15:57:25 test2 Keepalived_healthcheckers[19580]: SMTP alert successfully sent.
查看规则：IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0 [root@test2 ~]#
可以看到，规则又成功备添加回来了。。。。这说明keepalived可以实时检测后端服务，并刷新规则。。。
测试sorry_server的功能：在配置文件中添加下面这么一句话：virtual_server 10.0.10.200 80 { delay_loop 3 lb_algo rr lb_kind DR nat_mask 255.255.255.0# persistence_timeout 50 protocol TCPsorry_server 10.0.10.15 80 //我这是配置文件的一部分，看清楚是在什么位置添加的。。。 real_server 10.0.10.13 80 { weight 1
重启keepalived服务：[root@test2 keepalived]# service keepalived restartStopping keepalived: [ OK ]Starting keepalived: [ OK ][root@test2 keepalived]#查看规则：[root@test2 keepalived]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.13:80 Route 1 0 0 -> 10.0.10.14:80 Route 1 0 0 [root@test2 keepalived]#
停止后端所有web服务：[root@test4 ~]# service httpd stopStopping httpd: [ OK ][root@test4 ~]#[root@test3 logs]# service httpd stopStopping httpd: [ OK ][root@test3 logs]#
查看日志：[root@test2 keepalived]#tail -f /var/log/messageSep 5 16:05:15 test2 Keepalived_healthcheckers[25096]: Removing service [10.0.10.13]:80 from VS [10.0.10.200]:80Sep 5 16:05:15 test2 Keepalived_healthcheckers[25096]: Remote SMTP server [127.0.0.1]:25 connected.Sep 5 16:05:15 test2 Keepalived_healthcheckers[25096]: SMTP alert successfully sent.Sep 5 16:05:20 test2 Keepalived_healthcheckers[25096]: Error connecting server [10.0.10.14]:80.Sep 5 16:05:20 test2 Keepalived_healthcheckers[25096]: Removing service [10.0.10.14]:80 from VS [10.0.10.200]:80Sep 5 16:05:20 test2 Keepalived_healthcheckers[25096]: Lost quorum 1-0=1 > 0 for VS [10.0.10.200]:80Sep 5 16:05:20 test2 Keepalived_healthcheckers[25096]: Adding sorry server [10.0.10.15]:80 to VS [10.0.10.200]:80Sep 5 16:05:20 test2 Keepalived_healthcheckers[25096]: Removing alive servers from the pool for VS [10.0.10.200]:80Sep 5 16:05:20 test2 Keepalived_healthcheckers[25096]: Remote SMTP server [127.0.0.1]:25 connected.Sep 5 16:05:20 test2 Keepalived_healthcheckers[25096]: SMTP alert successfully sent.
查看规则：[root@test2 keepalived]# ipvsadm --list -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 10.0.10.200:80 rr -> 10.0.10.15:80 Route 1 0 0 You have new mail in /var/spool/mail/root[root@test2 keepalived]#
好，到这个地方，请你注意了，对比一样，当添加了sorry_server之后重启了keepalived服务，第一次查看规则，发现没有任何变化，并没有添加备用web的规则信息，当停止掉所有后端服务时此时在查看规则，发现自动生成了备用web的规则。。此时在客户端测试访问，发现访问到的是备用web上的内容。。
第五种：主备状态切换的报警 对于keepalived来说，特别重要的就是当前的主备状态，以便于我们根据需要进行调整。。。配置如下：vrrp_instance VI_1 { state BACKUP interface eth1 virtual_router_id 51 priority 99 advert_int 1 authentication { auth_type PASS auth_pass keepalivepass } virtual_ipaddress { 10.0.10.200/24 dev eth0 label eth0:0 }
track_script { check_lvs} track_interface { eth0} notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master 10.0.10.200" notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup 10.0.10.200" notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault 10.0.10.200" smtp_alert //这个必须有，是报警的开关}
上面的配置是个片段，报警配置是:notify开头的配置。上面的master，backup好理解，fault通常是跟心跳有关，例如心跳网卡宕了，检测不到心跳信息了等。。
一个简单的notify.sh的内容：#!/bin/bash#
Usage() { echo "Usage:`basename $0` {master|backup|fault} VIP"}Notify() { subject="`hostname`'s keepalived state changed to $1" mailBody="`date "+%F %T"`:`hostname`'s keepalived state change to $1,$VIP floating." echo $mailBody | mail -s "$subject" $contact}[ $# -lt 2 ] && Usage && exit 1
VIP=$2case $1 in master) Notify master ;; backup) Notify backup ;; fault) Notify fault ;; *) Usage exit 1 ;;esac
配置完成后，重启keepalived服务，当主备发生切换时，就会有邮件报警，在生产环境中，可以将邮件地址填写为自己的邮箱即可。。其次配置了这里的报警，就可以取消全局配置中的邮件报警了，notification_email相关的设置...,使用该方式，你可以任意定义报警的内容。。。
到此位置，关于keepalived+lvs的部署就说道这了，不可能面面俱到，但是也说的差不多了。。。至于keepalived的本身其它的内容设置，会在keepalived.conf配置文件分析中再聊一聊。。
在啰嗦一句，这文档有点长啊，不想在从头审核了，看到错误的话，你自己意会吧啊，不过我想应该不会有什么笔误吧。
结束！！！ 笨蛋的技术------不怕你不会！！！！