linux-lvs-nat模式，无法完成3次握手

[ningyougang]

ALL好！
nat模式配置好后，调度算法起作用的。但是当我通过自己的本机访问，就不能访问（只能在LB-SERVER本身上，通过wget http://vip-address/）才可以访问。
通过ipvsadm -lcn发现其TCP状态为SYN_RECV。
LB-SERVER(VIP:192.168.100.250 DIP:192.168.100.119)
RIP1:192.168.100.120 RIP:192.168.100.121。网关均指向192.168.100.119

然后通过tcpdump相关命令，发现其没有完成tcp三次握手。日志如下
在客户端(非LB-SERVER)执行，wget http://192.168.100.250/demo.html，通过tcpdump抓包结果如下
root@119-compute:/home/compute# tcpdump -n -tttt -i eth0 dst 192.168.100.250 and port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
2014-10-19 21:55:31.088510 IP 192.168.100.10.37280 > 192.168.100.250.80: Flags [S], seq 4161161539, win 29200, options [mss 1460,sackOK,TS val 80496110 ecr 0,nop,wscale 7], length 0
2014-10-19 21:55:32.085629 IP 192.168.100.10.37280 > 192.168.100.250.80: Flags [S], seq 4161161539, win 29200, options [mss 1460,sackOK,TS val 80496360 ecr 0,nop,wscale 7], length 0

root@120-compute:/usr/local/tomcat6# tcpdump -n -tttt -i eth0 dst 192.168.100.120 and port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
2014-10-19 21:58:10.628408 IP 192.168.100.10.37292 > 192.168.100.120.80: Flags [S], seq 2146963872, win 29200, options [mss 1460,sackOK,TS val 80533821 ecr 0,nop,wscale 7], length 0
2014-10-19 21:58:10.628543 IP 192.168.100.10.37292 > 192.168.100.120.80: Flags [R], seq 2146963873, win 0, length 0
2014-10-19 21:58:11.625935 IP 192.168.100.10.37292 > 192.168.100.120.80: Flags [S], seq 2146963872, win 29200, options [mss 1460,sackOK,TS val 80534071 ecr 0,nop,wscale 7], length 0
2014-10-19 21:58:11.626053 IP 192.168.100.10.37292 > 192.168.100.120.80: Flags [R], seq 2146963873, win 0, length 0

[ningyougang]

支持下支持下支持下

[小笨和漂向北方]

I am sorry to see that you are still stuck there! My gut feeling is that your virtual ip set up went wrong. Still, I strongly recommend this document, read through it and you will probably find answer. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Load_Balancer_Administration/index.html

[ningyougang]

引用 2 楼 micropentium6 的回复:

I am sorry to see that you are still stuck there! My gut feeling is that your virtual ip set up went wrong. Still, I strongly recommend this document, read through it and you will probably find answer. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Load_Balancer_Administration/index.html

在贴上我整个的安装过程 一、LB-SERVER配置 //1.先关闭防火墙和selinux,以防止端口屏蔽问题和莫名奇妙的安全问题 #ufw disable //关闭防火墙 #setenforce 0 //临时关闭 //2.安装ipvs #apt-get install ipvsadm //3.配置ip转发 #vi /etc/sysctl.conf net.ipv4.ip_forward=1 net.ipv4.conf.default.send_redirects=0 net.ipv4.conf.all.send_redirects=0 net.ipv4.conf.eth0.send_redirects=0 //永久生效 sysctl –p //4.配置vip //给eth0网卡在绑定1虚拟IP( 删除指令:ip addr del 192.168.100.250 dev eth0) //该配置方法重启机器失效,可以配置在/etc/network/interfaces文件里,永久生效 #ifconfig eth0:0 192.168.100.250 broadcast 192.168.100.255 netmask 255.255.255.0 //5.配置dip #vi /etc/network/interfaces auto eth0 iface eth0 inet static address 192.168.100.119 netmask 255.255.0.0 network 192.168.0.0 gateway 192.168.0.6 dns-nameservers 192.168.10.5 #service networking restart //重启网络 //6.通过ipvsadm添加虚拟服务器和真实服务器并指定负载均衡NAT模式和后台调度算法 ipvsadm –C //清除ipvs配置 ipvsadm -A -t 192.168.100.250:80 -s rr //加入虚拟机服务器,删除将-A调整为-D ipvsadm -a -t 192.168.100.250:80 -r 192.168.100.120:80 -m -w 1 //加入真实服务器120 ipvsadm -a -t 192.168.100.250:80 -r 192.168.100.121:80 -m -w 1 //加入真实服务器121 service ipvsadm save //保存 ipvsadm -L -n //查看ipvsadm配置情况 二、REAL-SERVER配置 //1.修改IP且默认网关指向LB-SERVER的IP( 临时:/sbin/route add default gw 192.168.100.119) #vi /etc/network/interfaces //编辑，需改网关 auto eth0 iface eth0 inet static address 192.168.100.120 netmask 255.255.0.0 network 192.168.0.0 #gateway 192.168.0.6 gateway 192.168.100.119 dns-nameservers 192.168.10.5 #service networking restart //重启网络服务 #netstat –r //查看默认网关 //2.配置IP转发 vi /etc/sysctl.conf net.ipv4.ip_forward=1 sysctl –p

[小笨和漂向北方]

引用 3 楼 ningyougang 的回复:

[quote=引用 2 楼 micropentium6 的回复:] I am sorry to see that you are still stuck there! My gut feeling is that your virtual ip set up went wrong. Still, I strongly recommend this document, read through it and you will probably find answer. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Load_Balancer_Administration/index.html

在贴上我整个的安装过程 一、LB-SERVER配置 //1.先关闭防火墙和selinux,以防止端口屏蔽问题和莫名奇妙的安全问题 #ufw disable //关闭防火墙 #setenforce 0 //临时关闭 //2.安装ipvs #apt-get install ipvsadm //3.配置ip转发 #vi /etc/sysctl.conf net.ipv4.ip_forward=1 net.ipv4.conf.default.send_redirects=0 net.ipv4.conf.all.send_redirects=0 net.ipv4.conf.eth0.send_redirects=0 //永久生效 sysctl –p //4.配置vip //给eth0网卡在绑定1虚拟IP( 删除指令:ip addr del 192.168.100.250 dev eth0) //该配置方法重启机器失效,可以配置在/etc/network/interfaces文件里,永久生效 #ifconfig eth0:0 192.168.100.250 broadcast 192.168.100.255 netmask 255.255.255.0 //5.配置dip #vi /etc/network/interfaces auto eth0 iface eth0 inet static address 192.168.100.119 netmask 255.255.0.0 network 192.168.0.0 gateway 192.168.0.6 dns-nameservers 192.168.10.5 #service networking restart //重启网络 //6.通过ipvsadm添加虚拟服务器和真实服务器并指定负载均衡NAT模式和后台调度算法 ipvsadm –C //清除ipvs配置 ipvsadm -A -t 192.168.100.250:80 -s rr //加入虚拟机服务器,删除将-A调整为-D ipvsadm -a -t 192.168.100.250:80 -r 192.168.100.120:80 -m -w 1 //加入真实服务器120 ipvsadm -a -t 192.168.100.250:80 -r 192.168.100.121:80 -m -w 1 //加入真实服务器121 service ipvsadm save //保存 ipvsadm -L -n //查看ipvsadm配置情况 二、REAL-SERVER配置 //1.修改IP且默认网关指向LB-SERVER的IP( 临时:/sbin/route add default gw 192.168.100.119) #vi /etc/network/interfaces //编辑，需改网关 auto eth0 iface eth0 inet static address 192.168.100.120 netmask 255.255.0.0 network 192.168.0.0 #gateway 192.168.0.6 gateway 192.168.100.119 dns-nameservers 192.168.10.5 #service networking restart //重启网络服务 #netstat –r //查看默认网关 //2.配置IP转发 vi /etc/sysctl.conf net.ipv4.ip_forward=1 sysctl –p[/quote] Thank you for sharing. If we get some spare time later this week, I might be able to take a look. I know why you can't follow the article I recommended: it's for RHEL and you probably have a ubuntu...

[ningyougang]

引用 2 楼 micropentium6 的回复:

I am sorry to see that you are still stuck there! My gut feeling is that your virtual ip set up went wrong. Still, I strongly recommend this document, read through it and you will probably find answer. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Load_Balancer_Administration/index.html

我在LB-SERVER上，通过tcmp来查看来自RS1上的日志，结果如下 root@119-compute:/home/compute# tcpdump -n -tttt -i eth0 src 192.168.100.120 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 2014-10-22 01:05:27.948842 ARP, Reply 192.168.100.120 is-at 08:57:00:d2:1d:e9, length 46 2014-10-22 01:06:36.972841 ARP, Reply 192.168.100.120 is-at 08:57:00:d2:1d:e9, length 46 看这个日志貌似跟ARP有关系。呵呵，查证中！

[ningyougang]

如果是从LB-SERVER内部访问wget http://192.168.100.250/demo.html 在LB-server上监控来自RS的日志如下（这个就是正常的） root@119-compute:/home/compute# tcpdump -n -tttt -i eth0 src 192.168.100.120 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 2014-10-22 01:34:33.655299 IP 192.168.100.120.80 > 192.168.20.100.56406: Flags [S.], seq 2796168726, ack 2055865296, win 28960, options [mss 1460,sackOK,TS val 111703158 ecr 41231478,nop,wscale 7], length 0 2014-10-22 01:34:33.655532 IP 192.168.100.120.80 > 192.168.20.100.56406: Flags [.], ack 122, win 227, options [nop,nop,TS val 111703158 ecr 41231478], length 0 2014-10-22 01:34:33.656536 IP 192.168.100.120.80 > 192.168.20.100.56406: Flags [P.], seq 1:292, ack 122, win 227, options [nop,nop,TS val 111703158 ecr 41231478], length 291 2014-10-22 01:34:33.657628 IP 192.168.100.120.80 > 192.168.20.100.56406: Flags [F.], seq 292, ack 123, win 227, options [nop,nop,TS val 111703159 ecr 41231479], length 0 2014-10-22 01:34:38.668843 ARP, Reply 192.168.100.120 is-at 08:57:00:d2:1d:e9, length 46 2014-10-22 01:34:38.669114 ARP, Request who-has 192.168.20.100 tell 192.168.100.120, length 46

[ningyougang]

引用 2 楼 micropentium6 的回复:

I am sorry to see that you are still stuck there! My gut feeling is that your virtual ip set up went wrong. Still, I strongly recommend this document, read through it and you will probably find answer. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Load_Balancer_Administration/index.html

会不是LB-SERVER网关的问题呢？或者整个网络的结构有问题呢？ VIP(eth0:0) :192.168.100.250 broadcast 192.168.100.255 netmask 255.255.255.0 DIP(eth0):信息如下： auto eth0 iface eth0 inet static address 192.168.100.119 netmask 255.255.0.0 network 192.168.0.0 gateway 192.168.0.6 #gateway 192.168.5.172 dns-nameservers 192.168.10.5 RS1(eth0):信息如下： auto eth0 iface eth0 inet static address 192.168.100.120 netmask 255.255.0.0 network 192.168.0.0 gateway 192.168.100.119 dns-nameservers 192.168.10.5

[小笨和漂向北方]

引用 5 楼 ningyougang 的回复:

[quote=引用 2 楼 micropentium6 的回复:] I am sorry to see that you are still stuck there! My gut feeling is that your virtual ip set up went wrong. Still, I strongly recommend this document, read through it and you will probably find answer. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Load_Balancer_Administration/index.html

会不是LB-SERVER网关的问题呢？或者整个网络的结构有问题呢？ VIP(eth0:0) :192.168.100.250 broadcast 192.168.100.255 netmask 255.255.255.0 DIP(eth0):信息如下： auto eth0 iface eth0 inet static address 192.168.100.119 netmask 255.255.0.0 network 192.168.0.0 gateway 192.168.0.6 #gateway 192.168.5.172 dns-nameservers 192.168.10.5 RS1(eth0):信息如下： auto eth0 iface eth0 inet static address 192.168.100.120 netmask 255.255.0.0 network 192.168.0.0 gateway 192.168.100.119 dns-nameservers 192.168.10.5[/quote] I am sorry I didn't have time to take a look on this... is it ok to set RS1's gateway as DIP's IP?

[ningyougang]

引用 6 楼 micropentium6 的回复:

[quote=引用 5 楼 ningyougang 的回复:] [quote=引用 2 楼 micropentium6 的回复:] I am sorry to see that you are still stuck there! My gut feeling is that your virtual ip set up went wrong. Still, I strongly recommend this document, read through it and you will probably find answer. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Load_Balancer_Administration/index.html

会不是LB-SERVER网关的问题呢？或者整个网络的结构有问题呢？ VIP(eth0:0) :192.168.100.250 broadcast 192.168.100.255 netmask 255.255.255.0 DIP(eth0):信息如下： auto eth0 iface eth0 inet static address 192.168.100.119 netmask 255.255.0.0 network 192.168.0.0 gateway 192.168.0.6 #gateway 192.168.5.172 dns-nameservers 192.168.10.5 RS1(eth0):信息如下： auto eth0 iface eth0 inet static address 192.168.100.120 netmask 255.255.0.0 network 192.168.0.0 gateway 192.168.100.119 dns-nameservers 192.168.10.5[/quote] I am sorry I didn't have time to take a look on this... is it ok to set RS1's gateway as DIP's IP?[/quote] 没有好！RS1的网关早就设置为DIP's IP地址了。

[ywskin_gmail]

我也遇到这个问题，请问楼主怎么解决的？