k8s calico overlay网络下抓包

一、背景

    因为机房xxx服务节点（172.16.8.161）需要回收搬迁，所以需要先迁移上面的流量，发现都是172.16.49.94这个k8s集群宿主机容器发出的请求

root@172-16-8-161:~# netstat -nat|grep 9090|grep 172.16.49.94
tcp6       0      0 172.16.8.161:9090       172.16.49.94:55968      ESTABLISHED
tcp6       0      0 172.16.8.161:9090       172.16.49.94:38430      ESTABLISHED
tcp6       0      0 172.16.8.161:9090       172.16.49.94:55524      ESTABLISHED
tcp6       0      0 172.16.8.161:9090       172.16.49.94:42136      ESTABLISHED
tcp6       0      0 172.16.8.161:9090       172.16.49.94:55972      ESTABLISHED
tcp6       0      0 172.16.8.161:9090       172.16.49.94:42752      ESTABLISHED
tcp6       0      0 172.16.8.161:9090       172.16.49.94:38436      ESTABLISHED
tcp6       0      0 172.16.8.161:9090       172.16.49.94:35680      ESTABLISHED
tcp6       0      0 172.16.8.161:9090       172.16.49.94:36676      ESTABLISHED

二、解决

一、遍历宿主机上面所有的容器网卡，然后通过tcpdump抓包
root@172-16-49-94:/data/tmp# for i in `ip a|grep cali|awk -F'[ :]'+ '{print$2}'`;do echo $i;timeout 180 tcpdump -vvvv -i $i dst 172.16.8.161 and port 9090 -w /data/tmp/$i.txt;done
root@172-16-49-94:/data/tmp# ll
total 168
-rw-r--r--. 1 tcpdump tcpdump   24 Jun 30 14:04 cali01a4ffa592d@if4.txt
-rw-r--r--. 1 tcpdump tcpdump 8492 Jun 30 12:58 cali02ac8d93ad8@if4.txt
-rw-r--r--. 1 tcpdump tcpdump   24 Jun 30 12:28 cali0e601ff8351@if4.txt
-rw-r--r--. 1 tcpdump tcpdump   24 Jun 30 13:19 cali0ff5657372e@if4.txt
-rw-r--r--. 1 tcpdump tcpdump   24 Jun 30 13:22 cali1e8e5529179@if4.txt
-rw-r--r--. 1 tcpdump tcpdump   24 Jun 30 13:58 cali26c6829ab29@if4.txt
-rw-r--r--. 1 tcpdump tcpdump   24 Jun 30 14:19 cali31b8c211055@if4.txt
-rw-r--r--. 1 tcpdump tcpdump   24 Jun 30 13:40 cali31f20848bf4@if4.txt

二、遍历生成的抓包文件之后，查询哪个网卡有发生对172.16.8.161:9090的请求，然后通过route -n命令根据网卡获取k8s容器IP
root@172-16-49-94:/data/tmp# for f in `ls *.txt`;do count=$(grep -c 172.16.8.161:9090 $f);if [ $count -gt 0 ];then eth=$(echo $f|awk -F'[ .@]' '{print$1}');ip=$(route -n|grep $eth|awk '{print$1}');echo $f $eth $count $ip;fi;done
cali02ac8d93ad8@if4.txt cali02ac8d93ad8 4 10.233.85.176
cali8cb537624a1@if4.txt cali8cb537624a1 2 10.233.85.167
caliac9e6f976af@if4.txt caliac9e6f976af 6 10.233.85.168
calib018b20c03a@if4.txt calib018b20c03a 1 10.233.85.182
calica32b3125e8@if4.txt calica32b3125e8 1 10.233.85.185

三、通过k8s集群get pods,定位容器ip
# kubect get pods -o wide|grep $ip

‍