pcap2txt

该程序的作用是将pcap文件里的前三个ssl/tls包的数据,转化成十进制提取到txt文件里:

  • dataset里有很多域名文件夹,每个文件夹下有很多的txt文件,记录着数据包,我们使不同域名文件夹下的txt文件个数相同
  • pcapnum_per_txt:每个txt文件,是由pcapnum_per_txt个pcap处理得到的
  • filelist是所有pcap文件名的列表,这些pcap要分txt_num次处理,每次处理的文件名存进new_list里
if __name__ == '__main__':
    path = '/home/new3/https/lx/login.weixin.qq.com'
    filelist = os.listdir(path)
    #print (len(filelist))

    pcapnum_per_txt = len(filelist) // txt_num

    for i in range(txt_num):
        print(str(i)+'.txt')
        new_list = filelist[i*pcapnum_per_txt:i*pcapnum_per_txt + pcapnum_per_txt]


        for file in new_list:
            print('The pcap file is: ' + file)
            filepath = os.path.join(path,file)
            parse_pcap(filepath)

        txt_name = str(i) + '.txt'
        fw = open(txt_name, "a+")
        for key in flow.keys():
            if key in new_list:
                if(len(flow[key]) == 3):
                    print(key)
                    for pkts in flow[key]:
                        for bytes in pkts:
                            fw.write(str(bytes)+" ")
                        fw.write("\n")
        print('The above pcap file is written in the txt file.')
        print ('\n')

parse_pcap

  • 先读取24字节的pcap文件头,然后在循环读【先16字节数据包头包含这个数据包的大小iplensave,再读iplensave大小的数据包】
  • 参数iplensave记录了当前数据包的长度
确定当前包是ssl/tls的方法:
  • tls是基于tcp的,由tcp封装
  • tls包的第一个字段标明了tls类型(content type),接下来的一个字段标明了version,大多数(目前发现)的content type 值只有20(0x14),22(0x16),23(0x17)三个值,version的第一个字节都是由0x03开头的
tls.png
  • mac层有14B,ip层一般有20B,tcp的长度不定长,由首部的header length字段给出了tcp层的长度,该字段只有4bits,位于tcp首部的第13B的前4b,该值转化为十进制再乘4就是整个tcp的长度
  • 首先判断包长度iplensave > 54,因为mac+ip+tcp最少需要54B,小于54B一定没有tls层
  • 判断iplensave - tcplen - iplen - maclen > 0,如果=0也没有tls层
  • 两个条件都满足,再判断tcp的下一字节是不是20,22或者23,version是不是3,两者都满足,则是tls包(巧合的概率很小可以忽略)


    tcp
def parse_pcap(filename):
    with open(filename, "rb") as file: 
        # Read 24-bytes pcap header 
        data = file.read(pcaphdrlen)
        (tag, maj, min, tzone, ts, ppsize, lt) = struct.unpack("=L2p2pLLLL", data)
        # pocket counter
        cnt = 0

        while data:
            # read packet header
            data = file.read(pkthdrlen)
            if not data:
                break
            (sec, microsec, iplensave, origlen) = struct.unpack("=LLLL", data)
            # print (sec, microsec, iplensave, origlen)
            #print iplensave
            data = file.read(iplensave)


            if iplensave > 54:
                tcplen = ord(data[46])//16*4
                if iplensave - tcplen - iplen - maclen > 0:
                    tlstype = maclen + iplen + tcplen
                    tlsversion = tlstype + 1
                    if (ord(data[tlstype]) == 20 or ord(data[tlstype]) == 22 or ord(data[tlstype]) == 23) and ord(data[tlsversion]) == 3:
                        processpacket(data)
                        cnt = cnt + 1

        print('The number of ssl/tls packets: ' + str(cnt))
        print('----------------------------------------------------------------------------------')

processpacket

  • 定义一个字典flow,key是pcap的文件名,因为我们输入的是一个域名文件夹下的所有pcap文件,使用文件名作为key不会有重复,value是该pcap文件下满足筛选条件(tls)的包,最多取三个
def processpacket(pkt):
    pkt = [ord(b) for b in str(pkt)]
    proto = pkt[23]

    srcip = "{0}.{1}.{2}.{3}".format(pkt[26], pkt[27], pkt[28], pkt[29])
    dstip = "{0}.{1}.{2}.{3}".format(pkt[30], pkt[31], pkt[32], pkt[33])

    sport = pkt[34] * 256 + pkt[35]
    dport = pkt[36] * 256 + pkt[37]

    pkt = preprocess(pkt, proto)
    # print file    

    tuple = file
    if tuple in flow:

        value = flow[tuple]
        if len(value) < 3:
            value.append(pkt)
            flow[tuple] = value

    else:
        value = []

        value.append(pkt)
        flow[tuple] = value

preprocess

  • 取1000字节,去掉mac和ip层的信息,从tcp开始截取
def preprocess(packet, proto):
    # remove mac and ip layer, start from tcp layer
    packet = packet[34:]
    #TCP
    if len(packet) < 1000:
        for j in range(1000 - len(packet)):
            packet.append(0)
    else:
        packet = packet[:1000]
    return packet

结果:

  • 横线部分是程序处理的pcap文件,ssl/tls包的个数是3的才会被写进txt里,小于3的不处理
  • 横线底下列出的是写进txt的pcap文件,列出的顺序就是写入txt的顺序
new3@new3:~/https/lx$ python parsepcap.py 
0.txt
The pcap file is: 159.226.121.15_54806_101.226.76.164_443_1556368053.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.117.158_44082_101.227.160.102_443_1556357309.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.171.251_5794_101.227.160.102_443_1556370252.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.244_60644_101.227.160.102_443_1556357290.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.253_12310_223.166.152.108_443_1556368139.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.81_7116_101.226.76.164_443_1556368107.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.20.7_35292_101.226.76.164_443_1556368849.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.117.215_1098_101.227.160.102_443_1556357592.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
159.226.25.81_7116_101.226.76.164_443_1556368107.pcap
159.226.171.251_5794_101.227.160.102_443_1556370252.pcap
159.226.117.215_1098_101.227.160.102_443_1556357592.pcap
159.226.35.244_60644_101.227.160.102_443_1556357290.pcap
159.226.20.7_35292_101.226.76.164_443_1556368849.pcap
The above pcap file is written in the txt file.


1.txt
The pcap file is: 159.226.171.251_34568_101.226.76.164_443_1556369112.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.121.15_49579_101.226.76.164_443_1556357432.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.171.251_34358_101.227.160.102_443_1556368031.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.113.225_14779_101.226.76.164_443_1556369020.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.121_53831_101.226.76.164_443_1556357383.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.171.251_14133_101.227.160.102_443_1556368271.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.182.51_49968_117.135.169.34_443_1556368833.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.91_36511_101.226.76.164_443_1556368921.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
159.226.25.91_36511_101.226.76.164_443_1556368921.pcap
159.226.171.251_14133_101.227.160.102_443_1556368271.pcap
159.226.182.51_49968_117.135.169.34_443_1556368833.pcap
159.226.171.251_34568_101.226.76.164_443_1556369112.pcap
159.226.171.251_34358_101.227.160.102_443_1556368031.pcap
The above pcap file is written in the txt file.


2.txt
The pcap file is: 159.226.95.33_12289_101.226.76.164_443_1556370074.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.43.54_48313_101.226.76.164_443_1556357346.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.132_54887_101.227.160.102_443_1556357319.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.121_58915_101.226.76.164_443_1556368004.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.91_32511_101.227.160.102_443_1556367961.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.172_5276_101.227.160.102_443_1556370303.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.113.225_33547_101.226.76.164_443_1556370161.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.121_58948_101.226.76.164_443_1556368244.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
159.226.25.91_32511_101.227.160.102_443_1556367961.pcap
159.226.95.33_12289_101.226.76.164_443_1556370074.pcap
159.226.43.54_48313_101.226.76.164_443_1556357346.pcap
159.226.35.172_5276_101.227.160.102_443_1556370303.pcap
159.226.118.121_58948_101.226.76.164_443_1556368244.pcap
159.226.113.225_33547_101.226.76.164_443_1556370161.pcap
The above pcap file is written in the txt file.


3.txt
The pcap file is: 159.226.117.158_7200_101.227.160.102_443_1556369012.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.199.87_60649_101.227.160.102_443_1556368176.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.110.25_65292_101.226.76.164_443_1556368947.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.171.251_1215_101.227.160.102_443_1556357407.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.113.225_55106_101.226.76.164_443_1556357379.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.81_7611_101.226.76.164_443_1556368229.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.117.158_23010_101.226.76.164_443_1556370332.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.138_51823_101.226.76.164_443_1556370162.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
159.226.171.251_1215_101.227.160.102_443_1556357407.pcap
159.226.199.87_60649_101.227.160.102_443_1556368176.pcap
159.226.25.81_7611_101.226.76.164_443_1556368229.pcap
The above pcap file is written in the txt file.


4.txt
The pcap file is: 159.226.35.244_53259_101.226.76.164_443_1556368032.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.21.20_53148_101.227.160.102_443_1556357379.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.244_53897_101.227.160.102_443_1556368813.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.117.158_12623_101.227.160.102_443_1556368952.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.244_55147_101.226.76.164_443_1556370313.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.231.165_52310_101.226.76.164_443_1556367992.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.81_6935_101.226.76.164_443_1556368047.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.177_55157_101.226.76.164_443_1556357535.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
159.226.35.177_55157_101.226.76.164_443_1556357535.pcap
159.226.35.244_55147_101.226.76.164_443_1556370313.pcap
159.226.35.244_53259_101.226.76.164_443_1556368032.pcap
159.226.21.20_53148_101.227.160.102_443_1556357379.pcap
159.226.25.81_6935_101.226.76.164_443_1556368047.pcap
159.226.231.165_52310_101.226.76.164_443_1556367992.pcap
159.226.35.244_53897_101.227.160.102_443_1556368813.pcap
159.226.117.158_12623_101.227.160.102_443_1556368952.pcap
The above pcap file is written in the txt file.


new3@new3:~/https/lx$ 

你可能感兴趣的:(pcap2txt)