python struct 学习笔记

python struct 学习笔记

part 1: introduce

python 是一门简洁的语言。为了与其他语言或平台（尤其在网络传输过程中）进行互相转换，有必要了解一下 python struct.

part 2: get help

NAME
    struct
FILE
    /usr/lib/python2.6/struct.py
FUNCTIONS
    calcsize(...)
        Return size of C struct described by format string fmt.
    pack(...)
        Return string containing values v1, v2, ... packed according to fmt.
    pack_into(...)
        Pack the values v1, v2, ... according to fmt.
        Write the packed bytes into the writable buffer buf starting at offset.
    unpack(...)
        Unpack the string containing packed C structure data, according to fmt.
        Requires len(string) == calcsize(fmt).
    unpack_from(...)
        Unpack the buffer, containing packed C structure data, according to
        fmt, starting at offset. Requires len(buffer[offset:]) >= calcsize(fmt).


part 3: instances

function 1: 
calcsize(...)  计算格式字符串对应的结果的长度

>>> b = struct.calcsize('HB')
>>> b
3
>>> b = struct.calcsize('!HBBBBBBIIIIBH')
>>> b
27
>>> b = struct.calcsize('HBBBBBBIIIIBH')
>>> b
28

function 2: 
pack(...) 根据格式字符串，将python值转换为字节流（字节数组）

>>> a = struct.pack("!HBBBBBBIIIIBH",48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> a
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01\x00\>>> repr(a)x02'

function 3:
pack_into(...) 与pack()类似。根据格式字符串，将python值转换为字节流（字节数组),并且将这字节流保存到一个buffer 中

>>> from ctypes import create_string_buffer
>>> buf = create_string_buffer(40)
>>> a = struct.pack_into("!HBBBBBBIIIIBH",buf,0,48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> buf


function 4：
unpack(...)  与pack() 相反。根据格式字符串，将字节流转换成python 数据类型。

>>> b = struct.unpack("!HBBBBBBIIIIBH",a)

>>> b
(48251, 0, 24, 0, 1, 0, 0, 842003, 4294963512, 2300851747, 368142189, 1, 2)


function 5:
unpack_from(...) 与unpack()类似。根据格式字符串，从一个buffer 中将字节流转换成python 数据类型

>>> b  = struct.unpack_from("!HBBBBBBIIIIBH",buf,0)
>>> b
(48251, 0, 24, 0, 1, 0, 0, 842003, 4294963512, 2300851747, 368142189, 1, 2)
>>> buf

# pack 之后得到的字节流（数组）也可以像序列一样来操作：

>>> a = struct.pack("!HBBBBBBIIIIBH",48251,0,24,0,1,0,0,842003,4294963512,2300851747,368142189,1,2)
>>> a
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01\x00\x02'
>>> len(a)
27
>>> head  = a[:25]
>>> head
'\xbc{\x00\x18\x00\x01\x00\x00\x00\x0c\xd9\x13\xff\xff\xf18\x89$6#\x15\xf1gm\x01'
>>> command = a[25:]
>>> command
'\x00\x02'
>>> command = a[24:]
>>> command
'\x01\x00\x02'
>>> command = a[-4:]
>>> command
'm\x01\x00\x02'
>>> command = a[-3:]
>>> command
'\x01\x00\x02'

part 4: big-endian & format

 Functions to convert between Python values and C structs represented
    as Python strings. It uses format strings (explained below) as compact
    descriptions of the lay-out of the C structs and the intended conversion
    to/from Python values.

    The optional first format char indicates byte order, size and alignment:
      @: native order, size & alignment (default)
      =: native order, std. size & alignment
      <: little-endian, std. size & alignment
      >: big-endian, std. size & alignment
      !: same as >

    The remaining chars indicate types of args and must match exactly;
    these can be preceded by a decimal repeat count:
      x: pad byte (no data); c:char; b:signed byte; B:unsigned byte;
      ?: _Bool (requires C99; if not available, char is used instead)
      h:short; H:unsigned short; i:int; I:unsigned int;
      l:long; L:unsigned long; f:float; d:double.
    Special cases (preceding decimal count indicates length):
      s:string (array of char); p: pascal string (with count byte).
    Special case (only available in native format):
      P:an integer type that is wide enough to hold a pointer.
    Special case (not in native mode unless 'long long' in platform C):
      q:long long; Q:unsigned long long
    Whitespace between formats is ignored.

大端模式：

字节的传输约定：按照B7~B0的顺序传输；
字的传输约定： 先传递高8位（B15~B8），再传递低8位，（B7~B0）；
双字的传输约定： 先传递高24位，（B31~B24），然后传递高16位（B23~B16），再传递高8位（B15~B8），最后传递低8位（B7~B0）。

>>> b  = struct.pack("!H",1)
>>> b
'\x00\x01'
>>> b  = struct.pack("H",1)
>>> b
'\x01\x00'
>>> b  = struct.pack("L",1)
>>> b
'\x01\x00\x00\x00\x00\x00\x00\x00'
>>> b  = struct.pack("!L",1)
>>> b
'\x00\x00\x00\x01'


格式化字符串：

# P     void *     long
>>> buffer = struct.pack("P",1)
>>> buffer
'\x01\x00\x00\x00\x00\x00\x00\x00'

# p     char[]     string
>>> buffer = struct.pack("p","string")
>>> buffer
'\x00'

# s     char[]     string
>>> buffer = struct.pack("ssss","string")
>>> buffer
's'
>>> buffer = struct.pack("sss","string","st","ff")
>>> buffer
'ssf'
>>> buffer = struct.pack("sss","string","j","ff")
>>> buffer
'sjf'
>>> buffer  = struct.pack("%ds"%len('jia xiaolei'),"jia xiaolei")
>>> buffer
'jia xiaolei'

# d     double     float
>>> buffer = struct.pack("d", 1.2)
>>> buffer
'333333\xf3?'

# f     float     float
>>> buffer = struct.pack("f", 1.2)
>>> buffer
'\x9a\x99\x99?'

#  这里很有用，Q 可以匹配几乎所有的东西 
Q     unsigned long long     long
>>> buffer = struct.pack("Q",25111)
>>> buffer
'\x17b\x00\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("Q",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'

# q     long long     long
>>> buffer = struct.pack("q",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("q",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'

# L     unsigned long     long

>>> buffer = struct.pack("L",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("L",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'

# l     long     integer
>>> buffer = struct.pack("l",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("l",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'
>>> buffer = struct.pack("l",47882545111)
>>> buffer
'\xd7\xa7\x05&\x0b\x00\x00\x00'
>>> buffer = struct.pack("l",2545111)
>>> buffer
'\xd7\xd5&\x00\x00\x00\x00\x00'

# I     unsigned int     integer or long

>>> buffer = struct.pack("I",2545111)
>>> buffer
'\xd7\xd5&\x00'
>>> buffer = struct.pack("I",47882545111)
__main__:1: DeprecationWarning: 'I' format requires 0 <= number <= 4294967295
#  如果传入的值的范围查过格式字符串说约定的范围，会报错

# i     int     integer
>>> buffer = struct.pack("i",5111)
>>> buffer
'\xf7\x13\x00\x00'
>>> buffer = struct.pack("i",775111)
>>> buffer
'\xc7\xd3\x0b\x00'

# H     unsigned short     integer
>>> buffer = struct.pack("H",775111)
Traceback (most recent call last):
  File "", line 1, in
struct.error: ushort format requires 0 <= number <= USHRT_MAX
>>> buffer = struct.pack("H",11)
>>> buffer
'\x0b\x00'
>>> buffer = struct.pack("H",711)
>>> buffer
'\xc7\x02'

# h     short     integer
>>> buffer = struct.pack("h",711)
>>> buffer
'\xc7\x02'
>>> buffer = struct.pack("",7811)
Traceback (most recent call last):
  File "", line 1, in
struct.error: pack requires exactly 0 arguments

# ?     _Bool     bool
>>> buffer = struct.pack("?",True)
>>> buffer
'\x01'
>>> buffer = struct.pack("?",False)
>>> buffer
'\x00'

# B     unsigned char     integer
>>> buffer = struct.pack("B",2)
>>> buffer
'\x02'
>>> buffer = struct.pack("B",15)
>>> buffer
'\x0f'

# b     signed char     integer
>>> buffer = struct.pack("b",15)
>>> buffer
'\x0f'
>>> buffer = struct.pack("b",1)
>>> buffer
'\x01'
>>> buffer = struct.pack("b",51)
>>> buffer
'3'
>>> buffer = struct.pack("b",51)
>>> buffer
'3'

# c     char     string of length 1
>>> buffer = struct.pack("c","1")
>>> buffer
'1'
>>> buffer = struct.pack("c","7")
>>> buffer
'7'

# x
>>> buffer = struct.pack("x",)
>>> buffer
'\x00'