CRC32/MPEG2
目录
- 常见CRC参数模型
- CRC32 0xEDB88320
- CRC32MPEG2
- crc在线计算
最近工作中需要使用crc32校验。硬件使用的CRC32-mpeg2算法，需要软件也使用相同的计算方法。这里我是需要使用python实现该算法。
常见CRC参数模型
在http://www.ip33.com/crc.html中有更多的参数模型，这里只列出两个crc32.
| 算法名称 | 多项式公式 | 宽度 | 多项式 POLY | 初始值 INIT | 结果异或值 XOROUT | 输入数据反转(REFIN) | 输出数据反转(REFOUT) |
|---|---|---|---|---|---|---|---|
| CRC-32 | x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + x2 + x + 1 | 32 | 04C11DB7 | FFFFFFFF | FFFFFFFF | true | true |
| CRC-32/MPEG-2 | x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + x2 + x + 1 | 32 | 04C11DB7 | FFFFFFFF | 00000000 | false | false |
CRC32 0xEDB88320
之前我在python中调用的校验接口是binascii.crc32。参考链接利用Python计算单片机bin文件的CRC32值
binascii.crc32从初始crc开始计算32位数据校验和。这与ZIP文件校验和是一致的。由于该算法是为校验和算法而设计的，所以不适合作为一般的哈希算法。https://docs.python.org/2/library/binascii.html
下面两个是在网上找到的计算方法，有源代码，一种是查找法，一种是直接计算。
参考链接：
CRC32 直接计算方法
CRC32 0xEDB88320
CRC32MPEG2
在网上找的一个mpeg2的python计算方法（python CRC32-mpeg2校验），但是计算结果与网页计算结果不一致。我没有去研究计算方法，有懂得网友可以评论一下。
后来在github上找到了另一份代码是直接通过计算得到crc32的计算结果的，后来回头找链接没有找到，很遗憾。该代码功能强大，现把代码贴上来。
"""Generic CRC implementation with many pre-defined CRC models.
This module is the adopted Python implementation of the JavaScript CRC
implementation by Bastian Molkenthin:
http://www.sunshine2k.de/coding/javascript/crc/crc_js.html
Copyright (c) 2015 Bastian Molkenthin
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
Usage:
.. code:: python
>>> from crc import crc32
>>> checksum = crc32(b"Hello world")
>>> hex(checksum)
>>> '0x1b851995'
Tests:
There are some very small unit tests that can be executed with ``pytest``:
.. code:: bash
pytest path/to/crc.py
"""
import array
def reflect(num, width):
"""Reverts bit order of the given number
Args:
num (int): Number that should be reflected
width (int): Size of the number in bits
"""
reflected = 0
for i in range(width):
if (num >> i) & 1 != 0:
reflected |= 1 << (width - 1 - i)
return reflected
def make_table(width):
"""Create static sized CRC lookup table and initialize it with ``0``.
For 8, 16, 32, and 64 bit width :class:`array.array` instances are used. For
all other widths it falls back to a generic Python :class:`list`.
Args:
width (int): Size of elements in bits
"""
initializer = (0 for _ in range(256))
if width == 8:
return array.array('B', initializer)
elif width == 16:
return array.array('H', initializer)
elif width == 32:
return array.array('L', initializer)
elif width == 64:
return array.array('Q', initializer)
else:
# Fallback to a generic list
return list(initializer)
class CRC(object):
"""Generic CRC model implemented with lookup tables.
The model parameter can are the constructor parameters.
Args:
width (int): Number of bits of the polynomial
polynomial (int): CRC polynomial
initial_value (int): Initial value of the checksum
final_xor_value (int): Value that will be XOR-ed with final checksum
input_reflected (bool): True, if each input byte should be reflected
result_reflected (bool): True, if the result should be reflected before
the final XOR is applied
Usage:
.. code:: python
from crc import CRC
# Definition of CRC-32 Ethernet. The crc module defines many common
# models already.
crc = CRC(32, 0x04c11db7, 0xffffffff, 0xffffffff, True, True)
# You can call the model to calculate the CRC checksum of byte
# string
assert crc(b"Hello world!") == 0x1b851995
"""
def __init__(self, width, polynomial, initial_value, final_xor_value,
input_reflected, result_reflected):
self.width = width
self.polynomial = polynomial
self.initial_value = initial_value
self.final_xor_value = final_xor_value
self.input_reflected = input_reflected
self.result_reflected = result_reflected
# Initialize casting mask to keep the correct width for dynamic Python
# integers
self.cast_mask = int('1' * self.width, base=2)
# Mask that can be applied to get the Most Significant Bit (MSB) if the
# number with given width
self.msb_mask = 0x01 << (self.width - 1)
# The lookup tables get initialized lazzily. This ensures that only
# tables are calculated that are actually needed.
self.table = None
self.reflected_table = None
def __call__(self, value):
"""Compute the CRC checksum with respect to the model parameters by using
a looup table algorithm.
Args:
value (bytes): Input bytes that should be checked
Returns:
int - CRC checksum
"""
# Use the reflection optimization if applicable
if self.input_reflected and self.result_reflected:
return self.fast_reflected(value)
# Lazy initialization of the lookup table
if self.table is None:
self.table = self.calculate_crc_table()
crc = self.initial_value
for cur_byte in value:
if self.input_reflected:
cur_byte = reflect(cur_byte, 8)
# Update the MSB of the CRC value with the next input byte
crc = (crc ^ (cur_byte << (self.width - 8))) & self.cast_mask
# This MSB byte value is the index into the lookup table
index = (crc >> (self.width - 8)) & 0xff
# Shift out the index
crc = (crc << 8) & self.cast_mask
# XOR-ing crc from the lookup table using the calculated index
crc = crc ^ self.table[index]
if self.result_reflected:
crc = reflect(crc, self.width)
# Final XBOR
return crc ^ self.final_xor_value
def fast_reflected(self, value):
"""If the input data and the result checksum are both reflected in the
current model, an optimized algorithm can be used that reflects the
looup table rather then the input data. This saves the reflection
operation of the input data.
"""
if not self.input_reflected or not self.result_reflected:
raise ValueError("Input and result must be reflected")
# Lazy initialization of the lookup table
if self.reflected_table is None:
self.reflected_table = self.calculate_crc_table_reflected()
crc = self.initial_value
for cur_byte in value:
# The LSB of the XOR-red remainder and the next byte is the index
# into the lookup table
index = (crc & 0xff) ^ cur_byte
# Shift out the index
crc = (crc >> 8) & self.cast_mask
# XOR-ing remainder from the loopup table
crc = crc ^ self.reflected_table[index]
# Final XBOR
return crc ^ self.final_xor_value
def calculate_crc_table(self):
table = make_table(self.width)
for divident in range(256):
cur_byte = (divident << (self.width - 8)) & self.cast_mask
for bit in range(8):
if (cur_byte & self.msb_mask) != 0:
cur_byte <<= 1
cur_byte ^= self.polynomial
else:
cur_byte <<= 1
table[divident] = cur_byte & self.cast_mask
return table
def calculate_crc_table_reflected(self):
table = make_table(self.width)
for divident in range(256):
reflected_divident = reflect(divident, 8)
cur_byte = (reflected_divident << (self.width - 8)) & self.cast_mask
for bit in range(8):
if (cur_byte & self.msb_mask) != 0:
cur_byte <<= 1
cur_byte ^= self.polynomial
else:
cur_byte <<= 1
cur_byte = reflect(cur_byte, self.width)
table[divident] = (cur_byte & self.cast_mask)
return table
# Known CRC algorihtms
crc8 = CRC(8, 0x07, 0x00, 0x00, False, False)
crc8_sae_j1850 = CRC(8, 0x1d, 0xff, 0xff, False, False)
crc8_sae_j1850_zero = CRC(8, 0x1d, 0x00, 0x00, False, False)
crc8_8h2f = CRC(8, 0x2f, 0xff, 0xff, False, False)
crc8_cdma2000 = CRC(8, 0x9b, 0xff, 0x00, False, False)
crc8_darc = CRC(8, 0x39, 0x00, 0x00, True, True)
crc8_dvb_s2 = CRC(8, 0xd5, 0x00, 0x00, False, False)
crc8_ebu = CRC(8, 0x1d, 0xff, 0x00, True, True)
crc8_icode = CRC(8, 0x1d, 0xfd, 0x00, False, False)
crc8_itu = CRC(8, 0x07, 0x00, 0x55, False, False)
crc8_maxim = CRC(8, 0x31, 0x00, 0x00, True, True)
crc8_rohc = CRC(8, 0x07, 0xff, 0x00, True, True)
crc8_wcdma = CRC(8, 0x9b, 0x00, 0x00, True, True)
crc16_ccit_zero = CRC(16, 0x1021, 0x0000, 0x0000, False, False)
crc16_arc = CRC(16, 0x8005, 0x0000, 0x0000, True, True)
crc16_aug_ccitt = CRC(16, 0x1021, 0x1d0f, 0x0000, False, False)
crc16_buypass = CRC(16, 0x8005, 0x0000, 0x0000, False, False)
crc16_ccitt_false = CRC(16, 0x1021, 0xffff, 0x0000, False, False)
crc16_cdma2000 = CRC(16, 0xc867, 0xffff, 0x0000, False, False)
crc16_dds_110 = CRC(16, 0x8005, 0x800d, 0x0000, False, False)
crc16_dect_r = CRC(16, 0x0589, 0x0000, 0x0001, False, False)
crc16_dect_x = CRC(16, 0x0589, 0x0000, 0x0000, False, False)
crc16_dnp = CRC(16, 0x3d65, 0x0000, 0xffff, True, True)
crc16_en_13757 = CRC(16, 0x3d65, 0x0000, 0xffff, False, False)
crc16_genibus = CRC(16, 0x1021, 0xffff, 0xffff, False, False)
crc16_maxim = CRC(16, 0x8005, 0x0000, 0xffff, True, True)
crc16_mcrf4xx = CRC(16, 0x1021, 0xffff, 0x0000, True, True)
crc16_riello = CRC(16, 0x1021, 0xb2aa, 0x0000, True, True)
crc16_t10_dif = CRC(16, 0x8bb7, 0x0000, 0x0000, False, False)
crc16_teledisk = CRC(16, 0xa097, 0x0000, 0x0000, False, False)
crc16_tms37157 = CRC(16, 0x1021, 0x89ec, 0x0000, True, True)
crc16_usb = CRC(16, 0x8005, 0xffff, 0xffff, True, True)
crc16_a = CRC(16, 0x1021, 0xc6c6, 0x0000, True, True)
crc16_kermit = CRC(16, 0x1021, 0x0000, 0x0000, True, True)
crc16_modbus = CRC(16, 0x8005, 0xffff, 0x0000, True, True)
crc16_x25 = CRC(16, 0x1021, 0xffff, 0xffff, True, True)
crc16_xmodem = CRC(16, 0x1021, 0x0000, 0x0000, False, False)
crc32 = CRC(32, 0x04c11db7, 0xffffffff, 0xffffffff, True, True)
crc32_bzip2 = CRC(32, 0x04c11db7, 0xffffffff, 0xffffffff, False, False)
crc32_c = CRC(32, 0x1edc6f41, 0xffffffff, 0xffffffff, True, True)
crc32_d = CRC(32, 0xa833982b, 0xffffffff, 0xffffffff, True, True)
crc32_mpeg2 = CRC(32, 0x04c11db7, 0xffffffff, 0x00000000, False, False)
crc32_posix = CRC(32, 0x04c11db7, 0x00000000, 0xffffffff, False, False)
crc32_q = CRC(32, 0x814141ab, 0x00000000, 0x00000000, False, False)
crc32_jamcrc = CRC(32, 0x04c11db7, 0xffffffff, 0x00000000, True, True)
crc32_xfer = CRC(32, 0x000000af, 0x00000000, 0x00000000, False, False)
# ----------
# Unit tests
# ----------
def test_crc16_cc():
assert crc16_ccit_zero(b'Hello world!') == 0x39db
def test_crc16_x25():
assert crc16_x25(b'Hello world!') == 0x8edb
def test_crc32():
assert crc32(b'Hello world!') == 0x1b851995
crc在线计算
https://crccalc.com
http://www.ip33.com/crc.html