CRC32/MPEG2

目录

  • 常见CRC参数模型
  • CRC32 0xEDB88320
  • CRC32MPEG2
  • crc在线计算

最近工作中需要使用crc32校验。硬件使用的CRC32-mpeg2算法,需要软件也使用相同的计算方法。这里我是需要使用python实现该算法。

常见CRC参数模型

在http://www.ip33.com/crc.html中有更多的参数模型,这里只列出两个crc32.

算法名称 多项式公式 宽度 多项式 POLY 初始值 INIT 结果异或值 XOROUT 输入数据反转(REFIN) 输出数据反转(REFOUT)
CRC-32 x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + x2 + x + 1 32 04C11DB7 FFFFFFFF FFFFFFFF true true
CRC-32/MPEG-2 x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + x2 + x + 1 32 04C11DB7 FFFFFFFF 00000000 false false

CRC32 0xEDB88320

之前我在python中调用的校验接口是binascii.crc32。参考链接利用Python计算单片机bin文件的CRC32值
binascii.crc32从初始crc开始计算32位数据校验和。这与ZIP文件校验和是一致的。由于该算法是为校验和算法而设计的,所以不适合作为一般的哈希算法。https://docs.python.org/2/library/binascii.html

下面两个是在网上找到的计算方法,有源代码,一种是查找法,一种是直接计算。
参考链接:
CRC32 直接计算方法
CRC32 0xEDB88320

CRC32MPEG2

在网上找的一个mpeg2的python计算方法(python CRC32-mpeg2校验),但是计算结果与网页计算结果不一致。我没有去研究计算方法,有懂得网友可以评论一下。

后来在github上找到了另一份代码是直接通过计算得到crc32的计算结果的,后来回头找链接没有找到,很遗憾。该代码功能强大,现把代码贴上来。

"""Generic CRC implementation with many pre-defined CRC models.
This module is the adopted Python implementation of the JavaScript CRC
implementation by Bastian Molkenthin:
    http://www.sunshine2k.de/coding/javascript/crc/crc_js.html
Copyright (c) 2015 Bastian Molkenthin
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
Usage:
    .. code:: python
        >>> from crc import crc32
        >>> checksum = crc32(b"Hello world")
        >>> hex(checksum)
        >>> '0x1b851995'
Tests:
    There are some very small unit tests that can be executed with ``pytest``:
    .. code:: bash
        pytest path/to/crc.py
"""

import array


def reflect(num, width):
    """Reverts bit order of the given number
    Args:
        num (int): Number that should be reflected
        width (int): Size of the number in bits
    """
    reflected = 0

    for i in range(width):
        if (num >> i) & 1 != 0:
            reflected |= 1 << (width - 1 - i)

    return reflected


def make_table(width):
    """Create static sized CRC lookup table and initialize it with ``0``.
    For 8, 16, 32, and 64 bit width :class:`array.array` instances are used. For
    all other widths it falls back to a generic Python :class:`list`.
    Args:
        width (int): Size of elements in bits
    """
    initializer = (0 for _ in range(256))

    if width == 8:
        return array.array('B', initializer)
    elif width == 16:
        return array.array('H', initializer)
    elif width == 32:
        return array.array('L', initializer)
    elif width == 64:
        return array.array('Q', initializer)
    else:
        # Fallback to a generic list
        return list(initializer)


class CRC(object):
    """Generic CRC model implemented with lookup tables.
    The model parameter can are the constructor parameters.
    Args:
        width (int): Number of bits of the polynomial
        polynomial (int): CRC polynomial
        initial_value (int): Initial value of the checksum
        final_xor_value (int): Value that will be XOR-ed with final checksum
        input_reflected (bool): True, if each input byte should be reflected
        result_reflected (bool): True, if the result should be reflected before
            the final XOR is applied
    Usage:
        .. code:: python
            from crc import CRC
            # Definition  of CRC-32 Ethernet. The crc module defines many common
            # models already.
            crc = CRC(32, 0x04c11db7, 0xffffffff, 0xffffffff, True, True)
            # You can call the model to calculate the CRC checksum of byte
            # string
            assert crc(b"Hello world!") == 0x1b851995
    """
    def __init__(self, width, polynomial, initial_value, final_xor_value,
                 input_reflected, result_reflected):
        self.width = width
        self.polynomial = polynomial
        self.initial_value = initial_value
        self.final_xor_value = final_xor_value
        self.input_reflected = input_reflected
        self.result_reflected = result_reflected

        # Initialize casting mask to keep the correct width for dynamic Python
        # integers
        self.cast_mask = int('1' * self.width, base=2)

        # Mask that can be applied to get the Most Significant Bit (MSB) if the
        # number with given width
        self.msb_mask = 0x01 << (self.width - 1)

        # The lookup tables get initialized lazzily. This ensures that only
        # tables are calculated that are actually needed.
        self.table = None
        self.reflected_table = None

    def __call__(self, value):
        """Compute the CRC checksum with respect to the model parameters by using
        a looup table algorithm.
        Args:
            value (bytes): Input bytes that should be checked
        Returns:
            int - CRC checksum
        """
        # Use the reflection optimization if applicable
        if self.input_reflected and self.result_reflected:
            return self.fast_reflected(value)

        # Lazy initialization of the lookup table
        if self.table is None:
            self.table = self.calculate_crc_table()

        crc = self.initial_value

        for cur_byte in value:
            if self.input_reflected:
                cur_byte = reflect(cur_byte, 8)

            # Update the MSB of the CRC value with the next input byte
            crc = (crc ^ (cur_byte << (self.width - 8))) & self.cast_mask

            # This MSB byte value is the index into the lookup table
            index = (crc >> (self.width - 8)) & 0xff

            # Shift out the index
            crc = (crc << 8) & self.cast_mask

            # XOR-ing crc from the lookup table using the calculated index
            crc = crc ^ self.table[index]

        if self.result_reflected:
            crc = reflect(crc, self.width)

        # Final XBOR
        return crc ^ self.final_xor_value

    def fast_reflected(self, value):
        """If the input data and the result checksum are both reflected in the
        current model, an optimized algorithm can be used that reflects the
        looup table rather then the input data. This saves the reflection
        operation of the input data.
        """
        if not self.input_reflected or not self.result_reflected:
            raise ValueError("Input and result must be reflected")

        # Lazy initialization of the lookup table
        if self.reflected_table is None:
            self.reflected_table = self.calculate_crc_table_reflected()

        crc = self.initial_value

        for cur_byte in value:
            # The LSB of the XOR-red remainder and the next byte is the index
            # into the lookup table
            index = (crc & 0xff) ^ cur_byte

            # Shift out the index
            crc = (crc >> 8) & self.cast_mask

            # XOR-ing remainder from the loopup table
            crc = crc ^ self.reflected_table[index]

        # Final XBOR
        return crc ^ self.final_xor_value


    def calculate_crc_table(self):
        table = make_table(self.width)

        for divident in range(256):
            cur_byte = (divident << (self.width - 8)) & self.cast_mask

            for bit in range(8):
                if (cur_byte & self.msb_mask) != 0:
                    cur_byte <<= 1
                    cur_byte ^= self.polynomial
                else:
                    cur_byte <<= 1

            table[divident] = cur_byte & self.cast_mask

        return table

    def calculate_crc_table_reflected(self):
        table = make_table(self.width)

        for divident in range(256):
            reflected_divident = reflect(divident, 8)
            cur_byte = (reflected_divident << (self.width - 8)) & self.cast_mask

            for bit in range(8):
                if (cur_byte & self.msb_mask) != 0:
                    cur_byte <<= 1
                    cur_byte ^= self.polynomial
                else:
                    cur_byte <<= 1

            cur_byte = reflect(cur_byte, self.width)

            table[divident] = (cur_byte & self.cast_mask)

        return table


# Known CRC algorihtms
crc8                = CRC(8, 0x07, 0x00, 0x00, False, False)
crc8_sae_j1850      = CRC(8, 0x1d, 0xff, 0xff, False, False)
crc8_sae_j1850_zero = CRC(8, 0x1d, 0x00, 0x00, False, False)
crc8_8h2f           = CRC(8, 0x2f, 0xff, 0xff, False, False)
crc8_cdma2000       = CRC(8, 0x9b, 0xff, 0x00, False, False)
crc8_darc           = CRC(8, 0x39, 0x00, 0x00, True, True)
crc8_dvb_s2         = CRC(8, 0xd5, 0x00, 0x00, False, False)
crc8_ebu            = CRC(8, 0x1d, 0xff, 0x00, True, True)
crc8_icode          = CRC(8, 0x1d, 0xfd, 0x00, False, False)
crc8_itu            = CRC(8, 0x07, 0x00, 0x55, False, False)
crc8_maxim          = CRC(8, 0x31, 0x00, 0x00, True, True)
crc8_rohc           = CRC(8, 0x07, 0xff, 0x00, True, True)
crc8_wcdma          = CRC(8, 0x9b, 0x00, 0x00, True, True)

crc16_ccit_zero     = CRC(16, 0x1021, 0x0000, 0x0000, False, False)
crc16_arc           = CRC(16, 0x8005, 0x0000, 0x0000, True, True)
crc16_aug_ccitt     = CRC(16, 0x1021, 0x1d0f, 0x0000, False, False)
crc16_buypass       = CRC(16, 0x8005, 0x0000, 0x0000, False, False)
crc16_ccitt_false   = CRC(16, 0x1021, 0xffff, 0x0000, False, False)
crc16_cdma2000      = CRC(16, 0xc867, 0xffff, 0x0000, False, False)
crc16_dds_110       = CRC(16, 0x8005, 0x800d, 0x0000, False, False)
crc16_dect_r        = CRC(16, 0x0589, 0x0000, 0x0001, False, False)
crc16_dect_x        = CRC(16, 0x0589, 0x0000, 0x0000, False, False)
crc16_dnp           = CRC(16, 0x3d65, 0x0000, 0xffff, True, True)
crc16_en_13757      = CRC(16, 0x3d65, 0x0000, 0xffff, False, False)
crc16_genibus       = CRC(16, 0x1021, 0xffff, 0xffff, False, False)
crc16_maxim         = CRC(16, 0x8005, 0x0000, 0xffff, True, True)
crc16_mcrf4xx       = CRC(16, 0x1021, 0xffff, 0x0000, True, True)
crc16_riello        = CRC(16, 0x1021, 0xb2aa, 0x0000, True, True)
crc16_t10_dif       = CRC(16, 0x8bb7, 0x0000, 0x0000, False, False)
crc16_teledisk      = CRC(16, 0xa097, 0x0000, 0x0000, False, False)
crc16_tms37157      = CRC(16, 0x1021, 0x89ec, 0x0000, True, True)
crc16_usb           = CRC(16, 0x8005, 0xffff, 0xffff, True, True)
crc16_a             = CRC(16, 0x1021, 0xc6c6, 0x0000, True, True)
crc16_kermit        = CRC(16, 0x1021, 0x0000, 0x0000, True, True)
crc16_modbus        = CRC(16, 0x8005, 0xffff, 0x0000, True, True)
crc16_x25           = CRC(16, 0x1021, 0xffff, 0xffff, True, True)
crc16_xmodem        = CRC(16, 0x1021, 0x0000, 0x0000, False, False)

crc32               = CRC(32, 0x04c11db7, 0xffffffff, 0xffffffff, True, True)
crc32_bzip2         = CRC(32, 0x04c11db7, 0xffffffff, 0xffffffff, False, False)
crc32_c             = CRC(32, 0x1edc6f41, 0xffffffff, 0xffffffff, True, True)
crc32_d             = CRC(32, 0xa833982b, 0xffffffff, 0xffffffff, True, True)
crc32_mpeg2         = CRC(32, 0x04c11db7, 0xffffffff, 0x00000000, False, False)
crc32_posix         = CRC(32, 0x04c11db7, 0x00000000, 0xffffffff, False, False)
crc32_q             = CRC(32, 0x814141ab, 0x00000000, 0x00000000, False, False)
crc32_jamcrc        = CRC(32, 0x04c11db7, 0xffffffff, 0x00000000, True, True)
crc32_xfer          = CRC(32, 0x000000af, 0x00000000, 0x00000000, False, False)


# ----------
# Unit tests
# ----------

def test_crc16_cc():
    assert crc16_ccit_zero(b'Hello world!') ==  0x39db

def test_crc16_x25():
    assert crc16_x25(b'Hello world!') ==  0x8edb

def test_crc32():
    assert crc32(b'Hello world!') == 0x1b851995

crc在线计算

https://crccalc.com
http://www.ip33.com/crc.html

你可能感兴趣的:(CRC32/MPEG2)