Fuzz进阶教学——基于机器学习的模糊测试相关工作

【参考文献】[1]王鹃,张冲,龚家新等.基于机器学习的模糊测试研究综述[J].信息网络安全,2023,23(08):1-16.

---

目录

 一、机器学习在测试用例生成中的应用

1、文件解析软件的测试用例生成

2、网络协议的测试用例生成

3、代码解析工具的测试用例生成

二、机器学习在测试用例变异中的应用

三、机器学习在种子调度与筛选中的应用

1、种子调度

2、测试用例筛选

 四、 基于机器学习的其他模糊测试

---

 一、机器学习在测试用例生成中的应用

1、文件解析软件的测试用例生成

• 不同的文件格式需要不同的文件解析应用软件打开，如果这些文件解析应用存在缺陷和漏洞，轻则会错误地解析文件信息，导致无法正确处理文件内容，重则会导致各种安全问题。例如，攻击者可以利用文件解析软件的漏洞窃取、篡改隐私数据。
• 针对文件解析应用的模糊测试经常使用基于语法的模糊测试，根据特定文件格式编写文件语法模型，并根据构造的模型生成测试用例。由于不同的文件格式差别较大，大 多文件格式拥有复杂的语法结构，导致人为分析构造正确的文件语法相当耗时费力，并且构造的文件语法模型的效果依赖于专家经验。
• 
• 【26】GODEFROID P, PELEG H, SINGH R. Learn&Fuzz: Machine Learning for Input Fuzzing[C]//IEEE. 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). New York: IEEE, 2017: 50-59.
• 【27】NASRABADI M Z, PARSA S, KALAEE A. Format-Aware Learn&Fuzz: Deep Test Data Generation for Efficient Fuzzing[J]. Neural Computing and Applications, 2021, 33(5): 1497-1513.
• 【28】SABLOTNY M, JENSEN B S, JOHNSON C W. Recurrent Neural Networks for Fuzz Testing Web Browsers[C]//Springer. Information Security and Cryptology-ICISC 2018. Berlin: Springer, 2019: 354-370.
• 【29】PADURARU C, MELEMCIUC M C. An Automatic Test Data Generation Tool Using Machine Learning[EB/OL]. (2018-10-16)[2022- 10-07]. https://pdfs.semanticscholar.org/82bd/ef8522578768ffa161a255eb9ce68b99e371. pdf.
• 【30】JITSUNARI Y, ARAHORI Y. Coverage-Guided Learning-Assisted Grammar-Based Fuzzing[C]//IEEE. 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). New York: IEEE, 2019: 275-280.
• 【31】CHENG Liang, ZHANG Yang, ZHANG Yi, et al. Optimizing Seed Inputs in Fuzzing with Machine Learning[C]//IEEE. 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). New York: IEEE, 2019: 244-245.

2、网络协议的测试用例生成

• 为了提供实时可控的通信，很多协议格式设计为未加密，而且不同的协议格式不同，需要对应的协议解析工具处理协议报文。开发人员在应用和解析这些网络协议时，可能出现逻辑错误，这对网络系统的安全稳定造成巨大威胁。
• 模糊测试可以对目标协议解析软件进行测试，试图找出软件中的实时错误和异常，常用的协议模糊测试同样是基于语法指导生成的。而这些语法大多通过解释协议文档或对网络行为进行逆 向工程来构建，其构造难度较高，且正确性无法保证。基于机器学习的语法生成为自动协议语法生成的发展带来了新的可能。
• 
• 【32】FAN Rong, CHANG Yaoyao. Machine Learning for BlackBox Fuzzing of Network Protocols[C]//Springer. Information and Communications Security-ICICS 2017. Berlin: Springer, 2018: 621-632.
• 【33】GAO Zicong, DONG Weiyu, CHANG Rui, et al. The Stacked Seq2Seq-Attention Model for Protocol Fuzzing[C]//IEEE. 2019 IEEE 7th International Conference on Computer Science and Network Technology (ICCSNT). New York: IEEE, 2019: 126-130. 
• 【34】LAI Yingxu, GAO Huijuan, LIU Jing. Vulnerability Mining Method for the Modbus TCP Using an Anti-Sample Fuzzer[J]. Sensors, 2020, 20(7): 20-40.
• 【35】ZHAO Hui, LI Zhihui, WEI Hansheng, et al. SeqFuzzer: An Industrial Protocol Fuzzing Framework from a Deep Learning Perspective[C]//IEEE. 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST). New York: IEEE, 2019: 59-67.
• 【36】HU Zhicheng, SHI Jianqi, HUANG Yanhong, et al. GANFuzz: A GAN-Based Industrial Network Protocol Fuzzing Framework[C]//ACM. Proceedings of the 15th ACM International Conference on Computing Frontiers. New York: ACM, 2018: 138-145.

3、代码解析工具的测试用例生成

• 编译器和解释器是较为重要的计算机系统软件，它们仍然存在一些缺陷，可能导致意外的程序执行，甚至影响应用的安全性。因此，如何对编译器和解释器进行测试、验证编译器的正确性，一直是研究人员 比较重视的问题。编译器和解释器等代码解析工具的测试用例就是对应语言的代码文件，这些文件不只是简单的文本文件，还具有复杂的编程语法和代码逻辑，因此生成这些编程语言代码文件存在一定的难度。
• 
• 【37】CUMMINS C, PETOUMENOS P, MURRAY A, et al. Compiler Fuzzing Through Deep Learning[C]//ACM. Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. New York: ACM, 2018: 95-105.
• 【38】LIU Xiao, LI Xiaoting, PRAJAPATI R, et al. DeepFuzz: Automatic Generation of Syntax Valid C Programs for Fuzz Testing[C]//ACM. Proceedings of the AAAI Conference on Artificial Intelligence. New York: ACM, 2019: 1044-1051.
• 【39】LEE S, HAN H S, CHA S K, et al. Montage: A Neural Network Language Model-Guided Javascript Engine Fuzzer[C]//USENIX. 29th USENIX Security Symposium. Berkley: USENIX, 2020: 2613-2630.
• 【40】YE Guixin, TANG Zhanyong, TAN S H, et al. Automated Conformance Testing for Javascript Engines via Deep Compiler Fuzzing[C]// ACM. Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation. New York: ACM, 2021: 435-450.

---

二、机器学习在测试用例变异中的应用

• 从理论上讲，所有测试用例生成方法都可以分为基于生成、基于变异以及结合生成和变异的方法。基于语法格式直接生成一个新的测试用例的方法存在生成难度高、效率低的问题。因此，基于变异的测试用例生成方法也是模糊测试研究中的一个重要方向。
• 传统测试用例变异方法经常是随机的，模糊测试处理器一般会平等地对待被测程序的每个部分，导致测试用例朝着无序的方向变异。而这种方式得到的测试用例大多是无意义的，无法触发异常或者漏洞，导致模糊测试效率很低。
• 因此，本文通过基于机器学习技术的模糊测试用例变异方法来学习程序特征和变异策略，指导变异过程生成更有意义的测试用例，或者缩小变异范围并提高变异效率。
• 
• 【44】RAJPAL M, BLUM W, SINGH R. Not All Bytes are Equal: Neural Byte Sieve for Fuzzing[EB/OL]. (2017-11-10)[2022-10-07]. https://arxiv. org/abs/1711.04596.
• 【45】SHE Dongdong, PEI Kexin, EPSTEIN D, et al. Neuzz: Efficient Fuzzing with Neural Program Smoothing[C]//IEEE. 2019 IEEE Symposium on Security and Privacy (SP). New York: IEEE, 2019: 803-817.
• 【46】SHE Dongdong, RAHUL K, YAN Lu, et al. MTFuzz: Fuzzing with a Multi-Task Neural Network[C]//ACM. Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020). New York: ACM, 2020: 737-749.
• 【47】WU Mingyuan, JIANG Ling, XIANG Jiahong, et al. Evaluating and Improving Neural Program-Smoothing-Based Fuzzing[C]//IEEE. 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE). New York: IEEE, 2022: 847-858.
• 【48】KARAMCHETI S, MANN G, ROSENBERG D. Adaptive GreyBox Fuzz-Testing with Thompson Sampling[C]//ACM. Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security. New York: ACM, 2018: 37-47.
• 【49】DROZD W, MICHAEL D W. FuzzerGym: A Competitive Framework for Fuzzing and Learning[EB/OL]. (2018-07-19)[2022-10-07]. https://arxiv. org/abs/1807.07490.
• 【50】BOTTINGER K, GODEFROID P, SINGH R. Deep Reinforcement Fuzzing[C]//IEEE. 2018 IEEE Security and Privacy Workshops (SPW). New York: IEEE, 2018: 116-122.
• 【51】WU Mingyuan, JIANG Ling, XIANG Jiahong, et al. One Fuzzing Strategy to Rule Them All[C]//IEEE. 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE). New York: IEEE, 2022: 1634- 1645.

---

三、机器学习在种子调度与筛选中的应用

1、种子调度

• 为了提高模糊测试效率，当前的大多数研究集中在对输入进行突变以增大完成目标的可能性上，但是在进行变异的种子调度方面做的工作很少。
• 在现有模糊测试过程中，常用的种子调度是通过固定的启发式算法来预测种子的效用，但是这种启发式算法不可泛化，因此无法适用于所有程序。基于机器学习的模糊测试调度与传统的调度过程不同，该方法能够挖掘出目标程序和待测种子的深层特征，从而发掘出种子在应用程序中触发新漏洞或者新代码路径的能力。机器学习模型通过分析目标应用或以往调度过程来获取可 以指导种子调度的信息。
• 
• 【52】ZHAO Yuyue, LI Yangyang, YANG Tengfei, et al. Suzzer: A Vulnerability-Guided Fuzzer Based on Deep Learning[C]//Springer. Information Security and Cryptology: 15th International Conference. Berlin: Springer, 2020: 134-153.
• 【53】LI Yuwei, JI Shouling, LYU Chenyang, et al. V-Fuzz: Vulnerability Prediction-Assisted Evolutionary Fuzzing for Binary Programs[J]. IEEE Transactions on Cybernetics, 2020, 52(5): 3745-3756.
• 【54】WANG Yunchao, WU Zehui, WEI Qiang, et al. NeuFuzz: Efficient Fuzzing with Deep Neural Network[J]. IEEE Access, 2019(7): 36340-36352.
• 【55】CHEN Yaohui, AHMADI M, FARKHANI R M, et al. MEUZZ: Smart Seed Scheduling for Hybrid Fuzzing[C]//USENIX. 23rd International Symposium on Research in Attacks, Intrusions and Defenses ({RAID} 2020). Berkley: USENIX, 2020: 77-92.
• 【56】WANG Jinghan, SONG Chengyu, YIN Heng. Reinforcement Learning-Based Hierarchical Seed Scheduling for Greybox Fuzzing[EB/OL]. (2021-01-01)[2022-10-07]. https://escholarship.org/uc/item/44p2v1gd.
• 【57】WANG Daimeng, ZHANG Zheng, ZHANG Hang, et al. SyzVegas:Beating Kernel Fuzzing Odds with Reinforcement Learning[C]// USENIX. 30th USENIX Security Symposium. Berkley: USENIX, 2021: 2741-2758.

2、测试用例筛选

• 在模糊测试过程中能够产生大量测试用例，其中并非所有测试用例都能触发异常或者覆盖新代码位置，是否能够有效减少变异和测试用例数量，对于提升模糊测试效率很重要。
• 较常见的方式是训练一个机器学习模型并将其作为分类器，预测待执行的测试用例是否可以触发新的覆盖率或者异常。但是基于不同的阶段和角度，机器学习模型应用的细节不同。
• 
• 【58】SUN Xiaoshan, FU Yu, DONG Yun, et al. Improving Fitness Function for Language Fuzzing with PCFG Model[C]//IEEE. 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC). New York: IEEE, 2018: 655-660.
• 【59】SANJEEV D, KEDRIAN J, JAN W, et al. A Flexible Framework for Expediting Bug Finding by Leveraging Past (Mis-)Behavior to Discover New Bugs[C]//ACM. Annual Computer Security Applications Conference (ACSAC’20). New York: ACM, 2020: 345-359.
• 【60】GONG Weiwei, ZHANG Gen, ZHOU Xu. Learn to Accelerate Identifying New Test Cases in Fuzzing[C]//Springer. International Conference on Security, Privacy and Anonymity in Computation, Communication and Storage. Berlin: Springer, 2017: 298-307.
• 【61】JOFFE L. Machine Learning Augmented Fuzzing[C]//IEEE. 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). New York: IEEE, 2018: 178-183.
• 【62】JOFFE L, CLARK D. Directing a Search Towards Execution Properties with a Learned Fitness Function[C]//IEEE. 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST). New York: IEEE, 2019: 206-216.
• 【63】ZONG Peiyuan, LYU Tao, WANG Dawei, et al. Fuzzguard: Filtering out Unreachable Inputs in Directed Grey-Box Fuzzing Through Deep Learning[C]//USENIX. 29th USENIX Security Symposium. Berkley: USENIX, 2020: 2255-2269.

---

 四、 基于机器学习的其他模糊测试

• 
• 【64】MING Liang, ZHAO Gang, HUANG Minhuan, et al. Remote Protocol Vulnerability Discovery for Intelligent Transportation Systems (ITS)[C]//IEEE. 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC). New York: IEEE, 2018: 923-929.
• 【65】TRIPATHI S, GRIECO G, RAWAT S. Exniffer: Learning to Prioritize Crashes by Assessing the Exploitability from Memory Dump[C]// IEEE. 2017 24th Asia-Pacific Software Engineering Conference (APSEC). New York: IEEE, 2017: 239-248.
• 【66】ZHANG Li, THING V. Assisting Vulnerability Detection by Prioritizing Crashes with Incremental Learning[C]//IEEE. TENCON 2018 IEEE Region 10 Conference. New York: IEEE, 2018: 2080-2085.
• 【67】YAN Guanhua, LU Junchen, SHU Zhan, et al. Exploitmeter: Combining Fuzzing with Machine Learning for Automated Evaluation of Software Exploitability[C]//IEEE. 2017 IEEE Symposium on Privacy-Aware Computing (PAC). New York: IEEE, 2017: 164-175. 
• 【68】YOU Wei, ZONG Peiyuan, CHEN Kai, et al. Semfuzz: SemanticsBased Automatic Generation of Proof-of-Concept Exploits[C]//ACM. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2017: 2139-2154.