Recently one of my projects is related to concurrent attack detector and analyzer. "syzkaller" is a famous linux kernel fuzzer built by Google. In this article, you're going to see the installation and usage of syzkaller.
The following instructions are for Ubuntu host, QEMU vm, x86-64 kernel (with QEMU installed).
GCC
Since syzkaller requires coverage support in GCC, we need to use a recent GCC version - 7.1.0. (Set $GCC to be the path to your gcc folder.)
svn checkout svn://gcc.gnu.org/svn/gcc/trunk $GCC
cd $GCC
svn ls -v ^/tags | grep gcc_7_1_0_release
svn up -r 247494
There's a typo in this version of gcc. To fix it, find "tree.h" and change the code at line 900:
// from
(tree_check2 (NODE, __FILE__, __LINE__, __FUNCTION__, \
CALL_EXPR, AGGR_INIT_EXPR)->base.u.bits.unsigned_flag)
// to
(TREE_CHECK2 (NODE, CALL_EXPR, \
AGGR_INIT_EXPR)->base.u.bits.unsigned_flag)
Install GCC prerequisite:
sudo apt-get install flex bison libc6-dev libc6-dev-i386 linux-libc-dev linux-libc-dev:i386 libgmp3-dev libmpfr-dev libmpc-dev build-essential bc
Build GCC
mkdir build
mkdir install
cd build/
../configure --enable-languages=c,c++ --disable-bootstrap --enable-checking=no --with-gnu-as --with-gnu-ld --with-ld=/usr/bin/ld.bfd --disable-multilib --prefix=$GCC/install/
make -j64
make install
Note that there will be an error with the path of install folder, change the path to be the absolute path then.
Now you should have GCC binaries in $GCC/install/bin/:
$ ls $GCC/install/bin/
c++ gcc-ar gcov-tool x86_64-pc-linux-gnu-gcc-7.0.0
cpp gcc-nm x86_64-pc-linux-gnu-c++ x86_64-pc-linux-gnu-gcc-ar
g++ gcc-ranlib x86_64-pc-linux-gnu-g++ x86_64-pc-linux-gnu-gcc-nm
gcc gcov x86_64-pc-linux-gnu-gcc x86_64-pc-linux-gnu-gcc-ranlib
Kernel
Checkour linux kernel source:
git clone https://github.com/torvalds/linux.git $KERNEL
Generate default configs:
cd $KERNEL
make defconfig
make kvmconfig
Now we need to enable some config options required for syzkaller. Edit .config file manually and enable:
CONFIG_KCOV=y
CONFIG_DEBUG_INFO=y
CONFIG_KASAN=y
CONFIG_KASAN_INLINE=y
Since enabling these options results in more sub options being available, we need to regenerate config. Run this and press enter each time when prompted for some config value to leave it as default:
make oldconfig
Build the kernel with previously built GCC:
make CC='$GCC/install/bin/gcc' -j64
Now you should have vmlinux (kernel binary)
and bzImage (packed kernel image):
$ ls $KERNEL/vmlinux
$KERNEL/vmlinux
$ ls $KERNEL/arch/x86/boot/bzImage $KERNEL/arch/x86/boot/bzImage
Image
Install debootstrap
sudo apt-get install debootstrap
Create a script with this as its content:
#!/bin/bash
# Copyright 2016 syzkaller project authors. All rights reserved.
# Use of this source code is governed by Apache 2 LICENSE that can be found in the LICENSE file.
# create-image.sh creates a minimal Debian-wheezy Linux image suitable for syzkaller.
set -eux
# Create a minimal Debian-wheezy distributive as a directory.
sudo rm -rf wheezy
mkdir -p wheezy
sudo debootstrap --include=openssh-server,curl,tar,gcc,libc6-dev,time,strace,sudo,less,psmisc wheezy wheezy
# Set some defaults and enable promtless ssh to the machine for root.
sudo sed -i '/^root/ { s/:x:/::/ }' wheezy/etc/passwd
echo 'T0:23:respawn:/sbin/getty -L ttyS0 115200 vt100' | sudo tee -a wheezy/etc/inittab
printf '\nauto eth0\niface eth0 inet dhcp\n' | sudo tee -a wheezy/etc/network/interfaces
echo 'debugfs /sys/kernel/debug debugfs defaults 0 0' | sudo tee -a wheezy/etc/fstab
echo "kernel.printk = 7 4 1 3" | sudo tee -a wheezy/etc/sysctl.conf
echo 'debug.exception-trace = 0' | sudo tee -a wheezy/etc/sysctl.conf
echo "net.core.bpf_jit_enable = 1" | sudo tee -a wheezy/etc/sysctl.conf
echo "net.core.bpf_jit_harden = 2" | sudo tee -a wheezy/etc/sysctl.conf
echo "net.ipv4.ping_group_range = 0 65535" | sudo tee -a wheezy/etc/sysctl.conf
echo -en "127.0.0.1\tlocalhost\n" | sudo tee wheezy/etc/hosts
echo "nameserver 8.8.8.8" | sudo tee -a wheezy/etc/resolve.conf
echo "syzkaller" | sudo tee wheezy/etc/hostname
sudo mkdir -p wheezy/root/.ssh/
rm -rf ssh
mkdir -p ssh
ssh-keygen -f ssh/id_rsa -t rsa -N ''
cat ssh/id_rsa.pub | sudo tee wheezy/root/.ssh/authorized_keys
# Build a disk image
dd if=/dev/zero of=wheezy.img bs=1M seek=2047 count=1
sudo mkfs.ext4 -F wheezy.img
sudo mkdir -p /mnt/wheezy
sudo mount -o loop wheezy.img /mnt/wheezy
sudo cp -a wheezy/. /mnt/wheezy/.
sudo umount /mnt/wheezy
Run it to get wheezy.img.
Go
install Go 1.8.1:
wget https://storage.googleapis.com/golang/go1.8.1.linux-amd64.tar.gz
tar -xf go1.8.1.linux-amd64.tar.gz
mv go goroot
export GOROOT=`pwd`/goroot
export PATH=$PATH:$GOROOT/bin
mkdir gopath
export GOPATH=`pwd`/gopath
Build syzkaller:
go get -u -d github.com/google/syzkaller/...
cd gopath/src/github.com/google/syzkaller/
mkdir workdir
make
Create manager config my.cfg like this:
{
"target": "linux/amd64",
"http": "127.0.0.1:56741",
"workdir": "/gopath/src/github.com/google/syzkaller/workdir",
"vmlinux": "/linux/upstream/vmlinux",
"image": "/image/wheezy.img",
"sshkey": "/image/ssh/id_rsa",
"syzkaller": "/gopath/src/github.com/google/syzkaller",
"procs": 8,
"type": "qemu",
"vm": {
"count": 4,
"kernel": "/linux/arch/x86/boot/bzImage",
"cpu": 2,
"mem": 2048
}
}
Run syzkaller manager:
./bin/syz-manager -config=my.cfg
Note
use nohup to run it independent of ssh process.
if there's error report like:
2017/10/02 10:39:16 mismatching git revisions:
manager=
fuzzer= d335103a64e75dad273f98d32c0a1ea5967a5549+
executor=d335103a64e75dad273f98d32c0a1ea5967a5549+
It seems that the git revision does not match. You can safely turn down the checking of git revisions by first "grep mismatching git revisions" to find the code location and then comment the error printf out.
If there's error report saying "pulseaudio pa_context_connect() failed", it's probably because you use sudo to execute syzkaller.
If there's error reminding you that access to kvm's permission denied, then find the location of kvm (usually it's /dev/kvm) and add permission to it.
Hope you could run syzkaller well now. Now you can go to the browser and go to the port number to see the webpage generated by syzkaller.
How to use syzkaller? Refer to https://github.com/google/syzkaller/blob/master/docs/usage.md