利用 AES 对 log4j 日志文件加密
总览
本文简要介绍了 AES 算法加密的方式，以及如何利用 AES 对 log4j 输出的日志进行加密。
背景
在互联网时代下，JAVA 大多用来做后端开发，由于后端的程序大多都部署在自己的服务器上，客户接触不到程序的日志文件，因此，多数情况下，日志是没有加密的必要，log4j 本身也没有提供加密的方法。但有些客户端软件仍然是用 java 编写，客户端安装在客户的 PC上，我们想要了解软件的运行状态以及出错原因，就必须记下日志，这些日志可能包含有一些敏感的信息，我们不希望用户能直接看到，因此对日志加密是很有必要的。
AES 加密
既然要进行加密，那么首先得选择一个可靠的加密算法，网上搜索了下，大概有这三种：DES、AES、RSA。，其中 RSA 的解密是基于大数的因式分解，虽然安全性极高，但解密效率比其它两种低得多，不太适合。 DES是美国联邦政府采用过的一种加密方式，由于它的密钥只有56位，因此算法的理论安全强度是 2 56 2^{56} 256，但随着计算机的飞速发展，每秒能处理的密钥数越来越多，DES将不能够提供足够的安全性，因此没过国家标准技术研究所开始征集了 AES 用以取代 DES，研究所对 AES的要求是速度快(比三重 DES 速度快)、安全性高(至少与三重 DES一样安全)，最终 Rijndael 算法脱颖而出。因此对日志的加密解密选用 AES 比较适合。
JAVA 中的 AES
Java 早已提供标准的 AES 算法供大家使用，这里我提供一个简单的 AES 加密工具类(需要引入 apache 的 codec)
import java.nio.charset.Charset;
import java.security.GeneralSecurityException;
import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import org.apache.commons.codec.binary.Base64;
/**
* @Title: AESUtil.java
* @Description: AES 加密解密工具类
* @author: weekdragon
* @date: 2018年7月18日 下午4:58:22
* @version V1.0
*/
public class AESUtil {
private static final String KEY_ALGORITHM = "ThisIsASecretKey";
private static final String DEFAULT_CIPHER_ALGORITHM = "AES/CBC/PKCS5Padding";// 默认的加密算法
private static Cipher cipher = null;
static {
try {
cipher = Cipher.getInstance(DEFAULT_CIPHER_ALGORITHM);
} catch (Exception e) {}
}
/**
* AES 加密操作
*
* @param content
* 待加密内容
* @param key
* 加密密码
* @return 返回Base64转码后的加密数据
*/
public static String encrypt(String content, String key) throws GeneralSecurityException {
if(cipher == null)return content;
byte[] raw = key.getBytes(Charset.forName("UTF-8"));
if (raw.length != 16) {
throw new IllegalArgumentException("Invalid key size.");
}
SecretKeySpec skeySpec = new SecretKeySpec(raw, "AES");
cipher.init(Cipher.ENCRYPT_MODE, skeySpec, new IvParameterSpec(new byte[16]));
byte[] doFinal = cipher.doFinal(content.getBytes(Charset.forName("UTF-8")));
return Base64.encodeBase64String(doFinal);
}
/**
* AES 解密操作
*
* @param content
* @param key
* @return
*/
public static String decrypt(String content, String key) throws GeneralSecurityException {
byte[] encrypted = Base64.decodeBase64(content);
byte[] raw = key.getBytes(Charset.forName("UTF-8"));
if (raw.length != 16) {
throw new IllegalArgumentException("Invalid key size.");
}
SecretKeySpec skeySpec = new SecretKeySpec(raw, "AES");
Cipher cipher = Cipher.getInstance(DEFAULT_CIPHER_ALGORITHM);
cipher.init(Cipher.DECRYPT_MODE, skeySpec, new IvParameterSpec(new byte[16]));
byte[] original = cipher.doFinal(encrypted);
return new String(original, Charset.forName("UTF-8"));
}
}
对于 AES 加密，可以选择不同的密钥长度(16的整数倍)，还可以选择不同的补齐方法(加密数据不足16位时的补位策略)，这里只做了个最简单的实现。
对日志加密
log4j 本身没有提供日志加密的方法，但是用户可以自定义日志的 Appender，这个 Appender 就是负责日志输出的东西，目前我研究的加密方式有两种：
第一种是对整个输出流加密，自定义一个加密的输出流
import javax.crypto.*;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.io.*;
import java.security.*;
public class FlushableCipherOutputStream extends OutputStream
{
private static int HEADER_LENGTH = 16;
private SecretKeySpec key;
private RandomAccessFile seekableFile;
private boolean flushGoesStraightToDisk;
private Cipher cipher;
private boolean needToRestoreCipherState;
/** the buffer holding one byte of incoming data */
private byte[] ibuffer = new byte[1];
/** the buffer holding data ready to be written out */
private byte[] obuffer;
/** Each time you call 'flush()', the data will be written to the operating system level, immediately available
* for other processes to read. However this is not the same as writing to disk, which might save you some
* data if there's a sudden loss of power to the computer. To protect against that, set 'flushGoesStraightToDisk=true'.
* Most people set that to 'false'. */
public FlushableCipherOutputStream(String fnm, SecretKeySpec _key, boolean append, boolean _flushGoesStraightToDisk)
throws IOException
{
this(new File(fnm), _key, append,_flushGoesStraightToDisk);
}
public FlushableCipherOutputStream(File file, SecretKeySpec _key, boolean append, boolean _flushGoesStraightToDisk)
throws IOException
{
super();
if (! append)
file.delete();
seekableFile = new RandomAccessFile(file,"rw");
flushGoesStraightToDisk = _flushGoesStraightToDisk;
key = _key;
try {
cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
byte[] iv = new byte[16];
byte[] headerBytes = new byte[HEADER_LENGTH];
long fileLen = seekableFile.length();
if (fileLen % 16L != 0L) {
throw new IllegalArgumentException("Invalid file length (not a multiple of block size)");
} else if (fileLen == 0L) {
// new file
// You can write a 16 byte file header here, including some file format number to represent the
// encryption format, in case you need to change the key or algorithm. E.g. "100" = v1.0.0
headerBytes[0] = 100;
seekableFile.write(headerBytes);
// Now appending the first IV
SecureRandom sr = new SecureRandom();
sr.nextBytes(iv);
seekableFile.write(iv);
cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));
} else if (fileLen <= 16 + HEADER_LENGTH) {
throw new IllegalArgumentException("Invalid file length (need 2 blocks for iv and data)");
} else {
// file length is at least 2 blocks
needToRestoreCipherState = true;
}
} catch (InvalidKeyException e) {
throw new IOException(e.getMessage());
} catch (NoSuchAlgorithmException e) {
throw new IOException(e.getMessage());
} catch (NoSuchPaddingException e) {
throw new IOException(e.getMessage());
} catch (InvalidAlgorithmParameterException e) {
throw new IOException(e.getMessage());
}
}
/**
* Writes one _byte_ to this output stream.
*/
public void write(int b) throws IOException {
if (needToRestoreCipherState)
restoreStateOfCipher();
ibuffer[0] = (byte) b;
obuffer = cipher.update(ibuffer, 0, 1);
if (obuffer != null) {
seekableFile.write(obuffer);
obuffer = null;
}
}
/** Writes a byte array to this output stream. */
public void write(byte data[]) throws IOException {
write(data, 0, data.length);
}
/**
* Writes len bytes from the specified byte array
* starting at offset off to this output stream.
*
* @param data the data.
* @param off the start offset in the data.
* @param len the number of bytes to write.
*/
public void write(byte data[], int off, int len) throws IOException
{
if (needToRestoreCipherState)
restoreStateOfCipher();
obuffer = cipher.update(data, off, len);
if (obuffer != null) {
seekableFile.write(obuffer);
obuffer = null;
}
}
/** The tricky stuff happens here. We finalise the cipher, write it out, but then rewind the
* stream so that we can add more bytes without padding. */
public void flush() throws IOException
{
try {
if (needToRestoreCipherState)
return; // It must have already been flushed.
byte[] obuffer = cipher.doFinal();
if (obuffer != null) {
seekableFile.write(obuffer);
if (flushGoesStraightToDisk)
seekableFile.getFD().sync();
needToRestoreCipherState = true;
}
} catch (IllegalBlockSizeException e) {
throw new IOException("Illegal block");
} catch (BadPaddingException e) {
throw new IOException("Bad padding");
}
}
private void restoreStateOfCipher() throws IOException
{
try {
// I wish there was a more direct way to snapshot a Cipher object, but it seems there's not.
needToRestoreCipherState = false;
byte[] iv = cipher.getIV(); // To help avoid garbage, re-use the old one if present.
if (iv == null)
iv = new byte[16];
seekableFile.seek(seekableFile.length() - 32);
seekableFile.read(iv);
byte[] lastBlockEnc = new byte[16];
seekableFile.read(lastBlockEnc);
cipher.init(Cipher.DECRYPT_MODE, key, new IvParameterSpec(iv));
byte[] lastBlock = cipher.doFinal(lastBlockEnc);
seekableFile.seek(seekableFile.length() - 16);
cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));
byte[] out = cipher.update(lastBlock);
assert out == null || out.length == 0;
} catch (Exception e) {
throw new IOException("Unable to restore cipher state");
}
}
public void close() throws IOException
{
flush();
seekableFile.close();
}
}
然后自定义一个 Appender ，重写 setFile 方法
public class AESRollingFileAppender extends RollingFileAppender {
public final static byte[] keyBytes = new byte[] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 };
public final static SecretKeySpec key = new SecretKeySpec(keyBytes, "AES");
public final static String cipherKey[] = new String[] { "AES/CBC/PKCS5Padding", "AES/CFB8/NoPadding" };
public final static String Encoding = "UTF-8";
private Writer fw;
@Override
public synchronized void setFile(String fileName, boolean append,
boolean bufferedIO, int bufferSize) throws IOException {
LogLog.debug("setFile called: " + fileName + ", " + append);
// It does not make sense to have immediate flush and bufferedIO.
if (bufferedIO) {
setImmediateFlush(false);
}
reset();
FlushableCipherOutputStream cstream = null;
try {
cstream = new FlushableCipherOutputStream(fileName, key, true, false);
} catch (Exception ex) {
LogLog.error("setFile error", ex);
ex.printStackTrace();
}
setEncoding(Encoding);
fw = createWriter(cstream);
if (bufferedIO) {
fw = new BufferedWriter(fw, bufferSize);
}
this.setQWForFiles(fw);
this.fileName = fileName;
this.fileAppend = append;
this.bufferedIO = bufferedIO;
this.bufferSize = bufferSize;
writeHeader();
LogLog.debug("setFile ended");
if (append) {
File f = new File(fileName);
((CountingQuietWriter) qw).setCount(f.length());
}
}
}
这种方法是在 https://stackoverflow.com/questions/10283637/how-to-append-to-aes-encrypted-file 上找到的，实际测试中，发现客户端在某些极端情况下，日志输出的可能不完整，导致日志文件格式损坏，从损坏点开始起，日志不能解密，这是目前一个缺陷。但对于稳定运行的程序，并且日志并发量比较高，用这种方式加密可以节约时间。
第二种是对单条日志加密，由于会对访问到私有变量，因此直接拷贝 RollingFileAppender 类，重写 subAppend 方法即可
/**
* This method differentiates RollingFileAppender from its super class.
*
* @since 0.9.0
*/
protected void subAppend(LoggingEvent event) {
this.qw.write(encrypt(this.layout.format(event)));
if (layout.ignoresThrowable()) {
String[] s = event.getThrowableStrRep();
if (s != null) {
int len = s.length;
for (int i = 0; i < len; i++) {
this.qw.write(encrypt(s[i]));
this.qw.write(encrypt(Layout.LINE_SEP));
}
}
}
if (shouldFlush(event)) {
this.qw.flush();
}
if (fileName != null && qw != null) {
long size = ((CountingQuietWriter) qw).getCount();
if (size >= maxFileSize && size >= nextRollover) {
rollOver();
}
}
}
private String encrypt(String content) {
try {
return AESUtil.encrypt(content, AES_KEY) + "\n";
} catch (Exception e) {
e.printStackTrace();
return content;
}
}
在 log4j 配置文件里替换掉默认的 Appender 即可
log4j.appender.AppenderName=package.AESRollingFileAppender
这种方式，日志是按行加密，每一条日志加密一行，相比前一种，加密时间有所增加，大概每 10w 条日志增加几秒钟的样子，对于客户端程序来说，日志记录得不会非常密集，并发量不会很高，完全可以满足要求，而且可以应对客户端在强退、断电等异常情况，即使日志记录不完整，也只会损坏单条，而不会影响全部。