Netty Buffer 通用byte 缓冲区使用

在java语言中直接操纵byte[]数组和通过ByteBuffer 接口来操纵都非常的繁琐和复杂。在此,netty 提供了ByteBuf来替代和增强ByteBuffer类的功能。

Creation of a buffer 

It is recommended to create a new buffer using the helper methods in Unpooled rather than calling an individual implementation's constructor.

如何创建ByteBuf ? 建议通过Unpooled 中的方法来创建一个新ByteBuf,而不是通过ByteBuf的构造方法来创建ByteBuf对象。

public final class Unpooled extends Object{}

Creates a new ByteBuf by allocating new space or by wrapping or copying existing byte arrays, byte buffers and a string.

Use static import

This classes is intended to be used with Java 5 static import statement:

import static io.netty.buffer.Unpooled.*;

ByteBuf heapBuffer    = buffer(128);
ByteBuf directBuffer  = directBuffer(256);
ByteBuf wrappedBuffer = wrappedBuffer(new byte[128], new byte[256]);
ByteBuf copiedBuffe r = copiedBuffer(ByteBuffer.allocate(128));

public interface ReferenceCounted {
   
    int refCnt();

    ReferenceCounted retain();

 
    ReferenceCounted retain(int increment);

    ReferenceCounted touch();


    ReferenceCounted touch(Object hint);

    /**
     * Decreases the reference count by {@code 1} and deallocates this object if the reference count reaches at
     * {@code 0}.
     *
     * @return {@code true} if and only if the reference count became {@code 0} and this object has been deallocated
     */
    boolean release();

    /**
     * Decreases the reference count by the specified {@code decrement} and deallocates this object if the reference
     * count reaches at {@code 0}.
     *
     * @return {@code true} if and only if the reference count became {@code 0} and this object has been deallocated
     */
    boolean release(int decrement);
}

A reference-counted object that requires explicit deallocation.

When a new ReferenceCounted is instantiated, it starts with the reference count of  1.

If the reference count is decreased to  0, the object will be deallocated explicitly

因为ByteBuf 继承了ReferencedCount 接口,所以必须通过release()来明确释放ByteBuf的引用计数。

所以,如下为使用netty  ByteBuf的基本模式。

//在使用netty ByteBuff时候,通过Unpooled提供的静态方式创建.
         ByteBuf byteBuf = Unpooled.buffer(1024);
         try {
        	 //按照顺序写输入数据
        	 byteBuf.writeInt(1);
        	 //按照顺序读取数据
        	 int i = byteBuf.readInt();
        	 System.out.println("i:"+i);
		} catch (Exception e) {
			// TODO: handle exception
		}
         finally {
			if(byteBuf!=null)
			{
 				byteBuf.release();
			}
		}

Who destroys it?

The general rule of thumb is that the party who accesses a reference-counted object lastly is responsible for the destruction of the reference-counted object. More specifically:

• If a [sending] component is supposed to pass a reference-counted object to another [receiving] component, the sending component usually does not need to destroy it but defers that decision to the receiving component.

• If a component consumes a reference-counted object and knows nothing else will access it anymore (i.e., does not pass along a reference to yet another component), the component should destroy it.

引用对象的释放原则：http://netty.io/wiki/reference-counted-objects.html

对于析构一个引用计数对象,一般是最后一个访问引用计数对象的来负责析构。

1 -如果一个sending 组件传递一个引用计数对象到另外一个receiving 组件中,那么通常sending组件不需要析构引用计数对象,而receiving 组件来负责析构引用对象。

2 -如果一个组件消耗一个引用计数对象并且明确知道该对象不会被其他对象使用,也不在传递该引用对象到其他组件中,那么该组件则负责引用计数对象的析构。

    public static void main( String[] args )
    {
    	//buf 在main方法中,buf传递到方法a()中,不负责释放buf,由a()方法释放
    	ByteBuf buf =Unpooled.buffer(1024);
        c(b(a(buf)));
    }
    public  static ByteBuf a(ByteBuf input) {
    	//a()方法中 input方法返回,input 还会继续使用,则由方法c()释放
        input.writeByte(42);
        return input;
    }
    public  static ByteBuf b(ByteBuf input) {
        try {
        	//input 不在使用,则由方法b()直接释放
            ByteBuf output = input.alloc().directBuffer(input.readableBytes() + 1);
            output.writeBytes(input);
            output.writeByte(42);
            return output;
        } finally {
            input.release();
        }
    }
    public static void c(ByteBuf input) {
        System.out.println(input);
        input.release();
    }

public class UnpooledHeapByteBuf extends AbstractReferenceCountedByteBuf 

{
    private final ByteBufAllocator alloc;
    byte[] array;
    private ByteBuffer tmpNioBuf;

}

在使用Unpooled.buffer()创建的ByteBuf底层包装byte[]数组。

public abstract class AbstractByteBuf extends ByteBuf 

{

    int readerIndex;
    int writerIndex;

}

在实现类中通过readerIndex和writeIndex来分别标记着读取数据时开始索引和写入数据时最后索引。

同时可以通过setter方法来改变readerIndex和writeIndex的位置,来读取和写入数据。

    	//在使用netty ByteBuff时候,通过Unpooled提供的静态方式创建.
         ByteBuf byteBuf = Unpooled.buffer(1024);
         try {
        	 //按照顺序写输入数据
        	 byteBuf.writeInt(1);
        	 //设置写入数据的开始索引,来覆盖上一次被写入的数据。在ByteBuf中索引是从零开始。
        	 byteBuf.writerIndex(0);
        	 //再次从头开始写入数据
        	 byteBuf.writeInt(2);
        	 //按照顺序读取数据
        	 int i = byteBuf.readInt();
        	 System.out.println("i:"+i);
		} catch (Exception e) {
			// TODO: handle exception
		}
         finally {
			if(byteBuf!=null)
			{
 				byteBuf.release();
			}
		}

ByteBuf.writeInt(int) 源代码剖析：

    @Override
    public ByteBuf writeInt(int value) {
        ensureAccessible();
        ensureWritable0(4);
        _setInt(writerIndex, value);
        writerIndex += 4;
        return this;
    }

首先判断该ReferencedCountObject 对象释放已经释放,然后自动扩容,确保可以写入数据,然后写入int数据的值,最后改变writeIndex的值。

1 -首先根据refCnt的值是否为零,判断引用计数对象是否被释放。

    /**
     * Should be called by every method that tries to access the buffers content to check
     * if the buffer was released before.
     */
    protected final void ensureAccessible() {
        if (checkAccessible && refCnt() == 0) {
            throw new IllegalReferenceCountException(0);
        }
    }

2 -根据规则进行扩容

    private void ensureWritable0(int minWritableBytes) {
        if (minWritableBytes <= writableBytes()) {
            return;
        }

        if (minWritableBytes > maxCapacity - writerIndex) {
            throw new IndexOutOfBoundsException(String.format(
                    "writerIndex(%d) + minWritableBytes(%d) exceeds maxCapacity(%d): %s",
                    writerIndex, minWritableBytes, maxCapacity, this));
        }

        // Normalize the current capacity to the power of 2.
        int newCapacity = alloc().calculateNewCapacity(writerIndex + minWritableBytes, maxCapacity);

        // Adjust to the new capacity.
        capacity(newCapacity);
    }

3 - static final Unsafe UNSAFE; UNSAFE.putInt(data, BYTE_ARRAY_BASE_OFFSET + index, value); 方法来写入数据.

在写入数据时,也是根据writeIndex值来写入数据的。

4- 最后改变writeIndex的值。

ByteBuf.readInt() 源代码剖析：

    @Override
    public int readInt() {
        checkReadableBytes0(4);
        int v = _getInt(readerIndex);
        readerIndex += 4;
        return v;
    }

首先判断ReferencedCountObject对象是否被释放,然后判断readIndex位置是否在已写入数据的范围内来读取数据,然后通过 Unsafe类来读取数据,最后增加readIndex的值。

1-判断引用计数对象,然后readerIndex > writerIndex - minimumReadableBytes 值,

    private void checkReadableBytes0(int minimumReadableBytes) {
        ensureAccessible();
        if (readerIndex > writerIndex - minimumReadableBytes) {
            throw new IndexOutOfBoundsException(String.format(
                    "readerIndex(%d) + length(%d) exceeds writerIndex(%d): %s",
                    readerIndex, minimumReadableBytes, writerIndex, this));
        }
    }

2 -通过UnSafe类来来读取数据，最后readerIndex += 4; 改变readerIndex 值。

    static int getInt(byte[] data, int index) {
        return UNSAFE.getInt(data, BYTE_ARRAY_BASE_OFFSET + index);
    }

Conversion to existing JDK types

Byte array

If a ByteBuf is backed by a byte array (i.e. byte[]), you can access it directly via the array() method. To determine if a buffer is backed by a byte array, hasArray() should be used.

NIO Buffers

If a ByteBuf can be converted into an NIO ByteBuffer which shares its content (i.e. view buffer), you can get it via the nioBuffer() method. To determine if a buffer can be converted into an NIO buffer, use nioBufferCount().

Strings

Various toString(Charset) methods convert a ByteBuf into a String. Please note that toString() is not a conversion method.

I/O Streams

Please refer to ByteBufInputStream and ByteBufOutputStream.

 ByteBuf heapBuffer    = buffer(128);
 ByteBuf directBuffer  = directBuffer(256);
 ByteBuf wrappedBuffer = wrappedBuffer(new byte[128], new byte[256]);
 ByteBuf copiedBuffe r = copiedBuffer(ByteBuffer.allocate(128));

io.netty.buffer  provides a generic buffer type called  ByteBuf . It is like  java.nio.ByteBuffer , but faster, more user-friendly, and extensible.

User-friendliness

Have you ever forgotten to call java.nio.ByteBuffer.flip() and wondered why the buffer does not contain anything? It never happens in ByteBuf because it has two indexes, one for reads and one for writes:

Performance

When a new  java.nio.ByteBuffer  is allocated, its content is filled with zeroes. This "zeroing" consumes CPU cycles and memory bandwidth. Normally, the buffer is then immediately filled from some data source, so the zeroing did no good.

To be reclaimed, java.nio.ByteBuffer relies on the JVM garbage collector. It works OK for heap buffers, but not direct buffers. By design, direct buffers are expected to live a long time. Thus, allocating many short-lived direct NIO buffers often causes an OutOfMemoryError. Also, deallocating a direct buffer explicitly using the (hidden, proprietary) API isn't very fast.

A ByteBuf's life cycle is bound to its reference count. When its count goes to zero, its underlying memory region (byte[] or direct buffer) is explicitly dereferenced, deallocated, or returned to the pool.

Netty also provides a solid buffer pool implementation that does not waste CPU cycles or memory bandwidth with zeroing its buffer:

ByteBufAllocator alloc = PooledByteBufAllocator.DEFAULT;
ByteBuf buf = alloc.directBuffer(1024);
...
buf.release(); // The direct buffer is returned to the pool.

However, reference counting isn't a holy grail. If the JVM garbage-collects the pooled buffer before its underlying memory region is returned to the pool, the leaks will eventually exhaust the pool.