NOTE

ByteBuf

Data structure, API, class hierarchy, and classification

JavaCreated Updated 1 min readhistorical

This is a historical learning note and may contain outdated or incomplete understanding.

Data Structure

There are two indexes: one marks the read position and one marks the write position.

0 < read < write < capacity

Data between 0 and the read index is invalid, data between the read and write indexes is readable, and data between the write index and capacity is writable.

API

readXXX reads data and moves the read index backward through the buffer.

writeXXX writes data and moves the write index backward through the buffer.

set does not move an index; it only changes data.

mark records the current read-index position so that it can later be restored through reset.

Class Hierarchy

ByteBuf consists of abstract methods.

AbstractByteBuf provides some basic skeleton implementations.

  • For example, getByte

It checks whether the index is valid and then calls the subclass implementation of _getByte.

@Override
public byte getByte(int index) {
    checkIndex(index);
    return _getByte(index);
}

protected abstract byte _getByte(int index);

Classification

It can mainly be classified along three dimensions.

pooled and unpooled

Whether the data is operated on in preallocated memory.

unsafe and non-unsafe

Whether JDK low-level NIO operations are used.

Using PooledUnsafeHeapByteBuf as an example, look at its _getByte method:

protected byte _getByte(int index) {
    return UnsafeByteBufUtil.getByte(memory, idx(index));
}

static byte getByte(byte[] array, int index) {
    return PlatformDependent.getByte(array, index);
}

public static byte getByte(byte[] data, int index) {
    return PlatformDependent0.getByte(data, index);
}

static byte getByte(byte[] data, int index) {
    // Finally calls Unsafe.
    return UNSAFE.getByte(data, BYTE_ARRAY_BASE_OFFSET + index);
}

Compare it with PooledHeapByteBuf._getByte:

protected byte _getByte(int index) {
    return HeapByteBufUtil.getByte(memory, idx(index));
}

static byte getByte(byte[] memory, int index) {
    // Does not call Unsafe; reads the array directly.
    return memory[index];
}

heap and direct

Whether the memory is allocated on the JVM heap or directly in OS memory.

Using UnPooledHeapByteBuf.getByte as an example:

public byte getByte(int index) {
    ensureAccessible();
    return _getByte(index);
}

protected byte _getByte(int index) {
    return HeapByteBufUtil.getByte(array, index);
}

static byte getByte(byte[] memory, int index) {
    // Reads from an array, so this is on the heap.
    return memory[index];
}

Compare it with UnPooledDirectByteBuf:

public byte getByte(int index) {
    ensureAccessible();
    return _getByte(index);
}


protected byte _getByte(int index) {
    // This buffer is a JDK NIO buffer.
    // See Unpooled.directBuffer.
    return buffer.get(index);
}
  • Unpooled.directBuffer
public static ByteBuf directBuffer(int initialCapacity) {
    return ALLOC.directBuffer(initialCapacity);
}

public ByteBuf directBuffer(int initialCapacity) {
    return directBuffer(initialCapacity, DEFAULT_MAX_CAPACITY);
}

public ByteBuf directBuffer(int initialCapacity, int maxCapacity) {
    if (initialCapacity == 0 && maxCapacity == 0) {
        return emptyBuf;
    }
    validate(initialCapacity, maxCapacity);
    return newDirectBuffer(initialCapacity, maxCapacity);
}

protected ByteBuf newDirectBuffer(int initialCapacity, int maxCapacity) {
    final ByteBuf buf;
    if (PlatformDependent.hasUnsafe()) {
        // Here.
        buf = noCleaner ? new InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf(this, initialCapacity, maxCapacity) :
                new InstrumentedUnpooledUnsafeDirectByteBuf(this, initialCapacity, maxCapacity);
    } else {
        buf = new InstrumentedUnpooledDirectByteBuf(this, initialCapacity, maxCapacity);
    }
    return disableLeakDetector ? buf : toLeakAwareBuffer(buf);
}
//......

public UnpooledUnsafeDirectByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
    super(maxCapacity);
    if (alloc == null) {
        throw new NullPointerException("alloc");
    }
    if (initialCapacity < 0) {
        throw new IllegalArgumentException("initialCapacity: " + initialCapacity);
    }
    if (maxCapacity < 0) {
        throw new IllegalArgumentException("maxCapacity: " + maxCapacity);
    }
    if (initialCapacity > maxCapacity) {
        throw new IllegalArgumentException(String.format(
                "initialCapacity(%d) > maxCapacity(%d)", initialCapacity, maxCapacity));
    }

    this.alloc = alloc;
    // ByteBuffer.allocateDirect(initialCapacity)
    // JDK low-level NIO allocates direct memory.
    setByteBuffer(allocateDirect(initialCapacity), false);
}

Discussion

Sign in with GitHub to comment. Discussions are stored as GitHub Issues.View on GitHub