feat: add Netty and Java NIO interview questions
Added 3 comprehensive interview documents covering Netty and Java NIO: **1. Netty Core Principles (Netty核心原理.md)** - Core components: Channel, EventLoop, ChannelPipeline, ByteBuf - Reactor threading model (single/multi-threaded, master-slave) - ByteBuf vs Java NIO ByteBuffer - Zero-copy implementation (4 approaches) - ChannelPipeline and ChannelHandler - TCP sticky/unpacking problem solutions - Heartbeat mechanism **2. Java NIO Core Principles (Java NIO核心原理.md)** - NIO vs BIO comparison - Three core components: Channel, Buffer, Selector - Selector multiplexing mechanism - Channel vs Stream differences - Buffer core attributes and usage - Non-blocking I/O implementation - Zero-copy with transferTo and MappedByteBuffer **3. Netty Practice Scenarios (Netty实战场景.md)** - High-performance RPC framework design - WebSocket server implementation - Million-connection IM system architecture - Memory leak detection and resolution - Graceful shutdown implementation - Heartbeat and reconnection mechanisms Each document includes: - Detailed problem descriptions - Complete code examples (Java) - Architecture diagrams - Best practices - Performance optimization tips - P7-level bonus points Total: 3 documents, covering: - Theoretical foundations - Practical implementations - Production scenarios - Performance tuning - Common pitfalls Suitable for backend P7 interview preparation. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
This commit is contained in:
527
10-中间件/Java NIO核心原理.md
Normal file
527
10-中间件/Java NIO核心原理.md
Normal file
@@ -0,0 +1,527 @@
|
||||
# Java NIO 核心原理
|
||||
|
||||
## 问题
|
||||
|
||||
1. 什么是 NIO？与 BIO 的区别是什么？
|
||||
2. NIO 的三大核心组件是什么？
|
||||
3. 什么是 Selector？如何实现多路复用？
|
||||
4. 什么是 Channel？与 Stream 的区别？
|
||||
5. 什么是 Buffer？如何理解 Buffer 的核心属性？
|
||||
6. NIO 如何实现非阻塞 I/O？
|
||||
7. 什么是零拷贝？如何实现？
|
||||
|
||||
---
|
||||
|
||||
## 标准答案
|
||||
|
||||
### 1. NIO vs BIO
|
||||
|
||||
#### **对比表**
|
||||
|
||||
| 特性 | BIO (Blocking I/O) | NIO (Non-blocking I/O) |
|
||||
|------|-------------------|----------------------|
|
||||
| **I/O 模型** | 阻塞 | 非阻塞 |
|
||||
| **线程模型** | 每连接一线程 | Reactor 模式 |
|
||||
| **并发能力** | 低 | 高 |
|
||||
| **编程复杂度** | 简单 | 复杂 |
|
||||
| **数据操作** | Stream | Channel + Buffer |
|
||||
| **适用场景** | 连接数少、高延迟 | 连接数多、高并发 |
|
||||
|
||||
#### **代码对比**
|
||||
|
||||
**BIO 实现**：
|
||||
```java
|
||||
// 传统 BIO - 阻塞式
|
||||
ServerSocket serverSocket = new ServerSocket(8080);
|
||||
|
||||
while (true) {
|
||||
// 阻塞等待连接
|
||||
Socket socket = serverSocket.accept();
|
||||
|
||||
// 每个连接一个线程
|
||||
new Thread(() -> {
|
||||
try {
|
||||
BufferedReader reader = new BufferedReader(
|
||||
new InputStreamReader(socket.getInputStream()));
|
||||
|
||||
// 阻塞读取数据
|
||||
String line = reader.readLine();
|
||||
|
||||
// 处理数据...
|
||||
} catch (IOException e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}).start();
|
||||
}
|
||||
```
|
||||
|
||||
**NIO 实现**：
|
||||
```java
|
||||
// NIO - 非阻塞式
|
||||
ServerSocketChannel serverChannel = ServerSocketChannel.open();
|
||||
serverChannel.configureBlocking(false);
|
||||
serverChannel.bind(new InetSocketAddress(8080));
|
||||
|
||||
Selector selector = Selector.open();
|
||||
serverChannel.register(selector, SelectionKey.OP_ACCEPT);
|
||||
|
||||
while (true) {
|
||||
// 非阻塞等待事件
|
||||
int readyCount = selector.select();
|
||||
|
||||
Set<SelectionKey> readyKeys = selector.selectedKeys();
|
||||
for (SelectionKey key : readyKeys) {
|
||||
if (key.isAcceptable()) {
|
||||
// 处理连接
|
||||
}
|
||||
if (key.isReadable()) {
|
||||
// 处理读
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. NIO 三大核心组件
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────┐
|
||||
│ Java NIO 架构 │
|
||||
├────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────┐ ┌─────────┐ │
|
||||
│ │Channel │◄──►│ Buffer │ │
|
||||
│ └────┬────┘ └─────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────┐ │
|
||||
│ │Selector │ │
|
||||
│ │(多路复用)│ │
|
||||
│ └─────────┘ │
|
||||
│ │
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
Channel: 数据通道（双向）
|
||||
Buffer: 数据容器
|
||||
Selector: 多路复用器
|
||||
```
|
||||
|
||||
#### **核心组件关系**
|
||||
|
||||
```java
|
||||
// 1. 打开 Channel
|
||||
FileChannel channel = FileChannel.open(Paths.get("data.txt"));
|
||||
|
||||
// 2. 分配 Buffer
|
||||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||||
|
||||
// 3. 读取数据到 Buffer
|
||||
channel.read(buffer);
|
||||
|
||||
// 4. 切换读写模式
|
||||
buffer.flip();
|
||||
|
||||
// 5. 处理数据
|
||||
while (buffer.hasRemaining()) {
|
||||
byte b = buffer.get();
|
||||
}
|
||||
|
||||
// 6. 清空 Buffer
|
||||
buffer.clear();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Selector 多路复用
|
||||
|
||||
#### **工作原理**
|
||||
|
||||
```
|
||||
Selector 多路复用模型:
|
||||
|
||||
┌──────────────────────────────────┐
|
||||
│ Selector │
|
||||
│ │
|
||||
│ select() - 阻塞等待就绪事件 │
|
||||
│ │
|
||||
│ ┌────┐ ┌────┐ ┌────┐ │
|
||||
│ │Ch1 │ │Ch2 │ │Ch3 │ ... │
|
||||
│ │就绪│ │就绪│ │就绪│ │
|
||||
│ └────┘ └────┘ └────┘ │
|
||||
└──────────────────────────────────┘
|
||||
▲
|
||||
│ one thread
|
||||
│
|
||||
┌─────────┴──────────┐
|
||||
│ Event Loop │
|
||||
│ - select() │
|
||||
│ - process() │
|
||||
│ - repeat │
|
||||
└────────────────────┘
|
||||
```
|
||||
|
||||
#### **SelectionKey 事件类型**
|
||||
|
||||
```java
|
||||
// 四种事件类型
|
||||
int OP_ACCEPT = 1 << 4; // 连接就绪（ServerSocketChannel）
|
||||
int OP_CONNECT = 1 << 3; // 连接完成（SocketChannel）
|
||||
int OP_READ = 1 << 0; // 读就绪
|
||||
int OP_WRITE = 1 << 2; // 写就绪
|
||||
|
||||
// 注册事件
|
||||
SelectionKey key = channel.register(selector,
|
||||
SelectionKey.OP_READ | SelectionKey.OP_WRITE);
|
||||
|
||||
// 判断事件类型
|
||||
if (key.isAcceptable()) { // 连接事件
|
||||
if (key.isReadable()) { // 读事件
|
||||
if (key.isWritable()) { // 写事件
|
||||
if (key.isConnectable()) { // 连接完成事件
|
||||
```
|
||||
|
||||
#### **完整示例**
|
||||
|
||||
```java
|
||||
// NIO Server
|
||||
ServerSocketChannel serverChannel = ServerSocketChannel.open();
|
||||
serverChannel.configureBlocking(false);
|
||||
serverChannel.bind(new InetSocketAddress(8080));
|
||||
|
||||
Selector selector = Selector.open();
|
||||
serverChannel.register(selector, SelectionKey.OP_ACCEPT);
|
||||
|
||||
while (true) {
|
||||
// 阻塞等待事件（超时 1 秒）
|
||||
int readyCount = selector.select(1000);
|
||||
|
||||
if (readyCount == 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 获取就绪的 SelectionKey
|
||||
Set<SelectionKey> readyKeys = selector.selectedKeys();
|
||||
Iterator<SelectionKey> iterator = readyKeys.iterator();
|
||||
|
||||
while (iterator.hasNext()) {
|
||||
SelectionKey key = iterator.next();
|
||||
iterator.remove(); // 移除已处理的 key
|
||||
|
||||
if (!key.isValid()) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 处理连接事件
|
||||
if (key.isAcceptable()) {
|
||||
ServerSocketChannel server = (ServerSocketChannel) key.channel();
|
||||
SocketChannel channel = server.accept();
|
||||
channel.configureBlocking(false);
|
||||
channel.register(selector, SelectionKey.OP_READ);
|
||||
}
|
||||
|
||||
// 处理读事件
|
||||
if (key.isReadable()) {
|
||||
SocketChannel channel = (SocketChannel) key.channel();
|
||||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||||
int bytesRead = channel.read(buffer);
|
||||
|
||||
if (bytesRead == -1) {
|
||||
// 连接关闭
|
||||
key.cancel();
|
||||
channel.close();
|
||||
} else {
|
||||
// 处理数据
|
||||
buffer.flip();
|
||||
byte[] data = new byte[buffer.remaining()];
|
||||
buffer.get(data);
|
||||
System.out.println("Received: " + new String(data));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. Channel vs Stream
|
||||
|
||||
#### **核心区别**
|
||||
|
||||
| 特性 | Stream (IO) | Channel (NIO) |
|
||||
|------|-------------|---------------|
|
||||
| **方向** | 单向（读/写） | 双向（读+写） |
|
||||
| **阻塞** | 阻塞 | 可配置阻塞/非阻塞 |
|
||||
| **缓冲** | 直接操作流 | 必须通过 Buffer |
|
||||
| **性能** | 较低 | 高（零拷贝） |
|
||||
|
||||
#### **Channel 类型**
|
||||
|
||||
```java
|
||||
// 1. FileChannel - 文件通道
|
||||
FileChannel fileChannel = FileChannel.open(Paths.get("data.txt"),
|
||||
StandardOpenOption.READ, StandardOpenOption.WRITE);
|
||||
|
||||
// 2. SocketChannel - TCP Socket
|
||||
SocketChannel socketChannel = SocketChannel.open();
|
||||
socketChannel.configureBlocking(false);
|
||||
socketChannel.connect(new InetSocketAddress("localhost", 8080));
|
||||
|
||||
// 3. ServerSocketChannel - TCP Server
|
||||
ServerSocketChannel serverChannel = ServerSocketChannel.open();
|
||||
serverChannel.bind(new InetSocketAddress(8080));
|
||||
|
||||
// 4. DatagramChannel - UDP
|
||||
DatagramChannel datagramChannel = DatagramChannel.open();
|
||||
datagramChannel.bind(new InetSocketAddress(8080));
|
||||
|
||||
// 5. Pipe.SinkChannel / Pipe.SourceChannel - 管道
|
||||
Pipe pipe = Pipe.open();
|
||||
Pipe.SinkChannel sinkChannel = pipe.sink();
|
||||
Pipe.SourceChannel sourceChannel = pipe.source();
|
||||
```
|
||||
|
||||
#### **FileChannel 示例**
|
||||
|
||||
```java
|
||||
// 文件复制（传统方式）
|
||||
FileChannel sourceChannel = FileChannel.open(Paths.get("source.txt"));
|
||||
FileChannel destChannel = FileChannel.open(Paths.get("dest.txt"),
|
||||
StandardOpenOption.CREATE, StandardOpenOption.WRITE);
|
||||
|
||||
// 方法1: 使用 Buffer
|
||||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||||
while (sourceChannel.read(buffer) != -1) {
|
||||
buffer.flip();
|
||||
destChannel.write(buffer);
|
||||
buffer.clear();
|
||||
}
|
||||
|
||||
// 方法2: 直接传输（零拷贝）
|
||||
sourceChannel.transferTo(0, sourceChannel.size(), destChannel);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Buffer 核心属性
|
||||
|
||||
#### **Buffer 结构**
|
||||
|
||||
```
|
||||
Buffer 核心属性:
|
||||
|
||||
┌───────┬──────────┬──────────┬─────────┐
|
||||
│ 0 │position │ limit │capacity │
|
||||
│ │ │ │ │
|
||||
│ 已处理 可读/写数据 │ 不可访问 │
|
||||
│ 数据 │ │ │
|
||||
└───────┴──────────┴──────────┴─────────┘
|
||||
↑ ↑ ↑
|
||||
mark position limit
|
||||
|
||||
操作方法:
|
||||
- mark(): 标记当前 position
|
||||
- reset(): 恢复到 mark 位置
|
||||
- clear(): position=0, limit=capacity
|
||||
- flip(): limit=position, position=0 (读→写)
|
||||
- rewind(): position=0, limit 不变
|
||||
```
|
||||
|
||||
#### **Buffer 使用流程**
|
||||
|
||||
```java
|
||||
// 1. 分配 Buffer
|
||||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||||
|
||||
// 初始状态: position=0, limit=1024, capacity=1024
|
||||
|
||||
// 2. 写入数据
|
||||
buffer.putInt(123);
|
||||
buffer.putLong(456L);
|
||||
buffer.put("Hello".getBytes());
|
||||
|
||||
// 写入后: position=17, limit=1024
|
||||
|
||||
// 3. 切换到读模式
|
||||
buffer.flip();
|
||||
|
||||
// flip 后: position=0, limit=17
|
||||
|
||||
// 4. 读取数据
|
||||
while (buffer.hasRemaining()) {
|
||||
byte b = buffer.get();
|
||||
}
|
||||
|
||||
// 读取后: position=17, limit=17
|
||||
|
||||
// 5. 清空 Buffer
|
||||
buffer.clear();
|
||||
|
||||
// clear 后: position=0, limit=1024, capacity=1024
|
||||
```
|
||||
|
||||
#### **Buffer 类型**
|
||||
|
||||
```java
|
||||
// 基本类型 Buffer
|
||||
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
|
||||
CharBuffer charBuffer = CharBuffer.allocate(1024);
|
||||
ShortBuffer shortBuffer = ShortBuffer.allocate(1024);
|
||||
IntBuffer intBuffer = IntBuffer.allocate(1024);
|
||||
LongBuffer longBuffer = LongBuffer.allocate(1024);
|
||||
FloatBuffer floatBuffer = FloatBuffer.allocate(1024);
|
||||
DoubleBuffer doubleBuffer = DoubleBuffer.allocate(1024);
|
||||
|
||||
// 直接内存 vs 堆内存
|
||||
ByteBuffer heapBuffer = ByteBuffer.allocate(1024); // 堆内存
|
||||
ByteBuffer directBuffer = ByteBuffer.allocateDirect(1024); // 直接内存
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. 非阻塞 I/O 实现
|
||||
|
||||
#### **阻塞 vs 非阻塞**
|
||||
|
||||
```java
|
||||
// 阻塞模式（默认）
|
||||
SocketChannel channel = SocketChannel.open();
|
||||
channel.configureBlocking(true); // 阻塞
|
||||
channel.connect(new InetSocketAddress("localhost", 8080));
|
||||
// 阻塞直到连接建立
|
||||
|
||||
// 非阻塞模式
|
||||
SocketChannel channel = SocketChannel.open();
|
||||
channel.configureBlocking(false); // 非阻塞
|
||||
channel.connect(new InetSocketAddress("localhost", 8080));
|
||||
// 立即返回
|
||||
|
||||
while (!channel.finishConnect()) {
|
||||
// 连接未完成，做其他事
|
||||
System.out.println("Connecting...");
|
||||
}
|
||||
```
|
||||
|
||||
#### **非阻塞读写**
|
||||
|
||||
```java
|
||||
SocketChannel channel = SocketChannel.open();
|
||||
channel.configureBlocking(false);
|
||||
|
||||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||||
|
||||
// 非阻塞读
|
||||
int bytesRead = channel.read(buffer);
|
||||
if (bytesRead == 0) {
|
||||
// 没有数据可用
|
||||
} else if (bytesRead == -1) {
|
||||
// 连接已关闭
|
||||
} else {
|
||||
// 读取到数据
|
||||
buffer.flip();
|
||||
// 处理数据...
|
||||
}
|
||||
|
||||
// 非阻塞写
|
||||
buffer.clear();
|
||||
buffer.put("Hello".getBytes());
|
||||
buffer.flip();
|
||||
|
||||
int bytesWritten = channel.write(buffer);
|
||||
if (bytesWritten == 0) {
|
||||
// 缓冲区满，稍后重试
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 7. 零拷贝实现
|
||||
|
||||
#### **传统 I/O vs 零拷贝**
|
||||
|
||||
```
|
||||
传统 I/O（4 次拷贝，4 次上下文切换）:
|
||||
1. 磁盘 → 内核缓冲区 (DMA)
|
||||
2. 内核缓冲区 → 用户缓冲区 (CPU)
|
||||
3. 用户缓冲区 → Socket 缓冲区 (CPU)
|
||||
4. Socket 缓冲区 → 网卡 (DMA)
|
||||
|
||||
零拷贝（2 次拷贝，2 次上下文切换）:
|
||||
1. 磁盘 → 内核缓冲区 (DMA)
|
||||
2. 内核缓冲区 → 网卡 (DMA)
|
||||
```
|
||||
|
||||
#### **transferTo 实现**
|
||||
|
||||
```java
|
||||
// 文件传输零拷贝
|
||||
FileChannel sourceChannel = FileChannel.open(Paths.get("source.txt"));
|
||||
FileChannel destChannel = FileChannel.open(Paths.get("dest.txt"),
|
||||
StandardOpenOption.CREATE, StandardOpenOption.WRITE);
|
||||
|
||||
// 直接在内核空间传输
|
||||
long position = 0;
|
||||
long count = sourceChannel.size();
|
||||
sourceChannel.transferTo(position, count, destChannel);
|
||||
```
|
||||
|
||||
#### **MappedByteBuffer（内存映射）**
|
||||
|
||||
```java
|
||||
// 内存映射文件
|
||||
FileChannel channel = FileChannel.open(Paths.get("data.txt"),
|
||||
StandardOpenOption.READ, StandardOpenOption.WRITE);
|
||||
|
||||
// 映射到内存
|
||||
MappedByteBuffer mappedBuffer = channel.map(
|
||||
FileChannel.MapMode.READ_WRITE, // 读写模式
|
||||
0, // 起始位置
|
||||
channel.size() // 映射大小
|
||||
);
|
||||
|
||||
// 直接操作内存（零拷贝）
|
||||
mappedBuffer.putInt(0, 123);
|
||||
int value = mappedBuffer.getInt(0);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## P7 加分项
|
||||
|
||||
### 深度理解
|
||||
- **多路复用原理**：理解 select、poll、epoll 的区别
|
||||
- **零拷贝原理**：理解 DMA、用户空间、内核空间
|
||||
- **内存管理**：堆内存 vs 直接内存，GC 影响
|
||||
|
||||
### 实战经验
|
||||
- **高并发场景**：Netty、Mina、Vert.x 框架使用
|
||||
- **文件处理**：大文件读写、内存映射文件
|
||||
- **网络编程**：自定义协议、粘包处理
|
||||
|
||||
### 性能优化
|
||||
- **Buffer 复用**：使用对象池减少 GC
|
||||
- **直接内存**：减少一次拷贝，但分配/释放成本高
|
||||
- **批量操作**：vectorized I/O（scatter/gather）
|
||||
|
||||
### 常见问题
|
||||
1. **Buffer 泄漏**：直接内存未释放
|
||||
2. **select 空转**：CPU 100% 问题
|
||||
3. **epoll 空轮询**：Linux kernel bug
|
||||
4. **文件描述符耗尽**：未关闭 Channel
|
||||
|
||||
---
|
||||
|
||||
## 总结
|
||||
|
||||
Java NIO 的核心优势：
|
||||
1. **高性能**：非阻塞 I/O、零拷贝
|
||||
2. **高并发**：单线程处理多连接
|
||||
3. **灵活性**：可配置阻塞/非阻塞
|
||||
4. **扩展性**：适合大规模分布式系统
|
||||
|
||||
**最佳实践**：
|
||||
- 高并发场景优先使用 NIO（Netty）
|
||||
- 大文件传输使用 transferTo
|
||||
- 理解 Buffer 的读写模式切换
|
||||
- 注意资源释放（Channel、Buffer）
|
||||
- 监控文件描述符使用
|
||||
Reference in New Issue
Block a user