457 lines
11 KiB
Markdown
457 lines
11 KiB
Markdown
# Java NIO 核心原理
|
||
|
||
## 问题
|
||
|
||
1. 什么是 NIO？与 BIO 的区别是什么？
|
||
2. NIO 的三大核心组件是什么？
|
||
3. 什么是 Selector？如何实现多路复用？
|
||
4. 什么是 Channel？与 Stream 的区别？
|
||
5. 什么是 Buffer？如何理解 Buffer 的核心属性？
|
||
6. NIO 如何实现非阻塞 I/O？
|
||
7. 什么是零拷贝？如何实现？
|
||
|
||
---
|
||
|
||
## 标准答案
|
||
|
||
### 1. NIO vs BIO
|
||
|
||
#### **对比表**
|
||
|
||
| 特性 | BIO (Blocking I/O) | NIO (Non-blocking I/O) |
|
||
|------|-------------------|----------------------|
|
||
| **I/O 模型** | 阻塞 | 非阻塞 |
|
||
| **线程模型** | 每连接一线程 | Reactor 模式 |
|
||
| **并发能力** | 低 | 高 |
|
||
| **编程复杂度** | 简单 | 复杂 |
|
||
| **数据操作** | Stream | Channel + Buffer |
|
||
| **适用场景** | 连接数少、高延迟 | 连接数多、高并发 |
|
||
|
||
#### **代码对比**
|
||
|
||
**BIO 实现**：
|
||
```java
|
||
// 传统 BIO - 阻塞式
|
||
ServerSocket serverSocket = new ServerSocket(8080);
|
||
|
||
while (true) {
|
||
// 阻塞等待连接
|
||
Socket socket = serverSocket.accept();
|
||
|
||
// 每个连接一个线程
|
||
new Thread(() -> {
|
||
try {
|
||
BufferedReader reader = new BufferedReader(
|
||
new InputStreamReader(socket.getInputStream()));
|
||
|
||
// 阻塞读取数据
|
||
String line = reader.readLine();
|
||
|
||
// 处理数据...
|
||
} catch (IOException e) {
|
||
e.printStackTrace();
|
||
}
|
||
}).start();
|
||
}
|
||
```
|
||
|
||
**NIO 实现**：
|
||
```java
|
||
// NIO - 非阻塞式
|
||
ServerSocketChannel serverChannel = ServerSocketChannel.open();
|
||
serverChannel.configureBlocking(false);
|
||
serverChannel.bind(new InetSocketAddress(8080));
|
||
|
||
Selector selector = Selector.open();
|
||
serverChannel.register(selector, SelectionKey.OP_ACCEPT);
|
||
|
||
while (true) {
|
||
// 非阻塞等待事件
|
||
int readyCount = selector.select();
|
||
|
||
Set<SelectionKey> readyKeys = selector.selectedKeys();
|
||
for (SelectionKey key : readyKeys) {
|
||
if (key.isAcceptable()) {
|
||
// 处理连接
|
||
}
|
||
if (key.isReadable()) {
|
||
// 处理读
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 2. NIO 三大核心组件
|
||
![[Java NIO三大核心组件架构.excalidraw]]
|
||
|
||
#### **核心组件关系**
|
||
|
||
```java
|
||
// 1. 打开 Channel
|
||
FileChannel channel = FileChannel.open(Paths.get("data.txt"));
|
||
|
||
// 2. 分配 Buffer
|
||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||
|
||
// 3. 读取数据到 Buffer
|
||
channel.read(buffer);
|
||
|
||
// 4. 切换读写模式
|
||
buffer.flip();
|
||
|
||
// 5. 处理数据
|
||
while (buffer.hasRemaining()) {
|
||
byte b = buffer.get();
|
||
}
|
||
|
||
// 6. 清空 Buffer
|
||
buffer.clear();
|
||
```
|
||
|
||
---
|
||
|
||
### 3. Selector 多路复用
|
||
|
||
#### **工作原理**
|
||
|
||
![[Selector多路复用模型]]
|
||
|
||
#### **SelectionKey 事件类型**
|
||
|
||
```java
|
||
// 四种事件类型
|
||
int OP_ACCEPT = 1 << 4; // 连接就绪（ServerSocketChannel）
|
||
int OP_CONNECT = 1 << 3; // 连接完成（SocketChannel）
|
||
int OP_READ = 1 << 0; // 读就绪
|
||
int OP_WRITE = 1 << 2; // 写就绪
|
||
|
||
// 注册事件
|
||
SelectionKey key = channel.register(selector,
|
||
SelectionKey.OP_READ | SelectionKey.OP_WRITE);
|
||
|
||
// 判断事件类型
|
||
if (key.isAcceptable()) { // 连接事件
|
||
if (key.isReadable()) { // 读事件
|
||
if (key.isWritable()) { // 写事件
|
||
if (key.isConnectable()) { // 连接完成事件
|
||
```
|
||
|
||
#### **完整示例**
|
||
|
||
```java
|
||
// NIO Server
|
||
ServerSocketChannel serverChannel = ServerSocketChannel.open();
|
||
serverChannel.configureBlocking(false);
|
||
serverChannel.bind(new InetSocketAddress(8080));
|
||
|
||
Selector selector = Selector.open();
|
||
serverChannel.register(selector, SelectionKey.OP_ACCEPT);
|
||
|
||
while (true) {
|
||
// 阻塞等待事件（超时 1 秒）
|
||
int readyCount = selector.select(1000);
|
||
|
||
if (readyCount == 0) {
|
||
continue;
|
||
}
|
||
|
||
// 获取就绪的 SelectionKey
|
||
Set<SelectionKey> readyKeys = selector.selectedKeys();
|
||
Iterator<SelectionKey> iterator = readyKeys.iterator();
|
||
|
||
while (iterator.hasNext()) {
|
||
SelectionKey key = iterator.next();
|
||
iterator.remove(); // 移除已处理的 key
|
||
|
||
if (!key.isValid()) {
|
||
continue;
|
||
}
|
||
|
||
// 处理连接事件
|
||
if (key.isAcceptable()) {
|
||
ServerSocketChannel server = (ServerSocketChannel) key.channel();
|
||
SocketChannel channel = server.accept();
|
||
channel.configureBlocking(false);
|
||
channel.register(selector, SelectionKey.OP_READ);
|
||
}
|
||
|
||
// 处理读事件
|
||
if (key.isReadable()) {
|
||
SocketChannel channel = (SocketChannel) key.channel();
|
||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||
int bytesRead = channel.read(buffer);
|
||
|
||
if (bytesRead == -1) {
|
||
// 连接关闭
|
||
key.cancel();
|
||
channel.close();
|
||
} else {
|
||
// 处理数据
|
||
buffer.flip();
|
||
byte[] data = new byte[buffer.remaining()];
|
||
buffer.get(data);
|
||
System.out.println("Received: " + new String(data));
|
||
}
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 4. Channel vs Stream
|
||
|
||
#### **核心区别**
|
||
|
||
| 特性 | Stream (IO) | Channel (NIO) |
|
||
|------|-------------|---------------|
|
||
| **方向** | 单向（读/写） | 双向（读+写） |
|
||
| **阻塞** | 阻塞 | 可配置阻塞/非阻塞 |
|
||
| **缓冲** | 直接操作流 | 必须通过 Buffer |
|
||
| **性能** | 较低 | 高（零拷贝） |
|
||
|
||
#### **Channel 类型**
|
||
|
||
```java
|
||
// 1. FileChannel - 文件通道
|
||
FileChannel fileChannel = FileChannel.open(Paths.get("data.txt"),
|
||
StandardOpenOption.READ, StandardOpenOption.WRITE);
|
||
|
||
// 2. SocketChannel - TCP Socket
|
||
SocketChannel socketChannel = SocketChannel.open();
|
||
socketChannel.configureBlocking(false);
|
||
socketChannel.connect(new InetSocketAddress("localhost", 8080));
|
||
|
||
// 3. ServerSocketChannel - TCP Server
|
||
ServerSocketChannel serverChannel = ServerSocketChannel.open();
|
||
serverChannel.bind(new InetSocketAddress(8080));
|
||
|
||
// 4. DatagramChannel - UDP
|
||
DatagramChannel datagramChannel = DatagramChannel.open();
|
||
datagramChannel.bind(new InetSocketAddress(8080));
|
||
|
||
// 5. Pipe.SinkChannel / Pipe.SourceChannel - 管道
|
||
Pipe pipe = Pipe.open();
|
||
Pipe.SinkChannel sinkChannel = pipe.sink();
|
||
Pipe.SourceChannel sourceChannel = pipe.source();
|
||
```
|
||
|
||
#### **FileChannel 示例**
|
||
|
||
```java
|
||
// 文件复制（传统方式）
|
||
FileChannel sourceChannel = FileChannel.open(Paths.get("source.txt"));
|
||
FileChannel destChannel = FileChannel.open(Paths.get("dest.txt"),
|
||
StandardOpenOption.CREATE, StandardOpenOption.WRITE);
|
||
|
||
// 方法1: 使用 Buffer
|
||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||
while (sourceChannel.read(buffer) != -1) {
|
||
buffer.flip();
|
||
destChannel.write(buffer);
|
||
buffer.clear();
|
||
}
|
||
|
||
// 方法2: 直接传输（零拷贝）
|
||
sourceChannel.transferTo(0, sourceChannel.size(), destChannel);
|
||
```
|
||
|
||
---
|
||
|
||
### 5. Buffer 核心属性
|
||
|
||
#### **Buffer 结构**
|
||
|
||
![[Buffer核心属性]]
|
||
|
||
#### **Buffer 使用流程**
|
||
|
||
```java
|
||
// 1. 分配 Buffer
|
||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||
|
||
// 初始状态: position=0, limit=1024, capacity=1024
|
||
|
||
// 2. 写入数据
|
||
buffer.putInt(123);
|
||
buffer.putLong(456L);
|
||
buffer.put("Hello".getBytes());
|
||
|
||
// 写入后: position=17, limit=1024
|
||
-
|
||
// 3. 切换到读模式
|
||
buffer.flip();
|
||
|
||
// flip 后: position=0, limit=17
|
||
|
||
// 4. 读取数据
|
||
while (buffer.hasRemaining()) {
|
||
byte b = buffer.get();
|
||
}
|
||
|
||
// 读取后: position=17, limit=17
|
||
|
||
// 5. 清空 Buffer
|
||
buffer.clear();
|
||
|
||
// clear 后: position=0, limit=1024, capacity=1024
|
||
```
|
||
|
||
#### **Buffer 类型**
|
||
|
||
```java
|
||
// 基本类型 Buffer
|
||
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
|
||
CharBuffer charBuffer = CharBuffer.allocate(1024);
|
||
ShortBuffer shortBuffer = ShortBuffer.allocate(1024);
|
||
IntBuffer intBuffer = IntBuffer.allocate(1024);
|
||
LongBuffer longBuffer = LongBuffer.allocate(1024);
|
||
FloatBuffer floatBuffer = FloatBuffer.allocate(1024);
|
||
DoubleBuffer doubleBuffer = DoubleBuffer.allocate(1024);
|
||
|
||
// 直接内存 vs 堆内存
|
||
ByteBuffer heapBuffer = ByteBuffer.allocate(1024); // 堆内存
|
||
ByteBuffer directBuffer = ByteBuffer.allocateDirect(1024); // 直接内存
|
||
```
|
||
|
||
---
|
||
|
||
### 6. 非阻塞 I/O 实现
|
||
|
||
#### **阻塞 vs 非阻塞**
|
||
|
||
```java
|
||
// 阻塞模式（默认）
|
||
SocketChannel channel = SocketChannel.open();
|
||
channel.configureBlocking(true); // 阻塞
|
||
channel.connect(new InetSocketAddress("localhost", 8080));
|
||
// 阻塞直到连接建立
|
||
|
||
// 非阻塞模式
|
||
SocketChannel channel = SocketChannel.open();
|
||
channel.configureBlocking(false); // 非阻塞
|
||
channel.connect(new InetSocketAddress("localhost", 8080));
|
||
// 立即返回
|
||
|
||
while (!channel.finishConnect()) {
|
||
// 连接未完成，做其他事
|
||
System.out.println("Connecting...");
|
||
}
|
||
```
|
||
|
||
#### **非阻塞读写**
|
||
|
||
```java
|
||
SocketChannel channel = SocketChannel.open();
|
||
channel.configureBlocking(false);
|
||
|
||
ByteBuffer buffer = ByteBuffer.allocate(1024);
|
||
|
||
// 非阻塞读
|
||
int bytesRead = channel.read(buffer);
|
||
if (bytesRead == 0) {
|
||
// 没有数据可用
|
||
} else if (bytesRead == -1) {
|
||
// 连接已关闭
|
||
} else {
|
||
// 读取到数据
|
||
buffer.flip();
|
||
// 处理数据...
|
||
}
|
||
|
||
// 非阻塞写
|
||
buffer.clear();
|
||
buffer.put("Hello".getBytes());
|
||
buffer.flip();
|
||
|
||
int bytesWritten = channel.write(buffer);
|
||
if (bytesWritten == 0) {
|
||
// 缓冲区满，稍后重试
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 7. 零拷贝实现
|
||
|
||
#### **传统 I/O vs 零拷贝**
|
||
|
||
![[零拷贝原理对比]]
|
||
|
||
#### **transferTo 实现**
|
||
|
||
```java
|
||
// 文件传输零拷贝
|
||
FileChannel sourceChannel = FileChannel.open(Paths.get("source.txt"));
|
||
FileChannel destChannel = FileChannel.open(Paths.get("dest.txt"),
|
||
StandardOpenOption.CREATE, StandardOpenOption.WRITE);
|
||
|
||
// 直接在内核空间传输
|
||
long position = 0;
|
||
long count = sourceChannel.size();
|
||
sourceChannel.transferTo(position, count, destChannel);
|
||
```
|
||
|
||
#### **MappedByteBuffer（内存映射）**
|
||
|
||
```java
|
||
// 内存映射文件
|
||
FileChannel channel = FileChannel.open(Paths.get("data.txt"),
|
||
StandardOpenOption.READ, StandardOpenOption.WRITE);
|
||
|
||
// 映射到内存
|
||
MappedByteBuffer mappedBuffer = channel.map(
|
||
FileChannel.MapMode.READ_WRITE, // 读写模式
|
||
0, // 起始位置
|
||
channel.size() // 映射大小
|
||
);
|
||
|
||
// 直接操作内存（零拷贝）
|
||
mappedBuffer.putInt(0, 123);
|
||
int value = mappedBuffer.getInt(0);
|
||
```
|
||
|
||
---
|
||
|
||
## P7 加分项
|
||
|
||
### 深度理解
|
||
- **多路复用原理**：理解 select、poll、epoll 的区别
|
||
- **零拷贝原理**：理解 DMA、用户空间、内核空间
|
||
- **内存管理**：堆内存 vs 直接内存，GC 影响
|
||
|
||
### 实战经验
|
||
- **高并发场景**：Netty、Mina、Vert.x 框架使用
|
||
- **文件处理**：大文件读写、内存映射文件
|
||
- **网络编程**：自定义协议、粘包处理
|
||
|
||
### 性能优化
|
||
- **Buffer 复用**：使用对象池减少 GC
|
||
- **直接内存**：减少一次拷贝，但分配/释放成本高
|
||
- **批量操作**：vectorized I/O（scatter/gather）
|
||
|
||
### 常见问题
|
||
1. **Buffer 泄漏**：直接内存未释放
|
||
2. **select 空转**：CPU 100% 问题
|
||
3. **epoll 空轮询**：Linux kernel bug
|
||
4. **文件描述符耗尽**：未关闭 Channel
|
||
|
||
---
|
||
|
||
## 总结
|
||
|
||
Java NIO 的核心优势：
|
||
1. **高性能**：非阻塞 I/O、零拷贝
|
||
2. **高并发**：单线程处理多连接
|
||
3. **灵活性**：可配置阻塞/非阻塞
|
||
4. **扩展性**：适合大规模分布式系统
|
||
|
||
**最佳实践**：
|
||
- 高并发场景优先使用 NIO（Netty）
|
||
- 大文件传输使用 transferTo
|
||
- 理解 Buffer 的读写模式切换
|
||
- 注意资源释放（Channel、Buffer）
|
||
- 监控文件描述符使用
|