refactor: rename files to Chinese and organize by category
Organized 50 interview questions into 12 categories: - 01-分布式系统 (9 files): 分布式事务, 分布式锁, 一致性哈希, CAP理论, etc. - 02-数据库 (2 files): MySQL索引优化, MyBatis核心原理 - 03-缓存 (5 files): Redis数据结构, 缓存问题, LRU算法, etc. - 04-消息队列 (1 file): RocketMQ/Kafka - 05-并发编程 (4 files): 线程池, 设计模式, 限流策略, etc. - 06-JVM (1 file): JVM和垃圾回收 - 07-系统设计 (8 files): 秒杀系统, 短链接, IM, Feed流, etc. - 08-算法与数据结构 (4 files): B+树, 红黑树, 跳表, 时间轮 - 09-网络与安全 (3 files): TCP/IP, 加密安全, 性能优化 - 10-中间件 (4 files): Spring Boot, Nacos, Dubbo, Nginx - 11-运维 (4 files): Kubernetes, CI/CD, Docker, 可观测性 - 12-面试技巧 (1 file): 面试技巧和职业规划 All files renamed to Chinese for better accessibility and organized into categorized folders for easier navigation. Generated with [Claude Code](https://claude.com/claude-code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
This commit is contained in:
476
questions/03-缓存/ConcurrentHashMap原理.md
Normal file
476
questions/03-缓存/ConcurrentHashMap原理.md
Normal file
@@ -0,0 +1,476 @@
|
||||
# ConcurrentHashMap 原理
|
||||
|
||||
## 问题
|
||||
|
||||
1. ConcurrentHashMap 在 JDK 1.7 和 1.8 中的实现有什么区别?
|
||||
2. ConcurrentHashMap 如何保证线程安全?
|
||||
3. ConcurrentHashMap 的 size() 方法是如何实现的?
|
||||
4. ConcurrentHashMap 和 Hashtable、Collections.synchronizedMap 的区别?
|
||||
5. 在实际项目中如何选择线程安全的 Map?
|
||||
|
||||
---
|
||||
|
||||
## 标准答案
|
||||
|
||||
### 1. JDK 1.7 vs 1.8
|
||||
|
||||
#### **JDK 1.7:分段锁**
|
||||
|
||||
**结构**:
|
||||
```
|
||||
ConcurrentHashMap
|
||||
↓
|
||||
Segment[](分段数组,默认 16 个)
|
||||
↓
|
||||
HashEntry[](每个 Segment 有自己的 HashEntry 数组)
|
||||
↓
|
||||
HashEntry(键值对)
|
||||
```
|
||||
|
||||
**特点**:
|
||||
- **分段锁**:每个 Segment 独立加锁(ReentrantLock)
|
||||
- **并发度**:默认 16(Segment 数量)
|
||||
- **粒度**:Segment 级别
|
||||
|
||||
**获取锁流程**:
|
||||
```java
|
||||
// 1. 计算 Segment 索引
|
||||
int hash = hash(key.hashCode());
|
||||
int segmentIndex = (hash >>> segmentShift) & segmentMask;
|
||||
|
||||
// 2. 获取 Segment
|
||||
Segment segment = segments[segmentIndex];
|
||||
|
||||
// 3. 加锁
|
||||
segment.lock();
|
||||
try {
|
||||
// 操作 HashEntry[]
|
||||
} finally {
|
||||
segment.unlock();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### **JDK 1.8:CAS + synchronized**
|
||||
|
||||
**结构**:
|
||||
```
|
||||
ConcurrentHashMap
|
||||
↓
|
||||
Node[] + TreeNode[](数组 + 链表 + 红黑树)
|
||||
↓
|
||||
Node / TreeNode
|
||||
```
|
||||
|
||||
**特点**:
|
||||
- **CAS + synchronized**:CAS 失败后使用 synchronized
|
||||
- **粒度**:Node 级别(更细)
|
||||
- **并发度**:理论上无限制(实际受限于数组大小)
|
||||
|
||||
**核心改进**:
|
||||
| 特性 | JDK 1.7 | JDK 1.8 |
|
||||
|------|---------|---------|
|
||||
| **锁机制** | ReentrantLock(分段) | CAS + synchronized(Node 级别) |
|
||||
| **锁粒度** | Segment 级别 | Node 级别(更细) |
|
||||
| **并发度** | 默认 16 | 理论无限制 |
|
||||
| **查询** | 需要加锁(不强一致) | 无锁(volatile) |
|
||||
| **红黑树** | 不支持 | 支持(链表长度 ≥ 8) |
|
||||
|
||||
---
|
||||
|
||||
### 2. JDK 1.8 实现原理
|
||||
|
||||
#### **核心数据结构**
|
||||
|
||||
```java
|
||||
public class ConcurrentHashMap<K, V> {
|
||||
// 数组
|
||||
transient volatile Node<K,V>[] table;
|
||||
|
||||
// 数组(扩容时使用)
|
||||
private transient volatile Node<K,V>[] nextTable;
|
||||
|
||||
// 基础计数器值
|
||||
private transient volatile long baseCount;
|
||||
|
||||
// 控制位(sizeCtl < 0:初始化或扩容;-1:正在初始化;<-1:扩容线程数)
|
||||
private transient volatile int sizeCtl;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### **Node 节点**
|
||||
|
||||
```java
|
||||
static class Node<K,V> implements Map.Entry<K,V> {
|
||||
final int hash;
|
||||
final K key;
|
||||
volatile V val;
|
||||
volatile Node<K,V> next; // 链表下一个节点
|
||||
}
|
||||
```
|
||||
|
||||
**注意**:`val` 和 `next` 都是 `volatile`,保证可见性。
|
||||
|
||||
---
|
||||
|
||||
#### **TreeNode(红黑树节点)**
|
||||
|
||||
```java
|
||||
static final class TreeNode<K,V> extends Node<K,V> {
|
||||
TreeNode<K,V> parent; // 父节点
|
||||
TreeNode<K,V> left; // 左子节点
|
||||
TreeNode<K,V> right; // 右子节点
|
||||
TreeNode<K,V> prev; // 前驱节点
|
||||
boolean red; // 颜色(红黑树)
|
||||
}
|
||||
```
|
||||
|
||||
**转换条件**:
|
||||
- 链表 → 红黑树:链表长度 ≥ 8 **且** 数组长度 ≥ 64
|
||||
- 红黑树 → 链表:红黑树节点数 ≤ 6
|
||||
|
||||
---
|
||||
|
||||
### 3. 核心 API 源码解析
|
||||
|
||||
#### **put() 方法**
|
||||
|
||||
```java
|
||||
public V put(K key, V value) {
|
||||
return putVal(key, value, false);
|
||||
}
|
||||
|
||||
final V putVal(K key, V value, boolean onlyIfAbsent) {
|
||||
if (key == null || value == null) throw new NullPointerException();
|
||||
|
||||
// 1. 计算哈希值(扰动函数,减少哈希冲突)
|
||||
int hash = spread(key.hashCode());
|
||||
int binCount = 0;
|
||||
|
||||
// 2. 无限循环(CAS + 重试)
|
||||
for (Node<K,V>[] tab = table;;) {
|
||||
Node<K,V> f; int n, i, fh;
|
||||
|
||||
// 2.1 初始化数组(延迟初始化)
|
||||
if (tab == null || (n = tab.length) == 0)
|
||||
tab = initTable();
|
||||
|
||||
// 2.2 计算索引位置
|
||||
else if ((f = tabAt(tab, i = (n - 1) & hash)) == null) {
|
||||
// 位置为空,CAS 插入
|
||||
if (casTabAt(tab, i, null, new Node<K,V>(hash, key, value, null)))
|
||||
break; // CAS 成功,退出
|
||||
}
|
||||
|
||||
// 2.3 扩容中(MOVED = -1)
|
||||
else if ((fh = f.hash) == MOVED)
|
||||
tab = helpTransfer(tab, f); // 帮助扩容
|
||||
|
||||
// 2.4 位置不为空(链表或红黑树)
|
||||
else {
|
||||
V oldVal = null;
|
||||
synchronized (f) { // 加锁(Node 级别)
|
||||
if (tabAt(tab, i) == f) {
|
||||
// 链表
|
||||
if (fh >= 0) {
|
||||
binCount = 1;
|
||||
for (Node<K,V> e = f;; ++binCount) {
|
||||
K ek;
|
||||
// 键已存在,更新值
|
||||
if (e.hash == hash &&
|
||||
((ek = e.key) == key ||
|
||||
(ek != null && key.equals(ek)))) {
|
||||
oldVal = e.val;
|
||||
if (!onlyIfAbsent)
|
||||
e.val = value;
|
||||
break;
|
||||
}
|
||||
|
||||
Node<K,V> pred = e;
|
||||
// 到达链表尾部,插入新节点
|
||||
if ((e = e.next) == null) {
|
||||
pred.next = new Node<K,V>(hash, key,
|
||||
value, null);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
// 红黑树
|
||||
else if (f instanceof TreeBin) {
|
||||
Node<K,V> p;
|
||||
binCount = 2;
|
||||
if ((p = ((TreeBin<K,V>)f).putTreeVal(hash, key,
|
||||
value)) != null) {
|
||||
oldVal = p.val;
|
||||
if (!onlyIfAbsent)
|
||||
p.val = value;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 2.5 链表转红黑树
|
||||
if (binCount != 0) {
|
||||
if (binCount >= TREEIFY_THRESHOLD)
|
||||
treeifyBin(tab, i); // TREEIFY_THRESHOLD = 8
|
||||
if (oldVal != null)
|
||||
return oldVal;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 3. 增加元素个数(LongAdder)
|
||||
addCount(1L, binCount);
|
||||
return null;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### **get() 方法**
|
||||
|
||||
```java
|
||||
public V get(Object key) {
|
||||
Node<K,V>[] tab; Node<K,V> e, p; int n, eh; K ek;
|
||||
int h = spread(key.hashCode());
|
||||
|
||||
// 1. 数组不为空且索引位置不为空
|
||||
if ((tab = table) != null && (n = tab.length) > 0 &&
|
||||
(e = tabAt(tab, (n - 1) & h)) != null) {
|
||||
|
||||
// 2. 第一个节点匹配
|
||||
if ((eh = e.hash) == h) {
|
||||
if ((ek = e.key) == key || (ek != null && key.equals(ek)))
|
||||
return e.val;
|
||||
}
|
||||
|
||||
// 3. 红黑树
|
||||
else if (eh < 0)
|
||||
return (p = e.find(h, key)) != null ? p.val : null;
|
||||
|
||||
// 4. 链表遍历
|
||||
while ((e = e.next) != null) {
|
||||
if (e.hash == h &&
|
||||
((ek = e.key) == key || (ek != null && key.equals(ek))))
|
||||
return e.val;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
```
|
||||
|
||||
**注意**:
|
||||
- **无锁读取**:整个 `get()` 方法无锁
|
||||
- **volatile 可见性**:`Node.val` 是 `volatile`,保证读到最新值
|
||||
|
||||
---
|
||||
|
||||
#### **size() 方法**
|
||||
|
||||
**问题**:如何高效统计元素个数?
|
||||
|
||||
**方案**:**LongAdder**(分段计数)
|
||||
|
||||
```java
|
||||
public int size() {
|
||||
long n = sumCount();
|
||||
return ((n < 0L) ? 0 :
|
||||
(n > (long)Integer.MAX_VALUE) ? Integer.MAX_VALUE :
|
||||
(int)n);
|
||||
}
|
||||
|
||||
final long sumCount() {
|
||||
// 计算所有 CounterCell 的和
|
||||
CounterCell[] as = counterCells;
|
||||
long sum = baseCount;
|
||||
if (as != null) {
|
||||
for (CounterCell a : as)
|
||||
if (a != null)
|
||||
sum += a.value;
|
||||
}
|
||||
return sum;
|
||||
}
|
||||
```
|
||||
|
||||
**LongAdder 原理**:
|
||||
```
|
||||
LongAdder
|
||||
↓
|
||||
baseCount(基础值)
|
||||
↓
|
||||
CounterCell[](计数器数组,避免 CAS 竞争)
|
||||
↓
|
||||
多线程更新时,随机选择一个 CounterCell 更新
|
||||
↓
|
||||
统计时,baseCount + 所有 CounterCell 的值
|
||||
```
|
||||
|
||||
**为什么高效?**
|
||||
- 高并发时,多线程更新不同的 CounterCell(无竞争)
|
||||
- 统计时才累加(牺牲实时性换取性能)
|
||||
|
||||
---
|
||||
|
||||
### 4. ConcurrentHashMap vs 其他 Map
|
||||
|
||||
#### **对比表**
|
||||
|
||||
| 特性 | HashMap | Hashtable | Collections.synchronizedMap | ConcurrentHashMap |
|
||||
|------|---------|-----------|----------------------------|-------------------|
|
||||
| **线程安全** | ❌ 否 | ✅ 是 | ✅ 是 | ✅ 是 |
|
||||
| **锁机制** | 无 | synchronized | synchronized | CAS + synchronized |
|
||||
| **锁粒度** | - | 整个表 | 整个表 | Node 级别 |
|
||||
| **并发度** | 无 | 低 | 低 | 高 |
|
||||
| **迭代器** | Fail-Fast | Fail-Safe | Fail-Safe | Fail-Safe |
|
||||
| **null 键值** | 允许 | 不允许 | 不允许 | 不允许 |
|
||||
|
||||
---
|
||||
|
||||
#### **详细对比**
|
||||
|
||||
**1. Hashtable(不推荐)**
|
||||
|
||||
```java
|
||||
// Hashtable 的 put 方法(整个表加锁)
|
||||
public synchronized V put(K key, V value) {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**问题**:
|
||||
- **锁粒度太大**:整个表加锁
|
||||
- **并发度低**:同一时刻只有一个线程能操作
|
||||
|
||||
---
|
||||
|
||||
**2. Collections.synchronizedMap()**
|
||||
|
||||
```java
|
||||
Map<String, String> map = Collections.synchronizedMap(new HashMap<>());
|
||||
```
|
||||
|
||||
**原理**:
|
||||
```java
|
||||
// 内部使用 mutex(Object)加锁
|
||||
public V put(K key, V value) {
|
||||
synchronized (mutex) { // 整个 Map 加锁
|
||||
return m.put(key, value);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**问题**:
|
||||
- **锁粒度太大**:整个 Map 加锁
|
||||
- **迭代器需要手动加锁**:
|
||||
```java
|
||||
Map<String, String> map = Collections.synchronizedMap(new HashMap<>());
|
||||
|
||||
// 遍历需要手动加锁
|
||||
synchronized (map) {
|
||||
for (Map.Entry<String, String> entry : map.entrySet()) {
|
||||
// ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**3. ConcurrentHashMap(推荐)**
|
||||
|
||||
```java
|
||||
Map<String, String> map = new ConcurrentHashMap<>();
|
||||
```
|
||||
|
||||
**优点**:
|
||||
- **锁粒度小**:Node 级别
|
||||
- **并发度高**:理论上无限制
|
||||
- **迭代器无需加锁**:Fail-Safe(弱一致迭代器)
|
||||
|
||||
---
|
||||
|
||||
### 5. 实际项目应用
|
||||
|
||||
#### **场景 1:本地缓存**
|
||||
|
||||
```java
|
||||
@Component
|
||||
public class LocalCache {
|
||||
private final ConcurrentHashMap<String, Object> cache = new ConcurrentHashMap<>();
|
||||
|
||||
public void put(String key, Object value) {
|
||||
cache.put(key, value);
|
||||
}
|
||||
|
||||
public Object get(String key) {
|
||||
return cache.get(key);
|
||||
}
|
||||
|
||||
public void remove(String key) {
|
||||
cache.remove(key);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### **场景 2:计数器**
|
||||
|
||||
```java
|
||||
@Component
|
||||
public class CounterService {
|
||||
private final ConcurrentHashMap<String, LongAdder> counters = new ConcurrentHashMap<>();
|
||||
|
||||
public void increment(String key) {
|
||||
counters.computeIfAbsent(key, k -> new LongAdder()).increment();
|
||||
}
|
||||
|
||||
public long get(String key) {
|
||||
LongAdder adder = counters.get(key);
|
||||
return adder != null ? adder.sum() : 0;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### **场景 3:去重表**
|
||||
|
||||
```java
|
||||
@Component
|
||||
public class DeduplicationService {
|
||||
private final ConcurrentHashMap<String, Boolean> dedupTable = new ConcurrentHashMap<>();
|
||||
|
||||
public boolean isDuplicate(String id) {
|
||||
return dedupTable.putIfAbsent(id, true) != null;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. 阿里 P7 加分项
|
||||
|
||||
**深度理解**:
|
||||
- 理解 `ConcurrentHashMap` 的扩容机制(多线程协同扩容)
|
||||
- 理解 `LongAdder` 的实现原理(Cell 数组 + CAS)
|
||||
- 理解 `ConcurrentHashMap` 的弱一致迭代器
|
||||
|
||||
**实战经验**:
|
||||
- 有使用 `ConcurrentHashMap` 解决并发问题的经验
|
||||
- 有 `ConcurrentHashMap` 性能调优的经验
|
||||
- 有处理 `ConcurrentHashMap` 相关线上问题的经验
|
||||
|
||||
**架构能力**:
|
||||
- 能根据业务特点选择合适的 Map 实现
|
||||
- 能设计高性能的并发数据结构
|
||||
- 有分布式缓存的设计经验
|
||||
|
||||
**技术选型**:
|
||||
- 了解 `ConcurrentSkipListMap`(跳表实现)
|
||||
- 了解 Guava 的 `LocalCache`(Caffeine)
|
||||
- 能根据场景选择本地缓存或分布式缓存
|
||||
524
questions/03-缓存/LRU缓存实现.md
Normal file
524
questions/03-缓存/LRU缓存实现.md
Normal file
@@ -0,0 +1,524 @@
|
||||
# LRU 缓存实现
|
||||
|
||||
## 数据结构原理
|
||||
|
||||
### 什么是 LRU 缓存?
|
||||
LRU(Least Recently Used)缓存是一种缓存淘汰算法,当缓存满时,会淘汰最近最少使用的数据。它基于局部性原理,认为最近使用的数据在将来也可能被再次使用。
|
||||
|
||||
### LRU 缓存的核心概念
|
||||
|
||||
1. **缓存容量**:缓存能存储的最大数据量
|
||||
2. **访问时间**:数据被访问的时间戳
|
||||
3. **淘汰策略**:当缓存满时,移除最久未使用的数据
|
||||
4. **访问模式**:数据访问的时间和频率模式
|
||||
|
||||
### LRU 缓存的工作原理
|
||||
|
||||
1. **数据访问**:当数据被访问(读或写)时,将其标记为最近使用
|
||||
2. **数据插入**:新数据插入时,如果缓存满,先淘汰最久未使用的数据
|
||||
3. **数据查找**:查找数据时,如果存在,将其标记为最近使用
|
||||
4. **缓存维护**:维护使用顺序,确保时间复杂度高效
|
||||
|
||||
## 图解说明
|
||||
|
||||
```
|
||||
LRU 缓存工作流程示例:
|
||||
|
||||
初始状态: [] (容量=3)
|
||||
|
||||
1. 插入 A -> [A]
|
||||
2. 插入 B -> [A, B]
|
||||
3. 插入 C -> [A, B, C]
|
||||
4. 访问 A -> [A, B, C] (A 被移到头部)
|
||||
5. 揓入 D -> [B, C, D] (A 被淘汰)
|
||||
6. 访问 C -> [B, C, D] (C 被移到头部)
|
||||
7. 揓入 E -> [C, D, E] (B 被淘汰)
|
||||
|
||||
访问顺序: A, B, C, A, D, C, E
|
||||
淘汰顺序: A, B
|
||||
```
|
||||
|
||||
### LRU 与其他缓存策略对比
|
||||
|
||||
| 策略 | 淘汰标准 | 适用场景 |
|
||||
|------|----------|----------|
|
||||
| LRU | 最近最少使用 | 一般访问模式 |
|
||||
| LFU | 最不经常使用 | 访问频率稳定 |
|
||||
| FIFO | 先进先出 | 流水式数据处理 |
|
||||
| Random | 随机淘汰 | 无法预测访问模式 |
|
||||
|
||||
## Java 代码实现
|
||||
|
||||
### 方法一:使用 LinkedHashMap(推荐)
|
||||
|
||||
```java
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
|
||||
public class LRUCache<K, V> extends LinkedHashMap<K, V> {
|
||||
private final int capacity;
|
||||
|
||||
public LRUCache(int capacity) {
|
||||
super(capacity, 0.75f, true);
|
||||
this.capacity = capacity;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
|
||||
return size() > capacity;
|
||||
}
|
||||
|
||||
// 测试用例
|
||||
public static void main(String[] args) {
|
||||
LRUCache<Integer, String> cache = new LRUCache<>(3);
|
||||
|
||||
cache.put(1, "A");
|
||||
cache.put(2, "B");
|
||||
cache.put(3, "C");
|
||||
System.out.println("Cache after insertion: " + cache);
|
||||
|
||||
cache.get(1);
|
||||
System.out.println("Cache after accessing 1: " + cache);
|
||||
|
||||
cache.put(4, "D");
|
||||
System.out.println("Cache after insertion 4: " + cache);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 方法二:手写实现(面试重点)
|
||||
|
||||
```java
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
|
||||
class LRUCacheNode<K, V> {
|
||||
K key;
|
||||
V value;
|
||||
LRUCacheNode<K, V> prev;
|
||||
LRUCacheNode<K, V> next;
|
||||
|
||||
public LRUCacheNode(K key, V value) {
|
||||
this.key = key;
|
||||
this.value = value;
|
||||
this.prev = null;
|
||||
this.next = null;
|
||||
}
|
||||
}
|
||||
|
||||
public class LRUCacheImpl<K, V> {
|
||||
private final int capacity;
|
||||
private final Map<K, LRUCacheNode<K, V>> cache;
|
||||
private final LRUCacheNode<K, V> head;
|
||||
private final LRUCacheNode<K, V> tail;
|
||||
|
||||
public LRUCacheImpl(int capacity) {
|
||||
this.capacity = capacity;
|
||||
this.cache = new HashMap<>();
|
||||
this.head = new LRUCacheNode<>(null, null);
|
||||
this.tail = new LRUCacheNode<>(null, null);
|
||||
head.next = tail;
|
||||
tail.prev = head;
|
||||
}
|
||||
|
||||
// 获取数据
|
||||
public V get(K key) {
|
||||
if (!cache.containsKey(key)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
LRUCacheNode<K, V> node = cache.get(key);
|
||||
moveToHead(node);
|
||||
return node.value;
|
||||
}
|
||||
|
||||
// 插入数据
|
||||
public void put(K key, V value) {
|
||||
if (cache.containsKey(key)) {
|
||||
// 更新已有节点
|
||||
LRUCacheNode<K, V> node = cache.get(key);
|
||||
node.value = value;
|
||||
moveToHead(node);
|
||||
} else {
|
||||
// 创建新节点
|
||||
LRUCacheNode<K, V> newNode = new LRUCacheNode<>(key, value);
|
||||
cache.put(key, newNode);
|
||||
addToHead(newNode);
|
||||
|
||||
// 淘汰策略
|
||||
if (cache.size() > capacity) {
|
||||
LRUCacheNode<K, V> last = removeTail();
|
||||
cache.remove(last.key);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 移除指定节点
|
||||
public void remove(K key) {
|
||||
if (!cache.containsKey(key)) {
|
||||
return;
|
||||
}
|
||||
|
||||
LRUCacheNode<K, V> node = cache.get(key);
|
||||
removeNode(node);
|
||||
cache.remove(key);
|
||||
}
|
||||
|
||||
// 清空缓存
|
||||
public void clear() {
|
||||
cache.clear();
|
||||
head.next = tail;
|
||||
tail.prev = head;
|
||||
}
|
||||
|
||||
// 获取缓存大小
|
||||
public int size() {
|
||||
return cache.size();
|
||||
}
|
||||
|
||||
// 检查是否包含键
|
||||
public boolean containsKey(K key) {
|
||||
return cache.containsKey(key);
|
||||
}
|
||||
|
||||
// 辅助方法:添加到头部
|
||||
private void addToHead(LRUCacheNode<K, V> node) {
|
||||
node.prev = head;
|
||||
node.next = head.next;
|
||||
head.next.prev = node;
|
||||
head.next = node;
|
||||
}
|
||||
|
||||
// 辅助方法:移除节点
|
||||
private void removeNode(LRUCacheNode<K, V> node) {
|
||||
node.prev.next = node.next;
|
||||
node.next.prev = node.prev;
|
||||
}
|
||||
|
||||
// 辅助方法:移动到头部
|
||||
private void moveToHead(LRUCacheNode<K, V> node) {
|
||||
removeNode(node);
|
||||
addToHead(node);
|
||||
}
|
||||
|
||||
// 辅助方法:移除尾部节点
|
||||
private LRUCacheNode<K, V> removeTail() {
|
||||
LRUCacheNode<K, V> last = tail.prev;
|
||||
removeNode(last);
|
||||
return last;
|
||||
}
|
||||
|
||||
// 打印缓存内容
|
||||
public void printCache() {
|
||||
LRUCacheNode<K, V> current = head.next;
|
||||
while (current != tail) {
|
||||
System.out.print("(" + current.key + "=" + current.value + ") ");
|
||||
current = current.next;
|
||||
}
|
||||
System.out.println();
|
||||
}
|
||||
|
||||
// 测试用例
|
||||
public static void main(String[] args) {
|
||||
LRUCacheImpl<Integer, String> cache = new LRUCacheImpl<>(3);
|
||||
|
||||
System.out.println("Inserting 1, 2, 3");
|
||||
cache.put(1, "A");
|
||||
cache.put(2, "B");
|
||||
cache.put(3, "C");
|
||||
cache.printCache();
|
||||
|
||||
System.out.println("Accessing 1");
|
||||
cache.get(1);
|
||||
cache.printCache();
|
||||
|
||||
System.out.println("Inserting 4");
|
||||
cache.put(4, "D");
|
||||
cache.printCache();
|
||||
|
||||
System.out.println("Removing 2");
|
||||
cache.remove(2);
|
||||
cache.printCache();
|
||||
|
||||
System.out.println("Clearing cache");
|
||||
cache.clear();
|
||||
cache.printCache();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 方法三:使用双向队列(Deque)
|
||||
|
||||
```java
|
||||
import java.util.Deque;
|
||||
import java.util.HashMap;
|
||||
import java.util.LinkedList;
|
||||
import java.util.Map;
|
||||
|
||||
public class LRUCacheWithDeque<K, V> {
|
||||
private final int capacity;
|
||||
private final Map<K, V> cache;
|
||||
private final Deque<K> accessQueue;
|
||||
|
||||
public LRUCacheWithDeque(int capacity) {
|
||||
this.capacity = capacity;
|
||||
this.cache = new HashMap<>();
|
||||
this.accessQueue = new LinkedList<>();
|
||||
}
|
||||
|
||||
public V get(K key) {
|
||||
if (!cache.containsKey(key)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// 更新访问顺序
|
||||
accessQueue.remove(key);
|
||||
accessQueue.addFirst(key);
|
||||
return cache.get(key);
|
||||
}
|
||||
|
||||
public void put(K key, V value) {
|
||||
if (cache.containsKey(key)) {
|
||||
// 更新已有数据
|
||||
cache.put(key, value);
|
||||
accessQueue.remove(key);
|
||||
accessQueue.addFirst(key);
|
||||
} else {
|
||||
// 添加新数据
|
||||
if (cache.size() >= capacity) {
|
||||
// 淘汰最久未使用的数据
|
||||
K lruKey = accessQueue.removeLast();
|
||||
cache.remove(lruKey);
|
||||
}
|
||||
cache.put(key, value);
|
||||
accessQueue.addFirst(key);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 时间复杂度分析
|
||||
|
||||
### 操作时间复杂度
|
||||
|
||||
| 操作 | 时间复杂度 | 说明 |
|
||||
|------|------------|------|
|
||||
| get(K) | O(1) | 哈希查找 + 双向链表操作 |
|
||||
| put(K,V) | O(1) | 哈希查找 + 双向链表操作 |
|
||||
| remove(K) | O(1) | 哈希删除 + 双向链表操作 |
|
||||
| clear() | O(1) | 清空哈希表和链表 |
|
||||
| size() | O(1) | 哈希表大小 |
|
||||
|
||||
### 空间复杂度
|
||||
|
||||
- O(n) - 存储 n 个键值对
|
||||
- 需要额外空间维护双向链表结构
|
||||
|
||||
### 性能分析
|
||||
|
||||
1. **最优实现**:HashMap + 双向链表 = O(1) 所有操作
|
||||
2. **次优实现**:LinkedHashMap = O(1) 操作,但依赖 JDK 实现
|
||||
3. **最差实现**:数组 + 遍历 = O(n) 操作
|
||||
|
||||
## 实际应用场景
|
||||
|
||||
### 1. Web 服务器缓存
|
||||
- **静态资源缓存**:CSS、JS、图片文件
|
||||
- **页面缓存**:动态生成的 HTML 页面
|
||||
- **API 响应缓存**:频繁调用的 API 结果
|
||||
|
||||
```java
|
||||
// Web 缓存示例
|
||||
public class WebCache {
|
||||
private final LRUCache<String, HttpResponse> cache;
|
||||
|
||||
public WebCache(int maxSize) {
|
||||
this.cache = new LRUCacheImpl<>(maxSize);
|
||||
}
|
||||
|
||||
public HttpResponse getPage(String url) {
|
||||
HttpResponse response = cache.get(url);
|
||||
if (response == null) {
|
||||
response = fetchFromOrigin(url);
|
||||
cache.put(url, response);
|
||||
}
|
||||
return response;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 数据库查询缓存
|
||||
- **ORM 缓存**:Hibernate、MyBatis 一级/二级缓存
|
||||
- **查询结果缓存**:复杂查询结果的缓存
|
||||
|
||||
```java
|
||||
// 数据库缓存示例
|
||||
public class QueryCache {
|
||||
private final LRUCache<String, ResultSet> queryCache;
|
||||
|
||||
public QueryCache(int maxSize) {
|
||||
this.queryCache = new LRUCacheImpl<>(maxSize);
|
||||
}
|
||||
|
||||
public ResultSet executeQuery(String sql) {
|
||||
ResultSet result = queryCache.get(sql);
|
||||
if (result == null) {
|
||||
result = executeSql(sql);
|
||||
if (result != null) {
|
||||
queryCache.put(sql, result);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 内存数据库
|
||||
- **Redis 缓存策略**:`maxmemory-policy allkeys-lru`
|
||||
- **本地缓存**:Ehcache、Caffeine
|
||||
|
||||
```java
|
||||
// 本地缓存示例
|
||||
public class LocalCache {
|
||||
private final LRUCache<String, Object> cache;
|
||||
|
||||
public LocalCache(int maxSize) {
|
||||
this.cache = new LRUCacheImpl<>(maxSize);
|
||||
}
|
||||
|
||||
public <T> T get(String key, Class<T> type) {
|
||||
Object value = cache.get(key);
|
||||
return type.cast(value);
|
||||
}
|
||||
|
||||
public void put(String key, Object value) {
|
||||
cache.put(key, value);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 消息队列缓冲
|
||||
- **消息去重**:防止重复处理消息
|
||||
- **请求合并**:合并短时间内多个相同请求
|
||||
|
||||
```java
|
||||
// 消息队列缓冲示例
|
||||
public class MessageBuffer {
|
||||
private final LRUCache<String, Message> messageBuffer;
|
||||
private final Queue<Message> messageQueue;
|
||||
|
||||
public MessageBuffer(int maxSize) {
|
||||
this.messageBuffer = new LRUCacheImpl<>(maxSize);
|
||||
this.messageQueue = new LinkedList<>();
|
||||
}
|
||||
|
||||
public void addMessage(Message message) {
|
||||
String key = message.getId();
|
||||
if (!messageBuffer.containsKey(key)) {
|
||||
messageBuffer.put(key, message);
|
||||
messageQueue.add(message);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 与其他缓存策略的对比
|
||||
|
||||
| 策略 | 时间复杂度 | 适用场景 | 优点 | 缺点 |
|
||||
|------|------------|----------|------|------|
|
||||
| LRU | O(1) | 一般访问模式 | 实现简单,效果好 | 对突发访问敏感 |
|
||||
| LFU | O(1) | 频率稳定场景 | 更好处理热点数据 | 实现较复杂 |
|
||||
| FIFO | O(1) | 流水式数据 | 实现简单 | 可能淘汰有用数据 |
|
||||
| Random | O(1) | 随机访问模式 | 实现最简单 | 性能不稳定 |
|
||||
|
||||
### LRU 的优缺点
|
||||
|
||||
**优点**:
|
||||
- 实现简单,易于理解
|
||||
- 性能稳定,时间复杂度 O(1)
|
||||
- 对大多数场景效果良好
|
||||
- JDK 已有成熟实现
|
||||
|
||||
**缺点**:
|
||||
- 对突发访问敏感(缓存污染)
|
||||
- 需要额外维护访问顺序
|
||||
- 内存占用相对较大
|
||||
- 无法区分临时访问和频繁访问
|
||||
|
||||
## 常见面试问题
|
||||
|
||||
### Q1: 如何实现 LRU 缓存?为什么选择 HashMap + 双向链表?
|
||||
**答**:
|
||||
1. **HashMap** 提供 O(1) 时间复杂度的查找
|
||||
2. **双向链表** 维护访问顺序,头节点最近访问,尾节点最久未访问
|
||||
3. 结合使用可实现所有操作的 O(1) 时间复杂度
|
||||
4. 其他方案(如数组)时间复杂度较高
|
||||
|
||||
### Q2: LRU 缓存存在什么问题?如何改进?
|
||||
**答**:
|
||||
**存在的问题**:
|
||||
- 缓存污染:一次性大量访问可能导致有用数据被淘汰
|
||||
- 无法区分临时访问和频繁访问
|
||||
|
||||
**改进方案**:
|
||||
1. **LFU (Least Frequently Used)**:记录访问频率
|
||||
2. **2Q (Two Queues)**:分为缓存队列和保留队列
|
||||
3. **ARC (Adaptive Replacement Cache)**:结合 LRU 和 LFU
|
||||
4. **LRU-K**:记录最近 K 次访问历史
|
||||
|
||||
### Q3: 缓存容量如何确定?
|
||||
**答**:
|
||||
考虑因素:
|
||||
1. **内存限制**:系统可用内存大小
|
||||
2. **访问模式**:数据访问频率和大小分布
|
||||
3. **性能要求**:需要达到的响应时间
|
||||
4. **命中率目标**:期望的缓存命中率
|
||||
5. **业务特点**:数据的时效性和重要性
|
||||
|
||||
### Q4: 如何处理缓存并发问题?
|
||||
**答**:
|
||||
解决方案:
|
||||
1. **使用线程安全容器**:如 `ConcurrentHashMap`
|
||||
2. **添加同步锁**:方法或代码块同步
|
||||
3. **使用读写锁**:提高并发性能
|
||||
4. **不可变对象**:避免并发修改问题
|
||||
|
||||
```java
|
||||
// 线程安全的 LRU 缓存
|
||||
public class ThreadSafeLRUCache<K, V> {
|
||||
private final LRUCacheImpl<K, V> cache;
|
||||
private final ReadWriteLock lock = new ReentrantReadWriteLock();
|
||||
|
||||
public V get(K key) {
|
||||
lock.readLock().lock();
|
||||
try {
|
||||
return cache.get(key);
|
||||
} finally {
|
||||
lock.readLock().unlock();
|
||||
}
|
||||
}
|
||||
|
||||
public void put(K key, V value) {
|
||||
lock.writeLock().lock();
|
||||
try {
|
||||
cache.put(key, value);
|
||||
} finally {
|
||||
lock.writeLock().unlock();
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Q5: 如何处理缓存穿透、击穿、雪崩?
|
||||
**答**:
|
||||
**缓存穿透**:
|
||||
- 查询不存在的数据
|
||||
- 解决方案:布隆过滤器、空值缓存
|
||||
|
||||
**缓存击穿**:
|
||||
- 大量请求同时查询过期热点数据
|
||||
- 解决方案:互斥锁、永不过期
|
||||
|
||||
**缓存雪崩**:
|
||||
- 大量缓存同时失效
|
||||
- 解决方案:随机过期时间、集群部署
|
||||
215
questions/03-缓存/Redis数据结构.md
Normal file
215
questions/03-缓存/Redis数据结构.md
Normal file
@@ -0,0 +1,215 @@
|
||||
# Redis 数据结构
|
||||
|
||||
## 问题
|
||||
|
||||
1. Redis 有哪些数据结构?底层实现是什么?
|
||||
2. String 类型的应用场景?
|
||||
3. Hash 和 String 的区别?
|
||||
4. List 的应用场景?
|
||||
5. Set 和 ZSet 的区别?
|
||||
6. Bitmap、HyperLogLog、GEO 的应用?
|
||||
|
||||
---
|
||||
|
||||
## 标准答案
|
||||
|
||||
### 1. Redis 数据类型
|
||||
|
||||
| 类型 | 底层实现 | 应用场景 |
|
||||
|------|---------|----------|
|
||||
| **String** | SDS | 缓存、计数器、分布式锁 |
|
||||
| **Hash** | 压缩列表/哈希表 | 对象存储、购物车 |
|
||||
| **List** | 双向链表/压缩列表 | 消息队列、最新列表 |
|
||||
| **Set** | 哈希表/整数集合 | 标签、共同关注 |
|
||||
| **ZSet** | 跳表/哈希表 | 排行榜、延时队列 |
|
||||
| **Bitmap** | String(位操作) | 签到、在线用户 |
|
||||
| **HyperLogLog** | String(基数统计) | UV 统计 |
|
||||
| **GEO** | ZSet(经纬度编码) | 附近的人 |
|
||||
|
||||
---
|
||||
|
||||
### 2. String(SDS - Simple Dynamic String)
|
||||
|
||||
**结构**:
|
||||
```c
|
||||
struct sdshdr {
|
||||
int len; // 已使用长度
|
||||
int free; // 剩余空间
|
||||
char buf[]; // 字节数组
|
||||
};
|
||||
```
|
||||
|
||||
**优势**:
|
||||
- O(1) 获取长度
|
||||
- 防止缓冲区溢出
|
||||
- 减少内存分配次数
|
||||
|
||||
**应用**:
|
||||
```bash
|
||||
# 缓存
|
||||
SET user:1001 '{"id":1001,"name":"Alice"}'
|
||||
GET user:1001
|
||||
|
||||
# 计数器
|
||||
INCR view_count:1001
|
||||
DECR stock:1001
|
||||
|
||||
# 分布式锁
|
||||
SET lock:order:1001 "uuid" NX PX 30000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Hash
|
||||
|
||||
**结构**:
|
||||
```bash
|
||||
HSET user:1001 name "Alice" age 25 email "alice@example.com"
|
||||
HGET user:1001 name
|
||||
HGETALL user:1001
|
||||
```
|
||||
|
||||
**底层**:
|
||||
- 字段少(< 512):压缩列表(ziplist)
|
||||
- 字段多(≥ 512):哈希表(hashtable)
|
||||
|
||||
**应用**:
|
||||
```java
|
||||
// 对象存储(推荐 Hash,而非 String)
|
||||
redisTemplate.opsForHash().putAll("user:1001", Map.of(
|
||||
"name", "Alice",
|
||||
"age", "25"
|
||||
));
|
||||
|
||||
// 购物车
|
||||
redisTemplate.opsForHash().put("cart:1001", "product:1001", "2");
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. List
|
||||
|
||||
**结构**:
|
||||
```bash
|
||||
LPUSH list:msgs "msg1" "msg2" "msg3" # 左侧插入
|
||||
RPOP list:msgs # 右侧弹出
|
||||
```
|
||||
|
||||
**底层**:
|
||||
- 元素少(< 512):压缩列表(ziplist)
|
||||
- 元素多(≥ 512):双向链表(linkedlist)
|
||||
- Redis 3.2+:quicklist(ziplist + linkedlist)
|
||||
|
||||
**应用**:
|
||||
```bash
|
||||
# 消息队列
|
||||
LPUSH queue:email '{"to":"alice@example.com","subject":"Hello"}'
|
||||
RPOP queue:email
|
||||
|
||||
# 最新列表
|
||||
LPUSH timeline:1001 "post1" "post2"
|
||||
LRANGE timeline:1001 0 9 # 最新 10 条
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Set vs ZSet
|
||||
|
||||
**Set(无序集合)**:
|
||||
```bash
|
||||
SADD tags:article:1001 "java" "redis" "mysql"
|
||||
SMEMBERS tags:article:1001
|
||||
SISMEMBER tags:article:1001 "java"
|
||||
SINTER tags:user:1001 tags:article:1001 # 交集
|
||||
```
|
||||
|
||||
**底层**:
|
||||
- 元素少(< 512):整数集合(intset)
|
||||
- 元素多(≥ 512):哈希表(hashtable)
|
||||
|
||||
---
|
||||
|
||||
**ZSet(有序集合)**:
|
||||
```bash
|
||||
ZADD rank:score 100 "player1" 200 "player2" 150 "player3"
|
||||
ZREVRANGE rank:score 0 9 WITHSCORES # Top 10
|
||||
ZRANK rank:score "player1" # 排名
|
||||
ZSCORE rank:score "player1" # 分数
|
||||
```
|
||||
|
||||
**底层**:
|
||||
- 元素少(< 128):压缩列表(ziplist)
|
||||
- 元素多(≥ 128):跳表(skiplist) + 哈希表(hashtable)
|
||||
|
||||
---
|
||||
|
||||
### 6. Bitmap
|
||||
|
||||
**原理**:用 bit 位表示状态(0 或 1)
|
||||
|
||||
```bash
|
||||
# 签到
|
||||
SETBIT sign:2024:02:28:1001 0 1 # 用户 1001 在第 0 天签到
|
||||
SETBIT sign:2024:02:28:1001 4 1 # 用户 1001 在第 4 天签到
|
||||
|
||||
# 统计签到天数
|
||||
BITCOUNT sign:2024:02:28:1001
|
||||
|
||||
# 用户 1001 和 1002 共同签到的天数
|
||||
BITOP AND result sign:2024:02:28:1001 sign:2024:02:28:1002
|
||||
BITCOUNT result
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 7. HyperLogLog
|
||||
|
||||
**用途**:基数统计(不重复元素个数)
|
||||
|
||||
**优点**:内存占用极小(12 KB)
|
||||
|
||||
```bash
|
||||
PFADD uv:2024:02:28 user:1001 user:1002 user:1003
|
||||
PFCOUNT uv:2024:02:28 # 3
|
||||
|
||||
# 合并多个 HyperLogLog
|
||||
PFMERGE uv:2024:02:01-28 uv:2024:02:01 uv:2024:02:02 ...
|
||||
```
|
||||
|
||||
**误差率**:< 1%
|
||||
|
||||
---
|
||||
|
||||
### 8. GEO(地理位置)
|
||||
|
||||
```bash
|
||||
# 添加位置
|
||||
GEOADD locations:users 116.404 39.915 "user:1001" # 北京
|
||||
|
||||
# 查找附近的人(5 km 内)
|
||||
GEORADIUS locations:users 116.404 39.915 5 km
|
||||
|
||||
# 计算距离
|
||||
GEODIST locations:users user:1001 user:1002
|
||||
```
|
||||
|
||||
**底层**:ZSet(经纬度编码为 score)
|
||||
|
||||
---
|
||||
|
||||
### 9. 阿里 P7 加分项
|
||||
|
||||
**深度理解**:
|
||||
- 理解 SDS 和 C 字符串的区别
|
||||
- 理解跳表的实现原理
|
||||
- 理解压缩列表的优缺点
|
||||
|
||||
**实战经验**:
|
||||
- 有选择合适数据类型的经验
|
||||
- 有大数据量下的优化经验
|
||||
- 有 Redis 内存优化的经验
|
||||
|
||||
**架构能力**:
|
||||
- 能设计基于 Redis 的业务方案
|
||||
- 能设计 Redis 集群方案
|
||||
- 能设计 Redis 监控体系
|
||||
0
questions/03-缓存/Redis架构.md
Normal file
0
questions/03-缓存/Redis架构.md
Normal file
1093
questions/03-缓存/缓存穿透击穿雪崩.md
Normal file
1093
questions/03-缓存/缓存穿透击穿雪崩.md
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user