forked from eaiadmin/rtos_llm_opt
422 lines
15 KiB
Markdown
422 lines
15 KiB
Markdown
# 方向5: 中断管理与实时性保障
|
||
|
||
## 1. 问题陈述
|
||
|
||
RTOS上的LLM推理不是孤立的系统。设备中还有其他硬实时任务(传感器、通信、控制)，它们与LLM任务共存于同一个RTOS内核中。核心问题：
|
||
|
||
1. **抢占冲突**: LLM的大计算量可能饿死其他实时任务
|
||
2. **中断风暴**: NPU完成中断 + 通信中断 + 传感器中断的并发处理
|
||
3. **Priority Inversion**: 低优先级的LLM任务可能阻塞高优先级任务
|
||
4. **资源竞争**: IRQ line、DMA channel、内存的共享竞争
|
||
|
||
## 2. 中断层级设计
|
||
|
||
### 2.1 中断优先级映射
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────┐
|
||
│ IRQ Hierarchy (示例: 基于ARM GIC) │
|
||
├─────────────────────────────────────────────────────────────────┤
|
||
│ │
|
||
│ Priority 255 (最高): Hard fault, Reset │
|
||
│ Priority 254: Watchdog, Critical error │
|
||
│ Priority 253: Real-time control (motor, actuator) │
|
||
│ Priority 252: Hard comm (ethernet MAC) │
|
||
│ Priority 251: Sensor interrupt (IMU, camera) │
|
||
│ Priority 250: Network interrupt (TCP timeout) │
|
||
│ ────────────────────────────────────────────── │
|
||
│ Priority 200: NPU/GPU completion interrupt │
|
||
│ Priority 199: DMA completion │
|
||
│ Priority 198: Timer interrupt (tick) │
|
||
│ ────────────────────────────────────────────── │
|
||
│ Priority 100: Software interrupt (task notification) │
|
||
│ Priority 50: Low-priority I2C/SPI │
|
||
│ Priority 0 (最低): Idle interrupt │
|
||
└─────────────────────────────────────────────────────────────────┘
|
||
|
||
Design Rule:
|
||
Real-time control tasks > NPU completion > DMA > Timer > Software
|
||
```
|
||
|
||
### 2.2 中断类型与处理策略
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────┐
|
||
│ IRQ Type | Handling Strategy │
|
||
├────────────────────┼────────────────────────────────────────┤
|
||
│ NPU Completion | 立即唤醒Attention task, 不延迟 │
|
||
│ DMA Complete | 标记完成, 通过event通知task │
|
||
│ Timer Tick | 标准tick, 用于调度时间片 │
|
||
│ Sensor Data | 高优先级, 触发数据处理task │
|
||
│ Comm Timeout | 中优先级, 可能触发重连task │
|
||
│ Control Command | 最高优先级, 触发控制task │
|
||
│ Error/Exception | 最高优先级, 触发error handling task │
|
||
└─────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
## 3. NPU/GPU完成中断处理
|
||
|
||
### 3.1 中断处理流程
|
||
|
||
```
|
||
NPU完成指令 → Interrupt Controller → RTOS IRQ Handler
|
||
|
||
1. NPU完成当前层 → 触发IRQ
|
||
2. ARM GIC收到中断 → 保存寄存器, 跳转到ISR
|
||
3. ISR: 读取NPU状态寄存器 → 确认完成 → 清中断
|
||
4. ISR: xSemaphoreGiveFromISR() → 唤醒等待task
|
||
5. ISR: 如果有更高优先级task ready → portYIELD_FROM_ISR()
|
||
6. 返回 → 恢复寄存器 → 执行task
|
||
```
|
||
|
||
### 3.2 ISR设计原则
|
||
|
||
```c
|
||
// NPU完成ISR (简化版)
|
||
volatile uint32_t npu_status_reg; // 硬件寄存器
|
||
SemaphoreHandle_t npu_done_sem; // RTOS信号量
|
||
|
||
void NPU_IRQ_Handler(void) {
|
||
// 1. 读取并确认NPU状态 (纯硬件操作)
|
||
uint32_t status = npu_status_reg;
|
||
if (status & NPU_STATUS_COMPLETE) {
|
||
// 2. 写寄存器清除中断标志
|
||
npu_status_reg &= ~NPU_STATUS_COMPLETE;
|
||
|
||
// 3. 释放信号量 (中断上下文)
|
||
BaseType_t xHigherPriorityTaskWoken = pdFALSE;
|
||
xSemaphoreGiveFromISR(npu_done_sem, &xHigherPriorityTaskWoken);
|
||
|
||
// 4. 如果有更高优先级task ready, 立即切换
|
||
if (xHigherPriorityTaskWoken) {
|
||
portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
|
||
}
|
||
}
|
||
}
|
||
|
||
// ISR设计原则:
|
||
// 1. 最小化: ISR中只做最少的操作 (<10μs)
|
||
// 2. 不阻塞: ISR中不使用sleep/等待
|
||
// 3. 用FromISR版本: 所有RTOS API都用ISR版本
|
||
// 4. 状态寄存器直接访问: 硬件寄存器直接读写
|
||
// 5. 事件通知为主: 优先使用event/group, 避免queue拷贝
|
||
```
|
||
|
||
## 4. Priority Inversion处理
|
||
|
||
### 4.1 经典场景分析
|
||
|
||
```
|
||
Priority Inversion in LLM Inference:
|
||
|
||
Task A (HIGH): Real-time control (needs NPU results)
|
||
Task B (MEDIUM): LLM Attention (uses NPU, holds mutex)
|
||
Task C (LOW): Background logging (locks NPU mutex)
|
||
|
||
Timeline:
|
||
t0: Task C acquires NPU mutex
|
||
t1: Task B starts, wants NPU → BLOCKED (waiting for C)
|
||
t2: Task A starts, wants NPU result → BLOCKED (waiting for B)
|
||
t3: Preemptive? No - C is LOW, B is MEDIUM
|
||
→ A waits for B, B waits for C
|
||
→ A is blocked by lower-priority C = PRIORITY INVERSION
|
||
|
||
Solution: Priority Inheritance Protocol (PIP)
|
||
t0: Task C acquires NPU mutex
|
||
t2: Task A blocked by B, B blocked by C
|
||
t3: C inherits A's priority → C becomes HIGH
|
||
t4: C releases mutex → C returns to LOW, A can proceed
|
||
```
|
||
|
||
### 4.2 RTOS中的PIP实现
|
||
|
||
```c
|
||
// PIP扩展: 适用于LLM推理的分级PIP
|
||
typedef struct {
|
||
uint32_t mutex_id;
|
||
uint32_t base_priority; // 持有者的基础优先级
|
||
uint32_t current_priority; // 继承后的优先级
|
||
TaskHandle_t holder; // 持有者
|
||
StackType_t *saved_stack; // 保存的栈指针
|
||
} pip_mutex_t;
|
||
|
||
void pip_take(pip_mutex_t *m) {
|
||
// 保存原始优先级
|
||
uint32_t orig_priority = task_get_priority(m->holder);
|
||
m->base_priority = orig_priority;
|
||
m->current_priority = orig_priority;
|
||
|
||
// 提升持有者优先级到最高等待者
|
||
uint32_t max_waiter = find_highest_waiting_priority(m->mutex_id);
|
||
task_set_priority(m->holder, max_waiter);
|
||
m->current_priority = max_waiter;
|
||
}
|
||
|
||
void pip_give(pip_mutex_t *m) {
|
||
// 恢复基础优先级
|
||
task_set_priority(m->holder, m->base_priority);
|
||
m->current_priority = m->base_priority;
|
||
}
|
||
|
||
// LLM特例: NPU mutex的PIP
|
||
// NPU mutex持有者通常是DMA task或NPU驱动
|
||
// 提升为Attention task的优先级
|
||
```
|
||
|
||
### 4.3 Enhanced PIP (ePIP) for LLM
|
||
|
||
```
|
||
Standard PIP的问题: 只继承一次, 可能传递到更高级
|
||
|
||
Enhanced PIP: 继承所有等待者的最高优先级
|
||
|
||
Scenario:
|
||
Task A (MAX): Control → waiting for NPU
|
||
Task B (HIGH): Attention → waiting for NPU
|
||
Task C (MED): KV Cache → waiting for NPU
|
||
Task D (LOW): DMA → holding NPU mutex
|
||
|
||
Standard PIP:
|
||
D inherits MAX (A's priority)
|
||
Problem: B (HIGH) may starve
|
||
|
||
Enhanced PIP:
|
||
D inherits MAX (A's priority)
|
||
B inherits MAX too (via queue, not mutex)
|
||
All high-priority tasks preempt D
|
||
|
||
RTOS Implementation:
|
||
// 使用Priority Queue记录所有等待者
|
||
typedef struct {
|
||
pip_mutex_t base;
|
||
PriorityQueueType_t wait_queue; // RTOS wait queue
|
||
uint32_t max_wait_priority; // 最高等待优先级
|
||
} epip_mutex_t;
|
||
|
||
void epip_take(epip_mutex_t *m) {
|
||
uint32_t max_prio = max_priority_in_queue(m->wait_queue);
|
||
task_set_priority(m->holder, max_prio);
|
||
m->max_wait_priority = max_prio;
|
||
}
|
||
```
|
||
|
||
## 5. 中断屏蔽与上下文切换
|
||
|
||
### 5.1 中断屏蔽策略
|
||
|
||
```
|
||
Critical Section (中断屏蔽):
|
||
┌─────────────────────────────────────────────────────────┐
|
||
│ Level 0: Global IRQ Disable │
|
||
│ 使用: 极短的critical section (<5μs) │
|
||
│ 场景: 写硬件寄存器, 更新共享计数器 │
|
||
│ 开销: 所有IRQ被屏蔽, 可能错过中断 │
|
||
│ │
|
||
│ Level 1: Task-level IRQ Disable │
|
||
│ 使用: task内critical section (≤50μs) │
|
||
│ 场景: 原子更新task状态, 修改task control block │
|
||
│ 开销: 只屏蔽当前task的中断, 其他task不受影响 │
|
||
│ │
|
||
│ Level 2: No IRQ Disable (锁+原子操作) │
|
||
│ 使用: 大多数critical section │
|
||
│ 场景: 修改共享数据结构, 使用atomic操作 │
|
||
│ 开销: 最小, 但需要确保原子性 │
|
||
└─────────────────────────────────────────────────────────┘
|
||
|
||
LLM推理中的critical sections:
|
||
1. Attention结果写入 (原子操作即可, 不需要全局disable)
|
||
2. KV Cache更新 (lock-free queue, 不需要disable)
|
||
3. Scheduler state update (task-level disable)
|
||
4. Precision change flag (global disable for <10μs)
|
||
```
|
||
|
||
### 5.2 上下文切换开销
|
||
|
||
```
|
||
Context Switch in RTOS:
|
||
1. Save current task registers (R0-R12, LR, PC, PSR)
|
||
2. Save/restore stack pointer
|
||
3. Update task control block
|
||
4. Restore next task registers
|
||
5. Restore stack pointer
|
||
6. Execute 'BX LR' (return from exception)
|
||
|
||
Cost:
|
||
Cortex-M7: ~10-50 cycles (with DCBP) ≈ 1-5μs @ 400MHz
|
||
ARM Cortex-A: ~100-500 cycles ≈ 50-250ns @ 2GHz
|
||
With MMU: ~2-10μs (TLB flush)
|
||
|
||
LLM Impact:
|
||
- 频繁context switch增加E2E延迟
|
||
- 每个switch增加 ~5μs (M7) ~2μs (A72)
|
||
- 假设每token 10次context switch: +50μs
|
||
|
||
Reduction strategies:
|
||
- Pin critical tasks to core (reduce switch)
|
||
- Use task notification instead of queue (reduce overhead)
|
||
- Batch task switches (wait for multiple events)
|
||
```
|
||
|
||
## 6. 实时性分析
|
||
|
||
### 6.1 Response Time Analysis (RTA) with Interrupts
|
||
|
||
```
|
||
Modified RTA for LLM with interrupts:
|
||
|
||
R_i = C_i + I_i + Σ_{j∈hp(i)} ⌈R_j / T_j⌉ × C_j
|
||
|
||
Where:
|
||
R_i: response time of task i
|
||
C_i: execution time of task i
|
||
I_i: interrupt-induced latency (中断导致的额外延迟)
|
||
Σ: interference from higher priority tasks
|
||
|
||
Interrupt-induced latency:
|
||
I_i = Σ_{k∈interrupts} (ISR_time_k + preemption_delay_k)
|
||
|
||
ISR_time_k: 中断k的处理时间
|
||
preemption_delay_k: 中断k导致的preemption开销
|
||
|
||
For LLM:
|
||
I_attention = ISR_time(npu_complete) + ISR_time(dma_complete)
|
||
+ preemption(attention → higher_priority_control)
|
||
|
||
Typical:
|
||
ISR_time(npu) ≈ 2μs
|
||
ISR_time(dma) ≈ 1μs
|
||
Preemption ≈ 5μs
|
||
Total interrupt overhead ≈ 8μs per inference step
|
||
```
|
||
|
||
### 6.2 中断导致的Jitter
|
||
|
||
```
|
||
Jitter来源分析:
|
||
┌─────────────────────────────────────────────────────────┐
|
||
│ Source | Latency (μs) | Probability │
|
||
├──────────────────────────┼──────────────┼────────────────┤
|
||
│ NPU completion IRQ | 2-5 | 100% (predictable) │
|
||
│ DMA completion IRQ | 1-3 | 100% (predictable) │
|
||
│ Timer tick | 1-2 | 100% (predictable) │
|
||
│ Preemption by control | 5-10 | Low (<5%) │
|
||
│ Preemption by comm | 5-15 | Medium (<20%) │
|
||
│ ISR overhead variance | 1-5 | High │
|
||
│ Cache miss (interrupt) | 10-50 | Variable │
|
||
└─────────────────────────────────────────────────────────┘
|
||
|
||
Worst-case Jitter:
|
||
Jitter = Max(R_i) - Min(R_i) across all inference steps
|
||
|
||
Predictable jitter: NPU/DMA IRQ (deterministic)
|
||
Unpredictable jitter: Preemption, Cache miss (stochastic)
|
||
|
||
Scheduling strategy:
|
||
- Minimize unpredictable jitter sources
|
||
- Bound predictable jitter through analysis
|
||
- Design with worst-case jitter in mind
|
||
```
|
||
|
||
## 7. 各平台的中断差异
|
||
|
||
### 7.1 MCU级
|
||
|
||
```
|
||
中断控制器: NVIC (ARM) / SCB (RISC-V)
|
||
中断数量: 10-50
|
||
中断优先级: 3-8 bits
|
||
特点:
|
||
- 中断延迟低 (<1μs)
|
||
- 中断嵌套简单 (固定层级)
|
||
- 无中断向量表动态重定位
|
||
- RTOS成熟, 中断集成简单
|
||
|
||
关键中断:
|
||
- SysTick (tick)
|
||
- NPU/ISP (如有)
|
||
- GPIO (传感器)
|
||
- UART/SPI (通信)
|
||
- DMA (数据传输)
|
||
```
|
||
|
||
### 7.2 SoC级
|
||
|
||
```
|
||
中断控制器: GICv3/v4 (ARM) / PLIC (RISC-V)
|
||
中断数量: 100-1000+
|
||
中断优先级: 8 bits
|
||
特点:
|
||
- 中断虚拟化支持 (但NPU驱动可能不暴露)
|
||
- 中断路由复杂 (多个master)
|
||
- MSI/MSI-X支持
|
||
- 中断聚合 (coalescing)
|
||
|
||
关键中断:
|
||
- ARM CPU cores (each has IRQ)
|
||
- NPU (interrupt via GIC)
|
||
- DMA controllers (multiple)
|
||
- Ethernet (MAC interrupt)
|
||
- Modem/Radio (connectivity)
|
||
- ISP (camera)
|
||
```
|
||
|
||
### 7.3 Edge盒子
|
||
|
||
```
|
||
中断控制器: GICv4 (Jetson Orin)
|
||
中断数量: 1000+
|
||
中断优先级: 8 bits
|
||
特点:
|
||
- 丰富的中断源
|
||
- PCIe MSIX支持
|
||
- GPU中断丰富 (渲染/计算)
|
||
- 中断亲和性可配置
|
||
|
||
关键中断:
|
||
- GPU (CUDA completion)
|
||
- NVLink (multi-GPU)
|
||
- PCIe (SSD/NPU)
|
||
- Ethernet
|
||
- Timer
|
||
```
|
||
|
||
### 7.4 Server级
|
||
|
||
```
|
||
中断控制器: GICv4 / MSI-X
|
||
中断数量: 数千
|
||
特点:
|
||
- 中断亲和性精细控制
|
||
- 中断聚合
|
||
- RSS (Receive Side Scattering)
|
||
- 中断向量池
|
||
|
||
RTOS角色:
|
||
- 主要处理网络/存储中断
|
||
- GPU中断由用户态处理
|
||
- RTOS保证关键路径上的中断响应
|
||
```
|
||
|
||
## 8. 关键设计决策
|
||
|
||
| 决策点 | 选项 | 推荐 | 理由 |
|
||
|-------|------|-----|------|
|
||
| PIP/ePIP | PIP / ePIP / No | **ePIP** | LLM场景优先级复杂 |
|
||
| ISR策略 | Minimal / Full | **Minimal** | ISR尽量短 |
|
||
| Critical Section | Global / Task-level / Lock-free | **混合** | 按场景选择 |
|
||
| Interrupt Mask | Disable / Priority / Vector | **Priority** | 灵活+实时 |
|
||
| Timer | Tick / Event | **Event** | 降低开销 |
|
||
| IRQ Affinity | Auto / Manual | **Manual** | 保证确定性 |
|
||
|
||
## 9. 开放研究问题
|
||
|
||
1. **Interrupt-aware Scheduling**: 将中断处理时间纳入调度分析？
|
||
2. **Interrupt Storm Handling**: 多中断并发时的优先级管理？
|
||
3. **Predictable Jitter**: 如何分析和保证jitter的确定性？
|
||
4. **Cross-core Interrupt**: 多核间的中断传播与同步？
|
||
5. **Interrupt-less Inference**: 轮询模式下的LLM调度？
|
||
|
||
---
|
||
|
||
*最后更新: 2026-09-17*
|