forked from eaiadmin/rtos_llm_opt
464 lines
14 KiB
Markdown
464 lines
14 KiB
Markdown
# 方向6: 能耗感知调度与热管理
|
||
|
||
## 1. 问题陈述
|
||
|
||
边缘/嵌入式设备运行LLM时，能耗和热是硬约束：
|
||
|
||
```
|
||
LLM推理能耗 = 计算能耗 + 内存带宽能耗 + 传输能耗
|
||
|
||
Example (Qwen2.5-1.5B, 100 tokens):
|
||
Compute (CPU): ~50mW × 2s = 100mJ
|
||
Memory (DDR): ~30mW × 2s = 60mJ
|
||
NPU (if used): ~200mW × 0.5s = 100mJ
|
||
Total: ~260mJ per inference (~200ms, 100 tokens)
|
||
|
||
Constraints:
|
||
- Battery: 3000mAh @ 3.8V = 40.68Wh = 146kJ
|
||
- Thermal: Tj < 85°C (Junction temperature)
|
||
- Form Factor: Passive cooling (no fan)
|
||
```
|
||
|
||
## 2. 能耗模型
|
||
|
||
### 2.1 各组件的能耗模型
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────┐
|
||
│ Component | Power (mW) | Active | Idle | Sleep │
|
||
├────────────────┼────────────┼────────┼──────┼───────────────┤
|
||
│ CPU Core | 80-200 | Active | 5 | 0.1 │
|
||
│ NPU | 100-500 | Active | 10 | 1 │
|
||
│ DDR | 50-150 | RW | 20 | 5 │
|
||
│ NPU Memory | 20-50 | Active | 5 | 1 │
|
||
│ Interconnect | 10-30 | Active | 2 | 0.5 │
|
||
│ GPU | 100-800 | Active | 15 | 5 │
|
||
└─────────────────────────────────────────────────────────────┘
|
||
|
||
Total Active Power: 300-1300mW
|
||
Total Idle Power: 40-100mW
|
||
Total Sleep Power: 1-10mW
|
||
|
||
Energy per inference step:
|
||
E = P_active × t_active + P_idle × t_idle + P_sleep × t_sleep
|
||
```
|
||
|
||
### 2.2 DVFS (Dynamic Voltage and Frequency Scaling) 模型
|
||
|
||
```
|
||
DVFS States (以Cortex-A72为例):
|
||
|
||
State | Frequency | Voltage | Power | Latency
|
||
-------|-----------|---------|---------|---------
|
||
S0 | 2.0 GHz | 1.2V | 200mW | 1x (baseline)
|
||
S1 | 1.5 GHz | 1.1V | 150mW | 1.33x
|
||
S2 | 1.0 GHz | 1.0V | 100mW | 2.0x
|
||
S3 | 0.5 GHz | 0.9V | 50mW | 4.0x
|
||
S4 | 100MHz | 0.8V | 15mW | 20x
|
||
|
||
Switching overhead: ~10-100μs (frequency ramp time)
|
||
|
||
Power-Frequency relationship:
|
||
P ∝ f × V² ∝ f × f^α ≈ f^(1+α)
|
||
|
||
where α ≈ 1-2 (depends on architecture)
|
||
|
||
So doubling frequency ≈ 2-4x power increase
|
||
```
|
||
|
||
### 2.3 能耗-延迟权衡
|
||
|
||
```
|
||
DVFS State | Latency | Power | Energy (per step) | Throughput
|
||
-----------|---------|-------|-------------------|----------
|
||
S0 (max) | 10ms | 200mW | 2.0mJ | 100 tok/s
|
||
S1 | 13ms | 150mW | 1.95mJ | 77 tok/s
|
||
S2 | 20ms | 100mW | 2.0mJ | 50 tok/s
|
||
S3 | 40ms | 50mW | 2.0mJ | 25 tok/s
|
||
S4 (min) | 200ms | 15mW | 3.0mJ | 5 tok/s
|
||
|
||
Insight:
|
||
- S1-S3 energy per step is similar (better power efficiency)
|
||
- S0: highest throughput, highest energy rate
|
||
- S4: lowest throughput, higher energy per step (overhead)
|
||
- Optimal: S1-S2 for latency-constrained, S2-S3 for energy-constrained
|
||
```
|
||
|
||
## 3. 能耗感知调度策略
|
||
|
||
### 3.1 静态调度策略
|
||
|
||
```
|
||
Strategy 1: Frequency Provisioning (固定频率)
|
||
1. Offline: Analyze workload, determine required frequency
|
||
2. Run at fixed DVFS state for entire inference
|
||
3. Simple, deterministic, but may waste energy
|
||
|
||
Example: Qwen2.5-1.5B inference at S1
|
||
- Guaranteed to meet 500ms deadline
|
||
- Uses 150mW instead of 200mW → 25% energy saved
|
||
- No runtime adaptation
|
||
|
||
Strategy 2: Power Capping
|
||
1. Set maximum power budget (e.g., 500mW)
|
||
2. Monitor power consumption
|
||
3. Scale frequency if exceeding budget
|
||
4. Reactive, but guarantees thermal safety
|
||
|
||
Example:
|
||
Power: ████████████████████░░░░░
|
||
^ ^
|
||
Starts at S0 Hits cap → Drops to S2
|
||
```
|
||
|
||
### 3.2 动态调度策略
|
||
|
||
```
|
||
Strategy 3: Adaptive DVFS (运行时调整)
|
||
1. Monitor: CPU load, temperature, battery level
|
||
2. Decide: Adjust DVFS state
|
||
3. Act: Switch frequency
|
||
4. Repeat: At each inference step
|
||
|
||
Decision variables:
|
||
- Current temperature (T)
|
||
- Battery level (B)
|
||
- Latency requirement (D)
|
||
- Thermal headroom (T_junction_max - T_current)
|
||
|
||
Decision function:
|
||
DVFS_state = f(T, B, D, thermal_headroom)
|
||
|
||
Pseudo-code:
|
||
if (temperature > 75°C):
|
||
drop_to_dvfs_state(S2)
|
||
elif (temperature > 70°C):
|
||
drop_to_dvfs_state(S1)
|
||
elif (battery < 20%):
|
||
drop_to_dvfs_state(S2)
|
||
elif (latency_requirement == strict):
|
||
run_at_dvfs_state(S0)
|
||
else:
|
||
run_at_dvfs_state(S1) # balanced
|
||
|
||
Strategy 4: Prediction-based Scheduling
|
||
1. Predict: Future workload (based on input size, sequence length)
|
||
2. Schedule: Set DVFS state in advance
|
||
3. Reduce: Switching overhead (pre-emptive adjustment)
|
||
|
||
Example:
|
||
Input: 2048 tokens (long input)
|
||
Predict: Prefill will be heavy → Set S0
|
||
During decode: Predict lighter → Drop to S1/S2
|
||
Before output: Predict completion → Drop to S3
|
||
|
||
Implementation:
|
||
Use RTOS timer + callback for predictive scheduling
|
||
```
|
||
|
||
### 3.3 任务级功耗控制
|
||
|
||
```
|
||
Task Power Profiling:
|
||
Task | Avg Power (mW) | Active Time (ms) | Energy (mJ)
|
||
--------------------|----------------|------------------|------------
|
||
Attention | 150 | 10 | 1.5
|
||
FFN | 180 | 8 | 1.44
|
||
KV Cache Write | 50 | 5 | 0.25
|
||
Tokenizer | 20 | 2 | 0.04
|
||
Control | 10 | 1 | 0.01
|
||
|
||
Task-level power control:
|
||
- Scale specific task's frequency based on urgency
|
||
- Attention: High frequency (latency-critical)
|
||
- KV Cache: Low frequency (IO-bound, less compute)
|
||
- Tokenizer: Variable (depends on input size)
|
||
|
||
RTOS Implementation:
|
||
void task_set_power_profile(task_t *task, PowerProfile profile) {
|
||
switch (profile) {
|
||
case PERFORMANCE:
|
||
task->frequency = MAX_FREQ;
|
||
task->voltage = MAX_VOLTAGE;
|
||
break;
|
||
case BALANCED:
|
||
task->frequency = MID_FREQ;
|
||
task->voltage = MID_VOLTAGE;
|
||
break;
|
||
case POWER_SAVING:
|
||
task->frequency = LOW_FREQ;
|
||
task->voltage = LOW_VOLTAGE;
|
||
break;
|
||
case ULTRA_LOW:
|
||
task->frequency = MIN_FREQ;
|
||
task->voltage = MIN_VOLTAGE;
|
||
break;
|
||
}
|
||
}
|
||
```
|
||
|
||
## 4. 热管理
|
||
|
||
### 4.1 热模型
|
||
|
||
```
|
||
Thermal Model (Simplified):
|
||
|
||
T_junction = T_ambient + P_total × R_thermal
|
||
|
||
Where:
|
||
T_junction: 芯片结温
|
||
T_ambient: 环境温度
|
||
P_total: 总功耗
|
||
R_thermal: 热阻 (package + heatsink + air)
|
||
|
||
Example (RK3588, passive cooling):
|
||
T_ambient = 40°C
|
||
R_thermal = 5°C/W
|
||
P_total = 5W (typical)
|
||
|
||
T_junction = 40 + 5 × 5 = 65°C
|
||
|
||
At P_total = 10W (LLM heavy):
|
||
T_junction = 40 + 5 × 10 = 90°C → Too hot!
|
||
|
||
Solution: Throttle to P_total = 6W
|
||
T_junction = 40 + 5 × 6 = 70°C → Acceptable
|
||
```
|
||
|
||
### 4.2 热感知调度
|
||
|
||
```
|
||
Thermal Throttling Strategy:
|
||
|
||
┌─────────────────────────────────────────────────────┐
|
||
│ Temperature Zone | Action │
|
||
├──────────────────────────────────────────────────────┤
|
||
│ Zone 1: < 60°C (Cool) | Full performance │
|
||
│ Zone 2: 60-70°C (Warm) | Light throttle (S1) │
|
||
│ Zone 3: 70-80°C (Hot) | Aggressive throttle (S2) │
|
||
│ Zone 4: 80-85°C (Hot) | Max throttle (S3) │
|
||
│ Zone 5: > 85°C (Critical)| Emergency shutdown │
|
||
└─────────────────────────────────────────────────────┘
|
||
|
||
Implementation:
|
||
Thermal Zone Controller (low-priority RTOS task):
|
||
|
||
void thermal_controller_task(void *params) {
|
||
for (;;) {
|
||
float temp = read_temperature_sensor();
|
||
ThermalZone zone = classify_temperature(temp);
|
||
|
||
// Adjust DVFS based on zone
|
||
switch (zone) {
|
||
case ZONE_COOL:
|
||
set_dvfs_state(S0);
|
||
break;
|
||
case ZONE_WARM:
|
||
set_dvfs_state(S1);
|
||
break;
|
||
case ZONE_HOT:
|
||
set_dvfs_state(S2);
|
||
reduce_llm_priority();
|
||
break;
|
||
case ZONE_HOTTER:
|
||
set_dvfs_state(S3);
|
||
reduce_llm_priority();
|
||
break;
|
||
case ZONE_CRITICAL:
|
||
emergency_shutdown();
|
||
break;
|
||
}
|
||
|
||
vTaskDelay(pdMS_TO_TICKS(100)); // Check every 100ms
|
||
}
|
||
}
|
||
```
|
||
|
||
### 4.3 热-调度联合优化
|
||
|
||
```
|
||
Joint Thermal-Scheduling Optimization:
|
||
|
||
Objective: Minimize energy, subject to thermal constraints
|
||
|
||
Variables:
|
||
- DVFS state per time slot
|
||
- Task scheduling per time slot
|
||
- Task placement (core assignment)
|
||
|
||
Constraints:
|
||
- T_junction(t) ≤ 85°C for all t
|
||
- Latency ≤ deadline
|
||
- Throughput ≥ minimum
|
||
|
||
Solution approach:
|
||
1. Model thermal dynamics (heat equation)
|
||
2. Predict temperature profile for each scheduling option
|
||
3. Select scheduling that minimizes energy without thermal violation
|
||
|
||
Simplified approach (RTOS-friendly):
|
||
- Use PID controller for temperature
|
||
- Setpoint: 75°C (target), 85°C (max)
|
||
- Control variable: DVFS state
|
||
- Disturbance: Task load changes
|
||
```
|
||
|
||
## 5. 空闲与低功耗状态管理
|
||
|
||
### 5.1 推理间隙的低功耗
|
||
|
||
```
|
||
LLM推理的idle间隙:
|
||
|
||
Prefill (20ms) → Decode (10ms × 100) → Idle
|
||
|
||
Decode间隙: 每层计算之间有微秒级间隙
|
||
Prefill-Decode间隙: 20ms的完全空闲
|
||
|
||
Power-down opportunities:
|
||
- NPU can enter sleep during decode间隙
|
||
- DDR can enter self-refresh during间隙
|
||
- CPU cores can sleep (if no pending tasks)
|
||
|
||
RTOS低功耗策略:
|
||
|
||
1. Tickless idle: 不使用固定tick, 动态调整tick间隔
|
||
2. Deep sleep: 推理间隙进入深睡眠
|
||
3. Clock gating: 禁用未使用外设的时钟
|
||
4. RAM retention: 深睡眠时保持RAM内容
|
||
|
||
Implementation:
|
||
// 推理间隙功耗管理
|
||
void manage_inference_power(void) {
|
||
if (npu_idle && cpu_idle) {
|
||
// Enter low-power mode
|
||
enter_deep_sleep();
|
||
|
||
// Wake on NPU interrupt or timer
|
||
sleep_until(wake_source);
|
||
|
||
// Restore state
|
||
restore_from_deep_sleep();
|
||
}
|
||
}
|
||
```
|
||
|
||
### 5.2 动态功耗监控
|
||
|
||
```
|
||
Power Monitoring (RTOS Task):
|
||
|
||
void power_monitor_task(void *params) {
|
||
for (;;) {
|
||
// Read power sensors
|
||
float cpu_power = read_power_sensor(CPU);
|
||
float npu_power = read_power_sensor(NPU);
|
||
float ddr_power = read_power_sensor(DDR);
|
||
float total_power = cpu_power + npu_power + ddr_power;
|
||
|
||
// Update power model
|
||
update_power_model(total_power);
|
||
|
||
// Check thresholds
|
||
if (total_power > POWER_LIMIT) {
|
||
trigger_throttling();
|
||
}
|
||
|
||
if (battery_level < BATTERY_WARN) {
|
||
notify_low_battery();
|
||
}
|
||
|
||
vTaskDelay(pdMS_TO_TICKS(10)); // Check every 10ms
|
||
}
|
||
}
|
||
```
|
||
|
||
## 6. 各平台的功耗-调度差异
|
||
|
||
### 6.1 MCU级
|
||
|
||
```
|
||
功耗特性:
|
||
- Sleep: < 1μW (deep sleep)
|
||
- Active: 50-200mW
|
||
- DVFS: 有限 (2-4 states)
|
||
- 热: 被动散热, R_thermal ~10°C/W
|
||
|
||
策略:
|
||
- 大量使用sleep模式
|
||
- 计算密集型任务集中执行, 然后sleep
|
||
- DVFS简单, 调度变化小
|
||
```
|
||
|
||
### 6.2 SoC级
|
||
|
||
```
|
||
功耗特性:
|
||
- Sleep: ~10mW (peripheral sleep)
|
||
- Active: 300-1300mW
|
||
- DVFS: 丰富 (8+ states)
|
||
- 热: 被动散热, R_thermal ~5°C/W
|
||
|
||
策略:
|
||
- 精细的DVFS控制
|
||
- 多核独立频率
|
||
- NPU专属低功耗模式
|
||
- 热感知调度 (关键)
|
||
```
|
||
|
||
### 6.3 Edge盒子
|
||
|
||
```
|
||
功耗特性:
|
||
- Sleep: ~50mW (standby)
|
||
- Active: 500-2000mW
|
||
- DVFS: 丰富 (10+ states)
|
||
- 热: 主动+被动散热
|
||
|
||
策略:
|
||
- 精细的热管理
|
||
- CPU-NPU负载均衡
|
||
- 功耗capping
|
||
- 预测性调度
|
||
```
|
||
|
||
### 6.4 Server级
|
||
|
||
```
|
||
功耗特性:
|
||
- Sleep: ~1W
|
||
- Active: 100-500W
|
||
- DVFS: 极丰富
|
||
- 热: 主动散热 (fan + heatsink)
|
||
|
||
策略:
|
||
- PUE优化
|
||
- GPU NVLink功耗管理
|
||
- NUMA功耗均衡
|
||
- 数据中心级功耗管理
|
||
```
|
||
|
||
## 7. 关键设计决策
|
||
|
||
| 决策点 | 选项 | 推荐 | 理由 |
|
||
|-------|------|-----|------|
|
||
| DVFS粒度 | Global / Per-core / Per-device | **Per-core** | 灵活性+实时性可接受 |
|
||
| 热管理 | PID / Rule-based / ML | **PID** | 确定性+可分析 |
|
||
| 空闲管理 | Tickless / Deep-sleep / Clock-gate | **混合** | 按场景选择 |
|
||
| 功耗监控 | Hardware / Software | **Hardware** | 精度+低开销 |
|
||
| 热感知 | Reactive / Predictive | **Predictive** | 减少延迟抖动 |
|
||
| 模式切换 | Manual / Auto / Hybrid | **Auto** | 自适应环境变化 |
|
||
|
||
## 8. 开放研究问题
|
||
|
||
1. **Predictive Power**: 基于输入预测功耗, 提前调整DVFS？
|
||
2. **Thermal-aware Task Placement**: 多核场景下的热均衡调度？
|
||
3. **Battery-aware Scheduling**: 电池电量变化时的调度策略自适应？
|
||
4. **Cross-component Thermal**: CPU+NPU+DDR的联合热建模？
|
||
5. **Power-perf SLA**: 同时满足性能和功耗SLA的调度？
|
||
|
||
---
|
||
|
||
*最后更新: 2026-09-17*
|