# 方向6: 能耗感知调度与热管理 ## 1. 问题陈述 边缘/嵌入式设备运行LLM时，能耗和热是硬约束： ``` LLM推理能耗 = 计算能耗 + 内存带宽能耗 + 传输能耗 Example (Qwen2.5-1.5B, 100 tokens): Compute (CPU): ~50mW × 2s = 100mJ Memory (DDR): ~30mW × 2s = 60mJ NPU (if used): ~200mW × 0.5s = 100mJ Total: ~260mJ per inference (~200ms, 100 tokens) Constraints: - Battery: 3000mAh @ 3.8V = 40.68Wh = 146kJ - Thermal: Tj < 85°C (Junction temperature) - Form Factor: Passive cooling (no fan) ``` ## 2. 能耗模型 ### 2.1 各组件的能耗模型 ``` ┌─────────────────────────────────────────────────────────────┐ │ Component | Power (mW) | Active | Idle | Sleep │ ├────────────────┼────────────┼────────┼──────┼───────────────┤ │ CPU Core | 80-200 | Active | 5 | 0.1 │ │ NPU | 100-500 | Active | 10 | 1 │ │ DDR | 50-150 | RW | 20 | 5 │ │ NPU Memory | 20-50 | Active | 5 | 1 │ │ Interconnect | 10-30 | Active | 2 | 0.5 │ │ GPU | 100-800 | Active | 15 | 5 │ └─────────────────────────────────────────────────────────────┘ Total Active Power: 300-1300mW Total Idle Power: 40-100mW Total Sleep Power: 1-10mW Energy per inference step: E = P_active × t_active + P_idle × t_idle + P_sleep × t_sleep ``` ### 2.2 DVFS (Dynamic Voltage and Frequency Scaling) 模型 ``` DVFS States (以Cortex-A72为例): State | Frequency | Voltage | Power | Latency -------|-----------|---------|---------|--------- S0 | 2.0 GHz | 1.2V | 200mW | 1x (baseline) S1 | 1.5 GHz | 1.1V | 150mW | 1.33x S2 | 1.0 GHz | 1.0V | 100mW | 2.0x S3 | 0.5 GHz | 0.9V | 50mW | 4.0x S4 | 100MHz | 0.8V | 15mW | 20x Switching overhead: ~10-100μs (frequency ramp time) Power-Frequency relationship: P ∝ f × V² ∝ f × f^α ≈ f^(1+α) where α ≈ 1-2 (depends on architecture) So doubling frequency ≈ 2-4x power increase ``` ### 2.3 能耗-延迟权衡 ``` DVFS State | Latency | Power | Energy (per step) | Throughput -----------|---------|-------|-------------------|---------- S0 (max) | 10ms | 200mW | 2.0mJ | 100 tok/s S1 | 13ms | 150mW | 1.95mJ | 77 tok/s S2 | 20ms | 100mW | 2.0mJ | 50 tok/s S3 | 40ms | 50mW | 2.0mJ | 25 tok/s S4 (min) | 200ms | 15mW | 3.0mJ | 5 tok/s Insight: - S1-S3 energy per step is similar (better power efficiency) - S0: highest throughput, highest energy rate - S4: lowest throughput, higher energy per step (overhead) - Optimal: S1-S2 for latency-constrained, S2-S3 for energy-constrained ``` ## 3. 能耗感知调度策略 ### 3.1 静态调度策略 ``` Strategy 1: Frequency Provisioning (固定频率) 1. Offline: Analyze workload, determine required frequency 2. Run at fixed DVFS state for entire inference 3. Simple, deterministic, but may waste energy Example: Qwen2.5-1.5B inference at S1 - Guaranteed to meet 500ms deadline - Uses 150mW instead of 200mW → 25% energy saved - No runtime adaptation Strategy 2: Power Capping 1. Set maximum power budget (e.g., 500mW) 2. Monitor power consumption 3. Scale frequency if exceeding budget 4. Reactive, but guarantees thermal safety Example: Power: ████████████████████░░░░░ ^ ^ Starts at S0 Hits cap → Drops to S2 ``` ### 3.2 动态调度策略 ``` Strategy 3: Adaptive DVFS (运行时调整) 1. Monitor: CPU load, temperature, battery level 2. Decide: Adjust DVFS state 3. Act: Switch frequency 4. Repeat: At each inference step Decision variables: - Current temperature (T) - Battery level (B) - Latency requirement (D) - Thermal headroom (T_junction_max - T_current) Decision function: DVFS_state = f(T, B, D, thermal_headroom) Pseudo-code: if (temperature > 75°C): drop_to_dvfs_state(S2) elif (temperature > 70°C): drop_to_dvfs_state(S1) elif (battery < 20%): drop_to_dvfs_state(S2) elif (latency_requirement == strict): run_at_dvfs_state(S0) else: run_at_dvfs_state(S1) # balanced Strategy 4: Prediction-based Scheduling 1. Predict: Future workload (based on input size, sequence length) 2. Schedule: Set DVFS state in advance 3. Reduce: Switching overhead (pre-emptive adjustment) Example: Input: 2048 tokens (long input) Predict: Prefill will be heavy → Set S0 During decode: Predict lighter → Drop to S1/S2 Before output: Predict completion → Drop to S3 Implementation: Use RTOS timer + callback for predictive scheduling ``` ### 3.3 任务级功耗控制 ``` Task Power Profiling: Task | Avg Power (mW) | Active Time (ms) | Energy (mJ) --------------------|----------------|------------------|------------ Attention | 150 | 10 | 1.5 FFN | 180 | 8 | 1.44 KV Cache Write | 50 | 5 | 0.25 Tokenizer | 20 | 2 | 0.04 Control | 10 | 1 | 0.01 Task-level power control: - Scale specific task's frequency based on urgency - Attention: High frequency (latency-critical) - KV Cache: Low frequency (IO-bound, less compute) - Tokenizer: Variable (depends on input size) RTOS Implementation: void task_set_power_profile(task_t *task, PowerProfile profile) { switch (profile) { case PERFORMANCE: task->frequency = MAX_FREQ; task->voltage = MAX_VOLTAGE; break; case BALANCED: task->frequency = MID_FREQ; task->voltage = MID_VOLTAGE; break; case POWER_SAVING: task->frequency = LOW_FREQ; task->voltage = LOW_VOLTAGE; break; case ULTRA_LOW: task->frequency = MIN_FREQ; task->voltage = MIN_VOLTAGE; break; } } ``` ## 4. 热管理 ### 4.1 热模型 ``` Thermal Model (Simplified): T_junction = T_ambient + P_total × R_thermal Where: T_junction: 芯片结温 T_ambient: 环境温度 P_total: 总功耗 R_thermal: 热阻 (package + heatsink + air) Example (RK3588, passive cooling): T_ambient = 40°C R_thermal = 5°C/W P_total = 5W (typical) T_junction = 40 + 5 × 5 = 65°C At P_total = 10W (LLM heavy): T_junction = 40 + 5 × 10 = 90°C → Too hot! Solution: Throttle to P_total = 6W T_junction = 40 + 5 × 6 = 70°C → Acceptable ``` ### 4.2 热感知调度 ``` Thermal Throttling Strategy: ┌─────────────────────────────────────────────────────┐ │ Temperature Zone | Action │ ├──────────────────────────────────────────────────────┤ │ Zone 1: < 60°C (Cool) | Full performance │ │ Zone 2: 60-70°C (Warm) | Light throttle (S1) │ │ Zone 3: 70-80°C (Hot) | Aggressive throttle (S2) │ │ Zone 4: 80-85°C (Hot) | Max throttle (S3) │ │ Zone 5: > 85°C (Critical)| Emergency shutdown │ └─────────────────────────────────────────────────────┘ Implementation: Thermal Zone Controller (low-priority RTOS task): void thermal_controller_task(void *params) { for (;;) { float temp = read_temperature_sensor(); ThermalZone zone = classify_temperature(temp); // Adjust DVFS based on zone switch (zone) { case ZONE_COOL: set_dvfs_state(S0); break; case ZONE_WARM: set_dvfs_state(S1); break; case ZONE_HOT: set_dvfs_state(S2); reduce_llm_priority(); break; case ZONE_HOTTER: set_dvfs_state(S3); reduce_llm_priority(); break; case ZONE_CRITICAL: emergency_shutdown(); break; } vTaskDelay(pdMS_TO_TICKS(100)); // Check every 100ms } } ``` ### 4.3 热-调度联合优化 ``` Joint Thermal-Scheduling Optimization: Objective: Minimize energy, subject to thermal constraints Variables: - DVFS state per time slot - Task scheduling per time slot - Task placement (core assignment) Constraints: - T_junction(t) ≤ 85°C for all t - Latency ≤ deadline - Throughput ≥ minimum Solution approach: 1. Model thermal dynamics (heat equation) 2. Predict temperature profile for each scheduling option 3. Select scheduling that minimizes energy without thermal violation Simplified approach (RTOS-friendly): - Use PID controller for temperature - Setpoint: 75°C (target), 85°C (max) - Control variable: DVFS state - Disturbance: Task load changes ``` ## 5. 空闲与低功耗状态管理 ### 5.1 推理间隙的低功耗 ``` LLM推理的idle间隙: Prefill (20ms) → Decode (10ms × 100) → Idle Decode间隙: 每层计算之间有微秒级间隙 Prefill-Decode间隙: 20ms的完全空闲 Power-down opportunities: - NPU can enter sleep during decode间隙 - DDR can enter self-refresh during间隙 - CPU cores can sleep (if no pending tasks) RTOS低功耗策略: 1. Tickless idle: 不使用固定tick, 动态调整tick间隔 2. Deep sleep: 推理间隙进入深睡眠 3. Clock gating: 禁用未使用外设的时钟 4. RAM retention: 深睡眠时保持RAM内容 Implementation: // 推理间隙功耗管理 void manage_inference_power(void) { if (npu_idle && cpu_idle) { // Enter low-power mode enter_deep_sleep(); // Wake on NPU interrupt or timer sleep_until(wake_source); // Restore state restore_from_deep_sleep(); } } ``` ### 5.2 动态功耗监控 ``` Power Monitoring (RTOS Task): void power_monitor_task(void *params) { for (;;) { // Read power sensors float cpu_power = read_power_sensor(CPU); float npu_power = read_power_sensor(NPU); float ddr_power = read_power_sensor(DDR); float total_power = cpu_power + npu_power + ddr_power; // Update power model update_power_model(total_power); // Check thresholds if (total_power > POWER_LIMIT) { trigger_throttling(); } if (battery_level < BATTERY_WARN) { notify_low_battery(); } vTaskDelay(pdMS_TO_TICKS(10)); // Check every 10ms } } ``` ## 6. 各平台的功耗-调度差异 ### 6.1 MCU级 ``` 功耗特性: - Sleep: < 1μW (deep sleep) - Active: 50-200mW - DVFS: 有限 (2-4 states) - 热: 被动散热, R_thermal ~10°C/W 策略: - 大量使用sleep模式 - 计算密集型任务集中执行, 然后sleep - DVFS简单, 调度变化小 ``` ### 6.2 SoC级 ``` 功耗特性: - Sleep: ~10mW (peripheral sleep) - Active: 300-1300mW - DVFS: 丰富 (8+ states) - 热: 被动散热, R_thermal ~5°C/W 策略: - 精细的DVFS控制 - 多核独立频率 - NPU专属低功耗模式 - 热感知调度 (关键) ``` ### 6.3 Edge盒子 ``` 功耗特性: - Sleep: ~50mW (standby) - Active: 500-2000mW - DVFS: 丰富 (10+ states) - 热: 主动+被动散热 策略: - 精细的热管理 - CPU-NPU负载均衡 - 功耗capping - 预测性调度 ``` ### 6.4 Server级 ``` 功耗特性: - Sleep: ~1W - Active: 100-500W - DVFS: 极丰富 - 热: 主动散热 (fan + heatsink) 策略: - PUE优化 - GPU NVLink功耗管理 - NUMA功耗均衡 - 数据中心级功耗管理 ``` ## 7. 关键设计决策 | 决策点 | 选项 | 推荐 | 理由 | |-------|------|-----|------| | DVFS粒度 | Global / Per-core / Per-device | **Per-core** | 灵活性+实时性可接受 | | 热管理 | PID / Rule-based / ML | **PID** | 确定性+可分析 | | 空闲管理 | Tickless / Deep-sleep / Clock-gate | **混合** | 按场景选择 | | 功耗监控 | Hardware / Software | **Hardware** | 精度+低开销 | | 热感知 | Reactive / Predictive | **Predictive** | 减少延迟抖动 | | 模式切换 | Manual / Auto / Hybrid | **Auto** | 自适应环境变化 | ## 8. 开放研究问题 1. **Predictive Power**: 基于输入预测功耗, 提前调整DVFS？ 2. **Thermal-aware Task Placement**: 多核场景下的热均衡调度？ 3. **Battery-aware Scheduling**: 电池电量变化时的调度策略自适应？ 4. **Cross-component Thermal**: CPU+NPU+DDR的联合热建模？ 5. **Power-perf SLA**: 同时满足性能和功耗SLA的调度？ --- *最后更新: 2026-09-17*