rtos_llm_opt/06-power-thermal.md

方向6: 能耗感知调度与热管理

1. 问题陈述

边缘/嵌入式设备运行LLM时，能耗和热是硬约束：

LLM推理能耗 = 计算能耗 + 内存带宽能耗 + 传输能耗

Example (Qwen2.5-1.5B, 100 tokens):
  Compute (CPU):     ~50mW × 2s = 100mJ
  Memory (DDR):      ~30mW × 2s = 60mJ
  NPU (if used):     ~200mW × 0.5s = 100mJ
  Total:             ~260mJ per inference (~200ms, 100 tokens)

Constraints:
  - Battery: 3000mAh @ 3.8V = 40.68Wh = 146kJ
  - Thermal: Tj < 85°C (Junction temperature)
  - Form Factor: Passive cooling (no fan)

2. 能耗模型

2.1 各组件的能耗模型

┌─────────────────────────────────────────────────────────────┐
│  Component     | Power (mW) | Active | Idle | Sleep         │
├────────────────┼────────────┼────────┼──────┼───────────────┤
│  CPU Core      | 80-200     | Active | 5    | 0.1           │
│  NPU           | 100-500    | Active | 10   | 1             │
│  DDR           | 50-150     | RW     | 20   | 5             │
│  NPU Memory    | 20-50      | Active | 5    | 1             │
│  Interconnect  | 10-30      | Active | 2    | 0.5           │
│  GPU           | 100-800    | Active | 15   | 5             │
└─────────────────────────────────────────────────────────────┘

Total Active Power: 300-1300mW
Total Idle Power:   40-100mW
Total Sleep Power:  1-10mW

Energy per inference step:
  E = P_active × t_active + P_idle × t_idle + P_sleep × t_sleep

2.2 DVFS (Dynamic Voltage and Frequency Scaling) 模型

DVFS States (以Cortex-A72为例):
  
  State  | Frequency | Voltage | Power   | Latency
  -------|-----------|---------|---------|---------
  S0     | 2.0 GHz   | 1.2V    | 200mW   | 1x (baseline)
  S1     | 1.5 GHz   | 1.1V    | 150mW   | 1.33x
  S2     | 1.0 GHz   | 1.0V    | 100mW   | 2.0x
  S3     | 0.5 GHz   | 0.9V    | 50mW    | 4.0x
  S4     | 100MHz    | 0.8V    | 15mW    | 20x

  Switching overhead: ~10-100μs (frequency ramp time)
  
  Power-Frequency relationship:
    P ∝ f × V² ∝ f × f^α ≈ f^(1+α)
    
    where α ≈ 1-2 (depends on architecture)
    
    So doubling frequency ≈ 2-4x power increase

2.3 能耗-延迟权衡

DVFS State | Latency | Power | Energy (per step) | Throughput
-----------|---------|-------|-------------------|----------
S0 (max)   | 10ms    | 200mW | 2.0mJ             | 100 tok/s
S1         | 13ms    | 150mW | 1.95mJ            | 77 tok/s
S2         | 20ms    | 100mW | 2.0mJ             | 50 tok/s
S3         | 40ms    | 50mW  | 2.0mJ             | 25 tok/s
S4 (min)   | 200ms   | 15mW  | 3.0mJ             | 5 tok/s

Insight:
  - S1-S3 energy per step is similar (better power efficiency)
  - S0: highest throughput, highest energy rate
  - S4: lowest throughput, higher energy per step (overhead)
  - Optimal: S1-S2 for latency-constrained, S2-S3 for energy-constrained

3. 能耗感知调度策略

3.1 静态调度策略

Strategy 1: Frequency Provisioning (固定频率)
  1. Offline: Analyze workload, determine required frequency
  2. Run at fixed DVFS state for entire inference
  3. Simple, deterministic, but may waste energy
  
  Example: Qwen2.5-1.5B inference at S1
    - Guaranteed to meet 500ms deadline
    - Uses 150mW instead of 200mW → 25% energy saved
    - No runtime adaptation

Strategy 2: Power Capping
  1. Set maximum power budget (e.g., 500mW)
  2. Monitor power consumption
  3. Scale frequency if exceeding budget
  4. Reactive, but guarantees thermal safety
  
  Example:
    Power: ████████████████████░░░░░
           ^                   ^
        Starts at S0       Hits cap → Drops to S2

3.2 动态调度策略

Strategy 3: Adaptive DVFS (运行时调整)
  1. Monitor: CPU load, temperature, battery level
  2. Decide: Adjust DVFS state
  3. Act: Switch frequency
  4. Repeat: At each inference step
  
  Decision variables:
    - Current temperature (T)
    - Battery level (B)
    - Latency requirement (D)
    - Thermal headroom (T_junction_max - T_current)
  
  Decision function:
    DVFS_state = f(T, B, D, thermal_headroom)
  
  Pseudo-code:
    if (temperature > 75°C):
        drop_to_dvfs_state(S2)
    elif (temperature > 70°C):
        drop_to_dvfs_state(S1)
    elif (battery < 20%):
        drop_to_dvfs_state(S2)
    elif (latency_requirement == strict):
        run_at_dvfs_state(S0)
    else:
        run_at_dvfs_state(S1)  # balanced

Strategy 4: Prediction-based Scheduling
  1. Predict: Future workload (based on input size, sequence length)
  2. Schedule: Set DVFS state in advance
  3. Reduce: Switching overhead (pre-emptive adjustment)
  
  Example:
    Input: 2048 tokens (long input)
    Predict: Prefill will be heavy → Set S0
    During decode: Predict lighter → Drop to S1/S2
    Before output: Predict completion → Drop to S3
    
  Implementation:
    Use RTOS timer + callback for predictive scheduling

3.3 任务级功耗控制

Task Power Profiling:
  Task                | Avg Power (mW) | Active Time (ms) | Energy (mJ)
  --------------------|----------------|------------------|------------
  Attention           | 150            | 10               | 1.5
  FFN                 | 180            | 8                | 1.44
  KV Cache Write      | 50             | 5                | 0.25
  Tokenizer           | 20             | 2                | 0.04
  Control             | 10             | 1                | 0.01

Task-level power control:
  - Scale specific task's frequency based on urgency
  - Attention: High frequency (latency-critical)
  - KV Cache: Low frequency (IO-bound, less compute)
  - Tokenizer: Variable (depends on input size)

RTOS Implementation:
  void task_set_power_profile(task_t *task, PowerProfile profile) {
      switch (profile) {
          case PERFORMANCE:
              task->frequency = MAX_FREQ;
              task->voltage = MAX_VOLTAGE;
              break;
          case BALANCED:
              task->frequency = MID_FREQ;
              task->voltage = MID_VOLTAGE;
              break;
          case POWER_SAVING:
              task->frequency = LOW_FREQ;
              task->voltage = LOW_VOLTAGE;
              break;
          case ULTRA_LOW:
              task->frequency = MIN_FREQ;
              task->voltage = MIN_VOLTAGE;
              break;
      }
  }

4. 热管理

4.1 热模型

Thermal Model (Simplified):
  
  T_junction = T_ambient + P_total × R_thermal
  
  Where:
    T_junction: 芯片结温
    T_ambient: 环境温度
    P_total: 总功耗
    R_thermal: 热阻 (package + heatsink + air)
  
  Example (RK3588, passive cooling):
    T_ambient = 40°C
    R_thermal = 5°C/W
    P_total = 5W (typical)
    
    T_junction = 40 + 5 × 5 = 65°C
    
    At P_total = 10W (LLM heavy):
    T_junction = 40 + 5 × 10 = 90°C → Too hot!
    
    Solution: Throttle to P_total = 6W
    T_junction = 40 + 5 × 6 = 70°C → Acceptable

4.2 热感知调度

Thermal Throttling Strategy:

  ┌─────────────────────────────────────────────────────┐
│  Temperature Zone       | Action                     │
├──────────────────────────────────────────────────────┤
│  Zone 1: < 60°C (Cool)  | Full performance           │
│  Zone 2: 60-70°C (Warm) | Light throttle (S1)        │
│  Zone 3: 70-80°C (Hot)  | Aggressive throttle (S2)   │
│  Zone 4: 80-85°C (Hot)  | Max throttle (S3)          │
│  Zone 5: > 85°C (Critical)| Emergency shutdown       │
└─────────────────────────────────────────────────────┘

Implementation:
  Thermal Zone Controller (low-priority RTOS task):
  
  void thermal_controller_task(void *params) {
      for (;;) {
          float temp = read_temperature_sensor();
          ThermalZone zone = classify_temperature(temp);
          
          // Adjust DVFS based on zone
          switch (zone) {
              case ZONE_COOL:
                  set_dvfs_state(S0);
                  break;
              case ZONE_WARM:
                  set_dvfs_state(S1);
                  break;
              case ZONE_HOT:
                  set_dvfs_state(S2);
                  reduce_llm_priority();
                  break;
              case ZONE_HOTTER:
                  set_dvfs_state(S3);
                  reduce_llm_priority();
                  break;
              case ZONE_CRITICAL:
                  emergency_shutdown();
                  break;
          }
          
          vTaskDelay(pdMS_TO_TICKS(100)); // Check every 100ms
      }
  }

4.3 热-调度联合优化

Joint Thermal-Scheduling Optimization:
  
  Objective: Minimize energy, subject to thermal constraints
  
  Variables:
    - DVFS state per time slot
    - Task scheduling per time slot
    - Task placement (core assignment)
  
  Constraints:
    - T_junction(t) ≤ 85°C for all t
    - Latency ≤ deadline
    - Throughput ≥ minimum
  
  Solution approach:
    1. Model thermal dynamics (heat equation)
    2. Predict temperature profile for each scheduling option
    3. Select scheduling that minimizes energy without thermal violation
    
  Simplified approach (RTOS-friendly):
    - Use PID controller for temperature
    - Setpoint: 75°C (target), 85°C (max)
    - Control variable: DVFS state
    - Disturbance: Task load changes

5. 空闲与低功耗状态管理

5.1 推理间隙的低功耗

LLM推理的idle间隙:
  
  Prefill (20ms) → Decode (10ms × 100) → Idle
  
  Decode间隙: 每层计算之间有微秒级间隙
  Prefill-Decode间隙: 20ms的完全空闲
  
  Power-down opportunities:
  - NPU can enter sleep during decode间隙
  - DDR can enter self-refresh during间隙
  - CPU cores can sleep (if no pending tasks)

RTOS低功耗策略:
  
  1. Tickless idle: 不使用固定tick, 动态调整tick间隔
  2. Deep sleep: 推理间隙进入深睡眠
  3. Clock gating: 禁用未使用外设的时钟
  4. RAM retention: 深睡眠时保持RAM内容
  
  Implementation:
    // 推理间隙功耗管理
    void manage_inference_power(void) {
        if (npu_idle && cpu_idle) {
            // Enter low-power mode
            enter_deep_sleep();
            
            // Wake on NPU interrupt or timer
            sleep_until(wake_source);
            
            // Restore state
            restore_from_deep_sleep();
        }
    }

5.2 动态功耗监控

Power Monitoring (RTOS Task):
  
  void power_monitor_task(void *params) {
      for (;;) {
          // Read power sensors
          float cpu_power = read_power_sensor(CPU);
          float npu_power = read_power_sensor(NPU);
          float ddr_power = read_power_sensor(DDR);
          float total_power = cpu_power + npu_power + ddr_power;
          
          // Update power model
          update_power_model(total_power);
          
          // Check thresholds
          if (total_power > POWER_LIMIT) {
              trigger_throttling();
          }
          
          if (battery_level < BATTERY_WARN) {
              notify_low_battery();
          }
          
          vTaskDelay(pdMS_TO_TICKS(10)); // Check every 10ms
      }
  }

6. 各平台的功耗-调度差异

6.1 MCU级

功耗特性:
  - Sleep: < 1μW (deep sleep)
  - Active: 50-200mW
  - DVFS: 有限 (2-4 states)
  - 热: 被动散热, R_thermal ~10°C/W

策略:
  - 大量使用sleep模式
  - 计算密集型任务集中执行, 然后sleep
  - DVFS简单, 调度变化小

6.2 SoC级

功耗特性:
  - Sleep: ~10mW (peripheral sleep)
  - Active: 300-1300mW
  - DVFS: 丰富 (8+ states)
  - 热: 被动散热, R_thermal ~5°C/W

策略:
  - 精细的DVFS控制
  - 多核独立频率
  - NPU专属低功耗模式
  - 热感知调度 (关键)

6.3 Edge盒子

功耗特性:
  - Sleep: ~50mW (standby)
  - Active: 500-2000mW
  - DVFS: 丰富 (10+ states)
  - 热: 主动+被动散热

策略:
  - 精细的热管理
  - CPU-NPU负载均衡
  - 功耗capping
  - 预测性调度

6.4 Server级

功耗特性:
  - Sleep: ~1W
  - Active: 100-500W
  - DVFS: 极丰富
  - 热: 主动散热 (fan + heatsink)

策略:
  - PUE优化
  - GPU NVLink功耗管理
  - NUMA功耗均衡
  - 数据中心级功耗管理

7. 关键设计决策

决策点	选项	推荐	理由
DVFS粒度	Global / Per-core / Per-device	Per-core	灵活性+实时性可接受
热管理	PID / Rule-based / ML	PID	确定性+可分析
空闲管理	Tickless / Deep-sleep / Clock-gate	混合	按场景选择
功耗监控	Hardware / Software	Hardware	精度+低开销
热感知	Reactive / Predictive	Predictive	减少延迟抖动
模式切换	Manual / Auto / Hybrid	Auto	自适应环境变化

8. 开放研究问题

1. Predictive Power: 基于输入预测功耗, 提前调整DVFS？
2. Thermal-aware Task Placement: 多核场景下的热均衡调度？
3. Battery-aware Scheduling: 电池电量变化时的调度策略自适应？
4. Cross-component Thermal: CPU+NPU+DDR的联合热建模？
5. Power-perf SLA: 同时满足性能和功耗SLA的调度？

---

最后更新: 2026-09-17