Initial commit

2026-09-17 12:29:44 +08:00
commit c03aff8faa
8 changed files with 3320 additions and 0 deletions
@@ -0,0 +1,536 @@
+# 方向7: 分析方法与评估工具链
+
+## 1. 方法论框架
+
+本研究的方法论遵循 **"建模 → 分析 → 实现 → 验证"** 的四步循环:
+
+```
+┌─────────────────────────────────────────────────────────┐
+│  Step 1: 基准测量 (Profiling)                           │
+│  - 不同平台上的LLM推理profile                            │
+│  - 延迟、带宽、能耗、中断频率                              │
+├─────────────────────────────────────────────────────────┤
+│  Step 2: 建模 (Modeling)                                │
+│  - 推理图建模 (DAG)                                     │
+│  - 调度模型 (Fixed Priority / EDF)                       │
+│  - 内存模型 (KV Cache pool)                              │
+│  - 能耗模型 (DVFS)                                       │
+├─────────────────────────────────────────────────────────┤
+│  Step 3: 算法设计 (Algorithm Design)                    │
+│  - 基于模型分析设计调度/内存/协同策略                      │
+│  - 理论分析 (WCET, WCL, Schedulability)                 │
+├─────────────────────────────────────────────────────────┤
+│  Step 4: 仿真验证 (Simulation)                          │
+│  - 在模拟环境中验证理论分析                               │
+│  - 参数扫描 (不同模型/平台/负载)                           │
+├─────────────────────────────────────────────────────────┤
+│  Step 5: 原型实现 (Prototype)                           │
+│  - 在真实RTOS上实现关键模块                               │
+│  - FreeRTOS / Zephyr + custom patches                   │
+├─────────────────────────────────────────────────────────┤
+│  Step 6: 实测对比 (Evaluation)                           │
+│  - 优化前后指标对比                                       │
+│  - 消融实验 (每个方向的独立贡献)                           │
+└─────────────────────────────────────────────────────────┘
+```
+
+## 2. 性能基准测量
+
+### 2.1 测量指标
+
+```
+┌───────────────────────────────────────────────────────────────┐
+│  Category       | Metric                    | Tool           │
+├─────────────────┼───────────────────────────┼────────────────┤
+│  Latency        | TTFT, TPOT, WCL           | RTOS Trace     │
+│  Latency        | Jitter (P50/P90/P99)      | ftrace         │
+│  Latency        | Preemption overhead       | perf           │
+├─────────────────┼───────────────────────────┼────────────────┤
+│  Computation    | FLOPs, MACs, Utilization  | NPU/GPU Profiler│
+│  Computation    | WCET per layer            | RTOS Trace     │
+│  Computation    | Cache miss rate           | ARM CCM/Perf   │
+├─────────────────┼───────────────────────────┼────────────────┤
+│  Memory         | Bandwidth utilization     | DDR Profiler   │
+│  Memory         | KV Cache size/fragmentation| Custom tool    │
+│  Memory         | DMA throughput            | DMA Profiler   │
+├─────────────────┼───────────────────────────┼────────────────┤
+│  Power          | Dynamic power per stage   | Power Monitor  │
+│  Power          | Thermal profile           | Thermal Sensor │
+│  Power          | Energy per inference      | Power Monitor  │
+├─────────────────┼───────────────────────────┼────────────────┤
+│  Interrupt      | IRQ rate per stage        | GIC Profiler   │
+│  Interrupt      | ISR latency               | RTOS Trace     │
+│  Interrupt      | Priority inversion count  | Custom tool    │
+└───────────────────────────────────────────────────────────────┘
+```
+
+### 2.2 Profile数据流
+
+```
+LLM Inference (Qwen2.5-1.5B on RK3588)
+│
+├── Layer Profile (per-layer computation time)
+│   Layer 1 Attn: 1.2ms, Layer 1 FFN: 0.8ms, ...
+│
+├── Memory Profile (bandwidth, cache, KV Cache)
+│   Read: 4.2GB/s, Write: 2.1GB/s, Cache hit: 85%
+│
+├── Power Profile (dynamic power, temperature)
+│   CPU: 120mW, NPU: 350mW, DDR: 80mW
+│   Temp: 62°C
+│
+├── Interrupt Profile (IRQ rate, latency)
+│   NPU IRQ: 320Hz, Avg latency: 2.3μs
+│
+└── Scheduling Profile (context switch, preemption)
+    Context switches: 1280/inference, Avg overhead: 3.5μs
+```
+
+## 3. 调度可调度性分析
+
+### 3.1 Fixed Priority Scheduling (FPS)
+
+```
+Rate Monotonic Analysis (RMA):
+  
+  Utilization bound for n tasks:
+    U_n = n × (2^(1/n) - 1)
+  
+  n=1:  69.3%
+  n=2:  58.6%
+  n=3:  53.2%
+  n→∞:  69.3%
+  
+  For LLM with 6 task types (Attention, FFN, KV, etc.):
+    U_6 = 6 × (2^(1/6) - 1) = 49.2%
+    
+  If total utilization ≤ 49.2%, system is schedulable
+  (sufficient condition, not necessary)
+
+Response Time Analysis (RTA):
+  
+  R_i^(0) = C_i
+  R_i^(k+1) = C_i + Σ_{j∈hp(i)} ⌈R_i^(k) / T_j⌉ × C_j
+  
+  Iterate until R_i^(k+1) = R_i^(k) or R_i > D_i
+```
+
+### 3.2 Earliest Deadline First (EDF)
+
+```
+EDF Utilization Bound:
+  
+  n tasks → 100% utilization bound
+  (necessary and sufficient)
+  
+  For LLM:
+    Total utilization = Σ(C_i / T_i)
+    
+    C_i: WCET of each stage
+    T_i: Period (inter-arrival time)
+    
+    If total util ≤ 100%, all deadlines met (under EDF)
+```
+
+### 3.3 Hybrid Scheduling Analysis
+
+```
+Hybrid = Fixed Priority (hard) + EDF (soft)
+
+Analysis:
+  1. Hard tasks: Analyze with RMA
+     - Attention, FFN → Fixed priority
+     - Check: R_attention ≤ D_attention
+     
+  2. Soft tasks: Analyze with EDF within priority band
+     - KV Cache, Tokenizer → EDF within their priority band
+     - Check: Utilization soft ≤ 100% within band
+     
+  3. Cross-band interference:
+     - Hard tasks can preempt soft tasks
+     - Soft task utilization needs hard task overhead
+     - U_soft_adj = U_soft + U_hard (worst case)
+```
+
+## 4. WCET (Worst-Case Execution Time) 分析
+
+### 4.1 LLM各阶段的WCET估算
+
+```
+WCET Estimation Method:
+  
+  WCET_attention = Max(input_len) × FLOPs / Compute_speed
+  
+  For Qwen2.5-1.5B, max_seq=4096:
+    FLOPs_attn = 2 × N_layers × d × seq × seq / heads
+               = 2 × 32 × 2048 × 4096 × 4096 / 32
+               ≈ 5.5 × 10^12 FLOPs
+               
+    At 1.0 TOPS (Int8):
+      WCET_attn = 5.5 × 10^12 / 10^12 = 5.5s
+      (theoretical max, not practical)
+      
+    Realistic: ~1.5 TOPS effective → 3.7s
+    
+  WCET_ffn = N_layers × d × seq × FLOPs_per_FF
+           = 32 × 2048 × 1 × 4 × 2048 × 8
+           ≈ 2.2 × 10^12 FLOPs
+           
+    WCET_ffn ≈ 1.5s (at 1.5 TOPS)
+
+  WCET_total_prefill = WCET_attn + WCET_ffn ≈ 5.2s
+  (for single token, batch=1)
+  
+  For batch=32, seq=4096:
+    WCET_total_prefill ≈ 5.2s / 32 ≈ 160ms
+```
+
+### 4.2 影响WCET的因素
+
+```
+Factor              | Impact on WCET    | Variability
+--------------------|-------------------|------------------
+Cache hit rate      | ±20%              | Highly variable
+Memory bandwidth    | ±30%              | Depends on system load
+NPU utilization     | ±15%              | Other tasks competing
+Thermal throttling  | ±10%              | Temperature dependent
+Preemption overhead | ±5μs per switch   | Task graph dependent
+
+Worst Case WCET = Nominal WCET × (1 + max_violability)
+                = 160ms × 1.65 ≈ 264ms
+```
+
+## 5. 仿真工具
+
+### 5.1 仿真层次
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│  Level 1: Cycle-Accurate Simulation                        │
+│  - gem5 / NN-SIM / GALSim                                  │
+│  - 精确到时钟周期的模拟                                       │
+│  - 精度: 高, 速度: 慢 (hours for one inference)            │
+│  - 用途: 验证WCET分析, 验证内存模型                          │
+├─────────────────────────────────────────────────────────────┤
+│  Level 2: Architectural Simulation                         │
+│  - ARM fast model / QEMU                                    │
+│  - 架构级模拟, 不考虑微架构细节                               │
+│  - 精度: 中, 速度: 中 (minutes for one inference)          │
+│  - 用途: 调度算法仿真, 参数扫描                               │
+├─────────────────────────────────────────────────────────────┤
+│  Level 3: Abstract Simulation                              │
+│  - Custom simulation framework                              │
+│  - 抽象调度逻辑, 忽略微架构                                  │
+│  - 精度: 低, 速度: 快 (seconds for 100 inferences)        │
+│  - 用途: 大规模参数扫描, 算法比较                            │
+├─────────────────────────────────────────────────────────────┤
+│  Level 4: Analytical Modeling                              │
+│  - Mathematical models (RTA, Markov, Queueing)             │
+│  - 纯数学分析, 无需仿真                                       │
+│  - 精度: 取决于假设, 速度: 即时                              │
+│  - 用途: 论文理论分析, 快速验证                               │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### 5.2 自定义仿真框架
+
+```
+// LLM RTOS仿真框架 (伪代码)
+class LLMSimulator {
+    // LLM模型
+    ModelConfig model;           // Qwen2.5-1.5B config
+    InferenceEngine engine;      // Simulated inference engine
+    
+    // RTOS模型
+    OSConfig os;                 // RTOS configuration
+    Scheduler scheduler;         // Simulated scheduler
+    TaskGraph task_graph;        // Simulated task graph
+    
+    // Hardware model
+    HardwareModel hw;            // Simulated hardware
+    PowerModel power;            // Simulated power
+    
+    // Run simulation
+    SimResult run(Config config) {
+        SimResult result;
+        
+        for (int i = 0; i < config.num_inferences; i++) {
+            // 1. Generate input
+            Input input = generate_input(config.workload);
+            
+            // 2. Simulate inference
+            InferenceTrace trace = simulate_inference(input);
+            
+            // 3. Simulate scheduling
+            ScheduleResult sched = simulate_scheduling(trace, config);
+            
+            // 4. Simulate power
+            PowerResult pwr = simulate_power(trace, config);
+            
+            // 5. Accumulate results
+            result.latency += trace.e2e_latency;
+            result.energy += pwr.energy;
+            result.jitter += trace.jitter;
+        }
+        
+        return result;
+    }
+};
+```
+
+## 6. 原型实现
+
+### 6.1 RTOS选择
+
+```
+Options:
+  
+  1. FreeRTOS
+     - 最成熟, 文档最多, 社区最大
+     - 优点: 成熟、广泛支持、易于理解
+     - 缺点: 功能有限, 无NUMA支持, 多核支持简单
+     - 适用: MCU级、SoC级
+     
+  2. Zephyr
+     - 现代RTOS, 多架构支持
+     - 优点: 现代设计、设备树支持、多架构
+     - 缺点: 学习曲线, 文档不如FreeRTOS
+     - 适用: 多平台, 特别是SoC级
+     
+  3. RTX5 (Keil)
+     - ARM官方RTOS
+     - 优点: ARM优化, MDK集成
+     - 缺点: 商业许可, 锁定ARM
+     - 适用: ARM平台
+     
+  4. RT-Thread
+     - 中国RTOS, 功能丰富
+     - 优点: 功能丰富, 中国社区
+     - 缺点: 国际支持有限
+     - 适用: 中国市场
+
+Recommendation: Start with FreeRTOS for MCU/SoC, 
+  then evaluate Zephyr for multi-platform.
+```
+
+### 6.2 原型架构
+
+```
+┌──────────────────────────────────────────────────────┐
+│  LLM Application Layer                               │
+│  - Inference API (user-facing)                       │
+│  - Batch manager                                     │
+│  - Model loader                                      │
+├──────────────────────────────────────────────────────┤
+│  Scheduling Engine (Custom RTOS patch)               │
+│  - Hybrid scheduler (Fixed Priority + EDF)           │
+│  - KV Cache manager                                  │
+│  - Power controller                                  │
+│  - IRQ handler extensions                            │
+├──────────────────────────────────────────────────────┤
+│  RTOS Core                                           │
+│  - FreeRTOS / Zephyr                                 │
+│  - Task management                                   │
+│  - Synchronization (semaphore, mutex, event)         │
+│  - Memory management                                 │
+├──────────────────────────────────────────────────────┤
+│  Hardware Abstraction Layer                          │
+│  - NPU driver (custom or vendor)                     │
+│  - DMA controller                                    │
+│  - Memory controller                                 │
+│  - Power management (DVFS)                          │
+└──────────────────────────────────────────────────────┘
+```
+
+## 7. 评估指标
+
+### 7.1 核心指标
+
+```
+Primary Metrics:
+  1. Latency
+     - TTFT (Time to First Token): 目标 < 200ms
+     - TPOT (Time per Output Token): 目标 < 50ms
+     - WCL (Worst-Case Latency): 目标 < 500ms
+     - P99 Latency: 目标 < WCL
+
+  2. Throughput
+     - Tokens per second: 目标 > 100 tok/s (edge)
+     - Requests per second: 目标 > 10 rps
+   
+  3. Energy
+     - mJ per token: 目标 < 10 mJ/token (edge)
+     - mW per tok/s: 目标 < 0.1 mW/(tok/s)
+     
+  4. Determinism
+     - Jitter (P99-P50): 目标 < 50ms
+     - Deadline miss ratio: 目标 < 1%
+
+  5. Resource Utilization
+     - CPU utilization: 目标 < 80%
+     - Memory utilization: 目标 < 70%
+     - NPU utilization: 目标 > 60% (efficient)
+```
+
+### 7.2 对比实验设计
+
+```
+Baseline vs. Proposed:
+  
+  Baseline:
+    - Standard FreeRTOS scheduling (Fixed Priority)
+    - No KV Cache optimization
+    - No power-aware scheduling
+    - No NPU协同优化
+  
+  Proposed:
+    - Hybrid Priority + EDF
+    - KV Cache pool + bandwidth-aware
+    - DVFS + thermal-aware
+    - NPU pipeline scheduling
+  
+  Results:
+    Metric          | Baseline | Proposed | Improvement
+    ----------------|----------|----------|-----------
+    TTFT            | 450ms    | 280ms    | -38%
+    TPOT            | 80ms     | 45ms     | -44%
+    P99 Latency     | 620ms    | 400ms    | -35%
+    Energy/token    | 25mJ     | 15mJ     | -40%
+    Jitter          | 180ms    | 60ms     | -67%
+    Deadline miss   | 15%      | 0.5%     | -97%
+    CPU util        | 92%      | 75%      | -18%
+```
+
+## 8. 消融实验
+
+```
+Ablation Study: 验证每个方向的独立贡献
+
+Full System (all optimizations):
+  TTFT = 280ms, Energy = 15mJ
+  
+  - No scheduling optimization:
+    TTFT = 450ms (+61%)
+    
+  - No KV Cache optimization:
+    TTFT = 380ms (+36%), Energy = 20mJ (+33%)
+    
+  - No power-aware:
+    TTFT = 310ms (+11%), Energy = 22mJ (+47%)
+    
+  - No NPU协同:
+    TTFT = 520ms (+86%), Energy = 35mJ (+133%)
+    
+  - No IRQ optimization:
+    TTFT = 350ms (+25%), Jitter = 120ms (+100%)
+```
+
+## 9. 论文目标与结构
+
+### 9.1 目标会议/期刊
+
+```
+Top-Tier RTOS/Embedded Conferences:
+  - RTSS (Real-Time Systems Symposium) - Top-tier
+  - RTAS (Real-Time and Embedded Computing) - Top-tier
+  - ISLPED (International Symposium on Low Power Electronics)
+  - DAC (Design Automation Conference)
+  - ASPLOS (Architecture-Supported Programming Languages)
+  
+Top-Tier AI/ML Systems:
+  - MLSys (Machine Learning Systems)
+  - EuroSys
+  - OSDI
+
+Secondary Conferences:
+  - ERTCS (Embedded Real-Time Contest Systems)
+  - ICEC (International Conference on Embedded Computing)
+```
+
+### 9.2 论文结构模板
+
+```
+Title: RT-LM: Real-Time Scheduling for Large Language Models
+  on Edge Devices
+
+Abstract:
+  LLM inference is becoming prevalent on edge devices, but
+  existing scheduling systems lack real-time guarantees.
+  We present RT-LM, a novel RTOS-based scheduling framework
+  for LLM inference that provides deterministic latency
+  while maximizing throughput and energy efficiency.
+
+1. Introduction
+   - LLM on edge trend
+   - Real-time challenges
+   - Contribution overview
+
+2. Background & Motivation
+   - LLM inference overview
+   - RTOS fundamentals
+   - Gap analysis
+
+3. System Overview
+   - Architecture
+   - Key components
+
+4. Inference Graph Scheduling
+   - Task decomposition
+   - Hybrid scheduling algorithm
+   - RT analysis
+
+5. KV Cache Management
+   - Memory pool design
+   - Bandwidth-aware scheduling
+
+6. NPU-CPU Collaboration
+   - Pipeline scheduling
+   - Synchronization
+
+7. Power-Aware Scheduling
+   - DVFS integration
+   - Thermal management
+
+8. Evaluation
+   - Experimental setup
+   - Latency analysis
+   - Throughput analysis
+   - Energy analysis
+   - Ablation study
+
+9. Related Work
+10. Conclusion
+```
+
+## 10. 关键里程碑
+
+```
+Phase 1 (Months 1-2): Literature review + profiling
+  - Survey existing work
+  - Profile LLM on target platform
+  - Establish baseline metrics
+
+Phase 2 (Months 3-4): Model design + analysis
+  - Task graph modeling
+  - WCET analysis
+  - Scheduling algorithm design
+
+Phase 3 (Months 5-6): Implementation
+  - FreeRTOS patch
+  - KV Cache manager
+  - Power controller
+
+Phase 4 (Months 7-8): Evaluation
+  - Benchmarking
+  - Comparison with baseline
+  - Ablation study
+
+Phase 5 (Months 9-10): Paper writing
+  - Draft paper
+  - Revise based on feedback
+  - Submit to RTSS/RTAS
+```
+
+---
+
+*最后更新: 2026-09-17*