feat(asr): 本地语音转写接入为一级路由 + 并行工作流合并提交
按用户指示做**一包提交**,不按工作流拆分。本提交刻意混合了多条并行线:
· 本地 ASR 接管:audio 成为与 chat/embed/image/video 同等的路由类别
(IsLocalRoute 单一判据、audio 健康探测、default_audio_route、
auto 占位、GET /api/ai/routes/audio、回退云端时界面明示「音频已出网」)
· LLM 调用层:ctx 贯穿、ToolCall/ToolSchema、EmptyCompletionError /
TransientUpstreamError(按错误类型而非文案判重试)
· 编排 Agent:general_assistant orchestrate/persistence/spec_driver
· 联网搜索:internal/search(playwright)
· 网盘:backend + 前端
· 前端 UI:导航/路由/工作台若干页
· 交付文档:DELIVERY.md / AR04 / 部署文档的「无 Python」表述据实改写,
新增 eai_agentplatform-asr.service、asr.env、clonezilla-cleanup 清 ~/asr-poc
不分拆的原因:dev 早期,粒度不该打断工作节奏。且实测过——这些改动
**在编译上是同一个单元**(llm.go 的 ctx 签名变更牵动 12 个调用点,
chat_message.go 的 ctx 改动又与编排重写同处一个 hunk),拆出来的中间态编不过。
详见 TOP_CODING_RULES.md G14.5 与 bugs_and_errors.md E09。
Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
@@ -4,19 +4,64 @@
|
||||
"image_gen": "image_route_openrouter_gpt_image_2",
|
||||
"video_gen": "video_route_openrouter_veo_3",
|
||||
"path_coach": "chat_route_lmuai_deepseek_v4_flash",
|
||||
"title_gen": "chat_route_lmuai_deepseek_v4_flash"
|
||||
"title_gen": "chat_route_lmuai_deepseek_v4_flash",
|
||||
"audio_transcribe": "audio_route_auto",
|
||||
"audio_transcribe_llm": "chat_route_lmuai_deepseek_v4_flash_doc"
|
||||
},
|
||||
"audio_routes": {
|
||||
"audio_route_local_whisper": {
|
||||
"base_url": "http://127.0.0.1:8090/v1",
|
||||
"description": "语音转写 · 本机 faster-whisper large-v3 + pyannote 3.1(音频不出本机)。由 deploy/eai_agentplatform-asr.service 常驻在 127.0.0.1:8090;服务不在时 audio_route_auto 会改走 fallback_routes 里的云端路由,并在转写产物里标注「音频已出本机」。timeout 定在 840s 而不是一小时:前端 api/audioSkill.js 给这一步的上限是 15 分钟,后端必须在它之前自己收手(否则用户看到的是 axios 超时,而后端还在跑),同时 840s 里还留得下云端那 600s 的回退预算",
|
||||
"endpoint": "/audio/transcriptions",
|
||||
"model": "large-v3",
|
||||
"provider": "local_asr",
|
||||
"short_model_name": "whisper large-v3",
|
||||
"short_route_name": "本地",
|
||||
"supports_speakers": true,
|
||||
"timeout_seconds": 840
|
||||
},
|
||||
"audio_route_siliconflow_diarize": {
|
||||
"description": "语音转写 · SiliconFlow / XingChen ASR Diarize(云端,音频会出本机;本地服务不可用时的回退,也可由管理员手动指定)",
|
||||
"endpoint": "/audio/transcriptions",
|
||||
"model": "XingChenAGI/XingChenASR-Diarize-V3.0",
|
||||
"provider": "siliconflow",
|
||||
"short_model_name": "XingChen ASR Diarize",
|
||||
"short_route_name": "SiliconFlow",
|
||||
"supports_speakers": true,
|
||||
"timeout_seconds": 600
|
||||
},
|
||||
"audio_route_siliconflow_qwen3": {
|
||||
"description": "语音转写 · SiliconFlow / Qwen3-ASR(云端,音频会出公网;无说话人分离)",
|
||||
"endpoint": "/audio/transcriptions",
|
||||
"model": "Qwen/Qwen3-ASR-1.7B",
|
||||
"provider": "siliconflow",
|
||||
"short_model_name": "Qwen3 ASR",
|
||||
"short_route_name": "SiliconFlow",
|
||||
"supports_speakers": false,
|
||||
"timeout_seconds": 600
|
||||
}
|
||||
},
|
||||
"chat_routes": {
|
||||
"chat_route_lmuai_deepseek_v4_flash": {
|
||||
"description": "LMUAI / DeepSeek Flash",
|
||||
"description": "LMUAI / DeepSeek Flash。max_tokens 从 4096 提到 32768(2026-09-26):4096 是当初只按「聊天」估的,但这条路由是 default_route,用户在对话框里选过模型后这个选择会被记进 localStorage 并**一路带到技能内部的加工步骤**(全仓库只有语音转写的第 3、4 步消费 ai_route_id,见 skills/api/audio_handlers.go 的 resolveAudioChatRoute)。于是它会顶掉技能自带的 _doc 路由,让「整理段落」在推理模型思考完之前就把预算耗光、正文 0 字。上限是天花板不是配额,模型写完就停,抬高不会让聊天变慢变贵(实测见 _doc 那条的说明)",
|
||||
"endpoint": "/chat/completions",
|
||||
"max_tokens": 4096,
|
||||
"max_tokens": 32768,
|
||||
"model": "deepseek-v4-flash",
|
||||
"provider": "lmuai",
|
||||
"short_model_name": "DeepSeek V4 Flash",
|
||||
"short_route_name": "LMUAI",
|
||||
"temperature": 0.7
|
||||
},
|
||||
"chat_route_lmuai_deepseek_v4_flash_doc": {
|
||||
"description": "LMUAI / DeepSeek Flash(长文输出档,max_tokens=32768)· 供逐字稿整理这类要产出一整篇文档的技能使用。为什么不复用上面那条 4096 的:这是推理模型,思考与正文共用 max_tokens,而思考的长度跟输入几乎不成比例(实测 1200 字的输入,思考照样写到 7169 字)。4096 下正文一个字都写不出来(六次实测只有一次成功)。再说为什么是 32768 而不是 8192:纪要步骤最后要把各段结果**归并**成一份,那是全流程最长的一次输入(实测 8628 字),8192 下 finish_reason=length、正文 0 字,16384 才出稿、32768 又快又稳。上限提高**不会**让逐块调用变慢或变贵——上限是天花板不是配额,模型写完就停:实测同一批 11 段,8192 与 32768 两档的用时(18.6s / 19.2s)与 completion token(均 4928 / 5094)基本一致。调小这个值之前请先看 audio_transcribe/prompts.go 里的实测表",
|
||||
"endpoint": "/chat/completions",
|
||||
"max_tokens": 32768,
|
||||
"model": "deepseek-v4-flash",
|
||||
"provider": "lmuai",
|
||||
"short_model_name": "DeepSeek V4 Flash(长文档)",
|
||||
"short_route_name": "LMUAI",
|
||||
"temperature": 0.7
|
||||
},
|
||||
"chat_route_ollama_qwen": {
|
||||
"description": "本地对话 · llama.cpp / Qwen3.5 4B",
|
||||
"endpoint": "/chat/completions",
|
||||
@@ -89,6 +134,7 @@
|
||||
"temperature": 0.7
|
||||
}
|
||||
},
|
||||
"default_audio_route": "audio_route_local_whisper",
|
||||
"default_embed_route": "embed_route_llamacpp_nomic",
|
||||
"default_route": "chat_route_lmuai_deepseek_v4_flash",
|
||||
"description": "eai_agentplatform AI 路由配置 — 按场景分类定义多条路由,支持 provider/模型切换",
|
||||
@@ -141,6 +187,9 @@
|
||||
"chat_route_ollama_qwen_fast",
|
||||
"chat_route_ollama_qwen"
|
||||
],
|
||||
"audio_route_local_whisper": [
|
||||
"audio_route_siliconflow_diarize"
|
||||
],
|
||||
"embed_route_llamacpp_nomic": [
|
||||
"embed_route_openrouter_text_v3"
|
||||
],
|
||||
|
||||
Reference in New Issue
Block a user