From 81366ca2730baf6a3594709e0c1cd18382c7bb15 Mon Sep 17 00:00:00 2001 From: eaiadmin Date: Sun, 27 Sep 2026 00:04:19 +0800 Subject: [PATCH] =?UTF-8?q?fix(audio):=20=E6=96=AD=E8=BF=9E=E5=90=8E?= =?UTF-8?q?=E8=83=BD=E6=8E=A5=E7=9D=80=E8=B7=91=E5=AE=8C=20=E2=80=94?= =?UTF-8?q?=E2=80=94=20=E5=A4=B1=E8=B4=A5=E6=B6=88=E6=81=AF=E4=B8=8A?= =?UTF-8?q?=E7=9A=84=E9=87=8D=E8=AF=95=20+=20=E6=AD=A5=E9=AA=A4=E7=BC=96?= =?UTF-8?q?=E5=8F=B7=E7=BB=9F=E4=B8=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 用户报「卡在步骤 3」，实际断在第 5 步 structure：请求发出 46ms 后浏览器 连接断开，Gin 的 Request.Context() 被取消，上游回 499，后端包成 400。 服务端与前端应用层都没有任何主动取消（全仓 grep cancel/AbortController 零命中），是一次客户端断连。 断连本身无法在代码里杜绝，但它暴露了三个真 bug： 1. 报错文案说「第 1 步」，右栏说第 5 步 —— 尾部两步按自己这一轮从 1 数。 新增 STEP_NUMBERS，报错与右栏共用同一份编号（这就是「步骤 3」这个 说法的来源：用户看到的第一个数字就对不上）。 2. 断连后界面上没有任何续跑入口，任务永远停在 4/6（右栏六步是只读的）。 失败的那条助手消息上挂「🔁 重试」。 3. 无脑重跑会在同一任务里留第二份「结构化纪要」，两份差别用户看不出来。 resumeAudioSkill 收 skipSteps，按 task_run.action_key 判「跑过没有」 —— 不用产物类型判，document/checklist 别的技能也在用。 另修：待确认那条消息改用 reactive()。messages 是 ref([])，push 进去的对象 模板读时才被代理，而代码改的是那个变量，改普通对象不触发重渲染， 用户会一直看到「正在继续整理...」。 验证（临时任务 100，已删，先备份 db）：CDP setBlockedURLs 掐掉 structure 复现同形断连 → 消息挂出重试按钮，文案与右栏同为第 5 步；后端补跑一次 structure（模拟响应丢失但已落库）→ 点重试只跑 minutes，落库 document 1 份、 checklist 1 份。反证：再手工跑一次 structure，同名产物立刻变 2 份。 详见 bugs_and_errors.md E14。 E15（未修，记录在案）：DeleteMyTask 只删 task_record，run 与 artifact 全成 孤儿，产物 content_text 是完整逐字稿。删除语义务必先定硬删还是软删。 Co-Authored-By: Claude Code --- bugs_and_errors.md | 82 +++++ .../config/ai_config.json.bak-asmr-tests | 248 ++++++++++++++ .../config/ai_config.json.bak-b-degrade | 303 ++++++++++++++++++ .../frontend/src/skills/shared/audioSkill.js | 36 ++- .../views/workbench/SmartAssistantPage.vue | 125 +++++++- 5 files changed, 784 insertions(+), 10 deletions(-) create mode 100644 eai_agentplatform/backend-go/config/ai_config.json.bak-asmr-tests create mode 100644 eai_agentplatform/backend-go/config/ai_config.json.bak-b-degrade diff --git a/bugs_and_errors.md b/bugs_and_errors.md index 257b6d5..dca19eb 100644 --- a/bugs_and_errors.md +++ b/bugs_and_errors.md @@ -620,6 +620,85 @@ torch / matplotlib 之类的 dotfile 也落在同一个可写位置。 --- +## E14 「卡在第 3 步」——三步说的不是同一步 + +**日期**：2026-09-26　**区域**：语音转写技能 / 失败恢复 + +**症状**：用户报「看起来在步骤 3 卡住了，没有完成工作」。界面实况：右栏 +「语音转写 · 技能工作流 **4 / 6** 完成」，第 1–4 步全绿，第 5 步「整理段落与重点」 +与第 6 步「提炼可交付纪要」都是「未开始」；聊天区最后一条停在 +「已记下你确认的说话人身份，正在继续整理段落与生成纪要...」。 + +**根因**：**用户说的第 3 步、右栏的第 5 步、报错文案里的第 1 步，是三个不同的编号。** + +- 真正断掉的是 `audio-transcribe:structure`（右栏第 5 步）。请求发出后 **46ms** + 浏览器↔dev server 的连接断开，Gin 的 `c.Request.Context()` 随之取消，上游 LMUAI + 回 499 `context canceled`，后端包成 400。 +- **不是服务端取消**：全仓 grep `WithCancel|WithTimeout|cancel()` 在 `internal/ai`、 + `internal/skills/api`、`internal/middleware`、`audio_transcribe` 中**零命中**； + 前端全 `src/` grep `AbortController|CancelToken|signal` 同样零命中。 + 日志里也没有 401（排除跳登录页杀请求），全文只此 1 次 `context canceled`。 + 结论：一次客户端断连，不是模型/路由/代码的问题。 +- 但**失败文案把它叫「第 1 步」** —— 尾部两步按**自己这一轮**从 1 数， + 而右栏按整条链数。用户看到的第一个数字就对不上，于是有了「卡在步骤 3」这个说法。 +- 更麻烦的是：断连之后**界面上没有任何续跑入口**。右栏那六个步骤是只读的， + 没有 `@click`，任务永远停在 4/6。 + +**修法**（三处，都在前端）： + +1. `audioSkill.js` 导出 `STEP_NUMBERS`，报错文案与右栏共用同一份编号 —— 两处各写一遍 + 就是这次对不上号的成因。 +2. 失败的那条助手消息上挂 `🔁 重试`（`SmartAssistantPage.vue`）。复用 `speakerConfirmBusy` + 做并发闸，不新造一套。 +3. `resumeAudioSkill` 收 `skipSteps`，由页面按 `task_run.action_key` 算出「哪几步跑过了」。 + **判据必须是 run 而不是产物类型**：产物类型（`document` / `checklist`）别的技能也在用， + 拿它判断会把别人的产物误认成自己的。 +4. 待确认那条消息要用 `reactive()` 包。`messages` 是 `ref([])`，push 进去的对象在模板里 + 读时才被代理，而代码改的是**那个变量** —— 改普通对象只改了原始值，不触发重渲染， + 用户会一直看到「正在继续整理...」。 + +**验证**（临时任务 100，用完即删）： + +- 用 CDP `Network.setBlockedURLs` 掐掉 `*/api/skills/audio/structure`，复现与线上同形的 + 断连 → 消息变失败态并挂出重试按钮。截图 `/tmp/shot_retry_a_failed.png`：红字 + 「第 5 步「整理段落与重点」失败：Network Error」，与右栏那个 5 是同一个。 +- 脚本直接对后端补跑一次 `structure`（模拟「后端跑完了，只是响应没回来」）→ 点重试 + → 只跑 `minutes`，跑完 6/6（`/tmp/shot_retry_b_done.png`）。 +- 落库核对：`document` 1 份、`checklist` 1 份，没有重复。 +- **反证**（先证明断言会失败）：再手工 POST 一次 `structure`，同名「结构化纪要」立刻变成 + **2 份** —— 这就是 `skipSteps` 缺席时会发生的事。 + +**怎么早点发现**： + +> **同一步在界面上有两个编号，迟早有一天会被当成两步。** 序号这种东西只有一份来源， +> 谁要显示都从那里取。 +> +> 而**失败现场必须自带出路**。把「怎么继续」放在一个用户当前看不到、也点不动的地方 +> （右栏只读列表），等于没放 —— 用户会做的是刷新，而刷新之后连失败的那条消息都没了。 + +--- + +## E15 删除任务只删了 `task_record`，run 与 artifact 全成孤儿 + +**日期**：2026-09-26　**区域**：后端（`internal/api/my_task.go`） + +**症状**：清理验证夹具时走真实接口 `DELETE /api/my/tasks/100`，返回 200。 +删前 `task_record` 1 条、`task_run` 7 条、`task_artifact` 6 条；删后 +**0 / 7 / 6** —— 只有主记录没了，run 与产物原样留着。 + +**根因**：`DeleteMyTask`（`my_task.go:46`）只调 `taskRecordDAO.Delete(&task)`，没有级联。 + +**影响**：产物的 `content_text` 是**完整逐字稿**（真实会议录音，本次夹具那份 1550 字， +线上任务是 26278 字）。也就是说「删掉任务」并不等于「删掉录音内容」。 +`DELIVERY.md` 第 3 节手工清单 C 项「清空测试数据」若按这个删法走， +库里会留下一批**没有任何入口能看到、也没人能删**的孤儿逐字稿。 + +**状态**：**未修**。删除语义（硬删 / 软删留痕）是产品决定，不擅自改。 +本次是我自己造的夹具，已手工清掉那 7 条 run 与 6 条产物（先备份 +`/tmp/eai_backup_before_retry_test.db`，删后核对为 0 / 0）。 + +--- + ## 待沉淀（还没写进规则的） - [ ] **装完必须导入自检**：带 C 扩展的包（torch / torchaudio / ctranslate2）装完立刻 import 一次（见 E01）。 @@ -634,3 +713,6 @@ torch / matplotlib 之类的 dotfile 也落在同一个可写位置。 ProtectSystem / ReadOnlyPaths）、并发模型（前台独占 vs 后台多请求）。 凡是「原型上验过」的东西，装成服务后必须**照着这三条重验一遍**。 这次的 E13 是账号那条，E12 是并发那条。 +- [ ] **E14/E15 的通用形状**：**多步流程的「第 N 步」只能有一个编号来源**（E14）； + **有从属数据的聚合根，删除必须级联，或者明确是软删**（E15）。 + 两条都属「界面/接口说了一件事，数据说另一件事」，候选合并成 P06 的一条。 diff --git a/eai_agentplatform/backend-go/config/ai_config.json.bak-asmr-tests b/eai_agentplatform/backend-go/config/ai_config.json.bak-asmr-tests new file mode 100644 index 0000000..b4828a0 --- /dev/null +++ b/eai_agentplatform/backend-go/config/ai_config.json.bak-asmr-tests @@ -0,0 +1,248 @@ +{ + "agent_routes": { + "embed_gen": "embed_route_llamacpp_nomic", + "image_gen": "image_route_openrouter_gpt_image_2", + "video_gen": "video_route_openrouter_veo_3", + "path_coach": "chat_route_lmuai_deepseek_v4_flash", + "title_gen": "chat_route_lmuai_deepseek_v4_flash", + "audio_transcribe": "audio_route_auto", + "audio_transcribe_llm": "chat_route_lmuai_deepseek_v4_flash_doc" + }, + "audio_routes": { + "audio_route_local_whisper": { + "base_url": "http://127.0.0.1:8090/v1", + "description": "语音转写 · 本机 faster-whisper large-v3 + pyannote 3.1（音频不出本机）。由 deploy/eai_agentplatform-asr.service 常驻在 127.0.0.1:8090；服务不在时 audio_route_auto 会改走 fallback_routes 里的云端路由，并在转写产物里标注「音频已出本机」。timeout 定在 840s 而不是一小时：前端 api/audioSkill.js 给这一步的上限是 15 分钟，后端必须在它之前自己收手（否则用户看到的是 axios 超时，而后端还在跑），同时 840s 里还留得下云端那 600s 的回退预算", + "endpoint": "/audio/transcriptions", + "model": "large-v3", + "provider": "local_asr", + "short_model_name": "whisper large-v3", + "short_route_name": "本地", + "supports_speakers": true, + "timeout_seconds": 840 + }, + "audio_route_siliconflow_diarize": { + "description": "语音转写 · SiliconFlow / XingChen ASR Diarize（云端，音频会出本机；本地服务不可用时的回退，也可由管理员手动指定）", + "endpoint": "/audio/transcriptions", + "model": "XingChenAGI/XingChenASR-Diarize-V3.0", + "provider": "siliconflow", + "short_model_name": "XingChen ASR Diarize", + "short_route_name": "SiliconFlow", + "supports_speakers": true, + "timeout_seconds": 600 + }, + "audio_route_siliconflow_qwen3": { + "description": "语音转写 · SiliconFlow / Qwen3-ASR（云端，音频会出公网；无说话人分离）", + "endpoint": "/audio/transcriptions", + "model": "Qwen/Qwen3-ASR-1.7B", + "provider": "siliconflow", + "short_model_name": "Qwen3 ASR", + "short_route_name": "SiliconFlow", + "supports_speakers": false, + "timeout_seconds": 600 + } + }, + "chat_routes": { + "chat_route_lmuai_deepseek_v4_flash": { + "description": "LMUAI / DeepSeek Flash。max_tokens 从 4096 提到 32768（2026-09-26）：4096 是当初只按「聊天」估的，但这条路由是 default_route，用户在对话框里选过模型后这个选择会被记进 localStorage 并**一路带到技能内部的加工步骤**（全仓库只有语音转写的第 3、4 步消费 ai_route_id，见 skills/api/audio_handlers.go 的 resolveAudioChatRoute）。于是它会顶掉技能自带的 _doc 路由，让「整理段落」在推理模型思考完之前就把预算耗光、正文 0 字。上限是天花板不是配额，模型写完就停，抬高不会让聊天变慢变贵（实测见 _doc 那条的说明）", + "endpoint": "/chat/completions", + "max_tokens": 32768, + "model": "deepseek-v4-flash", + "provider": "lmuai", + "short_model_name": "DeepSeek V4 Flash", + "short_route_name": "LMUAI", + "temperature": 0.7 + }, + "chat_route_lmuai_deepseek_v4_flash_doc": { + "description": "LMUAI / DeepSeek Flash（长文输出档，max_tokens=32768）· 供逐字稿整理这类要产出一整篇文档的技能使用。为什么不复用上面那条 4096 的：这是推理模型，思考与正文共用 max_tokens，而思考的长度跟输入几乎不成比例（实测 1200 字的输入，思考照样写到 7169 字）。4096 下正文一个字都写不出来（六次实测只有一次成功）。再说为什么是 32768 而不是 8192：纪要步骤最后要把各段结果**归并**成一份，那是全流程最长的一次输入（实测 8628 字），8192 下 finish_reason=length、正文 0 字，16384 才出稿、32768 又快又稳。上限提高**不会**让逐块调用变慢或变贵——上限是天花板不是配额，模型写完就停：实测同一批 11 段，8192 与 32768 两档的用时（18.6s / 19.2s）与 completion token（均 4928 / 5094）基本一致。调小这个值之前请先看 audio_transcribe/prompts.go 里的实测表", + "endpoint": "/chat/completions", + "max_tokens": 32768, + "model": "deepseek-v4-flash", + "provider": "lmuai", + "short_model_name": "DeepSeek V4 Flash（长文档）", + "short_route_name": "LMUAI", + "temperature": 0.7 + }, + "chat_route_ollama_qwen": { + "description": "本地对话 · llama.cpp / Qwen3.5 4B", + "endpoint": "/chat/completions", + "max_tokens": 512, + "model": "Qwen3.5-4B-Q4_K_M.gguf", + "provider": "llamacpp", + "short_model_name": "Qwen3.5 4B", + "short_route_name": "本地对话", + "temperature": 0.7, + "timeout_seconds": 90 + }, + "chat_route_ollama_qwen_fast": { + "description": "本地快聊 · llama.cpp / Qwen3.5 4B", + "endpoint": "/chat/completions", + "max_tokens": 512, + "model": "Qwen3.5-4B-Q4_K_M.gguf", + "provider": "llamacpp", + "short_model_name": "Qwen3.5 4B", + "short_route_name": "本地快聊", + "temperature": 0.3 + }, + "chat_route_openrouter_deepseek_v3": { + "description": "OpenRouter / DeepSeek V3.2", + "endpoint": "/chat/completions", + "max_tokens": 4096, + "model": "deepseek/deepseek-v3.2", + "provider": "openrouter", + "short_model_name": "DeepSeek V3.2", + "short_route_name": "OpenRouter", + "temperature": 0.6 + }, + "chat_route_openrouter_deepseek_v4_flash": { + "description": "OpenRouter / DeepSeek V4 Flash 0731", + "endpoint": "/chat/completions", + "max_tokens": 4096, + "model": "deepseek/deepseek-v4-flash-0731", + "provider": "openrouter", + "short_model_name": "DeepSeek V4 Flash 0731", + "short_route_name": "OpenRouter", + "temperature": 0.7 + }, + "chat_route_openrouter_gpt_5": { + "description": "OpenRouter / GPT 5.4", + "endpoint": "/chat/completions", + "max_tokens": 8192, + "model": "openai/gpt-5.4", + "provider": "openrouter", + "short_model_name": "GPT 5.4", + "short_route_name": "OpenRouter", + "temperature": 0.7 + }, + "chat_route_openrouter_grok_4": { + "description": "OpenRouter / Grok 4.20", + "endpoint": "/chat/completions", + "max_tokens": 4096, + "model": "x-ai/grok-4.20", + "provider": "openrouter", + "short_model_name": "Grok 4.20", + "short_route_name": "OpenRouter", + "temperature": 0.6 + }, + "chat_route_siliconflow_deepseek_v4_flash": { + "description": "SiliconFlow / DeepSeek V4 Flash", + "endpoint": "/chat/completions", + "max_tokens": 4096, + "model": "deepseek-ai/DeepSeek-V4-Flash", + "provider": "siliconflow", + "short_model_name": "DeepSeek V4 Flash", + "short_route_name": "SiliconFlow", + "temperature": 0.7 + } + }, + "default_audio_route": "audio_route_local_whisper", + "default_embed_route": "embed_route_llamacpp_nomic", + "default_route": "chat_route_lmuai_deepseek_v4_flash", + "description": "eai_agentplatform AI 路由配置 — 按场景分类定义多条路由，支持 provider/模型切换", + "embed_routes": { + "embed_route_llamacpp_nomic": { + "base_url": "http://127.0.0.1:8081/v1", + "description": "本地向量 · llama.cpp / Nomic Embed v1.5", + "endpoint": "/embeddings", + "model": "nomic-embed-text-v1.5.Q4_K_M.gguf", + "provider": "llamacpp", + "short_model_name": "Nomic Embed v1.5", + "short_route_name": "本地向量", + "timeout_seconds": 30 + }, + "embed_route_ollama_bge_m3": { + "description": "本地向量 · Ollama / BGE-M3", + "endpoint": "/embeddings", + "model": "bge-m3", + "provider": "ollama", + "short_model_name": "BGE-M3", + "short_route_name": "本地向量" + }, + "embed_route_openrouter_text_v3": { + "description": "OpenRouter / text-embedding-3-small", + "endpoint": "/embeddings", + "model": "openai/text-embedding-3-small", + "provider": "openrouter", + "short_model_name": "text-embedding-3-small", + "short_route_name": "OpenRouter" + } + }, + "fallback_routes": { + "chat_route_lmuai_deepseek_flash": [ + "chat_route_ollama_qwen_fast", + "chat_route_ollama_qwen" + ], + "chat_route_openrouter_deepseek_v3": [ + "chat_route_openrouter_grok_4", + "chat_route_ollama_qwen" + ], + "chat_route_openrouter_deepseek_v4_flash": [ + "chat_route_openrouter_deepseek_v3", + "chat_route_openrouter_grok_4", + "chat_route_ollama_qwen" + ], + "chat_route_openrouter_grok_4": [ + "chat_route_ollama_qwen" + ], + "chat_route_siliconflow_deepseek_v4_flash": [ + "chat_route_ollama_qwen_fast", + "chat_route_ollama_qwen" + ], + "audio_route_local_whisper": [ + "audio_route_siliconflow_diarize" + ], + "embed_route_llamacpp_nomic": [ + "embed_route_openrouter_text_v3" + ], + "embed_route_ollama_bge_m3": [ + "embed_route_openrouter_text_v3" + ], + "embed_route_openrouter_text_v3": [ + "embed_route_ollama_bge_m3" + ] + }, + "image_routes": { + "image_route_ollama_llava": { + "description": "本地图像 · Ollama / LLaVA", + "endpoint": "/chat/completions", + "max_tokens": 2048, + "model": "llava", + "provider": "ollama", + "short_model_name": "LLaVA", + "short_route_name": "本地图像", + "temperature": 0.7 + }, + "image_route_openrouter_flux": { + "description": "OpenRouter / FLUX 1.1 Pro", + "endpoint": "/images/generations", + "max_tokens": 1024, + "model": "black-forest-labs/flux-1.1-pro", + "provider": "openrouter", + "short_model_name": "FLUX 1.1 Pro", + "short_route_name": "OpenRouter", + "temperature": 1 + }, + "image_route_openrouter_gpt_image_2": { + "description": "OpenRouter / GPT 5.4 Image 2", + "endpoint": "/images/generations", + "max_tokens": 4096, + "model": "openai/gpt-5.4-image-2", + "provider": "openrouter", + "short_model_name": "GPT 5.4 Image 2", + "short_route_name": "OpenRouter", + "temperature": 1 + } + }, + "video_routes": { + "video_route_openrouter_veo_3": { + "description": "OpenRouter / Google Veo 3", + "endpoint": "/videos/generations", + "model": "google/veo-3", + "max_tokens": 4096, + "provider": "openrouter", + "short_model_name": "Veo 3", + "short_route_name": "OpenRouter", + "temperature": 0.7 + } + }, + "version": "2.1.0" +} diff --git a/eai_agentplatform/backend-go/config/ai_config.json.bak-b-degrade b/eai_agentplatform/backend-go/config/ai_config.json.bak-b-degrade new file mode 100644 index 0000000..f172fab --- /dev/null +++ b/eai_agentplatform/backend-go/config/ai_config.json.bak-b-degrade @@ -0,0 +1,303 @@ +{ + "agent_routes": { + "embed_gen": "embed_route_llamacpp_nomic", + "image_gen": "image_route_openrouter_gpt_image_2", + "video_gen": "video_route_openrouter_veo_3", + "path_coach": "chat_route_lmuai_deepseek_v4_flash", + "title_gen": "chat_route_lmuai_deepseek_v4_flash", + "audio_transcribe": "audio_route_auto", + "audio_transcribe_llm": "chat_route_lmuai_deepseek_v4_flash_doc" + }, + "audio_routes": { + "audio_route_local_whisper": { + "base_url": "http://127.0.0.1:8090/v1", + "description": "语音转写 · 本机 faster-whisper large-v3 + pyannote 3.1（音频不出本机）。由 deploy/eai_agentplatform-asr.service 常驻在 127.0.0.1:8090；服务不在时 audio_route_auto 会改走 fallback_routes 里的云端路由，并在转写产物里标注「音频已出本机」。timeout 定在 840s 而不是一小时：前端 api/audioSkill.js 给这一步的上限是 15 分钟，后端必须在它之前自己收手（否则用户看到的是 axios 超时，而后端还在跑），同时 840s 里还留得下云端那 600s 的回退预算", + "endpoint": "/audio/transcriptions", + "model": "large-v3", + "provider": "local_asr", + "short_model_name": "whisper large-v3", + "short_route_name": "本地", + "supports_speakers": true, + "timeout_seconds": 840 + }, + "audio_route_siliconflow_diarize": { + "description": "语音转写 · SiliconFlow / XingChen ASR Diarize（云端，音频会出本机；本地服务不可用时的回退，也可由管理员手动指定）", + "endpoint": "/audio/transcriptions", + "model": "XingChenAGI/XingChenASR-Diarize-V3.0", + "provider": "siliconflow", + "short_model_name": "XingChen ASR Diarize", + "short_route_name": "SiliconFlow", + "supports_speakers": true, + "timeout_seconds": 600 + }, + "audio_route_siliconflow_qwen3": { + "description": "语音转写 · SiliconFlow / Qwen3-ASR（云端，音频会出公网；无说话人分离）", + "endpoint": "/audio/transcriptions", + "model": "Qwen/Qwen3-ASR-1.7B", + "provider": "siliconflow", + "short_model_name": "Qwen3 ASR", + "short_route_name": "SiliconFlow", + "supports_speakers": false, + "timeout_seconds": 600 + }, + "audio_route_openrouter_whisper": { + "description": "语音转写 · OpenRouter / OpenAI whisper-1（云端，音频会出公网；无说话人分离）。此路只作同能力（无分离）路由之间的备份，不能顶替默认的本地路由 —— 回退链按 supports_speakers 过滤，能力不同不回退", + "endpoint": "/audio/transcriptions", + "model": "openai/whisper-1", + "provider": "openrouter", + "base_url": "https://openrouter.ai/api/v1", + "short_model_name": "whisper-1", + "short_route_name": "OpenRouter", + "supports_speakers": false, + "timeout_seconds": 600 + }, + "audio_route_openrouter_gpt4o_transcribe": { + "description": "语音转写 · OpenRouter / OpenAI gpt-4o-transcribe（云端，音频会出公网；无说话人分离）。实测比 whisper-1 快一倍且断句更整，无分离路由里的首选", + "endpoint": "/audio/transcriptions", + "model": "openai/gpt-4o-transcribe", + "provider": "openrouter", + "base_url": "https://openrouter.ai/api/v1", + "short_model_name": "gpt-4o-transcribe", + "short_route_name": "OpenRouter", + "supports_speakers": false, + "timeout_seconds": 600 + }, + "audio_route_siliconflow_asr_ultra": { + "description": "语音转写 · SiliconFlow / XingChen ASR V3.2 Ultra（云端，音频会出公网；无说话人分离）。同门的 V3.2（非 Ultra）实测同样可用、约快一倍，但效果略差，没有单独建路由", + "endpoint": "/audio/transcriptions", + "model": "XingChenAGI/XingChenASR-V3.2-Ultra", + "provider": "siliconflow", + "short_model_name": "XingChen V3.2 Ultra", + "short_route_name": "SiliconFlow", + "supports_speakers": false, + "timeout_seconds": 600 + } + }, + "chat_routes": { + "chat_route_lmuai_deepseek_v4_flash": { + "description": "LMUAI / DeepSeek Flash。max_tokens 从 4096 提到 32768（2026-09-26）：4096 是当初只按「聊天」估的，但这条路由是 default_route，用户在对话框里选过模型后这个选择会被记进 localStorage 并**一路带到技能内部的加工步骤**（全仓库只有语音转写的第 3、4 步消费 ai_route_id，见 skills/api/audio_handlers.go 的 resolveAudioChatRoute）。于是它会顶掉技能自带的 _doc 路由，让「整理段落」在推理模型思考完之前就把预算耗光、正文 0 字。上限是天花板不是配额，模型写完就停，抬高不会让聊天变慢变贵（实测见 _doc 那条的说明）", + "endpoint": "/chat/completions", + "max_tokens": 32768, + "model": "deepseek-v4-flash", + "provider": "lmuai", + "short_model_name": "DeepSeek V4 Flash", + "short_route_name": "LMUAI", + "temperature": 0.7 + }, + "chat_route_lmuai_deepseek_v4_flash_doc": { + "description": "LMUAI / DeepSeek Flash（长文输出档，max_tokens=32768）· 供逐字稿整理这类要产出一整篇文档的技能使用。为什么不复用上面那条 4096 的：这是推理模型，思考与正文共用 max_tokens，而思考的长度跟输入几乎不成比例（实测 1200 字的输入，思考照样写到 7169 字）。4096 下正文一个字都写不出来（六次实测只有一次成功）。再说为什么是 32768 而不是 8192：纪要步骤最后要把各段结果**归并**成一份，那是全流程最长的一次输入（实测 8628 字），8192 下 finish_reason=length、正文 0 字，16384 才出稿、32768 又快又稳。上限提高**不会**让逐块调用变慢或变贵——上限是天花板不是配额，模型写完就停：实测同一批 11 段，8192 与 32768 两档的用时（18.6s / 19.2s）与 completion token（均 4928 / 5094）基本一致。调小这个值之前请先看 audio_transcribe/prompts.go 里的实测表", + "endpoint": "/chat/completions", + "max_tokens": 32768, + "model": "deepseek-v4-flash", + "provider": "lmuai", + "short_model_name": "DeepSeek V4 Flash（长文档）", + "short_route_name": "LMUAI", + "temperature": 0.7 + }, + "chat_route_ollama_qwen": { + "description": "本地对话 · llama.cpp / Qwen3.5 4B", + "endpoint": "/chat/completions", + "max_tokens": 512, + "model": "Qwen3.5-4B-Q4_K_M.gguf", + "provider": "llamacpp", + "short_model_name": "Qwen3.5 4B", + "short_route_name": "本地对话", + "temperature": 0.7, + "timeout_seconds": 90 + }, + "chat_route_ollama_qwen_fast": { + "description": "本地快聊 · llama.cpp / Qwen3.5 4B", + "endpoint": "/chat/completions", + "max_tokens": 512, + "model": "Qwen3.5-4B-Q4_K_M.gguf", + "provider": "llamacpp", + "short_model_name": "Qwen3.5 4B", + "short_route_name": "本地快聊", + "temperature": 0.3 + }, + "chat_route_openrouter_deepseek_v3": { + "description": "OpenRouter / DeepSeek V3.2", + "endpoint": "/chat/completions", + "max_tokens": 4096, + "model": "deepseek/deepseek-v3.2", + "provider": "openrouter", + "short_model_name": "DeepSeek V3.2", + "short_route_name": "OpenRouter", + "temperature": 0.6 + }, + "chat_route_openrouter_deepseek_v4_flash": { + "description": "OpenRouter / DeepSeek V4 Flash 0731", + "endpoint": "/chat/completions", + "max_tokens": 4096, + "model": "deepseek/deepseek-v4-flash-0731", + "provider": "openrouter", + "short_model_name": "DeepSeek V4 Flash 0731", + "short_route_name": "OpenRouter", + "temperature": 0.7 + }, + "chat_route_openrouter_gpt_5": { + "description": "OpenRouter / GPT 5.4", + "endpoint": "/chat/completions", + "max_tokens": 8192, + "model": "openai/gpt-5.4", + "provider": "openrouter", + "short_model_name": "GPT 5.4", + "short_route_name": "OpenRouter", + "temperature": 0.7 + }, + "chat_route_openrouter_grok_4": { + "description": "OpenRouter / Grok 4.20", + "endpoint": "/chat/completions", + "max_tokens": 4096, + "model": "x-ai/grok-4.20", + "provider": "openrouter", + "short_model_name": "Grok 4.20", + "short_route_name": "OpenRouter", + "temperature": 0.6 + }, + "chat_route_siliconflow_deepseek_v4_flash": { + "description": "SiliconFlow / DeepSeek V4 Flash", + "endpoint": "/chat/completions", + "max_tokens": 4096, + "model": "deepseek-ai/DeepSeek-V4-Flash", + "provider": "siliconflow", + "short_model_name": "DeepSeek V4 Flash", + "short_route_name": "SiliconFlow", + "temperature": 0.7 + } + }, + "default_audio_route": "audio_route_local_whisper", + "default_embed_route": "embed_route_llamacpp_nomic", + "default_route": "chat_route_lmuai_deepseek_v4_flash", + "description": "eai_agentplatform AI 路由配置 — 按场景分类定义多条路由，支持 provider/模型切换", + "embed_routes": { + "embed_route_llamacpp_nomic": { + "base_url": "http://127.0.0.1:8081/v1", + "description": "本地向量 · llama.cpp / Nomic Embed v1.5", + "endpoint": "/embeddings", + "model": "nomic-embed-text-v1.5.Q4_K_M.gguf", + "provider": "llamacpp", + "short_model_name": "Nomic Embed v1.5", + "short_route_name": "本地向量", + "timeout_seconds": 30 + }, + "embed_route_ollama_bge_m3": { + "description": "本地向量 · Ollama / BGE-M3", + "endpoint": "/embeddings", + "model": "bge-m3", + "provider": "ollama", + "short_model_name": "BGE-M3", + "short_route_name": "本地向量" + }, + "embed_route_openrouter_text_v3": { + "description": "OpenRouter / text-embedding-3-small", + "endpoint": "/embeddings", + "model": "openai/text-embedding-3-small", + "provider": "openrouter", + "short_model_name": "text-embedding-3-small", + "short_route_name": "OpenRouter" + } + }, + "fallback_routes": { + "chat_route_lmuai_deepseek_flash": [ + "chat_route_ollama_qwen_fast", + "chat_route_ollama_qwen" + ], + "chat_route_openrouter_deepseek_v3": [ + "chat_route_openrouter_grok_4", + "chat_route_ollama_qwen" + ], + "chat_route_openrouter_deepseek_v4_flash": [ + "chat_route_openrouter_deepseek_v3", + "chat_route_openrouter_grok_4", + "chat_route_ollama_qwen" + ], + "chat_route_openrouter_grok_4": [ + "chat_route_ollama_qwen" + ], + "chat_route_siliconflow_deepseek_v4_flash": [ + "chat_route_ollama_qwen_fast", + "chat_route_ollama_qwen" + ], + "audio_route_local_whisper": [ + "audio_route_siliconflow_diarize" + ], + "embed_route_llamacpp_nomic": [ + "embed_route_openrouter_text_v3" + ], + "embed_route_ollama_bge_m3": [ + "embed_route_openrouter_text_v3" + ], + "embed_route_openrouter_text_v3": [ + "embed_route_ollama_bge_m3" + ], + "audio_route_openrouter_gpt4o_transcribe": [ + "audio_route_siliconflow_asr_ultra", + "audio_route_openrouter_whisper", + "audio_route_siliconflow_qwen3" + ], + "audio_route_siliconflow_asr_ultra": [ + "audio_route_openrouter_gpt4o_transcribe", + "audio_route_openrouter_whisper", + "audio_route_siliconflow_qwen3" + ], + "audio_route_openrouter_whisper": [ + "audio_route_openrouter_gpt4o_transcribe", + "audio_route_siliconflow_asr_ultra", + "audio_route_siliconflow_qwen3" + ], + "audio_route_siliconflow_qwen3": [ + "audio_route_openrouter_gpt4o_transcribe", + "audio_route_siliconflow_asr_ultra", + "audio_route_openrouter_whisper" + ], + "audio_route_siliconflow_diarize": [ + "audio_route_local_whisper" + ] + }, + "image_routes": { + "image_route_ollama_llava": { + "description": "本地图像 · Ollama / LLaVA", + "endpoint": "/chat/completions", + "max_tokens": 2048, + "model": "llava", + "provider": "ollama", + "short_model_name": "LLaVA", + "short_route_name": "本地图像", + "temperature": 0.7 + }, + "image_route_openrouter_flux": { + "description": "OpenRouter / FLUX 1.1 Pro", + "endpoint": "/images/generations", + "max_tokens": 1024, + "model": "black-forest-labs/flux-1.1-pro", + "provider": "openrouter", + "short_model_name": "FLUX 1.1 Pro", + "short_route_name": "OpenRouter", + "temperature": 1 + }, + "image_route_openrouter_gpt_image_2": { + "description": "OpenRouter / GPT 5.4 Image 2", + "endpoint": "/images/generations", + "max_tokens": 4096, + "model": "openai/gpt-5.4-image-2", + "provider": "openrouter", + "short_model_name": "GPT 5.4 Image 2", + "short_route_name": "OpenRouter", + "temperature": 1 + } + }, + "video_routes": { + "video_route_openrouter_veo_3": { + "description": "OpenRouter / Google Veo 3", + "endpoint": "/videos/generations", + "model": "google/veo-3", + "max_tokens": 4096, + "provider": "openrouter", + "short_model_name": "Veo 3", + "short_route_name": "OpenRouter", + "temperature": 0.7 + } + }, + "version": "2.1.0" +} diff --git a/eai_agentplatform/frontend/src/skills/shared/audioSkill.js b/eai_agentplatform/frontend/src/skills/shared/audioSkill.js index 18e0c15..46cd9e7 100644 --- a/eai_agentplatform/frontend/src/skills/shared/audioSkill.js +++ b/eai_agentplatform/frontend/src/skills/shared/audioSkill.js @@ -45,8 +45,13 @@ export function skillConsumesAudio(skillKey) { */ const STEP_ORDER = ['scope', 'transcribe', 'speakers'] -/** 确认之后才跑的两步。中间隔着一次人工确认，所以不能和上面合成一个循环。 */ -const RESUME_STEP_ORDER = ['structure', 'minutes'] +/** + * 确认之后才跑的两步。中间隔着一次人工确认，所以不能和上面合成一个循环。 + * + * 导出是给页面用的：失败后「重试」要先算出哪几步已经有产物了（见 + * audioTailDoneSteps），跳过的判断得和这里的顺序是同一份。 + */ +export const RESUME_STEP_ORDER = ['structure', 'minutes'] const STEP_LABELS = { scope: '确认音频范围与语言', @@ -57,6 +62,22 @@ const STEP_LABELS = { minutes: '提炼可交付纪要', } +/** + * 每一步在右栏工作流里的序号。 + * + * 报错文案里说的「第 N 步」必须和右栏那个数字是同一个 —— 原先尾部两步按 + * **自己这一轮**从 1 数（「第 1 步『整理段落与重点』失败」），而右栏把它叫第 5 步， + * 于是用户看到「卡住了」时根本对不上是哪一步（2026-09-26 实际发生过一次）。 + */ +const STEP_NUMBERS = { + scope: 1, + transcribe: 2, + speakers: 3, + 'speakers-confirm': 4, + structure: 5, + minutes: 6, +} + /** 从已挂载的附件里挑出第一个音频（挂多个时只处理第一个，不做隐式合并）。 */ export function pickAudioResource(resources) { const list = Array.isArray(resources) ? resources : [] @@ -143,7 +164,7 @@ export async function executeAudioSkill(context = {}) { } catch (error) { const done = results.length const tail = done > 0 ? `（前 ${done} 步的产物已保存，可在右侧产物区查看）` : '' - throw new Error(`第 ${index + 1} 步「${STEP_LABELS[step]}」失败：${error?.message || '请稍后重试'}${tail}`) + throw new Error(`第 ${STEP_NUMBERS[step]} 步「${STEP_LABELS[step]}」失败：${error?.message || '请稍后重试'}${tail}`) } } @@ -165,11 +186,17 @@ export async function executeAudioSkill(context = {}) { * * 为什么确认这件事要在这里再做一次调用、而不是让后端一路跑到底：确认是**人的动作**， * 中间可能隔几分钟、隔一次刷新、甚至隔一天。后端不为此挂一个等待中的请求。 + * + * `skipSteps` 是给「重试」用的：请求可能**已经在后端跑完并落库了，只是响应没回来** + * （浏览器连接断掉、代理掉线 —— 2026-09-26 第 5 步就这么失败过一次）。那时无脑重跑 + * 会在同一个任务里留下第二份「结构化纪要」，而两份的差别用户看不出来， + * 只会疑惑产物区为什么多了一条。调用方按任务里已有的 run 算好传进来。 */ export async function resumeAudioSkill(context = {}) { const definition = context?.definition || {} const taskId = Number(context?.taskId || 0) const aiRouteId = String(context?.aiRouteId || '').trim() + const skipSteps = new Set(Array.isArray(context?.skipSteps) ? context.skipSteps : []) if (!taskId) { throw new Error('找不到当前任务，无法继续。请刷新后重试。') } @@ -177,6 +204,7 @@ export async function resumeAudioSkill(context = {}) { const results = [] for (let index = 0; index < RESUME_STEP_ORDER.length; index += 1) { const step = RESUME_STEP_ORDER[index] + if (skipSteps.has(step)) continue try { const res = await runAudioSkillStep(step, { task_id: taskId, ai_route_id: aiRouteId }) if (!res?.data) throw new Error('服务没有返回结果') @@ -185,7 +213,7 @@ export async function resumeAudioSkill(context = {}) { } catch (error) { const done = results.length const tail = done > 0 ? '（「整理段落与重点」的产物已保存，可在右侧产物区查看）' : '' - throw new Error(`第 ${index + 1} 步「${STEP_LABELS[step]}」失败：${error?.message || '请稍后重试'}${tail}`) + throw new Error(`第 ${STEP_NUMBERS[step]} 步「${STEP_LABELS[step]}」失败：${error?.message || '请稍后重试'}${tail}`) } } diff --git a/eai_agentplatform/frontend/src/views/workbench/SmartAssistantPage.vue b/eai_agentplatform/frontend/src/views/workbench/SmartAssistantPage.vue index 78c44be..baee1d3 100644 --- a/eai_agentplatform/frontend/src/views/workbench/SmartAssistantPage.vue +++ b/eai_agentplatform/frontend/src/views/workbench/SmartAssistantPage.vue @@ -149,6 +149,16 @@
+ +
{{ msg.timestamp }}
@@ -208,7 +218,7 @@