init: 数字员工平台初始代码
包含前端(Vue3 + VueFlow 画布)、后端(Go)、文档体系。 - 工作台画布:节点拖放、连线模式、右键菜单、AI 助手 - 后端:连接器 API、专员种子数据 - 导航:左侧导航、工坊、市场、控制台
This commit is contained in:
@@ -0,0 +1,155 @@
|
||||
# BE01 — 认证模块设计
|
||||
|
||||
> **版本:V1.1 | 技术:JWT(python-jose)+ bcrypt | 参考:pj006-zhilianyuan2 auth/security.py + dependencies.py**
|
||||
|
||||
---
|
||||
|
||||
## 1. 模块职责
|
||||
|
||||
- 用户注册(仅管理员可创建账号)
|
||||
- 密码登录 → JWT 签发
|
||||
- Token 校验 + 用户状态检查
|
||||
- 角色守卫(普通用户 / 管理员)
|
||||
|
||||
## 2. 核心流程
|
||||
|
||||
```
|
||||
POST /api/auth/login
|
||||
→ 校验 username + password
|
||||
→ bcrypt verify
|
||||
→ 签发 JWT(含 sub=username, role, exp)
|
||||
→ 返回 { token, expires_in, user }
|
||||
|
||||
GET /api/auth/me
|
||||
→ Authorization: Bearer <token>
|
||||
→ 解码 JWT → 校验用户状态
|
||||
→ 返回用户信息
|
||||
```
|
||||
|
||||
## 3. 密码哈希(参考 zhilianyuan2 模式)
|
||||
|
||||
```python
|
||||
# core/security.py
|
||||
from passlib.context import CryptContext
|
||||
|
||||
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
return pwd_context.hash(password)
|
||||
|
||||
def verify_password(password: str, hashed: str) -> bool:
|
||||
return pwd_context.verify(password, hashed)
|
||||
```
|
||||
|
||||
## 4. JWT 签发与校验
|
||||
|
||||
```python
|
||||
# core/security.py
|
||||
from jose import jwt, JWTError
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
def create_access_token(
|
||||
*, subject: str, role: str, settings: Settings
|
||||
) -> str:
|
||||
issued_at = datetime.now(timezone.utc)
|
||||
payload = {
|
||||
"sub": subject,
|
||||
"role": role,
|
||||
"iat": issued_at,
|
||||
"exp": issued_at + timedelta(minutes=settings.jwt_expire_minutes),
|
||||
}
|
||||
return jwt.encode(payload, settings.jwt_secret, algorithm="HS256")
|
||||
|
||||
def decode_access_token(token: str, settings: Settings) -> dict:
|
||||
try:
|
||||
return jwt.decode(token, settings.jwt_secret, algorithms=["HS256"])
|
||||
except JWTError as e:
|
||||
raise AuthError(f"令牌无效或已过期:{e}")
|
||||
```
|
||||
|
||||
## 5. 依赖注入(参考 zhilianyuan2 dependencies.py)
|
||||
|
||||
```python
|
||||
# core/deps.py
|
||||
from fastapi import Depends
|
||||
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||
|
||||
async def get_current_user(
|
||||
credentials: HTTPAuthorizationCredentials | None = Depends(
|
||||
HTTPBearer(auto_error=False)
|
||||
),
|
||||
db: Session = Depends(get_db),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> User:
|
||||
"""解析 JWT + 校验用户状态"""
|
||||
if credentials is None:
|
||||
raise AuthError("缺少 Authorization Bearer 令牌")
|
||||
payload = decode_access_token(credentials.credentials, settings)
|
||||
username = payload.get("sub")
|
||||
user = db.query(User).filter(User.username == username).first()
|
||||
if user is None:
|
||||
raise AuthError("用户不存在")
|
||||
if user.status != "active":
|
||||
raise AuthError("账号已禁用")
|
||||
return user
|
||||
|
||||
def require_admin(current_user: User = Depends(get_current_user)) -> User:
|
||||
"""管理员角色守卫"""
|
||||
if current_user.role != "admin":
|
||||
raise ForbiddenError("需要管理员权限")
|
||||
return current_user
|
||||
```
|
||||
|
||||
## 6. API 路由
|
||||
|
||||
```python
|
||||
# api/auth.py
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
|
||||
router = APIRouter(prefix="/api/auth", tags=["auth"])
|
||||
|
||||
class LoginRequest(BaseModel):
|
||||
username: str
|
||||
password: str
|
||||
|
||||
class TokenResponse(BaseModel):
|
||||
token: str
|
||||
expires_in: int
|
||||
user: UserPublic
|
||||
|
||||
@router.post("/login")
|
||||
def login(request: LoginRequest, settings: SettingsDep):
|
||||
user = authenticate(request.username, request.password)
|
||||
token = create_access_token(subject=user.username, role=user.role, settings=settings)
|
||||
return {"data": {
|
||||
"token": token,
|
||||
"expires_in": settings.jwt_expire_minutes * 60,
|
||||
"user": UserPublic.from_orm(user),
|
||||
}}
|
||||
|
||||
@router.get("/me")
|
||||
def me(current_user: CurrentUser):
|
||||
return {"data": UserPublic.from_orm(current_user)}
|
||||
```
|
||||
|
||||
## 7. 数据表
|
||||
|
||||
```sql
|
||||
CREATE TABLE user (
|
||||
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
|
||||
username VARCHAR(64) NOT NULL UNIQUE,
|
||||
password_hash VARCHAR(256) NOT NULL,
|
||||
full_name VARCHAR(64) NOT NULL,
|
||||
role ENUM('employee','admin') NOT NULL DEFAULT 'employee',
|
||||
status ENUM('active','disabled') NOT NULL DEFAULT 'active',
|
||||
created_at DATETIME(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
|
||||
updated_at DATETIME(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6)
|
||||
);
|
||||
```
|
||||
|
||||
## 8. 安全约束
|
||||
|
||||
- 禁用账号即时失效:token 校验时检查 status
|
||||
- 前端路由守卫仅作 UX 隐藏,不以之为安全边界
|
||||
- 所有权限以后端鉴权为准
|
||||
@@ -0,0 +1,136 @@
|
||||
# BE02 — 考试模块设计
|
||||
|
||||
> **版本:V1.1 | 参考:pj006-zhilianyuan2 exam/routes.py + exam/service.py**
|
||||
|
||||
---
|
||||
|
||||
## 1. 模块职责
|
||||
|
||||
- 题库管理(管理员 CRUD)
|
||||
- 考试配置/组卷(管理员设置)
|
||||
- 学员端考试(开始/答题/交卷/判分)
|
||||
- 考试记录与回溯
|
||||
|
||||
## 2. 考试流程
|
||||
|
||||
```
|
||||
管理员端:
|
||||
录入题目 → 配置考试(名称/类型/题量/总分/合格线/时长/随机)
|
||||
|
||||
员工端:
|
||||
考试列表 → 查看封面/说明 → 开始考试 → 答题 → 交卷
|
||||
↓ ↓
|
||||
自测:即时显示对错+答案 正式考:存档(得分/明细/是否通过)
|
||||
```
|
||||
|
||||
## 3. 数据表关系
|
||||
|
||||
```
|
||||
question(题库)← exam_paper(考试配置,通过 domain+question_count 抽题)
|
||||
↓
|
||||
exam_record(每次交卷的记录)
|
||||
```
|
||||
|
||||
## 4. 题库 API
|
||||
|
||||
```python
|
||||
# 管理员 — 题目 CRUD
|
||||
GET /api/exam/questions?domain=company&status=active # 题目列表
|
||||
POST /api/exam/questions # 新增题目
|
||||
PUT /api/exam/questions/{id} # 编辑题目
|
||||
DELETE /api/exam/questions/{id} # 停用题目
|
||||
|
||||
# 管理员 — 考试配置
|
||||
GET /api/exam/papers # 考试配置列表
|
||||
POST /api/exam/papers # 创建考试
|
||||
PUT /api/exam/papers/{id} # 编辑考试
|
||||
DELETE /api/exam/papers/{id} # 停用考试
|
||||
```
|
||||
|
||||
## 5. 学员端考试 API
|
||||
|
||||
```python
|
||||
GET /api/exam/list # 我的考试列表
|
||||
GET /api/exam/cover?id={paperId} # 考试封面/说明
|
||||
POST /api/exam/start # 开始考试 → 下发题目
|
||||
POST /api/exam/submit # 交卷判分
|
||||
GET /api/exam/record # 我的考试记录
|
||||
GET /api/exam/record/{recordId} # 考试记录详情
|
||||
```
|
||||
|
||||
## 6. 判分逻辑(参考 zhilianyuan2 _is_correct)
|
||||
|
||||
```python
|
||||
# services/exam_service.py
|
||||
|
||||
def _is_correct(qtype: str, correct: list, user: any) -> bool:
|
||||
"""确定性判分"""
|
||||
if qtype == "multiple": # 多选题:集合相等
|
||||
return sorted(correct) == sorted(user) if user else False
|
||||
elif qtype == "judge": # 判断题:值相等
|
||||
return str(correct).lower() == str(user).lower()
|
||||
else: # 单选题:值相等
|
||||
return correct == user
|
||||
|
||||
def grade_paper(questions: list, answers: dict) -> dict:
|
||||
"""批卷:逐题比对 → 统计得分/正确数"""
|
||||
correct_count = 0
|
||||
total = len(questions)
|
||||
score_per_question = 100 / total if total else 0
|
||||
details = []
|
||||
|
||||
for q in questions:
|
||||
user_ans = answers.get(str(q["id"]))
|
||||
is_correct = _is_correct(q["type"], q["answer"], user_ans)
|
||||
if is_correct:
|
||||
correct_count += 1
|
||||
details.append({
|
||||
"question_id": q["id"],
|
||||
"is_correct": is_correct,
|
||||
"user_answer": user_ans,
|
||||
"correct_answer": q["answer"],
|
||||
})
|
||||
|
||||
score = round(score_per_question * correct_count)
|
||||
return {
|
||||
"score": score,
|
||||
"correct_count": correct_count,
|
||||
"wrong_count": total - correct_count,
|
||||
"passed": score >= paper.pass_score,
|
||||
"details": details,
|
||||
}
|
||||
```
|
||||
|
||||
## 7. 自测 vs 正式考区别
|
||||
|
||||
| 维度 | 自测 (self_test) | 正式考 (formal) |
|
||||
|------|-----------------|----------------|
|
||||
| 次数限制 | 不限 | 按配置(通常 1 次) |
|
||||
| 即时反馈 | 每题显示对错+答案 | 交卷后显示成绩 |
|
||||
| 成绩存档 | 不存 | 永久保存到 exam_record |
|
||||
| 答题明细 | 不存 | JSON 持久化 |
|
||||
|
||||
## 8. 考试记录设计
|
||||
|
||||
```json
|
||||
// exam_record.detail_json 示例
|
||||
{
|
||||
"questions": [
|
||||
{
|
||||
"question_id": 1,
|
||||
"stem": "博昇的主营业务包括?",
|
||||
"type": "single",
|
||||
"user_answer": "D",
|
||||
"correct_answer": "D",
|
||||
"is_correct": true,
|
||||
"explanation": "博昇双主营业务为资本咨询与AI产业落地"
|
||||
}
|
||||
],
|
||||
"time_spent_sec": 1200
|
||||
}
|
||||
```
|
||||
|
||||
## 9. 权限
|
||||
|
||||
- **员工:** 仅查看自己的考试记录
|
||||
- **管理员:** 查看全部考试记录(`/api/system/exam-records`)
|
||||
@@ -0,0 +1,233 @@
|
||||
# BE03 — 素材模块设计
|
||||
|
||||
> **版本:V1.1 | 技术:分片上传 + LibreOffice + PyMuPDF**
|
||||
|
||||
---
|
||||
|
||||
## 1. 模块职责
|
||||
|
||||
- 文件上传(直传 + 分片上传)
|
||||
- 素材审批流(待审批 → 通过/驳回)
|
||||
- 审批通过后异步文档转换管线
|
||||
- 文件预览
|
||||
- 转换状态查询
|
||||
|
||||
## 2. 素材状态流转
|
||||
|
||||
```
|
||||
员工提交 / 管理员上传
|
||||
│
|
||||
▼
|
||||
pending(待审批) ──┬─ approve → approved(已通过)
|
||||
│ │
|
||||
│ └─→ 触发异步转换管线
|
||||
│ │
|
||||
│ ├─ 文档 → LibreOffice 转 PDF → PyMuPDF 提取
|
||||
│ │ 文本 → 切片写入 knowledge_chunk
|
||||
│ └─ 视频/图片 → 仅标记预览可用
|
||||
│
|
||||
└─ reject → rejected(已驳回,前台不可见)
|
||||
```
|
||||
|
||||
## 3. 上传 API
|
||||
|
||||
### 直传(文档 ≤ 200MB)
|
||||
|
||||
```python
|
||||
POST /api/media/upload
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
Parameters:
|
||||
- file: 文件二进制
|
||||
- bind_type: company | product | course | none
|
||||
- bind_id: 绑定实体 ID(可选)
|
||||
|
||||
Response:
|
||||
{
|
||||
"media_id": 1,
|
||||
"status": "pending",
|
||||
"filename": "原始名称.pptx"
|
||||
}
|
||||
```
|
||||
|
||||
### 分片上传(视频 > 100MB)
|
||||
|
||||
```python
|
||||
# 1. 初始化
|
||||
POST /api/media/upload-init
|
||||
{
|
||||
"filename": "training.mp4",
|
||||
"file_size": 524288000,
|
||||
"bind_type": "course",
|
||||
"bind_id": 1
|
||||
}
|
||||
Response: { "upload_id": "uuid", "chunk_size": 5242880, "chunk_count": 100 }
|
||||
|
||||
# 2. 上传分片(循环调用)
|
||||
POST /api/media/upload-chunk
|
||||
Content-Type: multipart/form-data
|
||||
{
|
||||
"upload_id": "uuid",
|
||||
"chunk_index": 0,
|
||||
"file": <binary>
|
||||
}
|
||||
|
||||
# 3. 完成合并
|
||||
POST /api/media/upload-complete
|
||||
{ "upload_id": "uuid" }
|
||||
Response: { "media_id": 1, "status": "pending" }
|
||||
```
|
||||
|
||||
## 4. 审批 API
|
||||
|
||||
```python
|
||||
# 管理员
|
||||
GET /api/media/audit-list?status=pending&page=1&size=20
|
||||
|
||||
POST /api/media/audit/{mediaId}
|
||||
{
|
||||
"action": "approve", # approve | reject
|
||||
"reject_reason": "..." # 驳回时必填
|
||||
}
|
||||
```
|
||||
|
||||
## 5. 异步转换管线
|
||||
|
||||
```python
|
||||
# services/media_service.py
|
||||
import threading
|
||||
from datetime import datetime
|
||||
|
||||
def _async_convert_pipeline(media_id: int):
|
||||
"""审批通过后的异步转换管线"""
|
||||
try:
|
||||
media = db.query(MediaFile).get(media_id)
|
||||
|
||||
# 仅文档需要转换(PPT/Word/PDF)
|
||||
if media.file_ext in ("ppt", "pptx", "doc", "docx"):
|
||||
# Step 1: LibreOffice 转 PDF
|
||||
pdf_path = _libreoffice_to_pdf(media.stored_path)
|
||||
|
||||
# Step 2: PyMuPDF 提取文本
|
||||
text = _pymupdf_extract(pdf_path)
|
||||
|
||||
# Step 3: 按段落切片入库
|
||||
chunks = _split_into_chunks(text)
|
||||
for i, chunk_text in enumerate(chunks):
|
||||
db.add(KnowledgeChunk(
|
||||
media_file_id=media.id,
|
||||
source_type=media.file_ext,
|
||||
chunk_index=i,
|
||||
content=chunk_text,
|
||||
))
|
||||
elif media.file_ext == "pdf":
|
||||
# PDF 直接 PyMuPDF 提取
|
||||
text = _pymupdf_extract(media.stored_path)
|
||||
chunks = _split_into_chunks(text)
|
||||
for i, chunk_text in enumerate(chunks):
|
||||
db.add(KnowledgeChunk(media_file_id=media.id, ...))
|
||||
|
||||
# 视频/图片:不提取文本
|
||||
media.extracted = True
|
||||
db.commit()
|
||||
logger.info(f"转换完成: media_id={media_id}")
|
||||
except Exception as e:
|
||||
logger.error(f"转换失败: media_id={media_id}, error={e}")
|
||||
media.extracted = False # 标记失败可重试
|
||||
|
||||
def approve_media(media_id: int, auditor_id: int):
|
||||
"""审批通过 → 启动异步转换"""
|
||||
media = db.query(MediaFile).get(media_id)
|
||||
media.status = "approved"
|
||||
media.audit_by = auditor_id
|
||||
media.audit_at = datetime.utcnow()
|
||||
db.commit()
|
||||
|
||||
thread = threading.Thread(target=_async_convert_pipeline, args=(media_id,))
|
||||
thread.start()
|
||||
```
|
||||
|
||||
## 6. LibreOffice 转换接口
|
||||
|
||||
```python
|
||||
# utils/libreoffice.py
|
||||
import subprocess
|
||||
import requests
|
||||
|
||||
def libreoffice_convert(input_path: str, output_dir: str) -> str:
|
||||
"""调用 LibreOffice 容器将文档转 PDF"""
|
||||
# 方式1:本地安装 libreoffice
|
||||
subprocess.run([
|
||||
"libreoffice", "--headless", "--convert-to", "pdf",
|
||||
"--outdir", output_dir, input_path
|
||||
], check=True)
|
||||
|
||||
# 方式2:Docker 容器 HTTP 接口
|
||||
# response = requests.post(
|
||||
# f"{settings.libreoffice_url}/convert",
|
||||
# files={"file": open(input_path, "rb")}
|
||||
# )
|
||||
# return response.json()["pdf_path"]
|
||||
```
|
||||
|
||||
## 7. PyMuPDF 文本提取
|
||||
|
||||
```python
|
||||
# utils/pdf_extractor.py
|
||||
import fitz # PyMuPDF
|
||||
|
||||
def extract_text(pdf_path: str) -> str:
|
||||
"""提取 PDF 全部文本"""
|
||||
doc = fitz.open(pdf_path)
|
||||
text = ""
|
||||
for page in doc:
|
||||
text += page.get_text()
|
||||
doc.close()
|
||||
return text
|
||||
|
||||
def split_into_chunks(text: str, max_chars: int = 1000) -> list[str]:
|
||||
"""按段落 + 最大字符数切片"""
|
||||
paragraphs = text.split("\n\n")
|
||||
chunks = []
|
||||
current = ""
|
||||
for p in paragraphs:
|
||||
if len(current) + len(p) > max_chars:
|
||||
if current:
|
||||
chunks.append(current.strip())
|
||||
current = p
|
||||
else:
|
||||
current += "\n\n" + p if current else p
|
||||
if current:
|
||||
chunks.append(current.strip())
|
||||
return chunks
|
||||
```
|
||||
|
||||
## 8. 预览 API
|
||||
|
||||
```python
|
||||
GET /api/media/preview/{mediaId}
|
||||
# 仅 approved 素材可预览
|
||||
Response:
|
||||
{
|
||||
"preview_url": "/media/upload/uuid-filename.pdf",
|
||||
"file_ext": "pdf",
|
||||
"can_preview": true
|
||||
}
|
||||
|
||||
GET /api/media/status/{mediaId}
|
||||
# 查询素材状态(含提取进度)
|
||||
Response:
|
||||
{
|
||||
"status": "approved",
|
||||
"extracted": true,
|
||||
"chunk_count": 42
|
||||
}
|
||||
```
|
||||
|
||||
## 9. 安全约束
|
||||
|
||||
- 扩展名白名单:ppt/pptx/pdf/doc/docx/mp4/png/jpg/jpeg
|
||||
- 文件名重命名为 UUID,杜绝路径穿越
|
||||
- 上传目录对静态预览只读,禁止直接执行
|
||||
- 文件大小:文档 ≤ 200MB,视频 ≤ 2GB
|
||||
- MIME 类型校验 + 扩展名双重校验
|
||||
@@ -0,0 +1,350 @@
|
||||
# BE04 — AI PathCoach 模块设计
|
||||
|
||||
> **版本:V1.1 | httpx 适配器 + 配置链 + Fail Fast | 参考:pj006-zhilianyuan2 llm/openai_adapter.py**
|
||||
> **配置优先级:system_config 数据库表 → .env 文件**
|
||||
|
||||
---
|
||||
|
||||
## 1. 模块职责
|
||||
|
||||
- 全局 AI 聊天框的后端支持
|
||||
- 上下文注入(当前产品/课程信息自动带入)
|
||||
- 知识检索(MySQL FULLTEXT 召回 → Prompt 注入)
|
||||
- SSE 流式响应 + 非流式调用(快捷动作)
|
||||
- 3 个快捷动作(情景演练/查佣金/产品对比)
|
||||
|
||||
## 2. 架构
|
||||
|
||||
```
|
||||
用户消息 + 页面上下文
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ 知识检索 │
|
||||
│ MySQL FULLTEXT │
|
||||
│ → 匹配段落 │
|
||||
└──────┬───────────┘
|
||||
│ 上下文片段
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ Prompt 组装 │
|
||||
│ System Prompt │
|
||||
│ + 知识上下文 │
|
||||
│ + 对话历史 │
|
||||
└──────┬───────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────┐
|
||||
│ build_llm_adapter() 工厂 │
|
||||
│ → 读取配置(库→.env) │
|
||||
│ → Fail Fast 缺配置抛 501 │
|
||||
│ → 返回 OpenAICompatible │
|
||||
└──────┬───────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ httpx 调用 │
|
||||
│ /chat/completions│
|
||||
│ SSE 流式 / 非流式│
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
## 3. 配置优先级与 Fail Fast
|
||||
|
||||
```python
|
||||
# services/ai_service.py
|
||||
from __future__ import annotations
|
||||
import httpx
|
||||
from typing import Generator
|
||||
from dataclasses import dataclass, field
|
||||
from app.core.config import Settings
|
||||
from app.errors import AppError
|
||||
|
||||
|
||||
class LLMNotConfiguredError(AppError):
|
||||
"""LLM 未配置(缺 api_key / base_url / model)"""
|
||||
status_code = 501
|
||||
error_code = "llm_not_configured"
|
||||
|
||||
|
||||
@dataclass
|
||||
class LLMConfig:
|
||||
"""LLM 连接所需的三项配置"""
|
||||
base_url: str
|
||||
api_key: str
|
||||
model: str
|
||||
|
||||
|
||||
def resolve_llm_config(settings: Settings) -> LLMConfig:
|
||||
"""按优先级链解析 LLM 配置。缺任何一项即抛 LLMNotConfiguredError。
|
||||
|
||||
优先级(高 → 低):
|
||||
1. settings.llm_*(来自 system_config 数据库表)
|
||||
2. settings 中从 .env 读取的默认值
|
||||
"""
|
||||
base_url = getattr(settings, "llm_base_url", None) or ""
|
||||
api_key = getattr(settings, "llm_api_key", None) or ""
|
||||
model = getattr(settings, "llm_model", None) or ""
|
||||
|
||||
missing = []
|
||||
if not base_url:
|
||||
missing.append("llm_base_url")
|
||||
if not api_key:
|
||||
missing.append("llm_api_key")
|
||||
if not model:
|
||||
missing.append("llm_model")
|
||||
|
||||
if missing:
|
||||
raise LLMNotConfiguredError(
|
||||
f"LLM 服务未配置——缺失:{', '.join(missing)}。"
|
||||
f"请管理员在【系统参数配置】中补充。"
|
||||
)
|
||||
|
||||
return LLMConfig(base_url=base_url, api_key=api_key, model=model)
|
||||
```
|
||||
|
||||
## 4. LLM 适配器(httpx 实现,参考 zhilianyuan2 openai_adapter.py)
|
||||
|
||||
使用 httpx 替代 OpenAI SDK,减少依赖、更可控、支持 token usage 采集。
|
||||
|
||||
```python
|
||||
# services/ai_service.py
|
||||
|
||||
class LLMAdapter:
|
||||
"""OpenAI 兼容接口适配器(httpx 实现,无 openai SDK 依赖)"""
|
||||
|
||||
def __init__(self, *, config: LLMConfig, http_client: httpx.Client | None = None):
|
||||
self._base_url = config.base_url.rstrip("/")
|
||||
self._api_key = config.api_key
|
||||
self._model = config.model
|
||||
self._client = http_client or httpx.Client(timeout=60.0)
|
||||
# 最后一次流式调用的 token 用量(供日志埋点)
|
||||
self.last_stream_usage: dict[str, int] = {}
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
return {
|
||||
"Authorization": f"Bearer {self._api_key}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
def _payload(self, messages: list[dict], *, stream: bool, **kwargs) -> dict:
|
||||
payload = {
|
||||
"model": self._model,
|
||||
"messages": messages,
|
||||
"stream": stream,
|
||||
"temperature": kwargs.get("temperature", 0.7),
|
||||
"max_tokens": kwargs.get("max_tokens", 2048),
|
||||
}
|
||||
if stream:
|
||||
payload["stream_options"] = {"include_usage": True}
|
||||
return payload
|
||||
|
||||
def generate(self, messages: list[dict], **kwargs) -> str:
|
||||
"""非流式调用,返回完整正文。用于快捷动作等一次性请求。"""
|
||||
try:
|
||||
resp = self._client.post(
|
||||
f"{self._base_url}/chat/completions",
|
||||
headers=self._headers(),
|
||||
json=self._payload(messages, stream=False, **kwargs),
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
content = data["choices"][0]["message"]["content"]
|
||||
if not content or not content.strip():
|
||||
raise LLMError("LLM 返回空正文")
|
||||
return content
|
||||
except httpx.HTTPError as e:
|
||||
raise LLMError(f"LLM 调用失败:{e}") from e
|
||||
except (KeyError, ValueError) as e:
|
||||
raise LLMError(f"LLM 响应解析失败:{e}") from e
|
||||
|
||||
def generate_stream(self, messages: list[dict], **kwargs) -> Generator[str, None, None]:
|
||||
"""SSE 流式调用。逐 chunk yield 文本,末包采集 usage。"""
|
||||
self.last_stream_usage = {}
|
||||
try:
|
||||
with self._client.stream(
|
||||
"POST",
|
||||
f"{self._base_url}/chat/completions",
|
||||
headers=self._headers(),
|
||||
json=self._payload(messages, stream=True, **kwargs),
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
for line in resp.iter_lines():
|
||||
if not line or not line.startswith("data: "):
|
||||
continue
|
||||
payload = line[6:].strip()
|
||||
if payload == "[DONE]":
|
||||
break
|
||||
chunk = json.loads(payload)
|
||||
# 末包采集 usage(include_usage=true)
|
||||
usage = chunk.get("usage")
|
||||
if usage:
|
||||
self.last_stream_usage = {
|
||||
"input_tokens": int(usage.get("prompt_tokens", 0) or 0),
|
||||
"output_tokens": int(usage.get("completion_tokens", 0) or 0),
|
||||
}
|
||||
choices = chunk.get("choices", [])
|
||||
if not choices:
|
||||
continue
|
||||
delta = choices[0].get("delta", {})
|
||||
content = delta.get("content", "")
|
||||
if content:
|
||||
yield content
|
||||
except httpx.HTTPError as e:
|
||||
raise LLMError(f"LLM 流式调用失败:{e}") from e
|
||||
except (KeyError, ValueError) as e:
|
||||
raise LLMError(f"LLM 流式响应解析失败:{e}") from e
|
||||
|
||||
|
||||
def build_llm_adapter(settings: Settings) -> LLMAdapter:
|
||||
"""工厂方法:解析配置 → 构造适配器。配置不全即 Fail Fast。"""
|
||||
config = resolve_llm_config(settings)
|
||||
return LLMAdapter(config=config)
|
||||
```
|
||||
|
||||
## 5. 知识检索
|
||||
|
||||
```python
|
||||
# services/ai_service.py
|
||||
|
||||
def retrieve_knowledge(keywords: str, db: Session, top_k: int = 5) -> list[str]:
|
||||
"""MySQL 全文索引检索知识块"""
|
||||
results = db.execute(
|
||||
text(
|
||||
"SELECT content FROM knowledge_chunk "
|
||||
"WHERE MATCH(content) AGAINST(:keywords IN NATURAL LANGUAGE MODE) "
|
||||
"LIMIT :limit"
|
||||
),
|
||||
{"keywords": keywords, "limit": top_k},
|
||||
).fetchall()
|
||||
return [r[0] for r in results]
|
||||
```
|
||||
|
||||
## 6. System Prompt
|
||||
|
||||
```python
|
||||
SYSTEM_PROMPT = """你是一个博昇内部培训平台的 AI 助教 PathCoach。
|
||||
|
||||
你的职责:
|
||||
1. 解答公司介绍、产品知识、佣金规则、销售话术、业务规则相关的问题
|
||||
2. 严格依赖已审批知识库的内容回答
|
||||
3. 如果知识库中未找到相关资料,明确回答「未找到相关资料」,不得臆测
|
||||
|
||||
禁止行为:
|
||||
1. 禁止闲聊
|
||||
2. 禁止编造数据
|
||||
3. 禁止回答超出业务范围的问题
|
||||
4. 禁止泄露敏感信息
|
||||
|
||||
当前页面上下文:
|
||||
{page_context}
|
||||
|
||||
知识库相关片段:
|
||||
{knowledge_context}
|
||||
"""
|
||||
```
|
||||
|
||||
## 7. API 实现
|
||||
|
||||
```python
|
||||
# api/ai_chat.py
|
||||
from fastapi.responses import StreamingResponse
|
||||
|
||||
router = APIRouter(prefix="/api/ai-chat", tags=["ai_chat"])
|
||||
|
||||
@router.post("/message")
|
||||
def chat_message(
|
||||
request: ChatRequest,
|
||||
current_user: CurrentUser,
|
||||
db: Session = Depends(get_db),
|
||||
settings: Settings = Depends(get_settings),
|
||||
):
|
||||
"""SSE 流式对话"""
|
||||
# 1. 检索知识
|
||||
knowledge = retrieve_knowledge(request.message, db)
|
||||
|
||||
# 2. 组装 Prompt
|
||||
system = SYSTEM_PROMPT.format(
|
||||
page_context=json.dumps(request.context or {}),
|
||||
knowledge_context="\n\n".join(knowledge),
|
||||
)
|
||||
messages = [
|
||||
{"role": "system", "content": system},
|
||||
*request.history,
|
||||
{"role": "user", "content": request.message},
|
||||
]
|
||||
|
||||
# 3. 构建 LLM 适配器(缺配置即抛 501)
|
||||
llm = build_llm_adapter(settings)
|
||||
|
||||
def generate():
|
||||
for chunk in llm.generate_stream(messages):
|
||||
yield f"data: {json.dumps({'type': 'text', 'content': chunk})}\n\n"
|
||||
yield "data: {\"type\": \"done\"}\n\n"
|
||||
|
||||
return StreamingResponse(
|
||||
generate(), media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
|
||||
|
||||
@router.get("/quick-actions")
|
||||
def get_quick_actions():
|
||||
"""获取 3 个快捷按钮"""
|
||||
return {"data": {"actions": [
|
||||
{"id": "scenario", "label": "客户情景演练"},
|
||||
{"id": "commission", "label": "查询佣金/规则"},
|
||||
{"id": "compare", "label": "产品对比"},
|
||||
]}}
|
||||
|
||||
|
||||
@router.post("/quick-action")
|
||||
def trigger_quick_action(
|
||||
request: QuickActionRequest,
|
||||
settings: Settings = Depends(get_settings),
|
||||
):
|
||||
"""触发快捷动作(非流式 LLM 调用)"""
|
||||
llm = build_llm_adapter(settings)
|
||||
|
||||
if request.action_id == "commission":
|
||||
# 查佣金:构建 prompt → 非流式调用
|
||||
prompt = f"查询产品佣金信息,产品参数:{request.params}"
|
||||
resp = llm.generate([{"role": "user", "content": prompt}])
|
||||
return {"data": {"result": resp}}
|
||||
|
||||
elif request.action_id == "compare":
|
||||
prompt = f"对比以下产品:{request.params}"
|
||||
resp = llm.generate([{"role": "user", "content": prompt}])
|
||||
return {"data": {"result": resp}}
|
||||
|
||||
elif request.action_id == "scenario":
|
||||
# 情景演练:返回初始话术,后续走流式对话
|
||||
prompt = f"开始销售情景演练,场景参数:{request.params}"
|
||||
resp = llm.generate([{"role": "user", "content": prompt}])
|
||||
return {"data": {"result": resp, "mode": "scenario"}}
|
||||
```
|
||||
|
||||
## 8. 上下文注入规则
|
||||
|
||||
| 页面 | 自动注入上下文 | 说明 |
|
||||
|------|--------------|------|
|
||||
| 产品详情 | `product_id`, `product_name`, `product_code` | 自动带入当前产品 |
|
||||
| 课程详情 | `course_id`, `course_name`, `related_product` | 自动带入当前课程及关联产品 |
|
||||
| 公司介绍 | `page: "company_intro"` | 提示 AI 当前页为公司介绍 |
|
||||
|
||||
## 9. 配置项
|
||||
|
||||
| 配置键 | 来源 | 说明 |
|
||||
|--------|------|------|
|
||||
| `llm_base_url` | system_config 表 / .env | LLM 服务地址,如 `http://192.168.1.100:11434/v1` |
|
||||
| `llm_api_key` | system_config 表 / .env | API Key,本地 Ollama 可填 `ollama` |
|
||||
| `llm_model` | system_config 表 / .env | 模型名,如 `qwen2.5:7b` |
|
||||
|
||||
## 10. 错误处理
|
||||
|
||||
| 场景 | HTTP 状态 | 响应 |
|
||||
|------|----------|------|
|
||||
| LLM 未配置(缺 base_url/key/model) | 501 | `{"error": "llm_not_configured", "message": "请管理员在系统参数配置中补充..."}` |
|
||||
| LLM 调用超时/网络错误 | 502 | `{"error": "llm_request_failed", "message": "LLM 服务不可达,请检查网络连接"}` |
|
||||
| LLM 返回空正文 | 502 | `{"error": "llm_empty_response", "message": "LLM 返回空结果"}` |
|
||||
| LLM 响应格式异常 | 502 | `{"error": "llm_response_error", "message": "LLM 响应异常"}` |
|
||||
@@ -0,0 +1,226 @@
|
||||
# BE05 — 知识入库体系总设计(Knowledge Ingest)
|
||||
|
||||
> **版本:V1.1 | 定稿**
|
||||
> **定位**:把「素材入库」与「结构化知识入库」统一为一套可复用的知识入库体系,覆盖上传 → 生成 → 审批 → 入库全链路。
|
||||
> **配套**:BE03(素材转换管线)、BE04(AI 检索)、`docs/knowledge_source/README.md`(知识源格式契约)。
|
||||
|
||||
---
|
||||
|
||||
## 1. 模块职责
|
||||
|
||||
知识入库体系负责把两类知识源,统一经过「审批前置」流入三张消费表,最终支撑「可浏览 / 可考试 / 可 AI 检索」。
|
||||
|
||||
| 知识源类型 | 载体 | 审批单元 | 流入表 |
|
||||
|-----------|------|---------|--------|
|
||||
| **非结构化素材** | PPT/PDF/Word/视频/图片 | 逐文件 | knowledge_chunk(+ 预览) |
|
||||
| **结构化知识** | 知识源 md(docs/knowledge_source/) | 逐文档 | product / question / knowledge_chunk |
|
||||
|
||||
**核心原则(与 P02/P04 对齐)**:所有知识只有 `approved` 才生效;`pending` / `rejected` 一律不解析、不进库、前台不可见。
|
||||
|
||||
---
|
||||
|
||||
## 2. 完整目录结构(定稿)
|
||||
|
||||
```
|
||||
eaisalestrain_app/
|
||||
├── docs/
|
||||
│ └── knowledge_source/ # 知识源文档(权威源,纳入 git 版本管理)
|
||||
│ ├── README.md # 格式契约 + 答案契约 + 审批状态机
|
||||
│ ├── 01_通用规则.md
|
||||
│ ├── 02_资本咨询类.md
|
||||
│ ├── 03_资质认定辅导类.md
|
||||
│ ├── 04_AI咨询与实施类.md
|
||||
│ └── 05_企业级AI工具与平台.md
|
||||
│
|
||||
├── backend/
|
||||
│ ├── app/
|
||||
│ │ ├── api/
|
||||
│ │ │ ├── media.py # 【已有】素材上传/审批/预览
|
||||
│ │ │ └── knowledge.py # 【新增】知识源扫描/审批/摄入
|
||||
│ │ ├── models/
|
||||
│ │ │ ├── media_file.py # 【已有】素材表
|
||||
│ │ │ ├── knowledge_chunk.py # 【改造】来源扩展(见 §4)
|
||||
│ │ │ └── knowledge_source.py # 【新增】知识源文档表
|
||||
│ │ ├── services/
|
||||
│ │ │ ├── media_service.py # 【已有】LibreOffice/PyMuPDF 转换管线
|
||||
│ │ │ └── knowledge_service.py # 【新增】md 解析 + 摄入
|
||||
│ │ └── scripts/
|
||||
│ │ ├── init_db.py # 【已有】建表 + 种子
|
||||
│ │ └── ingest_knowledge.py # 【新增】知识源扫描/摄入脚本
|
||||
│ └── data/
|
||||
│ ├── media/ # 【已有】素材物理文件(git 忽略)
|
||||
│ │ ├── upload/
|
||||
│ │ └── _preview_cache/
|
||||
│ └── logs/ # 【已有】special_trace 按天日志
|
||||
│
|
||||
└── docker-compose.yml # 【待创建】
|
||||
```
|
||||
|
||||
**约定**:
|
||||
- `docs/knowledge_source/` 是知识源 md 的唯一入库口(权威源,git 管理,可 diff 可回滚)。
|
||||
- 运行时摄入**直接读取**该目录,不复制到 backend/data(单一事实源,避免双份漂移)。
|
||||
- 素材物理文件仍在 `backend/data/media/`(git 忽略)。
|
||||
|
||||
---
|
||||
|
||||
## 3. 两条流程的状态机(定稿)
|
||||
|
||||
### 流程 A:非结构化素材(BE03 已实现)
|
||||
|
||||
```
|
||||
员工上传 ──▶ media_file(pending) ──审批──▶ approved ──▶ 异步转换
|
||||
管理员上传 ──▶ media_file(approved) ──▶ 立即异步转换 │
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ 文档: LibreOffice→PDF→ │
|
||||
│ PyMuPDF 提取→切片 │
|
||||
│ 视频/图片: 仅预览 │
|
||||
└──────────┬───────────────┘
|
||||
▼
|
||||
knowledge_chunk
|
||||
```
|
||||
|
||||
### 流程 B:结构化知识源(本次新增)
|
||||
|
||||
```
|
||||
知识源 md(docs/knowledge_source/)
|
||||
│
|
||||
▼
|
||||
【扫描】ingest_knowledge.py / POST /api/knowledge/scan
|
||||
│ 解析 front-matter → 为每个 md 建 knowledge_source 记录
|
||||
▼
|
||||
knowledge_source(pending) ──审批──▶ approved ──▶ 解析摄入
|
||||
│ │
|
||||
│ ├─→ product(status=active)
|
||||
│ ├─→ question(status=active)
|
||||
│ └─→ knowledge_chunk(source=knowledge_source)
|
||||
│
|
||||
└─驳回──▶ rejected(理由必填,不生效)
|
||||
```
|
||||
|
||||
**对称性**:素材「审批通过→转换提取」,知识源「审批通过→解析摄入」。两条通道最终都汇入 `knowledge_chunk` 供 AI 检索。
|
||||
|
||||
---
|
||||
|
||||
## 4. 数据模型变更
|
||||
|
||||
### 4.1 新增 `knowledge_source` 表
|
||||
|
||||
```sql
|
||||
CREATE TABLE knowledge_source (
|
||||
id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
|
||||
title VARCHAR(256) NOT NULL COMMENT '文档标题',
|
||||
file_path VARCHAR(512) NOT NULL UNIQUE COMMENT 'md 相对路径(docs/knowledge_source/ 下)',
|
||||
category VARCHAR(64) NOT NULL COMMENT '分类:general/capital_consulting/qualification_counseling/ai_consulting/ai_tools_platform',
|
||||
domain ENUM('company','product','sales') NOT NULL DEFAULT 'product',
|
||||
source_version VARCHAR(32) NOT NULL COMMENT '源版本,如 V1.0',
|
||||
audit_status ENUM('pending','approved','rejected') NOT NULL DEFAULT 'pending',
|
||||
audit_by BIGINT UNSIGNED DEFAULT NULL COMMENT '审批人ID',
|
||||
audit_at DATETIME(6) DEFAULT NULL COMMENT '审批时间',
|
||||
reject_reason VARCHAR(512) DEFAULT NULL COMMENT '驳回理由',
|
||||
ingested TINYINT(1) NOT NULL DEFAULT 0 COMMENT '是否已摄入',
|
||||
created_at DATETIME(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
|
||||
updated_at DATETIME(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
|
||||
|
||||
INDEX idx_ks_status (audit_status),
|
||||
INDEX idx_ks_category (category),
|
||||
CONSTRAINT fk_ks_auditor FOREIGN KEY (audit_by) REFERENCES user(id)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
|
||||
```
|
||||
|
||||
### 4.2 改造 `knowledge_chunk` 表(来源扩展)
|
||||
|
||||
现状 `media_file_id BIGINT UNSIGNED NOT NULL` 强绑定素材。改造后:
|
||||
|
||||
```sql
|
||||
media_file_id BIGINT UNSIGNED DEFAULT NULL, -- 由 NOT NULL 改为可空
|
||||
knowledge_source_id BIGINT UNSIGNED DEFAULT NULL, -- 新增
|
||||
-- 约束:media_file_id 与 knowledge_source_id 二者必居其一(应用层校验,Fail Fast)
|
||||
```
|
||||
|
||||
对应 ORM `models/knowledge_chunk.py`:`media_file_id` 改为可空,新增 `knowledge_source_id` 外键与 relationship。
|
||||
|
||||
---
|
||||
|
||||
## 5. 模块清单(8 模块)
|
||||
|
||||
| # | 模块 | 状态 | 职责 | 落点 |
|
||||
|---|------|------|------|------|
|
||||
| 1 | 上传模块 | ✅ 已有 | 直传 + 分片(>100MB) | `api/media.py` |
|
||||
| 2 | 转换模块 | ✅ 已有 | LibreOffice → PDF | `services/media_service.py` |
|
||||
| 3 | 提取切片模块 | ✅ 已有 | PyMuPDF 提取 + 段落切片 | `services/media_service.py` |
|
||||
| 4 | **摄入模块** | 🆕 新增 | 解析知识源 md → product/question/chunk | `services/knowledge_service.py` |
|
||||
| 5 | 审批模块 | 🔧 扩展 | 素材审批(已有)+ 知识源审批(新增) | `api/media.py` + `api/knowledge.py` |
|
||||
| 6 | 入库模块 | 🔧 扩展 | 写 product/question/knowledge_chunk | `services/knowledge_service.py` |
|
||||
| 7 | 预览模块 | ✅ 已有 | approved 素材预览 | `api/media.py` |
|
||||
| 8 | 检索模块 | ✅ 已有 | MySQL FULLTEXT 召回 | `services/ai_service.py` |
|
||||
|
||||
---
|
||||
|
||||
## 6. 新增 API(knowledge.py)
|
||||
|
||||
```python
|
||||
router = APIRouter(prefix="/api/knowledge", tags=["knowledge"])
|
||||
|
||||
# ── 扫描(管理员)──
|
||||
POST /api/knowledge/scan
|
||||
# 扫描 docs/knowledge_source/*.md
|
||||
# 为新增/变更的 md 建 knowledge_source 记录(status=pending)
|
||||
# 已存在且未摄入的记录跳过;返回扫描结果列表
|
||||
|
||||
# ── 审批(管理员)──
|
||||
GET /api/knowledge/audit-list?status=pending
|
||||
POST /api/knowledge/audit/{source_id}
|
||||
# { "action": "approve" | "reject", "reject_reason": "..." }
|
||||
# approve → 触发摄入(同步解析写入 product/question/chunk,标记 ingested=1)
|
||||
# reject → 必填 reject_reason,不摄入
|
||||
|
||||
# ── 状态查询 ──
|
||||
GET /api/knowledge/status/{source_id}
|
||||
# { "audit_status", "ingested", "reject_reason" }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 摄入脚本(ingest_knowledge.py)
|
||||
|
||||
```python
|
||||
"""知识源摄入脚本:扫描 md → 建记录 → (审批通过后)解析入库
|
||||
|
||||
用法:
|
||||
cd backend && source venv/bin/activate
|
||||
python -m app.scripts.ingest_knowledge --scan # 仅扫描建 pending 记录
|
||||
python -m app.scripts.ingest_knowledge --ingest <source_id> # 摄入单个已审批源
|
||||
"""
|
||||
|
||||
# 解析规则(严格对齐 knowledge_source/README.md 格式契约):
|
||||
# 1. 读 YAML front-matter:category / domain / source_version
|
||||
# 2. 按 "## " 二级标题切块:
|
||||
# ## 结构化产品数据 → 解析 "### code name" + "key: value" 列表 → product
|
||||
# ## AI 检索知识 → 每个 "### 标题" 段落 → knowledge_chunk
|
||||
# ## 考试题目 → 每个 "### Qn" 的字段列表 → question
|
||||
# 3. 摄入后 product/question 置 status=active
|
||||
# 4. knowledge_chunk 写入 knowledge_source_id,media_file_id 留空
|
||||
```
|
||||
|
||||
**Fail Fast 约束(G02)**:解析失败(front-matter 缺失、section 缺块、字段缺失)立即抛错并中断该源摄入,不静默跳过、不写半截数据。
|
||||
|
||||
---
|
||||
|
||||
## 8. 格式契约(已定,见 knowledge_source/README.md)
|
||||
|
||||
- 每个 md:YAML front-matter(`category` / `domain` / `source_version`)+ 三个固定 `## ` section。
|
||||
- 题目答案契约:`judge=[bool]`、`single=[索引]`、`multiple=[索引列表]`。
|
||||
- 已生成 5 个知识源:`01_通用规则` + `02/03/04/05` 四大分类,共 28 产品 + 20 题。
|
||||
|
||||
---
|
||||
|
||||
## 9. 实现清单(代码侧,已完成)
|
||||
|
||||
- [x] `models/knowledge_source.py` 新建
|
||||
- [x] `models/knowledge_chunk.py` 来源扩展(media_file_id 可空 + knowledge_source_id)
|
||||
- [x] `services/knowledge_service.py`(md 解析 + 摄入)
|
||||
- [x] `api/knowledge.py`(scan / audit / status)
|
||||
- [x] `scripts/ingest_knowledge.py`
|
||||
- [x] `scripts/init_db.py` 追加 knowledge_source 建表 + knowledge_chunk 字段迁移
|
||||
- [x] `docs/db_schema.md`、`docs/api.md` 同步更新
|
||||
@@ -0,0 +1,226 @@
|
||||
# BE06 — AI 配置与算力点计费模块设计
|
||||
|
||||
> **版本:V1.0 | 从 pj034-oeamgt 完整移植 + 精简适配 | 参考:pj034 `core/ai_config.py`、`services/ai/router.py`、`models/ai_call_log.py`、`api/v1/ai_billing.py`**
|
||||
> **目标:把 pj034 成熟的「AI 路由/Provider/密钥配置 + 算力点计费 + 调用审计」体系,按 pj0231「极致极简、单公司、按用户计点」落地。**
|
||||
|
||||
---
|
||||
|
||||
## 1. 背景与目标
|
||||
|
||||
pj034 的 AI 系统已演进为「多层、多公司、多租户」的完整运维平台:平台级路由配置 + 公司级覆盖 + 算力点计费 + 调用审计 + 用量报表。pj0231 是内部培训平台,只需要其中与「AI 助教 PathCoach」相关的子集。
|
||||
|
||||
**移植范围(保留)**:
|
||||
1. AI 路由/Provider/密钥配置文件体系(`ai_config.json` + `ai_secrets.json`)——已落地
|
||||
2. Agent → 路由映射、默认路由、回退链(fallback)
|
||||
3. **按用户算力点计费**:每次 AI 调用按能力扣点 + 写调用日志
|
||||
4. AI 配置管理 API(读/写/热重载/密钥状态)+ 用量查询 API
|
||||
5. 前端:AI 配置 UI + AI 用量看板 + 用户剩余点数展示
|
||||
|
||||
**排除范围(pj034 专属,pj0231 不适用)**:
|
||||
| pj034 能力 | 排除理由 |
|
||||
|-----------|---------|
|
||||
| 图片生成 / 抠图 / 换背景 / AI 模特 / 卖点图 | 电商场景,培训平台无此需求 |
|
||||
| 多公司覆盖层 `company_ai_config`(AES 加密、billing_mode 分流) | 单公司内网部署 |
|
||||
| 套餐订阅 `pricing.py` / 配额守护 `quota_guard.py`(tier 阶梯) | 无订阅计费,改为按用户点数 |
|
||||
| `cost_cny`(人民币成本核算) | 内网,无对外结算 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 现状盘点(pj0231 已落地部分)
|
||||
|
||||
| 模块 | 文件 | 状态 |
|
||||
|------|------|------|
|
||||
| 配置加载器 | `internal/config/json_loader.go` | ✅ 已移植(RouteInfo/RouteConfig/AIConfig/AISecrets/ProviderSecretKey/ProviderDefaultBaseURL) |
|
||||
| 路由解析 | `GetRoute / GetFallbackRoutes / GetAllRoutes / GetRoutesByCategory` | ✅ |
|
||||
| 配置文件 | `config/ai_config.json` `ai_secrets.json` `ai_secrets.example.json` `platform.json` | ✅ |
|
||||
| LLM 客户端 | `internal/ai/llm.go`(NewClient/NewClientLegacy/Generate/GenerateFull/GenerateStream/Embed/GenerateWithFallback) | ✅ |
|
||||
| 混合检索 | `internal/ai/retrieve.go`(向量 + 关键词,embed 走 `embed_gen` agent) | ✅ |
|
||||
| 对话/快捷动作 | `internal/api/ai_chat.go`(ChatMessage SSE + QuickAction) | ✅ |
|
||||
| 路由选择器 | `internal/api/routes.go`(ListChatRoutes/ListEmbedRoutes) | ✅ |
|
||||
|
||||
**待补(本次工作)**:
|
||||
1. `ai_call_log` 表 + `User.ai_points` 字段(计费与审计)
|
||||
2. 扣点收口逻辑 `compute_credits` + `log_ai_call`
|
||||
3. 计费查询 API(按月/能力聚合,管理员全量 + 用户本人)
|
||||
4. AI 配置管理 API(读/写 `ai_config.json`、热重载、密钥状态)
|
||||
5. `ChatMessage`/`QuickAction` 接入「扣点 + 日志 + 回退链」
|
||||
6. 前端:AI 配置 UI(7.2.4)、AI 用量看板、剩余点数展示
|
||||
|
||||
---
|
||||
|
||||
## 3. pj034 计费体系分析(移植蓝本)
|
||||
|
||||
### 3.1 调用日志表 `ai_call_logs`(pj034)
|
||||
|
||||
| 字段 | 类型 | 说明 | pj0231 取舍 |
|
||||
|------|------|------|------------|
|
||||
| `id` | PK | | ✅ |
|
||||
| `company_id` | int | 公司维度 | ❌ 单公司,删除 |
|
||||
| `user_id` | int | 调用人 | ✅ |
|
||||
| `provider` | enum | provider | ✅ |
|
||||
| `capability` | enum | AI 能力 | ✅ |
|
||||
| `input_asset_id` / `output_asset_id` | int | 电商素材 | ❌ |
|
||||
| `route_id` | str | 路由 ID | ✅ |
|
||||
| `model_id` | str | 模型 | ✅ |
|
||||
| `input_summary` / `output_summary` | text | 摘要 | ❌ 极致极简 |
|
||||
| `raw_request` / `raw_response` | text | 原始 IO | ❌(审计可后补) |
|
||||
| `tokens_input` / `tokens_output` | int | token 用量 | ✅ |
|
||||
| `credits` | numeric | 成本单价 | ❌ |
|
||||
| `cost_cny` | numeric | 人民币成本 | ❌ |
|
||||
| `billing_mode` | enum | platform/self_managed | ❌ 单平台 |
|
||||
| `credits_charged` | int | 实扣点数 | ✅ |
|
||||
| `status` | enum | success/failed/... | ✅ |
|
||||
| `error_message` / `error_detail` | text | 错误 | ✅(保留 error_message) |
|
||||
| `http_status` | int | | ✅ |
|
||||
| `latency_ms` | int | 耗时 | ✅ |
|
||||
| `called_at` | datetime | 调用时间 | ✅ |
|
||||
|
||||
### 3.2 扣点收口 `compute_credits`(pj034)
|
||||
|
||||
```python
|
||||
CAPABILITY_CREDITS = {
|
||||
AiCapability.TEXT_DIAGNOSE: 1,
|
||||
AiCapability.BG_REMOVE: 1,
|
||||
AiCapability.BG_REPLACE: 2,
|
||||
AiCapability.MODEL_GEN: 5,
|
||||
AiCapability.TEXT_GEN: 1,
|
||||
AiCapability.AI_CHAT: 1, # 每轮对话扣 1 点
|
||||
AiCapability.IMAGE_VALIDATE: 1,
|
||||
}
|
||||
|
||||
def compute_credits(capability, billing_mode, status):
|
||||
if status != SUCCESS or billing_mode != PLATFORM:
|
||||
return 0
|
||||
return CAPABILITY_CREDITS.get(capability, 0)
|
||||
```
|
||||
|
||||
**核心规则**:只有「调用成功」才扣点;失败不扣点。扣点决策收口到 service 层,不在各端点散落判断。
|
||||
|
||||
### 3.3 计费聚合 API(pj034 `ai_billing.py`)
|
||||
|
||||
`GET /data/ai-billing?days=30&group_by=month|capability|provider|month_capability`
|
||||
|
||||
返回 `{ summary: {total_calls, success_calls, failed_calls, total_credits_charged}, buckets: [...] }`。MySQL 下按月聚合走 Go 侧 group(不依赖 `date_trunc`)。
|
||||
|
||||
---
|
||||
|
||||
## 4. pj0231 计费模型(精简)
|
||||
|
||||
### 4.1 能力 → 点数(CAPABILITY_CREDITS)
|
||||
|
||||
| capability | 含义 | 点数 |
|
||||
|-----------|------|------|
|
||||
| `ai_chat` | PathCoach 对话(每轮) | 1 |
|
||||
| `text_gen` | 快捷动作(情景演练/查佣金/产品对比) | 1 |
|
||||
| `embed` | 知识检索内部 embedding | 0(不扣,仅记审计) |
|
||||
|
||||
### 4.2 用户点数 `User.ai_points`
|
||||
|
||||
- `User` 表新增 `ai_points int`(默认 100,管理员可充值)。
|
||||
- 新用户默认值走 `system_config.ai_points_default`(默认 `100`)。
|
||||
- 管理员账号默认 `999999`(不限,避免管理员自己用没)。
|
||||
- 扣点规则:成功调用 → 扣 `compute_credits(...)` 点;失败不扣。
|
||||
- **点数不足**:返回 402 `ai_points_exhausted`,前端提示「AI 点数不足,请联系管理员充值」。
|
||||
|
||||
### 4.3 调用日志 `ai_call_log`(pj0231 精简版)
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `id` | uint PK | |
|
||||
| `user_id` | uint index | 调用人 |
|
||||
| `capability` | str | ai_chat / text_gen / embed |
|
||||
| `provider` | str | 实际命中的 provider |
|
||||
| `route_id` | str | 路由 ID |
|
||||
| `model` | str | 模型名 |
|
||||
| `tokens_input` | int | 输入 token |
|
||||
| `tokens_output` | int | 输出 token |
|
||||
| `credits_charged` | int | 实扣点数(0 = 未扣) |
|
||||
| `status` | str | success / failed |
|
||||
| `error_message` | str | 失败信息 |
|
||||
| `latency_ms` | int | 耗时 |
|
||||
| `created_at` | time | 调用时间 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 后端 API 设计(pj0231)
|
||||
|
||||
### 5.1 AI 配置管理(管理员)
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| GET | `/api/ai/config` | 读完整 `ai_config.json`(含 agent_routes + 分类 routes + fallback) |
|
||||
| PUT | `/api/ai/config` | 写回 `ai_config.json`(原子写 + 清缓存热生效) |
|
||||
| POST | `/api/ai/reload` | 热重载(清缓存,无需重启) |
|
||||
| GET | `/api/ai/secrets-status` | 各 provider 密钥是否已配置(仅 `configured: true/false`,不回显明文) |
|
||||
| GET | `/api/ai/routes/chat` | 现有:chat 路由选择器 |
|
||||
| GET | `/api/ai/routes/embed` | 现有:embed 路由选择器 |
|
||||
|
||||
### 5.2 算力点计费查询
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| GET | `/api/ai/usage?days=30&group_by=month` | 用量聚合(管理员 = 全量;员工 = 本人,后端按角色过滤) |
|
||||
| GET | `/api/ai/usage/users` | 管理员:按用户聚合的用量 + 剩余点数(充值入口数据源) |
|
||||
| GET | `/api/ai/me` | 员工:本人剩余点数 + 近 N 天用量 |
|
||||
|
||||
### 5.3 用户点数充值(管理员)
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| PUT | `/api/system/users/{id}` | 扩展现有接口:`ai_points` 字段可设置(充值/扣减) |
|
||||
|
||||
---
|
||||
|
||||
## 6. 前端 UI 设计(pj0231)
|
||||
|
||||
### 6.1 参数配置 7.2.4「AI 配置」改造
|
||||
|
||||
现有 `SystemConfigPage.vue` 的 `aiItems`(llm_base_url/llm_model 等明文 DB 字段)**替换**为 pj034 `PlatformSystem.vue` 的精简版:
|
||||
|
||||
- **Agent → 路由映射**:PathCoach 对话路由(`path_coach`)、快捷动作路由(`title_gen`)、Embedding 路由(`embed_gen`)各一个下拉选择(数据源 `/api/ai/routes/chat|embed`)
|
||||
- **默认路由 / 默认 Embedding 路由**:下拉选择
|
||||
- **密钥状态看板**:每个 provider 显示 `configured: true/false`(数据源 `/api/ai/secrets-status`)
|
||||
- **原始 JSON 编辑器**:折叠面板编辑完整 `ai_config.json`
|
||||
- **热重载 + 保存**:`POST /api/ai/reload` + `PUT /api/ai/config`,带脏检查
|
||||
|
||||
### 6.2 AI 用量看板(新增页面)
|
||||
|
||||
复刻 pj034 `AiUsage.vue` 的精简版:
|
||||
- 汇总卡:总调用 / 成功 / 失败 / 总消耗点数(去掉「¥ 花费」)
|
||||
- 时间窗切换:近 30 / 90 / 365 天
|
||||
- 分组明细:按月 / 按能力 / 按 provider
|
||||
- 管理员额外视角:按用户聚合(含剩余点数)
|
||||
|
||||
菜单位置:知识管理下新增「AI 用量」(管理员);员工入口放在 PathCoach 面板内(本人剩余点数 + 近 30 天用量)。
|
||||
|
||||
### 6.3 PathCoach 面板剩余点数
|
||||
|
||||
`PathCoachPanel.vue` 顶部显示「剩余 AI 点数:N」(数据源 `/api/ai/me`),每次对话/快捷动作完成后刷新。
|
||||
|
||||
---
|
||||
|
||||
## 7. 实施清单
|
||||
|
||||
- [x] 配置加载器 + 配置文件(已落地)
|
||||
- [x] LLM 客户端 + 回退链(`GenerateWithFallback` 已实现,待接入 ChatMessage)
|
||||
- [x] `model/ai_call_log.go` + `model/user.go` 加 `ai_points`
|
||||
- [x] `store/db.go` AutoMigrate 加 `AiCallLog`
|
||||
- [x] `internal/ai/credits.go`:CAPABILITY_CREDITS + compute_credits + log_ai_call
|
||||
- [x] `internal/api/ai_chat.go`:ChatMessage/QuickAction 接入扣点 + 日志 + 回退链
|
||||
- [x] `internal/api/ai_admin.go`:AI 配置读/写/热重载/密钥状态
|
||||
- [x] `internal/api/ai_usage.go`:用量聚合 + 按用户聚合 + 本人剩余点数
|
||||
- [x] `internal/api/system.go`:UpdateUser 支持 ai_points 充值
|
||||
- [x] `internal/api/router.go`:注册新路由
|
||||
- [x] 前端 `api/ai.js` + `api/system.js` 扩展
|
||||
- [x] `SystemConfigPage.vue` 7.2.4 改路由配置 UI
|
||||
- [x] 新增 `AiUsage.vue` 用量看板 + 路由/菜单
|
||||
- [x] `PathCoachPanel.vue` 剩余点数
|
||||
- [x] 重编译 + 重启 + 验证
|
||||
|
||||
---
|
||||
|
||||
## 8. 与既有设计的衔接
|
||||
|
||||
- **配置优先级**:AI 路由走 `ai_config.json`(文件)优先;旧的 `system_config` 表 `llm_base_url/llm_model` 字段**逐步废弃**,`ResolveLLM` 保留为兜底兼容(`NewClientLegacy`),新链路全部走 `GetRoute`。
|
||||
- **极致极简**:不引入多公司、订阅、人民币成本、原始 IO 存储;日志只保留审计必需的字段。
|
||||
- **审批前置**:AI 问答只检索已审批知识库(`knowledge_chunk`),与计费解耦。
|
||||
@@ -0,0 +1,18 @@
|
||||
# 04_Backend — 后端设计与 API
|
||||
|
||||
> **命名规则:** `BE{NN}_{描述}.md`
|
||||
> **用途:** 后端实现细节、API 设计、服务层设计
|
||||
>
|
||||
> **⚠️ 本目录 BE 文档为 V1.1 设计期历史快照(FastAPI/Python)。** 当前后端已重写为 **Go + Gin + GORM + MySQL 8.0 + FAISS**,见 `docs/changelog.md`(V1.2)。V1.4–V1.7 新增的岗位/积分/证书/部门/消息等接口以 `docs/changelog.md` 为准。
|
||||
|
||||
## 文件清单
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `README.md` | 本索引文件 |
|
||||
| `BE01_Auth_Module.md` | 认证模块设计(JWT + bcrypt) |
|
||||
| `BE02_Exam_Module.md` | 考试模块设计(题库/组卷/判分) |
|
||||
| `BE03_Media_Module.md` | 素材模块设计(上传/审批/转换管线) |
|
||||
| `BE04_AI_Chat_Module.md` | AI PathCoach 模块设计(V1 单一助手;智能体矩阵见 SY03) |
|
||||
| `BE05_Knowledge_Ingest_Module.md` | 知识入库体系设计(上传→生成→审批→入库) |
|
||||
| `BE06_AI_Config_Credits_Module.md` | AI 配置与算力点计费模块(移植自 pj034) |
|
||||
Reference in New Issue
Block a user