# BE03 — 素材模块设计

> **版本：V2.0 | 技术：Go + Gin + GORM + SQLite + systemd + LibreOffice + pdftotext**

---

## 1. 模块职责

- 文件上传（直传 + 分片上传）
- 素材审批流（待审批 → 通过/驳回）
- 审批通过后异步文档转换管线
- 文件预览
- 转换状态查询

---

## 2. 素材状态流转

```
员工提交 / 管理员上传
      │
      ▼
  pending（待审批） ──┬─ approve → approved（已通过）
      │               │
      │               └─→ 触发异步转换管线
      │                       │
      │                       ├─ 文档 → LibreOffice 转 PDF → pdftotext 提取
      │                       │   文本 → 切片写入 knowledge_chunk
      │                       └─ 视频/图片 → 仅标记预览可用
      │
      └─ reject → rejected（已驳回，前台不可见）
```

---

## 3. 数据模型

素材模块对应 `media_file` 表（`internal/model/media.go`）：

```go
// internal/model/media.go
type MediaFile struct {
    model.Base
    UserID     uint      `gorm:"not null;index"`
    Filename   string    `gorm:"size:256;not null"`
    StoredPath string    `gorm:"size:512;not null"`
    FileExt    string    `gorm:"size:16;not null"`       // ppt/pptx/pdf/doc/docx/mp4/png/jpg/jpeg
    FileSize   int64     `gorm:"not null"`
    BindType   string    `gorm:"size:32"`                // company|product|course|none
    BindID     *uint     `gorm:"index"`
    Status     string    `gorm:"size:16;not null;default:pending;index"` // pending|approved|rejected
    AuditBy    *uint
    AuditAt    *time.Time
    Extracted  bool      `gorm:"not null;default:false"`
    RejectReason string `gorm:"size:512"`
    CreatedAt  time.Time
    UpdatedAt  time.Time
}
```

---

## 4. 上传 API

### 直传（文档 ≤ 200MB）

```go
// backend-go/internal/api/media.go
func Upload(c *gin.Context) {
    // POST /api/media/upload
    // Content-Type: multipart/form-data

    file, _ := c.FormFile("file")
    bindType := c.PostForm("bind_type")       // company | product | course | none
    bindIDStr := c.PostForm("bind_id")        // 可选

    uid := middleware.GetUserID(c)
    mediaID, err := mediaSvc.Upload(c, uid, file, bindType, bindIDStr)
    if err != nil { c.JSON(500, web.FAIL); return }

    c.JSON(200, web.OK(gin.M{
        "media_id": mediaID,
        "status":   "pending",
        "filename": file.Filename,
    }))
}
```

Response (SQLite TEXT 列):

```json
{
  "code": 0,
  "data": {
    "media_id": 1,
    "status": "pending",
    "filename": "原始名称.pptx"
  }
}
```

### 分片上传（视频 > 100MB）

```go
// POST /api/media/upload-init
// → { "upload_id": "uuid", "chunk_size": 5242880, "chunk_count": 100 }

// POST /api/media/upload-chunk   (循环调用，Content-Type: multipart/form-data)
// → { "ok": true }

// POST /api/media/upload-complete
// → { "media_id": 1, "status": "pending" }
```

### 前端分片上传流程（Axios + FormData）

```js
// 1. 初始化
const initResp = await axios.post('/api/media/upload-init', {
  filename: 'training.mp4',
  file_size: 524288000,
  bind_type: 'course',
  bind_id: 1,
})
const { upload_id, chunk_size, chunk_count } = initResp.data.data

// 2. 逐片上传（并发或串行）
for (let i = 0; i < chunk_count; i++) {
  const chunk = file.slice(i * chunk_size, (i + 1) * chunk_size)
  await axios.post('/api/media/upload-chunk', chunk, {
    params: { upload_id, chunk_index: i },
    headers: { 'Content-Type': 'multipart/form-data' },
  })
}

// 3. 完成合并
await axios.post('/api/media/upload-complete', null, {
  params: { upload_id },
})
```

---

## 5. 审批 API

### 管理员端

| 方法 | 路径 | 说明 |
|------|------|------|
| GET | `/api/media/audit-list?status=pending&page=1&size=20` | 审批列表 |
| POST | `/api/media/audit/{mediaId}` | 审批通过/驳回 |

POST 请求体：

```json
{
  "action": "approve",       // approve | reject
  "reject_reason": "..."     // 驳回时必填
}
```

响应：

```json
{ "code": 0, "data": { "media_id": 1, "status": "approved", "audit_by": 1 } }
```

### Go handler 实现

```go
func AuditMedia(c *gin.Context) {
    mediaID, _ := strconv.ParseUint(c.Param("mediaId"), 10, 32)
    var req struct {
        Action        string `json:"action" binding:"required"` // approve | reject
        RejectReason  string `json:"reject_reason"`
    }
    c.ShouldBindJSON(&req)

    uid := middleware.GetUserID(c)
    err := mediaSvc.Audit(c, uint(mediaID), uid, req.Action, req.RejectReason)
    if err != nil { c.JSON(500, web.FAIL); return }

    c.JSON(200, web.OK(gin.M{"media_id": mediaID, "status": req.Action}))
}
```

---

## 6. 异步转换管线

审批通过后，后台 goroutine 执行转换管线：

```go
// backend-go/internal/api/media.go
func approveMedia(tx *gorm.DB, mediaID uint, auditorID uint) error {
    media := &model.MediaFile{}
    tx.Model(media).Where("id = ?", mediaID).First(media)

    media.Status = "approved"
    media.AuditBy = &auditorID
    now := time.Now()
    media.AuditAt = &now
    tx.Save(media)

    go func() { _asyncConvertPipeline(tx, mediaID) }()
    return nil
}

func _asyncConvertPipeline(tx *gorm.DB, mediaID uint) {
    media := &model.MediaFile{}
    tx.Model(media).Where("id = ?", mediaID).First(media)

    // 仅文档需要转换（PPT/Word/PDF）
    switch media.FileExt {
    case "ppt", "pptx", "doc", "docx":
        // Step 1: LibreOffice 转 PDF
        pdfPath := _libreofficeToPDF(media.StoredPath)
        // Step 2: pdftotext 提取文本
        text := _pdftotextExtract(pdfPath)
        // Step 3: 按段落切片写入 knowledge_chunk
        chunks := _splitIntoChunks(text)
        for i, chunkText := range chunks {
            tx.Create(&model.KnowledgeChunk{
                MediaFileID: media.ID,
                SourceType:  media.FileExt,
                ChunkIndex:  i,
                Content:     chunkText,
            })
        }
    case "pdf":
        text := _pdftotextExtract(media.StoredPath)
        chunks := _splitIntoChunks(text)
        for i, chunkText := range chunks {
            tx.Create(&model.KnowledgeChunk{
                MediaFileID: media.ID,
                SourceType:  "pdf",
                ChunkIndex:  i,
                Content:     chunkText,
            })
        }
    }

    // 视频/图片：不提取文本
    media.Extracted = true
    tx.Save(media)
}
```

---

## 7. LibreOffice 转换接口（systemd 裸进程）

```go
func _libreofficeToPDF(inputPath string) string {
    // 调用本地安装的 LibreOffice（systemd 部署，非 Docker）
    outDir := filepath.Dir(inputPath)
    cmd := exec.Command("libreoffice",
        "--headless",
        "--convert-to", "pdf",
        "--outdir", outDir,
        inputPath,
    )
    cmd.Run()
    return strings.TrimSuffix(inputPath, filepath.Ext(inputPath)) + ".pdf"
}
```

---

## 8. pdftotext 文本提取

```go
func _pdftotextExtract(pdfPath string) string {
    // 使用系统 pdftotext（poppler-utils）提取 PDF 文本
    cmd := exec.Command("pdftotext", "-layout", pdfPath, "-")
    out, _ := cmd.Output()
    return string(out)
}

func _splitIntoChunks(text string, maxChars int) []string {
    paragraphs := strings.Split(text, "\n\n")
    var chunks []string
    current := ""
    for _, p := range paragraphs {
        if len(current)+len(p) > maxChars {
            if current != "" {
                chunks = append(chunks, strings.TrimSpace(current))
            }
            current = p
        } else {
            if current != "" {
                current += "\n\n" + p
            } else {
                current = p
            }
        }
    }
    if current != "" {
        chunks = append(chunks, strings.TrimSpace(current))
    }
    return chunks
}
```

---

## 9. 预览与状态 API

| 方法 | 路径 | 说明 |
|------|------|------|
| GET | `/api/media/preview/{mediaId}` | 仅 approved 素材可预览 |
| GET | `/api/media/status/{mediaId}` | 查询素材状态（含提取进度） |

GET `/api/media/preview/{mediaId}` 响应：

```json
{
  "code": 0,
  "data": {
    "preview_url": "/media/upload/uuid-filename.pdf",
    "file_ext": "pdf",
    "can_preview": true
  }
}
```

GET `/api/media/status/{mediaId}` 响应：

```json
{
  "code": 0,
  "data": {
    "status": "approved",
    "extracted": true,
    "chunk_count": 42
  }
}
```

---

## 10. 安全约束

- 扩展名白名单：`ppt/pptx/pdf/doc/docx/mp4/png/jpg/jpeg`
- 文件名重命名为 UUID，杜绝路径穿越
- 上传目录对静态预览只读，禁止直接执行
- 文件大小：文档 ≤ 200MB，视频 ≤ 2GB
- MIME 类型校验 + 扩展名双重校验
- SQLite 单文件，数据目录 `/opt/eai_agentplatform/data/media/`