Sakurasan
f9e9a1572f
- 系统配置键 log_raw_requests：开启后，仅管理员账号的每次请求 在用量明细中保存客户端原始请求体与上游原始响应体 （流式含全部 SSE 事件），用于排障 - UsageLog 新增 raw_request / raw_response 字段（type:text） - AuthLLM 附带 user_role 供网关判断管理员 - gateway：10s TTL 缓存开关；streamResponse/bufferResponse 支持累积上游原始响应；recordUsage 填充原始字段 - 前端 SystemConfig 新增开关（会显著增加存储的提示） - 新增 doc/flow.md 网关调用流程示意图
277 lines
11 KiB
Markdown
277 lines
11 KiB
Markdown
# 网关调用流程示意图
|
||
|
||
> 对应实现：`backend/router/setRouter.go`（路由注册）、`backend/middleware/auth_llm.go`（鉴权）、
|
||
> `backend/internal/proxy/{gateway.go,handlers.go,convert/*}`（网关与三协议互转）、
|
||
> `backend/internal/channel/{channel.go,health.go}`（渠道路由与健康）、
|
||
> `backend/internal/usage/recorder.go`（用量异步落库）。
|
||
|
||
## 0. 总览
|
||
|
||
```
|
||
┌────────────────────────────────────────────────┐
|
||
│ Gin Router (/v1) │
|
||
│ ┌──────────────────────────────────────────┐ │
|
||
客户端 ───────────▶│ │ middleware.AuthLLM (密钥鉴权, 401拦截) │ │
|
||
Bearer sk-ot-… │ └──────────────────────────────────────────┘ │
|
||
│ ┌───────┬────────┬─────────┬─────────┐ │
|
||
│ │ chat │messages│responses│ models │ │
|
||
│ │Handle │Handle │ Handle │ Handle │ │
|
||
│ │Chat │Messages│Responses│ Models │ │
|
||
│ └───┬───┴───┬────┴────┬────┴─────────┘ │
|
||
│ └───────┴────┬────┘ │
|
||
│ ParseRequest │
|
||
│ (model/stream) │
|
||
│ │ │
|
||
│ Dispatch ◀── Candidates/Pick │
|
||
│ (转换+故障转移+用量记录) │
|
||
└───────────────────┼──────────────────────────────┘
|
||
│
|
||
▼
|
||
上游 /v1/* (chat|messages|responses)
|
||
```
|
||
|
||
## 1. 请求入口与鉴权
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
autonumber
|
||
participant C as 客户端
|
||
participant R as Gin /v1 路由
|
||
participant A as AuthLLM
|
||
participant DB as SQLite(APIKey)
|
||
participant H as HandleXxx
|
||
|
||
C->>R: POST /v1/chat/completions 等
|
||
Note over R: /v1 组挂 middleware.AuthLLM
|
||
R->>A: 进入中间件
|
||
A->>A: 提取 Bearer token（兼容无前缀直传）
|
||
alt 未携带 Authorization
|
||
A-->>C: 401 「未提供认证信息」
|
||
else token 长度 < 12 或 prefix 不匹配
|
||
A-->>C: 401 「无效的API密钥」
|
||
else prefix 命中
|
||
A->>DB: SELECT * WHERE key_prefix=? AND status=active
|
||
A->>A: sha256(token) == KeyHash ?
|
||
alt 哈希不一致
|
||
A-->>C: 401 「无效的API密钥」
|
||
else 校验通过
|
||
A->>H: c.Set(api_key, user_id) → 放行
|
||
end
|
||
end
|
||
```
|
||
|
||
> 关键点：`key_prefix` 取 `sk-ot-` 后前 12 位（`api.go` 创建密钥时 `keyValue[:12]`），
|
||
> `auth_llm.go` 用同一常量 `keyPrefixLen=12` 切片，避免越界 panic。
|
||
|
||
## 2. 三协议主调用流程
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
autonumber
|
||
participant C as 客户端
|
||
participant H as HandleChat/Messages/Responses
|
||
participant P as ParseRequest
|
||
participant G as Dispatch
|
||
participant S as ChannelService
|
||
participant U as usage.Recorder
|
||
participant UP as 上游(OpenRouter等)
|
||
|
||
C->>H: 请求体 (model, stream, messages/input…)
|
||
H->>P: ParseRequest(protocol)
|
||
P->>P: 读 body → 解析 model / stream
|
||
P-->>H: Request{Model, Stream, Protocol, Body, …}
|
||
H->>G: Dispatch(req)
|
||
|
||
G->>S: Candidates(req.Model)
|
||
S-->>G: []Candidate{Channel, Binding?}
|
||
G->>S: FilterHealthy(cands)
|
||
G->>S: Pick(cands) → 加权随机选定一个候选
|
||
Note over G,S: 绑定优先（携带 upstream_model 映射）；<br/>无绑定回退到权重最低的健康备用渠道
|
||
|
||
loop 故障转移（候选耗尽前）
|
||
G->>G: conversionTarget(ch, proto) → 渠道首选协议
|
||
alt 客户端协议 ≠ 渠道协议
|
||
G->>G: ConvertRequest(body, from, to) 转换请求体
|
||
Note over G: 走 convert 包（chat/messages/responses 互转）
|
||
end
|
||
alt 有 Binding.UpstreamModel
|
||
G->>G: rewriteModel(body, upstreamModel) 别名映射
|
||
end
|
||
G->>UP: POST {base}/v1/{path} (按渠道协议拼 URL/头)
|
||
alt 连接失败 或 429/5xx
|
||
S->>S: RecordFailure(ch) → 连续2次 degraded 熔断
|
||
G->>U: Record(error 事件, error_code)
|
||
Note over G: continue → 换下一个候选渠道
|
||
else 4xx
|
||
G-->>C: 透传上游错误体 (不重试)
|
||
else 2xx
|
||
S->>S: RecordSuccess(ch)
|
||
alt stream=true
|
||
G->>G: streamResponse → 逐块转发 + 累计 usage
|
||
else
|
||
G->>G: bufferResponse → 整体转发 + 提取 usage
|
||
end
|
||
G->>G: recordUsage (按模型定价计算 cost)
|
||
G->>U: Record(成功事件, tokens, cost)
|
||
G-->>C: 响应
|
||
end
|
||
end
|
||
Note over G,C: 全部候选失败 → 502/503
|
||
```
|
||
|
||
> **故障转移规则**（对齐参考实现 `doProxy`）：
|
||
> - 连接错误、429、5xx → 可重试，换下一个候选；
|
||
> - 4xx（如 400 参数错误）→ 透传上游错误体，不重试；
|
||
> - 无可用渠道（全部不健康/无绑定且无备用）→ 502/503 + `error_code=no_channel`。
|
||
|
||
## 3. 路由选择细节
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
A[客户端 model 名] --> B{存在启用模型行?}
|
||
B -- 是 --> C{有绑定且渠道健康?}
|
||
C -- 是 --> D[候选 = 绑定该模型的渠道<br/>排序 priority ASC, weight DESC, id ASC]
|
||
C -- 否 --> E
|
||
B -- 否 --> E[候选 = 全部启用渠道<br/>取权重最低的健康备用渠道]
|
||
D --> F[FilterHealthy 内存熔断过滤]
|
||
E --> F
|
||
F --> G[Pick 加权随机选中一个]
|
||
G --> H[Dispatch 开始尝试]
|
||
H --> I{尝试成功?}
|
||
I -- 失败可重试 --> J[RecordFailure + 换下一个]
|
||
J --> H
|
||
I -- 成功 --> K[RecordSuccess + 响应 + 记账]
|
||
J -. 全部耗尽 .-> L[502/503]
|
||
```
|
||
|
||
## 4. 跨协议转换（client ↔ 渠道原生协议）
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
subgraph 客户端协议
|
||
CHAT[/"chat<br/>chat/completions"/]
|
||
MSG[/"messages<br/>(Anthropic)"/]
|
||
RESP[/"responses<br/>(OpenAI)"/]
|
||
end
|
||
subgraph 中间模型
|
||
MID["Chat 形状<br/>(标准中间模型)"]
|
||
end
|
||
subgraph 渠道协议
|
||
UCHAT[/"chat"/]
|
||
UMSG[/"messages"/]
|
||
URESP[/"responses"/]
|
||
end
|
||
|
||
CHAT -->|直通| UCHAT
|
||
MSG -->|messagesToChat| MID -->|chatToMessages| UMSG
|
||
MSG -->|messagesToChat| MID -->|chatToResponses| URESP
|
||
RESP -->|responsesToChat| MID -->|chatToResponses| URESP
|
||
RESP -->|responsesToChat| MID -->|chatToMessages| UMSG
|
||
```
|
||
|
||
> 转换入口：`convert.ConvertRequest`（请求体）、`convert.ConvertResponse`（非流式响应）、
|
||
> `convert.NewStreamTransformer`（流式 SSE 逐行转换）。跨两跳时经 Chat 中转（如
|
||
> `responses→messages` = `responsesToChatReq` + `chatToMessagesReq`）。
|
||
|
||
## 5. 流式 / 非流式响应与用量提取
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
autonumber
|
||
participant G as Dispatch
|
||
participant S as streamResponse
|
||
participant B as bufferResponse
|
||
participant ACC as StreamUsageAccum
|
||
participant W as 客户端 Writer
|
||
participant UP as 上游
|
||
|
||
alt stream=true
|
||
G->>S: streamResponse(resp, clientProto, upstreamProto)
|
||
S->>UP: 按 \n\n 读块 (bufio)
|
||
loop 每个 SSE 块
|
||
S->>ACC: sseDataPayloads(chunk) → Feed(data, upstreamProto)
|
||
Note over ACC: 逐协议累计 usage 字段
|
||
alt 跨协议
|
||
S->>S: NewStreamTransformer(upstream→client).line(chunk)
|
||
end
|
||
S->>W: 写块 + Flush
|
||
alt 遇到流结束标记
|
||
Note over S: chat: data:[DONE]<br/>messages: message_stop<br/>responses: response.completed
|
||
S-->>G: 返回累计 TokenUsage
|
||
end
|
||
end
|
||
else stream=false
|
||
G->>B: bufferResponse(resp, clientProto, upstreamProto)
|
||
B->>B: io.ReadAll
|
||
B->>B: ExtractUsageJSON(body, upstreamProto)
|
||
alt 跨协议
|
||
B->>B: ConvertResponse(body, upstream→client)
|
||
else 直通
|
||
B->>B: CleanJSON(body) 去空白/SSE注释前缀
|
||
end
|
||
B-->>G: 返回 TokenUsage
|
||
end
|
||
G->>G: recordUsage(req, cand, ch, ev, tok)
|
||
Note over G: 定价 cost = (非缓存输入×输入价 + 缓存读×缓存价<br/>+ 缓存写×输出价 + 输出×输出价) / 1e6
|
||
G->>G: usageRec.Record(Event)
|
||
```
|
||
|
||
## 6. 用量异步落库
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
autonumber
|
||
participant G as Gateway
|
||
participant R as usage.Recorder
|
||
participant U as UsageDAO
|
||
participant D as DailyUsageDAO
|
||
participant DB as SQLite
|
||
|
||
G->>R: Record(Event) 每次请求(成功/错误/取消)
|
||
Note over R: 缓冲 channel (10000), 每 5s 或满 100 条 flush
|
||
R->>U: BatchCreate(UsageLog[])
|
||
R->>D: UpsertDailyUsage(UsageDaily) 按(user_id,model_id,date)增量累加
|
||
U->>DB: INSERT usage_logs
|
||
D->>DB: ON CONFLICT 累加 requests/input/output/cache/cost
|
||
```
|
||
|
||
> `usage_dailies` 用 `gorm.Expr("requests + ?")` 增量累加而非覆盖，保证多次 flush 不互相清零。
|
||
|
||
## 7. 渠道健康与熔断
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
A[请求失败] --> B[RecordFailure: consecutive++]
|
||
B --> C{consecutive >= 2?}
|
||
C -- 是 --> D[status=degraded + 5min cooldown]
|
||
C -- 否 --> E[仅计数]
|
||
D --> F{后续请求 Candidates}
|
||
F --> G{FilterHealthy 该渠道}
|
||
G -- degraded/cooldown 未过期 --> H[排除, 走其他渠道/备用]
|
||
G -- healthy --> I[参与选择]
|
||
D -. 冷却过期 .-> J[复位 healthy]
|
||
J --> I
|
||
K[健康检查周期探测成功] --> L[RecordSuccess: 复位 healthy]
|
||
```
|
||
|
||
> 说明：`health.go` 的 `StartPeriodicCheck`（默认 5min）会探测各渠道 `/models`，
|
||
> 成功调 `RecordSuccess` 复位；失败调 `RecordFailure` 进入熔断计数。
|
||
|
||
## 8. 关键代码锚点
|
||
|
||
| 环节 | 位置 |
|
||
|---|---|
|
||
| /v1 路由注册 + AuthLLM | `router/setRouter.go:146` |
|
||
| 密钥鉴权 | `middleware/auth_llm.go` |
|
||
| 请求解析 | `proxy/gateway.go:104 ParseRequest` |
|
||
| 主调度 + 故障转移 | `proxy/gateway.go:157 Dispatch` |
|
||
| 流式转发 + 结束检测 | `proxy/gateway.go:388 streamResponse` |
|
||
| 非流式转发 | `proxy/gateway.go:496 bufferResponse` |
|
||
| 用量记账 | `proxy/gateway.go:302 recordUsage` |
|
||
| 候选构建 | `channel/channel.go:51 Candidates` |
|
||
| 内存健康过滤 | `channel/channel.go:252 FilterHealthy` |
|
||
| 加权选择 | `channel/channel.go:222 Pick` |
|
||
| 失败熔断 | `channel/channel.go:139 RecordFailure` |
|
||
| 三协议互转 | `proxy/convert/{convert.go,json_chat.go,json_responses.go,stream_transform.go}` |
|
||
| 用量异步落库 | `usage/recorder.go:114 flush` |
|