ericwyuan/sentinel-home-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: ericwyuan:ae1c13589f9ef42c78d887124a60c11deb2dffca
Branches Tags
ericwyuan:main
...
compare: ericwyuan:44026b65eab39f5ab25c2ea192b5ce88cc84ed89
Branches Tags
ericwyuan:main

2 Commits
ae1c13589f ... 44026b65ea

Author SHA1 Message Date
ericwyuan
44026b65ea [阶段3] FAM-Core 瘦身为管理后台+Oracle-Sync 同步引擎，FAM-UI 改读同步镜像 - 删除 scheduler/dispatcher/poller/event_receiver/video_server，新增 oracle_sync 每30分钟拉增量写 sync_* 镜像表；member_manager/chat_handler 改走同步数据；UI 移除帧图改为事件时间线；DDL 新增 sync_videos/events/people/cursor 2026-08-21 10:38:26 +08:00
ericwyuan
9b1cc8f93b [阶段2] FAM-Edge 重构为整视频分析+同步接口+人物服务 - 移除切片/抽帧/队列，新增 oracle_db/person_service/qa/watch_processor/video_processor，api_gateway 提供 /api/oracle/sync 与 /api/oracle/people/correct 2026-08-21 10:38:15 +08:00
45 changed files with 2456 additions and 5138 deletions
Expand all files Collapse all files

337
README.md
View File

@@ -13,13 +13,15 @@
### 1.1 首期范围（已基本完成） ### 1.1 首期范围（已基本完成）
- **FAM-Core**（NAS 端单进程）：Task-Scheduler / Dispatcher / Event-Receiver / Chat-Handler / Member-Manager / Video-Server 六个子模块 > **新架构 v2（2026-08-21 重构）**：NAS 不再处理视频，仅作管理后台；视频分析全部上云（Oracle）。
- **FAM-Edge**（Oracle 端单进程）：接收视频上传 → FFmpeg 快速抽帧 → OpenCV 关键帧筛选 → 云端 VLM 视觉分析（Gemini/NVIDIA NIM）直出结构化 JSON → Edge 仅做格式化/校验 → 结果同步返回 全链路（**本地模型不参与视频分析**）
- **FAM-UI**（NAS 端）：Streamlit 直读 DB，事件列表 + 成员命名页 + AI 对话页 + 对话历史 - **FAM-Core**（NAS 端单进程）：仅 **Oracle-Sync**（每 30 分钟拉增量镜像）+ **Chat-Handler** + **Member-Manager** 三个子模块，CPU 占用极低
- **数据库六张表**：process_tasks / monitor_events / event_details / chat_history / family_members / daily_summaries（预留） - **FAM-Edge**（Oracle 端单进程）：rclone 实时同步 Google 硬盘视频 → 监听目录 → **整视频直传云端 VLM**（Gemini 用 Files API / NVIDIA 用整视频 `video_url`，不切片不抽帧）→ 结构化 JSON 落本地 SQLite → 对外提供 `/api/oracle/sync` 增量拉取接口（**本地模型不参与视频分析**）
- **任务状态机**：PENDING → PROCESSING → SUCCESS/FAILED，含退避重试与僵尸任务回收 - **FAM-UI**（NAS 端）：Streamlit 读本地同步镜像（sync_videos / sync_events / sync_people），事件时间轴 + 人物管理 + AI 对话 + 对话历史 + 统计
- **AI 对话**：查 event_details 拼上下文 → 经 FAM-Edge 问答编排（Gemini → NVIDIA → 本地 Ollama 兜底）生成回答 → 返回并写 chat_history - **数据库**：Oracle 侧 SQLite（videos/events/people/sync_cursor）；NAS 侧 MariaDB 镜像（sync_videos / sync_events / sync_people / sync_cursor）+ chat_history
- **交互式成员命名**：VLM 按特征提取"人物A/B/C"落库，用户命名后批量回溯更新历史，后续分析直接用真名 - **数据流向**：Google 硬盘 ──rclone──► 甲骨文本地 ──整视频分析──► Oracle SQLite ──每 30 分钟 NAS 拉取──► NAS MariaDB 镜像 ──► FAM-UI
- **AI 对话**：查 sync_events 拼上下文 → 经 FAM-Edge 问答编排（Gemini → NVIDIA → 本地 Ollama 兜底）生成回答 → 返回并写 chat_history
- **人物命名**：用户命名/合并某 label → 回推 Oracle `/api/oracle/people/correct`（manual 优先）→ 下一周期同步回 NAS；Oracle 独立 person_service 汇总全量人物 → LLM 合并为规范名 → 回灌视频提示
### 1.2 不在首期范围（推迟 v1.1+） ### 1.2 不在首期范围（推迟 v1.1+）
@@ -70,59 +72,58 @@ Orchestrator 视觉阶段按 `fallback` 模式顺序降级：Gemini → NVIDIA N
- Oracle 端 Ollama 端口 11434 不对外暴露，聊天请求经 FAM-Edge `/api/edge/chat` 代理转发 - Oracle 端 Ollama 端口 11434 不对外暴露，聊天请求经 FAM-Edge `/api/edge/chat` 代理转发
- Tailscale 两节点已安装在线，但 NAS tailscaled 为 userspace 模式且防火墙端口不通，暂走公网 IP - Tailscale 两节点已安装在线，但 NAS tailscaled 为 userspace 模式且防火墙端口不通，暂走公网 IP
### 2.2 部署拓扑与数据流（推送模式） ### 2.2 部署拓扑与数据流（新架构 v2：Oracle 分析 + NAS 镜像）
``` ```
┌─────────────────────────────── NAS (192.168.50.64) ──────────────────────────────┐ ┌──────────── Google 硬盘 ────────────┐
│ │ │ oraclenas@...gserviceaccount.com │
│ Surveillance Station ──► /volume1/surveillance/Generic_ONVIF-001/ │ │ (Cloud Sync 落盘目录) │
│ （YYYYMMDDAM / YYYYMMDDPM 两级目录，~30min/350MB） │ └──────────────┬───────────────────────┘
│ │ │ │ rclone 定时同步（systemd timer）
│ ▼ │
│ FAM-Core (Flask :8000, gunicorn) │
│ ├─ Task-Scheduler: 60s 轮询视频目录，稳定文件建 PENDING 任务 │
│ ├─ Dispatcher: 30s 轮询，multipart 上传视频 ──────────┐ │
│ │ (push_timeout=1800s，同步等待响应) │ │
│ ├─ 僵尸回收: PROCESSING 超 push_timeout+120s 重置 PENDING │
│ ├─ Event-Receiver: /api/core/callback/event（兼容保留） │
│ ├─ Chat-Handler ── /api/edge/chat 代理 ──────────────┐ │
│ ├─ Member-Manager / Video-Server(/media, token) │ │
│ ▼ │ │
│ MariaDB (sentinel_home_ai, 6 张表) │ │
│ │ │
│ FAM-UI (Streamlit :8501) 直读 DB │ │
└─────────────────────────────────────────────────────────┼─────────────────────────┘
│ HTTP (公网)
▼ ▼
┌──────────────────────── Oracle Cloud (129.146.203.203) ──────────────────────────┐ ┌──────────────────────── Oracle Cloud (129.146.203.203) ────────────────────────┐
│ FAM-Edge (Flask :5000, gunicorn --timeout 1800, 单 worker) │ │ FAM-Edge (Flask :5000) │
│ ├─ POST /api/edge/video/push（multipart 视频，同步分析，结果随响应返回） │ │ ├─ Watch-Processor: 30s 轮询 /opt/fam-edge/gdrive_videos，新视频串行处理 │
│ ├─ AI-Orchestrator: 健康检查 → 抽帧 → 选帧 → 压缩 → 云端VLM视觉直出结构化JSON → 格式化校验(无本地融合) │ │ ├─ Video-Processor: 整视频直传云端 VLM（不切片不抽帧） │
│ ├─ POST /api/edge/chat/ask（问答编排: Gemini→NVIDIA→本地Ollama 兜底） │ │ │ Gemini(Files API) → 失败 NVIDIA(整视频 video_url) → 再失败 FAILED │
│ ├─ Person-Service: 汇总人物 → LLM 合并规范名 → 回灌视频提示 │
│ ├─ OracleDB (SQLite): videos / events / people / sync_cursor │
│ └─ API: /api/oracle/sync (增量拉取) · /api/oracle/people/correct (命名校正) · │
│ /api/edge/chat/ask (问答编排 Gemini→NVIDIA→Ollama) │
└───────────────────────────────┬───────────────────────────────────────────────┘
│ HTTP GET /api/oracle/sync?since=&token= （每 30 分钟）
▼
┌─────────────────────────────── NAS (192.168.50.64) ────────────────────────────┐
│ FAM-Core (Flask :8000) │
│ ├─ Oracle-Sync: 唯一后台线程，拉增量写 MariaDB 镜像 + 维护 sync_cursor │
│ ├─ Chat-Handler: /api/chat/ask（查 sync_events 拼上下文 → 走 Oracle 编排） │
│ ├─ Member-Manager: /api/member/name|merge（回推 Oracle + 即时拉回） │
│ ▼ │ │ ▼ │
│ 云端: Gemini / NVIDIA NIM (视觉直出结构化 + 问答) 本地: Ollama :11434 (qwen2.5:7b, 仅问答兜底) │ │ MariaDB (sentinel_home_ai): sync_videos / sync_events / sync_people / │
└───────────────────────────────────────────────────────────────────────────────────┘ │ sync_cursor / chat_history │
│ FAM-UI (Streamlit :8501) 读同步镜像 │
└─────────────────────────────────────────────────────────────────────────────────┘
``` ```
### 2.3 主链路时序（推送模式） **网络要点（新架构）**：
- NAS → Oracle 仅一条出站 HTTPS/HTTP：`GET /api/oracle/sync`（拉取）与 `POST /api/oracle/people/correct`（命名回推），均走 Oracle 公网 IP:5000，token 鉴权
- Oracle Ollama :11434 不对外暴露，问答经 FAM-Edge `/api/edge/chat/ask` 代理
- Tailscale 两节点在线但 NAS 无法反向访问 Oracle，故全部走 NAS 主动出站拉取模式
1. Scheduler 扫描到新视频（修改时间 > 60s 且大小稳定）→ 写 `process_tasks`（PENDING） ### 2.3 主链路时序（新架构 v2）
2. Dispatcher 领取 PENDING 任务 → 状态置 PROCESSING → 读本地视频文件，multipart POST 到 Edge `/api/edge/video/push`，payload 含 task_id / camera_name / event_start_time（文件 mtime）/ known_members_context
3. Edge 同步执行： 1. Google 硬盘新视频 → rclone 定时同步到 Oracle `/opt/fam-edge/gdrive_videos`
- a. 保存上传视频到临时目录（超时 60s） 2. Watch-Processor 轮询发现新文件 → 登记到 Oracle `videos`（pending）
- b. FFmpeg 快速 seek（`-ss <ts> -frames:v 1`）粗抽候选帧，帧数随视频时长自适应 3. Video-Processor 串行处理：整视频上传 Gemini Files API（或 NVIDIA 整视频 `video_url`）→ 模型直出 `{global_summary, events[], people_mentioned[]}` → 写 Oracle `videos` + `events` + `people`
- c. OpenCV MSE 帧差分析筛选关键帧 → 压缩（长边 ≤ 1024px，JPEG 质量 80） 4. Person-Service 每 30 分钟汇总全量人物 → LLM 合并为规范名 → 更新 `people.canonical_name` → 生成 `known_members_context` 回灌后续视频提示
- d. 云端视觉模型按 `orchestrator.mode`（fallback）顺序降级：Gemini（timeout 30s）→ NVIDIA NIM（timeout 20s）；首个**直出结构化 JSON** 成功的模型即采用，两云端全失败 → 任务 FAILED 走重试（绝不回退本地 Ollama，本地模型不参与视频分析） 5. NAS Oracle-Sync 每 30 分钟 `GET /api/oracle/sync?since=<cursor>` → upsert 到本地 `sync_*` 镜像表 → 推进 `sync_cursor`
- e. `format_cloud_result` 格式化校验（无模型调用）：字段归一化、补 `source_providers=[provider]` / `compute_provider=[provider]`、缺失 `entities_json` 由 `frame_details` 推导、缺失 `global_summary` 时事实拼接 → 合法入库 schema 6. FAM-UI 读本地镜像展示；用户命名 → `POST /api/oracle/people/correct` 回推 Oracle，下一周期同步生效
- f. `event_end_time` = event_start_time + 视频时长（Edge 推算）
- g. `finally` 清理临时文件
4. Edge 把结果 JSON 直接作为 HTTP 响应返回（无 webhook）
5. Dispatcher 收到响应 → 调用 `apply_success_event()` 写 `monitor_events`（1 条聚合）+ `event_details`（每关键帧 1 条）+ upsert 未命名成员 → 任务置 SUCCESS；失败则退避重试（`min(60×(retry+1)×2, 600)`s，超 3 次 FAILED）
**容错设计**： **容错设计**：
- Dispatcher 僵尸回收：PROCESSING 状态超过 `push_timeout + 120s` 自动重置 PENDING（应对进程重启/Edge 重启导致 in-flight 请求丢失） - Oracle 单视频串行（`max_concurrent=1`）避免多视频抢占云端配额
- fam-core 日志双写：stdout + `fam-core/logs/fam-core.log`（daemon 模式下 stdout 不可见） - 视频分析失败（两云端均不可用）标记 `failed`，下一周期 cursor 仍包含它会被重试
- 所有日志带 `task_id` 作为 trace_id，各阶段耗时打 INFO - NAS 同步失败仅记日志，下一周期（30 分钟）自动重试，不阻塞 UI
- fam-core 日志双写：stdout + `fam-core/logs/fam-core.log`
--- ---
@@ -130,40 +131,42 @@ Orchestrator 视觉阶段按 `fallback` 模式顺序降级：Gemini → NVIDIA N
### 3.1 FAM-Core（NAS 端） ### 3.1 FAM-Core（NAS 端）
> NAS 不再处理视频，仅作管理后台。唯一常驻后台线程是 Oracle-Sync。
| 模块 | 文件 | 职责 | | 模块 | 文件 | 职责 |
|------|------|------| |------|------|------|
| Task-Scheduler | `scheduler/scheduler.py` | 60s 轮询视频目录（`os.walk` 递归，支持 AM/PM 子目录），`video_path` 去重，稳定文件建 PENDING 任务 | | Oracle-Sync | `oracle_sync/oracle_sync.py` | 唯一后台线程：每 30 分钟 `GET /api/oracle/sync?since=<cursor>&token=` 拉增量 → upsert 到 `sync_videos`/`sync_events`/`sync_people` → 推进 `sync_cursor`；`push_name_correct()` 回推命名校正；`trigger_now()` 立即同步 |
| Dispatcher | `dispatcher/dispatcher.py` | 30s 轮询 PENDING；multipart 上传视频至 Edge push 端点；收响应后经 `apply_success_event` 落库；僵尸 PROCESSING 回收；退避重试 | | Chat-Handler | `chat_handler/chat_handler.py` | `/api/chat/ask` 查 `sync_events` 拼上下文 → 经 Oracle `/api/edge/chat/ask` 问答编排（Gemini→NVIDIA→本地 Ollama 兜底）→ 写 chat_history |
| Event-Receiver | `event_receiver/event_receiver.py` | `/api/core/callback/event`（webhook 兼容保留）；核心逻辑抽为 `apply_success_event(data)` 供 Dispatcher 推送模式复用；未命名 abstract_label 自动 upsert `family_members` | | Member-Manager | `member_manager/member_manager.py` | `/api/member/unnamed` / `/api/member/list` / `/api/member/name` / `/api/member/merge`；命名/合并回推 Oracle 并即时拉回本地镜像 |
| Chat-Handler | `chat_handler/chat_handler.py` | `/api/chat/ask` 查 event_details 拼上下文 → 经 Edge `/api/edge/chat/ask` 问答编排（Gemini→NVIDIA→本地 Ollama 兜底）→ 写 chat_history；明细 > 50 条按小时聚合 | | 公共层 | `db_layer.py` / `config_loader.py` / `logger.py` | PyMySQL 连接（unix_socket）；同步镜像 CRUD；文件日志 |
| Member-Manager | `member_manager/member_manager.py` | `/api/member/unnamed` / `/api/member/name` / `/api/member/list`；命名后批量回溯 UPDATE event_details（MariaDB 不支持 `$[*]` JSON 路径，Python 层逐行更新） |
| Video-Server | `video_server/video_server.py` | `/media/<path>?token=xxx` 静态视频服务（推送模式下主链路不再使用，保留备用） | > 已删除：Task-Scheduler / Dispatcher / Poller / Event-Receiver / Video-Server（视频上传、切片、抽帧、关键帧落盘等职责全部迁移至 Oracle 端，NAS CPU 占用大幅降低）。
| 公共层 | `db_layer.py` / `config_loader.py` / `logger.py` | PyMySQL 连接（unix_socket）；datetime 空串归一化 NULL + NOT NULL 列兜底；文件日志 |
### 3.2 FAM-Edge（Oracle 端） ### 3.2 FAM-Edge（Oracle 端）
> 整视频分析，不切片、不抽帧、不依赖 OpenCV 人脸。
| 模块 | 文件 | 职责 | | 模块 | 文件 | 职责 |
|------|------|------| |------|------|------|
| API-Gateway | `api_gateway/api_gateway.py` | `POST /api/edge/video/push`（multipart 上传 + 同步分析 + 结果返回）；`POST /api/edge/video/analyze`（旧拉取模式，兼容保留）；`POST /api/edge/chat`（Ollama 代理）；`GET /health`；单并发控制（处理中返回 429） | | API-Gateway | `api_gateway/api_gateway.py` | `GET /api/oracle/sync`（增量拉取，since+token 校验）；`POST /api/oracle/people/correct`（命名校正）；`POST /api/edge/chat/ask`（问答编排）；`GET /health` |
| Video-Preprocessor | `video_preprocessor/preprocessor.py` | `save_upload` 保存上传视频；FFmpeg 快速 seek 粗抽候选帧（帧数自适应）；OpenCV MSE 帧差筛选关键帧（首末帧必选）；压缩；`compute_timestamps` 用 start+偏移算绝对时间戳；`video_duration` 供 event_end_time 推算 | | Watch-Processor | `watch_processor.py` | 30s 轮询 rclone 同步落地目录，登记新视频，串行触发 Video-Processor |
| AI-Orchestrator | `ai_orchestrator/orchestrator.py` | 模型健康检查 → 云端视觉适配器按 `orchestrator.mode`（fallback 顺序降级）调度，**直出结构化 JSON** → `format_cloud_result` 格式化校验（无本地融合）→ JSON schema 校验；`run_qa` 实现问答编排（Gemini→NVIDIA→本地 Ollama 兜底）；`process_push_task` 为推送模式入口（不触发 webhook）；记录各模型实际执行耗时与成功状态到 `compute_provider` 数组 | | Video-Processor | `video_processor.py` | 按 `vision_order` 调适配器 `analyze_video`（整视频）；首个成功即落库 Oracle `videos`+`events`+`people`；全失败标 `failed` |
| Model-Adapters | `model_adapters/` | `BaseModelAdapter` 抽象基类（`__init__` / `health_check` / `analyze_frames` / `chat` / `get_timeout` / 熔断器实例）；`build_adapter` 工厂函数按 `provider` 字段分发实例化；视觉适配器（gemini/nvidia）直出结构化 JSON，文本适配器（ollama）仅智能问答兜底 | | Person-Service | `person_service.py` | 汇总全量人物 → LLM 合并为规范名 → `set_canonical`；生成 `known_members_context` 回灌视频提示；manual 命名优先不被覆盖 |
| └ OllamaAdapter | `model_adapters/ollama_adapter.py` | requests 直调本地 REST `/api/generate`，`num_predict` 可配；**role: text, usage: qa_fallback**（仅智能问答兜底，不参与视觉分析、不参与融合） | | OracleDB | `oracle_db.py` | SQLite：videos / events / people / sync_cursor；`get_sync_delta(since)` 增量导出 |
| └ GeminiAdapter | `model_adapters/gemini_adapter.py` | requests 直调 Google REST `:generateContent`，**多图单请求直出结构化 JSON**；**role: vision**；`chat()` 参与问答 | | Model-Adapters | `model_adapters/` | `BaseModelAdapter.analyze_video(video_path, known_members_context, event_start_time)`；Gemini（Files API 整视频）/ NVIDIA（整视频 `video_url`，`num_frames=128`）/ Ollama（纯文本，不参与视频） |
| └ NvidiaVisionAdapter | `model_adapters/nvidia_adapter.py` | **基于 openai SDK**（NIM 兼容 OpenAI API 规范），`base_url=https://integrate.api.nvidia.com/v1`，`api_key` 从 `${NVIDIA_API_KEY}` 展开；**逐帧返回结构化单帧 JSON 并聚合为 frame_details**（NIM 限 1 图/请求）；`health_check` 调 `client.models.list()`；**role: vision**；`chat()` 参与问答 | | QA-Orchestrator | `qa.py` | 遍历所有适配器 `chat()`，Gemini→NVIDIA→Ollama 三级降级（仅问答） |
| Storage-Cleaner | `storage_cleaner/` | `finally` 删除临时视频与帧图片 |
### 3.3 FAM-UI（NAS 端） ### 3.3 FAM-UI（NAS 端）
Streamlit 应用（`fam-ui/src/app.py`），侧边栏切换页面： Streamlit 应用（`fam-ui/src/app.py`），侧边栏切换页面（均读本地同步镜像）：
| 页面 | 功能 | | 页面 | 功能 |
|------|------| |------|------|
| 📊 事件列表 | 按日期筛选 + 分页（20 条/页）展示 monitor_events | | 🕒 事件时间轴 | 视频会话列表（按处理后时间倒序）+ 选中会话的事件时间线（时间点 + 描述 + 人物/关注徽章，无帧图） |
| 👤 成员命名 | 列出未命名人物 + 特征描述，输入真名后调 `/api/member/name` 批量回溯 |
| 💬 AI 对话 | 输入框 + 调 `/api/chat/ask`；按 queried_person 预设快捷提问 | | 💬 AI 对话 | 输入框 + 调 `/api/chat/ask`；按 queried_person 预设快捷提问 |
| 📜 对话历史 | chat_history 倒序展示 | | 📝 对话历史 | chat_history 倒序展示 |
| 📈 统计图表 | compute_provider 占比（bar_chart） | | 👤 人物管理 | 按规范名/标签聚合，命名/合并（回推 Oracle）；不再展示帧照片 |
| 📈 统计图表 | 模型来源占比 / 关注事件 / 同步状态 |
--- ---
@@ -173,36 +176,46 @@ Streamlit 应用（`fam-ui/src/app.py`），侧边栏切换页面：
### 4.1 表清单 ### 4.1 表清单
> 新架构 v2：Oracle 侧用 SQLite（`videos`/`events`/`people`/`sync_cursor`），NAS 侧 MariaDB 仅保留 **同步镜像表 + 问答历史**。`process_tasks`/`monitor_events`/`event_details`/`family_members` 等旧表已不再写入（保留历史数据，未删除）。
**Oracle（SQLite，`oracle_db.py`）**
| 表 | 用途 | 关键字段 | | 表 | 用途 | 关键字段 |
|----|------|---------| |----|------|---------|
| `process_tasks` | 视频处理任务 | task_id, video_path, video_url, status(PENDING/PROCESSING/SUCCESS/FAILED), retry_count, max_retries, next_retry_at, error_message, failure_stage(ENUM) | | `videos` | 视频会话（每视频 1 行） | id, filename(UNIQUE), camera_name, status, summary_json, events_json, people_json, compute_provider, event_start_time, updated_at |
| `monitor_events` | 事件聚合（每任务 1 条） | event_id, task_id, event_start_time, event_end_time, camera_name, global_summary, entities_json(JSON), compute_provider(JSON 数组) | | `events` | 视频内时间点事件 | id, video_id, ts, description, person_list_json, is_attention_event |
| `event_details` | 每关键帧一条明细 | detail_id, event_id, task_id, frame_index, frame_timestamp, person, action, clothing, is_attention_event, source_providers(JSON) | | `people` | 规范人物（Oracle 维护） | id, label(UNIQUE), canonical_name, appearances, source(llm/manual) |
| `family_members` | 交互式命名 | member_id, abstract_label(如"人物A"), real_name(NULL=未命名), feature_description, first_seen_at, named_at, named_by | | `sync_cursor` | 同步游标 | key, value（上次 server_time） |
**NAS（MariaDB，同步镜像，`scripts/ddl.sql`）**
| 表 | 用途 | 关键字段 |
|----|------|---------|
| `sync_videos` | 视频会话镜像（对齐 Oracle videos） | id, filename, camera_name, status, summary_json, events_json, people_json, compute_provider, processed_at |
| `sync_events` | 事件镜像（对齐 Oracle events） | id, video_id, ts, description, person_list_json, is_attention_event |
| `sync_people` | 人物镜像（对齐 Oracle people） | id, label, canonical_name, appearances, source |
| `sync_cursor` | 同步游标 | key='last_since', value |
| `chat_history` | AI 问答记录 | chat_id, user_question, ai_answer, context_summary, queried_date, queried_person | | `chat_history` | AI 问答记录 | chat_id, user_question, ai_answer, context_summary, queried_date, queried_person |
| `daily_summaries` | 每日摘要（预留） | target_date, summary_text |
### 4.2 表关系与命名回溯 ### 4.2 表关系
``` ```
process_tasks (1) ─── (N) monitor_events (1) ─── (N) event_details Oracle: videos (1) ─── (N) events people 独立（label/canonical_name）
family_members 独立表: NAS 镜像: sync_videos (1) ─── (N) sync_events sync_people 独立
- event_details.person 存 abstract_label（未命名）或 real_name（命名后） chat_history 独立表（问答上下文摘要留存）
- 命名后: UPDATE event_details SET person = real_name WHERE person = abstract_label
- monitor_events.entities_json 由 Python 层解析逐行更新（MariaDB 不支持 $[*] 路径）
chat_history 独立表
``` ```
### 4.3 compute_provider / source_providers 命名回溯：用户命名某 `label` → `POST /api/oracle/people/correct` 设 `canonical_name`（manual 优先）→ 下一周期同步回 NAS `sync_people`；Oracle `person_service` 用规范名回灌视频提示，后续事件 `person_list_json` 直接带真名。
- `monitor_events.compute_provider`：JSON 数组，记录本次任务实际成功调用（**视觉分析**）的云端模型，如 `["gemini"]` 或 `["nvidia"]`；本地 Ollama 不参与视频分析，不会出现在该字段 ### 4.3 compute_provider
- `event_details.source_providers`：该条明细被哪些模型识别到（可能少于 compute_provider）
- 多模型交叉验证：多模型一致 → 可信度高；仅单一模型描述 → source_providers 仅含该模型；冲突 → 多数派为准 - `sync_videos.compute_provider`：字符串，记录该视频实际成功调用的视觉模型（`gemini` / `nvidia`）；本地 Ollama 不参与视频分析，不会出现在该字段
- 问答链路（Gemini→NVIDIA→Ollama 兜底）的 provider 体现在 `/api/edge/chat/ask` 响应的 `provider` 字段
### 4.4 兼容性注意 ### 4.4 兼容性注意
- MariaDB 10.11 严格模式：**空字符串不能插 DATETIME 列**（1292 错误）。`db_layer._dt_or_none` 将空串归一化 NULL；`event_end_time` NOT NULL 列按 end→start→NOW 兜底；`frame_timestamp` 空值兜底 NOW - MariaDB 10.11 严格模式：**空字符串不能插 DATETIME 列**（1292 错误）。同步表时间字段统一用 `VARCHAR(32)` 文本存储 Oracle 的 ISO 字符串，规避类型转换问题
- MariaDB 不支持 MySQL 的 `$[*]` JSON 通配路径与 `->` 操作符，JSON 字段在 Python 层处理 - MariaDB 不支持 MySQL 的 `$[*]` JSON 通配路径，人物统计按 `person_list_json LIKE '%name%'` 字符串匹配在 Python 层完成
--- ---
@@ -210,59 +223,70 @@ chat_history 独立表
### 5.1 FAM-Edge（Oracle :5000） ### 5.1 FAM-Edge（Oracle :5000）
**POST /api/edge/video/push**（主链路，推送模式） **GET /api/oracle/sync**（NAS 每 30 分钟拉增量，新架构主接口）
- 请求：`multipart/form-data`，字段 `video`(文件) / `task_id` / `camera_name` / `event_start_time` / `known_members_context` - 请求：`?since=<ISO 文本>&token=<ORACLE_SYNC_TOKEN>`（`since` 为空拉全量）
- 处理：同步执行完整分析流水线（可能耗时数分钟，gunicorn timeout 1800）
- 响应（200）： - 响应（200）：
```json ```json
{ {
"task_id": 289, "videos": [
"status": "success", {"id": 1, "filename": "2026-08-21_081500.mp4", "camera_name": "客厅",
"event_start_time": "2026-08-20 01:06:44", "status": "done", "summary_json": "...", "events_json": "[...]",
"event_end_time": "2026-08-20 01:07:13", "people_json": "[...]", "compute_provider": "gemini",
"camera_name": "客厅", "event_start_time": "2026-08-21 08:15:00", "updated_at": "2026-08-21 08:40:12"}
"global_summary": "...",
"entities_json": [{"person": "汤圆", "action": "...", "clothing": "..."}],
"frame_details": [
{"frame_index": 1, "frame_timestamp": "...", "person": "...", "action": "...",
"clothing": "...", "is_attention_event": false, "source_providers": ["gemini"]}
], ],
"compute_provider": ["gemini"] "events": [
{"id": 10, "video_id": 1, "ts": "00:01:23", "description": "汤圆在客厅玩耍",
"person_list_json": "[\"汤圆\"]", "is_attention_event": 0, "updated_at": "2026-08-21 08:40:12"}
],
"people": [
{"id": 1, "label": "人物A", "canonical_name": "汤圆", "source": "manual",
"appearances": 12, "updated_at": "2026-08-21 08:41:00"}
],
"server_time": "2026-08-21 08:41:30"
} }
``` ```
- 失败：`{"task_id": ..., "status": "failed", "failure_stage": "vlm_visual", "error_message": "..."}` - 401：token 校验失败
- 429：已有任务处理中（单并发）；503：全部模型不健康
**POST /api/edge/video/analyze** — 旧拉取模式（Edge 拉 video_url + webhook 回调），兼容保留，主链路不再使用 **POST /api/oracle/people/correct** — 命名校正回推：`{"label":"人物A","canonical_name":"汤圆","token":...}`（manual 优先，不被 LLM 覆盖）→ `{"status":"ok"}`
**POST /api/edge/chat/ask** — 智能问答编排（FAM-Core Chat-Handler 调用）：请求 `{"prompt"}` → 响应 `{"answer","provider"}`；内部按 Gemini → NVIDIA → 本地 Ollama 顺序，仅两云端都失败才用本地兜底 **POST /api/edge/chat/ask** — 智能问答编排（FAM-Core Chat-Handler 调用）：请求 `{"prompt","max_tokens"}` → 响应 `{"answer","provider"}`；内部按 Gemini → NVIDIA → 本地 Ollama 顺序，仅两云端都失败才用本地兜底
**POST /api/edge/chat** — Ollama 直连代理（兼容旧调用，保留） **GET /health** — 服务状态（含已处理视频数）
**GET /health** — 服务与模型健康状态（任务处理中可能无响应，单 worker 忙）
### 5.2 FAM-Core（NAS :8000） ### 5.2 FAM-Core（NAS :8000）
| 端点 | 方法 | 说明 | | 端点 | 方法 | 说明 |
|------|------|------| |------|------|------|
| `/health` | GET | 服务健康 | | `/health` | GET | 服务健康 |
| `/api/status` | GET | scheduler/dispatcher 运行状态 | | `/api/status` | GET | Oracle-Sync 同步状态（running / last_sync_at / last_error / cursor / last_count） |
| `/api/core/callback/event` | POST | Edge 回调（webhook 兼容保留）；推送模式下由 Dispatcher 内部调用 `apply_success_event` | | `/api/chat/ask` | POST | 用户问答：`{"question","queried_person","queried_date"}` → `{"answer","context_summary","chat_id"}`（上下文来自 sync_events） |
| `/api/chat/ask` | POST | 用户问答：`{"question","queried_person","queried_date"}` → `{"answer","context_summary","chat_id"}` |
| `/api/chat/history` | GET | 对话历史（`?date=` 或 `?person=&limit=`） | | `/api/chat/history` | GET | 对话历史（`?date=` 或 `?person=&limit=`） |
| `/api/member/unnamed` | GET | 未命名人物列表（含特征描述、出现次数） | | `/api/member/unnamed` | GET | 未命名人物列表（label / 出现次数 / 首见时间） |
| `/api/member/name` | POST | 命名：`{"abstract_label","real_name","named_by"}` → 批量回溯 event_details/entities_json，返回更新条数 | | `/api/member/list` | GET | 全部人物（label + canonical_name + 是否命名） |
| `/api/member/list` | GET | 全部成员 | | `/api/member/name` | POST | 命名：`{"label","canonical_name"}` → 回推 Oracle 并即时拉回本地镜像 |
| `/media/<path>?token=xxx` | GET | 视频静态服务（token 鉴权，推送模式下备用） | | `/api/member/merge` | POST | 合并：`{"source","target"}` → 将 source 并入 target 身份（统一 canonical_name） |
> 已删除：`/api/core/callback/event`、`/media/<path>`（视频处理职责已迁移至 Oracle）。
### 5.3 云端结构化输出 JSON Schema ### 5.3 云端结构化输出 JSON Schema
云端 VLM 直接产出结构化 JSON，经两道处理入库： 整视频直传云端 VLM，模型直接产出结构化 JSON（`analyze_video` 返回）：
1. **适配器内三层容错解析**（`json_parser.parse_vlm_json`）：直接 `json.loads` → 提取 markdown fence ` ```json ... ``` ` → 贪婪匹配最大 `{...}`；失败抛 `VLMOutputInvalidError` ```json
2. **`format_cloud_result` 归一化/校验**（无模型调用）：`frame_details` 必须为非空列表并做字段类型归一化；`source_providers` 缺失时补为 `[provider]`；`compute_provider` 置为本次成功 provider；`entities_json` 缺失时由 `frame_details` 按人物去重推导；`global_summary` 缺失时格式化拼接生成 {
"global_summary": "客厅监控摘要……",
"events": [
{"timestamp": "00:01:23", "description": "汤圆在客厅玩耍",
"people": ["汤圆"], "is_attention_event": false}
],
"people_mentioned": ["汤圆"]
}
```
任一步骤失败 → 任务 FAILED 走重试。`action` 由 AI 自由生成无枚举过滤，`is_attention_event` 由 AI 自行判断。 - Gemini 用 Files API 上传整视频后 `generateContent`；NVIDIA 用整视频 `video_url` + `num_frames=128`（模型内部自行采样帧），均不切片、不抽帧、不依赖 OpenCV
- `person_service` 汇总全量 `people_mentioned` → LLM 合并为规范名 → 生成 `known_members_context` 回灌后续视频提示，使模型用真名指代
- `action` / 描述由 AI 自由生成无枚举过滤，`is_attention_event` 由 AI 自行判断
--- ---
@@ -436,51 +460,62 @@ task_id=289（30s 测试片段）全链路打通：推送 5.7MB → Edge 分析
### 8.3 配置文件要点 ### 8.3 配置文件要点
**fam-core/config/config.yaml**（NAS，生产值）： **fam-core/config/config.yaml**（NAS，新架构 v2 —— 仅同步 + 问答）：
```yaml ```yaml
scheduler: server:
video_dir: "/volume1/surveillance/Generic_ONVIF-001" # 生产目录（YYYYMMDDAM/PM 两级子目录） port: 8000
# 285 个历史视频由占位 FAILED 任务占用路径，scheduler dedup 自动跳过（forward-only 模式） database: # MariaDB（unix_socket 优先）
dispatcher: unix_socket: "/run/mysqld/mysqld10.sock"
edge_url: "http://129.146.203.203:5000/api/edge/video/push" oracle_sync: # 唯一后台线程配置
push_timeout: 1800 base_url: "http://129.146.203.203:5000"
token: "${ORACLE_SYNC_TOKEN}" # 与 Oracle 端 sync_api.token 一致
interval_sec: 1800 # 每 30 分钟拉一次增量
timeout: 120
chat_handler: chat_handler:
qa_url: "http://129.146.203.203:5000/api/edge/chat/ask" # 问答统一走 Edge 编排（Gemini→NVIDIA→Ollama） qa_url: "http://129.146.203.203:5000/api/edge/chat/ask" # 问答统一走 Oracle 编排
timeout: 120 timeout: 120
``` ```
**fam-edge/config/config.yaml**（Oracle，多模型池配置，新架构）： **fam-edge/config/config.yaml**（Oracle，整视频分析 + 同步 + 人物服务）：
```yaml ```yaml
# 编排调度模式: fallback(顺序降级, 默认) | ensemble(并行交叉验证) server:
orchestrator: port: 5000
mode: "fallback" gdrive_sync: # rclone 同步落地目录监听
overall_timeout: 600 enabled: true
local_dir: "/opt/fam-edge/gdrive_videos"
# 多模型池配置（新框架：本地大模型不参与视频分析，仅智能问答兜底） watch_interval_sec: 30
# 视频分析链路: 云端 VLM 直出结构化 JSON → Edge format_cloud_result 格式化/校验 → 直存 NAS DB（无本地融合） camera_name: "客厅"
# 智能问答链路: Gemini → NVIDIA → 本地 Ollama（仅两云端都失败才启用本地兜底） parse_start_from_filename: true
oracle_db:
path: "/opt/fam-edge/data/oracle.db"
sync_api:
token: "${ORACLE_SYNC_TOKEN}" # NAS 拉取鉴权（与 NAS oracle_sync.token 一致）
person_service:
schedule_interval_sec: 1800 # 每 30 分钟重新汇总人物
model: "gemini"
video_processing:
max_concurrent: 1 # 单视频串行，避免抢占云端配额
timeout: 900
vision_order: ["gemini", "nvidia"]
models: models:
# 1. Google Gemini（视觉主 + 参与问答）
- provider: "gemini" - provider: "gemini"
role: "vision" # 视觉分析 + 问答（vision role 也参与 chat） role: "vision"
enabled: true model_name: "gemini-flash-latest"
model_name: "gemini-flash-latest" # v1beta 下 gemini-1.5-flash 会 404
api_key: "${GEMINI_API_KEY}" api_key: "${GEMINI_API_KEY}"
timeout: 30 timeout: 600
circuit_breaker:
enabled: true
threshold: 3
cooldown: 600
# 2. NVIDIA NIM 托管 API（视觉备 + 参与问答）
- provider: "nvidia" - provider: "nvidia"
role: "vision" # 视觉分析 + 问答（vision role 也参与 chat） role: "vision"
enabled: true model_name: "nvidia/nemotron-nano-12b-v2-vl" # 整视频 video_url 输入（内部采样帧）
model_name: "meta/llama-3.2-11b-vision-instruct" # 或 qwen/qwen2-vl-72b-instruct
base_url: "https://integrate.api.nvidia.com/v1" base_url: "https://integrate.api.nvidia.com/v1"
api_key: "${NVIDIA_API_KEY}" api_key: "${NVIDIA_API_KEY}"
timeout: 600
- provider: "ollama"
role: "text"
usage: "qa_fallback" # 仅智能问答兜底，不参与视频
model_name: "qwen2.5:7b"
``` api_key: "${NVIDIA_API_KEY}"
timeout: 20 timeout: 20
circuit_breaker: circuit_breaker:
enabled: true enabled: true

51
fam-core/config/config.yaml
View File

@@ -1,10 +1,11 @@
# FAM-Core 配置文件 (NAS 端) - 实际部署配置 # FAM-Core 配置文件 (NAS 端) - 新架构 v2（2026-08-21）
# 注: Tailscale 防火墙待修复，当前 edge_url 使用 Oracle 公网 IP #
# 异步队列模式 + 分块断点续传: # NAS 仅作管理后台，不再处理视频。唯一后台线程 Oracle-Sync 每 30 分钟
# 小文件(<=50MB): 直接上传 /enqueue # 从甲骨文 FAM-Edge 拉取增量镜像到本地 MariaDB（sync_videos/events/people）。
# 大文件(>50MB): 分块(20MB/块)上传 /chunk → /assemble 合并入队 # 所有视频分析在 Oracle 完成。
# → NAS Poller 定期从 /api/edge/results 拉取结果写库 #
# Ollama 未对外暴露，chat_handler 通过 FAM-Edge 代理 # Tailscale 当前无法从 NAS 反向访问 Oracle，故 base_url 用 Oracle 公网 IP。
# token 与 Oracle 端 sync_api.token 一致，均取自环境变量 ORACLE_SYNC_TOKEN。
server: server:
host: "0.0.0.0" host: "0.0.0.0"
@@ -18,36 +19,16 @@ database:
database: "sentinel_home_ai" database: "sentinel_home_ai"
unix_socket: "/run/mysqld/mysqld10.sock" unix_socket: "/run/mysqld/mysqld10.sock"
scheduler: # 甲骨文同步（每 30 分钟拉增量镜像）
scan_interval: 60 oracle_sync:
# 正式目录 /volume1/surveillance/Generic_ONVIF-001 # FAM-Edge 对外同步接口地址（端口同其 server.port=5000）
video_dir: "/volume1/surveillance/Generic_ONVIF-001" base_url: "http://129.146.203.203:5000"
video_extensions: [".mp4", ".mkv", ".avi"] # 与 Oracle 端 sync_api.token 一致（环境变量注入，避免明文入库）
file_stable_seconds: 60 token: "${ORACLE_SYNC_TOKEN}"
camera_name: "客厅" interval_sec: 1800 # 拉取间隔（秒），默认 30 分钟
timeout: 120 # 单次拉取超时（秒）
dispatcher:
poll_interval: 30
edge_url: "http://129.146.203.203:5000/api/edge/video/enqueue"
max_retries: 5 # 文件级重试次数（分块级重试另计，每块3次）
stale_timeout: 600 # PROCESSING 超时回收（10分钟）
poller:
poll_interval: 30
results_url: "http://129.146.203.203:5000/api/edge/results"
batch_size: 10
timeout: 30
video_server:
base_url: "http://127.0.0.1:8000/media"
token: "sentinel-media-2026"
video_dir: "/volume1/surveillance"
chat_handler: chat_handler:
# 智能问答统一走 FAM-Edge 编排端点（Gemini → NVIDIA → 本地 Ollama 兜底） # 智能问答统一走 FAM-Edge 编排端点（Gemini → NVIDIA → 本地 Ollama 兜底）
qa_url: "http://129.146.203.203:5000/api/edge/chat/ask" qa_url: "http://129.146.203.203:5000/api/edge/chat/ask"
timeout: 120 timeout: 120
storage:
# 关键帧落盘目录（event_receiver 写入，fam-ui 读取展示时间轴）
frame_image_dir: "/volume1/web/sentinel-home-ai/fam-ui/static/frames"

76
fam-core/src/fam_core/app.py
View File

@@ -1,12 +1,14 @@
""" """
FAM-Core 主应用 - Flask 单进程 FAM-Core 主应用 - Flask 单进程（新架构 v2）
承载: Task-Scheduler / Dispatcher / Poller / Event-Receiver / Chat-Handler / Member-Manager / Video-Server 承载: Oracle-Sync（每 30 分钟拉取增量镜像）+ Member-Manager + Chat-Handler
NAS 不再处理视频：无 Scheduler / Dispatcher / Poller / Event-Receiver / Video-Server。
所有视频分析在 Oracle 完成，NAS 仅作管理后台拉取展示，CPU 占用大幅降低。
""" """
import os import os
import sys import sys
import time
import threading
from flask import Flask, jsonify from flask import Flask, jsonify
# 确保包路径 # 确保包路径
@@ -14,83 +16,49 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from .config_loader import load_config from .config_loader import load_config
from .logger import setup_logger from .logger import setup_logger
from .scheduler.scheduler import TaskScheduler from .oracle_sync import get_sync
from .dispatcher.dispatcher import Dispatcher
from .poller.poller import Poller
from .event_receiver.event_receiver import event_bp
from .chat_handler.chat_handler import chat_bp from .chat_handler.chat_handler import chat_bp
from .member_manager.member_manager import member_bp from .member_manager.member_manager import member_bp
from .video_server.video_server import video_bp
logger = setup_logger('fam-core.app') logger = setup_logger('fam-core.app')
app = Flask(__name__) app = Flask(__name__)
# 注册蓝图 # 注册蓝图
app.register_blueprint(event_bp)
app.register_blueprint(chat_bp) app.register_blueprint(chat_bp)
app.register_blueprint(member_bp) app.register_blueprint(member_bp)
app.register_blueprint(video_bp)
# 健康检查 # 健康检查
@app.route('/health', methods=['GET']) @app.route('/health', methods=['GET'])
def health(): def health():
return jsonify({"status": "ok", "service": "fam-core"}), 200 return jsonify({"status": "ok", "service": "fam-core"}), 200
# 初始化后台线程
_scheduler = None
_dispatcher = None
_poller = None
# 初始化后台同步线程（NAS 唯一常驻线程）
_sync = None
try: try:
_scheduler = TaskScheduler() _sync = get_sync()
_scheduler.start() _sync.start()
logger.info("Task-Scheduler 已启动") logger.info("Oracle-Sync 已启动")
# 启动后立刻拉一次，前端无需等待首个周期即有数据
try:
_sync.trigger_now()
logger.info("启动首次同步完成")
except Exception as e:
logger.warning(f"启动首次同步失败（后续周期会重试）: {e}")
except Exception as e: except Exception as e:
logger.error(f"Task-Scheduler 启动失败: {e}") logger.error(f"Oracle-Sync 启动失败: {e}")
try:
_dispatcher = Dispatcher()
_dispatcher.start()
logger.info("Dispatcher 已启动")
except Exception as e:
logger.error(f"Dispatcher 启动失败: {e}")
try:
_poller = Poller()
_poller.start()
logger.info("Poller 已启动")
except Exception as e:
logger.error(f"Poller 启动失败: {e}")
@app.route('/api/status', methods=['GET']) @app.route('/api/status', methods=['GET'])
def status(): def status():
"""系统状态（检查线程实际存活）""" """系统状态"""
return jsonify({ return jsonify({
"scheduler_running": _scheduler.is_alive() if _scheduler else False, "service": "fam-core",
"dispatcher_running": _dispatcher.is_alive() if _dispatcher else False, "sync": _sync.status() if _sync else {"running": False, "error": "未初始化"},
"poller_running": _poller.is_alive() if _poller else False,
}), 200 }), 200
def _watchdog_run():
"""看门狗：每 60s 检查线程存活，崩溃自动重启"""
logger.info("Watchdog 线程启动，检查间隔 60s")
while True:
time.sleep(60)
for comp, name in [(_scheduler, 'Scheduler'), (_dispatcher, 'Dispatcher'), (_poller, 'Poller')]:
if comp and hasattr(comp, 'check_and_restart'):
try:
comp.check_and_restart()
except Exception as e:
logger.error(f"Watchdog 重启 {name} 失败: {e}", exc_info=True)
_watchdog_thread = threading.Thread(target=_watchdog_run, daemon=True, name='watchdog')
_watchdog_thread.start()
if __name__ == '__main__': if __name__ == '__main__':
cfg = load_config() cfg = load_config()
port = cfg.get('server', {}).get('port', 8000) port = cfg.get('server', {}).get('port', 8000)

95
fam-core/src/fam_core/chat_handler/chat_handler.py
View File

@@ -1,17 +1,19 @@
""" """
Chat-Handler - Flask 蓝图，接收用户问答 Chat-Handler - Flask 蓝图，接收用户问答（新架构 v2）
处理逻辑: 逻辑:
1. 根据 queried_person 和 queried_date 查询 event_details 1. 按 queried_person + queried_date 从 sync_events 拉取相关事件
2. 拼接上下文（每条明细一行） （person_list_json 含该名且 ts 落在日期内）
3. 若明细条数 > 50，按小时聚合成摘要 2. 拼接上下文文本（每事件一行：时间 + 摄像头 + 描述 + 人物 + 是否关注）
4. POST Oracle Ollama 纯文本模式，调问答 Prompt 3. 调 Oracle FAM-Edge 问答编排端点（Gemini → NVIDIA → 本地 Ollama 兜底）
5. 插入 chat_history 4. 插入 chat_history
6. 返回回答 5. 返回回答
注意: 上下文来自 Oracle 已分析好的事件摘要，不做本地视频处理。
""" """
import json
import requests import requests
from flask import Blueprint, request, jsonify from flask import Blueprint, request, jsonify
from collections import defaultdict
from ..logger import setup_logger from ..logger import setup_logger
from ..config_loader import load_config from ..config_loader import load_config
@@ -21,9 +23,9 @@ logger = setup_logger('fam-core.chat_handler')
chat_bp = Blueprint('chat_handler', __name__) chat_bp = Blueprint('chat_handler', __name__)
CHAT_SYSTEM_PROMPT = """你是家庭监控助手。根据以下今日监控数据，回答用户问题。 CHAT_SYSTEM_PROMPT = """你是家庭监控助手。根据以下监控数据，回答用户问题。
今日数据（按时间顺序，每条一行）： 监控数据（按时间顺序，每条一行）：
{context} {context}
已知家庭成员: {members} 已知家庭成员: {members}
@@ -33,41 +35,27 @@ CHAT_SYSTEM_PROMPT = """你是家庭监控助手。根据以下今日监控数
要求: 要求:
- 只基于上述数据回答，不要编造 - 只基于上述数据回答，不要编造
- 按时间顺序总结 - 按时间顺序总结
- 若有关注事件（跌倒、哭闹等），重点提示 - 若有关注事件（跌倒、哭闹、陌生人等），重点提示
- 若当天没有该人员的数据，明确说"今天没有观察到{person}" - 若当天没有该人员的数据，明确说"今天没有观察到{person}"
- 用自然语言回答，不要输出 JSON - 用自然语言回答，不要输出 JSON
""" """
def _format_details(details): def _format_events(rows):
"""将 event_details 格式化为文本行""" """将 sync_events 查询行格式化为上下文文本"""
lines = [] lines = []
for d in details: for r in rows:
timestamp = d['frame_timestamp'].strftime('%H:%M') if hasattr(d['frame_timestamp'], 'strftime') else str(d['frame_timestamp']) ts = (r.get('ts') or '')[:16] # 'YYYY-MM-DD HH:MM'
camera = d.get('camera_name', '') camera = r.get('camera_name') or ''
person = d.get('person', '') desc = r.get('description') or ''
action = d.get('action', '') try:
clothing = d.get('clothing', '') persons = json.loads(r['person_list_json']) if isinstance(r['person_list_json'], str) else (r['person_list_json'] or [])
attention = ' [关注事件]' if d.get('is_attention_event') else '' except (ValueError, TypeError):
lines.append(f"[{timestamp} {camera}] {person} {action} ({clothing}){attention}") persons = []
return '\n'.join(lines) persons = [str(p) for p in persons]
attention = ' [关注事件]' if r.get('is_attention_event') else ''
person_str = ','.join(persons) if persons else '无人'
def _aggregate_by_hour(details): lines.append(f"[{ts} {camera}] {person_str}: {desc}{attention}")
"""当明细 > 50 条时，按小时聚合"""
hourly = defaultdict(list)
for d in details:
ts = d['frame_timestamp']
hour_key = ts.strftime('%Y-%m-%d %H:00') if hasattr(ts, 'strftime') else str(ts)
hourly[hour_key].append(d)
lines = []
for hour, items in sorted(hourly.items()):
persons = set(i.get('person', '') for i in items)
actions = set(i.get('action', '') for i in items)
has_attention = any(i.get('is_attention_event') for i in items)
attention = ' [含关注事件]' if has_attention else ''
lines.append(f"[{hour}] {','.join(persons)}: {','.join(actions)}{attention}")
return '\n'.join(lines) return '\n'.join(lines)
@@ -80,7 +68,6 @@ def _call_edge_qa(prompt: str) -> str:
timeout = cfg.get('chat_handler', {}).get('timeout', 120) timeout = cfg.get('chat_handler', {}).get('timeout', 120)
resp = requests.post(qa_url, json={"prompt": prompt}, timeout=timeout) resp = requests.post(qa_url, json={"prompt": prompt}, timeout=timeout)
if resp.status_code == 200: if resp.status_code == 200:
data = resp.json() data = resp.json()
answer = data.get('answer', '') answer = data.get('answer', '')
@@ -108,38 +95,27 @@ def chat_ask():
logger.info(f"Chat: person={queried_person}, date={queried_date}, question={question}") logger.info(f"Chat: person={queried_person}, date={queried_date}, question={question}")
# 1. 查询 event_details rows = db_layer.query_sync_events_for_person_date(queried_person, queried_date)
details = db_layer.query_event_details(queried_person, queried_date)
# 2. 拼接上下文 if len(rows) == 0:
if len(details) == 0:
# 无数据
answer = f"今天没有观察到{queried_person}。" answer = f"今天没有观察到{queried_person}。"
context_summary = "查询 event_details 0 条" context_summary = "查询 sync_events 0 条"
elif len(details) > 50:
context = _aggregate_by_hour(details)
context_summary = f"查询 event_details {len(details)} 条，按小时聚合为 {len(set(d['frame_timestamp'].strftime('%Y-%m-%d %H') for d in details))} 行"
else: else:
context = _format_details(details) context = _format_events(rows)
context_summary = f"查询 event_details {len(details)} 条，时间范围 {details[0]['frame_timestamp']} - {details[-1]['frame_timestamp']}" context_summary = f"查询 sync_events {len(rows)} 条"
members = db_layer.get_sync_known_members_context()
if len(details) > 0:
# 构建完整 Prompt
members = db_layer.get_known_members_context()
prompt = CHAT_SYSTEM_PROMPT.format( prompt = CHAT_SYSTEM_PROMPT.format(
context=context, context=context,
members=members or f"{queried_person}", members=members or queried_person,
question=question, question=question,
person=queried_person person=queried_person
) )
try: try:
answer = _call_edge_qa(prompt) answer = _call_edge_qa(prompt)
except Exception as e: except Exception as e:
logger.error(f"问答编排调用失败: {e}") logger.error(f"问答编排调用失败: {e}")
return jsonify({"error": f"AI 调用失败: {e}"}), 503 return jsonify({"error": f"AI 调用失败: {e}"}), 503
# 3. 写入 chat_history
chat_id = db_layer.insert_chat_history( chat_id = db_layer.insert_chat_history(
user_question=question, user_question=question,
ai_answer=answer, ai_answer=answer,
@@ -163,7 +139,6 @@ def chat_history():
limit = int(request.args.get('limit', 20)) limit = int(request.args.get('limit', 20))
history = db_layer.get_chat_history(limit=limit, date_filter=date, person_filter=person) history = db_layer.get_chat_history(limit=limit, date_filter=date, person_filter=person)
# datetime 序列化
for h in history: for h in history:
for k, v in h.items(): for k, v in h.items():
if hasattr(v, 'isoformat'): if hasattr(v, 'isoformat'):

909
fam-core/src/fam_core/db_layer.py
View File

@@ -1,6 +1,17 @@
""" """
数据库访问层 - MariaDB 连接管理与 CRUD 操作 数据库访问层 - MariaDB 连接管理与同步镜像 CRUD
使用 PyMySQL (纯 Python, ~45KB) 连接 MariaDB 服务器
新架构 (2026-08-21 重构):
NAS 不再处理视频，仅作为管理后台。
Oracle (FAM-Edge) 处理整视频分析后存 SQLite，NAS 每 30 分钟拉增量，
镜像到本地三张表:
sync_videos : 视频会话（全局摘要 + 事件 JSON + 人物 JSON）
sync_events : 视频拆出的时间点事件（描述 + 涉及人物 + 是否关注）
sync_people : 规范人物表（label + canonical_name，Oracle 维护）
sync_cursor : 同步游标（上次成功拉取到的 server_time）
本层只服务同步镜像 + 问答历史，旧 process_tasks/event_details/monitor_events/
family_members 相关逻辑已全部移除（视频处理职责已迁移至 Oracle）。
""" """
import json import json
import pymysql import pymysql
@@ -41,283 +52,339 @@ def get_conn():
# ============================================================ # ============================================================
# process_tasks 操作 # 同步镜像：sync_videos
# ============================================================ # ============================================================
def create_task(video_path: str, video_url: str) -> int: def upsert_sync_videos(rows: List[Dict]) -> int:
"""创建新任务""" """批量 upsert Oracle 传来的 videos 增量。rows 为 Oracle 端 dict 列表。"""
if not rows:
return 0
conn = get_conn() conn = get_conn()
n = 0
try: try:
cursor = conn.cursor() cur = conn.cursor()
cursor.execute( for r in rows:
"INSERT INTO process_tasks (video_path, video_url, status) VALUES (%s, %s, 'PENDING')", cur.execute(
(video_path, video_url) """INSERT INTO sync_videos
(id, drive_file_id, filename, camera_name, duration_sec,
event_start_time, status, summary_json, events_json,
people_json, compute_provider, created_at, updated_at,
processed_at, synced_at)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s, NOW())
ON DUPLICATE KEY UPDATE
drive_file_id=VALUES(drive_file_id),
filename=VALUES(filename),
camera_name=VALUES(camera_name),
duration_sec=VALUES(duration_sec),
event_start_time=VALUES(event_start_time),
status=VALUES(status),
summary_json=VALUES(summary_json),
events_json=VALUES(events_json),
people_json=VALUES(people_json),
compute_provider=VALUES(compute_provider),
created_at=VALUES(created_at),
updated_at=VALUES(updated_at),
processed_at=VALUES(processed_at),
synced_at=NOW()""",
(r.get('id'), r.get('drive_file_id'), r.get('filename'),
r.get('camera_name'), r.get('duration_sec') or 0,
r.get('event_start_time'), r.get('status'),
r.get('summary_json'), r.get('events_json'), r.get('people_json'),
r.get('compute_provider'), r.get('created_at'),
r.get('updated_at'), r.get('processed_at'))
) )
n += 1
conn.commit() conn.commit()
task_id = cursor.lastrowid return n
logger.info(f"[task_id={task_id}] task created: {video_path}")
return task_id
finally: finally:
conn.close() conn.close()
def get_pending_tasks(limit=10) -> List[Dict]: def get_sync_videos(limit=15, offset=0, date_filter=None) -> List[Dict]:
"""获取待处理任务""" """获取视频会话列表（已完成优先），支持日期筛选与分页。
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute(
"SELECT * FROM process_tasks WHERE status = 'PENDING' ORDER BY created_at ASC LIMIT %s",
(limit,)
)
return cursor.fetchall()
finally:
conn.close()
排序按 COALESCE(processed_at, updated_at, created_at) 降序。
def get_tasks_by_status(status: str, limit=10) -> List[Dict]: date_filter 形如 '2026-08-21'，匹配 processed_at 前缀。
"""按状态获取任务"""
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute(
"SELECT * FROM process_tasks WHERE status = %s ORDER BY created_at ASC LIMIT %s",
(status, limit)
)
return cursor.fetchall()
finally:
conn.close()
def update_task_status(task_id: int, status: str, error_message: str = None,
failure_stage: str = None):
"""更新任务状态"""
valid_stages = {'download', 'extract', 'vlm_visual', 'vlm_fusion', 'callback', 'process'}
if failure_stage and failure_stage not in valid_stages:
failure_stage = 'callback'
conn = get_conn()
try:
cursor = conn.cursor()
cursor.execute(
"UPDATE process_tasks SET status=%s, error_message=%s, failure_stage=%s WHERE task_id=%s",
(status, error_message, failure_stage, task_id)
)
conn.commit()
finally:
conn.close()
def increment_retry(task_id: int, next_retry_at: datetime):
"""递增重试次数"""
conn = get_conn()
try:
cursor = conn.cursor()
cursor.execute(
"UPDATE process_tasks SET retry_count=retry_count+1, next_retry_at=%s, status='PENDING' WHERE task_id=%s",
(next_retry_at, task_id)
)
conn.commit()
finally:
conn.close()
def reclaim_stale_processing(timeout_seconds: int) -> List[int]:
"""回收僵尸 PROCESSING 任务：updated_at 早于 timeout_seconds 前的任务重置为 PENDING
场景：Dispatcher 推送过程中进程重启/Edge 重启导致 in-flight 请求丢失，
任务停留在 PROCESSING 无人处理。重置后由常规重试机制接管。
返回被回收的 task_id 列表。
""" """
conn = get_conn() conn = get_conn()
try: try:
cursor = conn.cursor() cur = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute(
"SELECT task_id FROM process_tasks "
"WHERE status='PROCESSING' AND updated_at < NOW() - INTERVAL %s SECOND",
(timeout_seconds,)
)
task_ids = [row[0] for row in cursor.fetchall()]
if task_ids:
placeholders = ','.join(['%s'] * len(task_ids))
cursor.execute(
f"UPDATE process_tasks SET status='PENDING' WHERE task_id IN ({placeholders})",
task_ids
)
conn.commit()
return task_ids
finally:
conn.close()
def get_task(task_id: int) -> Optional[Dict]:
"""获取单个任务"""
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute("SELECT * FROM process_tasks WHERE task_id = %s", (task_id,))
return cursor.fetchone()
finally:
conn.close()
def get_video_url_exists(video_path: str) -> bool:
"""检查视频是否已有对应任务（避免重复）"""
conn = get_conn()
try:
cursor = conn.cursor()
cursor.execute(
"SELECT COUNT(*) FROM process_tasks WHERE video_path = %s",
(video_path,)
)
return cursor.fetchone()[0] > 0
finally:
conn.close()
# ============================================================
# monitor_events 操作
# ============================================================
def _dt_or_none(value):
"""datetime 字段归一化（MariaDB 严格模式兼容）
- 空串/None/'None'/'null' → NULL
- ISO 8601（2026-08-20T01:06:44Z / 2026-08-20T01:06:44.123+00:00）→ '2026-08-20 01:06:44'
- 已为标准格式则原样返回
"""
if value is None:
return None
s = str(value).strip()
if s in ('', 'None', 'null', 'NaN'):
return None
# 归一化 ISO 8601 -> 'YYYY-MM-DD HH:MM:SS'
s2 = s.replace('T', ' ').replace('Z', '').replace('z', '')
if '+' in s2[10:]:
s2 = s2[:s2.index('+')]
if '.' in s2:
s2 = s2[:s2.index('.')]
try:
dt = datetime.strptime(s2, '%Y-%m-%d %H:%M:%S')
return dt.strftime('%Y-%m-%d %H:%M:%S')
except Exception:
return None
def insert_event(task_id: int, event_start_time: str, event_end_time: str,
camera_name: str, global_summary: str, entities_json: list,
compute_provider: list) -> int:
"""插入事件聚合记录"""
conn = get_conn()
try:
cursor = conn.cursor()
# event_start/end_time 为 NOT NULL 列: 空值兜底
# end 缺失 → 用 start；start 也缺失 → 用当前时间
dt_start = _dt_or_none(event_start_time)
dt_end = _dt_or_none(event_end_time)
if not dt_end:
dt_end = dt_start
if not dt_start:
from datetime import datetime as _dt
dt_start = dt_end = _dt.now().strftime('%Y-%m-%d %H:%M:%S')
cursor.execute(
"""INSERT INTO monitor_events
(task_id, event_start_time, event_end_time, camera_name,
global_summary, entities_json, compute_provider)
VALUES (%s, %s, %s, %s, %s, %s, %s)""",
(task_id, dt_start, dt_end, camera_name,
global_summary, json.dumps(entities_json, ensure_ascii=False),
json.dumps(compute_provider, ensure_ascii=False))
)
conn.commit()
return cursor.lastrowid
finally:
conn.close()
# ============================================================
# event_details 操作
# ============================================================
def insert_event_detail(event_id: int, task_id: int, frame_index: int,
frame_timestamp: str, camera_name: str,
person: str, action: str, clothing: str,
is_attention_event: bool, source_providers: list):
"""插入事件明细"""
conn = get_conn()
try:
cursor = conn.cursor()
# frame_timestamp 为 NOT NULL 列: 空值兜底为当前时间
dt_ts = _dt_or_none(frame_timestamp)
if not dt_ts:
from datetime import datetime as _dt
dt_ts = _dt.now().strftime('%Y-%m-%d %H:%M:%S')
cursor.execute(
"""INSERT INTO event_details
(event_id, task_id, frame_index, frame_timestamp, camera_name,
person, action, clothing, is_attention_event, source_providers)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)""",
(event_id, task_id, frame_index, dt_ts, camera_name,
person, action, clothing, is_attention_event,
json.dumps(source_providers, ensure_ascii=False))
)
conn.commit()
finally:
conn.close()
def query_event_details(person: str, queried_date: str) -> List[Dict]:
"""查询某人在某天的事件明细"""
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute(
"""SELECT frame_timestamp, camera_name, person, action, clothing,
is_attention_event
FROM event_details
WHERE person = %s AND DATE(frame_timestamp) = %s
ORDER BY frame_timestamp ASC""",
(person, queried_date)
)
return cursor.fetchall()
finally:
conn.close()
def get_recent_events(limit=20, offset=0, date_filter=None) -> List[Dict]:
"""获取事件列表（分页 + 日期筛选）"""
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
if date_filter: if date_filter:
cursor.execute( cur.execute(
"""SELECT me.event_id, me.task_id, me.event_start_time, me.event_end_time, """SELECT id, filename, camera_name, event_start_time, status,
me.camera_name, me.global_summary, me.compute_provider, summary_json, events_json, people_json, compute_provider,
me.created_at, processed_at, updated_at,
(SELECT COUNT(*) FROM event_details ed WHERE ed.event_id = me.event_id) AS detail_count (SELECT COUNT(*) FROM sync_events se WHERE se.video_id = sync_videos.id) AS event_count
FROM monitor_events me FROM sync_videos
WHERE DATE(me.event_start_time) = %s WHERE status='done' AND processed_at LIKE %s
ORDER BY me.event_start_time DESC ORDER BY COALESCE(processed_at, updated_at, created_at) DESC
LIMIT %s OFFSET %s""", LIMIT %s OFFSET %s""",
(date_filter, limit, offset) (f'{date_filter}%', limit, offset))
)
else: else:
cursor.execute( cur.execute(
"""SELECT me.event_id, me.task_id, me.event_start_time, me.event_end_time, """SELECT id, filename, camera_name, event_start_time, status,
me.camera_name, me.global_summary, me.compute_provider, summary_json, events_json, people_json, compute_provider,
me.created_at, processed_at, updated_at,
(SELECT COUNT(*) FROM event_details ed WHERE ed.event_id = me.event_id) AS detail_count (SELECT COUNT(*) FROM sync_events se WHERE se.video_id = sync_videos.id) AS event_count
FROM monitor_events me FROM sync_videos
ORDER BY me.event_start_time DESC WHERE status='done'
ORDER BY COALESCE(processed_at, updated_at, created_at) DESC
LIMIT %s OFFSET %s""", LIMIT %s OFFSET %s""",
(limit, offset) (limit, offset))
) return cur.fetchall()
return cursor.fetchall() finally:
conn.close()
def get_sync_video(video_id: int) -> Optional[Dict]:
conn = get_conn()
try:
cur = conn.cursor(pymysql.cursors.DictCursor)
cur.execute("SELECT * FROM sync_videos WHERE id=%s", (video_id,))
return cur.fetchone()
finally: finally:
conn.close() conn.close()
# ============================================================ # ============================================================
# chat_history 操作 # 同步镜像：sync_events
# ============================================================
def upsert_sync_events(rows: List[Dict]) -> int:
"""批量 upsert Oracle 传来的 events 增量。"""
if not rows:
return 0
conn = get_conn()
n = 0
try:
cur = conn.cursor()
for r in rows:
cur.execute(
"""INSERT INTO sync_events
(id, video_id, ts, description, person_list_json,
is_attention_event, updated_at, synced_at)
VALUES (%s,%s,%s,%s,%s,%s,%s, NOW())
ON DUPLICATE KEY UPDATE
video_id=VALUES(video_id),
ts=VALUES(ts),
description=VALUES(description),
person_list_json=VALUES(person_list_json),
is_attention_event=VALUES(is_attention_event),
updated_at=VALUES(updated_at),
synced_at=NOW()""",
(r.get('id'), r.get('video_id'), r.get('ts'), r.get('description'),
r.get('person_list_json'), 1 if r.get('is_attention_event') else 0,
r.get('updated_at'))
)
n += 1
conn.commit()
return n
finally:
conn.close()
def get_sync_events_for_video(video_id: int) -> List[Dict]:
conn = get_conn()
try:
cur = conn.cursor(pymysql.cursors.DictCursor)
cur.execute(
"""SELECT id, video_id, ts, description, person_list_json,
is_attention_event, updated_at
FROM sync_events WHERE video_id=%s ORDER BY ts ASC""",
(video_id,))
return cur.fetchall()
finally:
conn.close()
def query_sync_events_for_person_date(person: str, date_str: str) -> List[Dict]:
"""问答上下文：某人在某天的事件。
说明: Oracle 事件 ts 为视频内相对时间点（如 00:01:23），不是绝对日期，
因此按所属视频的 processed_at 日期过滤，再按 person_list_json 命中人名。
person 可为真名或抽象标签（Oracle 回灌上下文用真名，但历史标签也保留）。
"""
conn = get_conn()
try:
cur = conn.cursor(pymysql.cursors.DictCursor)
cur.execute(
"""SELECT e.ts, e.description, e.person_list_json, e.is_attention_event,
v.camera_name, v.filename, v.event_start_time, v.processed_at
FROM sync_events e
JOIN sync_videos v ON e.video_id = v.id
WHERE v.processed_at LIKE %s AND e.person_list_json LIKE %s
ORDER BY v.processed_at ASC, e.ts ASC""",
(f'{date_str}%', f'%{person}%'))
return cur.fetchall()
finally:
conn.close()
# ============================================================
# 同步镜像：sync_people
# ============================================================
def upsert_sync_people(rows: List[Dict]) -> int:
"""批量 upsert Oracle 传来的 people 增量。"""
if not rows:
return 0
conn = get_conn()
n = 0
try:
cur = conn.cursor()
for r in rows:
cur.execute(
"""INSERT INTO sync_people
(id, label, canonical_name, first_seen, appearances,
source, updated_at, synced_at)
VALUES (%s,%s,%s,%s,%s,%s,%s, NOW())
ON DUPLICATE KEY UPDATE
label=VALUES(label),
canonical_name=VALUES(canonical_name),
first_seen=VALUES(first_seen),
appearances=VALUES(appearances),
source=VALUES(source),
updated_at=VALUES(updated_at),
synced_at=NOW()""",
(r.get('id'), r.get('label'), r.get('canonical_name'),
r.get('first_seen'), r.get('appearances') or 0,
r.get('source'), r.get('updated_at')))
n += 1
conn.commit()
return n
finally:
conn.close()
def get_sync_people() -> List[Dict]:
conn = get_conn()
try:
cur = conn.cursor(pymysql.cursors.DictCursor)
cur.execute(
"SELECT id, label, canonical_name, first_seen, appearances, source, updated_at "
"FROM sync_people ORDER BY id ASC")
return cur.fetchall()
finally:
conn.close()
def get_sync_named_members() -> List[str]:
"""已命名成员的真名列表（供 UI 下拉 / 快捷选择）。"""
conn = get_conn()
try:
cur = conn.cursor(pymysql.cursors.DictCursor)
cur.execute(
"SELECT DISTINCT canonical_name FROM sync_people "
"WHERE canonical_name IS NOT NULL AND canonical_name != '' "
"ORDER BY canonical_name ASC")
return [r['canonical_name'] for r in cur.fetchall()]
finally:
conn.close()
def get_sync_known_members_context() -> str:
"""获取人物清单文本，注入问答 Prompt，让模型用真名指代。"""
conn = get_conn()
try:
cur = conn.cursor(pymysql.cursors.DictCursor)
cur.execute(
"SELECT label, canonical_name FROM sync_people ORDER BY id ASC")
rows = cur.fetchall()
if not rows:
return ""
parts = []
for r in rows:
name = r['canonical_name'] or r['label']
if r['canonical_name'] and r['canonical_name'] != r['label']:
parts.append(f"- {name}（标识：{r['label']}）")
else:
parts.append(f"- {name}")
return "\n".join(parts)
finally:
conn.close()
# ============================================================
# 同步游标
# ============================================================
def get_sync_cursor() -> str:
conn = get_conn()
try:
cur = conn.cursor()
cur.execute("SELECT value FROM sync_cursor WHERE key='last_since'")
row = cur.fetchone()
return row[0] if row else ''
finally:
conn.close()
def set_sync_cursor(value: str):
conn = get_conn()
try:
cur = conn.cursor()
cur.execute(
"""INSERT INTO sync_cursor (key, value) VALUES ('last_since', %s)
ON DUPLICATE KEY UPDATE value=VALUES(value)""",
(value,))
conn.commit()
finally:
conn.close()
# ============================================================
# 统计
# ============================================================
def get_sync_stats(date_str: str = None) -> Dict:
"""概览统计：视频数 / 事件数 / 关注事件数 / 出现人物数（按 date 可选过滤）。
人物数通过对 sync_events.person_list_json LIKE 统计（MariaDB 不支持 JSON 数组展开）。
"""
conn = get_conn()
try:
cur = conn.cursor(pymysql.cursors.DictCursor)
# 视频/事件/关注数
if date_str:
cur.execute(
"""SELECT
COUNT(*) AS videos,
(SELECT COUNT(*) FROM sync_events se
JOIN sync_videos sv ON se.video_id=sv.id
WHERE sv.processed_at LIKE %s) AS events,
(SELECT COALESCE(SUM(se.is_attention_event),0) FROM sync_events se
JOIN sync_videos sv ON se.video_id=sv.id
WHERE sv.processed_at LIKE %s) AS attention
FROM sync_videos sv WHERE sv.processed_at LIKE %s""",
(f'{date_str}%', f'{date_str}%', f'{date_str}%'))
else:
cur.execute(
"""SELECT
(SELECT COUNT(*) FROM sync_videos WHERE status='done') AS videos,
(SELECT COUNT(*) FROM sync_events) AS events,
(SELECT COALESCE(SUM(is_attention_event),0) FROM sync_events) AS attention""")
stat = cur.fetchone() or {}
# 人物数（distinct label 命中 sync_events）
cur.execute("SELECT id, label, canonical_name FROM sync_people")
people = cur.fetchall()
# 基于 person_list_json 命中计数：逐 label 统计命中事件数
person_hits = 0
for p in people:
name = p['canonical_name'] or p['label']
cur.execute(
"SELECT COUNT(*) c FROM sync_events WHERE person_list_json LIKE %s",
(f'%{name}%',))
if cur.fetchone()['c'] > 0:
person_hits += 1
stat['people'] = person_hits
return stat
finally:
conn.close()
# ============================================================
# chat_history（保留：问答历史）
# ============================================================ # ============================================================
def insert_chat_history(user_question: str, ai_answer: str, def insert_chat_history(user_question: str, ai_answer: str,
@@ -326,15 +393,15 @@ def insert_chat_history(user_question: str, ai_answer: str,
"""插入对话记录""" """插入对话记录"""
conn = get_conn() conn = get_conn()
try: try:
cursor = conn.cursor() cur = conn.cursor()
cursor.execute( cur.execute(
"""INSERT INTO chat_history """INSERT INTO chat_history
(user_question, ai_answer, context_summary, queried_date, queried_person) (user_question, ai_answer, context_summary, queried_date, queried_person)
VALUES (%s, %s, %s, %s, %s)""", VALUES (%s, %s, %s, %s, %s)""",
(user_question, ai_answer, context_summary, queried_date, queried_person) (user_question, ai_answer, context_summary, queried_date, queried_person)
) )
conn.commit() conn.commit()
return cursor.lastrowid return cur.lastrowid
finally: finally:
conn.close() conn.close()
@@ -343,7 +410,7 @@ def get_chat_history(limit=20, date_filter=None, person_filter=None) -> List[Dic
"""获取对话历史""" """获取对话历史"""
conn = get_conn() conn = get_conn()
try: try:
cursor = conn.cursor(pymysql.cursors.DictCursor) cur = conn.cursor(pymysql.cursors.DictCursor)
conditions = [] conditions = []
params = [] params = []
if date_filter: if date_filter:
@@ -353,327 +420,11 @@ def get_chat_history(limit=20, date_filter=None, person_filter=None) -> List[Dic
conditions.append("queried_person = %s") conditions.append("queried_person = %s")
params.append(person_filter) params.append(person_filter)
where = f"WHERE {' AND '.join(conditions)}" if conditions else "" where = f"WHERE {' AND '.join(conditions)}" if conditions else ""
params.extend([limit]) params.append(limit)
cursor.execute( cur.execute(
f"SELECT * FROM chat_history {where} ORDER BY created_at DESC LIMIT %s", f"SELECT * FROM chat_history {where} ORDER BY created_at DESC LIMIT %s",
params params
) )
return cursor.fetchall() return cur.fetchall()
finally:
conn.close()
# ============================================================
# family_members 操作
# ============================================================
def upsert_family_member(abstract_label: str, feature_description: str,
first_seen_at: str):
"""upsert 家庭成员（abstract_label 唯一）"""
conn = get_conn()
try:
cursor = conn.cursor()
cursor.execute(
"""INSERT INTO family_members (abstract_label, feature_description, first_seen_at)
VALUES (%s, %s, %s)
ON DUPLICATE KEY UPDATE abstract_label = abstract_label""",
(abstract_label, feature_description, first_seen_at)
)
conn.commit()
finally:
conn.close()
def get_unnamed_members() -> List[Dict]:
"""获取未命名成员列表"""
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute(
"""SELECT fm.abstract_label, fm.feature_description, fm.first_seen_at,
(SELECT COUNT(*) FROM event_details ed WHERE ed.person = fm.abstract_label) AS event_count
FROM family_members fm
WHERE fm.real_name IS NULL AND fm.is_active = TRUE
ORDER BY fm.first_seen_at ASC"""
)
return cursor.fetchall()
finally:
conn.close()
def get_all_members(include_named=True, include_unnamed=True) -> List[Dict]:
"""获取所有成员"""
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
conditions = []
if include_named and include_unnamed:
pass # 全部
elif include_named:
conditions.append("real_name IS NOT NULL")
elif include_unnamed:
conditions.append("real_name IS NULL")
where = f"WHERE {' AND '.join(conditions)}" if conditions else ""
cursor.execute(
f"""SELECT * FROM family_members {where}
ORDER BY first_seen_at ASC"""
)
return cursor.fetchall()
finally:
conn.close()
def name_member(abstract_label: str, real_name: str, named_by: str) -> Dict:
"""命名/重命名成员 + 批量回溯更新历史记录
- 标签未入库（如 AI 新输出的 人物1）自动注册
- 已命名成员可重命名（旧真名一并回溯替换）
- person 为多人组合字符串（'张三, 汤圆'），用 REPLACE 替换其中目标
"""
conn = get_conn()
try:
cursor = conn.cursor()
# 1. 查现有记录，拿到旧真名
cursor.execute(
"SELECT member_id, real_name FROM family_members WHERE abstract_label = %s",
(abstract_label,)
)
row = cursor.fetchone()
old_name = None
if row:
old_name = row[1]
else:
cursor.execute(
"""INSERT INTO family_members (abstract_label, feature_description, first_seen_at)
VALUES (%s, %s, NOW())""",
(abstract_label, f'由命名操作自动注册: {real_name}')
)
# 2. 更新 family_members
cursor.execute(
"""UPDATE family_members
SET real_name = %s, named_at = NOW(), named_by = %s
WHERE abstract_label = %s""",
(real_name, named_by, abstract_label)
)
# 3. 批量回溯更新 event_details
# 目标出现的三种形态: 独占整字段 / 多人组合内 / AI 直呼旧真名
updated_details_count = 0
for old in {abstract_label, old_name} - {None}:
if old == real_name:
continue
cursor.execute(
"UPDATE event_details SET person = %s WHERE person = %s",
(real_name, old)
)
updated_details_count += cursor.rowcount
cursor.execute(
"""UPDATE event_details
SET person = REPLACE(person, %s, %s)
WHERE person LIKE %s AND person <> %s""",
(old, real_name, f'%{old}%', real_name)
)
updated_details_count += cursor.rowcount
# 4. 批量回溯更新 monitor_events.entities_json
# MariaDB 10.11 不支持 MySQL 的 $[*] 通配符 JSON 路径，
# 改用 Python 层解析 + 逐行更新
import json as _json
updated_events_count = 0
targets = {abstract_label, old_name} - {None}
if targets and targets != {real_name}:
like_conds = ' OR '.join(['entities_json LIKE %s'] * len(targets))
like_args = [f'%{t}%' for t in targets]
cursor.execute(
f"SELECT event_id, entities_json FROM monitor_events WHERE {like_conds}",
tuple(like_args)
)
for eid, entities_raw in cursor.fetchall():
if not entities_raw:
continue
try:
entities = _json.loads(entities_raw) if isinstance(entities_raw, str) else entities_raw
except (ValueError, TypeError):
continue
changed = False
if isinstance(entities, list):
for ent in entities:
if isinstance(ent, dict) and ent.get('person') in targets:
ent['person'] = real_name
changed = True
if changed:
cursor.execute(
"UPDATE monitor_events SET entities_json = %s WHERE event_id = %s",
(_json.dumps(entities, ensure_ascii=False), eid)
)
updated_events_count += 1
conn.commit()
return {
"abstract_label": abstract_label,
"real_name": real_name,
"renamed_from": old_name,
"updated_event_details_count": updated_details_count,
"updated_monitor_events_count": updated_events_count
}
except Exception as e:
conn.rollback()
raise e
finally:
conn.close()
def merge_member(source_key: str, target_key: str, named_by: str = '管理员') -> Dict:
"""合并人物: source 并入 target（用户判断两帧是同一人时）
source_key/target_key 可为 abstract_label 或 real_name。
- event_details.person: 独占/组合字符串内的 source 一律替换为 target 显示名
- monitor_events.entities_json: person 字段替换
- family_members: source 行置 is_active=0（保留历史），target 未入库则注册
"""
if source_key == target_key:
return {"error": "source 与 target 不能相同"}
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
def resolve(key):
cursor.execute(
"""SELECT member_id, abstract_label, real_name FROM family_members
WHERE is_active = TRUE AND (abstract_label = %s OR real_name = %s)
ORDER BY real_name IS NULL LIMIT 1""",
(key, key))
return cursor.fetchone()
src = resolve(source_key)
tgt = resolve(target_key)
if not src:
return {"error": f"人物 {source_key} 不存在"}
if not tgt:
# target 是未入库的裸标签（如 人物1），注册后作为目标
cursor.execute(
"""INSERT INTO family_members (abstract_label, feature_description, first_seen_at)
VALUES (%s, %s, NOW())""",
(target_key, f'合并操作自动注册: {source_key} 并入'))
tgt = {'abstract_label': target_key, 'real_name': None}
target_display = tgt['real_name'] or tgt['abstract_label']
# source 的所有称呼: 抽象标签 + 旧真名（多人组合里两种都可能出现）
source_names = {src['abstract_label']}
if src['real_name']:
source_names.add(src['real_name'])
updated_details = 0
for name in source_names:
if name == target_display:
continue
cursor.execute(
"UPDATE event_details SET person = %s WHERE person = %s",
(target_display, name))
updated_details += cursor.rowcount
cursor.execute(
"""UPDATE event_details
SET person = REPLACE(person, %s, %s)
WHERE person LIKE %s AND person <> %s""",
(name, target_display, f'%{name}%', target_display))
updated_details += cursor.rowcount
# entities_json 逐行替换
import json as _json
updated_events = 0
like_conds = ' OR '.join(['entities_json LIKE %s'] * len(source_names))
cursor.execute(
f"SELECT event_id, entities_json FROM monitor_events WHERE {like_conds}",
tuple(f'%{n}%' for n in source_names))
for row in cursor.fetchall():
raw = row['entities_json']
if not raw:
continue
try:
entities = _json.loads(raw) if isinstance(raw, str) else raw
except (ValueError, TypeError):
continue
changed = False
if isinstance(entities, list):
for ent in entities:
if isinstance(ent, dict) and ent.get('person') in source_names:
ent['person'] = target_display
changed = True
if changed:
cursor.execute(
"UPDATE monitor_events SET entities_json = %s WHERE event_id = %s",
(_json.dumps(entities, ensure_ascii=False), row['event_id']))
updated_events += 1
# source 行停用
cursor.execute(
"UPDATE family_members SET is_active = 0, updated_at = NOW() WHERE member_id = %s",
(src['member_id'],))
conn.commit()
return {
"source": source_key,
"target": target_display,
"merged_names": sorted(source_names),
"updated_event_details_count": updated_details,
"updated_monitor_events_count": updated_events
}
except Exception as e:
conn.rollback()
raise e
finally:
conn.close()
def get_known_members_context() -> str:
"""获取已命名+未命名成员清单，用于注入 VLM Prompt"""
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute(
"SELECT abstract_label, real_name, feature_description FROM family_members WHERE is_active = TRUE"
)
members = cursor.fetchall()
if not members:
return ""
parts = []
for m in members:
name = m['real_name'] if m['real_name'] else m['abstract_label']
feat = m['feature_description'] or ''
status = f"(real_name={m['real_name']})" if m['real_name'] else f"(abstract_label={m['abstract_label']}, 未命名)"
parts.append(f"{name}: {feat} {status}")
return "; ".join(parts)
finally:
conn.close()
# ============================================================
# compute_provider 统计
# ============================================================
def get_compute_provider_stats() -> List[Dict]:
"""获取 compute_provider 分布统计"""
conn = get_conn()
try:
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute(
"""SELECT
JSON_UNQUOTE(JSON_EXTRACT(item, '$')) AS provider,
COUNT(*) AS count
FROM monitor_events,
JSON_TABLE(compute_provider, '$[*]'
COLUMNS(item VARCHAR(50) PATH '$'
)) AS jt
GROUP BY provider
ORDER BY count DESC"""
)
return cursor.fetchall()
except Exception:
# MariaDB 旧版不支持 JSON_TABLE，降级方案
cursor.execute("SELECT compute_provider, COUNT(*) AS count FROM monitor_events GROUP BY compute_provider")
return cursor.fetchall()
finally: finally:
conn.close() conn.close()

4
fam-core/src/fam_core/dispatcher/__init__.py
View File

@@ -1,4 +0,0 @@
"""Dispatcher 包"""
from .dispatcher import Dispatcher
__all__ = ["Dispatcher"]

517
fam-core/src/fam_core/dispatcher/dispatcher.py
View File

@@ -1,517 +0,0 @@
"""
Dispatcher - 30s 轮询 PENDING 任务，上传视频至 Edge 异步队列
流程（异步队列模式 + NAS 预压缩 + 分块断点续传）:
1. 读取任务对应的本地视频文件
2. 大文件 (>20MB): NAS 端 FFmpeg 预压缩 (480p/CRF28, ~36x 压缩比)
（根因: NAS→Oracle 跨境上行带宽仅 ~0.5-0.9MB/s，360MB 原始上传需 10-20min 且频繁超时）
3a. 压缩后小文件 (<=20MB): 直接 multipart 上传至 /enqueue
3b. 仍超阈值: 分块上传 (5MB/块) 至 /chunk，支持断点续传，最后调 /assemble 合并入队
4. Edge 保存视频 + 入 SQLite 队列，返回 202
5. Dispatcher 标记任务为 PROCESSING（已派发，等待 Poller 拉取结果）
6. Poller 线程定期从 Edge /api/edge/results 拉取结果，写库后标记 SUCCESS
退避重试: min(30 * (retry_count + 1), 300) 秒
分块级重试: 每块最多重试 3 次
看门狗: 线程崩溃后自动重启
"""
import os
import io
import re
import time
import math
import shutil
import subprocess
import threading
import requests
from datetime import datetime, timedelta
from ..logger import setup_logger, log_task
from ..config_loader import load_config
from .. import db_layer
logger = setup_logger('fam-core.dispatcher')
CHUNK_SIZE = 5 * 1024 * 1024 # 5MB per chunk (reliable at ~1Mbps upload)
CHUNK_THRESHOLD = 20 * 1024 * 1024 # files > 20MB use chunked upload
COMPRESS_THRESHOLD = 20 * 1024 * 1024 # files > 20MB get pre-compressed before upload
COMPRESS_DIR = '/tmp/fam_compressed'
COMPRESS_CACHE_TTL = 24 * 3600 # 压缩缓存保留 24h（供上传失败重试复用）
MAX_CHUNK_RETRIES = 3
# Synology 系统 ffmpeg 被裁剪（无 h264 编解码），CodecPack 的 ffmpeg41 带 libx264
FFMPEG_CANDIDATES = [
'/var/packages/CodecPack/target/bin/ffmpeg41',
'/usr/local/bin/ffmpeg',
]
def _safe_remove(path):
"""忽略不存在/清理失败的删除"""
try:
if os.path.isfile(path):
os.remove(path)
except OSError:
pass
class Dispatcher:
"""任务下发器，30s 轮询（异步队列 + 分块断点续传）"""
def __init__(self):
cfg = load_config()
self.poll_interval = cfg.get('dispatcher', {}).get('poll_interval', 30)
self.edge_url = cfg.get('dispatcher', {}).get('edge_url',
'http://localhost:5000/api/edge/video/enqueue')
self.max_retries = cfg.get('dispatcher', {}).get('max_retries', 3)
self.camera_name = cfg.get('scheduler', {}).get('camera_name', '默认摄像头')
self.stale_timeout = cfg.get('dispatcher', {}).get('stale_timeout', 600)
self.compress_timeout = cfg.get('dispatcher', {}).get('compress_timeout', 3600)
self.ffmpeg = self._find_ffmpeg(cfg)
# 推导 Edge base URL
self.edge_base = self.edge_url.rsplit('/api/edge/video/enqueue', 1)[0]
self.chunk_url = f"{self.edge_base}/api/edge/video/chunk"
self.chunks_query_url = f"{self.edge_base}/api/edge/video/chunks"
self.assemble_url = f"{self.edge_base}/api/edge/video/assemble"
self._running = False
self._thread = None
@staticmethod
def _find_ffmpeg(cfg):
"""查找可用 ffmpeg：配置优先，其次 CodecPack（带 libx264），最后 PATH"""
configured = cfg.get('dispatcher', {}).get('ffmpeg_path')
candidates = ([configured] if configured else []) + FFMPEG_CANDIDATES
for path in candidates:
if os.path.isfile(path) and os.access(path, os.X_OK):
return path
return shutil.which('ffmpeg')
def _calculate_backoff(self, retry_count):
"""退避策略: min(30 * (retry_count + 1), 300)"""
return min(30 * (retry_count + 1), 300)
def _should_retry(self, task):
"""检查任务是否可以重试"""
if task['retry_count'] >= task['max_retries']:
return False
if task['next_retry_at']:
now = datetime.now()
if now < task['next_retry_at']:
return False
return True
def _probe_duration(self, video_path):
"""用 ffmpeg 解析视频时长（NAS 无独立 ffprobe），失败返回 0"""
if not self.ffmpeg:
return 0
try:
r = subprocess.run(
[self.ffmpeg, '-i', video_path],
capture_output=True, timeout=30)
m = re.search(r'Duration:\s*(\d+):(\d+):(\d+(?:\.\d+)?)',
r.stderr.decode('utf-8', 'ignore'))
if m:
return int(m.group(1)) * 3600 + int(m.group(2)) * 60 + float(m.group(3))
except (subprocess.TimeoutExpired, OSError):
pass
return 0
def _build_payload(self, task):
"""构建推送元数据，注入已知成员清单与事件时间"""
video_path = task['video_path']
payload = {
"task_id": str(task['task_id']),
"camera_name": self.camera_name,
"known_members_context": db_layer.get_known_members_context(),
"event_start_time": "",
"event_end_time": "",
}
try:
mtime = os.path.getmtime(video_path)
# mtime 是录制结束时刻，开始时间 = 结束时间 - 视频时长
duration = self._probe_duration(video_path)
start_dt = datetime.fromtimestamp(mtime - duration)
payload["event_start_time"] = start_dt.strftime('%Y-%m-%d %H:%M:%S')
except OSError:
pass
return payload
def _compress_video(self, task_id, video_path):
"""NAS 端预压缩: 480p/CRF28/veryfast，静态监控场景实测 ~36x 压缩比
成功返回压缩文件路径；失败返回 None（回退原始文件分块上传）。
压缩产物缓存在 /tmp/fam_compressed/task_{id}/，重试时源文件未变则复用。
"""
if not self.ffmpeg:
logger.warning(f"[task_id={task_id}] ffmpeg 不可用，跳过预压缩")
return None
self._cleanup_compress_cache()
out_dir = os.path.join(COMPRESS_DIR, f'task_{task_id}')
out_path = os.path.join(out_dir, os.path.basename(video_path))
# 缓存复用：源文件未变且压缩产物有效
try:
if (os.path.isfile(out_path)
and os.path.getsize(out_path) > 0
and os.path.getmtime(out_path) >= os.path.getmtime(video_path)):
logger.info(f"[task_id={task_id}] 复用压缩缓存: {out_path}")
return out_path
except OSError:
pass
os.makedirs(out_dir, exist_ok=True)
src_size = os.path.getsize(video_path)
start = time.time()
# 写临时文件（带 PID 防多进程冲突），成功后原子 rename —
# 服务被 kill 时 ffmpeg 成为孤儿继续写 tmp，缓存目录中
# 只会出现完整产物，杜绝半成品被复用（曾导致上传损坏视频）
tmp_path = f"{out_path}.{os.getpid()}.tmp"
cmd = [
self.ffmpeg, '-y',
'-i', video_path,
# 第二级 scale 向下取偶（h264 要求偶数尺寸；force_divisible_by 需 ffmpeg>=4.3）
'-vf', ('scale=854:480:force_original_aspect_ratio=decrease,'
'scale=trunc(iw/2)*2:trunc(ih/2)*2'),
'-c:v', 'libx264', '-preset', 'veryfast', '-crf', '28',
'-an',
'-f', 'mp4', # .tmp 扩展名无法推断 muxer，必须显式指定
tmp_path,
]
try:
result = subprocess.run(
cmd, capture_output=True, text=True,
timeout=self.compress_timeout,
)
except subprocess.TimeoutExpired:
_safe_remove(tmp_path)
logger.error(f"[task_id={task_id}] 压缩超时 ({self.compress_timeout}s)，回退原始上传")
return None
except OSError as e:
logger.error(f"[task_id={task_id}] 启动 ffmpeg 失败: {e}")
return None
if result.returncode != 0 or not os.path.isfile(tmp_path) or os.path.getsize(tmp_path) == 0:
_safe_remove(tmp_path)
stderr_tail = (result.stderr or '')[-300:]
logger.error(f"[task_id={task_id}] 压缩失败 (rc={result.returncode}): {stderr_tail}")
return None
os.replace(tmp_path, out_path)
dst_size = os.path.getsize(out_path)
elapsed = time.time() - start
logger.info(f"[task_id={task_id}] 预压缩完成: {src_size/1048576:.1f}MB → "
f"{dst_size/1048576:.1f}MB ({src_size/max(dst_size,1):.1f}x)，耗时 {elapsed:.0f}s")
return out_path
@staticmethod
def _cleanup_compress_cache():
"""清理超过 TTL 的压缩缓存目录 + 孤儿 .tmp 残留（tmpfs 空间有限）"""
try:
if not os.path.isdir(COMPRESS_DIR):
return
cutoff = time.time() - COMPRESS_CACHE_TTL
for entry in os.listdir(COMPRESS_DIR):
path = os.path.join(COMPRESS_DIR, entry)
try:
if os.path.isdir(path):
if os.path.getmtime(path) < cutoff:
shutil.rmtree(path, ignore_errors=True)
continue
# 清理超过 1h 的 .tmp 残留（孤儿 ffmpeg 产物）
for fn in os.listdir(path):
if fn.endswith('.tmp') and \
os.path.getmtime(os.path.join(path, fn)) < time.time() - 3600:
_safe_remove(os.path.join(path, fn))
except OSError:
continue
except OSError:
pass
def _dispatch_one(self, task):
"""上传视频至 Edge 异步队列（大文件先预压缩，自动选择直接/分块模式）"""
task_id = task['task_id']
video_path = task['video_path']
if not video_path or not os.path.isfile(video_path):
db_layer.update_task_status(
task_id, 'FAILED',
error_message=f"视频文件不存在: {video_path}",
failure_stage='callback'
)
logger.error(f"[task_id={task_id}] 视频文件不存在，标记 FAILED: {video_path}")
return
file_size = os.path.getsize(video_path)
size_mb = file_size / (1024 * 1024)
db_layer.update_task_status(task_id, 'PROCESSING')
# 跨境上行带宽受限（实测 ~0.5-0.9MB/s），大文件先预压缩再上传
upload_path = video_path
if file_size > COMPRESS_THRESHOLD:
compressed = self._compress_video(task_id, video_path)
if compressed:
upload_path = compressed
file_size = os.path.getsize(upload_path)
size_mb = file_size / (1024 * 1024)
# 压缩耗时较长，重置 stale 计时基准（reclaim 按 updated_at 判断）
db_layer.update_task_status(task_id, 'PROCESSING')
else:
logger.warning(f"[task_id={task_id}] 压缩失败，回退原始文件上传 ({size_mb:.1f}MB)")
payload = self._build_payload(task)
if file_size > CHUNK_THRESHOLD:
logger.info(f"[task_id={task_id}] 大文件分块上传: {size_mb:.1f}MB, "
f"{math.ceil(file_size / CHUNK_SIZE)} 块")
self._dispatch_chunked(task, payload, upload_path, file_size)
else:
log_task(logger, task_id, 'dispatcher',
f'直接上传: {self.edge_url} ({size_mb:.1f}MB)')
self._dispatch_direct(task, payload, upload_path)
def _dispatch_direct(self, task, payload, video_path):
"""小文件直接上传至 /enqueue
注意 timeout 第一参数: urllib3 发送 multipart body 期间 socket
timeout 取的是 connect timeout 值（实测传 60s 则 60s 整超时），
而非 read timeout——跨境 1.4MB/s 下 17MB 需 ~12s，必须给足。
"""
task_id = task['task_id']
try:
with open(video_path, 'rb') as fh:
resp = requests.post(
self.edge_url,
data=payload,
files={'video': (os.path.basename(video_path), fh, 'video/mp4')},
timeout=(120, 300)
)
except requests.RequestException as e:
logger.error(f"[task_id={task_id}] 上传失败: {e}")
self._schedule_retry(task)
return
if resp.status_code == 202:
try:
data = resp.json()
queue_id = data.get('queue_id', '?')
logger.info(f"[task_id={task_id}] 已入 Edge 队列 (queue_id={queue_id})，等待 Poller 拉取结果")
except ValueError:
logger.info(f"[task_id={task_id}] 已入 Edge 队列，等待 Poller 拉取结果")
return
if resp.status_code == 429:
logger.warning(f"[task_id={task_id}] Edge 队列满 (429)，回到 PENDING 稍后重试")
db_layer.update_task_status(task_id, 'PENDING')
return
logger.error(f"[task_id={task_id}] Edge 返回异常状态码: {resp.status_code}")
self._schedule_retry(task)
def _query_uploaded_chunks(self, task_id, expected_total=None):
"""查询 Edge 端已上传分块列表
返回 (uploaded_set, edge_total_chunks)。
如果 expected_total 与 edge_total 不匹配（chunk_size 变更），
返回空集让 Edge 自动清理旧分块。
"""
try:
resp = requests.get(
self.chunks_query_url,
params={'task_id': task_id},
timeout=(30, 15)
)
if resp.status_code == 200:
data = resp.json()
uploaded = set(data.get('uploaded_chunks', []))
edge_total = data.get('total_chunks', 0)
if expected_total and edge_total and edge_total != expected_total:
logger.warning(f"[task_id={task_id}] Edge total_chunks={edge_total} "
f"≠ expected={expected_total}（chunk_size 已变更），从头上传")
return set(), edge_total
return uploaded, edge_total
logger.warning(f"[task_id={task_id}] 查询已上传分块返回 {resp.status_code}，将全量重传")
except requests.RequestException as e:
logger.warning(f"[task_id={task_id}] 查询已上传分块失败（将全量重传）: {e}")
return set(), 0
def _dispatch_chunked(self, task, payload, video_path, file_size):
"""大文件分块上传 + 断点续传
1. 查询 Edge 端已上传分块（断点续传）
2. 上传缺失分块（每块最多重试 3 次，失败后查询 Edge 确认是否实际收到）
3. 全部分块上传后调用 /assemble 合并入队
"""
task_id = task['task_id']
total_chunks = math.ceil(file_size / CHUNK_SIZE)
filename = os.path.basename(video_path)
# 1. 查询已上传分块（断点续传）
uploaded_set, edge_total = self._query_uploaded_chunks(task_id, expected_total=total_chunks)
if uploaded_set:
logger.info(f"[task_id={task_id}] 断点续传: 已有 {len(uploaded_set)}/{total_chunks} 块")
# 2. 上传缺失分块
try:
with open(video_path, 'rb') as fh:
for idx in range(total_chunks):
if idx in uploaded_set:
continue
chunk_data = fh.read(CHUNK_SIZE)
if not chunk_data:
break
success = False
for attempt in range(MAX_CHUNK_RETRIES):
try:
cresp = requests.post(
self.chunk_url,
data={
'task_id': str(task_id),
'chunk_index': str(idx),
'total_chunks': str(total_chunks),
'filename': filename,
},
files={'chunk': (f'chunk_{idx}', io.BytesIO(chunk_data))},
timeout=(60, 180)
)
if cresp.status_code == 200:
success = True
break
logger.warning(f"[task_id={task_id}] 分块 {idx} 返回 {cresp.status_code}（尝试 {attempt+1}/{MAX_CHUNK_RETRIES}）")
except requests.RequestException as e:
logger.warning(f"[task_id={task_id}] 分块 {idx} 上传失败（尝试 {attempt+1}/{MAX_CHUNK_RETRIES}）: {e}")
if attempt < MAX_CHUNK_RETRIES - 1:
time.sleep(5 * (attempt + 1))
if not success:
uploaded_now, _ = self._query_uploaded_chunks(task_id)
if idx in uploaded_now:
logger.info(f"[task_id={task_id}] 分块 {idx} 虽超时但 Edge 已收到，继续下一块")
uploaded_set.add(idx)
continue
logger.error(f"[task_id={task_id}] 分块 {idx} 确认未收到，安排文件级重试")
self._schedule_retry(task)
return
uploaded_set.add(idx)
if (idx + 1) % 5 == 0 or idx == total_chunks - 1:
logger.info(f"[task_id={task_id}] 分块进度: {idx + 1}/{total_chunks}")
except IOError as e:
logger.error(f"[task_id={task_id}] 读取视频文件失败: {e}")
self._schedule_retry(task)
return
# 3. 合并 + 入队
try:
aresp = requests.post(
self.assemble_url,
data={
'task_id': str(task_id),
'camera_name': payload.get('camera_name', ''),
'event_start_time': payload.get('event_start_time', ''),
'known_members_context': payload.get('known_members_context', ''),
},
timeout=(10, 60)
)
except requests.RequestException as e:
logger.error(f"[task_id={task_id}] 合并请求失败: {e}")
self._schedule_retry(task)
return
if aresp.status_code == 202:
try:
data = aresp.json()
queue_id = data.get('queue_id', '?')
asm_size = data.get('size_mb', '?')
logger.info(f"[task_id={task_id}] 分块合并入队成功 (queue_id={queue_id}, {asm_size}MB)，等待 Poller 拉取结果")
except ValueError:
logger.info(f"[task_id={task_id}] 分块合并入队成功，等待 Poller 拉取结果")
return
logger.error(f"[task_id={task_id}] 合并端点返回 {aresp.status_code}: {aresp.text[:200]}")
self._schedule_retry(task)
def _schedule_retry(self, task):
"""调度重试"""
task_id = task['task_id']
if task['retry_count'] >= self.max_retries:
db_layer.update_task_status(
task_id, 'FAILED',
error_message=f"超过最大重试次数 {self.max_retries}",
failure_stage='callback'
)
logger.error(f"[task_id={task_id}] 超过最大重试次数，标记为 FAILED")
return
backoff = self._calculate_backoff(task['retry_count'])
next_retry = datetime.now() + timedelta(seconds=backoff)
db_layer.increment_retry(task_id, next_retry)
logger.info(f"[task_id={task_id}] 安排重试 #{task['retry_count']+1}，{backoff}s 后执行 (at {next_retry})")
def _poll_once(self):
"""执行一次轮询"""
# 回收僵尸任务：PROCESSING 超过 stale_timeout 说明 Edge 丢失了任务
try:
stale_ids = db_layer.reclaim_stale_processing(self.stale_timeout)
for tid in stale_ids:
logger.warning(f"[task_id={tid}] PROCESSING 超时 {self.stale_timeout}s，回收为 PENDING 重试")
except Exception as e:
logger.error(f"僵尸任务回收失败: {e}", exc_info=True)
tasks = db_layer.get_pending_tasks(limit=1)
for task in tasks:
if task['retry_count'] >= task['max_retries']:
db_layer.update_task_status(
task['task_id'], 'FAILED',
error_message=f"超过最大重试次数 {task['max_retries']}",
failure_stage='callback'
)
logger.warning(f"[task_id={task['task_id']}] retry_count={task['retry_count']} >= max_retries={task['max_retries']}，标记 FAILED")
continue
if self._should_retry(task):
try:
self._dispatch_one(task)
except Exception as e:
logger.error(f"[task_id={task['task_id']}] dispatch 异常: {e}", exc_info=True)
def _run(self):
"""线程主循环"""
logger.info(f"Dispatcher 启动 (enqueue + 分块模式)，轮询间隔 {self.poll_interval}s")
while self._running:
try:
self._poll_once()
except Exception as e:
logger.error(f"轮询异常: {e}", exc_info=True)
time.sleep(self.poll_interval)
def start(self):
"""启动下发线程"""
if self._running:
return
self._running = True
self._thread = threading.Thread(target=self._run, daemon=True, name='dispatcher')
self._thread.start()
def is_alive(self):
"""线程是否存活"""
return self._thread is not None and self._thread.is_alive()
def check_and_restart(self):
"""看门狗：线程崩溃后自动重启"""
if self._running and not self.is_alive():
logger.warning("Dispatcher 线程已死亡，正在重启...")
self._thread = threading.Thread(target=self._run, daemon=True, name='dispatcher')
self._thread.start()
def stop(self):
"""停止下发线程"""
self._running = False
if self._thread:
self._thread.join(timeout=5)

4
fam-core/src/fam_core/event_receiver/__init__.py
View File

@@ -1,4 +0,0 @@
"""Event-Receiver 包"""
from .event_receiver import event_bp
__all__ = ["event_bp"]

172
fam-core/src/fam_core/event_receiver/event_receiver.py
View File

@@ -1,172 +0,0 @@
"""
Event-Receiver - Flask 蓝图，接收 Edge 回调，写库
处理逻辑:
1. 成功回调: 插入 monitor_events 1 条 + 遍历 frame_details 逐条插入 event_details
2. frame_details 携带的关键帧 base64 落盘到 fam-ui 静态目录（供时间轴展示）
3. 对未命名的 abstract_label 自动 upsert 到 family_members
4. 更新 process_tasks 状态为 SUCCESS
5. 失败回调: 更新任务状态为 FAILED，记录 failure_stage
"""
import os
import re
import json
import base64
from flask import Blueprint, request, jsonify
from ..logger import setup_logger
from ..config_loader import load_config
from .. import db_layer
logger = setup_logger('fam-core.event_receiver')
event_bp = Blueprint('event_receiver', __name__)
# 关键帧落盘目录（fam-ui 读取展示；fam-core 与 fam-ui 同机部署）
_cfg = load_config()
FRAME_IMAGE_DIR = _cfg.get('storage', {}).get(
'frame_image_dir',
'/volume1/web/sentinel-home-ai/fam-ui/static/frames')
# 匹配 "人物A" / "人物B" 等 abstract_label
_ABSTRACT_LABEL_PATTERN = re.compile(r'^人物[A-Z]$')
def _is_abstract_label(person: str) -> bool:
"""判断是否为未命名的 abstract_label"""
return bool(_ABSTRACT_LABEL_PATTERN.match(person))
def _save_frame_images(event_id: int, frame_details: list) -> int:
"""把 frame_details 中的 base64 关键帧落盘，返回成功张数
同时写 meta.json（frame_index -> face_count），UI 据此挑选有人像的帧做头像。
"""
saved = 0
face_counts = {}
for frame in frame_details:
img_b64 = frame.pop('frame_image', None)
faces = frame.pop('face_count', None)
if not img_b64:
continue
idx = frame.get('frame_index', 0)
try:
out_dir = os.path.join(FRAME_IMAGE_DIR, f'event_{event_id}')
os.makedirs(out_dir, exist_ok=True)
out_path = os.path.join(out_dir, f'frame_{idx}.jpg')
with open(out_path, 'wb') as f:
f.write(base64.b64decode(img_b64))
if faces is not None:
face_counts[str(idx)] = int(faces)
saved += 1
except Exception as e:
logger.warning(f"[event_id={event_id}] 关键帧落盘失败 frame_{idx}: {e}")
if face_counts:
try:
import json as _json
with open(os.path.join(out_dir, 'meta.json'), 'w') as f:
_json.dump(face_counts, f)
except Exception as e:
logger.warning(f"[event_id={event_id}] meta.json 写入失败: {e}")
if saved:
logger.info(f"[event_id={event_id}] 关键帧落盘 {saved} 张 -> {FRAME_IMAGE_DIR}")
return saved
def _upsert_abstract_members(frame_details: list):
"""对未命名的 abstract_label 自动 upsert 到 family_members"""
seen = {}
for frame in frame_details:
person = frame.get('person', '')
if _is_abstract_label(person):
clothing = frame.get('clothing', '')
action = frame.get('action', '')
feature = f"{clothing}、{action}" if clothing and action else clothing or action
timestamp = frame.get('frame_timestamp', '')
if person not in seen:
seen[person] = (feature, timestamp)
for label, (feature, ts) in seen.items():
db_layer.upsert_family_member(label, feature, ts)
logger.info(f"upsert family_member: {label} (feature={feature})")
def apply_success_event(task_id, data: dict) -> int:
"""将成功结果写库，返回 event_id
供两条路径复用:
- webhook 回调路由 (拉取模式)
- Dispatcher 收到推送模式同步响应后直接落库
"""
# 1. 插入 monitor_events
event_id = db_layer.insert_event(
task_id=task_id,
event_start_time=data['event_start_time'],
event_end_time=data['event_end_time'],
camera_name=data.get('camera_name', ''),
global_summary=data.get('global_summary', ''),
entities_json=data.get('entities_json', []),
compute_provider=data.get('compute_provider', [])
)
# 2. 关键帧图片落盘（先落盘再入库：pop 掉 base64 后 insert，避免大字段进 DB）
frame_details = data.get('frame_details', [])
try:
_save_frame_images(event_id, frame_details)
except Exception as e:
logger.warning(f"[event_id={event_id}] 关键帧落盘异常（不影响入库）: {e}")
# 3. 遍历 frame_details 逐条插入
for frame in frame_details:
db_layer.insert_event_detail(
event_id=event_id,
task_id=task_id,
frame_index=frame.get('frame_index', 0),
frame_timestamp=frame.get('frame_timestamp', ''),
camera_name=frame.get('camera_name', data.get('camera_name', '')),
person=frame.get('person', '未知'),
action=frame.get('action', ''),
clothing=frame.get('clothing', ''),
is_attention_event=frame.get('is_attention_event', False),
source_providers=frame.get('source_providers', [])
)
# 3. 对未命名的 abstract_label 自动 upsert
_upsert_abstract_members(frame_details)
# 4. 更新任务状态
db_layer.update_task_status(task_id, 'SUCCESS')
logger.info(f"[task_id={task_id}] 事件处理完成: event_id={event_id}, frame_details={len(frame_details)}条")
return event_id
@event_bp.route('/api/core/callback/event', methods=['POST'])
def receive_event():
"""接收 Edge 回调"""
data = request.get_json(silent=True)
if not data:
return jsonify({"error": "Invalid JSON"}), 400
task_id = data.get('task_id')
status = data.get('status')
logger.info(f"[task_id={task_id}] 收到回调: status={status}")
if status == 'success':
try:
event_id = apply_success_event(task_id, data)
return jsonify({"status": "ok", "event_id": event_id}), 200
except Exception as e:
logger.error(f"[task_id={task_id}] 处理回调失败: {e}", exc_info=True)
db_layer.update_task_status(task_id, 'FAILED', error_message=str(e), failure_stage='callback')
return jsonify({"error": str(e)}), 500
elif status == 'failed':
failure_stage = data.get('failure_stage', '')
error_message = data.get('error_message', '')
db_layer.update_task_status(task_id, 'FAILED', error_message=error_message, failure_stage=failure_stage)
logger.error(f"[task_id={task_id}] 任务失败: stage={failure_stage}, error={error_message}")
return jsonify({"status": "ok"}), 200
else:
return jsonify({"error": f"Unknown status: {status}"}), 400

156
fam-core/src/fam_core/member_manager/member_manager.py
View File

@@ -1,14 +1,20 @@
""" """
Member-Manager - Flask 蓝图，成员命名管理 Member-Manager - Flask 蓝图，人物命名管理（新架构 v2）
1. GET /api/member/unnamed - 列出未命名人物 Oracle 是人物规范的唯一真源。NAS 命名操作:
2. POST /api/member/name - 命名人物 + 批量回溯更新 1. POST /api/member/name 命名/重命名某 label -> 回推 Oracle + 立即拉回
3. GET /api/member/list - 列出所有成员 2. POST /api/member/merge 将两个 label 合并为同一身份（统一 canonical_name）
3. GET /api/member/list 列出所有人物（label + canonical_name）
4. GET /api/member/unnamed 列出未命名人物（canonical_name 为空）
命名流程: 调 Oracle /api/oracle/people/correct 设置 manual 规范名 ->
立即 trigger_now() 拉回最新 people 镜像 -> 前端刷新即见结果。
""" """
from flask import Blueprint, request, jsonify from flask import Blueprint, request, jsonify
from ..logger import setup_logger from ..logger import setup_logger
from .. import db_layer from .. import db_layer
from ..oracle_sync import get_sync
logger = setup_logger('fam-core.member_manager') logger = setup_logger('fam-core.member_manager')
@@ -17,86 +23,122 @@ member_bp = Blueprint('member_manager', __name__)
@member_bp.route('/api/member/unnamed', methods=['GET']) @member_bp.route('/api/member/unnamed', methods=['GET'])
def list_unnamed(): def list_unnamed():
"""列出未命名人物""" """列出未命名人物（canonical_name 为空）"""
members = db_layer.get_unnamed_members() members = db_layer.get_sync_people()
# datetime 序列化
result = [] result = []
for m in members: for m in members:
canonical = m.get('canonical_name')
if not canonical:
result.append({ result.append({
"abstract_label": m['abstract_label'], "label": m['label'],
"feature_description": m['feature_description'], "appearances": m.get('appearances', 0),
"first_seen_at": m['first_seen_at'].isoformat() if hasattr(m['first_seen_at'], 'isoformat') else str(m['first_seen_at']), "first_seen": m.get('first_seen'),
"event_count": m['event_count']
}) })
return jsonify({"unnamed_members": result}), 200 return jsonify({"unnamed_members": result}), 200
@member_bp.route('/api/member/list', methods=['GET'])
def list_members():
"""列出所有人物（按 canonical_name 或 label 展示）"""
members = db_layer.get_sync_people()
result = []
for m in members:
canonical = m.get('canonical_name')
result.append({
"label": m['label'],
"canonical_name": canonical,
"display_name": canonical or m['label'],
"is_named": bool(canonical),
"appearances": m.get('appearances', 0),
"source": m.get('source'),
"first_seen": m.get('first_seen'),
})
return jsonify({"members": result}), 200
@member_bp.route('/api/member/name', methods=['POST']) @member_bp.route('/api/member/name', methods=['POST'])
def name_member(): def name_member():
"""命名人物 + 批量回溯更新历史记录""" """命名人物（回推 Oracle + 立即拉回本地镜像）
请求: {"label": "人物A", "canonical_name": "张三"}
"""
data = request.get_json(silent=True) data = request.get_json(silent=True)
if not data: if not data:
return jsonify({"error": "Invalid JSON"}), 400 return jsonify({"error": "Invalid JSON"}), 400
abstract_label = data.get('abstract_label') label = (data.get('label') or '').strip()
real_name = data.get('real_name') canonical_name = (data.get('canonical_name') or '').strip()
named_by = data.get('named_by', '管理员') if not label or not canonical_name:
return jsonify({"error": "缺少必填字段: label, canonical_name"}), 400
if not abstract_label or not real_name: logger.info(f"命名: {label} -> {canonical_name}（回推 Oracle）")
return jsonify({"error": "缺少必填字段: abstract_label, real_name"}), 400 ok, err = get_sync().push_name_correct(label, canonical_name)
if not ok:
logger.info(f"命名: {abstract_label} -> {real_name}") return jsonify({"error": f"回推 Oracle 失败: {err}"}), 502
# 立即拉回最新 people 镜像，前端无需等待下一个 30 分钟周期
try: try:
result = db_layer.name_member(abstract_label, real_name, named_by) get_sync().trigger_now()
if 'error' in result:
return jsonify(result), 404
return jsonify(result), 200
except Exception as e: except Exception as e:
logger.error(f"命名失败: {e}", exc_info=True) logger.warning(f"命名后即时拉回失败（下一个周期会自动同步）: {e}")
return jsonify({"error": str(e)}), 500
members = db_layer.get_sync_people()
return jsonify({
"status": "ok",
"label": label,
"canonical_name": canonical_name,
"members": [{
"label": m['label'],
"canonical_name": m.get('canonical_name'),
"display_name": m.get('canonical_name') or m['label'],
} for m in members]
}), 200
@member_bp.route('/api/member/merge', methods=['POST']) @member_bp.route('/api/member/merge', methods=['POST'])
def merge_member(): def merge_member():
"""合并人物（用户判断两个标签是同一人时，source 并入 target）""" """合并人物：将 source 并入 target 身份（统一 canonical_name）
若 target 已命名 -> 用其 canonical_name；否则用 target label 作为规范名。
请求: {"source": "人物B", "target": "张三" 或 "人物A"}
"""
data = request.get_json(silent=True) data = request.get_json(silent=True)
if not data: if not data:
return jsonify({"error": "Invalid JSON"}), 400 return jsonify({"error": "Invalid JSON"}), 400
source_key = data.get('source') source = (data.get('source') or '').strip()
target_key = data.get('target') target = (data.get('target') or '').strip()
if not source_key or not target_key: if not source or not target:
return jsonify({"error": "缺少必填字段: source, target"}), 400 return jsonify({"error": "缺少必填字段: source, target"}), 400
if source == target:
return jsonify({"error": "source 与 target 不能相同"}), 400
# 解析 target 的规范名
members = {m['label']: m for m in db_layer.get_sync_people()}
target_row = members.get(target)
if target_row and target_row.get('canonical_name'):
canonical = target_row['canonical_name']
else:
canonical = target # target 未命名 -> 以 label 作为规范名
logger.info(f"合并: {source} -> {canonical}（回推 Oracle）")
ok, err = get_sync().push_name_correct(source, canonical)
if not ok:
return jsonify({"error": f"回推 Oracle 失败: {err}"}), 502
logger.info(f"合并人物: {source_key} -> {target_key}")
try: try:
result = db_layer.merge_member(source_key, target_key) get_sync().trigger_now()
if 'error' in result:
return jsonify(result), 404
return jsonify(result), 200
except Exception as e: except Exception as e:
logger.error(f"合并失败: {e}", exc_info=True) logger.warning(f"合并后即时拉回失败（下一个周期会自动同步）: {e}")
return jsonify({"error": str(e)}), 500
members = db_layer.get_sync_people()
@member_bp.route('/api/member/list', methods=['GET']) return jsonify({
def list_members(): "status": "ok",
"""列出所有成员""" "source": source,
include_named = request.args.get('include_named', 'true').lower() == 'true' "target_canonical": canonical,
include_unnamed = request.args.get('include_unnamed', 'true').lower() == 'true' "members": [{
"label": m['label'],
members = db_layer.get_all_members(include_named, include_unnamed) "canonical_name": m.get('canonical_name'),
result = [] "display_name": m.get('canonical_name') or m['label'],
for m in members: } for m in members]
result.append({ }), 200
"member_id": m['member_id'],
"abstract_label": m['abstract_label'],
"real_name": m['real_name'],
"feature_description": m['feature_description'],
"first_seen_at": m['first_seen_at'].isoformat() if hasattr(m['first_seen_at'], 'isoformat') else str(m['first_seen_at']),
"named_at": m['named_at'].isoformat() if m.get('named_at') and hasattr(m['named_at'], 'isoformat') else None,
"named_by": m.get('named_by'),
"is_active": m['is_active']
})
return jsonify({"members": result}), 200

170
fam-core/src/fam_core/oracle_sync.py Normal file
View File

@@ -0,0 +1,170 @@
"""
Oracle-Sync - NAS 端唯一后台线程
职责:
1. 每 interval_sec（默认 1800s = 30 分钟）从甲骨文 FAM-Edge 拉取增量:
GET {base_url}/api/oracle/sync?since=<cursor>&token=<token>
返回 {videos, events, people, server_time}，写入本地 MariaDB 镜像表
(sync_videos / sync_events / sync_people)，并推进 sync_cursor。
2. 接收命名校正回推: POST {base_url}/api/oracle/people/correct
{label, canonical_name, token} —— 手动命名（manual 优先，不被 LLM 覆盖）。
数据流向（新架构 v2）:
Google 硬盘 --rclone--> 甲骨文本地 --> 整视频分析 --> Oracle SQLite
--> [本线程每 30 分钟拉增量] --> NAS MariaDB 镜像 --> fam-ui 读取展示
NAS 不再处理任何视频，CPU 占用显著降低。
"""
import time
import threading
import requests
from datetime import datetime
from .logger import setup_logger
from .config_loader import load_config
from . import db_layer
logger = setup_logger('fam-core.oracle_sync')
_SYNC_INSTANCE = None
def get_sync():
"""模块级单例（app.py 启动时创建并 start，其余模块经此获取）"""
global _SYNC_INSTANCE
if _SYNC_INSTANCE is None:
_SYNC_INSTANCE = OracleSync()
return _SYNC_INSTANCE
class OracleSync:
def __init__(self):
cfg = load_config().get('oracle_sync', {})
self.base_url = cfg.get('base_url', 'http://129.146.203.203:5000').rstrip('/')
self.token = cfg.get('token', '')
self.interval_sec = int(cfg.get('interval_sec', 1800))
self.timeout = int(cfg.get('timeout', 120))
self._running = False
self._thread = None
self._last_sync_at = None
self._last_error = None
self._last_count = None
# ------------------------------------------------------------------
def _pull_once(self) -> bool:
"""执行一次增量拉取。返回是否成功。"""
since = db_layer.get_sync_cursor() or ''
params = {'since': since, 'token': self.token}
try:
resp = requests.get(
f"{self.base_url}/api/oracle/sync",
params=params, timeout=(10, self.timeout))
except requests.RequestException as e:
self._last_error = f"请求失败: {e}"
logger.error(f"拉取同步失败: {e}")
return False
if resp.status_code == 401:
self._last_error = "token 校验失败"
logger.error("同步 token 校验失败 (401)，请检查 oracle_sync.token 配置")
return False
if resp.status_code != 200:
self._last_error = f"HTTP {resp.status_code}"
logger.error(f"同步返回异常: {resp.status_code} {resp.text[:200]}")
return False
try:
data = resp.json()
except ValueError:
self._last_error = "非 JSON 响应"
logger.error("同步返回非 JSON 响应")
return False
videos = data.get('videos', []) or []
events = data.get('events', []) or []
people = data.get('people', []) or []
server_time = data.get('server_time', '') or ''
n_videos = db_layer.upsert_sync_videos(videos)
n_events = db_layer.upsert_sync_events(events)
n_people = db_layer.upsert_sync_people(people)
if server_time:
db_layer.set_sync_cursor(server_time)
self._last_sync_at = datetime.now()
self._last_error = None
self._last_count = (n_videos, n_events, n_people)
logger.info(
f"同步完成: videos+{n_videos} events+{n_events} people+{n_people} "
f"since={since!r} -> server_time={server_time}")
return True
# ------------------------------------------------------------------
def push_name_correct(self, label: str, canonical_name: str):
"""回推命名校正到 Oracle（手动命名优先级最高，不被 LLM 覆盖）。
返回 (success: bool, error: str)
"""
label = (label or '').strip()
canonical_name = (canonical_name or '').strip()
if not label or not canonical_name:
return False, "缺少 label / canonical_name"
try:
resp = requests.post(
f"{self.base_url}/api/oracle/people/correct",
json={"label": label, "canonical_name": canonical_name,
"token": self.token},
timeout=(10, 30))
except requests.RequestException as e:
logger.error(f"命名校正回推失败: {e}")
return False, str(e)
if resp.status_code == 200:
return True, ""
msg = f"HTTP {resp.status_code}: {resp.text[:200]}"
logger.error(f"命名校正回推失败: {msg}")
return False, msg
# ------------------------------------------------------------------
def _run(self):
logger.info(f"OracleSync 线程启动，间隔 {self.interval_sec}s，目标 {self.base_url}")
while self._running:
try:
self._pull_once()
except Exception as e:
self._last_error = str(e)
logger.error(f"同步异常: {e}", exc_info=True)
# 分段休眠，便于 stop 快速唤醒
for _ in range(self.interval_sec):
if not self._running:
break
time.sleep(1)
def start(self):
if self._running:
return
self._running = True
self._thread = threading.Thread(target=self._run, daemon=True, name='oracle-sync')
self._thread.start()
def is_alive(self):
return self._thread is not None and self._thread.is_alive()
def stop(self):
self._running = False
if self._thread:
self._thread.join(timeout=5)
def trigger_now(self) -> bool:
"""立即触发一次同步（命名后即时拉回 / 手动）。"""
return self._pull_once()
def status(self) -> dict:
return {
"running": self.is_alive(),
"last_sync_at": self._last_sync_at.isoformat() if self._last_sync_at else None,
"last_error": self._last_error,
"last_count": self._last_count,
"cursor": db_layer.get_sync_cursor(),
"interval_sec": self.interval_sec,
}

0
fam-core/src/fam_core/poller/__init__.py
View File

139
fam-core/src/fam_core/poller/poller.py
View File

@@ -1,139 +0,0 @@
"""
Poller - 定期从 Edge 拉取已完成的任务结果，写入 MariaDB
流程:
1. 每 N 秒请求 Edge /api/edge/results?limit=10
2. 遍历结果列表，对每个 nas_task_id:
- success: 调用 apply_success_event 写入 monitor_events + event_details，标记 SUCCESS
- failed: 更新任务状态为 FAILED，记录 failure_stage 和 error_message
3. Edge 端自动标记已拉取的结果为 delivered
"""
import time
import threading
import requests
from ..logger import setup_logger, log_task
from ..config_loader import load_config
from .. import db_layer
from ..event_receiver.event_receiver import apply_success_event
logger = setup_logger('fam-core.poller')
class Poller:
"""结果拉取器，定期从 Edge 拉取处理结果"""
def __init__(self):
cfg = load_config()
poller_cfg = cfg.get('poller', {})
self.poll_interval = poller_cfg.get('poll_interval', 30)
self.results_url = poller_cfg.get('results_url', 'http://localhost:5000/api/edge/results')
self.batch_size = poller_cfg.get('batch_size', 10)
self.timeout = poller_cfg.get('timeout', 30)
self._running = False
self._thread = None
def _handle_result(self, item: dict):
"""处理单个结果"""
nas_task_id = item.get('nas_task_id')
result = item.get('result')
if not nas_task_id:
logger.warning(f"结果缺少 nas_task_id，跳过: {item}")
return
if result is None:
logger.error(f"[task_id={nas_task_id}] Edge 返回空结果，标记 FAILED")
db_layer.update_task_status(
nas_task_id, 'FAILED',
error_message='Edge returned empty result',
failure_stage='callback'
)
return
status = result.get('status')
if status == 'success':
try:
event_id = apply_success_event(nas_task_id, result)
log_task(logger, nas_task_id, 'poller', f'结果落库成功: event_id={event_id}')
except Exception as e:
logger.error(f"[task_id={nas_task_id}] 结果落库失败: {e}", exc_info=True)
db_layer.update_task_status(
nas_task_id, 'FAILED', error_message=str(e), failure_stage='callback')
elif status == 'failed':
error_message = result.get('error_message', 'unknown')
failure_stage = result.get('failure_stage', '')
logger.error(f"[task_id={nas_task_id}] Edge 处理失败: stage={failure_stage}, error={error_message}")
db_layer.update_task_status(
nas_task_id, 'FAILED', error_message=error_message, failure_stage=failure_stage)
else:
logger.warning(f"[task_id={nas_task_id}] 未知状态: {status}")
def _poll_once(self):
"""执行一次拉取"""
try:
resp = requests.get(
self.results_url,
params={'limit': self.batch_size},
timeout=(10, 15)
)
except requests.RequestException as e:
logger.error(f"拉取结果失败: {e}")
return
if resp.status_code != 200:
logger.warning(f"Edge 返回 {resp.status_code}")
return
try:
data = resp.json()
except ValueError:
logger.error("Edge 返回非 JSON 响应")
return
results = data.get('results', [])
if not results:
return
logger.info(f"拉取到 {len(results)} 条结果")
for item in results:
try:
self._handle_result(item)
except Exception as e:
task_id = item.get('nas_task_id', '?')
logger.error(f"[task_id={task_id}] 处理结果异常: {e}", exc_info=True)
def _run(self):
"""线程主循环"""
logger.info(f"Poller 启动，轮询间隔 {self.poll_interval}s，目标: {self.results_url}")
while self._running:
try:
self._poll_once()
except Exception as e:
logger.error(f"轮询异常: {e}", exc_info=True)
time.sleep(self.poll_interval)
def start(self):
"""启动拉取线程"""
if self._running:
return
self._running = True
self._thread = threading.Thread(target=self._run, daemon=True, name='poller')
self._thread.start()
def is_alive(self):
"""线程是否存活"""
return self._thread is not None and self._thread.is_alive()
def check_and_restart(self):
"""看门狗：线程崩溃后自动重启"""
if self._running and not self.is_alive():
logger.warning("Poller 线程已死亡，正在重启...")
self._thread = threading.Thread(target=self._run, daemon=True, name='poller')
self._thread.start()
def stop(self):
"""停止拉取线程"""
self._running = False
if self._thread:
self._thread.join(timeout=5)

4
fam-core/src/fam_core/scheduler/__init__.py
View File

@@ -1,4 +0,0 @@
"""Task-Scheduler 包"""
from .scheduler import TaskScheduler
__all__ = ["TaskScheduler"]

122
fam-core/src/fam_core/scheduler/scheduler.py
View File

@@ -1,122 +0,0 @@
"""
Task-Scheduler - 60s 轮询视频目录，创建 PENDING 任务
判定视频完成（is_complete）:
- 修改时间 > 60s（文件已停止写入）
- 文件大小稳定（连续两次检查大小一致）
"""
import os
import time
import threading
from datetime import datetime
from ..logger import setup_logger, log_task
from ..config_loader import load_config
from .. import db_layer
logger = setup_logger('fam-core.scheduler')
class TaskScheduler:
"""视频目录扫描器，60s 轮询"""
def __init__(self):
cfg = load_config()
self.scan_interval = cfg.get('scheduler', {}).get('scan_interval', 60)
self.video_dir = cfg.get('scheduler', {}).get('video_dir', '/volume1/surveillance')
self.media_base_url = cfg.get('video_server', {}).get('base_url', 'http://127.0.0.1:8000/media')
self.video_extensions = cfg.get('scheduler', {}).get('video_extensions', ['.mp4', '.mkv', '.avi'])
self.file_stable_seconds = cfg.get('scheduler', {}).get('file_stable_seconds', 60)
self.camera_name = cfg.get('scheduler', {}).get('camera_name', '默认摄像头')
# 文件大小缓存，用于判断文件是否稳定
self._file_sizes: dict = {} # path -> size
self._running = False
self._thread = None
def _is_video(self, filename):
return any(filename.lower().endswith(ext) for ext in self.video_extensions)
def _is_complete(self, filepath):
"""判断视频是否已停止写入"""
try:
stat = os.stat(filepath)
now = time.time()
# 修改时间距当前 > stable_seconds
if now - stat.st_mtime < self.file_stable_seconds:
return False
# 文件大小稳定（与上次检查一致）
prev_size = self._file_sizes.get(filepath)
if prev_size is not None and prev_size == stat.st_size:
return True
self._file_sizes[filepath] = stat.st_size
return False
except OSError:
return False
def _build_video_url(self, filepath):
"""构建 Video-Server 下载 URL"""
filename = os.path.basename(filepath)
token = load_config().get('video_server', {}).get('token', '')
return f"{self.media_base_url}/{filename}?token={token}"
def scan_once(self):
"""执行一次扫描"""
if not os.path.isdir(self.video_dir):
logger.warning(f"视频目录不存在: {self.video_dir}")
return
new_count = 0
for root, dirs, files in os.walk(self.video_dir):
for filename in files:
if not self._is_video(filename):
continue
filepath = os.path.join(root, filename)
if not self._is_complete(filepath):
continue
# 检查是否已有任务
if db_layer.get_video_url_exists(filepath):
continue
# 创建新任务
video_url = self._build_video_url(filepath)
task_id = db_layer.create_task(filepath, video_url)
new_count += 1
log_task(logger, task_id, 'scheduler', f'新任务: {filename}')
if new_count > 0:
logger.info(f"本次扫描发现 {new_count} 个新视频")
def _run(self):
"""线程主循环"""
logger.info(f"Task-Scheduler 启动，扫描间隔 {self.scan_interval}s，目录: {self.video_dir}")
while self._running:
try:
self.scan_once()
except Exception as e:
logger.error(f"扫描异常: {e}", exc_info=True)
time.sleep(self.scan_interval)
def start(self):
"""启动调度线程"""
if self._running:
return
self._running = True
self._thread = threading.Thread(target=self._run, daemon=True, name='task-scheduler')
self._thread.start()
def is_alive(self):
"""线程是否存活"""
return self._thread is not None and self._thread.is_alive()
def check_and_restart(self):
"""看门狗：线程崩溃后自动重启"""
if self._running and not self.is_alive():
logger.warning("Scheduler 线程已死亡，正在重启...")
self._thread = threading.Thread(target=self._run, daemon=True, name='task-scheduler')
self._thread.start()
def stop(self):
"""停止调度线程"""
self._running = False
if self._thread:
self._thread.join(timeout=5)

4
fam-core/src/fam_core/video_server/__init__.py
View File

@@ -1,4 +0,0 @@
"""Video-Server 包"""
from .video_server import video_bp
__all__ = ["video_bp"]

52
fam-core/src/fam_core/video_server/video_server.py
View File

@@ -1,52 +0,0 @@
"""
Video-Server - Flask 蓝图，提供 mp4 静态下载
路由带 ?token=xxx 鉴权
无 token 或 token 错误返回 403
文件不存在返回 404
"""
import os
from flask import Blueprint, request, send_from_directory, jsonify
from ..logger import setup_logger
from ..config_loader import load_config
logger = setup_logger('fam-core.video_server')
video_bp = Blueprint('video_server', __name__)
_config = None
def _get_config():
global _config
if _config is None:
_config = load_config()
return _config
@video_bp.route('/media/<path:filename>', methods=['GET'])
def serve_video(filename):
"""提供视频文件下载，带 token 鉴权"""
cfg = _get_config()
token = cfg.get('video_server', {}).get('token', '')
video_dir = cfg.get('video_server', {}).get('video_dir', '/volume1/surveillance')
# 鉴权
req_token = request.args.get('token', '')
if not token or req_token != token:
logger.warning(f"鉴权失败: {filename}, token={req_token}")
return jsonify({"error": "Forbidden"}), 403
# 检查文件
filepath = os.path.join(video_dir, filename)
if not os.path.isfile(filepath):
logger.warning(f"文件不存在: {filepath}")
return jsonify({"error": "Not Found"}), 404
logger.info(f"提供视频: {filename}")
return send_from_directory(
os.path.dirname(filepath),
os.path.basename(filepath),
as_attachment=True
)

164
fam-core/tools/backfill_frames.py
View File

@@ -1,164 +0,0 @@
"""为历史事件补抽关键帧图（视频仍存于 NAS 时）
Edge 注入关键帧 base64 上线前落库的事件没有帧图，本脚本从原始视频
按 frame_timestamp - event_start_time 偏移重新抽帧，补齐到 UI 静态目录。
用法:
venv/bin/python tools/backfill_frames.py [--dry-run] [--event-id N]
特性:
- 幂等: 单帧文件已存在即跳过，帧数齐全的事件整条跳过
- NAS ffmpeg41 无 image2 muxer，必须用 -f singlejpeg 输出 jpg
- 抽帧尺寸与 NAS 预压缩一致（480p 等比缩放），-q:v 5 约 60-100KB/张
"""
import os
import re
import sys
import argparse
import subprocess
from datetime import datetime
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', 'src'))
import pymysql
from fam_core.config_loader import load_config
FFMPEG = '/var/packages/CodecPack/target/bin/ffmpeg41'
def video_duration(path: str) -> float:
"""解析 ffmpeg header 里的 Duration（无 ffprobe 环境）"""
try:
r = subprocess.run([FFMPEG, '-i', path], capture_output=True, text=True, timeout=60)
m = re.search(r'Duration:\s*(\d+):(\d+):(\d+)', r.stderr)
if m:
h, mi, s = (int(x) for x in m.groups())
return h * 3600 + mi * 60 + s
except Exception:
pass
return 0.0
def extract_frame(video: str, offset: float, out_path: str) -> bool:
cmd = [
FFMPEG, '-y',
'-ss', f'{offset:.1f}',
'-i', video,
'-frames:v', '1',
'-vf', 'scale=854:480:force_original_aspect_ratio=decrease,scale=trunc(iw/2)*2:trunc(ih/2)*2',
'-q:v', '5',
'-f', 'singlejpeg',
out_path,
]
try:
r = subprocess.run(cmd, capture_output=True, timeout=120)
return r.returncode == 0 and os.path.getsize(out_path) > 1024
except Exception:
return False
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--dry-run', action='store_true', help='只统计不落盘')
ap.add_argument('--event-id', type=int, default=None, help='只处理指定事件')
args = ap.parse_args()
cfg = load_config()
frame_dir = cfg.get('storage', {}).get(
'frame_image_dir', '/volume1/web/sentinel-home-ai/fam-ui/static/frames')
db = cfg['database']
conn = pymysql.connect(
host=db.get('host', '127.0.0.1'),
port=db.get('port', 3306),
user=db.get('user', 'root'),
password=db.get('password', ''),
database=db.get('database', 'sentinel_home_ai'),
unix_socket=db.get('unix_socket'),
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor,
)
try:
with conn.cursor() as cur:
sql = """
SELECT me.event_id, me.task_id, me.event_start_time, pt.video_path,
(SELECT COUNT(*) FROM event_details ed
WHERE ed.event_id = me.event_id) AS detail_count
FROM monitor_events me
LEFT JOIN process_tasks pt ON pt.task_id = me.task_id
"""
params = ()
if args.event_id:
sql += ' WHERE me.event_id = %s'
params = (args.event_id,)
sql += ' ORDER BY me.event_id'
cur.execute(sql, params)
events = cur.fetchall()
stat = {'skip_complete': 0, 'skip_no_video': 0, 'extracted': 0, 'failed': 0, 'events': 0}
for ev in events:
eid = ev['event_id']
edir = os.path.join(frame_dir, f'event_{eid}')
existing = {f for f in os.listdir(edir) if f.endswith('.jpg')} if os.path.isdir(edir) else set()
with conn.cursor() as cur:
cur.execute(
"""SELECT frame_index, frame_timestamp FROM event_details
WHERE event_id = %s ORDER BY frame_index""",
(eid,))
frames = cur.fetchall()
missing = [f for f in frames if f'frame_{f["frame_index"]}.jpg' not in existing]
if not missing:
stat['skip_complete'] += 1
continue
video = ev.get('video_path') or ''
source = '原始视频'
if not video or not os.path.isfile(video):
# 原始视频被监控保留策略清理时，回退到 dispatcher 压缩缓存（480p 副本）
cached = os.path.join(
f'/tmp/fam_compressed/task_{ev["task_id"]}', os.path.basename(video)) if video else ''
if cached and os.path.isfile(cached):
video = cached
source = '压缩缓存'
if not video or not os.path.isfile(video):
print(f'[event {eid}] 原始视频与压缩缓存均不存在，跳过 {len(missing)} 帧: {video}')
stat['skip_no_video'] += 1
continue
dur = video_duration(video)
start = ev['event_start_time']
if isinstance(start, str):
start = datetime.strptime(start[:19], '%Y-%m-%d %H:%M:%S')
os.makedirs(edir, exist_ok=True)
stat['events'] += 1
print(f'[event {eid}] 补 {len(missing)}/{len(frames)} 帧 · {source} (视频 {os.path.basename(video)}, {dur:.0f}s)')
for f in missing:
out_path = os.path.join(edir, f'frame_{f["frame_index"]}.jpg')
ts = f['frame_timestamp']
if isinstance(ts, str):
ts = datetime.strptime(ts[:19], '%Y-%m-%d %H:%M:%S')
offset = (ts - start).total_seconds()
offset = max(1.0, min(offset, max(1.0, dur - 2)))
if args.dry_run:
print(f' dry-run frame_{f["frame_index"]} @ {offset:.0f}s')
continue
ok = extract_frame(video, offset, out_path)
stat['extracted' if ok else 'failed'] += 1
if not ok:
print(f' 失败 frame_{f["frame_index"]} @ {offset:.0f}s')
if os.path.exists(out_path):
os.remove(out_path)
print(f"\n完成: 事件 {stat['events']} 个已补 | 抽帧成功 {stat['extracted']} 失败 {stat['failed']} "
f"| 齐全跳过 {stat['skip_complete']} | 视频缺失跳过 {stat['skip_no_video']}")
finally:
conn.close()
if __name__ == '__main__':
main()

118
fam-core/tools/backfill_mark_frames.py
View File

@@ -1,118 +0,0 @@
#!/usr/bin/env python3
"""存量关键帧批量补红框（计算在 Edge/Oracle，NAS 只编排与存图）
流程: 遍历 frames/event_*/frame_*.jpg -> 分批(8张)上传 Edge /api/edge/mark_frames
-> 用标记后的图覆盖原文件 -> 写 meta.json (frame_index -> face_count)
幂等: 已有 meta.json 的事件跳过；--force 强制重跑
备份: 首次覆盖前原文件备份到 frames_orig/event_*/
"""
import argparse
import base64
import json
import os
import shutil
import sys
import requests
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'src'))
from fam_core.config_loader import load_config # noqa: E402
FRAME_DIR = load_config().get('storage', {}).get(
'frame_image_dir', '/volume1/web/sentinel-home-ai/fam-ui/static/frames')
MARK_URL = load_config().get('poller', {}).get(
'results_url', 'http://129.146.203.203:5000/api/edge/results'
).rsplit('/', 1)[0] + '/mark_frames'
BATCH = 8
def mark_batch(images):
"""images: [(key, jpeg_bytes)] -> {key: (marked_bytes, faces)}"""
payload = {
'images': [
{'key': k, 'data': base64.b64encode(b).decode('ascii')}
for k, b in images
]
}
resp = requests.post(MARK_URL, json=payload, timeout=(30, 120))
resp.raise_for_status()
out = {}
for r in resp.json().get('results', []):
if r.get('data'):
out[r['key']] = (base64.b64decode(r['data']), r.get('faces', 0))
else:
out[r['key']] = (None, 0)
return out
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--force', action='store_true', help='忽略已有 meta.json 重跑')
ap.add_argument('--event', type=int, help='只处理指定 event_id')
args = ap.parse_args()
orig_root = os.path.join(os.path.dirname(FRAME_DIR.rstrip('/')), 'frames_orig')
stat = {'events': 0, 'frames': 0, 'faces': 0, 'skip': 0, 'fail': 0}
for name in sorted(os.listdir(FRAME_DIR)):
if not name.startswith('event_'):
continue
eid = int(name.split('_')[1])
if args.event and eid != args.event:
continue
edir = os.path.join(FRAME_DIR, name)
meta_path = os.path.join(edir, 'meta.json')
frames = sorted(
f for f in os.listdir(edir)
if f.startswith('frame_') and f.endswith('.jpg'))
if not frames:
continue
if os.path.exists(meta_path) and not args.force:
stat['skip'] += 1
continue
# 备份原件（一次）
bak_dir = os.path.join(orig_root, name)
if not os.path.isdir(bak_dir):
os.makedirs(bak_dir, exist_ok=True)
for f in frames:
shutil.copy2(os.path.join(edir, f), os.path.join(bak_dir, f))
face_counts = {}
for i in range(0, len(frames), BATCH):
batch = []
for f in frames[i:i + BATCH]:
with open(os.path.join(edir, f), 'rb') as fh:
batch.append((f, fh.read()))
try:
marked = mark_batch(batch)
except Exception as e:
print(f'[event {eid}] 批次失败 (跳过 {len(batch)} 帧): {e}')
stat['fail'] += len(batch)
continue
for f, _ in batch:
data, faces = marked.get(f, (None, 0))
if data is None:
stat['fail'] += 1
continue
with open(os.path.join(edir, f), 'wb') as fh:
fh.write(data)
idx = f[len('frame_'):-len('.jpg')]
face_counts[idx] = faces
stat['frames'] += 1
stat['faces'] += faces
if face_counts:
with open(meta_path, 'w') as fh:
json.dump(face_counts, fh)
stat['events'] += 1
print(f'[event {eid}] 标记 {len(face_counts)}/{len(frames)} 帧, '
f'人脸合计 {sum(face_counts.values())}')
print(f"\n完成: 事件 {stat['events']} | 帧标记 {stat['frames']} "
f"(含人脸帧人脸数 {stat['faces']}) | 已标跳过 {stat['skip']} | 失败 {stat['fail']}")
print(f'备份目录: {orig_root}')
if __name__ == '__main__':
main()

101
fam-edge/config/config.yaml
View File

@@ -1,78 +1,58 @@
# FAM-Edge 配置文件 (Oracle 端) - 多模型池配置 # FAM-Edge 配置文件 (Oracle 端) - 新架构 v2
# Tailscale: Oracle=100.74.137.126, NAS=100.70.234.39
# #
# 异步队列模式: # 新架构（2026-08-21 重构）:
# NAS 上传视频 → /api/edge/video/enqueue 入 SQLite 队列 → 消费者线程异步处理 # 1. 不再切片/抽帧：整视频直传云端 VLM（Gemini 用 Files API，NVIDIA 用整视频 video_url）
# → NAS Poller 从 /api/edge/results 拉取结果 # 2. 视频来源：rclone 从 Google 硬盘实时同步到本地 local_dir，监听目录处理新视频
# 速率限制: Gemini 1000RPM x2 burst, NVIDIA 40RPM x2 burst # 3. Oracle 自建 SQLite 库存储所有视频摘要/事件/人物，并对外提供同步接口供 NAS 拉取
# 4. 独立 person_service 汇总全量人物 -> LLM 合并为规范人物表 -> 回灌视频提示
# 5. NAS 仅作管理后台，每 30 分钟从甲骨文拉增量镜像到本地 MariaDB
# NAS 端回调地址（旧 webhook 模式保留，异步模式不使用） # Oracle 端 HTTP 服务
nas:
webhook_url: "http://100.70.234.39:8000/api/core/callback/event"
media_base_url: "http://100.70.234.39:8000/media"
media_token: "sentinel-media-2026"
# Oracle 端服务
server: server:
host: "0.0.0.0" host: "0.0.0.0"
port: 5000 port: 5000
max_concurrent_tasks: 1 max_concurrent_tasks: 1
# 异步任务队列 # Google 硬盘同步（rclone 负责同步落地，本段仅描述监听行为）
queue: gdrive_sync:
db_path: "/opt/fam-edge/data/fam_queue.db" enabled: true
upload_dir: "/tmp/fam_uploads" local_dir: "/opt/fam-edge/gdrive_videos" # rclone 同步落地目录（video_processing 监听此目录）
poll_interval: 10 # 消费者轮询间隔（秒） watch_interval_sec: 30 # 监听新视频的轮询间隔
# API 速率限制 (RPM)，burst_factor=2 表示突发容量为 2 倍 RPM camera_name: "客厅" # 摄像头名称（注入视频提示）
rate_limit: # 文件名解析开始时间：监控文件名含时间戳时使用（如 2026-08-21_081500.mp4）
gemini_rpm: 1000 parse_start_from_filename: true
nvidia_rpm: 40
burst_factor: 2
# 编排调度模式: fallback(顺序降级, 默认) | ensemble(并行交叉验证) # Oracle 本地库（视频摘要/事件/人物）
orchestrator: oracle_db:
mode: "fallback" path: "/opt/fam-edge/data/oracle.db"
overall_timeout: 600
# 关键帧筛选参数（自适应：帧数随视频时长动态计算） # NAS 拉取同步接口鉴权 token（与 NAS oracle_sync.token 一致）
video: sync_api:
candidate_per_minute: 2 # 每分钟粗抽候选帧数 token: "${ORACLE_SYNC_TOKEN}"
candidate_min: 30 # 候选帧下限（短视频保底）
candidate_max: 120 # 候选帧上限（超长视频截断）
key_frame_interval_sec: 150 # 关键帧间隔（秒），每2.5分钟1张
min_key_frames: 5 # 关键帧下限（帧差不足时补足到此数）
max_key_frames_floor: 8 # 关键帧上限的下限（短视频保底）
max_key_frames_cap: 30 # 关键帧上限（超长视频截断）
mse_threshold: 500
jpeg_quality: 80
max_long_edge: 1024
# 超时（秒） # 人物识别服务
timeout: person_service:
download: 60 enabled: true
vlm_visual: 600 schedule_interval_sec: 1800 # 每 30 分钟重新汇总一次人物
vlm_fusion: 300 model: "gemini" # 用哪个模型做人物合并（vision 模型也支持纯文本）
callback: 30
overall: 1800
# 多模型池配置（新框架：本地大模型不参与视频分析，仅智能问答兜底） # 视频处理
# video_processing:
# 视频分析链路（推送模式）: max_concurrent: 1
# 云端 VLM 直接产出结构化 JSON (global_summary / entities_json / frame_details) timeout: 900 # 单视频分析超时（整视频上云较慢）
# -> Edge 仅做格式化/校验 (format_cloud_result) -> 直接回写 NAS，无本地融合步骤 # 降级顺序：先 gemini 整视频，失败再 nvidia 整视频；两者都失败 -> 标记 failed
# 视觉角色: Gemini(主) -> NVIDIA NIM(备) 顺序降级; 两云端全失败 -> 任务 FAILED 走重试 vision_order: ["gemini", "nvidia"]
#
# 智能问答链路: # 智能问答降级链（与视频分析独立）：Gemini -> NVIDIA -> 本地 Ollama
# Gemini -> NVIDIA -> 本地 Ollama (仅当两云端都失败才启用本地兜底)
models: models:
- provider: "gemini" - provider: "gemini"
role: "vision" role: "vision"
enabled: true enabled: true
model_name: "gemini-flash-latest" # 主模型（每日免费配额 20 请求，按模型独立） model_name: "gemini-flash-latest" # 主模型（每日免费配额 20 请求，按模型独立）
fallback_models: # 429 配额耗尽/503 过载时依次切换 fallback_models:
- "gemini-flash-lite-latest" - "gemini-flash-lite-latest"
api_key: "${GEMINI_API_KEY}" api_key: "${GEMINI_API_KEY}"
timeout: 90 timeout: 600
circuit_breaker: circuit_breaker:
enabled: true enabled: true
threshold: 5 threshold: 5
@@ -81,16 +61,17 @@ models:
- provider: "nvidia" - provider: "nvidia"
role: "vision" role: "vision"
enabled: true enabled: true
model_name: "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning" # Omni 原生视频输入；llama-3.2-11b-vision 仅逐帧 # Nemotron Nano 12B v2 VL：NIM 官方支持整视频 video_url 输入（内部自行采样帧）
model_name: "nvidia/nemotron-nano-12b-v2-vl"
base_url: "https://integrate.api.nvidia.com/v1" base_url: "https://integrate.api.nvidia.com/v1"
api_key: "${NVIDIA_API_KEY}" api_key: "${NVIDIA_API_KEY}"
timeout: 120 timeout: 600
circuit_breaker: circuit_breaker:
enabled: true enabled: true
threshold: 5 threshold: 5
cooldown: 300 cooldown: 300
# 本地模型：纯文本 qwen2.5:7b，仅参与智能问答，作为 Gemini/NVIDIA 都失败时的兜底 # 本地模型：纯文本 qwen2.5:7b，仅参与智能问答兜底
- provider: "ollama" - provider: "ollama"
role: "text" role: "text"
usage: "qa_fallback" usage: "qa_fallback"

534
fam-edge/src/fam_edge/ai_orchestrator/orchestrator.py
View File

@@ -1,534 +0,0 @@
"""
AI-Orchestrator - 多模型编排
视频分析链路（新框架）:
1. 加载所有启用的模型适配器
2. 健康检查
3. 抽帧 + 关键帧筛选 + 压缩
4. 云端 VLM 视觉分析（Gemini 主 / NVIDIA 兜底），直出结构化 JSON
5. format_cloud_result：对云端结果做**格式化/校验**（无本地模型调用，不汇总摘要）
6. 同步返回 NAS → 落库
智能问答链路（新框架）:
- run_qa：Gemini → NVIDIA → 本地 Ollama（仅当两云端都失败才用本地兜底）
"""
import time
import json
import base64
import requests
from datetime import datetime, timedelta
from concurrent.futures import ThreadPoolExecutor, as_completed, TimeoutError as FuturesTimeout
from typing import Dict, List, Optional, Tuple
from ..logger import setup_logger, log_task
from ..config_loader import load_config
from ..model_adapters.adapter_factory import build_adapters
from ..model_adapters.base_adapter import BaseModelAdapter
from ..video_preprocessor.preprocessor import VideoPreprocessor
from .json_parser import VLMOutputInvalidError, validate_schema
logger = setup_logger('fam-edge.orchestrator')
class AIOrchestrator:
"""AI 编排器"""
def __init__(self):
self.config = load_config()
self.adapters: List[BaseModelAdapter] = build_adapters(self.config.get('models', []))
self.timeout_cfg = self.config.get('timeout', {})
def health_check_all(self) -> List[BaseModelAdapter]:
"""健康检查，返回健康的适配器列表"""
healthy = []
for adapter in self.adapters:
try:
if adapter.health_check():
healthy.append(adapter)
except Exception as e:
logger.error(f"适配器 {adapter.provider_name} 健康检查异常: {e}")
return healthy
def run_visual_analysis(self, adapters: List[BaseModelAdapter],
frame_paths: List[str],
frame_timestamps: List[str],
known_members_context: str,
rate_limiter=None,
video_path: str = None,
event_start_time: str = '') -> Dict[str, dict]:
"""视觉分析阶段：仅 role=vision 的适配器参与
支持 analyze_video 的适配器（如 NVIDIA Omni）优先走原生视频输入，
失败自动降级回逐帧图片模式。
orchestrator.mode:
- fallback (默认): 按 config 顺序依次尝试，首个成功即采用（单元素 dict）
- ensemble: 并行所有健康 vision 模型，全部成功结果都保留（交叉验证）
rate_limiter: 可选 RateLimiter 实例，按 provider 限速（2x burst）
"""
vision_adapters = [a for a in adapters if getattr(a, 'role', 'vision') == 'vision']
if not vision_adapters:
logger.error("没有 vision 角色的可用适配器")
return {}
mode = self.config.get('orchestrator', {}).get('mode', 'fallback')
if mode == 'ensemble':
return self._run_visual_ensemble(
vision_adapters, frame_paths, frame_timestamps,
known_members_context, rate_limiter)
# fallback: 顺序降级，首个成功即采用
model_outputs = {}
for adapter in vision_adapters:
if adapter.get_circuit_breaker().is_open():
logger.warning(f"[{adapter.provider_name}] 熔断器 OPEN，跳过")
continue
# 速率限制：按 provider 获取 token（2x burst）
if rate_limiter:
acquired = rate_limiter.acquire(adapter.provider_name, timeout=300)
if not acquired:
logger.warning(f"[{adapter.provider_name}] 速率限制超时，跳过")
continue
start = time.time()
try:
output = None
if video_path and hasattr(adapter, 'analyze_video'):
try:
logger.info(f"[{adapter.provider_name}] 尝试原生视频输入分析")
output = adapter.analyze_video(
video_path, frame_timestamps, known_members_context,
event_start_time=event_start_time)
if not output:
logger.warning(f"[{adapter.provider_name}] 视频模式失败，降级逐帧模式")
except Exception as ve:
logger.warning(f"[{adapter.provider_name}] 视频模式异常: {ve}，降级逐帧模式")
output = None
if not output:
output = adapter.analyze_frames(
frame_paths, frame_timestamps, known_members_context)
duration_ms = int((time.time() - start) * 1000)
if output:
adapter.get_circuit_breaker().record_success()
log_task(logger, 0, f'model_{adapter.provider_name}',
f'视觉分析成功', duration_ms=duration_ms)
model_outputs[adapter.provider_name] = output
logger.info(f"fallback 采用 [{adapter.provider_name}]，停止降级")
break
else:
adapter.get_circuit_breaker().record_failure()
logger.warning(f"[{adapter.provider_name}] 视觉分析返回空，降级下一模型")
except Exception as e:
logger.error(f"[{adapter.provider_name}] 视觉分析异常: {e}")
adapter.get_circuit_breaker().record_failure()
return model_outputs
def _run_visual_ensemble(self, vision_adapters, frame_paths,
frame_timestamps, known_members_context,
rate_limiter=None) -> Dict[str, dict]:
"""并行调用所有健康 vision 模型，保留全部成功结果（交叉验证）"""
model_outputs = {}
max_timeout = max((a.get_timeout() for a in vision_adapters), default=240)
with ThreadPoolExecutor(max_workers=len(vision_adapters)) as pool:
futures = {}
for adapter in vision_adapters:
if adapter.get_circuit_breaker().is_open():
logger.warning(f"[{adapter.provider_name}] 熔断器 OPEN，跳过")
continue
# 速率限制：按 provider 获取 token（2x burst）
if rate_limiter:
acquired = rate_limiter.acquire(adapter.provider_name, timeout=300)
if not acquired:
logger.warning(f"[{adapter.provider_name}] 速率限制超时，跳过")
continue
future = pool.submit(
adapter.analyze_frames,
frame_paths, frame_timestamps, known_members_context)
futures[future] = adapter.provider_name
for future in as_completed(futures, timeout=max_timeout + 10):
provider = futures[future]
start = time.time()
try:
adapter = next(a for a in vision_adapters if a.provider_name == provider)
output = future.result(timeout=adapter.get_timeout())
duration_ms = int((time.time() - start) * 1000)
if output:
model_outputs[provider] = output
adapter.get_circuit_breaker().record_success()
log_task(logger, 0, f'model_{provider}',
f'视觉分析成功，输出长度={len(output)}', duration_ms=duration_ms)
else:
adapter.get_circuit_breaker().record_failure()
logger.warning(f"[{provider}] 视觉分析返回空")
except FuturesTimeout:
logger.warning(f"[{provider}] 视觉分析超时")
adapter = next(a for a in vision_adapters if a.provider_name == provider)
adapter.get_circuit_breaker().record_failure()
except Exception as e:
logger.error(f"[{provider}] 视觉分析异常: {e}")
adapter = next(a for a in vision_adapters if a.provider_name == provider)
adapter.get_circuit_breaker().record_failure()
return model_outputs
@staticmethod
def _attach_frame_images(frame_details: List[dict], frame_paths: List[str]) -> None:
"""把关键帧图片 base64 附加到 frame_details（按位置对齐视觉分析输入帧）
附带人脸红框标记与 face_count（NAS 落盘 meta.json，UI 据此挑有人像的头像）
"""
from ..frame_marker import mark_jpeg
for i, fd in enumerate(frame_details):
if i >= len(frame_paths):
break
try:
with open(frame_paths[i], 'rb') as f:
raw = f.read()
marked, faces = mark_jpeg(raw)
fd['frame_image'] = base64.b64encode(marked).decode('ascii')
fd['face_count'] = faces
except OSError as e:
logger.warning(f"关键帧图片读取失败: {frame_paths[i]}: {e}")
def format_cloud_result(self, provider: str, raw_result: dict,
known_members_context: str = '',
task_id: int = 0) -> dict:
"""格式化云端 VLM 直出的结构化结果（**无本地模型调用**）。
- 云端模型已产出结构化数据（frame_details / 可选 global_summary / entities_json）
- 本方法仅做：字段归一化、source_providers 与 compute_provider 填充、
entities 推导、global_summary 缺失时格式化生成
- 解析/校验失败抛 VLMOutputInvalidError
"""
if not isinstance(raw_result, dict):
raise VLMOutputInvalidError("云端视觉模型未返回结构化数据(dict)")
data = dict(raw_result)
frame_details = data.get('frame_details')
if not isinstance(frame_details, list) or not frame_details:
raise VLMOutputInvalidError("云端结果缺少非空的 frame_details")
# 归一化每条 frame_detail
normalized = []
for f in frame_details:
if not isinstance(f, dict):
continue
sp = f.get('source_providers')
if not isinstance(sp, list) or not sp:
sp = [provider]
normalized.append({
"frame_index": int(f.get("frame_index", len(normalized) + 1)),
"frame_timestamp": str(f.get("frame_timestamp", "")),
"person": str(f.get("person", "无人")),
"action": str(f.get("action", "")),
"clothing": str(f.get("clothing", "")),
"is_attention_event": bool(f.get("is_attention_event", False)),
"source_providers": [str(p) for p in sp],
})
if not normalized:
raise VLMOutputInvalidError("frame_details 解析后为空")
data['frame_details'] = normalized
# compute_provider：本次实际成功的云端模型
data['compute_provider'] = [provider]
# entities_json：缺失时由 frame_details 推导（按人物去重）
if not data.get('entities_json'):
seen = set()
ents = []
for f in normalized:
p = f['person']
if p and p != '无人' and p not in seen:
seen.add(p)
ents.append({
"person": p,
"action": f['action'],
"clothing": f['clothing'],
})
data['entities_json'] = ents
# global_summary：云端未给则格式化生成（非 LLM 汇总，仅拼接事实）
if not data.get('global_summary'):
data['global_summary'] = self._build_summary_from_frames(normalized)
return validate_schema(data)
def _build_summary_from_frames(self, frame_details: List[dict]) -> str:
"""当云端模型未提供 global_summary 时，由 frame_details 格式化生成摘要。
注意：这是确定性事实拼接，非 LLM 二次汇总。"""
persons = {}
has_attention = False
for f in frame_details:
p = f['person']
if p and p != '无人':
persons.setdefault(p, set()).add(f['action'])
if f.get('is_attention_event'):
has_attention = True
if not persons:
summary = "整个时段内画面中未检测到人物出现，主要为环境静态画面。"
else:
parts = []
for p, acts in persons.items():
acts_desc = "、".join(sorted(a for a in acts if a)) or "无明显动作"
parts.append(f"{p}（{acts_desc}）")
summary = f"时段内检测到：{'；'.join(parts)}。"
if has_attention:
summary += " ⚠️ 存在需关注的异常事件。"
return summary
def run_qa(self, prompt: str, max_tokens: int = 512) -> Tuple[Optional[str], Optional[str]]:
"""智能问答编排：Gemini → NVIDIA → 本地 Ollama（仅当两云端都失败才用本地兜底）。
返回 (answer, provider)；全部失败返回 (None, None)。
"""
qa_order = ['gemini', 'nvidia', 'ollama']
for name in qa_order:
adapter = next((a for a in self.adapters if a.provider_name == name), None)
if adapter is None:
logger.warning(f"[qa] 未配置模型 {name}，跳过")
continue
try:
if adapter.get_circuit_breaker().is_open():
logger.warning(f"[qa] {name} 熔断器 OPEN，跳过")
continue
answer = adapter.chat(prompt, max_tokens=max_tokens)
if answer:
logger.info(f"[qa] 由 {name} 回答（长度={len(answer)}）")
return answer, name
logger.warning(f"[qa] {name} 返回空")
except Exception as e:
logger.error(f"[qa] {name} 调用异常: {e}")
return None, None
def send_callback(self, webhook_url: str, task_id: int,
result: dict, camera_name: str = '',
event_start_time: str = '', event_end_time: str = ''):
"""回调 NAS"""
payload = {
"task_id": task_id,
"status": "success",
"event_start_time": event_start_time,
"event_end_time": event_end_time,
"camera_name": camera_name,
"global_summary": result.get('global_summary', ''),
"entities_json": result.get('entities_json', []),
"frame_details": result.get('frame_details', []),
"compute_provider": result.get('compute_provider', []),
"error_message": None
}
callback_timeout = self.timeout_cfg.get('callback', 30)
max_retries = 3
for attempt in range(max_retries):
try:
resp = requests.post(webhook_url, json=payload, timeout=callback_timeout)
if resp.status_code == 200:
log_task(logger, task_id, 'callback', '回调成功')
return
else:
logger.warning(f"[task_id={task_id}] 回调返回 {resp.status_code}，重试 {attempt+1}/{max_retries}")
except Exception as e:
logger.warning(f"[task_id={task_id}] 回调异常: {e}，重试 {attempt+1}/{max_retries}")
raise Exception(f"回调失败，已重试 {max_retries} 次")
def send_failure_callback(self, webhook_url: str, task_id: int,
failure_stage: str, error_message: str):
"""发送失败回调"""
payload = {
"task_id": task_id,
"status": "failed",
"failure_stage": failure_stage,
"error_message": error_message
}
try:
requests.post(webhook_url, json=payload, timeout=30)
except Exception as e:
logger.error(f"[task_id={task_id}] 失败回调也失败: {e}")
def process_task(self, task_data: dict):
"""端到端处理任务（拉取模式，webhook 回调）"""
task_id = task_data.get('task_id')
video_url = task_data.get('video_url')
webhook_url = task_data.get('webhook_url')
known_members = task_data.get('known_members_context', '')
logger.info(f"[task_id={task_id}] ====== 开始处理任务 ======")
start_time = time.time()
# 1. 健康检查
healthy_adapters = self.health_check_all()
if not healthy_adapters:
logger.error(f"[task_id={task_id}] 所有模型不健康，返回 503")
self.send_failure_callback(webhook_url, task_id, 'vlm_visual', 'All models unhealthy')
return 503
# 2. 下载 + 抽帧
preprocessor = VideoPreprocessor(task_id)
try:
# 下载
video_path = preprocessor.download_video(video_url)
# 抽帧
candidate_frames = preprocessor.extract_candidate_frames(video_path)
if not candidate_frames:
raise Exception("抽帧失败，无候选帧")
# 关键帧筛选
key_frames = preprocessor.select_key_frames(candidate_frames)
# 压缩
compressed_frames = preprocessor.compress_frames(key_frames)
if not compressed_frames:
raise Exception("压缩后无可用帧")
# 计算时间戳
event_start_time = task_data.get('event_start_time', '')
frame_timestamps = preprocessor.compute_timestamps(
video_path, len(compressed_frames), event_start_time
)
# 3. 并行视觉分析
model_outputs = self.run_visual_analysis(
healthy_adapters, compressed_frames, frame_timestamps,
known_members, video_path=video_path,
event_start_time=event_start_time
)
if not model_outputs:
raise Exception('All models failed in visual analysis')
# 4. 云端直出结果格式化（无本地融合）
provider = next(iter(model_outputs))
fusion_result = self.format_cloud_result(
provider, model_outputs[provider], known_members, task_id)
# 5. 回调
# 从视频文件名推断 camera_name
camera_name = task_data.get('camera_name', '')
event_end_time = task_data.get('event_end_time', '')
self.send_callback(
webhook_url, task_id, fusion_result,
camera_name=camera_name,
event_start_time=event_start_time,
event_end_time=event_end_time
)
total_ms = int((time.time() - start_time) * 1000)
log_task(logger, task_id, 'overall', f'任务完成', duration_ms=total_ms)
except VLMOutputInvalidError as e:
logger.error(f"[task_id={task_id}] 云端结果格式化失败: {e}")
self.send_failure_callback(webhook_url, task_id, 'vlm_fusion', str(e))
except Exception as e:
logger.error(f"[task_id={task_id}] 任务处理失败: {e}", exc_info=True)
self.send_failure_callback(webhook_url, task_id, 'download', str(e))
finally:
# 6. 清理
if 'preprocessor' in locals():
preprocessor.cleanup()
return 200
def process_push_task(self, task_data: dict, video_path: str,
preprocessor: 'VideoPreprocessor',
rate_limiter=None) -> dict:
"""推送模式：同步处理上传的视频，结果直接返回（无 webhook 回调）
rate_limiter: 可选 RateLimiter 实例，按 provider 限速（2x burst）
返回 payload 结构与原 webhook 回调一致:
- 成功: {task_id, status: "success", event_start_time, ..., frame_details, ...}
- 失败: {task_id, status: "failed", failure_stage, error_message}
"""
task_id = task_data.get('task_id')
known_members = task_data.get('known_members_context', '')
event_start_time = task_data.get('event_start_time', '')
logger.info(f"[task_id={task_id}] ====== 开始处理推送任务 ======")
start_time = time.time()
try:
# 1. 健康检查
healthy_adapters = self.health_check_all()
if not healthy_adapters:
logger.error(f"[task_id={task_id}] 所有模型不健康")
return {
"task_id": task_id, "status": "failed",
"failure_stage": "vlm_visual",
"error_message": "All models unhealthy"
}
# 2. 抽帧（视频已由调用方保存到本地，无需下载）
candidate_frames = preprocessor.extract_candidate_frames(video_path)
if not candidate_frames:
raise Exception("抽帧失败，无候选帧")
key_frames = preprocessor.select_key_frames(candidate_frames)
compressed_frames = preprocessor.compress_frames(key_frames)
if not compressed_frames:
raise Exception("压缩后无可用帧")
frame_timestamps = preprocessor.compute_timestamps(
video_path, len(compressed_frames), event_start_time
)
# event_end_time 未提供时，用 start + 视频时长推算（DB 列 NOT NULL）
event_end_time = task_data.get('event_end_time', '')
if not event_end_time and event_start_time and preprocessor.video_duration > 0:
try:
start_dt = datetime.strptime(event_start_time, '%Y-%m-%d %H:%M:%S')
event_end_time = (
start_dt + timedelta(seconds=int(preprocessor.video_duration))
).strftime('%Y-%m-%d %H:%M:%S')
except ValueError:
pass
# 3. 并行视觉分析
model_outputs = self.run_visual_analysis(
healthy_adapters, compressed_frames, frame_timestamps,
known_members, rate_limiter, video_path=video_path,
event_start_time=event_start_time
)
if not model_outputs:
raise Exception('All models failed in visual analysis')
# 4. 云端直出结果格式化（无本地融合）
provider = next(iter(model_outputs))
fusion_result = self.format_cloud_result(
provider, model_outputs[provider], known_members, task_id)
# 5. 附加关键帧图片（NAS 落盘后供 UI 时间轴展示）
frame_details = fusion_result.get('frame_details', [])
self._attach_frame_images(frame_details, compressed_frames)
total_ms = int((time.time() - start_time) * 1000)
log_task(logger, task_id, 'overall', '推送任务完成', duration_ms=total_ms)
return {
"task_id": task_id,
"status": "success",
"event_start_time": event_start_time,
"event_end_time": event_end_time,
"camera_name": task_data.get('camera_name', ''),
"global_summary": fusion_result.get('global_summary', ''),
"entities_json": fusion_result.get('entities_json', []),
"frame_details": frame_details,
"compute_provider": fusion_result.get('compute_provider', []),
"error_message": None
}
except VLMOutputInvalidError as e:
logger.error(f"[task_id={task_id}] VLM 输出解析失败: {e}")
return {
"task_id": task_id, "status": "failed",
"failure_stage": "vlm_fusion", "error_message": str(e)
}
except Exception as e:
logger.error(f"[task_id={task_id}] 推送任务处理失败: {e}", exc_info=True)
return {
"task_id": task_id, "status": "failed",
"failure_stage": "process", "error_message": str(e)
}

572
fam-edge/src/fam_edge/api_gateway/api_gateway.py
View File

@@ -1,517 +1,103 @@
""" """
API-Gateway - Flask 蓝图，接收任务 API-Gateway - Flask 蓝图（新架构 v2）
模式: 端点:
1. enqueue (异步): NAS 上传视频 → Edge 入队 → 立即返回 → 消费者异步处理 → NAS 轮询拉取结果 GET /api/oracle/sync NAS 每 30 分钟拉取增量（since + token 校验）
2. push (同步, 兼容保留): NAS 上传 → Edge 同步处理 → 结果随响应返回 POST /api/oracle/people/correct NAS 推送手动命名校正（label -> canonical_name）
3. analyze (旧拉取模式, 兼容保留) POST /api/edge/chat/ask 智能问答编排（Gemini -> NVIDIA -> Ollama）
GET /health 健康检查
已移除（旧推送/分块/队列模式）: /video/push, /enqueue, /chunk, /assemble,
/results, /queue/stats, /mark_frames
""" """
import os import os
import base64
import threading
import requests
from flask import Blueprint, request, jsonify from flask import Blueprint, request, jsonify
from ..logger import setup_logger from ..logger import setup_logger
from ..ai_orchestrator.orchestrator import AIOrchestrator from .. import state
from ..video_preprocessor.preprocessor import VideoPreprocessor from ..qa import QAOrchestrator
from ..queue import queue_manager
logger = setup_logger('fam-edge.api_gateway') logger = setup_logger('fam-edge.api_gateway')
api_bp = Blueprint('api_gateway', __name__) api_bp = Blueprint('api_gateway', __name__)
_current_task_lock = threading.Lock() _qa = None
_currently_processing = False
_orchestrator = None
def get_orchestrator(): def get_qa():
global _orchestrator global _qa
if _orchestrator is None: if _qa is None:
_orchestrator = AIOrchestrator() _qa = QAOrchestrator()
return _orchestrator return _qa
@api_bp.route('/api/edge/video/analyze', methods=['POST']) def _check_token() -> bool:
def receive_task(): expected = _sync_token()
"""接收分析任务""" token = request.args.get('token') or request.form.get('token') or \
global _currently_processing (request.get_json(silent=True) or {}).get('token', '')
return bool(expected) and token == expected
_SYNC_TOK = None
def _sync_token():
global _SYNC_TOK
if _SYNC_TOK is None:
from ..config_loader import load_config
_SYNC_TOK = load_config().get('sync_api', {}).get('token', '${ORACLE_SYNC_TOKEN}')
if _SYNC_TOK.startswith('${') and _SYNC_TOK.endswith('}'):
_SYNC_TOK = os.environ.get(_SYNC_TOK[2:-1], '')
return _SYNC_TOK or ''
@api_bp.route('/api/oracle/sync', methods=['GET'])
def sync_pull():
"""NAS 拉取增量数据。since=ISO 时间字符串（默认 '' 拉全量）。
返回: {videos:[...], events:[...], people:[...], server_time}
"""
if not _check_token():
return jsonify({"error": "unauthorized"}), 401
since = request.args.get('since', '')
try:
delta = state.get_db().get_sync_delta(since)
except Exception as e:
logger.error(f"sync_pull 异常: {e}")
return jsonify({"error": str(e)}), 500
return jsonify(delta), 200
@api_bp.route('/api/oracle/people/correct', methods=['POST'])
def people_correct():
"""NAS 手动命名校正推送。
请求: {"label": "人物A", "canonical_name": "张三", "token": "..."}
更新 people 表（manual 优先，不被 LLM 覆盖），立即重算 known_members_context。
"""
if not _check_token():
return jsonify({"error": "unauthorized"}), 401
data = request.get_json(silent=True) data = request.get_json(silent=True)
if not data: if not data:
return jsonify({"error": "Invalid JSON"}), 400 return jsonify({"error": "Invalid JSON"}), 400
label = (data.get('label') or '').strip()
task_id = data.get('task_id') canonical = (data.get('canonical_name') or '').strip()
video_url = data.get('video_url') if not label or not canonical:
webhook_url = data.get('webhook_url') return jsonify({"error": "缺少 label / canonical_name"}), 400
if not task_id or not video_url or not webhook_url:
return jsonify({"error": "缺少必填字段: task_id, video_url, webhook_url"}), 400
logger.info(f"[task_id={task_id}] 收到任务: {video_url}")
# 并发控制
with _current_task_lock:
if _currently_processing:
logger.warning(f"[task_id={task_id}] 队列已满 (当前有任务处理中)，返回 429")
return jsonify({"error": "Queue full", "retry_after": 60}), 429
_currently_processing = True
# 异步处理
def _process():
global _currently_processing
try: try:
orch = get_orchestrator() state.get_db().set_canonical(label, canonical, source='manual')
orch.process_task(data)
except Exception as e: except Exception as e:
logger.error(f"[task_id={task_id}] 处理异常: {e}", exc_info=True) logger.error(f"people_correct 异常: {e}")
finally:
with _current_task_lock:
_currently_processing = False
thread = threading.Thread(target=_process, daemon=True, name=f'task-{task_id}')
thread.start()
return jsonify({"status": "accepted", "task_id": task_id}), 202
@api_bp.route('/api/edge/video/enqueue', methods=['POST'])
def enqueue_task():
"""异步模式：接收 multipart 视频上传，入队后立即返回
NAS 上传视频 → Edge 保存到磁盘 + 入 SQLite 队列 → 返回 task_id
消费者线程异步处理，NAS 通过 /api/edge/results 拉取结果
"""
task_id_raw = request.form.get('task_id')
file = request.files.get('video')
if not task_id_raw or not file:
return jsonify({"error": "缺少必填字段: task_id, video"}), 400
try:
task_id = int(task_id_raw)
except ValueError:
return jsonify({"error": "task_id 必须是整数"}), 400
camera_name = request.form.get('camera_name', '')
event_start_time = request.form.get('event_start_time', '')
known_members_context = request.form.get('known_members_context', '')
upload_dir = os.environ.get('FAM_UPLOAD_DIR', '/tmp/fam_uploads')
os.makedirs(upload_dir, exist_ok=True)
video_filename = f"task_{task_id}_{file.filename}"
video_path = os.path.join(upload_dir, video_filename)
try:
file.save(video_path)
size_mb = os.path.getsize(video_path) / 1024 / 1024
logger.info(f"[task_id={task_id}] 入队: {file.filename} ({size_mb:.1f}MB)")
queue_id = queue_manager.enqueue(
nas_task_id=task_id,
video_filename=file.filename,
video_path=video_path,
camera_name=camera_name,
event_start_time=event_start_time,
known_members_context=known_members_context,
)
return jsonify({
"status": "queued",
"task_id": task_id,
"queue_id": queue_id,
}), 202
except Exception as e:
logger.error(f"[task_id={task_id}] 入队失败: {e}", exc_info=True)
if os.path.exists(video_path):
os.remove(video_path)
return jsonify({"error": str(e)}), 500 return jsonify({"error": str(e)}), 500
return jsonify({"status": "ok", "label": label, "canonical_name": canonical}), 200
# ========== 分块上传（断点续传）==========
CHUNK_SIZE = 20 * 1024 * 1024 # 20MB per chunk
def _chunk_dir(task_id: int) -> str:
upload_dir = os.environ.get('FAM_UPLOAD_DIR', '/tmp/fam_uploads')
d = os.path.join(upload_dir, f"task_{task_id}")
os.makedirs(d, exist_ok=True)
return d
@api_bp.route('/api/edge/video/chunk', methods=['POST'])
def upload_chunk():
"""接收单个分块，保存到 task_{id}/chunk_{index:04d}
断点续传：同一 task_id + chunk_index 重复上传会覆盖，
NAS 端可通过 /chunks 查询已上传分块，跳过已有的。
"""
task_id_raw = request.form.get('task_id')
chunk_index_raw = request.form.get('chunk_index')
total_chunks_raw = request.form.get('total_chunks')
filename = request.form.get('filename', 'video.mp4')
chunk_file = request.files.get('chunk')
if not task_id_raw or not chunk_index_raw or not chunk_file:
return jsonify({"error": "缺少必填字段: task_id, chunk_index, chunk"}), 400
try:
task_id = int(task_id_raw)
chunk_index = int(chunk_index_raw)
total_chunks = int(total_chunks_raw) if total_chunks_raw else 0
except ValueError:
return jsonify({"error": "task_id/chunk_index 必须是整数"}), 400
d = _chunk_dir(task_id)
chunk_path = os.path.join(d, f"chunk_{chunk_index:04d}")
try:
# 检查 total_chunks 是否变化（chunk_size 变更导致），自动清理旧分块并更新元数据
import json
meta_path = os.path.join(d, "meta.json")
if total_chunks and os.path.exists(meta_path):
try:
with open(meta_path) as mf:
old_meta = json.load(mf)
if old_meta.get('total_chunks') and old_meta['total_chunks'] != total_chunks:
logger.warning(f"[task_id={task_id}] total_chunks 变更 "
f"({old_meta['total_chunks']}→{total_chunks})，清理旧分块")
for fn in os.listdir(d):
if fn.startswith('chunk_'):
os.remove(os.path.join(d, fn))
with open(meta_path, 'w') as f:
json.dump({"filename": filename, "total_chunks": total_chunks}, f)
except (json.JSONDecodeError, IOError):
pass
chunk_file.save(chunk_path)
size_kb = os.path.getsize(chunk_path) / 1024
# 写元数据（首次上传时）
if not os.path.exists(meta_path):
meta = {"filename": filename, "total_chunks": total_chunks}
with open(meta_path, 'w') as f:
json.dump(meta, f)
# 统计已上传分块
uploaded = sorted([
int(fn.split('_')[1]) for fn in os.listdir(d)
if fn.startswith('chunk_') and len(fn.split('_')) == 2
])
logger.info(f"[task_id={task_id}] 分块 {chunk_index}/{total_chunks} 上传成功 "
f"({size_kb:.0f}KB, 已上传 {len(uploaded)}/{total_chunks})")
return jsonify({
"status": "ok",
"task_id": task_id,
"chunk_index": chunk_index,
"uploaded_count": len(uploaded),
"total_chunks": total_chunks,
}), 200
except Exception as e:
logger.error(f"[task_id={task_id}] 分块上传失败: {e}", exc_info=True)
return jsonify({"error": str(e)}), 500
@api_bp.route('/api/edge/video/chunks', methods=['GET'])
def query_chunks():
"""查询已上传分块列表（断点续传：NAS 重启后查询跳过已有分块）"""
task_id_raw = request.args.get('task_id')
if not task_id_raw:
return jsonify({"error": "缺少 task_id"}), 400
try:
task_id = int(task_id_raw)
except ValueError:
return jsonify({"error": "task_id 必须是整数"}), 400
d = _chunk_dir(task_id)
uploaded = sorted([
int(fn.split('_')[1]) for fn in os.listdir(d)
if fn.startswith('chunk_') and len(fn.split('_')) == 2
]) if os.path.isdir(d) else []
total = 0
meta_path = os.path.join(d, "meta.json")
if os.path.exists(meta_path):
import json
try:
with open(meta_path) as f:
total = json.load(f).get('total_chunks', 0)
except (json.JSONDecodeError, IOError):
pass
return jsonify({
"task_id": task_id,
"uploaded_chunks": uploaded,
"uploaded_count": len(uploaded),
"total_chunks": total,
}), 200
@api_bp.route('/api/edge/video/assemble', methods=['POST'])
def assemble_chunks():
"""合并所有分块为完整视频文件，入 SQLite 队列
NAS 上传完全部分块后调用此端点触发合并 + 入队。
"""
task_id_raw = request.form.get('task_id')
if not task_id_raw:
return jsonify({"error": "缺少 task_id"}), 400
try:
task_id = int(task_id_raw)
except ValueError:
return jsonify({"error": "task_id 必须是整数"}), 400
camera_name = request.form.get('camera_name', '')
event_start_time = request.form.get('event_start_time', '')
known_members_context = request.form.get('known_members_context', '')
d = _chunk_dir(task_id)
# 读取元数据
import json
meta_path = os.path.join(d, "meta.json")
if not os.path.exists(meta_path):
return jsonify({"error": "元数据不存在，请先上传分块"}), 400
try:
with open(meta_path) as f:
meta = json.load(f)
except json.JSONDecodeError:
return jsonify({"error": "元数据损坏"}), 500
filename = meta.get('filename', 'video.mp4')
total_chunks = meta.get('total_chunks', 0)
# 检查分块完整性
chunk_files = sorted([
fn for fn in os.listdir(d)
if fn.startswith('chunk_') and len(fn.split('_')) == 2
])
if total_chunks and len(chunk_files) < total_chunks:
missing = total_chunks - len(chunk_files)
return jsonify({
"error": f"分块不完整: {len(chunk_files)}/{total_chunks}，缺 {missing} 块",
"uploaded_count": len(chunk_files),
"total_chunks": total_chunks,
}), 400
# 合并分块
upload_dir = os.environ.get('FAM_UPLOAD_DIR', '/tmp/fam_uploads')
video_filename = f"task_{task_id}_{filename}"
video_path = os.path.join(upload_dir, video_filename)
try:
with open(video_path, 'wb') as out:
for cf in chunk_files:
chunk_path = os.path.join(d, cf)
with open(chunk_path, 'rb') as chunk_f:
out.write(chunk_f.read())
size_mb = os.path.getsize(video_path) / 1024 / 1024
logger.info(f"[task_id={task_id}] 分块合并完成: {filename} ({size_mb:.1f}MB, {len(chunk_files)} 块)")
# 清理分块目录
import shutil
shutil.rmtree(d, ignore_errors=True)
# 入队
queue_id = queue_manager.enqueue(
nas_task_id=task_id,
video_filename=filename,
video_path=video_path,
camera_name=camera_name,
event_start_time=event_start_time,
known_members_context=known_members_context,
)
return jsonify({
"status": "queued",
"task_id": task_id,
"queue_id": queue_id,
"size_mb": round(size_mb, 1),
}), 202
except Exception as e:
logger.error(f"[task_id={task_id}] 合并失败: {e}", exc_info=True)
return jsonify({"error": str(e)}), 500
@api_bp.route('/api/edge/results', methods=['GET'])
def get_results():
"""返回已完成但未拉取的结果，标记为已交付"""
limit = int(request.args.get('limit', 10))
results = queue_manager.get_undelivered_results(limit=limit)
import json
payload = []
task_ids = []
for r in results:
try:
result_json = json.loads(r['result_json']) if r['result_json'] else None
except json.JSONDecodeError:
result_json = None
if r['status'] == 'FAILED':
result_json = {
"status": "failed",
"error_message": r['error_message'] or 'unknown',
"failure_stage": r['failure_stage'] or '',
}
payload.append({
"nas_task_id": r['nas_task_id'],
"result": result_json,
})
task_ids.append(r['id'])
if task_ids:
queue_manager.mark_delivered(task_ids)
return jsonify({"results": payload, "count": len(payload)}), 200
@api_bp.route('/api/edge/queue/stats', methods=['GET'])
def queue_stats():
"""队列状态统计"""
stats = queue_manager.get_queue_stats()
return jsonify(stats), 200
@api_bp.route('/api/edge/video/push', methods=['POST'])
def receive_push_task():
"""推送模式：接收 multipart 视频上传，同步分析，结果随 HTTP 响应返回
NAS 无法被 Oracle 反向访问（Tailscale 不通），因此改为 NAS 主动上传视频，
Edge 用 OpenCV 场景变化检测抽帧后分析，摘要直接放在响应里带回。
"""
global _currently_processing
task_id_raw = request.form.get('task_id')
file = request.files.get('video')
if not task_id_raw or not file:
return jsonify({"error": "缺少必填字段: task_id, video"}), 400
try:
task_id = int(task_id_raw)
except ValueError:
return jsonify({"error": "task_id 必须是整数"}), 400
logger.info(f"[task_id={task_id}] 收到推送任务: {file.filename}")
# 并发控制（同步处理，占用整个请求周期）
with _current_task_lock:
if _currently_processing:
logger.warning(f"[task_id={task_id}] 已有任务处理中，返回 429")
return jsonify({"error": "Queue full", "retry_after": 60}), 429
_currently_processing = True
preprocessor = None
try:
preprocessor = VideoPreprocessor(task_id)
video_path = preprocessor.save_upload(file)
task_data = {
"task_id": task_id,
"camera_name": request.form.get('camera_name', ''),
"event_start_time": request.form.get('event_start_time', ''),
"event_end_time": request.form.get('event_end_time', ''),
"known_members_context": request.form.get('known_members_context', ''),
}
result = get_orchestrator().process_push_task(task_data, video_path, preprocessor)
return jsonify(result), 200
except Exception as e:
logger.error(f"[task_id={task_id}] 推送任务异常: {e}", exc_info=True)
return jsonify({
"task_id": task_id, "status": "failed",
"failure_stage": "upload", "error_message": str(e)
}), 200
finally:
if preprocessor is not None:
preprocessor.cleanup()
with _current_task_lock:
_currently_processing = False
@api_bp.route('/api/edge/mark_frames', methods=['POST'])
def mark_frames():
"""NAS 存量关键帧批量补红框（检测计算在 Edge，NAS 只存图）"""
data = request.get_json(silent=True)
if not data:
return jsonify({"error": "Invalid JSON"}), 400
images = data.get('images')
if not isinstance(images, list) or not images or len(images) > 12:
return jsonify({"error": "images 需要 1-12 项 [{key, data}]"}), 400
from ..frame_marker import mark_jpeg
results = []
for item in images:
key = item.get('key', '')
b64 = item.get('data', '')
try:
marked, faces = mark_jpeg(base64.b64decode(b64))
results.append({
"key": key,
"data": base64.b64encode(marked).decode('ascii'),
"faces": faces
})
except Exception as e:
logger.warning(f"补标失败 {key}: {e}")
results.append({"key": key, "data": None, "faces": 0, "error": str(e)})
return jsonify({"results": results}), 200
@api_bp.route('/health', methods=['GET'])
def health():
"""健康检查"""
global _currently_processing
orch = get_orchestrator()
healthy = orch.health_check_all()
if not healthy:
return jsonify({
"status": "unavailable",
"healthy_models": [],
"processing": _currently_processing
}), 503
return jsonify({
"status": "ok",
"healthy_models": [a.provider_name for a in healthy],
"processing": _currently_processing
}), 200
@api_bp.route('/api/edge/chat', methods=['POST'])
def chat_proxy():
"""代理转发至本地 Ollama /api/generate（兼容旧调用，Ollama 未对外暴露）"""
data = request.get_json(silent=True)
if not data:
return jsonify({"error": "Invalid JSON"}), 400
try:
resp = requests.post(
'http://127.0.0.1:11434/api/generate',
json=data,
timeout=data.get('options', {}).get('timeout', 120)
)
return jsonify(resp.json()), resp.status_code
except requests.RequestException as e:
logger.error(f"Chat proxy error: {e}")
return jsonify({"error": f"Ollama unreachable: {e}"}), 502
@api_bp.route('/api/edge/chat/ask', methods=['POST']) @api_bp.route('/api/edge/chat/ask', methods=['POST'])
def chat_ask(): def chat_ask():
"""智能问答编排：Gemini → NVIDIA → 本地 Ollama（两云端都失败才用本地兜底） """智能问答编排：Gemini → NVIDIA → 本地 Ollama（两云端都失败才用本地兜底）
请求: {"prompt": "..."} 请求: {"prompt": "...", "max_tokens": 512}
响应: {"answer": "...", "provider": "gemini"|"nvidia"|"ollama"} 响应: {"answer": "...", "provider": "gemini"|"nvidia"|"ollama"}
""" """
data = request.get_json(silent=True) data = request.get_json(silent=True)
@@ -519,12 +105,24 @@ def chat_ask():
return jsonify({"error": "缺少必填字段: prompt"}), 400 return jsonify({"error": "缺少必填字段: prompt"}), 400
prompt = data['prompt'] prompt = data['prompt']
max_tokens = int(data.get('max_tokens', 512)) max_tokens = int(data.get('max_tokens', 1024))
answer, provider = get_orchestrator().run_qa(prompt, max_tokens=max_tokens) answer, provider = get_qa().run_qa(prompt, max_tokens=max_tokens)
if answer is None: if answer is None:
return jsonify({ return jsonify({
"error": "所有模型均不可用（Gemini / NVIDIA / Ollama 全部失败）" "error": "所有模型均不可用（Gemini / NVIDIA / Ollama 全部失败）"
}), 503 }), 503
return jsonify({"answer": answer, "provider": provider}), 200 return jsonify({"answer": answer, "provider": provider}), 200
@api_bp.route('/health', methods=['GET'])
def health():
"""健康检查"""
try:
db = state.get_db()
vids = db._conn.execute(
"SELECT COUNT(*) c FROM videos WHERE status='done'").fetchone()['c']
return jsonify({"status": "ok", "processed_videos": vids}), 200
except Exception as e:
return jsonify({"status": "error", "error": str(e)}), 500

32
fam-edge/src/fam_edge/app.py
View File

@@ -1,7 +1,10 @@
""" """
FAM-Edge 主应用 - Flask 单进程 FAM-Edge 主应用 - Flask 单进程（新架构 v2）
承载: API-Gateway / Video-Preprocessor / AI-Orchestrator / Storage-Cleaner / Queue-Consumer 承载:
- API-Gateway（同步拉取 / 命名校正 / 智能问答）
- WatchProcessor（监听 Google 硬盘同步落地目录，整视频分析）
- PersonService（人物汇总合并，定时）
""" """
import os import os
import sys import sys
@@ -12,7 +15,9 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from .config_loader import load_config from .config_loader import load_config
from .logger import setup_logger from .logger import setup_logger
from .api_gateway.api_gateway import api_bp from .api_gateway.api_gateway import api_bp
from .queue.consumer import get_consumer from . import state
from .watch_processor import WatchProcessor
from .person_service import PersonService
logger = setup_logger('fam-edge.app') logger = setup_logger('fam-edge.app')
@@ -22,17 +27,24 @@ app.register_blueprint(api_bp)
@app.route('/', methods=['GET']) @app.route('/', methods=['GET'])
def root(): def root():
return jsonify({"service": "fam-edge", "version": "2.0"}), 200 return jsonify({"service": "fam-edge", "version": "2.0",
"mode": "drive-sync + whole-video analysis"}), 200
# 启动消费者线程（异步任务队列） # 启动监听处理器 + 人物服务
_consumer = None _watch = None
_person = None
try: try:
_consumer = get_consumer() db = state.get_db()
_consumer.start() _watch = WatchProcessor(db)
logger.info("Queue-Consumer 已启动") _watch.start()
logger.info("WatchProcessor 已启动")
_person = PersonService(db)
_person.start()
logger.info("PersonService 已启动")
except Exception as e: except Exception as e:
logger.error(f"Queue-Consumer 启动失败: {e}") logger.error(f"后台服务启动失败: {e}", exc_info=True)
if __name__ == '__main__': if __name__ == '__main__':

71
fam-edge/src/fam_edge/frame_marker.py
View File

@@ -1,71 +0,0 @@
"""
Frame-Marker - 关键帧人脸红框标记
Edge 端统一做检测计算（NAS ARM 太弱），NAS 只存图零计算:
- orchestrator 分析后、回传前: 画红框 + 统计人脸数
- /api/edge/mark_frames: NAS 存量帧批量补标
"""
import os
import threading
import cv2
import numpy as np
from .logger import setup_logger
from .config_loader import load_config
logger = setup_logger('fam-edge.frame_marker')
_lock = threading.Lock()
_detector = None
DEFAULT_MODEL = '/opt/fam-edge/models/yunet.onnx'
def _get_detector():
global _detector
if _detector is not None:
return _detector
with _lock:
if _detector is not None:
return _detector
path = load_config().get('frame_marker', {}).get('model_path', DEFAULT_MODEL)
if not os.path.isfile(path):
logger.warning(f"YuNet 模型不存在，跳过红框标记: {path}")
return None
det = cv2.FaceDetectorYN_create(path, '', (320, 320), score_threshold=0.6)
_detector = det
logger.info(f"YuNet 人脸检测器就绪: {path}")
return det
def mark_jpeg(jpeg_bytes: bytes):
"""在 JPEG 帧图上画人脸红框
返回 (标记后的 JPEG bytes, 人脸数)。检测失败/无模型时原样返回。
"""
det = _get_detector()
if det is None:
return jpeg_bytes, 0
img = cv2.imdecode(np.frombuffer(jpeg_bytes, np.uint8), cv2.IMREAD_COLOR)
if img is None:
return jpeg_bytes, 0
h, w = img.shape[:2]
with _lock:
det.setInputSize((w, h))
_, faces = det.detect(img)
if faces is None or len(faces) == 0:
return jpeg_bytes, 0
for f in faces:
x, y, fw, fh = int(f[0]), int(f[1]), int(f[2]), int(f[3])
# 人脸框外扩 40%，远处小脸也能看清
pad_w, pad_h = int(fw * 0.4), int(fh * 0.4)
x1 = max(0, x - pad_w)
y1 = max(0, y - pad_h)
x2 = min(w, x + fw + pad_w)
y2 = min(h, y + fh + pad_h)
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 0, 255), 2)
ok, buf = cv2.imencode('.jpg', img, [cv2.IMWRITE_JPEG_QUALITY, 85])
if not ok:
return jpeg_bytes, 0
return buf.tobytes(), len(faces)

39
fam-edge/src/fam_edge/model_adapters/base_adapter.py
View File

@@ -3,13 +3,21 @@
新增模型只需继承此类并实现方法: 新增模型只需继承此类并实现方法:
1. health_check() -> bool 1. health_check() -> bool
2. analyze_frames(frame_paths, frame_timestamps, known_members_context) -> Optional[dict] 2. analyze_video(video_path, known_members_context, event_start_time) -> Optional[dict]
- 视觉分析：输入帧图片路径 + 时间戳 + 成员清单，直接输出**结构化结果 dict** - 整视频分析：直接把完整视频交给云端 VLM（本地不切片、不抽帧）。
(含 frame_details 等，详见 format_cloud_result 约定)。 - 模型内部自行采样帧，输出结构化结果 dict。失败/超时返回 None。
- 失败/超时返回 None。 - 返回约定:
{
"global_summary": str, # 整段视频摘要
"events": [ # 有用时间点 + 画面信息
{"timestamp": "2026-08-21 08:15:30", # 绝对北京时间（event_start_time 推算）
"description": str,
"people": [str],
"is_attention_event": bool}, ...],
"people_mentioned": [str], # 本视频出现的人物标识/真名
}
3. chat(prompt) -> Optional[str] 3. chat(prompt) -> Optional[str]
- 纯文本问答（智能问答场景），返回文本或 None。 - 纯文本问答（智能问答场景），返回文本或 None。
- 默认实现抛 NotImplementedError；文本/视觉模型按需实现。
4. get_timeout() -> int 4. get_timeout() -> int
5. get_circuit_breaker() -> CircuitBreaker 5. get_circuit_breaker() -> CircuitBreaker
""" """
@@ -36,22 +44,13 @@ class BaseModelAdapter(ABC):
pass pass
@abstractmethod @abstractmethod
def analyze_frames(self, frame_paths: List[str], def analyze_video(self, video_path: str,
frame_timestamps: List[str], known_members_context: str,
known_members_context: str) -> Optional[Dict]: event_start_time: str = '') -> Optional[Dict]:
"""视觉分析：输入帧图片路径 + 时间戳 + 成员清单， """整视频分析：把完整视频交给云端 VLM，输出结构化结果 dict。
直接输出结构化结果 dict（含 frame_details 等）。失败/超时返回 None。
约定返回结构（云端模型直出，Edge 仅做格式化校验，不再本地融合）: 本地不切片、不抽帧；模型内部自行采样帧。
{ 失败/超时返回 None。
"global_summary": "整个时段整体摘要(可选，缺失时由 Edge 格式化生成)",
"entities_json": [{"person","action","clothing"}] (可选，缺失时由 frame_details 推导),
"frame_details": [
{"frame_index":int, "frame_timestamp":str, "person":str,
"action":str, "clothing":str, "is_attention_event":bool,
"source_providers":[provider]}
]
}
""" """
pass pass

268
fam-edge/src/fam_edge/model_adapters/gemini_adapter.py
View File

@@ -2,15 +2,16 @@
GeminiAdapter - Google Gemini 云端 VLM 适配器 GeminiAdapter - Google Gemini 云端 VLM 适配器
provider_name = "gemini" provider_name = "gemini"
模型: gemini-flash-latest (v1beta 下 gemini-1.5-flash 会 404，用 flash-latest 别名) 模型: gemini-flash-latest
角色: vision (视觉分析直出结构化 JSON) + 智能问答 角色: vision (整视频直出结构化 JSON) + 智能问答
健康检查: GET /v1beta/models?key=... 健康检查: GET /v1beta/models?key=...
熔断器: 启用 熔断器: 启用
视觉分析: 多图单请求直出结构化 JSON（global_summary/entities_json/frame_details） 整视频分析: 用 Files API 上传完整视频 -> generateContent 直出结构化 JSON
（本地不切片、不抽帧；Gemini 原生支持长视频）
""" """
import os import os
import time import time
import base64 import json
import requests import requests
from typing import Dict, List, Optional from typing import Dict, List, Optional
@@ -23,16 +24,15 @@ logger = setup_logger('fam-edge.gemini_adapter')
class GeminiAdapter(BaseModelAdapter): class GeminiAdapter(BaseModelAdapter):
"""Gemini 云端 VLM 适配器 (视觉直出结构化 JSON + 文本问答)""" """Gemini 云端 VLM 适配器 (整视频直出结构化 JSON + 文本问答)"""
def __init__(self, config: dict): def __init__(self, config: dict):
super().__init__("gemini", config) super().__init__("gemini", config)
self.model_name = config.get('model_name', 'gemini-flash-latest') self.model_name = config.get('model_name', 'gemini-flash-latest')
# 免费层配额按模型独立（20 请求/天/模型），fallback 链用于跨模型借用配额
self.model_chain = [self.model_name] + [ self.model_chain = [self.model_name] + [
m for m in config.get('fallback_models', []) if m and m != self.model_name] m for m in config.get('fallback_models', []) if m and m != self.model_name]
self.api_key = self._resolve_key(config.get('api_key', '')) self.api_key = self._resolve_key(config.get('api_key', ''))
self.timeout = config.get('timeout', 30) self.timeout = config.get('timeout', 600)
cb_cfg = config.get('circuit_breaker', {}) cb_cfg = config.get('circuit_breaker', {})
self._cb = CircuitBreaker( self._cb = CircuitBreaker(
threshold=cb_cfg.get('threshold', 3), threshold=cb_cfg.get('threshold', 3),
@@ -69,76 +69,131 @@ class GeminiAdapter(BaseModelAdapter):
return False return False
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# 视觉分析：多图单请求，直出结构化 JSON # 整视频分析：Files API 上传 -> generateContent
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def analyze_frames(self, frame_paths: List[str], def analyze_video(self, video_path: str,
frame_timestamps: List[str], known_members_context: str,
known_members_context: str) -> Optional[Dict]: event_start_time: str = '') -> Optional[Dict]:
if self._cb.is_open(): if self._cb.is_open():
logger.warning("Gemini 熔断器 OPEN，跳过调用") logger.warning("Gemini 熔断器 OPEN，跳过视频分析")
return None return None
if not self.api_key: if not self.api_key:
logger.warning("Gemini API Key 未配置，跳过调用") logger.warning("Gemini API Key 未配置，跳过视频分析")
return None return None
if not frame_paths: if not os.path.isfile(video_path):
logger.warning("Gemini 无帧可分析") logger.warning(f"Gemini 视频文件不存在: {video_path}")
return None return None
parts = [] file_uri = self._upload_file(video_path)
ts_map = {} if not file_uri:
for i, (path, ts) in enumerate(zip(frame_paths, frame_timestamps), 1): self._cb.record_failure()
return None
prompt = self._build_video_prompt(known_members_context, event_start_time)
try: try:
with open(path, 'rb') as f: text = self._generate_video(file_uri, prompt, max_tokens=4096, temperature=0.2)
img = base64.b64encode(f.read()).decode('utf-8')
except Exception as e:
logger.error(f"读取图片失败 {path}: {e}")
continue
parts.append({"inline_data": {"mime_type": "image/jpeg", "data": img}})
parts.append({"text": f"[图片{i}] 时间: {ts}"})
ts_map[i] = ts
if not parts:
return None
parts.insert(0, {"text": self._build_structured_prompt(known_members_context)})
try:
text = self._generate(parts, max_tokens=2048, temperature=0.2)
if text is None: if text is None:
self._cb.record_failure() self._cb.record_failure()
return None return None
try: try:
result = parse_vlm_json(text) result = parse_vlm_json(text)
# 确保 frame_details 的 frame_timestamp 与标注一致 result = self._normalize(result)
for f in result.get('frame_details', []): if not result or 'events' not in result:
idx = f.get('frame_index') logger.error(f"Gemini 视频输出缺少 events: {text[:150]}")
if isinstance(idx, int) and idx in ts_map and not f.get('frame_timestamp'): self._cb.record_failure()
f['frame_timestamp'] = ts_map[idx] return None
for f in result.get('frame_details', []): result['compute_provider'] = 'gemini'
if 'source_providers' not in f or not f.get('source_providers'):
f['source_providers'] = ['gemini']
self._cb.record_success() self._cb.record_success()
logger.info(f"Gemini 视觉分析完成，frame_details={len(result.get('frame_details', []))}") logger.info(f"Gemini 整视频分析完成，events={len(result.get('events', []))}")
return result return result
except VLMOutputInvalidError as e: except VLMOutputInvalidError as e:
logger.error(f"Gemini 输出无法解析为 JSON: {e}") logger.error(f"Gemini 视频输出无法解析为 JSON: {e}")
self._cb.record_failure() self._cb.record_failure()
return None return None
except requests.Timeout: except requests.Timeout:
logger.warning(f"Gemini 视觉分析超时 ({self.timeout}s)") logger.warning(f"Gemini 视频分析超时 ({self.timeout}s)")
self._cb.record_failure()
except Exception as e:
logger.error(f"Gemini 视觉分析异常: {e}")
self._cb.record_failure() self._cb.record_failure()
return None return None
except Exception as e:
logger.error(f"Gemini 视频分析异常: {e}")
self._cb.record_failure()
return None
finally:
self._delete_file(file_uri)
def _generate(self, parts: List[dict], max_tokens: int, def _upload_file(self, video_path: str) -> Optional[str]:
temperature: float) -> Optional[str]: """用 Files API 上传完整视频，返回可引用 URI。"""
"""带模型 fallback 链的 generateContent 调用 name = os.path.basename(video_path)
upload_url = f"{self._base_url}/files?key={self.api_key}"
try:
with open(video_path, 'rb') as f:
data = f.read()
except OSError as e:
logger.error(f"读取视频失败 {video_path}: {e}")
return None
headers = {
"X-Goog-Upload-Protocol": "raw",
"X-Goog-Upload-File-Name": name,
"Content-Type": "video/mp4",
}
try:
resp = requests.post(upload_url, headers=headers, data=data, timeout=300)
except requests.Timeout:
logger.warning("Gemini 文件上传超时 (300s)")
return None
except Exception as e:
logger.error(f"Gemini 文件上传异常: {e}")
return None
if resp.status_code not in (200, 201):
logger.warning(f"Gemini 文件上传失败 HTTP {resp.status_code}: {resp.text[:200]}")
return None
try:
info = resp.json().get('file', {})
uri = info.get('uri')
file_name = info.get('name')
state = info.get('state')
except (ValueError, KeyError):
logger.warning("Gemini 文件上传响应解析失败")
return None
if not uri:
return None
# 等待 ACTIVE（大文件可能还在处理）
if state != 'ACTIVE' and file_name:
uri = self._wait_active(file_name)
return uri
- 429（每日免费配额耗尽，按模型独立）→ 立即换下一个模型，不重试 def _wait_active(self, file_name: str, max_wait: int = 120) -> Optional[str]:
- 503（模型过载，临时性）→ 同模型退避 3s 重试一次，仍失败换下一个 url = f"{self._base_url}/{file_name}?key={self.api_key}"
""" deadline = time.time() + max_wait
while time.time() < deadline:
try:
r = requests.get(url, timeout=15)
if r.status_code == 200:
j = r.json()
if j.get('state') == 'ACTIVE':
return j.get('uri')
except Exception:
pass
time.sleep(5)
logger.warning(f"Gemini 文件 {file_name} 未在 {max_wait}s 内 ACTIVE")
return None
def _delete_file(self, file_uri: str):
if not file_uri or 'files/' not in file_uri:
return
name = file_uri.split('files/', 1)[-1]
try:
requests.delete(f"{self._base_url}/files/{name}?key={self.api_key}", timeout=15)
except Exception:
pass
def _generate_video(self, file_uri: str, prompt: str,
max_tokens: int, temperature: float) -> Optional[str]:
"""带模型 fallback 链的 generateContent（视频文件引用）调用。"""
parts = [
{"file_data": {"mime_type": "video/mp4", "file_uri": file_uri}},
{"text": prompt},
]
for model in self.model_chain: for model in self.model_chain:
for attempt in range(2): for attempt in range(2):
try: try:
@@ -146,14 +201,15 @@ class GeminiAdapter(BaseModelAdapter):
f"{self._base_url}/models/{model}:generateContent?key={self.api_key}", f"{self._base_url}/models/{model}:generateContent?key={self.api_key}",
json={"contents": [{"parts": parts}], json={"contents": [{"parts": parts}],
"generationConfig": { "generationConfig": {
"temperature": temperature, "maxOutputTokens": max_tokens}}, "temperature": temperature,
"maxOutputTokens": max_tokens}},
timeout=self.timeout timeout=self.timeout
) )
except requests.Timeout: except requests.Timeout:
logger.warning(f"Gemini [{model}] 请求超时 ({self.timeout}s)") logger.warning(f"Gemini [{model}] 视频请求超时 ({self.timeout}s)")
break break
except Exception as e: except Exception as e:
logger.error(f"Gemini [{model}] 请求异常: {e}") logger.error(f"Gemini [{model}] 视频请求异常: {e}")
break break
if resp.status_code == 200: if resp.status_code == 200:
@@ -164,55 +220,76 @@ class GeminiAdapter(BaseModelAdapter):
).strip() if cands else '' ).strip() if cands else ''
if text: if text:
if model != self.model_name: if model != self.model_name:
logger.info(f"Gemini 主模型不可用，由 fallback 模型 [{model}] 出结果") logger.info(f"Gemini 主模型不可用，由 fallback [{model}] 出结果")
return text return text
logger.warning(f"Gemini [{model}] 返回空文本") logger.warning(f"Gemini [{model}] 返回空文本")
continue continue
detail = resp.text[:150].replace('\n', ' ') detail = resp.text[:150].replace('\n', ' ')
if resp.status_code == 429: if resp.status_code == 429:
logger.warning(f"Gemini [{model}] 429 每日免费配额耗尽，切换下一模型") logger.warning(f"Gemini [{model}] 429 配额耗尽，切换下一模型")
break break
if resp.status_code == 503: if resp.status_code == 503:
if attempt == 0: if attempt == 0:
logger.warning(f"Gemini [{model}] 503 过载，3s 后重试") logger.warning(f"Gemini [{model}] 503 过载，3s 后重试")
time.sleep(3) time.sleep(3)
continue continue
logger.warning(f"Gemini [{model}] 503 重试仍失败，切换下一模型")
break break
logger.warning(f"Gemini [{model}] HTTP {resp.status_code}: {detail}") logger.warning(f"Gemini [{model}] HTTP {resp.status_code}: {detail}")
break break
return None return None
def _build_structured_prompt(self, known_members: str) -> str: @staticmethod
return f"""你是家庭监控视频分析助手。下面按时间顺序排列了多张监控截图。 def _normalize(result: dict) -> dict:
请分析整个时段，只输出合法 JSON（不要 markdown、不要任何解释文字），结构如下： """统一字段名：frame_details -> events（兼容旧结构）。"""
events = result.get('events')
if events is None and 'frame_details' in result:
events = []
for f in result['frame_details']:
events.append({
"timestamp": f.get('frame_timestamp', ''),
"description": f.get('action', ''),
"people": [f.get('person', '')] if f.get('person') else [],
"is_attention_event": bool(f.get('is_attention_event', False)),
})
if events is None:
events = []
people = result.get('people_mentioned') or result.get('entities_json') or []
if isinstance(people, list) and people and isinstance(people[0], dict):
people = [p.get('person', '') for p in people]
people = [p for p in people if p]
return {
"global_summary": result.get('global_summary', ''),
"events": events,
"people_mentioned": people,
}
def _build_video_prompt(self, known_members: str, event_start_time: str) -> str:
start_hint = ""
if event_start_time:
start_hint = f"\n视频开始时间（北京时间）约为：{event_start_time}。请据此推算每个事件的绝对时间戳。"
return f"""你是家庭监控视频分析助手。下面是一段完整监控录像（已整段上传）。
请观看整段视频，提取其中有用的信息，只输出合法 JSON（不要 markdown、不要任何解释文字），结构如下：
{{ {{
"global_summary": "整个时段的整体摘要，简体中文，2-4 句，客观描述人物与主要活动", "global_summary": "整个时段的整体摘要，简体中文，2-4 句，客观描述人物与主要活动",
"entities_json": [ "events": [
{{"person": "人物标识(匹配已知成员用真名，否则用'人物A'/'人物B'...)", "action": "主要动作", "clothing": "衣着"}}
],
"frame_details": [
{{ {{
"frame_index": 图片序号(从1开始，与[图片N]标注对应), "timestamp": "事件发生时的绝对北京时间(格式 YYYY-MM-DD HH:MM:SS)",
"frame_timestamp": "该帧的时间戳(用[图片N]标注里的时间)", "description": "该时间点的画面/动作信息摘要（谁、在做什么、位置）",
"person": "该帧画面中的人物或'无人'", "people": ["出现在该时刻的人物，用已知成员真名或'人物A'/'人物B'"],
"action": "该帧可见动作", "is_attention_event": false
"clothing": "该帧衣着(颜色+类型)",
"is_attention_event": false,
"source_providers": ["gemini"]
}} }}
] ],
}} "people_mentioned": ["本视频出现过的所有人物标识/真名"]
}}{start_hint}
规则： 规则：
1. 只描述客观画面，不要猜测或想象。 1. 只描述客观画面，不要猜测或想象。
2. frame_details 每帧一条，frame_index 与上方[图片N]序号对应，frame_timestamp 用标注时间。 2. events 提取视频中"有意义的时间点"（人物出现/动作变化/异常），不要逐秒罗列；timestamp 用绝对北京时间。
3. 已知家庭成员（按特征匹配，匹配到用 real_name，否则用"人物X"）： 3. 已知家庭成员（按特征匹配，匹配到用 real_name，否则用"人物X"）：
{known_members or '（暂无已知成员）'} {known_members or '（暂无已知成员）'}
4. is_attention_event：是否为跌倒、危险、异常哭闹等需关注事件（没有则为 false）。 4. is_attention_event：是否为跌倒、危险、异常哭闹等需关注事件（没有则为 false）。
5. 没有人物出现的帧 person 填"无人"，action 填""。""" 5. 没有人物出现的时段不要单独成 event；people 留空数组。"""
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# 智能问答：纯文本 # 智能问答：纯文本
@@ -222,11 +299,42 @@ class GeminiAdapter(BaseModelAdapter):
logger.warning("Gemini API Key 未配置，跳过问答") logger.warning("Gemini API Key 未配置，跳过问答")
return None return None
try: try:
return self._generate([{"text": prompt}], max_tokens=max_tokens, temperature=0.3) return self._generate_text(prompt, max_tokens=max_tokens, temperature=0.3)
except Exception as e: except Exception as e:
logger.error(f"Gemini 问答异常: {e}") logger.error(f"Gemini 问答异常: {e}")
return None return None
def _generate_text(self, text: str, max_tokens: int, temperature: float) -> Optional[str]:
"""纯文本 generateContent（复用模型 fallback 链）。"""
for model in self.model_chain:
try:
resp = requests.post(
f"{self._base_url}/models/{model}:generateContent?key={self.api_key}",
json={"contents": [{"parts": [{"text": text}]}],
"generationConfig": {
"temperature": temperature,
"maxOutputTokens": max_tokens}},
timeout=self.timeout
)
except requests.Timeout:
logger.warning(f"Gemini [{model}] 问答超时")
continue
except Exception as e:
logger.error(f"Gemini [{model}] 问答异常: {e}")
continue
if resp.status_code == 200:
cands = resp.json().get('candidates', [])
out = ''.join(
p.get('text', '')
for p in (cands[0].get('content', {}) if cands else {}).get('parts', [])
).strip() if cands else ''
if out:
return out
elif resp.status_code == 429:
logger.warning(f"Gemini [{model}] 429，切换模型")
continue
return None
def get_timeout(self) -> int: def get_timeout(self) -> int:
return self.timeout return self.timeout

327
fam-edge/src/fam_edge/model_adapters/nvidia_adapter.py
View File

@@ -2,23 +2,15 @@
NvidiaVisionAdapter - NVIDIA NIM 云端 VLM 适配器 NvidiaVisionAdapter - NVIDIA NIM 云端 VLM 适配器
provider_name = "nvidia" provider_name = "nvidia"
模型: nvidia/nemotron-3-nano-omni-30b-a3b-reasoning (Omni, 原生视频输入) 模型: nvidia/nemotron-nano-12b-v2-vl（NIM 官方支持整视频 video_url 输入，内部自行采样帧）
角色: vision (视觉分析直出结构化 JSON) + 智能问答 角色: vision (整视频直出结构化 JSON) + 智能问答
SDK: openai (NIM 兼容 OpenAI API 规范) SDK: openai (NIM 兼容 OpenAI API 规范)
整视频分析: 整视频 base64 经 video_url 单次调用 —— 本地不切片、不抽帧
视频模式 (analyze_video): 按关键帧时间点截取 ±1.5s 片段拼接集锦视频
(片段左上角叠加原始时间戳)，base64 后经 video_url 单次调用 —
模型看到动态画面而非静态帧，动作/轨迹识别显著优于逐帧图片。
图片模式 (analyze_frames): 逐帧 image_url 调用（无视频文件时的降级路径）。
注意: nemotron-omni 是 reasoning 模型，max_tokens 需给足（reasoning 消耗 token）。
""" """
import os import os
import base64 import base64
import json import json
import re import re
import subprocess
import tempfile
from typing import Dict, List, Optional from typing import Dict, List, Optional
from .base_adapter import BaseModelAdapter from .base_adapter import BaseModelAdapter
@@ -32,20 +24,17 @@ try:
except ImportError: except ImportError:
OpenAI = None OpenAI = None
VIDEO_SEGMENT_PAD = 1.5 # 关键帧前后各截取秒数
HIGHLIGHT_WIDTH = 640 # 集锦视频宽度（保持宽高比）
class NvidiaVisionAdapter(BaseModelAdapter): class NvidiaVisionAdapter(BaseModelAdapter):
"""NVIDIA NIM 云端 VLM 适配器 (视频集锦单次调用; 逐帧降级; 文本问答)""" """NVIDIA NIM 云端 VLM 适配器 (整视频单次调用; 文本问答)"""
def __init__(self, config: dict): def __init__(self, config: dict):
super().__init__("nvidia", config) super().__init__("nvidia", config)
self.model_name = config.get( self.model_name = config.get(
'model_name', 'nvidia/nemotron-3-nano-omni-30b-a3b-reasoning') 'model_name', 'nvidia/nemotron-nano-12b-v2-vl')
self.api_key = self._resolve_key(config.get('api_key', '')) self.api_key = self._resolve_key(config.get('api_key', ''))
self.base_url = config.get('base_url', 'https://integrate.api.nvidia.com/v1') self.base_url = config.get('base_url', 'https://integrate.api.nvidia.com/v1')
self.timeout = config.get('timeout', 120) self.timeout = config.get('timeout', 600)
cb_cfg = config.get('circuit_breaker', {}) cb_cfg = config.get('circuit_breaker', {})
self._cb = CircuitBreaker( self._cb = CircuitBreaker(
threshold=cb_cfg.get('threshold', 3), threshold=cb_cfg.get('threshold', 3),
@@ -78,127 +67,29 @@ class NvidiaVisionAdapter(BaseModelAdapter):
return False return False
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# 视频模式：集锦视频 + video_url 单次调用（主路径） # 整视频分析：base64 整视频 -> video_url 单次调用
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@staticmethod
def _ts_to_seconds(ts: str) -> float:
"""'2026-08-20 04:34:10'（生产格式）/ 'HH:MM:SS' -> 当日秒偏移"""
s = str(ts).strip()
# 绝对时间格式: 取时间部分（同一天 30min 片段内足够）
date_part, _, time_part = s.partition(' ')
if time_part and '-' in date_part:
s = time_part
parts = s.split(':')
try:
if len(parts) == 3:
return int(parts[0]) * 3600 + int(parts[1]) * 60 + float(parts[2])
if len(parts) == 2:
return int(parts[0]) * 60 + float(parts[1])
return float(ts)
except ValueError:
return -1.0
@staticmethod
def _probe_duration(video_path: str) -> float:
"""ffprobe 解析视频时长，失败返回 0"""
try:
r = subprocess.run(
['ffprobe', '-v', 'quiet', '-show_entries', 'format=duration',
'-of', 'csv=p=0', video_path],
capture_output=True, timeout=30)
return float(r.stdout.decode().strip() or 0)
except (subprocess.TimeoutExpired, OSError, ValueError):
return 0.0
def _build_highlight_video(self, video_path: str, frame_timestamps: List[str],
event_start_time: str = '') -> Optional[str]:
"""按关键帧时间点截取 ±pad 秒片段，叠加时间戳后拼接集锦视频
时间戳以 2026-08-20 04-34-10 形式叠加（连字符避免 ffmpeg drawtext 冒号转义）。
偏移换算: 绝对时间戳 - 视频开始时间（event_start_time 缺失时，
时间戳值本身须已是视频内偏移，如 HH:MM:SS 相对时间）。
"""
start_sec = self._ts_to_seconds(event_start_time) if event_start_time else 0.0
duration = self._probe_duration(video_path)
clips = []
for ts in frame_timestamps:
sec = self._ts_to_seconds(ts)
if sec < 0:
continue
if start_sec > 0:
sec -= start_sec
if sec < 0:
sec += 86400 # 跨午夜
if duration > 0 and (sec < -VIDEO_SEGMENT_PAD
or sec > duration - 0.5):
logger.info(f"NVIDIA 跳过超界片段: {ts} -> {sec:.1f}s (视频 {duration:.0f}s)")
continue
clips.append((sec, str(ts).replace(':', '-')))
if not clips:
return None
out_path = os.path.join(
tempfile.mkdtemp(prefix='nim_highlight_'), 'highlight.mp4')
cmd = ['ffmpeg', '-y', '-loglevel', 'error']
for start, _ in clips:
cmd += ['-ss', f'{start:.2f}', '-t', f'{VIDEO_SEGMENT_PAD * 2}', '-i', video_path]
parts = []
for i, (_, label) in enumerate(clips):
parts.append(
f"[{i}:v]fps=15,scale={HIGHLIGHT_WIDTH}:-2,"
f"drawtext=text='ts {label}':x=8:y=8:fontsize=22:"
f"fontcolor=white:box=1:boxcolor=black@0.6[v{i}]")
concat_in = ''.join(f'[v{i}]' for i in range(len(clips)))
parts.append(f'{concat_in}concat=n={len(clips)}:v=1:a=0[out]')
cmd += ['-filter_complex', ';'.join(parts), '-map', '[out]',
'-r', '15',
'-c:v', 'libx264', '-preset', 'veryfast', '-crf', '28',
'-an', out_path]
try:
subprocess.run(cmd, check=True, capture_output=True, timeout=120)
except subprocess.TimeoutExpired:
logger.warning("NVIDIA 集锦视频生成超时")
return None
except subprocess.CalledProcessError as e:
logger.warning(f"NVIDIA 集锦视频生成失败: {e.stderr.decode()[:200] if e.stderr else e}")
return None
size = os.path.getsize(out_path)
logger.info(f"NVIDIA 集锦视频生成: {len(clips)} 片段, {size // 1024}KB")
if size < 1024:
return None
return out_path
def analyze_video(self, video_path: str, def analyze_video(self, video_path: str,
frame_timestamps: List[str],
known_members_context: str, known_members_context: str,
event_start_time: str = '') -> Optional[Dict]: event_start_time: str = '') -> Optional[Dict]:
"""原生视频输入分析: 集锦片段 -> video_url 单次调用"""
if self._cb.is_open(): if self._cb.is_open():
logger.warning("NVIDIA 熔断器 OPEN，跳过视频分析") logger.warning("NVIDIA 熔断器 OPEN，跳过视频分析")
return None return None
if self._client is None: if self._client is None:
logger.warning("NVIDIA 客户端未初始化，跳过视频分析") logger.warning("NVIDIA 客户端未初始化，跳过视频分析")
return None return None
if not os.path.isfile(video_path):
highlight = self._build_highlight_video( logger.warning(f"NVIDIA 视频文件不存在: {video_path}")
video_path, frame_timestamps, event_start_time)
if not highlight:
logger.warning("NVIDIA 集锦视频不可用，降级逐帧模式")
return None return None
try: try:
with open(highlight, 'rb') as f: with open(video_path, 'rb') as f:
b64 = base64.b64encode(f.read()).decode('utf-8') b64 = base64.b64encode(f.read()).decode('utf-8')
except Exception as e: except Exception as e:
logger.warning(f"NVIDIA 读取集锦视频失败: {e}") logger.warning(f"NVIDIA 读取视频失败: {e}")
return None return None
finally:
try:
os.remove(highlight)
os.rmdir(os.path.dirname(highlight))
except OSError:
pass
prompt = self._build_video_prompt(frame_timestamps, known_members_context) prompt = self._build_video_prompt(known_members_context, event_start_time)
try: try:
resp = self._client.chat.completions.create( resp = self._client.chat.completions.create(
model=self.model_name, model=self.model_name,
@@ -208,134 +99,47 @@ class NvidiaVisionAdapter(BaseModelAdapter):
"url": f"data:video/mp4;base64,{b64}"}} "url": f"data:video/mp4;base64,{b64}"}}
]}], ]}],
temperature=0.2, temperature=0.2,
max_tokens=3072, max_tokens=4096,
# NIM 扩展：控制视频采样帧数（模型上限 128 帧）
extra_body={"media_io_kwargs": {"video": {"num_frames": 128}}},
timeout=self.timeout timeout=self.timeout
) )
content = resp.choices[0].message.content content = resp.choices[0].message.content
if not content: if not content:
logger.warning("NVIDIA 视频分析返回空 content") logger.warning("NVIDIA 视频分析返回空 content")
self._cb.record_failure()
return None return None
data = self._parse_single_frame_json(content) data = self._parse_json(content)
if not data or 'frame_details' not in data: if not data or 'events' not in data:
logger.warning(f"NVIDIA 视频 JSON 解析失败: {content[:150]}") logger.warning(f"NVIDIA 视频 JSON 解析失败: {content[:150]}")
return None self._cb.record_failure()
frame_details = self._normalize_frame_details(data, frame_timestamps)
if not frame_details:
return None return None
self._cb.record_success() self._cb.record_success()
logger.info(f"NVIDIA 视频分析完成，frame_details={len(frame_details)}") logger.info(f"NVIDIA 整视频分析完成，events={len(data.get('events', []))}")
result = {"frame_details": frame_details} return {
if data.get('global_summary'): "global_summary": str(data.get('global_summary', '')),
result['global_summary'] = str(data['global_summary']) "events": data.get('events', []),
if data.get('entities_json'): "people_mentioned": data.get('people_mentioned', []),
result['entities_json'] = data['entities_json'] "compute_provider": "nvidia",
return result }
except Exception as e: except Exception as e:
self._cb.record_failure() self._cb.record_failure()
logger.warning(f"NVIDIA 视频分析异常: {e}") logger.warning(f"NVIDIA 视频分析异常: {e}")
return None return None
def _normalize_frame_details(self, data: dict, @staticmethod
frame_timestamps: List[str]) -> List[Dict]: def _parse_json(content: str) -> Optional[dict]:
"""归一化模型输出的 frame_details，按已知时间戳对齐"""
details = []
for i, item in enumerate(data.get('frame_details', []), 1):
if not isinstance(item, dict):
continue
ts = str(item.get('frame_timestamp',
frame_timestamps[i - 1] if i <= len(frame_timestamps) else ''))
details.append({
"frame_index": i,
"frame_timestamp": ts,
"person": str(item.get('person', '无人')),
"action": str(item.get('action', '')),
"clothing": str(item.get('clothing', '')),
"is_attention_event": bool(item.get('is_attention_event', False)),
"source_providers": ["nvidia"],
})
return details
def _build_video_prompt(self, frame_timestamps: List[str],
known_members: str) -> str:
ts_list = '\n'.join(f' 片段{i}: 原始时间 {ts}' for i, ts in enumerate(frame_timestamps, 1))
return f"""你是家庭监控视频分析助手。下面的视频是由一段长时间监控录像中抽取的片段集锦，
共 {len(frame_timestamps)} 个片段（每个约 3 秒），按顺序拼接。每个片段左上角叠加了
原始时间戳（ts 后的 2026-08-20 04-34-10 表示北京时间 2026年8月20日 04:34:10）。
片段时间对照：
{ts_list}
只输出合法 JSON（不要 markdown、不要解释），结构如下：
{{
"frame_details": [
{{
"frame_timestamp": "<片段原始时间>",
"person": "片段中的人物或'无人'",
"action": "片段中人物的动作（动态观察，如走动/跑动/坐下）",
"clothing": "衣着(颜色+类型)",
"is_attention_event": false
}}
],
"global_summary": "整段录像的综合摘要",
"entities_json": [{{"person": "人物名或人物X", "action": "行为概括", "clothing": "衣着"}}]
}}
规则：
1. 只描述客观画面，不猜测。
2. 已知家庭成员（按特征匹配，匹配到用 real_name，否则用"人物X"）：
{known_members or '（暂无已知成员）'}
3. is_attention_event：跌倒、危险、异常哭闹等需关注事件（没有则为 false）。
4. 无人出现的片段 person 填"无人"，action 填""。"""
# ------------------------------------------------------------------
# 图片模式：逐帧调用（降级路径，无视频文件时使用）
# ------------------------------------------------------------------
def analyze_frames(self, frame_paths: List[str],
frame_timestamps: List[str],
known_members_context: str) -> Optional[Dict]:
if self._cb.is_open():
logger.warning("NVIDIA 熔断器 OPEN，跳过调用")
return None
if self._client is None:
logger.warning("NVIDIA 客户端未初始化，跳过调用")
return None
if not frame_paths:
logger.warning("NVIDIA 无帧可分析")
return None
frame_details = []
ok = False
for i, (path, ts) in enumerate(zip(frame_paths, frame_timestamps), 1):
detail = self._analyze_one_structured(path, ts, i, known_members_context)
if detail:
frame_details.append(detail)
ok = True
if not ok:
self._cb.record_failure()
return None
self._cb.record_success()
logger.info(f"NVIDIA 视觉分析完成，frame_details={len(frame_details)}")
# NVIDIA 单帧无法跨帧综合 global_summary，交由 Edge format_cloud_result 格式化生成
return {"frame_details": frame_details}
def _parse_single_frame_json(self, content: str) -> Optional[dict]:
"""轻量解析单帧 JSON（不要求全 schema，仅提取字段）"""
content = content.strip() content = content.strip()
# 直接解析
try: try:
return json.loads(content) return json.loads(content)
except json.JSONDecodeError: except json.JSONDecodeError:
pass pass
# 提取 markdown fence
fence = re.search(r'```(?:json)?\s*(\{.*?\})\s*```', content, re.DOTALL) fence = re.search(r'```(?:json)?\s*(\{.*?\})\s*```', content, re.DOTALL)
if fence: if fence:
try: try:
return json.loads(fence.group(1)) return json.loads(fence.group(1))
except json.JSONDecodeError: except json.JSONDecodeError:
pass pass
# 贪婪匹配最大 {...}
brace = re.search(r'\{.*\}', content, re.DOTALL) brace = re.search(r'\{.*\}', content, re.DOTALL)
if brace: if brace:
try: try:
@@ -344,68 +148,35 @@ class NvidiaVisionAdapter(BaseModelAdapter):
pass pass
return None return None
def _analyze_one_structured(self, path: str, ts: str, idx: int, def _build_video_prompt(self, known_members: str, event_start_time: str) -> str:
known_members: str) -> Optional[Dict]: start_hint = ""
try: if event_start_time:
with open(path, 'rb') as f: start_hint = f"\n视频开始时间（北京时间）约为：{event_start_time}。请据此推算每个事件的绝对时间戳。"
b64 = base64.b64encode(f.read()).decode('utf-8') return f"""你是家庭监控视频分析助手。下面是一段完整监控录像（已整段上传）。
except Exception as e: 请观看整段视频，提取其中有用的信息，只输出合法 JSON（不要 markdown、不要解释），结构如下：
logger.error(f"读取图片失败 {path}: {e}")
return None
prompt = self._build_structured_prompt(ts, idx, known_members)
try:
resp = self._client.chat.completions.create(
model=self.model_name,
messages=[{"role": "user", "content": [
{"type": "text", "text": prompt},
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}
]}],
temperature=0.2,
max_tokens=512,
timeout=self.timeout
)
content = resp.choices[0].message.content
if not content:
return None
data = self._parse_single_frame_json(content)
if not data:
logger.warning(f"NVIDIA 单帧 JSON 解析失败: {content[:120]}")
return None
return {
"frame_index": idx,
"frame_timestamp": str(data.get("frame_timestamp", ts)),
"person": str(data.get("person", "无人")),
"action": str(data.get("action", "")),
"clothing": str(data.get("clothing", "")),
"is_attention_event": bool(data.get("is_attention_event", False)),
"source_providers": ["nvidia"],
}
except Exception as e:
logger.warning(f"NVIDIA 单帧异常: {e}")
return None
def _build_structured_prompt(self, ts: str, idx: int, known_members: str) -> str:
return f"""你是家庭监控视频分析助手。请看这张监控截图（拍摄时间 {ts}）。
只输出合法 JSON（不要 markdown、不要解释），结构如下：
{{ {{
"frame_timestamp": "{ts}", "global_summary": "整个时段的整体摘要，简体中文，2-4 句",
"person": "该帧画面中的人物或'无人'", "events": [
"action": "该帧可见动作", {{
"clothing": "该帧衣着(颜色+类型)", "timestamp": "事件发生时的绝对北京时间(YYYY-MM-DD HH:MM:SS)",
"description": "该时刻画面/动作信息摘要",
"people": ["出现在该时刻的人物，用已知成员真名或'人物A'"],
"is_attention_event": false "is_attention_event": false
}} }}
],
"people_mentioned": ["本视频出现过的所有人物标识/真名"]
}}{start_hint}
规则： 规则：
1. 只描述客观画面，不猜测。 1. 只描述客观画面，不猜测。
2. 已知家庭成员（按特征匹配，匹配到用 real_name，否则用"人物X"）： 2. events 提取有意义的时间点（人物出现/动作变化/异常），timestamp 用绝对北京时间。
3. 已知家庭成员（按特征匹配，匹配到用 real_name，否则用"人物X"）：
{known_members or '（暂无已知成员）'} {known_members or '（暂无已知成员）'}
3. is_attention_event：是否为跌倒、危险、异常哭闹等需关注事件（没有则为 false）。 4. is_attention_event：跌倒、危险、异常哭闹等需关注事件（没有则为 false）。"""
4. 没有人物出现的帧 person 填"无人"，action 填""。"""
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# 智能问答：纯文本（reasoning 模型，max_tokens 需给足） # 智能问答：纯文本
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def chat(self, prompt: str, max_tokens: int = 2048) -> Optional[str]: def chat(self, prompt: str, max_tokens: int = 2048) -> Optional[str]:
if self._client is None: if self._client is None:

7
fam-edge/src/fam_edge/model_adapters/ollama_adapter.py
View File

@@ -111,6 +111,13 @@ class OllamaAdapter(BaseModelAdapter):
self._cb.record_failure() self._cb.record_failure()
return None return None
def analyze_video(self, video_path: str,
known_members_context: str,
event_start_time: str = '') -> Optional[Dict]:
"""Ollama 为纯文本模型，不参与视频分析，返回 None（降级链不会选它做视频）。"""
logger.info("Ollama 为纯文本模型，跳过视频分析")
return None
def get_timeout(self) -> int: def get_timeout(self) -> int:
return self.timeout return self.timeout

238
fam-edge/src/fam_edge/oracle_db.py Normal file
View File

@@ -0,0 +1,238 @@
"""
Oracle 本地库（SQLite） - 视频摘要 / 事件 / 人物 存储
表结构:
videos : 每个被处理的视频一个记录（含全局摘要 + 事件列表 + 人物列表，JSON 冗余存储便于查询）
events : 视频拆出的事件（时间点 + 描述 + 涉及人物）
people : 规范人物表（canonical_name + 别名），由 person_service 维护
sync_cursor: 同步游标（NAS 拉取用，记录最后成功同步时间）
对外提供:
- upsert_video / get_pending_videos / mark_video_processed
- upsert_event
- upsert_person / get_known_members_context
- get_sync_delta(since_iso) -> 增量数据（供 NAS 拉取）
- set_cursor / get_cursor
"""
import os
import json
import sqlite3
from datetime import datetime, timezone, timedelta
from typing import Dict, List, Optional
logger = None # 延迟注入，避免循环 import
def _now_iso() -> str:
return datetime.now(timezone(timedelta(hours=8))).strftime('%Y-%m-%d %H:%M:%S')
class OracleDB:
def __init__(self, db_path: str):
os.makedirs(os.path.dirname(db_path), exist_ok=True)
self.db_path = db_path
self._conn = sqlite3.connect(db_path, check_same_thread=False)
self._conn.row_factory = sqlite3.Row
self._conn.execute("PRAGMA journal_mode=WAL")
self._init_schema()
# ------------------------------------------------------------------
def _init_schema(self):
c = self._conn
c.executescript("""
CREATE TABLE IF NOT EXISTS videos (
id INTEGER PRIMARY KEY AUTOINCREMENT,
drive_file_id TEXT,
filename TEXT UNIQUE,
local_path TEXT,
camera_name TEXT,
duration_sec REAL,
event_start_time TEXT,
status TEXT DEFAULT 'pending',
summary_json TEXT,
events_json TEXT,
people_json TEXT,
compute_provider TEXT,
created_at TEXT,
updated_at TEXT,
processed_at TEXT
);
CREATE TABLE IF NOT EXISTS events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
video_id INTEGER,
ts TEXT,
description TEXT,
person_list_json TEXT,
is_attention_event INTEGER DEFAULT 0,
FOREIGN KEY(video_id) REFERENCES videos(id)
);
CREATE TABLE IF NOT EXISTS people (
id INTEGER PRIMARY KEY AUTOINCREMENT,
label TEXT UNIQUE,
canonical_name TEXT,
first_seen TEXT,
appearances INTEGER DEFAULT 0,
source TEXT DEFAULT 'llm',
updated_at TEXT
);
CREATE TABLE IF NOT EXISTS sync_cursor (
key TEXT PRIMARY KEY,
value TEXT
);
CREATE INDEX IF NOT EXISTS idx_videos_updated ON videos(updated_at);
CREATE INDEX IF NOT EXISTS idx_events_video ON events(video_id);
""")
self._conn.commit()
# ------------------------------------------------------------------
# videos
# ------------------------------------------------------------------
def get_video_by_filename(self, filename: str) -> Optional[sqlite3.Row]:
cur = self._conn.execute("SELECT * FROM videos WHERE filename=?", (filename,))
return cur.fetchone()
def ensure_video(self, filename: str, local_path: str,
camera_name: str = '', event_start_time: str = '',
duration_sec: float = 0.0, drive_file_id: str = '') -> int:
"""视频进入监听目录时登记；已存在则更新路径。返回 video_id。"""
now = _now_iso()
row = self.get_video_by_filename(filename)
if row:
self._conn.execute(
"UPDATE videos SET local_path=?, camera_name=?, event_start_time=?, "
"duration_sec=?, updated_at=? WHERE id=?",
(local_path, camera_name, event_start_time, duration_sec, now, row['id']))
self._conn.commit()
return row['id']
cur = self._conn.execute(
"INSERT INTO videos (drive_file_id, filename, local_path, camera_name, "
"duration_sec, event_start_time, status, created_at, updated_at) "
"VALUES (?,?,?,?,?,?, 'pending', ?, ?)",
(drive_file_id, filename, local_path, camera_name, duration_sec,
event_start_time, now, now))
self._conn.commit()
return cur.lastrowid
def get_pending_videos(self, limit: int = 1) -> List[sqlite3.Row]:
cur = self._conn.execute(
"SELECT * FROM videos WHERE status IN ('pending','failed') "
"ORDER BY id ASC LIMIT ?", (limit,))
return cur.fetchall()
def mark_video_processed(self, video_id: int, summary: str, events: List[dict],
people: List[str], compute_provider: str):
now = _now_iso()
self._conn.execute(
"UPDATE videos SET status='done', summary_json=?, events_json=?, "
"people_json=?, compute_provider=?, updated_at=?, processed_at=? WHERE id=?",
(summary, json.dumps(events, ensure_ascii=False), json.dumps(people, ensure_ascii=False),
compute_provider, now, now, video_id))
# 事件落独立表，便于 NAS 拉取
self._conn.execute("DELETE FROM events WHERE video_id=?", (video_id,))
for ev in events:
self._conn.execute(
"INSERT INTO events (video_id, ts, description, person_list_json, "
"is_attention_event) VALUES (?,?,?,?,?)",
(video_id, ev.get('timestamp', ''), ev.get('description', ''),
json.dumps(ev.get('people', []), ensure_ascii=False),
1 if ev.get('is_attention_event') else 0))
self._conn.commit()
def mark_video_failed(self, video_id: int, error: str = ''):
now = _now_iso()
self._conn.execute(
"UPDATE videos SET status='failed', summary_json=?, updated_at=? WHERE id=?",
(error, now, video_id))
self._conn.commit()
def get_all_videos(self) -> List[sqlite3.Row]:
return self._conn.execute(
"SELECT * FROM videos WHERE status='done' ORDER BY id ASC").fetchall()
# ------------------------------------------------------------------
# people
# ------------------------------------------------------------------
def upsert_person(self, label: str, canonical_name: str = '', source: str = 'llm',
first_seen: str = ''):
now = _now_iso()
row = self._conn.execute("SELECT * FROM people WHERE label=?", (label,)).fetchone()
if row:
# manual 覆盖 llm；llm 不覆盖 manual
if source == 'manual' or row['source'] != 'manual':
self._conn.execute(
"UPDATE people SET canonical_name=?, source=?, appearances=appearances+1, "
"updated_at=? WHERE label=?",
(canonical_name or row['canonical_name'], source, now, label))
else:
self._conn.execute(
"UPDATE people SET appearances=appearances+1, updated_at=? WHERE label=?",
(now, label))
else:
self._conn.execute(
"INSERT INTO people (label, canonical_name, first_seen, appearances, "
"source, updated_at) VALUES (?,?,?,1,?,?)",
(label, canonical_name, first_seen or now, source, now))
self._conn.commit()
def set_canonical(self, label: str, canonical_name: str, source: str = 'manual'):
"""手动命名：设置规范名（label 可视为别名）。"""
self.upsert_person(label, canonical_name, source='manual')
def get_people(self) -> List[sqlite3.Row]:
return self._conn.execute("SELECT * FROM people ORDER BY id ASC").fetchall()
def get_known_members_context(self) -> str:
"""生成 known_members_context 文本，注入视频提示让模型用真名。"""
rows = self.get_people()
lines = []
for r in rows:
name = r['canonical_name'] or r['label']
if name and name != r['label']:
lines.append(f"- {name}（别名/标识：{r['label']}）")
else:
lines.append(f"- {name}")
return '\n'.join(lines) if lines else ''
# ------------------------------------------------------------------
# 同步导出（供 NAS 拉取）
# ------------------------------------------------------------------
def get_sync_delta(self, since_iso: str) -> Dict:
"""返回 since 之后变更的 videos / events / people。"""
videos = self._conn.execute(
"SELECT * FROM videos WHERE updated_at > ? ORDER BY id ASC", (since_iso,)
).fetchall()
events = self._conn.execute(
"SELECT * FROM events WHERE updated_at > ? ORDER BY id ASC", (since_iso,)
).fetchall() if False else self._conn.execute(
"SELECT e.* FROM events e JOIN videos v ON e.video_id=v.id "
"WHERE v.updated_at > ? ORDER BY e.id ASC", (since_iso,)).fetchall()
people = self._conn.execute(
"SELECT * FROM people WHERE updated_at > ? ORDER BY id ASC", (since_iso,)
).fetchall()
def _ser(row):
d = dict(row)
return d
return {
"videos": [_ser(v) for v in videos],
"events": [_ser(e) for e in events],
"people": [_ser(p) for p in people],
"server_time": _now_iso(),
}
# ------------------------------------------------------------------
# 同步游标
# ------------------------------------------------------------------
def get_cursor(self, key: str) -> str:
row = self._conn.execute("SELECT value FROM sync_cursor WHERE key=?", (key,)).fetchone()
return row['value'] if row else ''
def set_cursor(self, key: str, value: str):
self._conn.execute(
"INSERT INTO sync_cursor (key, value) VALUES (?, ?) "
"ON CONFLICT(key) DO UPDATE SET value=excluded.value", (key, value))
self._conn.commit()
def close(self):
self._conn.close()

174
fam-edge/src/fam_edge/person_service.py Normal file
View File

@@ -0,0 +1,174 @@
"""
PersonService - 独立人物识别/汇总服务
职责:
1. 汇总所有视频中出现的人物（来自 OracleDB.people / videos.people_json）
2. 用 LLM（Gemini）将跨视频的人物标签合并为规范身份集
（不再依赖 OpenCV 人脸，纯靠视频 LLM 输出的人物标签 + 上下文由大模型判断合并）
3. 维护 people 表 canonical_name，生成 known_members_context
4. 回灌给视频分析提示（video_processor 每次分析前读取 known_members_context）
5. 接收 NAS 手动命名校正（source='manual' 优先，不被 LLM 覆盖）
注意: 无 face embedding，合并基于标签文本 + 描述上下文的大模型判断，保守合并。
"""
import json
import re
import threading
import time
from typing import Dict, List, Optional
from .logger import setup_logger
from .config_loader import load_config
from .model_adapters.adapter_factory import build_adapters
from . import oracle_db
logger = setup_logger('fam-edge.person_service')
class PersonService:
def __init__(self, db: oracle_db.OracleDB):
self.config = load_config()
self.db = db
self.interval = self.config.get('person_service', {}).get('schedule_interval_sec', 1800)
self.enabled = self.config.get('person_service', {}).get('enabled', True)
self._timer: Optional[threading.Timer] = None
self._stop = False
# 选一个 vision 模型做合并（通常 gemini）
adapters = build_adapters(self.config.get('models', []))
model_name = self.config.get('person_service', {}).get('model', 'gemini')
self._llm = next((a for a in adapters if a.provider_name == model_name), None)
if self._llm is None and adapters:
self._llm = adapters[0]
# ------------------------------------------------------------------
def reconcile(self):
"""汇总 + LLM 合并一次。可由定时或手动触发。"""
# 1. 先把所有视频的 people_mentioned 同步进 people 表（标签级）
for v in self.db.get_all_videos():
try:
people = json.loads(v['people_json'] or '[]')
except (ValueError, TypeError):
people = []
for p in people:
if p and p not in ('无人', '无'):
self.db.upsert_person(p, source='llm')
# 2. 收集未命名（无 canonical 或 canonical==label）的标签 + 描述样本
rows = self.db.get_people()
manual = {r['label']: r['canonical_name'] for r in rows if r['source'] == 'manual' and r['canonical_name']}
unnamed = [r for r in rows if not r['canonical_name'] or r['canonical_name'] == r['label']]
if not unnamed:
logger.info("PersonService: 无待合并人物，跳过 LLM 合并")
return
samples = self._collect_descriptions([r['label'] for r in unnamed])
mapping = self._llm_merge(unnamed, samples)
if not mapping:
return
for label, canonical in mapping.items():
if label in manual:
continue # 手动命名优先
if canonical and canonical != label:
self.db.set_canonical(label, canonical, source='llm')
logger.info(f"PersonService: LLM 合并完成，更新 {len(mapping)} 条")
def _collect_descriptions(self, labels: List[str]) -> Dict[str, List[str]]:
"""从 events 表收集每个标签出现时的描述样本。"""
samples: Dict[str, List[str]] = {l: [] for l in labels}
rows = self.db._conn.execute(
"SELECT description, person_list_json FROM events").fetchall()
for r in rows:
try:
plist = json.loads(r['person_list_json'] or '[]')
except (ValueError, TypeError):
plist = []
for p in plist:
if p in samples and len(samples[p]) < 3 and r['description']:
samples[p].append(r['description'])
return samples
def _llm_merge(self, unnamed: List, samples: Dict[str, List[str]]) -> Dict[str, str]:
"""请 LLM 把标签合并为规范名。返回 {label: canonical}。"""
if self._llm is None:
logger.warning("PersonService: 无可用的 LLM 适配器，跳过合并")
return {}
lines = []
for r in unnamed:
label = r['label']
desc = '；'.join(samples.get(label, [])) or '（无描述）'
lines.append(f"- {label}：出现场景 {desc}")
prompt = f"""你是家庭监控人物汇总助手。下面是若干人物标识及其出现场景描述。
请判断哪些标识指向同一个人，并为每个人输出一个稳定的规范名（用'人物A'/'人物B'这类占位，
或若场景描述足以区分则保留原标识）。只输出 JSON，格式：
{{"<原标识>": "<规范名>", ...}}
不要编造真实姓名，仅做去重/合并。
待处理人物：
{chr(10).join(lines)}"""
try:
text = self._llm.chat(prompt, max_tokens=1024)
except Exception as e:
logger.warning(f"PersonService: LLM 调用失败: {e}")
return {}
if not text:
return {}
return self._parse_mapping(text, {r['label'] for r in unnamed})
@staticmethod
def _parse_mapping(text: str, valid_labels) -> Dict[str, str]:
text = text.strip()
try:
data = json.loads(text)
except json.JSONDecodeError:
fence = re.search(r'```(?:json)?\s*(\{.*?\})\s*```', text, re.DOTALL)
if fence:
try:
data = json.loads(fence.group(1))
except json.JSONDecodeError:
return {}
else:
brace = re.search(r'\{.*\}', text, re.DOTALL)
if brace:
try:
data = json.loads(brace.group(0))
except json.JSONDecodeError:
return {}
else:
return {}
out = {}
for k, v in data.items():
if k in valid_labels and v and isinstance(v, str):
out[k] = v
return out
# ------------------------------------------------------------------
# 定时循环
# ------------------------------------------------------------------
def start(self):
if not self.enabled:
logger.info("PersonService 未启用")
return
self.reconcile() # 启动即跑一次
self._schedule_next()
def _schedule_next(self):
if self._stop:
return
self._timer = threading.Timer(self.interval, self._tick)
self._timer.daemon = True
self._timer.start()
def _tick(self):
if self._stop:
return
try:
self.reconcile()
except Exception as e:
logger.error(f"PersonService tick 异常: {e}")
self._schedule_next()
def stop(self):
self._stop = True
if self._timer:
self._timer.cancel()

35
fam-edge/src/fam_edge/qa.py Normal file
View File

@@ -0,0 +1,35 @@
"""
QA - 智能问答编排
run_qa(prompt): 按 models 顺序尝试 chat()，首个成功返回 (answer, provider)。
顺序 = vision 模型(Gemini -> NVIDIA) + text 模型(Ollama 兜底)。
即 Gemini -> NVIDIA -> Ollama 三级降级。
"""
from typing import Optional, Tuple
from .logger import setup_logger
from .config_loader import load_config
from .model_adapters.adapter_factory import build_adapters
logger = setup_logger('fam-edge.qa')
class QAOrchestrator:
def __init__(self):
self.config = load_config()
self.adapters = build_adapters(self.config.get('models', []))
def run_qa(self, prompt: str,
max_tokens: int = 1024) -> Tuple[Optional[str], Optional[str]]:
"""依次尝试各适配器的 chat()，返回 (answer, provider)。"""
for adapter in self.adapters:
try:
answer = adapter.chat(prompt, max_tokens=max_tokens)
except Exception as e:
logger.warning(f"QA {adapter.provider_name} 异常: {e}")
continue
if answer:
logger.info(f"QA 命中 provider={adapter.provider_name}")
return answer, adapter.provider_name
logger.info(f"QA {adapter.provider_name} 无返回，降级下一模型")
return None, None

5
fam-edge/src/fam_edge/queue/__init__.py
View File

@@ -1,5 +0,0 @@
"""
SQLite 异步任务队列 (Oracle 端)
"""
from . import queue_manager
from .consumer import get_consumer

128
fam-edge/src/fam_edge/queue/consumer.py
View File

@@ -1,128 +0,0 @@
"""
消费者线程 - 从 SQLite 队列消费任务，限速处理
策略：Gemini 优先 → NVIDIA 兜底（与现有 orchestrator 一致）
速率：按 API 限制速度的 2 倍设置突发容量
"""
import os
import time
import threading
from typing import Optional
from ..logger import setup_logger
from ..config_loader import load_config
from ..rate_limiter import RateLimiter
from ..ai_orchestrator.orchestrator import AIOrchestrator
from ..video_preprocessor.preprocessor import VideoPreprocessor
from . import queue_manager
logger = setup_logger('fam-edge.consumer')
# 从配置加载速率限制参数
_cfg = load_config()
_queue_cfg = _cfg.get('queue', {})
_rate_cfg = _queue_cfg.get('rate_limit', {})
GEMINI_RPM = _rate_cfg.get('gemini_rpm', 1000)
NVIDIA_RPM = _rate_cfg.get('nvidia_rpm', 40)
BURST_FACTOR = _rate_cfg.get('burst_factor', 2)
POLL_INTERVAL = _queue_cfg.get('poll_interval', 10)
# 同步 SQLite DB 路径到环境变量（供 queue_manager 读取）
os.environ.setdefault('FAM_QUEUE_DB', _queue_cfg.get('db_path', '/opt/fam-edge/data/fam_queue.db'))
os.environ.setdefault('FAM_UPLOAD_DIR', _queue_cfg.get('upload_dir', '/tmp/fam_uploads'))
class Consumer:
def __init__(self):
self._running = False
self._thread = None
self._orchestrator = AIOrchestrator()
self._rate_limiter = RateLimiter()
self._rate_limiter.register('gemini', GEMINI_RPM, burst_factor=BURST_FACTOR)
self._rate_limiter.register('nvidia', NVIDIA_RPM, burst_factor=BURST_FACTOR)
self._poll_interval = POLL_INTERVAL
def _process_one(self, task: dict) -> bool:
task_id = task['id']
nas_task_id = task['nas_task_id']
video_path = task['video_path']
logger.info(f"[nas_task={nas_task_id}] 消费者开始处理")
preprocessor = None
try:
preprocessor = VideoPreprocessor(nas_task_id)
task_data = {
"task_id": nas_task_id,
"camera_name": task.get('camera_name', ''),
"event_start_time": task.get('event_start_time', ''),
"event_end_time": "",
"known_members_context": task.get('known_members_context', ''),
}
result = self._orchestrator.process_push_task(
task_data, video_path, preprocessor, self._rate_limiter
)
if result.get('status') == 'success':
import json
queue_manager.mark_success(task_id, json.dumps(result, ensure_ascii=False))
logger.info(f"[nas_task={nas_task_id}] 消费者处理成功")
return True
else:
error = result.get('error_message', 'unknown')
stage = result.get('failure_stage', '')
queue_manager.mark_failed(task_id, error, stage)
logger.error(f"[nas_task={nas_task_id}] 消费者处理失败: {error}")
return False
except Exception as e:
logger.error(f"[nas_task={nas_task_id}] 消费者异常: {e}", exc_info=True)
queue_manager.mark_failed(task_id, str(e), 'process')
return False
finally:
if preprocessor is not None:
preprocessor.cleanup()
try:
if video_path and __import__('os').path.exists(video_path):
__import__('os').remove(video_path)
logger.info(f"[nas_task={nas_task_id}] 清理视频文件: {video_path}")
except Exception:
pass
def _run(self):
logger.info(f"消费者线程启动，轮询间隔 {self._poll_interval}s")
logger.info(f"速率限制: Gemini {GEMINI_RPM}RPM x2 burst, NVIDIA {NVIDIA_RPM}RPM x2 burst")
while self._running:
try:
task = queue_manager.claim_next()
if task is None:
time.sleep(self._poll_interval)
continue
self._process_one(task)
except Exception as e:
logger.error(f"消费者循环异常: {e}", exc_info=True)
time.sleep(self._poll_interval)
def start(self):
if self._running:
return
self._running = True
self._thread = threading.Thread(target=self._run, daemon=True, name='consumer')
self._thread.start()
logger.info("消费者线程已启动")
def stop(self):
self._running = False
logger.info("消费者线程已停止")
_consumer: Optional[Consumer] = None
def get_consumer() -> Consumer:
global _consumer
if _consumer is None:
_consumer = Consumer()
return _consumer

209
fam-edge/src/fam_edge/queue/queue_manager.py
View File

@@ -1,209 +0,0 @@
"""
SQLite 队列管理器 - Oracle 端异步任务队列
表结构:
- task_queue: 任务队列 (PENDING → PROCESSING → SUCCESS/FAILED)
- 元数据: delivered 标记 NAS 是否已拉取结果
"""
import os
import sqlite3
import json
import threading
from typing import Optional, List, Dict
DB_PATH = os.environ.get('FAM_QUEUE_DB', '/opt/fam-edge/data/fam_queue.db')
_init_lock = threading.Lock()
_initialized = False
def _get_conn() -> sqlite3.Connection:
global _initialized
if not _initialized:
with _init_lock:
if not _initialized:
_init_db()
_initialized = True
conn = sqlite3.connect(DB_PATH, timeout=30)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA journal_mode=WAL")
return conn
_TASK_QUEUE_DDL = """
CREATE TABLE {name} (
id INTEGER PRIMARY KEY AUTOINCREMENT,
nas_task_id INTEGER NOT NULL,
video_filename TEXT NOT NULL,
video_path TEXT NOT NULL,
camera_name TEXT DEFAULT '',
event_start_time TEXT DEFAULT '',
known_members_context TEXT DEFAULT '',
status TEXT DEFAULT 'PENDING',
result_json TEXT,
error_message TEXT,
failure_stage TEXT,
retry_count INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now', '+8 hours')),
updated_at TEXT DEFAULT (datetime('now', '+8 hours')),
delivered INTEGER DEFAULT 0,
UNIQUE(nas_task_id)
)
"""
def _init_db():
os.makedirs(os.path.dirname(DB_PATH), exist_ok=True)
conn = sqlite3.connect(DB_PATH)
# DEFAULT 约束固化在表 schema 中，CREATE TABLE IF NOT EXISTS 不会更新已存在的旧表
# （旧表 DEFAULT 是 localtime，UTC 机器上=UTC）。检测到旧 schema 时重建表迁移。
row = conn.execute(
"SELECT sql FROM sqlite_master WHERE type='table' AND name='task_queue'"
).fetchone()
if row is not None and 'localtime' in (row[0] or ''):
conn.execute("BEGIN IMMEDIATE")
conn.execute(_TASK_QUEUE_DDL.format(name='task_queue_new'))
conn.execute("INSERT INTO task_queue_new SELECT * FROM task_queue")
conn.execute("DROP TABLE task_queue")
conn.execute("ALTER TABLE task_queue_new RENAME TO task_queue")
conn.commit()
conn.execute(_TASK_QUEUE_DDL.format(name='task_queue').replace(
'CREATE TABLE task_queue', 'CREATE TABLE IF NOT EXISTS task_queue'))
conn.execute("CREATE INDEX IF NOT EXISTS idx_status ON task_queue(status)")
conn.execute("CREATE INDEX IF NOT EXISTS idx_delivered ON task_queue(delivered)")
conn.commit()
conn.close()
def enqueue(nas_task_id: int, video_filename: str, video_path: str,
camera_name: str, event_start_time: str,
known_members_context: str) -> int:
conn = _get_conn()
try:
cur = conn.execute(
"INSERT OR IGNORE INTO task_queue "
"(nas_task_id, video_filename, video_path, camera_name, event_start_time, "
"known_members_context, created_at, updated_at) "
"VALUES (?, ?, ?, ?, ?, ?, datetime('now','+8 hours'), datetime('now','+8 hours'))",
(nas_task_id, video_filename, video_path, camera_name, event_start_time, known_members_context)
)
conn.commit()
if cur.rowcount == 0:
row = conn.execute(
"SELECT id, status, delivered FROM task_queue WHERE nas_task_id=?",
(nas_task_id,)
).fetchone()
if row is None:
return 0
# NAS 重新派发: FAILED（或已交付的 SUCCESS）重置为 PENDING，用新上传的视频重跑。
# SUCCESS 且未交付的行不动，避免丢失待 Poller 拉取的结果。
if row['status'] == 'FAILED' or (row['status'] == 'SUCCESS' and row['delivered']):
conn.execute(
"UPDATE task_queue SET status='PENDING', result_json=NULL, error_message=NULL, "
"failure_stage=NULL, retry_count=0, delivered=0, video_filename=?, video_path=?, "
"camera_name=?, event_start_time=?, known_members_context=?, "
"updated_at=datetime('now','+8 hours') WHERE id=?",
(video_filename, video_path, camera_name, event_start_time,
known_members_context, row['id'])
)
conn.commit()
return row['id']
return cur.lastrowid
finally:
conn.close()
def claim_next() -> Optional[Dict]:
conn = _get_conn()
try:
conn.execute("BEGIN IMMEDIATE")
row = conn.execute(
"SELECT * FROM task_queue WHERE status='PENDING' ORDER BY id LIMIT 1"
).fetchone()
if row:
conn.execute(
"UPDATE task_queue SET status='PROCESSING', updated_at=datetime('now','+8 hours') WHERE id=?",
(row['id'],)
)
conn.commit()
return dict(row)
conn.rollback()
return None
except Exception:
conn.rollback()
return None
finally:
conn.close()
def mark_success(task_id: int, result_json: str):
conn = _get_conn()
try:
conn.execute(
"UPDATE task_queue SET status='SUCCESS', result_json=?, updated_at=datetime('now','+8 hours') WHERE id=?",
(result_json, task_id)
)
conn.commit()
finally:
conn.close()
def mark_failed(task_id: int, error_message: str, failure_stage: str = ''):
conn = _get_conn()
try:
conn.execute(
"UPDATE task_queue SET status='FAILED', error_message=?, failure_stage=?, "
"updated_at=datetime('now','+8 hours') WHERE id=?",
(error_message, failure_stage, task_id)
)
conn.commit()
finally:
conn.close()
def get_undelivered_results(limit: int = 10) -> List[Dict]:
conn = _get_conn()
try:
# FAILED 也需交付: 否则 NAS 永远收不到失败结果，任务卡 PROCESSING
# 直至僵尸回收后无意义地重传 22MB 视频
rows = conn.execute(
"SELECT * FROM task_queue WHERE status IN ('SUCCESS','FAILED') AND delivered=0 "
"ORDER BY id LIMIT ?", (limit,)
).fetchall()
return [dict(r) for r in rows]
finally:
conn.close()
def mark_delivered(task_ids: List[int]):
if not task_ids:
return
conn = _get_conn()
try:
placeholders = ','.join('?' * len(task_ids))
conn.execute(
f"UPDATE task_queue SET delivered=1, updated_at=datetime('now','+8 hours') "
f"WHERE id IN ({placeholders})", task_ids
)
conn.commit()
finally:
conn.close()
def get_queue_stats() -> Dict:
conn = _get_conn()
try:
stats = {}
for status in ['PENDING', 'PROCESSING', 'SUCCESS', 'FAILED']:
row = conn.execute(
"SELECT COUNT(*) as cnt FROM task_queue WHERE status=?", (status,)
).fetchone()
stats[status] = row['cnt']
row = conn.execute(
"SELECT COUNT(*) as cnt FROM task_queue "
"WHERE status IN ('SUCCESS','FAILED') AND delivered=0"
).fetchone()
stats['UNDELIVERED'] = row['cnt']
return stats
finally:
conn.close()

48
fam-edge/src/fam_edge/rate_limiter.py
View File

@@ -1,48 +0,0 @@
"""
Rate Limiter - Token Bucket 算法
按 API 限制速度的 2 倍设置突发容量，按 API 限制速度持续补充。
"""
import time
import threading
class TokenBucket:
def __init__(self, rpm: int, burst_factor: int = 2):
self.capacity = rpm * burst_factor
self.refill_rate = rpm / 60.0
self.tokens = float(self.capacity)
self.last_refill = time.monotonic()
self._lock = threading.Lock()
def acquire(self, tokens: int = 1, timeout: float = 300.0) -> bool:
deadline = time.monotonic() + timeout
while True:
with self._lock:
now = time.monotonic()
elapsed = now - self.last_refill
self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
self.last_refill = now
if self.tokens >= tokens:
self.tokens -= tokens
return True
wait = (tokens - self.tokens) / self.refill_rate
if time.monotonic() + wait > deadline:
return False
time.sleep(min(wait, 1.0))
class RateLimiter:
"""多 API 速率限制管理"""
def __init__(self):
self._buckets = {}
def register(self, name: str, rpm: int, burst_factor: int = 2):
self._buckets[name] = TokenBucket(rpm, burst_factor)
def acquire(self, name: str, tokens: int = 1, timeout: float = 300.0) -> bool:
bucket = self._buckets.get(name)
if bucket is None:
return True
return bucket.acquire(tokens, timeout)

19
fam-edge/src/fam_edge/state.py Normal file
View File

@@ -0,0 +1,19 @@
"""
state - 进程内共享单例（OracleDB 实例）
watch_processor / person_service / api_gateway 都通过 get_db() 访问同一个 SQLite 连接，
避免重复打开与循环 import。
"""
from . import oracle_db
from .config_loader import load_config
_db = None
def get_db() -> oracle_db.OracleDB:
global _db
if _db is None:
cfg = load_config()
path = cfg.get('oracle_db', {}).get('path', '/opt/fam-edge/data/oracle.db')
_db = oracle_db.OracleDB(path)
return _db

4
fam-edge/src/fam_edge/storage_cleaner/__init__.py
View File

@@ -1,4 +0,0 @@
"""Storage-Cleaner 包"""
from .cleaner import StorageCleaner
__all__ = ["StorageCleaner"]

82
fam-edge/src/fam_edge/storage_cleaner/cleaner.py
View File

@@ -1,82 +0,0 @@
"""
Storage-Cleaner - 临时文件清理
首期仅 finally 清理（不做 Cron 兜底）
- 删除下载的视频文件
- 删除粗抽候选帧
- 删除压缩关键帧
- 清理任务工作目录
"""
import os
import shutil
from ..logger import setup_logger, log_task
logger = setup_logger('fam-edge.storage_cleaner')
class StorageCleaner:
"""临时文件清理器"""
def __init__(self, task_id: int, work_dir: str):
self.task_id = task_id
self.work_dir = work_dir
def cleanup(self):
"""清理整个工作目录"""
try:
if os.path.exists(self.work_dir):
# 统计清理前大小
total_size = 0
for dirpath, dirnames, filenames in os.walk(self.work_dir):
for f in filenames:
fp = os.path.join(dirpath, f)
try:
total_size += os.path.getsize(fp)
except OSError:
pass
shutil.rmtree(self.work_dir)
size_mb = total_size / (1024 * 1024)
log_task(logger, self.task_id, 'cleanup',
f'已清理工作目录: {self.work_dir} ({size_mb:.1f}MB)')
else:
log_task(logger, self.task_id, 'cleanup',
f'工作目录不存在，无需清理: {self.work_dir}')
except PermissionError as e:
logger.warning(f"[task_id={self.task_id}] 清理权限不足: {e}")
except Exception as e:
logger.warning(f"[task_id={self.task_id}] 清理异常: {e}")
def cleanup_file(self, filepath: str):
"""清理单个文件"""
try:
if os.path.exists(filepath):
os.remove(filepath)
logger.info(f"[task_id={self.task_id}] 已删除: {filepath}")
except Exception as e:
logger.warning(f"[task_id={self.task_id}] 删除文件失败 {filepath}: {e}")
@staticmethod
def cleanup_stale_dirs(base_dir='/tmp/fam_media', max_age_hours=24):
"""清理超期的残留目录（超过 max_age_hours 的 task_* 目录）
首期不通过 Cron 调用，可在进程启动时手动执行一次。
"""
if not os.path.isdir(base_dir):
return
import time
now = time.time()
max_age_seconds = max_age_hours * 3600
for entry in os.listdir(base_dir):
entry_path = os.path.join(base_dir, entry)
if not os.path.isdir(entry_path) or not entry.startswith('task_'):
continue
try:
dir_mtime = os.path.getmtime(entry_path)
if now - dir_mtime > max_age_seconds:
shutil.rmtree(entry_path)
logger.info(f"清理超期残留目录: {entry_path}")
except Exception as e:
logger.warning(f"清理残留目录失败 {entry_path}: {e}")

4
fam-edge/src/fam_edge/video_preprocessor/__init__.py
View File

@@ -1,4 +0,0 @@
"""Video-Preprocessor 包"""
from .preprocessor import VideoPreprocessor
__all__ = ["VideoPreprocessor"]

316
fam-edge/src/fam_edge/video_preprocessor/preprocessor.py
View File

@@ -1,316 +0,0 @@
"""
Video-Preprocessor - 视频预处理
流程:
1. 下载视频（超时 60s）
2. 根据视频时长自适应计算候选帧数，FFmpeg 等距粗抽
3. 根据视频时长自适应计算关键帧数，OpenCV 帧差分析筛选（MSE 阈值）
4. 压缩（长边 ≤ 1024px，JPEG 质量 80）
自适应规则:
- 候选帧: max(candidate_min, duration_min * candidate_per_minute), 上限 candidate_max
- 关键帧: max(min_key_frames, duration / key_frame_interval_sec), 上限 max_key_frames_cap
例: 30分钟视频 → 候选60张 → 关键帧12张（每2.5分钟1张）
例: 3分钟视频 → 候选30张 → 关键帧8张（保底）
异常兜底:
- ffprobe 失败 -> 退化为按 60s 间隔抽帧
- 帧差分析异常 -> 退化为等距抽 min_key_frames 帧
- OpenCV 压缩失败 -> 跳过该帧，记录 WARN
"""
import os
import time
import subprocess
import requests
import cv2
import numpy as np
from typing import List, Tuple, Optional
from ..logger import setup_logger, log_task
from ..config_loader import load_config
logger = setup_logger('fam-edge.preprocessor')
class VideoPreprocessor:
"""视频预处理器"""
def __init__(self, task_id: int):
self.task_id = task_id
cfg = load_config()
video_cfg = cfg.get('video', {})
self.candidate_per_minute = video_cfg.get('candidate_per_minute', 2)
self.candidate_min = video_cfg.get('candidate_min', 30)
self.candidate_max = video_cfg.get('candidate_max', 120)
self.key_frame_interval_sec = video_cfg.get('key_frame_interval_sec', 150)
self.min_key_frames = video_cfg.get('min_key_frames', 5)
self.max_key_frames_floor = video_cfg.get('max_key_frames_floor', 8)
self.max_key_frames_cap = video_cfg.get('max_key_frames_cap', 30)
self.mse_threshold = video_cfg.get('mse_threshold', 500)
self.jpeg_quality = video_cfg.get('jpeg_quality', 80)
self.max_long_edge = video_cfg.get('max_long_edge', 1024)
timeout_cfg = cfg.get('timeout', {})
self.download_timeout = timeout_cfg.get('download', 60)
# 视频时长（秒），在 extract_candidate_frames 中填充
self.video_duration = 0.0
# 临时目录
self.work_dir = f"/tmp/fam_media/task_{task_id}"
self.video_path = os.path.join(self.work_dir, f"video_{task_id}.mp4")
self.frames_dir = os.path.join(self.work_dir, "frames")
self.keyframes_dir = os.path.join(self.work_dir, "keyframes")
def download_video(self, video_url: str) -> str:
"""下载视频"""
os.makedirs(self.work_dir, exist_ok=True)
start = time.time()
log_task(logger, self.task_id, 'download', f'开始下载: {video_url}')
resp = requests.get(video_url, stream=True, timeout=self.download_timeout)
if resp.status_code != 200:
raise Exception(f"下载失败: HTTP {resp.status_code}")
with open(self.video_path, 'wb') as f:
for chunk in resp.iter_content(chunk_size=8192):
f.write(chunk)
duration_ms = int((time.time() - start) * 1000)
size_mb = os.path.getsize(self.video_path) / (1024 * 1024)
log_task(logger, self.task_id, 'download', f'下载完成: {size_mb:.1f}MB', duration_ms=duration_ms)
return self.video_path
def save_upload(self, file_storage) -> str:
"""保存推送模式上传的视频文件（multipart），替代 download_video"""
os.makedirs(self.work_dir, exist_ok=True)
start = time.time()
file_storage.save(self.video_path)
duration_ms = int((time.time() - start) * 1000)
size_mb = os.path.getsize(self.video_path) / (1024 * 1024)
log_task(logger, self.task_id, 'upload',
f'保存上传视频: {size_mb:.1f}MB', duration_ms=duration_ms)
return self.video_path
def _get_video_duration(self, video_path: str) -> float:
"""用 ffprobe 获取视频时长（秒）"""
try:
cmd = [
'ffprobe', '-v', 'error',
'-show_entries', 'format=duration',
'-of', 'default=noprint_wrappers=1:nokey=1',
video_path
]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
if result.returncode == 0:
return float(result.stdout.strip())
except Exception as e:
logger.warning(f"[task_id={self.task_id}] ffprobe 失败: {e}")
return 0.0
def extract_candidate_frames(self, video_path: str) -> List[str]:
"""等距粗抽候选帧（数量随视频时长自适应，使用快速 seek）"""
os.makedirs(self.frames_dir, exist_ok=True)
duration = self._get_video_duration(video_path)
self.video_duration = duration
if duration > 0:
duration_min = duration / 60
# 自适应候选帧数：每分钟 candidate_per_minute 张，保底 candidate_min，上限 candidate_max
candidate_count = min(
max(self.candidate_min, int(duration_min * self.candidate_per_minute)),
self.candidate_max
)
interval = duration / candidate_count
else:
# 兜底: 每 60s 抽一帧
interval = 60
candidate_count = 0
logger.warning(f"[task_id={self.task_id}] ffprobe 失败，退化为 60s 间隔抽帧")
# 快速 seek 逐帧提取（比 fps 滤镜快 6-8 倍，ARM CPU 上尤甚）
timestamps = [i * interval for i in range(candidate_count)] if candidate_count > 0 else []
if not timestamps:
# 兜底: 未知时长，用 ffprobe 不可用时按 60s 间隔
timestamps = [i * 60 for i in range(30)]
for i, ts in enumerate(timestamps):
output_path = os.path.join(self.frames_dir, f'frame_{i+1:04d}.jpg')
cmd = [
'ffmpeg', '-ss', f'{ts:.1f}',
'-i', video_path,
'-frames:v', '1',
'-q:v', '2',
output_path
]
try:
subprocess.run(cmd, capture_output=True, timeout=30, check=True)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired) as e:
logger.warning(f"[task_id={self.task_id}] seek 到 {ts:.1f}s 失败: {e}")
# 收集候选帧路径
frames = sorted([
os.path.join(self.frames_dir, f)
for f in os.listdir(self.frames_dir)
if f.endswith('.jpg')
])
log_task(logger, self.task_id, 'extract',
f'视频时长 {duration:.0f}s, 快速 seek 粗抽 {len(frames)} 张候选帧 (目标 {candidate_count})')
return frames
def _compute_adaptive_key_frame_counts(self) -> Tuple[int, int]:
"""根据视频时长自适应计算关键帧下限和上限"""
if self.video_duration > 0:
# 每隔 key_frame_interval_sec 秒 1 张关键帧
adaptive = int(self.video_duration / self.key_frame_interval_sec)
max_kf = min(max(self.max_key_frames_floor, adaptive), self.max_key_frames_cap)
else:
max_kf = self.max_key_frames_floor
min_kf = max(self.min_key_frames, max_kf // 2)
return min_kf, max_kf
def select_key_frames(self, candidate_frames: List[str]) -> List[str]:
"""帧差分析筛选关键帧（数量随视频时长自适应）"""
min_kf, max_kf = self._compute_adaptive_key_frame_counts()
log_task(logger, self.task_id, 'select_keyframes',
f'自适应关键帧: min={min_kf}, max={max_kf} (视频时长 {self.video_duration:.0f}s)')
if len(candidate_frames) <= min_kf:
return candidate_frames[:max_kf]
try:
# 加载所有候选帧
images = []
for path in candidate_frames:
img = cv2.imread(path)
if img is not None:
images.append((path, img))
if len(images) < 2:
return candidate_frames[:max_kf]
# 计算每帧与前一关键帧的 MSE
key_indices = [0] # 首帧必选
last_key_img = images[0][1]
for i in range(1, len(images)):
mse = self._compute_mse(last_key_img, images[i][1])
if mse > self.mse_threshold:
key_indices.append(i)
last_key_img = images[i][1]
# 末帧必选
if key_indices[-1] != len(images) - 1:
key_indices.append(len(images) - 1)
# 若 < min_kf，从剩余中均匀补足
if len(key_indices) < min_kf:
remaining = [i for i in range(len(images)) if i not in key_indices]
step = max(1, len(remaining) // (min_kf - len(key_indices)))
for i in range(0, len(remaining), step):
if len(key_indices) >= min_kf:
break
key_indices.append(remaining[i])
key_indices.sort()
# 若 > max_kf，按差异值降序取前 N
if len(key_indices) > max_kf:
# 计算每个关键帧与前一帧的差异
diffs = []
for idx in key_indices[1:-1]: # 不含首末帧
diff = self._compute_mse(images[idx-1][1], images[idx][1])
diffs.append((idx, diff))
diffs.sort(key=lambda x: x[1], reverse=True)
# 保留首末帧 + 差异最大的
keep = {0, len(images)-1}
for idx, _ in diffs[:max_kf - 2]:
keep.add(idx)
key_indices = sorted(keep)
key_frames = [images[i][0] for i in key_indices]
log_task(logger, self.task_id, 'select_keyframes', f'筛选 {len(key_frames)} 张关键帧')
return key_frames
except Exception as e:
logger.warning(f"[task_id={self.task_id}] 帧差分析异常: {e}，退化为等距抽 {min_kf} 帧")
step = max(1, len(candidate_frames) // min_kf)
return candidate_frames[::step][:min_kf]
def _compute_mse(self, img1, img2) -> float:
"""计算两帧的 MSE"""
# 转灰度并统一尺寸
h = min(img1.shape[0], img2.shape[0])
w = min(img1.shape[1], img2.shape[1])
g1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
g2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
g1 = cv2.resize(g1, (w, h))
g2 = cv2.resize(g2, (w, h))
diff = g1.astype(np.float64) - g2.astype(np.float64)
mse = np.mean(diff ** 2)
return float(mse)
def compress_frames(self, frame_paths: List[str]) -> List[str]:
"""压缩关键帧（长边 ≤ max_long_edge，JPEG 质量 80）"""
os.makedirs(self.keyframes_dir, exist_ok=True)
compressed = []
for i, path in enumerate(frame_paths):
out_path = os.path.join(self.keyframes_dir, f"keyframe_{i+1:02d}.jpg")
try:
img = cv2.imread(path)
if img is None:
logger.warning(f"[task_id={self.task_id}] 读取图片失败: {path}")
continue
h, w = img.shape[:2]
if max(h, w) > self.max_long_edge:
scale = self.max_long_edge / max(h, w)
img = cv2.resize(img, (int(w * scale), int(h * scale)))
cv2.imwrite(out_path, img, [cv2.IMWRITE_JPEG_QUALITY, self.jpeg_quality])
compressed.append(out_path)
except Exception as e:
logger.warning(f"[task_id={self.task_id}] 压缩失败 {path}: {e}")
continue
log_task(logger, self.task_id, 'compress', f'压缩 {len(compressed)} 张关键帧')
return compressed
def compute_timestamps(self, video_path: str, frame_count: int,
event_start_time: str) -> List[str]:
"""计算每帧的绝对时间戳 = 视频开始时间 + 帧偏移"""
from datetime import datetime, timedelta
duration = self._get_video_duration(video_path)
if duration <= 0:
duration = frame_count * 60 # 兜底
interval = duration / frame_count
from datetime import timedelta, timezone
# 统一北京时区: 视频均为北京时间录制，Edge 机器是 UTC，
# fallback 不能用本地 datetime.now()
try:
start_dt = datetime.fromisoformat(event_start_time.replace('Z', '+00:00'))
if start_dt.tzinfo is not None:
start_dt = start_dt.astimezone(timezone(timedelta(hours=8))).replace(tzinfo=None)
except Exception:
start_dt = datetime.now(timezone(timedelta(hours=8))).replace(tzinfo=None)
timestamps = []
for i in range(frame_count):
offset = interval * i
ts = start_dt + timedelta(seconds=offset)
timestamps.append(ts.strftime('%Y-%m-%d %H:%M:%S'))
return timestamps
def cleanup(self):
"""清理临时文件"""
import shutil
try:
if os.path.exists(self.work_dir):
shutil.rmtree(self.work_dir)
log_task(logger, self.task_id, 'cleanup', f'清理临时目录: {self.work_dir}')
except Exception as e:
logger.warning(f"[task_id={self.task_id}] 清理失败: {e}")

132
fam-edge/src/fam_edge/video_processor.py Normal file
View File

@@ -0,0 +1,132 @@
"""
VideoProcessor - 整视频分析编排
流程（不再切片/抽帧）:
1. 从 OracleDB 取当前 known_members_context（已命名/合并的人物）
2. 按 vision_order 依次调适配器的 analyze_video（Gemini 整视频 -> NVIDIA 整视频）
3. 首个成功结果 -> 归一化 -> 写 OracleDB（videos + events 表）
4. 把本视频 people_mentioned 更新进 people 表（供 person_service 后续合并）
降级: 全部视觉模型失败 -> 标记视频 failed（不再本地融合）
"""
import os
import re
from datetime import datetime, timedelta, timezone
from typing import Dict, List, Optional
from .logger import setup_logger
from .config_loader import load_config
from .model_adapters.adapter_factory import build_adapters
from .model_adapters.base_adapter import BaseModelAdapter
from . import oracle_db
logger = setup_logger('fam-edge.video_processor')
def _parse_event_start_from_filename(filename: str) -> str:
"""从监控文件名解析开始时间（北京时间）。示例: 2026-08-21_081500.mp4"""
m = re.search(r'(\d{4})[-_](\d{2})[-_](\d{2})[_-]?(\d{2})(\d{2})(\d{2})', filename)
if m:
y, mo, d, hh, mm, ss = m.groups()
try:
dt = datetime(int(y), int(mo), int(d), int(hh), int(mm), int(ss))
return dt.strftime('%Y-%m-%d %H:%M:%S')
except ValueError:
pass
# 退而求其次: 2026-08-21 08-15-00 等
m2 = re.search(r'(\d{4}-\d{2}-\d{2})[ _T-]+(\d{2})[-:](\d{2})[-:](\d{2})', filename)
if m2:
return f"{m2.group(1)} {m2.group(2)}:{m2.group(3)}:{m2.group(4)}"
return ''
class VideoProcessor:
def __init__(self, db: oracle_db.OracleDB):
self.config = load_config()
self.db = db
self.vision_order = self.config.get('video_processing', {}).get(
'vision_order', ['gemini', 'nvidia'])
self.vision_timeout = self.config.get('video_processing', {}).get('timeout', 900)
self.parse_start = self.config.get('gdrive_sync', {}).get(
'parse_start_from_filename', True)
adapters = build_adapters(self.config.get('models', []))
self.vision_adapters: Dict[str, BaseModelAdapter] = {
a.provider_name: a for a in adapters if a.get_role() == 'vision'}
def _ordered_vision_adapters(self) -> List[BaseModelAdapter]:
ordered = []
for name in self.vision_order:
if name in self.vision_adapters:
ordered.append(self.vision_adapters[name])
# 追加未在顺序里但启用的视觉适配器
for name, a in self.vision_adapters.items():
if name not in self.vision_order:
ordered.append(a)
return ordered
def process_video(self, video_id: int, filename: str, local_path: str) -> bool:
"""处理一个视频记录，返回是否成功。"""
if not os.path.isfile(local_path):
logger.error(f"[video_id={video_id}] 文件不存在，跳过: {local_path}")
self.db.mark_video_failed(video_id, "file_missing")
return False
camera_name = self.db.get_video_by_filename(filename)['camera_name'] or ''
event_start = ''
if self.parse_start:
event_start = _parse_event_start_from_filename(filename)
# 回写解析到的开始时间
if event_start:
self.db._conn.execute(
"UPDATE videos SET event_start_time=? WHERE id=?",
(event_start, video_id))
self.db._conn.commit()
known = self.db.get_known_members_context()
logger.info(f"[video_id={video_id}] 开始整视频分析: {filename} "
f"(event_start={event_start}, known_members={'有' if known else '无'})")
last_err = "no_vision_adapter"
for adapter in self._ordered_vision_adapters():
try:
logger.info(f"[video_id={video_id}] 尝试 {adapter.provider_name} 整视频分析")
result = adapter.analyze_video(local_path, known, event_start)
except Exception as e:
logger.error(f"[video_id={video_id}] {adapter.provider_name} 异常: {e}")
last_err = str(e)
continue
if result:
self._store_result(video_id, result)
return True
else:
last_err = f"{adapter.provider_name}_failed"
logger.warning(f"[video_id={video_id}] {adapter.provider_name} 未返回结果，降级下一模型")
logger.error(f"[video_id={video_id}] 所有视觉模型失败，标记 failed: {last_err}")
self.db.mark_video_failed(video_id, last_err)
return False
def _store_result(self, video_id: int, result: Dict):
events = result.get('events', [])
people = result.get('people_mentioned', [])
summary = result.get('global_summary', '')
provider = result.get('compute_provider', 'unknown')
# 归一化 events 时间戳（若模型给的是相对偏移，这里不强制；以模型输出为准）
norm_events = []
for ev in events:
norm_events.append({
"timestamp": str(ev.get('timestamp', '')),
"description": str(ev.get('description', '')),
"people": [str(p) for p in ev.get('people', []) if p],
"is_attention_event": bool(ev.get('is_attention_event', False)),
})
self.db.mark_video_processed(video_id, summary, norm_events, people, provider)
# 更新 people 表（标签级，待 person_service 合并）
for p in people:
if p and p not in ('无人', '无'):
self.db.upsert_person(p, source='llm')
logger.info(f"[video_id={video_id}] 已落库: summary={len(summary)}字, "
f"events={len(norm_events)}, people={people}")

89
fam-edge/src/fam_edge/watch_processor.py Normal file
View File

@@ -0,0 +1,89 @@
"""
WatchProcessor - 监听 Google 硬盘同步落地目录，处理新视频
流程:
1. rclone 已把 Google 硬盘目录实时同步到 local_dir（视频文件）
2. 每 watch_interval_sec 轮询一次 local_dir
3. 发现未在 videos 表登记的文件 -> ensure_video 登记
4. 取 pending/failed 的视频，逐个整视频分析（max_concurrent=1，串行避免过载）
"""
import os
import time
import threading
from typing import List
from .logger import setup_logger
from .config_loader import load_config
from . import oracle_db
from .video_processor import VideoProcessor
logger = setup_logger('fam-edge.watch_processor')
VIDEO_EXTS = ('.mp4', '.mkv', '.avi', '.mov', '.ts')
class WatchProcessor:
def __init__(self, db: oracle_db.OracleDB):
self.config = load_config()
self.db = db
self.local_dir = self.config.get('gdrive_sync', {}).get('local_dir', '/opt/fam-edge/gdrive_videos')
self.interval = self.config.get('gdrive_sync', {}).get('watch_interval_sec', 30)
self.camera_name = self.config.get('gdrive_sync', {}).get('camera_name', '摄像头')
self.max_concurrent = self.config.get('video_processing', {}).get('max_concurrent', 1)
self.processor = VideoProcessor(db)
self._running = False
self._thread = None
def _scan_files(self) -> List[str]:
if not os.path.isdir(self.local_dir):
logger.warning(f"监听目录不存在: {self.local_dir}")
return []
out = []
for fn in sorted(os.listdir(self.local_dir)):
if fn.lower().endswith(VIDEO_EXTS):
out.append(fn)
return out
def _register_new(self, files: List[str]):
for fn in files:
if self.db.get_video_by_filename(fn) is None:
path = os.path.join(self.local_dir, fn)
self.db.ensure_video(fn, path, camera_name=self.camera_name)
logger.info(f"登记新视频: {fn}")
def _process_pending(self):
pending = self.db.get_pending_videos(limit=self.max_concurrent)
for row in pending:
try:
self.processor.process_video(row['id'], row['filename'], row['local_path'])
except Exception as e:
logger.error(f"处理视频 {row['filename']} 异常: {e}", exc_info=True)
self.db.mark_video_failed(row['id'], f"watch_error: {e}")
def _run(self):
logger.info(f"WatchProcessor 启动，监听 {self.local_dir}，间隔 {self.interval}s")
while self._running:
try:
files = self._scan_files()
self._register_new(files)
self._process_pending()
except Exception as e:
logger.error(f"WatchProcessor 轮询异常: {e}", exc_info=True)
# 处理完一小批后休眠
for _ in range(self.interval):
if not self._running:
break
time.sleep(1)
def start(self):
if self._running:
return
self._running = True
self._thread = threading.Thread(target=self._run, daemon=True, name='watch')
self._thread.start()
def is_alive(self):
return self._thread is not None and self._thread.is_alive()
def stop(self):
self._running = False

8
fam-ui/config/config.yaml
View File

@@ -1,5 +1,5 @@
# FAM-UI 配置文件 (NAS 端) - 实际部署配置 # FAM-UI 配置文件 (NAS 端) - 新架构 v2（2026-08-21）
# Tailscale: NAS=100.70.234.39 # NAS 仅作管理后台，前端读本地 MariaDB 同步镜像，不再读取关键帧图片。
core_url: "http://127.0.0.1:8000" core_url: "http://127.0.0.1:8000"
@@ -10,7 +10,3 @@ database:
password: "iLoveJava5!" password: "iLoveJava5!"
database: "sentinel_home_ai" database: "sentinel_home_ai"
unix_socket: "/run/mysqld/mysqld10.sock" unix_socket: "/run/mysqld/mysqld10.sock"
storage:
# 关键帧目录（fam-core event_receiver 落盘，UI 读取展示时间轴）
frame_image_dir: "/volume1/web/sentinel-home-ai/fam-ui/static/frames"

801
fam-ui/src/app.py
View File

File diff suppressed because it is too large Load Diff

61
scripts/ddl.sql
View File

@@ -121,6 +121,67 @@ CREATE TABLE IF NOT EXISTS family_members (
INDEX idx_is_active (is_active) INDEX idx_is_active (is_active)
) ENGINE=InnoDB COMMENT='家庭成员表（交互式命名）'; ) ENGINE=InnoDB COMMENT='家庭成员表（交互式命名）';
-- ============================================================
-- 7. 甲骨文同步镜像表（新架构 v2，2026-08-21）
-- NAS 每 30 分钟从甲骨文 FAM-Edge 拉增量，镜像到本地，仅作展示
-- 字段对齐 Oracle 端 SQLite 库（oracle_db.py）
-- ============================================================
-- 7.1 视频会话表（Oracle videos 镜像）
CREATE TABLE IF NOT EXISTS sync_videos (
id INT PRIMARY KEY COMMENT 'Oracle videos.id',
drive_file_id VARCHAR(255) COMMENT 'Google 硬盘文件 ID',
filename VARCHAR(500) NOT NULL UNIQUE COMMENT '视频文件名（唯一）',
camera_name VARCHAR(50) COMMENT '摄像头名称/位置',
duration_sec DOUBLE DEFAULT 0 COMMENT '视频时长（秒）',
event_start_time VARCHAR(32) COMMENT '视频开始时间（文本）',
status VARCHAR(20) DEFAULT 'pending' COMMENT 'pending/done/failed',
summary_json LONGTEXT COMMENT '全局摘要文本',
events_json LONGTEXT COMMENT '事件列表 JSON 数组（冗余，便于查询）',
people_json LONGTEXT COMMENT '人物列表 JSON 数组',
compute_provider VARCHAR(255) COMMENT '模型来源，如 gemini / nvidia',
created_at VARCHAR(32),
updated_at VARCHAR(32),
processed_at VARCHAR(32),
synced_at DATETIME DEFAULT CURRENT_TIMESTAMP COMMENT '最近一次同步写入时间',
INDEX idx_processed (processed_at),
INDEX idx_status (status)
) ENGINE=InnoDB COMMENT='甲骨文视频会话镜像表';
-- 7.2 事件明细表（Oracle events 镜像）
CREATE TABLE IF NOT EXISTS sync_events (
id INT PRIMARY KEY COMMENT 'Oracle events.id',
video_id INT NOT NULL COMMENT '关联 sync_videos.id',
ts VARCHAR(32) COMMENT '事件时间点（文本）',
description TEXT COMMENT '事件描述',
person_list_json LONGTEXT COMMENT '涉及人物 JSON 数组（字符串或标签）',
is_attention_event TINYINT(1) DEFAULT 0 COMMENT 'AI 判断是否为关注事件',
updated_at VARCHAR(32),
synced_at DATETIME DEFAULT CURRENT_TIMESTAMP,
INDEX idx_video (video_id),
INDEX idx_ts (ts)
) ENGINE=InnoDB COMMENT='甲骨文事件镜像表';
-- 7.3 人物规范表（Oracle people 镜像）
CREATE TABLE IF NOT EXISTS sync_people (
id INT PRIMARY KEY COMMENT 'Oracle people.id',
label VARCHAR(100) NOT NULL UNIQUE COMMENT '抽象标识，如"人物A"',
canonical_name VARCHAR(100) COMMENT '规范名（用户命名或 LLM 合并），NULL 表示未命名',
first_seen VARCHAR(32) COMMENT '首次出现时间',
appearances INT DEFAULT 0 COMMENT '出现次数',
source VARCHAR(20) DEFAULT 'llm' COMMENT 'llm / manual（manual 优先不被覆盖）',
updated_at VARCHAR(32),
synced_at DATETIME DEFAULT CURRENT_TIMESTAMP,
INDEX idx_label (label),
INDEX idx_canonical (canonical_name)
) ENGINE=InnoDB COMMENT='甲骨文人物规范镜像表';
-- 7.4 同步游标表
CREATE TABLE IF NOT EXISTS sync_cursor (
key VARCHAR(50) PRIMARY KEY,
value VARCHAR(64) COMMENT '上次成功拉取到的 server_time（ISO 文本）'
) ENGINE=InnoDB COMMENT='同步游标表';
-- ============================================================ -- ============================================================
-- 验证 -- 验证
-- ============================================================ -- ============================================================