Compare commits
10 Commits
feb096ff3d
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
deaebf22b4 | ||
|
|
3b5f7db51d | ||
|
|
a494d60361 | ||
|
|
da60365214 | ||
|
|
1964e976f4 | ||
|
|
dc284f4bf5 | ||
|
|
0238030128 | ||
|
|
2ad0472578 | ||
|
|
db4f46140b | ||
|
|
9c4db682d5 |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -42,3 +42,6 @@ logs/
|
||||
# Env
|
||||
.env
|
||||
.env.local
|
||||
|
||||
# 其他独立项目（各有独立仓库，勿提交进本仓库）
|
||||
ai-gateway/
|
||||
|
||||
43
.workbuddy/memory/2026-08-31.md
Normal file
43
.workbuddy/memory/2026-08-31.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# 2026-08-31 工作日志
|
||||
|
||||
## fam-core 登录接入 auth-hub 统一登录（已部署 + 端到端验证）
|
||||
|
||||
### 目标（用户明确）
|
||||
`http://129.146.26.249:8000`（fam-core 公网入口，frp → NAS:8000）的登录改走 auth-hub
|
||||
统一登录，账号 `ericwyuan` / `iLoveJava5` 登录。
|
||||
|
||||
### 现状与动作
|
||||
- SSO 代码本地早已提交（`8120d2a feat(fam-core): 登录改接 auth-hub 统一登录`），NAS 未部署（旧密码登录版）。
|
||||
- **auth-hub 侧（甲骨文 /opt/auth-hub）**：
|
||||
- FAM-Core client `a2PJTZuKQYxt1oGh`（已存在）→ 轮换 secret 得明文
|
||||
`yZfJfD3xJmrCj9PQlbt7P8An9ebZ-nT2f3v4BTtOSu8`（旧密钥失效）
|
||||
- redirect_uris 增加 `http://129.146.26.249:8000/api/auth/callback`（原 :80 回调保留，
|
||||
auth-hub 精确匹配，用户入口是 :8000 所以回调也要 :8000 路径）
|
||||
- **NAS 侧（fam-core）**：
|
||||
- venv 补装 `PyJWT==2.13.0` + `cryptography==50.0.1`（RS256 验签必需）
|
||||
- 部署新 `auth.py`（cat 管道覆盖）
|
||||
- `.env` 追加 4 个变量：`AUTH_HUB_ISSUER=http://129.146.26.249:5300`、
|
||||
`AUTH_HUB_CLIENT_ID=a2PJTZuKQYxt1oGh`、`AUTH_HUB_CLIENT_SECRET=<上面>`、
|
||||
`AUTH_HUB_REDIRECT_URI=http://129.146.26.249:8000/api/auth/callback`
|
||||
- 清 __pycache__、杀双 gunicorn（今天 18:56 又一个旧进程 + 8/29 旧 master）、单实例重启
|
||||
|
||||
### 端到端验证（curl cookie jar 全链路）
|
||||
`/login` 302 → auth-hub `/authorize`（PKCE）→ hub 登录（ericwyuan/iLoveJava5 200）→
|
||||
`authorize` 302 回 `:8000/api/auth/callback?code&state` → fam-core 换 token+JWKS 验签 →
|
||||
种 fam_session → `/api/auth/check` = `{"authed":true}`，`/api/status` 200 ✓
|
||||
|
||||
### 注意
|
||||
- 本地 repo 3 个提交待 push（8120d2a SSO + 2 docs），等用户确认推送
|
||||
- NAS 仍跑旧版其余代码（webhook 移除等未部署，独立待办）
|
||||
- 密码登录 `/api/login` 已移除；fam-ui SPA 无登录逻辑，不受影响
|
||||
|
||||
## 补充：webhook/旧前端清理部署到 NAS（用户确认"清理"）
|
||||
|
||||
- 全量比对 NAS vs repo：仅 `app.py`、`motion_bp.py`、`config.yaml` 三处是旧版（其余已同步）
|
||||
- 同步：3 个文件（app.py 移除 static_bp/refresh_camera_map 调用；motion_bp.py 移除 webhook 端点；
|
||||
config.yaml 移除 camera_name_to_id/enrich_window_sec）
|
||||
- 删除废弃：`static_app.py`、`fam-ui/` 目录、根 `start_ui.sh`、`scripts/start_ui.sh`
|
||||
- 清 pycache + 单实例重启
|
||||
- 验证：启动日志无 `refresh_camera_map` 告警；`/api/ss/webhook` → 401（已移除白名单与路由）；
|
||||
SSO 登录 `/login` 302 → auth-hub 正常、`{"authed":true}`；`/api/ss/status` 200；Oracle 同步正常
|
||||
- 至此 NAS fam-core 代码与本地 repo 全量一致（除 .env/venv/logs）
|
||||
39
.workbuddy/memory/2026-09-03.md
Normal file
39
.workbuddy/memory/2026-09-03.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# 2026-09-03 同步 1062 复发 + Oracle 盒子过载
|
||||
|
||||
## 问题
|
||||
用户报 sync 状态卡在「同步中」+ `1062 Duplicate entry '...mp4' for key 'filename'`。
|
||||
游标停在 `2026-09-02 13:55:21`。
|
||||
|
||||
## 根因（与 8/29 identity_map 同一类）
|
||||
Oracle FAM-Edge 的 SQLite 视频/人物 id 整体重排（2303 段 → 3408/4600+ 段），
|
||||
NAS 镜像表 `sync_videos`/`sync_people` 旧 upsert 以 Oracle id 当主键插入，
|
||||
与已存在的同 `filename`/`label` 行在 UNIQUE 键上二次冲突 → 1062，整批中止、游标不推进。
|
||||
量化：本次积压 348 videos 中 **114 行**会 1062；9 people 中 **6 行**会 1062。
|
||||
events/model_calls 仅 PK id（ON DUPLICATE 按 id 更新，不会 1062）→ 无需改。
|
||||
|
||||
## 修复（已部署 NAS + 提交推送 0238030）
|
||||
- `upsert_sync_videos`：改以 `filename` 业务键去重，命中就地 UPDATE 保留 NAS 原 id
|
||||
（防 sync_events/model_calls/identity_map 的 video_id 外键失效），Oracle id 落
|
||||
`oracle_id` 列溯源；未命中插入优先用 Oracle id 对齐子表引用，主键冲突回退自增。
|
||||
- `upsert_sync_people`：同模式，以 `label` 业务键去重（people.id 无外键引用）。
|
||||
- `scripts/ddl.sql`：sync_videos/sync_people 的 id 改 NAS 本地自增 + 新增 oracle_id 列。
|
||||
- NAS 现网迁移：两表加 oracle_id、回填 oracle_id=id、id 改 AUTO_INCREMENT，重启。
|
||||
- 验证：单批补拉 videos+348 events+1017 people+9 model_calls+500 identity_map+238，
|
||||
游标 → `2026-09-03 08:51:11`，无 1062。目标行保留 id=2303、oracle_id=3408。
|
||||
|
||||
## 第二个问题：Oracle 盒子过载/挂死（未解决，环境层）
|
||||
NAS 日志 08:51–08:56 持续：`推送运动事件到 Oracle 失败 Connection reset` /
|
||||
`500 {"error":"database is locked"}` / `RemoteDisconnected`。
|
||||
FAM-Edge /health 先 200（status ok）后变 HTTP 000；最终整盒 129.146.26.249 全暗
|
||||
（:80/:5000/:5300/:7000/:8123 均 000），但 TCP :22/:5000 端口 OPEN、SSH 握手被拒
|
||||
（kex 阶段 Connection closed）→ 进程卡死/资源耗尽（4C23G 上 FAM-Edge 分析管线 +
|
||||
ai-gateway + garmin + auth-hub + frps + caddy 挤一起，SQLite 锁竞争 + 可能 OOM）。
|
||||
**SSH 连不进去，无法远程修复。** 需用户在 Oracle Cloud 控制台重启 VM，或等其自恢复。
|
||||
恢复后 NAS 的 oracle_sync 后台线程（30min 周期）会自动从游标 08:51:11 续拉，
|
||||
motion_notifier 也会恢复推送。
|
||||
|
||||
## 经验
|
||||
- 所有以 Oracle id 为镜像主键的表迟早会因 Oracle 库重建/重排踩坑。已修：
|
||||
identity_map(8/29)、videos+people(9/3)。events/model_calls 目前 PK-only 安全，
|
||||
但若将来 Oracle 重建导致子表引用错位，需同样改造（加 oracle_id + 业务键）。
|
||||
- Oracle 盒子服务过多挤在 4C23G，建议拆分或给 FAM-Edge SQLite 开 WAL + 限速。
|
||||
@@ -26,6 +26,11 @@
|
||||
手动 setsid 会端口冲突；venv `/opt/fam-edge/venv`；库 `/opt/fam-edge/data/oracle.db`；
|
||||
素材 `/opt/fam-edge/gdrive_videos`；logs 目录 ubuntu 可写。
|
||||
**前端亦托管于此机 Caddy :80**（fam-ui SPA + /api 反代回 NAS fam-core :8000 经 frp 隧道）；旧机 129.146.203.203 已退役。
|
||||
- **登录（2026-08-31 起）**：fam-core 不再自己校验密码，登录全权走 auth-hub 统一登录
|
||||
（OAuth2 Auth Code + PKCE / OIDC，`http://129.146.26.249:5300`）。fam-core client
|
||||
`a2PJTZuKQYxt1oGh`，回调 `:8000/api/auth/callback`（auth-hub 侧注册了 :80 和 :8000 两个回调）。
|
||||
NAS `.env` 配 `AUTH_HUB_ISSUER/CLIENT_ID/CLIENT_SECRET/REDIRECT_URI`（缺任一 fail closed）。
|
||||
账号：`ericwyuan / iLoveJava5`（hub 端）。NAS venv 需 PyJWT + cryptography（RS256 验签）。
|
||||
- 部署：本地 git 提交 push Gitea → tar 管道（NAS 直接解包；Oracle `--strip-components=1` 到临时目录再 cp，避免动 data/venv）。
|
||||
- **NAS 部署坑（2026-08-29 踩过）**：拷贝 `.py` 会带旧 mtime，若 `.pyc` 比 `.py` 新，Python 加载陈旧字节码
|
||||
（旧逻辑上线，曾导致 1062 假象）。部署后必须 `find src -name __pycache__ -exec rm -rf {} +` 再重启；
|
||||
|
||||
142
PROGRESS.md
142
PROGRESS.md
@@ -506,3 +506,145 @@ AI 分析瓶颈: 帧5耗时 229s (疑似 ARM CPU 热降频), 其余帧 50-65s
|
||||
- 修复：导航改为页面顶部 `st.segmented_control` 横向分段控制器，任何屏宽直接可见；侧边栏保留品牌标题和任务队列状态。
|
||||
|
||||
浏览器验证：顶部分段导航 5 页可达、成员命名页显示 张三/汤圆/爸、AI 对话页表单正常、无报错。
|
||||
|
||||
## 摄像头对外入口收口为单一域名 smart-camera.zichuan.xyz (2026-09-01)
|
||||
|
||||
**决策**：摄像头系统（fam-ui 前端 + NAS fam-core :8000 后端）对外只暴露一个域名 `smart-camera.zichuan.xyz`（前端 `/` + 后端 `/api`），下线独立的 `api.zichuan.xyz`，并从 `oracle.zichuan.xyz` 移除摄像头相关路由（其 `/ai`、`/fam`、`/wordpress` 仍保留给其它服务）。
|
||||
|
||||
**改动（基础设施，不在本仓库）**：
|
||||
- Oracle Caddy（`/etc/caddy/Caddyfile`）：新增 `smart-camera.zichuan.xyz` 块（`/` 静态 fam-ui + `/api`、`/login` 反代 `127.0.0.1:8000`，经 frp 隧道回源 NAS）；删除 `api.zichuan.xyz` 块；`oracle.zichuan.xyz` 与 `:80` 兜底块移除摄像头 root + `/api` 路由（改 404）。
|
||||
- DNS：新增 `smart-camera.zichuan.xyz → 129.146.26.249`（RecordId 2387553886）。
|
||||
- NAS `/volume1/web/sentinel-home-ai/.env`：`AUTH_HUB_ISSUER` 改为 `https://auth.zichuan.xyz`、`AUTH_HUB_REDIRECT_URI` 改为 `https://smart-camera.zichuan.xyz/api/auth/callback`，且 `AUTH_HUB_*` 四行补全 `export` 前缀（见下）。
|
||||
- auth-hub FAM-Core 客户端 `a2PJTZuKQYxt1oGh` 回调改为 `https://smart-camera.zichuan.xyz/api/auth/callback`（保留旧 IP 作兜底）。
|
||||
|
||||
**顺带修复两个生产 Bug**（见 `scripts/start_core.sh`）：
|
||||
1. 脚本先 `cd "$APP_DIR"` 再算相对 `SCRIPT_DIR`，导致 `source .env` 路径错位、`set -e` 下直接退出，gunicorn 起不来。
|
||||
2. `.env` 里 `AUTH_HUB_*` 是裸赋值（无 `export`），`source` 后不进环境，gunicorn 子进程读不到 → 登录一直 503 fail-closed（与之前“生产未部署”一致）。脚本加 `set -a` 包裹 source 修复。
|
||||
|
||||
**验证**：`smart-camera.zichuan.xyz/` 200、`/login` 302→auth.zichuan.xyz（redirect_uri=smart-camera）、`/api/*` 401；`oracle.zichuan.xyz/` 404、`/fam` 200；`api.zichuan.xyz` 已不通。fam-core 经修复脚本重启后 health ok。
|
||||
|
||||
## SPA 登录打通：未登录自动跳转 OIDC（2026-09-01）
|
||||
|
||||
**问题**：域名收口后访问 `https://smart-camera.zichuan.xyz/timeline` 显示「⚠ 未登录 / 登陆不了啊」。后端 `/login`（302→auth-hub）与 Caddy `/login` 反代此前已验证可用，根因在**前端零登录逻辑**：`fam-ui` 调 `/api/*` 拿到 401 后只在页面上渲染字面错误 `data.error`（"未登录"），从不发起 OIDC 跳转，也没有任何登录入口。
|
||||
|
||||
**改动（本仓库 `fam-ui/`）**：
|
||||
- `src/api.js` `request()`：捕获 401 时 `window.location.href = '/login'` 触发统一登录（后端经 auth-hub 走 Authorization Code + PKCE）；用模块级 `_redirectingToLogin` 开关保证单次会话只跳一次，并 `return new Promise(()=>{})` 阻止调用方继续渲染错误态；`/api/auth/check` 等白名单接口不会 401，不受影响。
|
||||
- `src/api.js` `api` 对象新增 `authCheck: () => request('/api/auth/check')`。
|
||||
- `src/App.vue`：`onMounted` 调 `api.authCheck()` 维护 `authed` 状态；左侧栏 + 移动端顶栏新增「🔑 登录」入口（`<a href="/login">`）与「👋 退出登录」按钮（`POST /api/logout` 后回 `/` 由后端 401 自动跳登录）。
|
||||
|
||||
**部署**：`npm run build` → `dist/` 经 `tar | ssh ubuntu@129.146.26.249` 覆盖 `/var/www/fam-ui`（macOS `._*` 元数据已清）。
|
||||
|
||||
**端到端验证**：
|
||||
- `smart-camera.zichuan.xyz/timeline` → 200（SPA）
|
||||
- `/login` → 302 → `auth.zichuan.xyz/authorize?...&redirect_uri=https://smart-camera.zichuan.xyz/api/auth/callback`
|
||||
- `/api/auth/check`（无 cookie）→ `{"authed":false}`（白名单，不 401）
|
||||
- `/api/ui/videos`（无 cookie）→ 401 → 前端据此自动跳 `/login`
|
||||
|
||||
登录链路已通：未登录访问任意页面 → 首个 401 → 自动跳 auth-hub 登录 → 回调种 `fam_session` cookie → 回 `/timeline` 正常加载。
|
||||
|
||||
## 统一登录从 NAS 迁到甲骨文 fam-edge（2026-09-12）
|
||||
|
||||
**触发**：`smart-camera.zichuan.xyz/login` 报 502。排查链路：静态页 `/`、`/timeline` 200（Caddy 读本机磁盘），
|
||||
只有 `/login` 和 `/api/*` 502；甲骨文本机 `curl 127.0.0.1:8000/health` 0.6 秒空响应（curl exit 52，
|
||||
Caddy 日志 `msg:"EOF"`），frps 正常、同隧道的 gitea 也正常 → NAS 上 fam-core 进程没了。
|
||||
局域网直扫 NAS：22/2222/3000/5000/5001/3306 全 OPEN，**只有 8000 closed**，NAS 本身没事。
|
||||
根因是登录入口挂在 NAS 上，而 NAS 上的 fam-core 没有任何守护（DSM 无 systemd），挂了不会自启。
|
||||
|
||||
**决策**：登录是入口，不该依赖家里的机器。整体迁到甲骨文的 fam-edge（前端静态文件、auth-hub 本来就在这台），
|
||||
NAS fam-core 退化成纯数据接口。
|
||||
|
||||
**改动**：
|
||||
- 新增 `fam-edge/src/fam_edge/auth.py`：`/login`、`/api/auth/callback`、`/api/logout`、
|
||||
`/api/auth/check`、`/api/auth/verify`（给 Caddy forward_auth）。跟旧实现三处关键差异：
|
||||
1. 换 token / 拉 JWKS 走 `AUTH_HUB_INTERNAL_BASE`（本机 :5300），不再跨公网 TLS——旧链路上
|
||||
`PyJWKClient` 用 urllib + 系统 CA（群晖易 CERTIFICATE_VERIFY_FAILED）、两机时钟偏差会让 `iat`
|
||||
显得来自未来，这两个坑一起消失；但 `iss` 校验和浏览器跳转仍用公网 issuer
|
||||
2. 会话改无状态 HS256 签名 cookie，服务重启不掉线（旧实现进程内 token 表）
|
||||
3. **回调失败渲染错误页，不再 302 回 `/login`**——旧实现失败即跳 `/login`，而 auth-hub 只要还有
|
||||
会话就立刻再签一个 code 跳回来，两边对跳成死循环，浏览器只报「重定向次数过多」，
|
||||
既看不到登录页也看不到原因（这正是 9/1 那次「跳不到登录页」的成因）
|
||||
- 删除 `fam-core/src/fam_core/auth.py` + `tests/test_auth.py`，`app.py` 去掉 `init_auth`；
|
||||
fam-core 不再有任何鉴权，改由甲骨文 Caddy `forward_auth` 前置拦截
|
||||
- `fam-edge/tests/test_auth.py` 16 个用例（含「失败分支绝不 302」的回归测试、
|
||||
「服务端走内网但 iss 按公网校验」、「cookie 无状态」）；fam-edge 157 / fam-core 15 全绿
|
||||
- 写测试时逮到自己写的一个 bug：会话 cookie 也套了 60 秒 leeway（本进程自签自验根本不需要），
|
||||
会让每个会话白白多活 60 秒，已改成只有 id_token 用 leeway
|
||||
|
||||
**遗留风险（已记入 README/DEPLOY）**：frps 在甲骨文绑的是 `*:8000` 且 iptables 明确放行，
|
||||
fam-core 去掉鉴权后，绕过 Caddy 直连 `129.146.26.249:8000` 就是无门禁的全量数据接口，
|
||||
必须靠 `iptables -I INPUT 1 -p tcp --dport 8000 ! -i lo -j DROP` 兜底。
|
||||
另：登录进程并到 fam-edge 后，fam-edge 正在跑视频分析时登录响应可能变慢（单 worker 4 线程）。
|
||||
|
||||
## 全量迁云：NAS 只剩推送进程（2026-09-13）
|
||||
|
||||
**触发**：前一天刚把登录迁到甲骨文，隔天 NAS 上的 fam-core 又挂了导致数据接口 502。
|
||||
用户一句话点破：「NAS 上只有一个无状态的通知甲骨文的服务，剩下的全部在甲骨文呀」。
|
||||
盘点后确认这个判断成立——甲骨文的 SQLite 才是权威数据源（videos 3113 / events 16159 /
|
||||
people 60 / model_calls 9876），NAS 的 MariaDB 全是它的镜像，前端读的数据本来就产自
|
||||
甲骨文，绕了一圈回家又绕回来。
|
||||
|
||||
**改动**：
|
||||
- `fam-core/db_layer.py` 从 725 行重写成 377 行：MySQL 镜像查询 → 直读 fam-edge 的
|
||||
SQLite。5 个 `upsert_sync_*`（约 300 行去重逻辑，8/29 和 9/3 两次 1062 事故的发源地）
|
||||
连同 `oracle_sync.py` 整个删除。SQL 方言：`JSON_CONTAINS` → `json_each`（前置
|
||||
`json_valid`，历史脏数据不会把查询搞崩）、`LEFT()` → `substr()`、`%s` → `?`。
|
||||
函数名 `get_sync_*` 一并改掉——已经没有 sync 这回事了，留着名字会误导。
|
||||
- 新增 `fam-core/edge_client.py`：写操作（改名/删除）、帧图头像、服务状态都打给同机
|
||||
fam-edge，全走 127.0.0.1，不出公网。
|
||||
- 新增 `fam-notifier/`：把 `motion_notifier.py` 从 fam-core 拆出来，游标从 MariaDB
|
||||
换成本地 JSON 文件。NAS 上从此没有 Flask、没有数据库、没有监听端口，只有一个
|
||||
单向推送进程。
|
||||
- fam-core 移到甲骨文 `/opt/fam-core`（systemd，gunicorn -w 2，**只绑 127.0.0.1:5401**——
|
||||
5400 被 chat-relay 占了）。Caddy 的 `/api/*` 从"frp 隧道回源 NAS"改成同机反代，
|
||||
forward_auth 闸门不变。
|
||||
- 前端跟着删：侧边栏同步面板、统计页同步状态、服务状态页的"NAS 同步"卡片和
|
||||
"立即同步"按钮（背后的镜像层已不存在）。"NAS 同步"换成"NAS 运动推送"，读
|
||||
fam-edge activity 新增的 `motion` 段（心跳年龄 + 最近事件）。
|
||||
- 顺带修掉一个隐蔽 bug：镜像表为保外键稳定用的是 NAS 本地自增 id，而帧图接口要的是
|
||||
甲骨文的 id，两边在 9/3 那次 id 重排后就对不上了。现在只有一套 id。
|
||||
|
||||
**测试**：fam-core 21（新增 12 个 db_layer 用例：脏 JSON 不崩、人物精确匹配不误伤
|
||||
"人物B"、日期过滤、统计口径、chat_history 懒建表）、fam-notifier 6、fam-edge 157，全绿。
|
||||
|
||||
**验证**：甲骨文 `/api/ui/stats` 返回 videos 2965 / events 16159 / attention 13 /
|
||||
people 59；外网 `/` `/timeline` 200、`/login` 302、`/api/*` 未登录 401——**全程 NAS
|
||||
上的 fam-core 是停着的**，这就是迁云的验收标准。
|
||||
|
||||
**待办**：NAS 侧部署 fam-notifier（只能用户手动，密码登录）；chat_history 一次性迁移
|
||||
（`fam-core/scripts/import_chat_history.py`，幂等）；frpc.toml 里的 8000 映射可删。
|
||||
|
||||
## 修复 fam-edge 多线程共用 SQLite 连接（2026-09-13）
|
||||
|
||||
**触发**：用户问「甲骨文剩余硬盘小于 10G 会删东西的逻辑怎么没了？」。查下来逻辑一直在、
|
||||
也没被迁云动过（`disk_guard.py` 最后一次改动还是 8/28 新增它那次），但它 95% 的检查在空转：
|
||||
|
||||
```
|
||||
DiskGuard「本轮清理完成」 128 次
|
||||
DiskGuard「检查异常」 2837 次 ← 22 倍
|
||||
```
|
||||
|
||||
失败原因全是 `cannot start a transaction within a transaction`。表现就是磁盘剩余空间在
|
||||
2.7GB 和 16GB 之间来回荡——清理能不能成功全靠运气，赶上 rclone 集中下载
|
||||
（实测 5 分钟写入 12.6GB）就掉进危险区。
|
||||
|
||||
**根因**：`OracleDB.__init__` 建一条 `sqlite3.connect(check_same_thread=False)` 的连接
|
||||
给全进程共用，而 VideoQueue / PersonService / DiskGuard 三个后台线程 + gunicorn 的 4 个
|
||||
请求线程都在并发读写它。sqlite3 的连接对象本来就不是可并发共享的，事务状态互相踩踏。
|
||||
同一个根因在线上刷出三类错误，累计：`database is locked` 71604 次、
|
||||
`cannot start a transaction within a transaction` 378 次、`no more rows available` 92 次
|
||||
（后者堆栈落在 `self._conn.commit()`，是游标被别的线程重置的典型症状）。
|
||||
运动事件推送被 500 打回也是它——NAS 侧失败批次不推进游标，下一轮补推，所以没丢事件。
|
||||
|
||||
**修复**：`_conn` 改成 `@property`，从 `threading.local()` 取当前线程的连接，没有就新建
|
||||
（WAL + busy_timeout=10000）。`close()` 相应改成收掉所有线程开过的连接。因为外部调用方
|
||||
（如 `api_gateway` 的 activity 端点）也在直接用 `db._conn.execute(...)`，做成 property
|
||||
可以让全部现有调用点原样工作，不用逐个改。`_write_lock` 保留，复合写语义不变。
|
||||
|
||||
**测试**：新增 2 个用例（8 线程 × 25 轮并发读写、close 要收掉所有连接）。在旧代码上
|
||||
稳定复现同族错误 `cannot commit transaction - SQL statements in progress`，修复后通过；
|
||||
fam-edge 全套 159 个测试绿。
|
||||
|
||||
**同批修掉的第二个竞态**：生产者列目录之后、读 mtime 之前，DiskGuard 可能刚好把那个
|
||||
文件清掉（两个后台线程的正常竞态），`os.path.getmtime` 抛 FileNotFoundError，代价是
|
||||
**整轮扫描中断**——排在后面的新素材本轮全都登记不上。改成捕获 OSError 跳过该文件，
|
||||
新增 `fam-edge/tests/test_video_queue.py` 两个用例覆盖（被删的跳过 / 仍在写入的照样跳过）。
|
||||
|
||||
70
README.md
70
README.md
@@ -63,9 +63,9 @@ Orchestrator 视觉阶段按 `fallback` 模式顺序降级：Gemini → NVIDIA N
|
||||
|
||||
| 节点 | 角色 | 硬件 | IP | 服务与端口 |
|
||||
|------|------|------|-----|-----------|
|
||||
| Synology NAS | FAM-Core + 数据库（采集 + 聚合 + 镜像层） | DS220+ (Geminilake), DSM 7 | 家庭局域网 192.168.50.64 | FAM-Core :8000（API 聚合 + 轮询 SS + OracleSync）, MariaDB :3306, Surveillance Station :5000；FAM-UI 已迁至云服务器（见下） |
|
||||
| Oracle Cloud（云服务器） | FAM-Edge + AI-Gateway + Caddy（前端） | Ampere A1 4C23G ARM64（无 GPU）, Ubuntu 20.04 | 公网 129.146.26.249 / Tailscale（已安装未启用，备用） | FAM-Edge :5000（systemd 守护，视频分析 + 转发问答）, **AI-Gateway :5100（systemd 守护，对外监听，问答模型降级链）**, Caddy :80（托管 FAM-UI SPA + 反代 /api/* → NAS FAM-Core）, Ollama :11434（仅本地，被 AI-Gateway 调用） |
|
||||
| 家庭网络 | 用户入口 | 普通终端 | 公网 / 家庭网络 | 浏览器访问 `http://129.146.26.249/`（Caddy 托管前端，/api 经 frp 隧道回源 NAS FAM-Core :8000） |
|
||||
| Synology NAS | 摄像头录像 + 运动事件推送（**仅此而已**，2026-09-13 起） | DS220+ (Geminilake), DSM 7 | 家庭局域网 192.168.50.64 | Surveillance Station :5000（录像机本体，搬不走）, **fam-notifier**（无端口的推送进程，轮询 SS → 推甲骨文）。MariaDB 与 FAM-Core 已随镜像层一起下线 |
|
||||
| Oracle Cloud（云服务器） | **除录像外的全部**：视频分析 + 数据 + 接口 + 登录 + 前端 | Ampere A1 4C23G ARM64（无 GPU）, Ubuntu 20.04 | 公网 129.146.26.249 / Tailscale（已安装未启用，备用） | FAM-Edge :5000（systemd，视频分析 + 统一登录 + SQLite 权威库）, **FAM-Core :5401（systemd，UI/对话/成员接口，仅监听 127.0.0.1）**, AI-Gateway :5100（systemd，问答模型降级链）, Caddy :80/:443（前端 + 反代 + forward_auth 鉴权）, Ollama :11434（仅本地） |
|
||||
| 家庭网络 | 用户入口 | 普通终端 | 公网 / 家庭网络 | 浏览器访问 `https://smart-camera.zichuan.xyz/`（全部由甲骨文提供，**不依赖 NAS 在线**） |
|
||||
|
||||
**网络要点（推送模式）**：
|
||||
- 服务间通信只有两条出站：**NAS → Oracle 公网 IP:5000**（① `GET /api/oracle/sync` 拉增量 + `POST /api/oracle/people/correct` 命名回推 ② `POST /api/ss/motion` 推送运动侦测事件）。Edge 不需要反向访问 NAS
|
||||
@@ -98,19 +98,22 @@ Orchestrator 视觉阶段按 `fallback` 模式顺序降级：Gemini → NVIDIA N
|
||||
│ ├─ /v1/chat/completions：NVIDIA → Gemini（多 Key 轮换）→ 本地 Ollama 降级链 │
|
||||
│ ├─ Bearer token 鉴权（fail-closed），对外 0.0.0.0:5100，非本系统专属 │
|
||||
│ └─ 独立 .env（/opt/ai-gateway/.env，NVIDIA/Gemini key 与 FAM-Edge 各自一份） │
|
||||
│ Caddy :80：托管 FAM-UI SPA + 反代 /api/* → NAS FAM-Core（frp 隧道 :8000） │
|
||||
└───────────────────────────────┬───────────────────────────────────────────────┘
|
||||
│ HTTP GET /api/oracle/sync?since=&token= （每 30 分钟）
|
||||
▼
|
||||
│ FAM-Core (Flask :5401，仅 127.0.0.1) │
|
||||
│ ├─ UI-API: /api/ui/*（直读下面那个 SQLite，无镜像、无后台线程） │
|
||||
│ ├─ Chat-Handler: /api/chat/ask（查 events 拼上下文 → 转 fam-edge 编排） │
|
||||
│ ├─ Member-Manager: /api/member/name|merge（转 fam-edge，它负责合并人物） │
|
||||
│ └─ chat_history: 本服务唯一写的表，落在同一个 SQLite 里 │
|
||||
│ SQLite /opt/fam-edge/data/oracle.db（WAL）：videos / events / people / │
|
||||
│ model_calls / person_identity_map / ss_motion_events / chat_history │
|
||||
│ Caddy :80/:443：前端静态 + /login 与 /api/auth/* → fam-edge │
|
||||
│ + /api/* 先 forward_auth 再反代 fam-core（唯一鉴权闸门） │
|
||||
└───────────────────────────────▲───────────────────────────────────────────────┘
|
||||
│ POST /api/ss/motion（单向推送 + 5 分钟心跳）
|
||||
│
|
||||
┌─────────────────────────────── NAS (192.168.50.64) ────────────────────────────┐
|
||||
│ FAM-Core (Flask :8000) │
|
||||
│ ├─ Oracle-Sync: 唯一后台线程，拉增量写 MariaDB 镜像 + 维护 sync_cursor │
|
||||
│ ├─ Chat-Handler: /api/chat/ask（查 sync_events 拼上下文 → 走 Oracle 编排） │
|
||||
│ ├─ Member-Manager: /api/member/name|merge（回推 Oracle + 即时拉回） │
|
||||
│ ▼ │
|
||||
│ MariaDB (sentinel_home_ai): sync_videos / sync_events / sync_people / │
|
||||
│ sync_cursor / chat_history │
|
||||
│ （FAM-UI 已迁至云服务器 Caddy :80，见上） │
|
||||
│ Surveillance Station :5000（录像机本体，摄像头插在这台上） │
|
||||
│ fam-notifier：轮询 SS EventCenter → 推运动事件给甲骨文。无端口、无数据库， │
|
||||
│ 游标存本地 JSON 文件。挂了重启即可，不影响网站。 │
|
||||
└─────────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
@@ -315,15 +318,23 @@ chat_history 独立表（问答上下文摘要留存）
|
||||
**GET /api/oracle/avatar** — 人物头像：`?label=&w=` → jpeg（从该人物候选事件 `person_appearances` bbox 裁剪）
|
||||
**GET /health** — 服务状态（含已处理视频数）
|
||||
|
||||
### 5.2 FAM-Core（NAS :8000）
|
||||
**登录相关端点**（2026-09-12 从 NAS FAM-Core 迁入，见 `fam-edge/src/fam_edge/auth.py`）
|
||||
|
||||
| 端点 | 方法 | 说明 |
|
||||
|------|------|------|
|
||||
| `/login` | GET | 登录入口：生成 PKCE 参数后 302 跳 auth-hub 公网 `/authorize` |
|
||||
| `/api/auth/callback` | GET | auth-hub 回跳：用 `AUTH_HUB_INTERNAL_BASE`（本机 :5300）换 token + 拉 JWKS 验签，成功后种 HttpOnly cookie `fam_session`（HS256 无状态签名，2 小时）；**任何一步失败都渲染错误页，绝不 302 回 `/login`**——旧实现失败即跳 `/login`，而 auth-hub 有会话时会立刻再签发 code，两边对跳成死循环 |
|
||||
| `/api/logout` | POST | 退出登录（清 cookie，不影响 auth-hub 上的 SSO 会话） |
|
||||
| `/api/auth/check` | GET | 登录态检查：`{"authed": true\|false, "username": "..."}` |
|
||||
| `/api/auth/verify` | ANY | 给甲骨文 Caddy 的 `forward_auth` 用：已登录 204，未登录 401 JSON |
|
||||
|
||||
### 5.2 FAM-Core（甲骨文 127.0.0.1:5401）
|
||||
|
||||
> 2026-09-13 起 FAM-Core 与 FAM-Edge 同机，直读它的 SQLite（不再有 MariaDB 镜像层），且**只监听 127.0.0.1**——唯一客户端是同机 Caddy。登录端点在 FAM-Edge（见 §5.1），鉴权由 Caddy 的 `forward_auth` 前置完成，所以本服务自身不做任何校验。
|
||||
|
||||
| 端点 | 方法 | 说明 |
|
||||
|------|------|------|
|
||||
| `/health` | GET | 服务健康（**免登录**） |
|
||||
| `/login` | GET | 登录入口：302 跳转 auth-hub `/authorize`（Authorization Code + PKCE，**免登录**） |
|
||||
| `/api/auth/callback` | GET | auth-hub 登录回跳：拿 `code` 换 token、验 `id_token` 签名后种 HttpOnly cookie `fam_session`（2 小时，**免登录**） |
|
||||
| `/api/logout` | POST | 退出登录（清本地 cookie，不影响 auth-hub 上的登录态） |
|
||||
| `/api/auth/check` | GET | 登录态检查：`{"authed": true\|false}`（**免登录**） |
|
||||
| `/api/status` | GET | Oracle-Sync + MotionNotifier 状态（sync running/cursor；motion poll_enabled/pushed_total/heartbeat） |
|
||||
| `/api/chat/ask` | POST | 用户问答：`{"question","queried_person","queried_date"}` → `{"answer","context_summary","chat_id"}`（上下文来自 sync_events） |
|
||||
| `/api/chat/history` | GET | 对话历史（`?date=` 或 `?person=&limit=`） |
|
||||
@@ -540,17 +551,27 @@ task_id=289（30s 测试片段）全链路打通：推送 5.7MB → Edge 分析
|
||||
|
||||
| 组件 | 节点 | 路径 | 启动 |
|
||||
|------|------|------|------|
|
||||
| FAM-Core | NAS | `/volume1/web/sentinel-home-ai/fam-core/` | `bash start_core.sh`（gunicorn -w 1 :8000，source .env 注入 DSM_*/ORACLE_SYNC_TOKEN/AUTH_HUB_*） |
|
||||
| FAM-Core | Oracle | `/opt/fam-core/` | **systemd `fam-core.service`**（gunicorn -w 2 --threads 4，绑 127.0.0.1:5401）；部署代码后 `sudo systemctl restart fam-core` |
|
||||
| fam-notifier | NAS | `/volume1/web/sentinel-home-ai/fam-notifier/` | `setsid bash scripts/start_notifier.sh &`（DSM 无 systemd，挂了不自启）；NAS 上唯一运行的本项目进程 |
|
||||
| FAM-UI | 云服务器（甲骨文，129.146.26.249） | `/var/www/fam-ui/` | Vue3 构建产物，由 Caddy :80 静态托管（无需独立进程）；本地改代码后 `npm run build`，`dist/` rsync/tar 到云服务器 |
|
||||
| FAM-Edge | Oracle | `/opt/fam-edge/` | **systemd `fam-edge.service` 守护**（Restart=always）；部署代码后 `sudo systemctl restart fam-edge`（勿手动 setsid，会端口冲突） |
|
||||
| **AI-Gateway** | Oracle | `/opt/ai-gateway/` | **systemd `ai-gateway.service` 守护**（Restart=always），独立 venv（Python 3.8）；gunicorn 绑定 `0.0.0.0:5100`（对外直接开放，非仅本机）；部署代码后 `sudo systemctl restart ai-gateway` |
|
||||
| Ollama | Oracle | systemd 托管 | 环境变量 `OLLAMA_KEEP_ALIVE=-1`；**被 AI-Gateway 调用，不再被 FAM-Edge 调用** |
|
||||
|
||||
> **外网访问（frp 内网穿透）**：NAS 跑 `frpc`（`/etc/frp/frpc.toml`，S99frpc.sh 守护），映射到云服务器 `129.146.26.249`（frps :7000）：
|
||||
> - `3000` → NAS Gitea、`8500` → NAS WordPress(8088)、**`8000` → NAS FAM-Core（本系统）**
|
||||
> - 前端入口 `http://129.146.26.249/`（Caddy 托管 FAM-UI，/api 经 frp 隧道反代回 NAS FAM-Core :8000），**需登录**（见 §5.2 `/login`）
|
||||
> - `3000` → NAS Gitea、`8500` → NAS WordPress(8088)
|
||||
> - **`8000` → NAS FAM-Core 这条已于 2026-09-13 迁云后废弃**（本系统不再有任何"回源 NAS"的流量，可从 frpc.toml 删除）
|
||||
> - 前端入口 `https://smart-camera.zichuan.xyz/`：静态页、接口、登录全部由甲骨文提供，**NAS 离线也能正常访问**（只是不再有新的运动事件推进来），**需登录**（见 §5.1 `/login`）
|
||||
>
|
||||
> **登录校验（2026-08-22 新增，2026-08-23 改为 fail-closed，2026-08-31 接入 auth-hub 统一登录）**：FAM-Core 不再自己保存/校验密码，全权委托给独立部署的 [auth-hub](http://129.146.26.249:5300)（OAuth2 Authorization Code + PKCE / OIDC，同一 Oracle 主机 :5300，与本系统完全独立部署）——`/login` 302 跳到 auth-hub `/authorize`，登录后带 `code` 跳回本服务 `/api/auth/callback`，服务端换 token、验完 `id_token` 签名（JWKS）后种本地会话 cookie。全站拦截仍在 `auth.py`：页面未登录 302 `/login`，`/api/*` 未登录 401；接入参数 `AUTH_HUB_ISSUER`/`AUTH_HUB_CLIENT_ID`/`AUTH_HUB_CLIENT_SECRET`/`AUTH_HUB_REDIRECT_URI`（NAS `.env` 配置，**无硬编码默认值**，任一没配置直接拒绝所有登录，fail closed）；`AUTH_HUB_REDIRECT_URI` 必须与 auth-hub 用 `manage_clients create` 登记的 redirect_uri 逐字符一致（auth-hub 只做精确匹配，不做前缀/子串匹配）。白名单免登录：`/login` `/api/auth/callback` `/api/logout` `/api/auth/check` `/health` `/assets/*`（`/api/ss/webhook` 已随端点一起移除，2026-08-25）。本地会话仍是进程内 token + HttpOnly cookie（**2 小时**），过期或重启 fam-core 需重新走一遍 auth-hub 登录；auth-hub 侧注册是任何人都能自助注册（管理员审批后才能登录），**未对 fam-core 侧再加用户名白名单**——任何在 auth-hub 上审批通过的账号登录后都能访问本系统，这是有意识的取舍（自用场景，见项目记忆）。
|
||||
> **登录校验（2026-08-22 新增；2026-08-31 接入 auth-hub 统一登录；2026-09-12 整体迁到甲骨文 FAM-Edge）**：登录流程和会话校验现在都在甲骨文 FAM-Edge 的 `auth.py`（§5.1），NAS FAM-Core 自身不再做任何鉴权。迁移的直接原因：NAS 或 frp 隧道一挂，`smart-camera.zichuan.xyz/login` 直接 502，**连登录页都打不开**——登录是入口，不该依赖家里那台机器。
|
||||
>
|
||||
> - **链路**：`/login`（Caddy → 本机 fam-edge :5000）→ 302 到 auth-hub 公网 `/authorize`（Authorization Code + PKCE）→ 用户在 [auth-hub](https://auth.zichuan.xyz) 登录 → 回跳 `/api/auth/callback`（仍是 fam-edge）→ 走**本机** `http://127.0.0.1:5300` 换 token、拉 JWKS 验 `id_token` 签名 → 种 cookie `fam_session`
|
||||
> - **为什么服务端调用走本机**：旧实现是 NAS 跨公网访问 `https://auth.zichuan.xyz`，那条链路上 `jwt.PyJWKClient` 用 urllib + 系统 CA（不像 requests 自带 certifi），群晖上容易 `CERTIFICATE_VERIFY_FAILED`；两台机器的时钟偏差还会让 `iat` 看起来来自未来。同机直连把这两个坑一起消掉。但 `iss` 校验和浏览器跳转仍用公网 `AUTH_HUB_ISSUER`
|
||||
> - **拦截**：甲骨文 Caddy 对 `/api/*` 做 `forward_auth` → fam-edge `/api/auth/verify`，200/204 才反代回 NAS；`/login`、`/api/auth/*`、`/api/logout` 直接由 fam-edge 处理（Caddy 按路径具体程度排序，这几条稳定排在 `/api/*` 前面）
|
||||
> - **NAS :8000 必须靠防火墙兜底**：frps 把它暴露在公网（历史入口 `http://129.146.26.249:8000`），绕过 Caddy 直连就没有任何鉴权。甲骨文 iptables 只允许本机访问 :8000，见 `docs/DEPLOY.md` §2.3
|
||||
> - **参数**（`/opt/fam-edge/.env`，**无硬编码默认值**，缺任一项拒绝所有登录 fail closed）：`AUTH_HUB_ISSUER`（公网，浏览器跳转 + `iss` 校验）/ `AUTH_HUB_INTERNAL_BASE`（本机，换 token + JWKS）/ `AUTH_HUB_CLIENT_ID` / `AUTH_HUB_CLIENT_SECRET` / `AUTH_HUB_REDIRECT_URI`（必须跟 auth-hub 登记的逐字符一致，只做精确匹配）/ `FAM_SESSION_SECRET`（会话签名密钥，≥32 字节）
|
||||
> - **会话是无状态签名 cookie**（HS256，2 小时），fam-edge 重启不掉线（旧实现是进程内 token 表，fam-core 一重启全员下线）；要强制全员下线就换掉 `FAM_SESSION_SECRET` 再重启
|
||||
> - auth-hub 侧任何审批通过的账号登录后都能访问本系统，**未再加用户名白名单**——自用场景的有意取舍（见项目记忆）
|
||||
|
||||
> Oracle 部署方式：本地 git 提交 push Gitea → tar 管道到 `/opt/fam-edge`（`--strip-components=1` 解临时目录再 cp，避免动 data/venv/gdrive_videos）。**AI-Gateway 是独立 git 仓库**（http://192.168.50.64:3000/ericwyuan/ai-gateway，见 §10.4），同样 tar 管道部署到 `/opt/ai-gateway`，互不影响。
|
||||
|
||||
@@ -703,6 +724,7 @@ print('NVIDIA NIM 连接成功:', response.choices[0].message.content)
|
||||
- NAS scp 子系统被禁用，同样用 stdin 管道传文件
|
||||
- **Oracle fam-edge 由 systemd `fam-edge.service` 守护（Restart=always）**：部署代码后必须 `sudo systemctl restart fam-edge`；手动 `setsid` 启动会和守护打架（端口 `Connection in use`）
|
||||
- NAS 远端 kill gunicorn 用 `ps aux | grep "[f]am-core/venv/bin/gunicorn"` 字符类技巧（pkill/pgrep 会匹配 SSH 自身命令行导致断连）
|
||||
- **sqlite3 连接不能跨线程共享**（2026-09-13 修）：`OracleDB` 原来在 `__init__` 里建一条 `check_same_thread=False` 的连接给全进程用，VideoQueue / PersonService / DiskGuard 三个后台线程加 gunicorn 请求线程并发读写它，事务状态互相踩踏，线上累计刷出 `database is locked` 71604 次、`cannot start a transaction within a transaction` 378 次、`no more rows available` 92 次；DiskGuard 的清理被打断 2837 次，磁盘守护形同虚设（剩余空间在 2.7G 和 16G 之间来回荡）。改成 `threading.local()` 每线程一条连接后，WAL 下读不互斥、写由 SQLite 自己排队。**新增后台线程时不要再去共用某一条连接对象**
|
||||
- fam-core 启动模块路径是 `src.fam_core.app:app`（不是 `fam_core.app:app`）；`start_core.sh` 会 source 仓库根 `.env` 注入 `DSM_*/ORACLE_SYNC_TOKEN`
|
||||
- Edge 单 worker 处理任务期间 `/health` 可能不响应，属正常
|
||||
- **运动数据清理**：切换架构/重新提取时清 Oracle `videos/events/people` + `motion_clips/`（保留 `ss_motion_events` 与素材）与 NAS `sync_*` 镜像，重启两端自动重新分割分析
|
||||
|
||||
165
docs/DEPLOY.md
165
docs/DEPLOY.md
@@ -1,54 +1,58 @@
|
||||
# 部署指南（v3 运动事件驱动架构，2026-08-22 同步）
|
||||
# 部署指南（2026-09-13 全量迁云后同步）
|
||||
|
||||
## 1. NAS 端部署 (FAM-Core + MariaDB；FAM-UI 见 §1.3，部署在云服务器)
|
||||
> **一句话架构**：除了录像本身，全部跑在甲骨文。NAS 只剩 Surveillance Station
|
||||
> 和一个往甲骨文推运动事件的进程（fam-notifier）——NAS 离线不影响网站访问。
|
||||
|
||||
### 1.1 MariaDB
|
||||
## 1. NAS 端部署（只有 fam-notifier）
|
||||
|
||||
2026-09-13 迁云后 NAS 上**不再有** MariaDB、FAM-Core、FAM-UI：镜像层整个删除
|
||||
（数据权威源本来就在甲骨文），接口和前端也都搬走了。旧部署的清理见 §1.2。
|
||||
|
||||
### 1.1 fam-notifier（轮询 Surveillance Station → 推甲骨文）
|
||||
```bash
|
||||
# 通过 Synology 套件中心安装 MariaDB（10.11），unix_socket=/run/mysqld/mysqld10.sock
|
||||
|
||||
# 执行 DDL
|
||||
python3 scripts/init_db.py --host 127.0.0.1 --port 3306 --user root --password <密码>
|
||||
```
|
||||
|
||||
### 1.2 FAM-Core
|
||||
```bash
|
||||
cd /volume1/web/sentinel-home-ai/fam-core
|
||||
cd /volume1/web/sentinel-home-ai/fam-notifier
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
pip install -r requirements.txt # 只有 requests + PyYAML
|
||||
|
||||
# 配置
|
||||
cp config/config.yaml.example config/config.yaml
|
||||
# 编辑 config.yaml：数据库密码、oracle_sync.base_url/token、motion_notifier（DSM 凭据走 .env）
|
||||
# 编辑 config.yaml：camera_ids、oracle_base_url；DSM 凭据与 token 走仓库根 .env
|
||||
# 仓库根 .env 需提供 DSM_ACCOUNT / DSM_PASSWORD / ORACLE_SYNC_TOKEN
|
||||
|
||||
# 仓库根 .env 提供 DSM_ACCOUNT/DSM_PASSWORD/ORACLE_SYNC_TOKEN/AUTH_HUB_*（start_core.sh 会 source）
|
||||
# 启动（DSM 没有 systemd，用 setsid 脱离 SSH 会话；挂了不会自启，需要人工重启）
|
||||
cd /volume1/web/sentinel-home-ai/fam-notifier
|
||||
setsid bash scripts/start_notifier.sh >> logs/start.log 2>&1 &
|
||||
|
||||
# 登录改接 auth-hub 统一登录（2026-08-31），先在 auth-hub 侧注册本站点 client：
|
||||
# ssh 到 auth-hub 所在 Oracle 主机，cd /opt/auth-hub
|
||||
# python -m auth_hub.manage_clients create "FAM-Core" "http://129.146.26.249/api/auth/callback"
|
||||
# 输出的 client_id/client_secret 明文只显示这一次，立即抄进 .env
|
||||
# 仓库根 .env 补齐四项（缺任一项 fam-core 直接拒绝所有登录，fail closed）：
|
||||
# AUTH_HUB_ISSUER=http://129.146.26.249:5300
|
||||
# AUTH_HUB_CLIENT_ID=<上一步输出>
|
||||
# AUTH_HUB_CLIENT_SECRET=<上一步输出>
|
||||
# AUTH_HUB_REDIRECT_URI=http://129.146.26.249/api/auth/callback # 必须跟注册时的 redirect_uri 逐字符一致
|
||||
|
||||
# 启动（含 Oracle-Sync + MotionNotifier 轮询）
|
||||
cd /volume1/web/sentinel-home-ai && bash start_core.sh
|
||||
# 确认在跑（进程没有监听端口，只能看进程和日志）
|
||||
ps aux | grep "[f]am_notifier"
|
||||
tail -f logs/fam-notifier.log # 应每 60s 轮询、每 5min 心跳
|
||||
```
|
||||
|
||||
### 1.3 FAM-UI（Vue3 SPA，托管于云服务器 Caddy :80）
|
||||
游标存在 `fam-notifier/data/cursor.json`（迁云前存在 MariaDB）。删掉也不致命：
|
||||
窗口回看会把最近的事件补推一遍，甲骨文按 event_id 幂等落库。
|
||||
|
||||
### 1.2 清理旧部署（迁云一次性动作）
|
||||
```bash
|
||||
# 1) 停掉 NAS 上的 fam-core（它已经不该再跑了；跑着也没用，前端不再连它）
|
||||
ps aux | grep "[f]am-core/venv/bin/gunicorn" # 字符类写法：pkill 会杀掉 SSH 自己
|
||||
kill <上面的 pid>
|
||||
|
||||
# 2) frpc 里的 8000 映射可以删了（/etc/frp/frpc.toml 的 fam-core 段），
|
||||
# 甲骨文侧对应的 iptables DROP 规则也就没有存在意义了
|
||||
# 3) MariaDB 的 sentinel_home_ai 库确认 chat_history 已迁移（§2.4）后再考虑删
|
||||
```
|
||||
|
||||
### 1.3 FAM-UI（Vue3 SPA，构建后托管在甲骨文 Caddy）
|
||||
```bash
|
||||
cd /Users/ericwyuan/Desktop/Work/sentinel-home-ai/fam-ui # 本地开发机
|
||||
npm install
|
||||
npm run build # 产物 fam-ui/dist/
|
||||
# 部署 dist 到云服务器（Caddy 静态目录，/api 经 frp 隧道反代回 NAS）：
|
||||
tar czf - fam-ui/dist | ssh -i ~/.ssh/oracle_new ubuntu@129.146.26.249 \
|
||||
'sudo tar xzf - --strip-components=2 -C /var/www/fam-ui && sudo chown -R ubuntu:ubuntu /var/www/fam-ui'
|
||||
# 浏览器访问 http://129.146.26.249/（Caddy 托管，/api/* 反代 NAS fam-core）
|
||||
tar czf - -C fam-ui dist | ssh -i ~/.ssh/oracle_new ubuntu@129.146.26.249 \
|
||||
'sudo tar xzf - --strip-components=1 -C /var/www/fam-ui && sudo chown -R ubuntu:ubuntu /var/www/fam-ui'
|
||||
# 浏览器访问 https://smart-camera.zichuan.xyz/
|
||||
```
|
||||
|
||||
## 2. Oracle 端部署 (FAM-Edge + Ollama + FFmpeg)
|
||||
## 2. Oracle 端部署 (FAM-Edge + FAM-Core + Ollama + FFmpeg)
|
||||
|
||||
### 2.1 系统依赖
|
||||
```bash
|
||||
@@ -74,6 +78,89 @@ sudo systemctl enable fam-edge
|
||||
sudo systemctl restart fam-edge # 部署代码后必须用 systemctl 重启，勿手动 setsid
|
||||
```
|
||||
|
||||
### 2.3 统一登录（2026-09-12 从 NAS 迁入，fam-edge + Caddy + 防火墙三件套）
|
||||
|
||||
登录入口不再依赖 NAS。三处缺一不可：
|
||||
|
||||
```bash
|
||||
# (1) /opt/fam-edge/.env 追加六项（缺任一项 fam-edge 拒绝所有登录，fail closed）
|
||||
# client_secret 用 rotate-secret 现拿，明文只显示一次：
|
||||
# cd /opt/auth-hub && venv/bin/python -m auth_hub.manage_clients rotate-secret <client_id>
|
||||
# 会话密钥自己生成：openssl rand -hex 32
|
||||
export AUTH_HUB_ISSUER=https://auth.zichuan.xyz # 公网：浏览器跳转 + id_token 的 iss 校验
|
||||
export AUTH_HUB_INTERNAL_BASE=http://127.0.0.1:5300 # 本机：换 token + 拉 JWKS，不走公网 TLS
|
||||
export AUTH_HUB_CLIENT_ID=<auth-hub 注册的 client_id>
|
||||
export AUTH_HUB_CLIENT_SECRET=<rotate-secret 输出>
|
||||
export AUTH_HUB_REDIRECT_URI=https://smart-camera.zichuan.xyz/api/auth/callback # 与 auth-hub 登记的逐字符一致
|
||||
export FAM_SESSION_SECRET=<openssl rand -hex 32> # 会话 cookie 签名密钥，换掉即全员下线
|
||||
|
||||
# (2) venv 补依赖（Python 3.8，pip 会自动选到兼容版本）后重启
|
||||
/opt/fam-edge/venv/bin/pip install "PyJWT>=2.8.0" "cryptography>=42.0.0"
|
||||
sudo systemctl restart fam-edge
|
||||
|
||||
# (3) Caddy：登录端点走本机 fam-edge，其余 /api/* 先 forward_auth 再回源 NAS
|
||||
# 改完先 caddy validate 再 reload，见下方 Caddyfile 片段
|
||||
sudo caddy validate --adapter caddyfile --config /etc/caddy/Caddyfile && sudo systemctl reload caddy
|
||||
```
|
||||
|
||||
```caddyfile
|
||||
smart-camera.zichuan.xyz {
|
||||
handle /login { reverse_proxy 127.0.0.1:5000 }
|
||||
handle /api/auth/* { reverse_proxy 127.0.0.1:5000 }
|
||||
handle /api/logout { reverse_proxy 127.0.0.1:5000 }
|
||||
handle /api/* {
|
||||
forward_auth 127.0.0.1:5000 { uri /api/auth/verify } # 删了等于数据接口全裸
|
||||
reverse_proxy 127.0.0.1:8000
|
||||
}
|
||||
handle { root * /var/www/fam-ui; encode gzip; try_files {path} /index.html; file_server }
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
# (4) 防火墙：frps 把 NAS fam-core 的 :8000 暴露在公网，而 fam-core 自身已无鉴权，
|
||||
# 必须只放行本机（Caddy）访问，否则绕过 Caddy 就能读到全部数据
|
||||
sudo iptables -I INPUT 1 -p tcp --dport 8000 ! -i lo -j DROP
|
||||
sudo netfilter-persistent save # 否则重启后规则丢失
|
||||
```
|
||||
|
||||
### 2.4 FAM-Core（UI/对话/成员接口，2026-09-13 从 NAS 迁入）
|
||||
|
||||
```bash
|
||||
# 代码（本地开发机 → 甲骨文）
|
||||
tar czf - --exclude=venv --exclude=__pycache__ --exclude=logs --exclude='config/config.yaml' fam-core | \
|
||||
ssh -i ~/.ssh/oracle_new ubuntu@129.146.26.249 'tar xzf - --strip-components=1 -C /opt/fam-core'
|
||||
|
||||
# 甲骨文上：venv + 配置
|
||||
python3 -m venv /opt/fam-core/venv
|
||||
/opt/fam-core/venv/bin/pip install -r /opt/fam-core/requirements.txt
|
||||
cp /opt/fam-core/config/config.yaml.example /opt/fam-core/config/config.yaml
|
||||
# .env 只需要 ORACLE_SYNC_TOKEN（与 fam-edge 的一致），config_loader 会自动加载
|
||||
grep "^export ORACLE_SYNC_TOKEN=" /opt/fam-edge/.env > /opt/fam-core/.env && chmod 600 /opt/fam-core/.env
|
||||
|
||||
sudo systemctl restart fam-core # 单元见 /etc/systemd/system/fam-core.service
|
||||
curl -s http://127.0.0.1:5401/api/status # {"db":{"ok":true},...}
|
||||
```
|
||||
|
||||
**端口 5401 不是笔误**：5400 被这台机器上的 chat-relay 占了。fam-core 只绑
|
||||
127.0.0.1，外部进不来，唯一客户端是同机 Caddy（先 forward_auth 再反代）。
|
||||
|
||||
**chat_history 迁移（一次性）**：这是 NAS MariaDB 里唯一不是镜像的表。
|
||||
在 NAS 上导出，再从本地导入甲骨文的 SQLite：
|
||||
```bash
|
||||
# NAS 上导出（用 fam-core 旧 venv 里的 PyMySQL）
|
||||
ssh -p 2222 ericwyuan@192.168.50.64 '/volume1/web/sentinel-home-ai/fam-core/venv/bin/python -c "
|
||||
import pymysql, json, yaml
|
||||
cfg = yaml.safe_load(open(\"/volume1/web/sentinel-home-ai/fam-core/config/config.yaml\"))[\"database\"]
|
||||
c = pymysql.connect(host=\"127.0.0.1\", user=cfg[\"user\"], password=cfg[\"password\"], database=cfg[\"database\"], cursorclass=pymysql.cursors.DictCursor)
|
||||
cur = c.cursor(); cur.execute(\"SELECT * FROM chat_history ORDER BY chat_id\")
|
||||
print(json.dumps(cur.fetchall(), ensure_ascii=False, default=str))
|
||||
"' > /tmp/chat_history.json
|
||||
|
||||
# 本地 → 甲骨文导入（幂等：按 chat_id 跳过已存在的）
|
||||
cat /tmp/chat_history.json | ssh -i ~/.ssh/oracle_new ubuntu@129.146.26.249 \
|
||||
'/opt/fam-core/venv/bin/python /opt/fam-core/scripts/import_chat_history.py'
|
||||
```
|
||||
|
||||
## 3. 代码同步（tar 管道，scp 在 NAS 被禁用）
|
||||
|
||||
```bash
|
||||
@@ -91,10 +178,12 @@ tar czf - --exclude=venv --exclude=__pycache__ --exclude=data --exclude=gdrive_v
|
||||
|
||||
| 项目 | 命令 | 预期 |
|
||||
|------|------|------|
|
||||
| NAS MariaDB | `mysql -u root -p -e "SHOW DATABASES"` | 包含 sentinel_home_ai |
|
||||
| NAS FAM-Core | `curl http://localhost:8000/health` | `{"status":"ok"}` |
|
||||
| NAS 运动监测 | `curl http://localhost:8000/api/ss/status` | `poll_enabled: true, running: true` |
|
||||
| 云服务器 FAM-UI | 浏览器访问 `http://129.146.26.249/`（Caddy 托管） | 未登录被 302 到 auth-hub 登录页；登录（审批过的 auth-hub 账号）后跳回展示 Vue3 SPA（时间轴/人物管理/统计） |
|
||||
| 云服务器 FAM-UI | 浏览器访问 `https://smart-camera.zichuan.xyz/` | 未登录时 `/api/*` 返回 401，前端自动跳 `/login` → auth-hub 登录页；登录后跳回展示 Vue3 SPA |
|
||||
| 登录不依赖 NAS | `curl -sI https://smart-camera.zichuan.xyz/login`（此时哪怕 fam-core 是停的） | 302 到 `auth.zichuan.xyz/authorize?...`，**不再是 502** |
|
||||
| 鉴权闸门 | `curl -s -o /dev/null -w '%{http_code}' https://smart-camera.zichuan.xyz/api/ui/videos` | 401（Caddy forward_auth 拦下，没到 NAS） |
|
||||
| NAS fam-notifier | NAS 上 `ps aux \| grep "[f]am_notifier"` | 有进程；`fam-notifier/logs/fam-notifier.log` 每 60s 一条轮询、每 5min 一条心跳 |
|
||||
| 甲骨文 FAM-Core | `curl -s http://127.0.0.1:5401/api/status` | `{"db":{"ok":true,...}}`；`/api/ui/stats` 返回真实计数 |
|
||||
| NAS 离线也能用 | 关掉 NAS 后访问 `https://smart-camera.zichuan.xyz/timeline` | 页面与数据照常（只是不再有新运动事件）；**这是本次迁云的验收标准** |
|
||||
| Oracle FAM-Edge | `curl http://localhost:5000/health` | `{"status":"ok","queue_alive":true}` |
|
||||
| Oracle 运动事件 | `sqlite3 /opt/fam-edge/data/oracle.db "SELECT COUNT(*) FROM ss_motion_events"` | >0（NAS 推送） |
|
||||
| Oracle 运动片段 | `ls /opt/fam-edge/motion_clips/` | 存在 motion_*.mp4（素材分割产物） |
|
||||
|
||||
@@ -1,32 +1,28 @@
|
||||
# FAM-Core 配置文件 (NAS 端) - 新架构 v2
|
||||
# FAM-Core 配置文件（2026-09-13 起跑在甲骨文，与 fam-edge 同机）
|
||||
# 复制此文件为 config.yaml 并修改实际值
|
||||
#
|
||||
# NAS 仅作管理后台，不再处理视频。唯一后台线程 Oracle-Sync 每 30 分钟
|
||||
# 从甲骨文 FAM-Edge 拉取增量镜像到本地 MariaDB（sync_videos/events/people）。
|
||||
# 所有视频分析在 Oracle 完成。
|
||||
# 本服务是纯查询/转发层：直读 fam-edge 的 SQLite，写操作转给 fam-edge。
|
||||
# 没有后台线程，没有自己的数据库，重启不影响任何数据。
|
||||
|
||||
server:
|
||||
host: "0.0.0.0"
|
||||
port: 8000
|
||||
# 只监听回环：唯一的客户端是同机 Caddy（它做 forward_auth 鉴权后才反代进来）。
|
||||
# 迁云前这里是 0.0.0.0:8000 并经 frp 暴露到公网，绕过 Caddy 就能拿到全部数据。
|
||||
host: "127.0.0.1"
|
||||
port: 5401
|
||||
|
||||
database:
|
||||
host: "127.0.0.1"
|
||||
port: 3306
|
||||
user: "root"
|
||||
password: ""
|
||||
database: "sentinel_home_ai"
|
||||
unix_socket: "/run/mysqld/mysqld10.sock"
|
||||
# fam-edge 的库，本服务只读它写的表（唯一写的是 chat_history）。
|
||||
# fam-edge 侧已开 WAL，多进程读写安全。
|
||||
path: "/opt/fam-edge/data/oracle.db"
|
||||
|
||||
# 甲骨文同步（每 30 分钟拉增量镜像）
|
||||
oracle_sync:
|
||||
# FAM-Edge 对外同步接口地址（端口同其 server.port=5000）
|
||||
base_url: "http://<oracle-public-ip>:5000"
|
||||
# 与 Oracle 端 sync_api.token 一致
|
||||
token: ""
|
||||
interval_sec: 1800 # 拉取间隔（秒），默认 30 分钟
|
||||
timeout: 120 # 单次拉取超时（秒）
|
||||
# 同机 fam-edge：写操作（改名/删除）、帧图头像、服务状态都打给它
|
||||
edge:
|
||||
base_url: "http://127.0.0.1:5000"
|
||||
token: "${ORACLE_SYNC_TOKEN}" # 与 fam-edge 的 sync_api.token 一致
|
||||
timeout: 60
|
||||
|
||||
chat_handler:
|
||||
# 智能问答统一走 FAM-Edge 编排端点（Gemini → NVIDIA → 本地 Ollama 兜底）
|
||||
qa_url: "http://<oracle-public-ip>:5000/api/edge/chat/ask"
|
||||
# 智能问答走 fam-edge 编排端点（NVIDIA → Gemini → 本地 Ollama 兜底）
|
||||
qa_url: "http://127.0.0.1:5000/api/edge/chat/ask"
|
||||
qa_stream_url: "http://127.0.0.1:5000/api/edge/chat/ask/stream"
|
||||
timeout: 120
|
||||
|
||||
@@ -1,7 +1,4 @@
|
||||
flask>=3.0.0
|
||||
gunicorn>=21.2.0
|
||||
PyMySQL>=1.1.0
|
||||
PyYAML>=6.0
|
||||
requests>=2.31.0
|
||||
PyJWT>=2.8.0
|
||||
cryptography>=42.0.0
|
||||
|
||||
57
fam-core/scripts/import_chat_history.py
Normal file
57
fam-core/scripts/import_chat_history.py
Normal file
@@ -0,0 +1,57 @@
|
||||
"""把 NAS MariaDB 导出的 chat_history 导入甲骨文 SQLite（2026-09-13 迁云一次性脚本）。
|
||||
|
||||
chat_history 是 NAS 那套库里唯一"不是镜像"的表——其余 sync_* 都能从甲骨文重新
|
||||
读出来，只有问答历史是本地产生的，迁云时必须搬过来。
|
||||
|
||||
用法（JSON 从 stdin 进来，导出命令见 docs/DEPLOY.md §2.4）：
|
||||
cat chat_history.json | /opt/fam-core/venv/bin/python scripts/import_chat_history.py
|
||||
|
||||
幂等：按 chat_id 跳过已存在的行，重复执行不会产生重复记录。
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(
|
||||
os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'src'))
|
||||
|
||||
from fam_core import db_layer # noqa: E402
|
||||
|
||||
_COLS = ('chat_id', 'user_question', 'ai_answer', 'context_summary',
|
||||
'queried_date', 'queried_person', 'created_at')
|
||||
|
||||
|
||||
def main():
|
||||
try:
|
||||
rows = json.load(sys.stdin)
|
||||
except ValueError as e:
|
||||
print(f"stdin 不是合法 JSON: {e}", file=sys.stderr)
|
||||
return 1
|
||||
if not isinstance(rows, list):
|
||||
print("期望一个 JSON 数组", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
conn = db_layer.get_conn()
|
||||
try:
|
||||
db_layer._ensure_chat_schema(conn)
|
||||
inserted = skipped = 0
|
||||
for r in rows:
|
||||
cid = r.get('chat_id')
|
||||
if cid is not None and conn.execute(
|
||||
"SELECT 1 FROM chat_history WHERE chat_id=?", (cid,)).fetchone():
|
||||
skipped += 1
|
||||
continue
|
||||
conn.execute(
|
||||
"INSERT INTO chat_history ({}) VALUES ({})".format(
|
||||
','.join(_COLS), ','.join('?' * len(_COLS))),
|
||||
tuple(None if r.get(c) is None else str(r.get(c)) for c in _COLS))
|
||||
inserted += 1
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
print(f"导入 {inserted} 条，跳过 {skipped} 条（chat_id 已存在）")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -1,10 +1,20 @@
|
||||
"""
|
||||
FAM-Core 主应用 - Flask 单进程（新架构 v2）
|
||||
FAM-Core 主应用 - Flask（2026-09-13 从 NAS 迁到甲骨文）
|
||||
|
||||
承载: Oracle-Sync（每 30 分钟拉取增量镜像）+ Member-Manager + Chat-Handler
|
||||
承载: UI-API + Member-Manager + Chat-Handler + Img-Proxy —— 全是查询与转发，
|
||||
**没有任何后台线程**，进程随时可重启，不持有任何状态。
|
||||
|
||||
NAS 不再处理视频：无 Scheduler / Dispatcher / Poller / Event-Receiver / Video-Server。
|
||||
所有视频分析在 Oracle 完成，NAS 仅作管理后台拉取展示，CPU 占用大幅降低。
|
||||
迁云前它跑在 NAS 上，还扛着两个后台线程，现在都不在这里了：
|
||||
- Oracle-Sync（每 30 分钟把甲骨文数据拉一份镜像进 MariaDB）—— 整个删除。
|
||||
本服务现在与 fam-edge 同机，直接读它的 SQLite（见 db_layer.py 开头）。
|
||||
- MotionNotifier（轮询 Surveillance Station 推运动事件）—— 留在 NAS，
|
||||
拆成独立的 fam-notifier（摄像头插在 NAS 上，这部分搬不走）。
|
||||
|
||||
两件跟安全有关的事：
|
||||
- 本服务只监听 127.0.0.1（config.yaml 的 server.host），唯一的客户端是同机
|
||||
Caddy。不像迁云前那样经 frp 把 :8000 暴露到公网。
|
||||
- 登录校验也不在这里：Caddy 用 forward_auth 打 fam-edge 的 /api/auth/verify，
|
||||
通过了才反代进来。
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
@@ -16,87 +26,52 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from .config_loader import load_config
|
||||
from .logger import setup_logger
|
||||
from .oracle_sync import get_sync
|
||||
from .motion_notifier.motion_notifier import get_motion_notifier
|
||||
from .chat_handler.chat_handler import chat_bp
|
||||
from .member_manager.member_manager import member_bp
|
||||
from .img_proxy import img_bp
|
||||
from .motion_bp import motion_bp
|
||||
from .ui_api import ui_bp
|
||||
from .auth import auth_bp, init_auth
|
||||
from . import db_layer, edge_client
|
||||
|
||||
logger = setup_logger('fam-core.app')
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
# 注册蓝图：全部 /api/* 路由（前端已迁云服务器 Caddy:80，NAS 不再托管 SPA 静态文件）
|
||||
app.register_blueprint(chat_bp)
|
||||
app.register_blueprint(member_bp)
|
||||
app.register_blueprint(img_bp)
|
||||
app.register_blueprint(motion_bp)
|
||||
app.register_blueprint(ui_bp)
|
||||
app.register_blueprint(auth_bp)
|
||||
|
||||
# 登录校验全局拦截（页面未登录 302 /login；/api/* 未登录 401）
|
||||
init_auth(app)
|
||||
|
||||
# 健康检查
|
||||
@app.route('/health', methods=['GET'])
|
||||
def health():
|
||||
return jsonify({"status": "ok", "service": "fam-core"}), 200
|
||||
|
||||
|
||||
# 初始化后台同步线程（NAS 唯一常驻线程）
|
||||
_sync = None
|
||||
try:
|
||||
_sync = get_sync()
|
||||
_sync.start()
|
||||
logger.info("Oracle-Sync 已启动")
|
||||
# 启动后立刻拉一次，前端无需等待首个周期即有数据
|
||||
try:
|
||||
_sync.trigger_now()
|
||||
logger.info("启动首次同步完成")
|
||||
except Exception as e:
|
||||
logger.warning(f"启动首次同步失败（后续周期会重试）: {e}")
|
||||
except Exception as e:
|
||||
logger.error(f"Oracle-Sync 启动失败: {e}")
|
||||
|
||||
# 初始化运动监测通知服务（NAS 轮询 SS EventCenter 主路径）
|
||||
_notifier = None
|
||||
try:
|
||||
_notifier = get_motion_notifier()
|
||||
_notifier.start()
|
||||
logger.info("MotionNotifier 已初始化")
|
||||
except Exception as e:
|
||||
logger.error(f"MotionNotifier 初始化失败: {e}")
|
||||
|
||||
|
||||
@app.route('/api/status', methods=['GET'])
|
||||
def status():
|
||||
"""系统状态"""
|
||||
"""服务自检：库能不能读、fam-edge 在不在。
|
||||
|
||||
迁云前这里报告的是镜像同步状态（游标/周期/上次增量条数），镜像没了之后
|
||||
那些字段不再存在，前端侧边栏的同步面板也一并去掉了。
|
||||
"""
|
||||
db_ok, db_err = True, None
|
||||
try:
|
||||
conn = db_layer.get_conn()
|
||||
try:
|
||||
conn.execute("SELECT 1 FROM videos LIMIT 1").fetchone()
|
||||
finally:
|
||||
conn.close()
|
||||
except Exception as e:
|
||||
db_ok, db_err = False, str(e)
|
||||
return jsonify({
|
||||
"service": "fam-core",
|
||||
"sync": _sync.status() if _sync else {"running": False, "error": "未初始化"},
|
||||
"motion": _notifier.status() if _notifier else {"running": False, "error": "未初始化"},
|
||||
"db": {"ok": db_ok, "error": db_err},
|
||||
"edge_base_url": edge_client.base_url(),
|
||||
}), 200
|
||||
|
||||
|
||||
@app.route('/api/sync/trigger', methods=['POST'])
|
||||
def sync_trigger():
|
||||
"""手动立即触发一次甲骨文增量同步（服务状态页"立即同步"按钮）。
|
||||
|
||||
正常情况下后台线程每 30 分钟自动拉一次；这个接口给用户想立刻看到最新数据
|
||||
时用，跟后台线程共用同一把拉取锁（oracle_sync._pull_lock），不会并发重复拉。
|
||||
"""
|
||||
if not _sync:
|
||||
return jsonify({"error": "同步服务未初始化"}), 503
|
||||
ok = _sync.trigger_now()
|
||||
if not ok:
|
||||
return jsonify({"status": "failed", "error": _sync.status().get("last_error")}), 502
|
||||
return jsonify({"status": "ok", **_sync.status()}), 200
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
cfg = load_config()
|
||||
port = cfg.get('server', {}).get('port', 8000)
|
||||
app.run(host='0.0.0.0', port=port, debug=False)
|
||||
server = cfg.get('server', {})
|
||||
app.run(host=server.get('host', '127.0.0.1'),
|
||||
port=server.get('port', 5401), debug=False)
|
||||
|
||||
@@ -1,242 +0,0 @@
|
||||
"""
|
||||
Auth - FAM-Core 登录校验（2026-08-22 新增；2026-08-31 接入 auth-hub 统一登录改造）
|
||||
|
||||
背景：NAS :8000（fam-core + FAM-UI）通过 frp 暴露到外网后，需要先登录才能访问。
|
||||
|
||||
- 登录方式：不再自己校验账号密码，全权委托给独立部署的 auth-hub（OAuth2
|
||||
Authorization Code + PKCE / OIDC）。/login 直接 302 跳到 auth-hub 的
|
||||
/authorize，用户在 auth-hub 上登录后带 code 跳回本服务的
|
||||
/api/auth/callback，服务端拿 code 换 token、验完 id_token 签名后建立本地
|
||||
会话——本服务自身不再保存、也不再校验任何密码。
|
||||
- 接入参数从环境变量读（.env 注入，由 start_core.sh source）：
|
||||
`AUTH_HUB_ISSUER` / `AUTH_HUB_CLIENT_ID` / `AUTH_HUB_CLIENT_SECRET` /
|
||||
`AUTH_HUB_REDIRECT_URI`（必须跟在 auth-hub 用 manage_clients 注册时登记的
|
||||
redirect_uri 逐字符一致，auth-hub 只做精确匹配）。四个变量任一没配置时直接
|
||||
拒绝登录（fail closed）——跟之前账号密码时代的取舍一致：本服务监听
|
||||
0.0.0.0，没有"配置不全就退回某种默认放行"这种兜底。
|
||||
- 登录态：进程内 token 表 + HttpOnly cookie（fam_session），2 小时有效；
|
||||
重启进程后需重新登录（可接受，见 config/auth 说明）。这一段跟接入 SSO
|
||||
之前完全一样，只是"怎么发这个 cookie"变了，下游（`is_authed()`/白名单/
|
||||
拦截逻辑）不用改。
|
||||
- 拦截策略（app.before_request 全局生效）：
|
||||
* 页面路径未登录 -> 302 重定向 /login
|
||||
* /api/* 未登录 -> 401 JSON
|
||||
- 白名单免登录：
|
||||
* /login /api/auth/callback /api/logout /api/auth/check —— 登录流程本身
|
||||
* /health —— 内部健康检查
|
||||
* /api/ss/webhook —— Surveillance Station 推送无法携带登录态
|
||||
* /assets/*、/favicon.ico —— SPA 静态资源
|
||||
"""
|
||||
import os
|
||||
import secrets
|
||||
import time
|
||||
from base64 import urlsafe_b64encode
|
||||
from hashlib import sha256
|
||||
from urllib.parse import urlencode
|
||||
|
||||
import jwt
|
||||
import requests
|
||||
from flask import Blueprint, jsonify, make_response, redirect, request
|
||||
|
||||
from .logger import setup_logger
|
||||
|
||||
logger = setup_logger('fam-core.auth')
|
||||
|
||||
auth_bp = Blueprint('auth', __name__)
|
||||
|
||||
_SESSION_TTL = 2 * 3600 # cookie 有效期 2 小时
|
||||
_PENDING_TTL = 10 * 60 # /login -> /api/auth/callback 之间的等待上限
|
||||
_sessions = {} # token -> 过期时间戳（进程内；重启需重新登录）
|
||||
_pending = {} # state -> {verifier, expires}（进程内；PKCE 用）
|
||||
_warned_unconfigured = False # 只在第一次拒绝登录时打一条警告日志，别刷屏
|
||||
|
||||
_jwks_client = None # jwt.PyJWKClient 单例，内建 JWKS 缓存
|
||||
|
||||
|
||||
def _auth_hub_config():
|
||||
"""auth-hub 接入参数，四个变量都是必需的。"""
|
||||
return {
|
||||
'issuer': os.environ.get('AUTH_HUB_ISSUER', ''),
|
||||
'client_id': os.environ.get('AUTH_HUB_CLIENT_ID', ''),
|
||||
'client_secret': os.environ.get('AUTH_HUB_CLIENT_SECRET', ''),
|
||||
'redirect_uri': os.environ.get('AUTH_HUB_REDIRECT_URI', ''),
|
||||
}
|
||||
|
||||
|
||||
def _require_auth_hub_config():
|
||||
"""校验四个环境变量齐全，缺任何一个都拒绝（fail closed）。返回配置 dict 或 None。"""
|
||||
global _warned_unconfigured
|
||||
cfg = _auth_hub_config()
|
||||
if not all(cfg.values()):
|
||||
if not _warned_unconfigured:
|
||||
logger.error(
|
||||
"AUTH_HUB_ISSUER/CLIENT_ID/CLIENT_SECRET/REDIRECT_URI 未配置齐全，"
|
||||
"拒绝所有登录——请在 .env 里设置后重启 fam-core")
|
||||
_warned_unconfigured = True
|
||||
return None
|
||||
return cfg
|
||||
|
||||
|
||||
def _get_jwks_client(jwks_uri: str):
|
||||
global _jwks_client
|
||||
if _jwks_client is None or _jwks_client.uri != jwks_uri:
|
||||
_jwks_client = jwt.PyJWKClient(jwks_uri)
|
||||
return _jwks_client
|
||||
|
||||
|
||||
def is_authed() -> bool:
|
||||
tok = request.cookies.get('fam_session')
|
||||
if not tok:
|
||||
return False
|
||||
exp = _sessions.get(tok)
|
||||
if not exp:
|
||||
return False
|
||||
if time.time() > exp:
|
||||
_sessions.pop(tok, None)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _create_session() -> str:
|
||||
tok = secrets.token_hex(24)
|
||||
_sessions[tok] = time.time() + _SESSION_TTL
|
||||
return tok
|
||||
|
||||
|
||||
def _cleanup_pending():
|
||||
now = time.time()
|
||||
expired = [s for s, v in _pending.items() if v['expires'] < now]
|
||||
for s in expired:
|
||||
_pending.pop(s, None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 白名单（免登录）
|
||||
# ---------------------------------------------------------------------------
|
||||
_WHITELIST_EXACT = {
|
||||
'/login', '/api/auth/callback', '/api/logout', '/api/auth/check',
|
||||
'/health', '/favicon.ico',
|
||||
}
|
||||
_WHITELIST_PREFIX = ('/assets/',)
|
||||
|
||||
|
||||
def _is_whitelisted(path: str) -> bool:
|
||||
if path in _WHITELIST_EXACT:
|
||||
return True
|
||||
return any(path.startswith(p) for p in _WHITELIST_PREFIX)
|
||||
|
||||
|
||||
def init_auth(app):
|
||||
"""注册全局登录拦截。需在注册完所有蓝图后调用。"""
|
||||
|
||||
@app.before_request
|
||||
def _guard():
|
||||
path = request.path
|
||||
if _is_whitelisted(path):
|
||||
return None
|
||||
if is_authed():
|
||||
return None
|
||||
if path.startswith('/api/'):
|
||||
return jsonify({"error": "未登录", "code": 401}), 401
|
||||
return redirect('/login')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 端点
|
||||
# ---------------------------------------------------------------------------
|
||||
@auth_bp.route('/login', methods=['GET'])
|
||||
def login_page():
|
||||
"""不再自己渲染登录表单，直接跳到 auth-hub 走 Authorization Code + PKCE。"""
|
||||
if is_authed():
|
||||
return redirect('/')
|
||||
|
||||
cfg = _require_auth_hub_config()
|
||||
if not cfg:
|
||||
return jsonify({"error": "登录服务未配置"}), 503
|
||||
|
||||
_cleanup_pending()
|
||||
verifier = secrets.token_urlsafe(48)
|
||||
challenge = urlsafe_b64encode(sha256(verifier.encode('ascii')).digest()).rstrip(b'=').decode('ascii')
|
||||
state = secrets.token_hex(24)
|
||||
_pending[state] = {'verifier': verifier, 'expires': time.time() + _PENDING_TTL}
|
||||
|
||||
params = {
|
||||
'response_type': 'code',
|
||||
'client_id': cfg['client_id'],
|
||||
'redirect_uri': cfg['redirect_uri'],
|
||||
'scope': 'openid profile',
|
||||
'state': state,
|
||||
'code_challenge': challenge,
|
||||
'code_challenge_method': 'S256',
|
||||
}
|
||||
return redirect(f"{cfg['issuer']}/authorize?{urlencode(params)}")
|
||||
|
||||
|
||||
@auth_bp.route('/api/auth/callback', methods=['GET'])
|
||||
def auth_callback():
|
||||
cfg = _require_auth_hub_config()
|
||||
if not cfg:
|
||||
return jsonify({"error": "登录服务未配置"}), 503
|
||||
|
||||
if request.args.get('error'):
|
||||
logger.warning(f"auth-hub 登录被拒绝: {request.args.get('error')}")
|
||||
return redirect('/login')
|
||||
|
||||
state = request.args.get('state', '')
|
||||
code = request.args.get('code', '')
|
||||
pending = _pending.pop(state, None)
|
||||
if not pending or pending['expires'] < time.time() or not code:
|
||||
logger.warning("auth-hub 回调 state 缺失/过期/重放，拒绝")
|
||||
return redirect('/login')
|
||||
|
||||
try:
|
||||
resp = requests.post(
|
||||
f"{cfg['issuer']}/token",
|
||||
data={
|
||||
'grant_type': 'authorization_code',
|
||||
'code': code,
|
||||
'redirect_uri': cfg['redirect_uri'],
|
||||
'client_id': cfg['client_id'],
|
||||
'client_secret': cfg['client_secret'],
|
||||
'code_verifier': pending['verifier'],
|
||||
}, timeout=(10, 15))
|
||||
except requests.RequestException as e:
|
||||
logger.error(f"auth-hub /token 请求失败: {e}")
|
||||
return redirect('/login')
|
||||
|
||||
if resp.status_code != 200:
|
||||
logger.warning(f"auth-hub /token 拒绝: {resp.status_code} {resp.text[:200]}")
|
||||
return redirect('/login')
|
||||
|
||||
id_token = (resp.json() or {}).get('id_token', '')
|
||||
try:
|
||||
jwks_client = _get_jwks_client(f"{cfg['issuer']}/.well-known/jwks.json")
|
||||
signing_key = jwks_client.get_signing_key_from_jwt(id_token)
|
||||
claims = jwt.decode(id_token, signing_key.key, algorithms=['RS256'],
|
||||
audience=cfg['client_id'], issuer=cfg['issuer'])
|
||||
except jwt.PyJWTError as e:
|
||||
logger.warning(f"id_token 验签/校验失败: {e}")
|
||||
return redirect('/login')
|
||||
|
||||
username = claims.get('preferred_username', '')
|
||||
tok = _create_session()
|
||||
resp2 = make_response(redirect('/'))
|
||||
resp2.set_cookie('fam_session', tok, max_age=_SESSION_TTL,
|
||||
httponly=True, samesite='Lax', path='/')
|
||||
logger.info(f"登录成功 username={username}")
|
||||
return resp2
|
||||
|
||||
|
||||
@auth_bp.route('/api/logout', methods=['POST'])
|
||||
def logout():
|
||||
tok = request.cookies.get('fam_session')
|
||||
if tok:
|
||||
_sessions.pop(tok, None)
|
||||
resp = make_response(jsonify({"ok": True}))
|
||||
resp.delete_cookie('fam_session', path='/')
|
||||
return resp
|
||||
|
||||
|
||||
@auth_bp.route('/api/auth/check', methods=['GET'])
|
||||
def check():
|
||||
return jsonify({"authed": is_authed()})
|
||||
@@ -47,7 +47,7 @@ def _call_edge_qa(prompt: str) -> str:
|
||||
"""调用 FAM-Edge 问答编排端点（Gemini → NVIDIA → 本地 Ollama 兜底）"""
|
||||
cfg = load_config()
|
||||
qa_url = cfg.get('chat_handler', {}).get(
|
||||
'qa_url', 'http://129.146.26.249:5000/api/edge/chat/ask'
|
||||
'qa_url', 'http://127.0.0.1:5000/api/edge/chat/ask'
|
||||
)
|
||||
timeout = cfg.get('chat_handler', {}).get('timeout', 120)
|
||||
|
||||
@@ -79,7 +79,7 @@ def chat_ask():
|
||||
|
||||
logger.info(f"Chat: person={queried_person}, date={queried_date}, question={question}")
|
||||
|
||||
rows = db_layer.query_sync_events_for_person_date(queried_person, queried_date)
|
||||
rows = db_layer.query_events_for_person_date(queried_person, queried_date)
|
||||
|
||||
if len(rows) == 0:
|
||||
answer = f"今天没有观察到{queried_person}。"
|
||||
@@ -87,7 +87,7 @@ def chat_ask():
|
||||
else:
|
||||
context = _format_events(rows)
|
||||
context_summary = f"查询 sync_events {len(rows)} 条"
|
||||
members = db_layer.get_sync_known_members_context()
|
||||
members = db_layer.get_known_members_context()
|
||||
prompt = build_chat_prompt(
|
||||
context=context,
|
||||
members=members or queried_person,
|
||||
@@ -134,7 +134,7 @@ def chat_ask_stream():
|
||||
return jsonify({"error": "缺少必填字段: question, queried_person, queried_date"}), 400
|
||||
|
||||
logger.info(f"Chat(stream): person={queried_person}, date={queried_date}, question={question}")
|
||||
rows = db_layer.query_sync_events_for_person_date(queried_person, queried_date)
|
||||
rows = db_layer.query_events_for_person_date(queried_person, queried_date)
|
||||
|
||||
def sse(obj):
|
||||
return f"data: {json.dumps(obj, ensure_ascii=False)}\n\n"
|
||||
@@ -157,14 +157,14 @@ def chat_ask_stream():
|
||||
yield sse({"type": "context", "count": len(rows), "summary": context_summary,
|
||||
"preview": context[:800]})
|
||||
|
||||
members = db_layer.get_sync_known_members_context()
|
||||
members = db_layer.get_known_members_context()
|
||||
prompt = build_chat_prompt(
|
||||
context=context, members=members or queried_person,
|
||||
question=question, queried_person=queried_person)
|
||||
|
||||
cfg = load_config()
|
||||
stream_url = cfg.get('chat_handler', {}).get(
|
||||
'qa_stream_url', 'http://129.146.26.249:5000/api/edge/chat/ask/stream')
|
||||
'qa_stream_url', 'http://127.0.0.1:5000/api/edge/chat/ask/stream')
|
||||
timeout = cfg.get('chat_handler', {}).get('timeout', 120)
|
||||
|
||||
full_answer = []
|
||||
|
||||
@@ -1,11 +1,42 @@
|
||||
"""
|
||||
配置加载器 - 从 config.yaml 读取配置
|
||||
配置加载器 - 从 config.yaml 读取配置，支持 ${ENV_VAR} 解析
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import yaml
|
||||
|
||||
|
||||
def _load_env_file():
|
||||
"""加载部署目录下的 .env（支持 export KEY=VALUE 格式）。
|
||||
|
||||
迁云前是 start_core.sh 负责 source .env 再起 gunicorn；现在由 systemd 拉起，
|
||||
没有那一步，所以在这里兜底加载（跟 fam-edge 的做法一致）。
|
||||
已存在的环境变量不覆盖。
|
||||
"""
|
||||
path = os.environ.get('FAM_ENV_FILE') or os.path.join(
|
||||
os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))),
|
||||
'.env')
|
||||
if not os.path.isfile(path):
|
||||
return
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line or line.startswith('#'):
|
||||
continue
|
||||
if line.startswith('export '):
|
||||
line = line[7:].strip()
|
||||
if '=' not in line:
|
||||
continue
|
||||
key, _, value = line.partition('=')
|
||||
key = key.strip()
|
||||
value = value.strip().strip('"').strip("'")
|
||||
if key and key not in os.environ:
|
||||
os.environ[key] = value
|
||||
|
||||
|
||||
_load_env_file()
|
||||
|
||||
|
||||
def _resolve_env_vars(value):
|
||||
"""递归解析字符串中的 ${ENV_VAR} 引用"""
|
||||
if isinstance(value, str):
|
||||
|
||||
@@ -1,269 +1,176 @@
|
||||
"""
|
||||
数据库访问层 - MariaDB 连接管理与同步镜像 CRUD
|
||||
DB-Layer - 直读 fam-edge 的 SQLite 库（2026-09-13 迁云重写）
|
||||
|
||||
新架构 (2026-08-21 重构):
|
||||
NAS 不再处理视频，仅作为管理后台。
|
||||
Oracle (FAM-Edge) 处理整视频分析后存 SQLite，NAS 每 30 分钟拉增量，
|
||||
镜像到本地三张表:
|
||||
sync_videos : 视频会话（全局摘要 + 事件 JSON + 人物 JSON）
|
||||
sync_events : 视频拆出的时间点事件（描述 + 涉及人物 + 是否关注）
|
||||
sync_people : 规范人物表（label + canonical_name，Oracle 维护）
|
||||
sync_cursor : 同步游标（上次成功拉取到的 server_time）
|
||||
**这个文件以前是什么样**：fam-core 跑在 NAS 上，每 30 分钟把甲骨文的数据拉一份
|
||||
镜像进 MariaDB（`sync_videos`/`sync_events`/…），前端读镜像。725 行里有一半是
|
||||
镜像 upsert 的去重逻辑——8/29 和 9/3 两次线上事故（1062 主键冲突、游标卡死）
|
||||
都出在那一半。
|
||||
|
||||
本层只服务同步镜像 + 问答历史，旧 process_tasks/event_details/monitor_events/
|
||||
family_members 相关逻辑已全部移除（视频处理职责已迁移至 Oracle）。
|
||||
**现在**：fam-core 跟 fam-edge 同机，直接打开它的 SQLite 库读，镜像层整个不存在了。
|
||||
连带消失的还有一个隐蔽 bug：镜像表为了保外键稳定用的是 NAS 本地自增 id，而帧图
|
||||
接口要的是甲骨文的 id，两边在 9/3 那次 id 重排后就对不上了。现在只有一套 id。
|
||||
|
||||
要点：
|
||||
- 库文件是 fam-edge 的（`database.path`），**本模块只读它写的表**，唯一写的表是
|
||||
`chat_history`（问答历史，迁云时从 NAS MariaDB 搬过来的，fam-edge 不碰）。
|
||||
- fam-edge 那边开了 WAL，读不会阻塞它的写；这边同样设 busy_timeout 兜底。
|
||||
- 视频/人物的写操作（改名、删除）不在这里做，走 `edge_client` 打给 fam-edge——
|
||||
它除了改库还要合并人物、删磁盘素材。
|
||||
"""
|
||||
import json
|
||||
import pymysql
|
||||
from datetime import datetime
|
||||
from typing import Optional, List, Dict, Any
|
||||
import sqlite3
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
from .config_loader import load_config
|
||||
from .logger import setup_logger
|
||||
|
||||
logger = setup_logger('fam-core.db')
|
||||
|
||||
_config = None
|
||||
_db_path = None
|
||||
_chat_schema_ready = False
|
||||
|
||||
|
||||
def get_config():
|
||||
global _config
|
||||
if _config is None:
|
||||
_config = load_config()
|
||||
return _config
|
||||
def _now() -> str:
|
||||
return datetime.now(timezone(timedelta(hours=8))).strftime('%Y-%m-%d %H:%M:%S')
|
||||
|
||||
|
||||
def get_conn():
|
||||
"""获取数据库连接（单 worker gunicorn，无需连接池）"""
|
||||
cfg = get_config().get('database', {})
|
||||
kwargs = dict(
|
||||
host=cfg.get('host', '127.0.0.1'),
|
||||
port=cfg.get('port', 3306),
|
||||
user=cfg.get('user', 'root'),
|
||||
password=cfg.get('password', ''),
|
||||
database=cfg.get('database', 'sentinel_home_ai'),
|
||||
charset='utf8mb4',
|
||||
autocommit=False
|
||||
)
|
||||
unix_socket = cfg.get('unix_socket')
|
||||
if unix_socket:
|
||||
kwargs['unix_socket'] = unix_socket
|
||||
return pymysql.connect(**kwargs)
|
||||
def get_config() -> Dict:
|
||||
return load_config()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# 同步镜像：sync_videos
|
||||
# ============================================================
|
||||
|
||||
def upsert_sync_videos(rows: List[Dict]) -> int:
|
||||
"""批量 upsert Oracle 传来的 videos 增量。rows 为 Oracle 端 dict 列表。"""
|
||||
if not rows:
|
||||
return 0
|
||||
conn = get_conn()
|
||||
n = 0
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
for r in rows:
|
||||
cur.execute(
|
||||
"""INSERT INTO sync_videos
|
||||
(id, drive_file_id, filename, camera_name, duration_sec,
|
||||
event_start_time, status, summary_json, events_json,
|
||||
people_json, compute_provider, created_at, updated_at,
|
||||
processed_at, synced_at)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s, NOW())
|
||||
ON DUPLICATE KEY UPDATE
|
||||
drive_file_id=VALUES(drive_file_id),
|
||||
filename=VALUES(filename),
|
||||
camera_name=VALUES(camera_name),
|
||||
duration_sec=VALUES(duration_sec),
|
||||
event_start_time=VALUES(event_start_time),
|
||||
status=VALUES(status),
|
||||
summary_json=VALUES(summary_json),
|
||||
events_json=VALUES(events_json),
|
||||
people_json=VALUES(people_json),
|
||||
compute_provider=VALUES(compute_provider),
|
||||
created_at=VALUES(created_at),
|
||||
updated_at=VALUES(updated_at),
|
||||
processed_at=VALUES(processed_at),
|
||||
synced_at=NOW()""",
|
||||
(r.get('id'), r.get('drive_file_id'), r.get('filename'),
|
||||
r.get('camera_name'), r.get('duration_sec') or 0,
|
||||
r.get('event_start_time'), r.get('status'),
|
||||
r.get('summary_json'), r.get('events_json'), r.get('people_json'),
|
||||
r.get('compute_provider'), r.get('created_at'),
|
||||
r.get('updated_at'), r.get('processed_at'))
|
||||
)
|
||||
n += 1
|
||||
conn.commit()
|
||||
return n
|
||||
finally:
|
||||
conn.close()
|
||||
def _path() -> str:
|
||||
global _db_path
|
||||
if _db_path is None:
|
||||
_db_path = load_config().get('database', {}).get(
|
||||
'path', '/opt/fam-edge/data/oracle.db')
|
||||
return _db_path
|
||||
|
||||
|
||||
def get_sync_videos(limit=15, offset=0, date_filter=None) -> List[Dict]:
|
||||
"""获取视频会话列表（已完成优先），支持日期筛选与分页。
|
||||
def get_conn() -> sqlite3.Connection:
|
||||
"""每次调用开一个连接（请求级，跟改写前的 MySQL 用法一致）。
|
||||
|
||||
排序/日期维度按视频实际录制时间（event_start_time，文件名解析），
|
||||
为空回退 processed_at/updated_at/created_at。
|
||||
date_filter 形如 '2026-08-21'。
|
||||
只返回"有内容"的会话：运动片段（filename 前缀 motion_）或含事件的视频；
|
||||
整段素材分割 0 段的空会话（历史素材无运动事件）不展示，避免淹没时间轴。
|
||||
busy_timeout：fam-edge 的写事务提交时会短暂持锁，这里等而不是立刻报
|
||||
`database is locked`（9/3 那次 Oracle 过载时刷过一片这个错）。
|
||||
"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
# 录制时间优先，回退分析时间
|
||||
date_expr = "COALESCE(NULLIF(event_start_time,''), processed_at, updated_at, created_at)"
|
||||
# 注意：pymysql execute 用 % 做参数占位符，SQL 字面量不能含 %，故用 LEFT 判断 motion_ 前缀
|
||||
content_filter = ("LEFT(v.filename, 7) = 'motion_' "
|
||||
"OR EXISTS(SELECT 1 FROM sync_events se WHERE se.video_id = v.id)")
|
||||
if date_filter:
|
||||
cur.execute(
|
||||
f"""SELECT v.id, v.filename, v.camera_name, v.event_start_time, v.status,
|
||||
conn = sqlite3.connect(_path(), timeout=10)
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA busy_timeout=10000")
|
||||
return conn
|
||||
|
||||
|
||||
def _rows(cur) -> List[Dict]:
|
||||
return [dict(r) for r in cur.fetchall()]
|
||||
|
||||
|
||||
def _row(cur) -> Optional[Dict]:
|
||||
r = cur.fetchone()
|
||||
return dict(r) if r else None
|
||||
|
||||
|
||||
# 人物命中判断：person_list_json 是 JSON 数组文本，用 json_each 展开精确匹配。
|
||||
# 必须先 json_valid——历史数据里有非 JSON 的脏值，直接 json_each 会整条查询报错。
|
||||
_PERSON_HIT = ("{col} IS NOT NULL AND json_valid({col}) "
|
||||
"AND EXISTS(SELECT 1 FROM json_each({col}) WHERE json_each.value = ?)")
|
||||
|
||||
# 日期维度：录制时间优先（文件名解析出来的），回退分析时间
|
||||
_DATE_EXPR = "COALESCE(NULLIF({a}.event_start_time,''), {a}.processed_at, {a}.updated_at, {a}.created_at)"
|
||||
|
||||
# 只展示"有内容"的会话：运动片段，或含事件的视频。
|
||||
# 整段素材分割 0 段的空会话（历史素材无运动事件）不展示，避免淹没时间轴。
|
||||
_CONTENT_FILTER = ("substr(v.filename, 1, 7) = 'motion_' "
|
||||
"OR EXISTS(SELECT 1 FROM events se WHERE se.video_id = v.id)")
|
||||
|
||||
|
||||
# ============================================================
|
||||
# videos / events
|
||||
# ============================================================
|
||||
def get_videos(limit=15, offset=0, date_filter=None) -> List[Dict]:
|
||||
"""视频会话列表（事件时间轴左侧），按录制时间倒序，支持日期筛选与分页。"""
|
||||
d = _DATE_EXPR.format(a='v')
|
||||
sql = f"""SELECT v.id, v.filename, v.camera_name, v.event_start_time, v.status,
|
||||
v.summary_json, v.events_json, v.people_json, v.compute_provider,
|
||||
v.processed_at, v.updated_at,
|
||||
(SELECT COUNT(*) FROM sync_events se WHERE se.video_id = v.id) AS event_count
|
||||
FROM sync_videos v
|
||||
WHERE v.status='done' AND ({content_filter}) AND {date_expr} LIKE %s
|
||||
ORDER BY {date_expr} DESC
|
||||
LIMIT %s OFFSET %s""",
|
||||
(SELECT COUNT(*) FROM events se WHERE se.video_id = v.id) AS event_count
|
||||
FROM videos v
|
||||
WHERE v.status='done' AND ({_CONTENT_FILTER}) {{extra}}
|
||||
ORDER BY {d} DESC
|
||||
LIMIT ? OFFSET ?"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
if date_filter:
|
||||
cur = conn.execute(sql.format(extra=f"AND {d} LIKE ?"),
|
||||
(f'{date_filter}%', limit, offset))
|
||||
else:
|
||||
cur.execute(
|
||||
f"""SELECT v.id, v.filename, v.camera_name, v.event_start_time, v.status,
|
||||
v.summary_json, v.events_json, v.people_json, v.compute_provider,
|
||||
v.processed_at, v.updated_at,
|
||||
(SELECT COUNT(*) FROM sync_events se WHERE se.video_id = v.id) AS event_count
|
||||
FROM sync_videos v
|
||||
WHERE v.status='done' AND ({content_filter})
|
||||
ORDER BY {date_expr} DESC
|
||||
LIMIT %s OFFSET %s""",
|
||||
(limit, offset))
|
||||
return cur.fetchall()
|
||||
cur = conn.execute(sql.format(extra=''), (limit, offset))
|
||||
return _rows(cur)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_video(video_id: int) -> Optional[Dict]:
|
||||
def get_video(video_id: int) -> Optional[Dict]:
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute("SELECT * FROM sync_videos WHERE id=%s", (video_id,))
|
||||
return cur.fetchone()
|
||||
return _row(conn.execute("SELECT * FROM videos WHERE id=?", (video_id,)))
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def delete_sync_video(video_id: int):
|
||||
"""删除本地镜像里的这条视频（events 先删，再删 videos）。
|
||||
|
||||
增量同步（get_sync_delta）只做 upsert，感知不到 Oracle 那边的物理删除，
|
||||
所以删除操作不能走"转发 Oracle + trigger_now() 拉增量"这条老路，必须在
|
||||
Oracle 确认删除成功后由调用方显式清理本地镜像。
|
||||
"""
|
||||
def get_events_for_video(video_id: int) -> List[Dict]:
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
cur.execute("DELETE FROM sync_events WHERE video_id=%s", (video_id,))
|
||||
cur.execute("DELETE FROM sync_videos WHERE id=%s", (video_id,))
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# 同步镜像：sync_events
|
||||
# ============================================================
|
||||
|
||||
def upsert_sync_events(rows: List[Dict]) -> int:
|
||||
"""批量 upsert Oracle 传来的 events 增量。"""
|
||||
if not rows:
|
||||
return 0
|
||||
conn = get_conn()
|
||||
n = 0
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
for r in rows:
|
||||
cur.execute(
|
||||
"""INSERT INTO sync_events
|
||||
(id, video_id, ts, description, person_list_json,
|
||||
person_appearances_json, is_attention_event,
|
||||
updated_at, synced_at)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s, NOW())
|
||||
ON DUPLICATE KEY UPDATE
|
||||
video_id=VALUES(video_id),
|
||||
ts=VALUES(ts),
|
||||
description=VALUES(description),
|
||||
person_list_json=VALUES(person_list_json),
|
||||
person_appearances_json=VALUES(person_appearances_json),
|
||||
is_attention_event=VALUES(is_attention_event),
|
||||
updated_at=VALUES(updated_at),
|
||||
synced_at=NOW()""",
|
||||
(r.get('id'), r.get('video_id'), r.get('ts'), r.get('description'),
|
||||
r.get('person_list_json'), r.get('person_appearances_json'),
|
||||
1 if r.get('is_attention_event') else 0,
|
||||
r.get('updated_at'))
|
||||
)
|
||||
n += 1
|
||||
conn.commit()
|
||||
return n
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_events_for_video(video_id: int) -> List[Dict]:
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute(
|
||||
return _rows(conn.execute(
|
||||
"""SELECT id, video_id, ts, description, person_list_json,
|
||||
is_attention_event, updated_at
|
||||
FROM sync_events WHERE video_id=%s ORDER BY ts ASC""",
|
||||
(video_id,))
|
||||
return cur.fetchall()
|
||||
is_attention_event
|
||||
FROM events WHERE video_id=? ORDER BY ts ASC""", (video_id,)))
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_people_clips(label: str, limit: int = 10) -> List[Dict]:
|
||||
"""某人物（label 或 canonical_name）出现过的运动片段列表。
|
||||
|
||||
匹配：label 本身 + 其 canonical_name 下全部 label；命中 sync_events 的
|
||||
person_list_json（JSON_CONTAINS）→ 关联 video（优先运动片段 motion_）。
|
||||
返回按 event_start_time 倒序，含 first_ts（该人物在片段内最早事件 ts，
|
||||
前端缩略图定位用）与 clip_events（片段内该人物事件数）。
|
||||
"""
|
||||
def get_attention_events(limit: int = 200) -> List[Dict]:
|
||||
"""需关注事件（统计图表页用），按录制日期倒序。"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
return _rows(conn.execute(
|
||||
"""SELECT COALESCE(NULLIF(v.event_start_time,''), v.processed_at) AS ev_date,
|
||||
e.person_list_json
|
||||
FROM events e JOIN videos v ON e.video_id=v.id
|
||||
WHERE e.is_attention_event = 1
|
||||
ORDER BY ev_date DESC LIMIT ?""", (limit,)))
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_people_clips(label: str, limit: int = 10) -> List[Dict]:
|
||||
"""某人物出现过的运动片段列表。
|
||||
|
||||
匹配 label 本身 + 同一 canonical_name 下的全部 label；返回含 first_ts
|
||||
（该人物在片段内最早事件时间点，前端缩略图定位用）与 clip_events。
|
||||
"""
|
||||
hit_e = _PERSON_HIT.format(col='e.person_list_json')
|
||||
hit_e2 = _PERSON_HIT.format(col='e2.person_list_json')
|
||||
hit_e3 = _PERSON_HIT.format(col='e3.person_list_json')
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
labels = {label}
|
||||
cur.execute(
|
||||
"SELECT label, canonical_name FROM sync_people WHERE label=%s OR canonical_name=%s",
|
||||
(label, label))
|
||||
for r in cur.fetchall():
|
||||
cn = r.get('canonical_name')
|
||||
if cn:
|
||||
cur.execute("SELECT label FROM sync_people WHERE canonical_name=%s", (cn,))
|
||||
labels.update(x['label'] for x in cur.fetchall())
|
||||
clips = []
|
||||
seen = set()
|
||||
for r in _rows(conn.execute(
|
||||
"SELECT label, canonical_name FROM people WHERE label=? OR canonical_name=?",
|
||||
(label, label))):
|
||||
if r.get('canonical_name'):
|
||||
labels.update(x['label'] for x in _rows(conn.execute(
|
||||
"SELECT label FROM people WHERE canonical_name=?", (r['canonical_name'],))))
|
||||
clips, seen = [], set()
|
||||
for lb in sorted(labels):
|
||||
cur.execute(
|
||||
"""SELECT DISTINCT v.id AS video_id, v.filename, v.event_start_time,
|
||||
cur = conn.execute(
|
||||
f"""SELECT DISTINCT v.id AS video_id, v.filename, v.event_start_time,
|
||||
v.duration_sec, v.summary_json, v.camera_name,
|
||||
(SELECT MIN(e2.ts) FROM sync_events e2
|
||||
WHERE e2.video_id=v.id AND e2.person_list_json IS NOT NULL
|
||||
AND JSON_CONTAINS(e2.person_list_json, JSON_QUOTE(%s), '$')) AS first_ts,
|
||||
(SELECT COUNT(*) FROM sync_events e3
|
||||
WHERE e3.video_id=v.id AND e3.person_list_json IS NOT NULL
|
||||
AND JSON_CONTAINS(e3.person_list_json, JSON_QUOTE(%s), '$')) AS clip_events
|
||||
FROM sync_events e
|
||||
JOIN sync_videos v ON e.video_id=v.id
|
||||
WHERE e.person_list_json IS NOT NULL
|
||||
AND JSON_CONTAINS(e.person_list_json, JSON_QUOTE(%s), '$')
|
||||
AND v.status='done'""",
|
||||
(SELECT MIN(e2.ts) FROM events e2
|
||||
WHERE e2.video_id=v.id AND {hit_e2}) AS first_ts,
|
||||
(SELECT COUNT(*) FROM events e3
|
||||
WHERE e3.video_id=v.id AND {hit_e3}) AS clip_events
|
||||
FROM events e JOIN videos v ON e.video_id=v.id
|
||||
WHERE {hit_e} AND v.status='done'""",
|
||||
(lb, lb, lb))
|
||||
for row in cur.fetchall():
|
||||
for row in _rows(cur):
|
||||
if row['video_id'] not in seen:
|
||||
seen.add(row['video_id'])
|
||||
clips.append(row)
|
||||
@@ -273,212 +180,55 @@ def get_sync_people_clips(label: str, limit: int = 10) -> List[Dict]:
|
||||
conn.close()
|
||||
|
||||
|
||||
def query_sync_events_for_person_date(person: str, date_str: str) -> List[Dict]:
|
||||
def query_events_for_person_date(person: str, date_str: str) -> List[Dict]:
|
||||
"""问答上下文：某人在某天的事件。
|
||||
|
||||
说明: Oracle 事件 ts 为视频内相对时间点（如 00:01:23），不是绝对日期，
|
||||
因此按所属视频的录制日期（event_start_time，回退 processed_at）过滤，
|
||||
再按 person_list_json 命中人名。
|
||||
person 可为真名或抽象标签（Oracle 回灌上下文用真名，但历史标签也保留）。
|
||||
事件 ts 是视频内的相对时间点（如 00:01:23），不是绝对日期，所以按所属视频的
|
||||
录制日期过滤，再按 person_list_json 命中人名。person 可为真名或抽象标签。
|
||||
"""
|
||||
hit = _PERSON_HIT.format(col='e.person_list_json')
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute(
|
||||
"""SELECT e.ts, e.description, e.person_list_json, e.is_attention_event,
|
||||
return _rows(conn.execute(
|
||||
f"""SELECT e.ts, e.description, e.person_list_json, e.is_attention_event,
|
||||
v.camera_name, v.filename, v.event_start_time, v.processed_at
|
||||
FROM sync_events e
|
||||
JOIN sync_videos v ON e.video_id = v.id
|
||||
WHERE COALESCE(NULLIF(v.event_start_time,''), v.processed_at) LIKE %s
|
||||
AND e.person_list_json IS NOT NULL
|
||||
AND JSON_CONTAINS(e.person_list_json, JSON_QUOTE(%s), '$')
|
||||
FROM events e JOIN videos v ON e.video_id = v.id
|
||||
WHERE COALESCE(NULLIF(v.event_start_time,''), v.processed_at) LIKE ?
|
||||
AND {hit}
|
||||
ORDER BY COALESCE(NULLIF(v.event_start_time,''), v.processed_at) ASC, e.ts ASC""",
|
||||
(f'{date_str}%', person))
|
||||
return cur.fetchall()
|
||||
(f'{date_str}%', person)))
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# 同步镜像：sync_people
|
||||
# people
|
||||
# ============================================================
|
||||
|
||||
def upsert_sync_people(rows: List[Dict]) -> int:
|
||||
"""批量 upsert Oracle 传来的 people 增量。"""
|
||||
if not rows:
|
||||
return 0
|
||||
conn = get_conn()
|
||||
n = 0
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
for r in rows:
|
||||
cur.execute(
|
||||
"""INSERT INTO sync_people
|
||||
(id, label, canonical_name, first_seen, appearances,
|
||||
source, features_json, display_uid, updated_at, synced_at)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s, NOW())
|
||||
ON DUPLICATE KEY UPDATE
|
||||
label=VALUES(label),
|
||||
canonical_name=VALUES(canonical_name),
|
||||
first_seen=VALUES(first_seen),
|
||||
appearances=VALUES(appearances),
|
||||
source=VALUES(source),
|
||||
features_json=VALUES(features_json),
|
||||
display_uid=VALUES(display_uid),
|
||||
updated_at=VALUES(updated_at),
|
||||
synced_at=NOW()""",
|
||||
(r.get('id'), r.get('label'), r.get('canonical_name'),
|
||||
r.get('first_seen'), r.get('appearances') or 0,
|
||||
r.get('source'), r.get('features_json'),
|
||||
r.get('display_uid'), r.get('updated_at')))
|
||||
n += 1
|
||||
conn.commit()
|
||||
return n
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_people() -> List[Dict]:
|
||||
def get_people() -> List[Dict]:
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute(
|
||||
return _rows(conn.execute(
|
||||
"SELECT id, label, canonical_name, first_seen, appearances, source, "
|
||||
"features_json, display_uid, updated_at "
|
||||
"FROM sync_people ORDER BY id ASC")
|
||||
return cur.fetchall()
|
||||
"features_json, display_uid, updated_at FROM people ORDER BY id ASC"))
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def upsert_sync_model_calls(rows: List[Dict]) -> int:
|
||||
"""批量 upsert Oracle 传来的 model_calls 增量（幂等，重复覆盖）。"""
|
||||
if not rows:
|
||||
return 0
|
||||
conn = get_conn()
|
||||
n = 0
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
for r in rows:
|
||||
cur.execute(
|
||||
"""INSERT INTO sync_model_calls
|
||||
(id, provider, model, video_id, filename, started_at,
|
||||
duration_sec, success, error, created_at, synced_at)
|
||||
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s, NOW())
|
||||
ON DUPLICATE KEY UPDATE
|
||||
provider=VALUES(provider),
|
||||
model=VALUES(model),
|
||||
video_id=VALUES(video_id),
|
||||
filename=VALUES(filename),
|
||||
started_at=VALUES(started_at),
|
||||
duration_sec=VALUES(duration_sec),
|
||||
success=VALUES(success),
|
||||
error=VALUES(error),
|
||||
created_at=VALUES(created_at),
|
||||
synced_at=NOW()""",
|
||||
(r.get('id'), r.get('provider'), r.get('model'),
|
||||
r.get('video_id'), r.get('filename'), r.get('started_at'),
|
||||
r.get('duration_sec') or 0, 1 if r.get('success') else 0,
|
||||
(r.get('error') or '')[:500], r.get('created_at')))
|
||||
n += 1
|
||||
conn.commit()
|
||||
return n
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def upsert_sync_identity_map(rows: List[Dict]) -> int:
|
||||
"""批量 upsert Oracle 传来的 person_identity_map 增量（幂等，重复覆盖）。
|
||||
|
||||
v2 修复 (2026-08-29)：不再以 Oracle 的 person_identity_map.id 作为 NAS 主键。
|
||||
该 id 只是 Oracle 端代理主键，一旦 Oracle 库重建/恢复，id 会被复用（实测出现
|
||||
id=744 同时对应 NAS 上的 (1302,'人物E') 与 Oracle 现在的 (1073,'汤圆')），
|
||||
旧写法会先按 id 撞主键、再 UPDATE 成新组合，触发 uq_video_raw_uid 二次冲突
|
||||
报 1062。现改为：NAS 本地自增 id 做主键，业务键 (video_id, raw_uid) 唯一，
|
||||
Oracle 的 id 只落 oracle_id 列作溯源参考。
|
||||
"""
|
||||
if not rows:
|
||||
return 0
|
||||
conn = get_conn()
|
||||
n = 0
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
for r in rows:
|
||||
cur.execute(
|
||||
"""INSERT INTO sync_identity_map
|
||||
(oracle_id, video_id, raw_uid, canonical_name, source, updated_at, synced_at)
|
||||
VALUES (%s,%s,%s,%s,%s,%s, NOW())
|
||||
ON DUPLICATE KEY UPDATE
|
||||
oracle_id=VALUES(oracle_id),
|
||||
canonical_name=VALUES(canonical_name),
|
||||
source=VALUES(source),
|
||||
updated_at=VALUES(updated_at),
|
||||
synced_at=NOW()""",
|
||||
(r.get('id'), r.get('video_id'), r.get('raw_uid'),
|
||||
r.get('canonical_name'), r.get('source'), r.get('updated_at')))
|
||||
n += 1
|
||||
conn.commit()
|
||||
return n
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_model_calls(limit: int = 200) -> List[Dict]:
|
||||
"""最近模型调用记录（前端统计展示）。"""
|
||||
def get_named_members() -> List[str]:
|
||||
"""已命名成员的真名列表（UI 下拉用）。"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute(
|
||||
"SELECT id, provider, model, video_id, filename, started_at, "
|
||||
"duration_sec, success, error, created_at "
|
||||
"FROM sync_model_calls ORDER BY id DESC LIMIT %s", (limit,))
|
||||
return cur.fetchall()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_model_calls_stats() -> Dict:
|
||||
"""模型调用统计：按 provider+model 聚合成功/失败/平均耗时。"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute(
|
||||
"""SELECT provider, model,
|
||||
SUM(CASE WHEN success=1 THEN 1 ELSE 0 END) AS ok_cnt,
|
||||
SUM(CASE WHEN success=0 THEN 1 ELSE 0 END) AS fail_cnt,
|
||||
ROUND(AVG(duration_sec), 1) AS avg_duration,
|
||||
MAX(created_at) AS last_call
|
||||
FROM sync_model_calls
|
||||
GROUP BY provider, model ORDER BY provider, model""")
|
||||
return cur.fetchall()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_named_members() -> List[str]:
|
||||
"""已命名成员的真名列表（供 UI 下拉 / 快捷选择）。"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute(
|
||||
"SELECT DISTINCT canonical_name FROM sync_people "
|
||||
return [r['canonical_name'] for r in _rows(conn.execute(
|
||||
"SELECT DISTINCT canonical_name FROM people "
|
||||
"WHERE canonical_name IS NOT NULL AND canonical_name != '' "
|
||||
"ORDER BY canonical_name ASC")
|
||||
return [r['canonical_name'] for r in cur.fetchall()]
|
||||
"ORDER BY canonical_name ASC"))]
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_known_members_context() -> str:
|
||||
"""获取人物清单文本，注入问答 Prompt，让模型用真名指代。"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute(
|
||||
"SELECT label, canonical_name FROM sync_people ORDER BY id ASC")
|
||||
rows = cur.fetchall()
|
||||
if not rows:
|
||||
return ""
|
||||
def get_known_members_context() -> str:
|
||||
"""人物清单文本，注入问答 Prompt，让模型用真名指代。"""
|
||||
rows = get_people()
|
||||
parts = []
|
||||
for r in rows:
|
||||
name = r['canonical_name'] or r['label']
|
||||
@@ -487,159 +237,120 @@ def get_sync_known_members_context() -> str:
|
||||
else:
|
||||
parts.append(f"- {name}")
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
# ============================================================
|
||||
# model_calls
|
||||
# ============================================================
|
||||
def get_model_calls(limit: int = 200) -> List[Dict]:
|
||||
conn = get_conn()
|
||||
try:
|
||||
return _rows(conn.execute(
|
||||
"SELECT id, provider, model, video_id, filename, started_at, "
|
||||
"duration_sec, success, error, created_at "
|
||||
"FROM model_calls ORDER BY id DESC LIMIT ?", (limit,)))
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_model_calls_stats() -> List[Dict]:
|
||||
"""按 provider+model 聚合成功/失败/平均耗时。"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
return _rows(conn.execute(
|
||||
"""SELECT provider, model,
|
||||
SUM(CASE WHEN success=1 THEN 1 ELSE 0 END) AS ok_cnt,
|
||||
SUM(CASE WHEN success=0 THEN 1 ELSE 0 END) AS fail_cnt,
|
||||
ROUND(AVG(duration_sec), 1) AS avg_duration,
|
||||
MAX(created_at) AS last_call
|
||||
FROM model_calls GROUP BY provider, model ORDER BY provider, model"""))
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# 同步游标
|
||||
# 概览统计
|
||||
# ============================================================
|
||||
|
||||
def get_sync_cursor() -> str:
|
||||
def get_stats(date_str: str = None) -> Dict:
|
||||
"""视频数 / 事件数 / 关注事件数 / 出现人物数（可按日期过滤）。"""
|
||||
d = _DATE_EXPR.format(a='sv')
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
cur.execute("SELECT `value` FROM sync_cursor WHERE `key`='last_since'")
|
||||
row = cur.fetchone()
|
||||
return row[0] if row else ''
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def set_sync_cursor(value: str):
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
cur.execute(
|
||||
"""INSERT INTO sync_cursor (`key`, `value`) VALUES ('last_since', %s)
|
||||
ON DUPLICATE KEY UPDATE `value`=VALUES(`value`)""",
|
||||
(value,))
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# 运动通知游标（MotionNotifier 增量去重用，复用 sync_cursor 表）
|
||||
# ============================================================
|
||||
|
||||
def get_motion_cursor() -> str:
|
||||
"""返回上次推送到甲骨文的最大 SS 事件 id（字符串），无则返回空。"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
cur.execute("SELECT `value` FROM sync_cursor WHERE `key`='motion_last_event_id'")
|
||||
row = cur.fetchone()
|
||||
return row[0] if row else ''
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def set_motion_cursor(value: str):
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
cur.execute(
|
||||
"""INSERT INTO sync_cursor (`key`, `value`) VALUES ('motion_last_event_id', %s)
|
||||
ON DUPLICATE KEY UPDATE `value`=VALUES(`value`)""",
|
||||
(value,))
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# 统计
|
||||
# ============================================================
|
||||
|
||||
def get_attention_events(limit: int = 200) -> List[Dict]:
|
||||
"""需关注事件列表（供统计图表页展示日期 + 涉及人物），按录制日期倒序。"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
cur.execute(
|
||||
"""SELECT COALESCE(NULLIF(v.event_start_time,''), v.processed_at) AS ev_date,
|
||||
e.person_list_json
|
||||
FROM sync_events e JOIN sync_videos v ON e.video_id=v.id
|
||||
WHERE e.is_attention_event = 1
|
||||
ORDER BY ev_date DESC LIMIT %s""",
|
||||
(limit,))
|
||||
return cur.fetchall()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def get_sync_stats(date_str: str = None) -> Dict:
|
||||
"""概览统计：视频数 / 事件数 / 关注事件数 / 出现人物数（按 date 可选过滤）。
|
||||
|
||||
人物数通过对 sync_events.person_list_json LIKE 统计（MariaDB 不支持 JSON 数组展开）。
|
||||
"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
# 视频/事件/关注数（日期维度=录制时间 event_start_time，回退 processed_at）
|
||||
_D = "COALESCE(NULLIF(sv.event_start_time,''), sv.processed_at)"
|
||||
if date_str:
|
||||
cur.execute(
|
||||
"""SELECT
|
||||
(SELECT COUNT(*) FROM sync_videos sv
|
||||
like = f'{date_str}%'
|
||||
stat = _row(conn.execute(
|
||||
f"""SELECT
|
||||
(SELECT COUNT(*) FROM videos sv
|
||||
WHERE sv.status='done'
|
||||
AND (LEFT(sv.filename, 7) = 'motion_'
|
||||
OR EXISTS(SELECT 1 FROM sync_events se WHERE se.video_id=sv.id))
|
||||
AND {d} LIKE %s) AS videos,
|
||||
(SELECT COUNT(*) FROM sync_events se
|
||||
JOIN sync_videos sv ON se.video_id=sv.id
|
||||
WHERE {d} LIKE %s) AS events,
|
||||
(SELECT COALESCE(SUM(se.is_attention_event),0) FROM sync_events se
|
||||
JOIN sync_videos sv ON se.video_id=sv.id
|
||||
WHERE {d} LIKE %s) AS attention""".format(d=_D),
|
||||
(f'{date_str}%', f'{date_str}%', f'{date_str}%'))
|
||||
AND (substr(sv.filename,1,7)='motion_'
|
||||
OR EXISTS(SELECT 1 FROM events se WHERE se.video_id=sv.id))
|
||||
AND {d} LIKE ?) AS videos,
|
||||
(SELECT COUNT(*) FROM events se JOIN videos sv ON se.video_id=sv.id
|
||||
WHERE {d} LIKE ?) AS events,
|
||||
(SELECT COALESCE(SUM(se.is_attention_event),0) FROM events se
|
||||
JOIN videos sv ON se.video_id=sv.id
|
||||
WHERE {d} LIKE ?) AS attention""",
|
||||
(like, like, like))) or {}
|
||||
else:
|
||||
cur.execute(
|
||||
stat = _row(conn.execute(
|
||||
"""SELECT
|
||||
(SELECT COUNT(*) FROM sync_videos
|
||||
WHERE status='done'
|
||||
AND (LEFT(filename, 7) = 'motion_'
|
||||
OR EXISTS(SELECT 1 FROM sync_events se WHERE se.video_id=sync_videos.id))) AS videos,
|
||||
(SELECT COUNT(*) FROM sync_events) AS events,
|
||||
(SELECT COALESCE(SUM(is_attention_event),0) FROM sync_events) AS attention""")
|
||||
stat = cur.fetchone() or {}
|
||||
(SELECT COUNT(*) FROM videos sv
|
||||
WHERE sv.status='done'
|
||||
AND (substr(sv.filename,1,7)='motion_'
|
||||
OR EXISTS(SELECT 1 FROM events se WHERE se.video_id=sv.id))) AS videos,
|
||||
(SELECT COUNT(*) FROM events) AS events,
|
||||
(SELECT COALESCE(SUM(is_attention_event),0) FROM events) AS attention""")) or {}
|
||||
|
||||
# 人物数（distinct label 命中 sync_events）
|
||||
cur.execute("SELECT id, label, canonical_name FROM sync_people")
|
||||
people = cur.fetchall()
|
||||
# 基于 person_list_json 命中计数：逐 label 统计命中事件数
|
||||
person_hits = 0
|
||||
for p in people:
|
||||
# 出现人物数：逐 label/真名在事件里查有没有命中（沿用改写前的口径）
|
||||
hits = 0
|
||||
for p in _rows(conn.execute("SELECT label, canonical_name FROM people")):
|
||||
name = p['canonical_name'] or p['label']
|
||||
cur.execute(
|
||||
"SELECT COUNT(*) c FROM sync_events WHERE person_list_json LIKE %s",
|
||||
(f'%{name}%',))
|
||||
if cur.fetchone()['c'] > 0:
|
||||
person_hits += 1
|
||||
stat['people'] = person_hits
|
||||
c = conn.execute("SELECT COUNT(*) AS c FROM events "
|
||||
"WHERE person_list_json LIKE ?", (f'%{name}%',)).fetchone()
|
||||
if c and c['c'] > 0:
|
||||
hits += 1
|
||||
stat['people'] = hits
|
||||
return stat
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ============================================================
|
||||
# chat_history（保留：问答历史）
|
||||
# chat_history（本模块唯一写的表；2026-09-13 从 NAS MariaDB 迁入）
|
||||
# ============================================================
|
||||
def _ensure_chat_schema(conn):
|
||||
global _chat_schema_ready
|
||||
if _chat_schema_ready:
|
||||
return
|
||||
conn.executescript("""
|
||||
CREATE TABLE IF NOT EXISTS chat_history (
|
||||
chat_id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
user_question TEXT NOT NULL,
|
||||
ai_answer TEXT NOT NULL,
|
||||
context_summary TEXT,
|
||||
queried_date TEXT,
|
||||
queried_person TEXT,
|
||||
created_at TEXT
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_chat_created ON chat_history(created_at);
|
||||
""")
|
||||
conn.commit()
|
||||
_chat_schema_ready = True
|
||||
|
||||
|
||||
def insert_chat_history(user_question: str, ai_answer: str,
|
||||
context_summary: str, queried_date: str,
|
||||
queried_person: str) -> int:
|
||||
"""插入对话记录"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor()
|
||||
cur.execute(
|
||||
_ensure_chat_schema(conn)
|
||||
cur = conn.execute(
|
||||
"""INSERT INTO chat_history
|
||||
(user_question, ai_answer, context_summary, queried_date, queried_person)
|
||||
VALUES (%s, %s, %s, %s, %s)""",
|
||||
(user_question, ai_answer, context_summary, queried_date, queried_person)
|
||||
)
|
||||
(user_question, ai_answer, context_summary, queried_date,
|
||||
queried_person, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)""",
|
||||
(user_question, ai_answer, context_summary, queried_date,
|
||||
queried_person, _now()))
|
||||
conn.commit()
|
||||
return cur.lastrowid
|
||||
finally:
|
||||
@@ -647,24 +358,20 @@ def insert_chat_history(user_question: str, ai_answer: str,
|
||||
|
||||
|
||||
def get_chat_history(limit=20, offset=0, date_filter=None, person_filter=None) -> List[Dict]:
|
||||
"""获取对话历史（分页）"""
|
||||
conn = get_conn()
|
||||
try:
|
||||
cur = conn.cursor(pymysql.cursors.DictCursor)
|
||||
conditions = []
|
||||
params = []
|
||||
_ensure_chat_schema(conn)
|
||||
conds, params = [], []
|
||||
if date_filter:
|
||||
conditions.append("queried_date = %s")
|
||||
conds.append("queried_date = ?")
|
||||
params.append(date_filter)
|
||||
if person_filter:
|
||||
conditions.append("queried_person = %s")
|
||||
conds.append("queried_person = ?")
|
||||
params.append(person_filter)
|
||||
where = f"WHERE {' AND '.join(conditions)}" if conditions else ""
|
||||
where = f"WHERE {' AND '.join(conds)}" if conds else ""
|
||||
params.extend([limit, offset])
|
||||
cur.execute(
|
||||
f"SELECT * FROM chat_history {where} ORDER BY created_at DESC LIMIT %s OFFSET %s",
|
||||
params
|
||||
)
|
||||
return cur.fetchall()
|
||||
return _rows(conn.execute(
|
||||
f"SELECT * FROM chat_history {where} ORDER BY created_at DESC LIMIT ? OFFSET ?",
|
||||
params))
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
102
fam-core/src/fam_core/edge_client.py
Normal file
102
fam-core/src/fam_core/edge_client.py
Normal file
@@ -0,0 +1,102 @@
|
||||
"""
|
||||
Edge-Client - 访问同机 fam-edge 的薄客户端（2026-09-13 迁云后新增）
|
||||
|
||||
迁云之前这些调用散在 `oracle_sync.py` 里（NAS 跨公网访问甲骨文），随镜像层一起
|
||||
删掉了。现在 fam-core 与 fam-edge 同机，全部走 127.0.0.1：不出公网、无 TLS、
|
||||
无隧道，token 仍然带着（fam-edge 侧的鉴权没变，且它同时对公网监听 :5000）。
|
||||
|
||||
只保留三类调用：
|
||||
- push_name_correct / push_video_delete —— 写操作仍由 fam-edge 处理，因为它
|
||||
除了改库还要做人物合并、删磁盘素材，不是单纯一条 SQL
|
||||
- get_activity —— 服务状态页
|
||||
- fetch_image —— 事件帧图 / 人物头像（图源和裁剪都在 fam-edge）
|
||||
"""
|
||||
import requests
|
||||
|
||||
from .config_loader import load_config
|
||||
from .logger import setup_logger
|
||||
|
||||
logger = setup_logger('fam-core.edge_client')
|
||||
|
||||
_cfg = None
|
||||
|
||||
|
||||
def _conf():
|
||||
global _cfg
|
||||
if _cfg is None:
|
||||
c = load_config().get('edge', {})
|
||||
_cfg = {
|
||||
'base_url': (c.get('base_url') or 'http://127.0.0.1:5000').rstrip('/'),
|
||||
'token': c.get('token') or '',
|
||||
'timeout': int(c.get('timeout', 60)),
|
||||
}
|
||||
return _cfg
|
||||
|
||||
|
||||
def base_url():
|
||||
return _conf()['base_url']
|
||||
|
||||
|
||||
def token():
|
||||
return _conf()['token']
|
||||
|
||||
|
||||
def _post(rel: str, payload: dict):
|
||||
"""返回 (ok, err)。写操作统一用这个，失败时把原因带回给前端。"""
|
||||
c = _conf()
|
||||
body = dict(payload)
|
||||
if c['token']:
|
||||
body['token'] = c['token']
|
||||
try:
|
||||
resp = requests.post(f"{c['base_url']}{rel}", json=body, timeout=(5, c['timeout']))
|
||||
except requests.RequestException as e:
|
||||
logger.error(f"fam-edge {rel} 请求失败: {e}")
|
||||
return False, str(e)
|
||||
if resp.status_code != 200:
|
||||
logger.warning(f"fam-edge {rel} 返回 {resp.status_code}: {resp.text[:160]}")
|
||||
return False, f"HTTP {resp.status_code} {resp.text[:120]}"
|
||||
return True, ''
|
||||
|
||||
|
||||
def push_name_correct(label: str, canonical_name: str):
|
||||
return _post('/api/oracle/people/correct',
|
||||
{'label': label, 'canonical_name': canonical_name})
|
||||
|
||||
|
||||
def push_identity_correct(video_id: int, current_name: str, new_name: str):
|
||||
return _post('/api/oracle/identity/correct',
|
||||
{'video_id': video_id, 'current_name': current_name, 'new_name': new_name})
|
||||
|
||||
|
||||
def push_video_delete(video_id: int):
|
||||
return _post('/api/oracle/video/delete', {'video_id': video_id})
|
||||
|
||||
|
||||
def get_activity():
|
||||
"""服务状态页用：fam-edge 的队列 / 分割 / 模型活动快照。返回 (data, err)。"""
|
||||
c = _conf()
|
||||
try:
|
||||
r = requests.get(f"{c['base_url']}/api/oracle/activity",
|
||||
params={'token': c['token']}, timeout=(5, 15))
|
||||
except requests.RequestException as e:
|
||||
return None, f"连接 fam-edge 失败: {e}"
|
||||
if r.status_code != 200:
|
||||
return None, f"fam-edge activity HTTP {r.status_code}"
|
||||
return r.json(), None
|
||||
|
||||
|
||||
def fetch_image(rel: str, params: dict):
|
||||
"""帧图 / 头像透传，失败返回 None（调用方给 404）。"""
|
||||
c = _conf()
|
||||
p = dict(params)
|
||||
if c['token']:
|
||||
p['token'] = c['token']
|
||||
try:
|
||||
resp = requests.get(f"{c['base_url']}{rel}", params=p, timeout=(5, c['timeout']))
|
||||
except requests.RequestException as e:
|
||||
logger.error(f"fam-edge {rel} 请求失败: {e}")
|
||||
return None
|
||||
if resp.status_code == 200 and resp.content:
|
||||
return resp.content
|
||||
logger.warning(f"fam-edge {rel} 返回 {resp.status_code}")
|
||||
return None
|
||||
@@ -1,14 +1,12 @@
|
||||
"""
|
||||
Img-Proxy - NAS 端图片代理（图源在 Oracle，计算全部在 Oracle）
|
||||
Img-Proxy - 图片代理（图源与裁剪都在 fam-edge）
|
||||
|
||||
浏览器不直连 Oracle（避免公网暴露 5000 端口与 token 外泄），
|
||||
而是访问 NAS fam-core 的 /api/proxy/*，由 NAS 出网到 Oracle 拉取 jpeg 回传。
|
||||
|
||||
Oracle 侧已有磁盘缓存 / VLM 人物定位裁剪，NAS 端仅透传，不做图像计算。
|
||||
浏览器不直连 fam-edge（避免 token 外泄），而是访问 /api/proxy/*，由本服务转一手。
|
||||
迁云后这一跳是同机 127.0.0.1，纯透传，不做图像计算。
|
||||
"""
|
||||
from flask import Blueprint, Response, request
|
||||
|
||||
from .oracle_sync import get_sync
|
||||
from . import edge_client
|
||||
from .logger import setup_logger
|
||||
|
||||
logger = setup_logger('fam-core.img_proxy')
|
||||
@@ -17,21 +15,7 @@ img_bp = Blueprint('img_proxy', __name__)
|
||||
|
||||
|
||||
def _fetch(rel: str, params: dict):
|
||||
import requests
|
||||
sync = get_sync() # 复用 oracle_sync 已解析好的 base_url/token/timeout，避免两处配置各读一份
|
||||
params = dict(params)
|
||||
if sync.token:
|
||||
params['token'] = sync.token
|
||||
try:
|
||||
resp = requests.get(f"{sync.base_url}{rel}", params=params,
|
||||
timeout=(10, sync.timeout))
|
||||
if resp.status_code == 200 and resp.content:
|
||||
return resp.content
|
||||
logger.warning(f"Oracle {rel} 返回 {resp.status_code}: {resp.text[:120]}")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"Oracle {rel} 请求失败: {e}")
|
||||
return None
|
||||
return edge_client.fetch_image(rel, params)
|
||||
|
||||
|
||||
@img_bp.route('/api/proxy/frame', methods=['GET'])
|
||||
|
||||
@@ -14,7 +14,7 @@ from flask import Blueprint, request, jsonify
|
||||
|
||||
from ..logger import setup_logger
|
||||
from .. import db_layer
|
||||
from ..oracle_sync import get_sync
|
||||
from .. import edge_client
|
||||
|
||||
logger = setup_logger('fam-core.member_manager')
|
||||
|
||||
@@ -24,7 +24,7 @@ member_bp = Blueprint('member_manager', __name__)
|
||||
@member_bp.route('/api/member/unnamed', methods=['GET'])
|
||||
def list_unnamed():
|
||||
"""列出未命名人物（canonical_name 为空）"""
|
||||
members = db_layer.get_sync_people()
|
||||
members = db_layer.get_people()
|
||||
result = []
|
||||
for m in members:
|
||||
canonical = m.get('canonical_name')
|
||||
@@ -40,7 +40,7 @@ def list_unnamed():
|
||||
@member_bp.route('/api/member/list', methods=['GET'])
|
||||
def list_members():
|
||||
"""列出所有人物（按 canonical_name 或 label 展示）"""
|
||||
members = db_layer.get_sync_people()
|
||||
members = db_layer.get_people()
|
||||
result = []
|
||||
for m in members:
|
||||
canonical = m.get('canonical_name')
|
||||
@@ -58,7 +58,7 @@ def list_members():
|
||||
|
||||
@member_bp.route('/api/member/name', methods=['POST'])
|
||||
def name_member():
|
||||
"""命名人物（回推 Oracle + 立即拉回本地镜像）
|
||||
"""命名人物（交给 fam-edge 落库 + 合并人物）
|
||||
|
||||
请求: {"label": "人物A", "canonical_name": "张三"}
|
||||
"""
|
||||
@@ -71,18 +71,13 @@ def name_member():
|
||||
if not label or not canonical_name:
|
||||
return jsonify({"error": "缺少必填字段: label, canonical_name"}), 400
|
||||
|
||||
logger.info(f"命名: {label} -> {canonical_name}（回推 Oracle）")
|
||||
ok, err = get_sync().push_name_correct(label, canonical_name)
|
||||
logger.info(f"命名: {label} -> {canonical_name}（回推 fam-edge）")
|
||||
ok, err = edge_client.push_name_correct(label, canonical_name)
|
||||
if not ok:
|
||||
return jsonify({"error": f"回推 Oracle 失败: {err}"}), 502
|
||||
return jsonify({"error": f"回推 fam-edge 失败: {err}"}), 502
|
||||
|
||||
# 立即拉回最新 people 镜像，前端无需等待下一个 30 分钟周期
|
||||
try:
|
||||
get_sync().trigger_now()
|
||||
except Exception as e:
|
||||
logger.warning(f"命名后即时拉回失败（下一个周期会自动同步）: {e}")
|
||||
|
||||
members = db_layer.get_sync_people()
|
||||
members = db_layer.get_people()
|
||||
return jsonify({
|
||||
"status": "ok",
|
||||
"label": label,
|
||||
@@ -114,24 +109,20 @@ def merge_member():
|
||||
return jsonify({"error": "source 与 target 不能相同"}), 400
|
||||
|
||||
# 解析 target 的规范名
|
||||
members = {m['label']: m for m in db_layer.get_sync_people()}
|
||||
members = {m['label']: m for m in db_layer.get_people()}
|
||||
target_row = members.get(target)
|
||||
if target_row and target_row.get('canonical_name'):
|
||||
canonical = target_row['canonical_name']
|
||||
else:
|
||||
canonical = target # target 未命名 -> 以 label 作为规范名
|
||||
|
||||
logger.info(f"合并: {source} -> {canonical}（回推 Oracle）")
|
||||
ok, err = get_sync().push_name_correct(source, canonical)
|
||||
logger.info(f"合并: {source} -> {canonical}（回推 fam-edge）")
|
||||
ok, err = edge_client.push_name_correct(source, canonical)
|
||||
if not ok:
|
||||
return jsonify({"error": f"回推 Oracle 失败: {err}"}), 502
|
||||
return jsonify({"error": f"回推 fam-edge 失败: {err}"}), 502
|
||||
|
||||
try:
|
||||
get_sync().trigger_now()
|
||||
except Exception as e:
|
||||
logger.warning(f"合并后即时拉回失败（下一个周期会自动同步）: {e}")
|
||||
|
||||
members = db_layer.get_sync_people()
|
||||
members = db_layer.get_people()
|
||||
return jsonify({
|
||||
"status": "ok",
|
||||
"source": source,
|
||||
@@ -163,15 +154,11 @@ def identity_correct():
|
||||
if not video_id or not current_name or not new_name:
|
||||
return jsonify({"error": "缺少必填字段: video_id, current_name, new_name"}), 400
|
||||
|
||||
logger.info(f"人物纠错: video_id={video_id} {current_name} -> {new_name}（回推 Oracle）")
|
||||
ok, err = get_sync().push_identity_correct(video_id, current_name, new_name)
|
||||
logger.info(f"人物纠错: video_id={video_id} {current_name} -> {new_name}（回推 fam-edge）")
|
||||
ok, err = edge_client.push_identity_correct(video_id, current_name, new_name)
|
||||
if not ok:
|
||||
return jsonify({"error": f"回推 Oracle 失败: {err}"}), 502
|
||||
return jsonify({"error": f"回推 fam-edge 失败: {err}"}), 502
|
||||
|
||||
try:
|
||||
get_sync().trigger_now()
|
||||
except Exception as e:
|
||||
logger.warning(f"纠错后即时拉回失败（下一个周期会自动同步）: {e}")
|
||||
|
||||
return jsonify({
|
||||
"status": "ok", "video_id": video_id,
|
||||
|
||||
@@ -1,20 +0,0 @@
|
||||
"""
|
||||
运动监测状态接口（fam-core）
|
||||
|
||||
运动事件采集走 MotionNotifier 轮询 SS EventCenter.Event.List（真实
|
||||
event_id/start_time/duration），不再提供 /api/ss/webhook 接收端点
|
||||
（SS Webhook 行动规则已弃用，2026-08-25 移除）。
|
||||
"""
|
||||
from flask import Blueprint, jsonify
|
||||
|
||||
from .logger import setup_logger
|
||||
from .motion_notifier.motion_notifier import get_motion_notifier
|
||||
|
||||
logger = setup_logger('fam-core.motion_bp')
|
||||
|
||||
motion_bp = Blueprint('motion_bp', __name__)
|
||||
|
||||
|
||||
@motion_bp.route('/api/ss/status', methods=['GET'])
|
||||
def ss_status():
|
||||
return jsonify(get_motion_notifier().status()), 200
|
||||
@@ -1,4 +0,0 @@
|
||||
"""MotionNotifier 包：NAS 端运动监测通知服务。"""
|
||||
from .motion_notifier import MotionNotifier, get_motion_notifier
|
||||
|
||||
__all__ = ["MotionNotifier", "get_motion_notifier"]
|
||||
@@ -1,234 +0,0 @@
|
||||
"""
|
||||
Oracle-Sync - NAS 端唯一后台线程
|
||||
|
||||
职责:
|
||||
1. 每 interval_sec（默认 1800s = 30 分钟）从甲骨文 FAM-Edge 拉取增量:
|
||||
GET {base_url}/api/oracle/sync?since=<cursor>&token=<token>
|
||||
返回 {videos, events, people, server_time}，写入本地 MariaDB 镜像表
|
||||
(sync_videos / sync_events / sync_people)，并推进 sync_cursor。
|
||||
2. 接收命名校正回推: POST {base_url}/api/oracle/people/correct
|
||||
{label, canonical_name, token} —— 手动命名（manual 优先，不被 LLM 覆盖）。
|
||||
|
||||
数据流向（新架构 v2）:
|
||||
Google 硬盘 --rclone--> 甲骨文本地 --> 整视频分析 --> Oracle SQLite
|
||||
--> [本线程每 30 分钟拉增量] --> NAS MariaDB 镜像 --> fam-ui 读取展示
|
||||
|
||||
NAS 不再处理任何视频，CPU 占用显著降低。
|
||||
"""
|
||||
import time
|
||||
import threading
|
||||
import requests
|
||||
from datetime import datetime
|
||||
|
||||
from .logger import setup_logger
|
||||
from .config_loader import load_config
|
||||
from . import db_layer
|
||||
|
||||
logger = setup_logger('fam-core.oracle_sync')
|
||||
|
||||
_SYNC_INSTANCE = None
|
||||
|
||||
|
||||
def get_sync():
|
||||
"""模块级单例（app.py 启动时创建并 start，其余模块经此获取）"""
|
||||
global _SYNC_INSTANCE
|
||||
if _SYNC_INSTANCE is None:
|
||||
_SYNC_INSTANCE = OracleSync()
|
||||
return _SYNC_INSTANCE
|
||||
|
||||
|
||||
class OracleSync:
|
||||
def __init__(self):
|
||||
cfg = load_config().get('oracle_sync', {})
|
||||
self.base_url = cfg.get('base_url', 'http://129.146.26.249:5000').rstrip('/')
|
||||
self.token = cfg.get('token', '')
|
||||
self.interval_sec = int(cfg.get('interval_sec', 1800))
|
||||
self.timeout = int(cfg.get('timeout', 120))
|
||||
self._running = False
|
||||
self._thread = None
|
||||
self._last_sync_at = None
|
||||
self._last_error = None
|
||||
self._last_count = None
|
||||
# 拉取互斥锁：trigger_now（命名后即时拉回）与后台 _run 并发时只允许一个执行
|
||||
self._pull_lock = threading.Lock()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
def _pull_once(self) -> bool:
|
||||
"""执行一次增量拉取（加锁防并发双拉）。返回是否成功。"""
|
||||
with self._pull_lock:
|
||||
return self._pull_once_locked()
|
||||
|
||||
def _pull_once_locked(self) -> bool:
|
||||
since = db_layer.get_sync_cursor() or ''
|
||||
params = {'since': since, 'token': self.token}
|
||||
try:
|
||||
resp = requests.get(
|
||||
f"{self.base_url}/api/oracle/sync",
|
||||
params=params, timeout=(10, self.timeout))
|
||||
except requests.RequestException as e:
|
||||
self._last_error = f"请求失败: {e}"
|
||||
logger.error(f"拉取同步失败: {e}")
|
||||
return False
|
||||
|
||||
if resp.status_code == 401:
|
||||
self._last_error = "token 校验失败"
|
||||
logger.error("同步 token 校验失败 (401)，请检查 oracle_sync.token 配置")
|
||||
return False
|
||||
if resp.status_code != 200:
|
||||
self._last_error = f"HTTP {resp.status_code}"
|
||||
logger.error(f"同步返回异常: {resp.status_code} {resp.text[:200]}")
|
||||
return False
|
||||
|
||||
try:
|
||||
data = resp.json()
|
||||
except ValueError:
|
||||
self._last_error = "非 JSON 响应"
|
||||
logger.error("同步返回非 JSON 响应")
|
||||
return False
|
||||
|
||||
videos = data.get('videos', []) or []
|
||||
events = data.get('events', []) or []
|
||||
people = data.get('people', []) or []
|
||||
model_calls = data.get('model_calls', []) or []
|
||||
identity_map = data.get('identity_map', []) or []
|
||||
server_time = data.get('server_time', '') or ''
|
||||
|
||||
n_videos = db_layer.upsert_sync_videos(videos)
|
||||
n_events = db_layer.upsert_sync_events(events)
|
||||
n_people = db_layer.upsert_sync_people(people)
|
||||
n_calls = db_layer.upsert_sync_model_calls(model_calls)
|
||||
n_identity = db_layer.upsert_sync_identity_map(identity_map)
|
||||
|
||||
if server_time:
|
||||
db_layer.set_sync_cursor(server_time)
|
||||
|
||||
self._last_sync_at = datetime.now()
|
||||
self._last_error = None
|
||||
self._last_count = (n_videos, n_events, n_people, n_calls, n_identity)
|
||||
logger.info(
|
||||
f"同步完成: videos+{n_videos} events+{n_events} people+{n_people} "
|
||||
f"model_calls+{n_calls} identity_map+{n_identity} since={since!r} "
|
||||
f"-> server_time={server_time}")
|
||||
return True
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
def push_name_correct(self, label: str, canonical_name: str):
|
||||
"""回推命名校正到 Oracle（手动命名优先级最高，不被 LLM 覆盖）。
|
||||
|
||||
返回 (success: bool, error: str)
|
||||
"""
|
||||
label = (label or '').strip()
|
||||
canonical_name = (canonical_name or '').strip()
|
||||
if not label or not canonical_name:
|
||||
return False, "缺少 label / canonical_name"
|
||||
try:
|
||||
resp = requests.post(
|
||||
f"{self.base_url}/api/oracle/people/correct",
|
||||
json={"label": label, "canonical_name": canonical_name,
|
||||
"token": self.token},
|
||||
timeout=(10, 30))
|
||||
except requests.RequestException as e:
|
||||
logger.error(f"命名校正回推失败: {e}")
|
||||
return False, str(e)
|
||||
if resp.status_code == 200:
|
||||
return True, ""
|
||||
msg = f"HTTP {resp.status_code}: {resp.text[:200]}"
|
||||
logger.error(f"命名校正回推失败: {msg}")
|
||||
return False, msg
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
def push_identity_correct(self, video_id: int, current_name: str, new_name: str):
|
||||
"""回推事件时间轴/人物管理"这个人识别错了"纠错到 Oracle。
|
||||
|
||||
跟 push_name_correct 的区别：这个按 (video_id, 当前展示名) 定位，只改这
|
||||
一段视频里错认的那个人，不影响同名字符串在其他视频里的映射（人物 uid
|
||||
只在单次视频分析内稳定，跨视频复用同一字符串完全可能是不同真人）。
|
||||
|
||||
返回 (success: bool, error: str)
|
||||
"""
|
||||
try:
|
||||
video_id = int(video_id)
|
||||
except (TypeError, ValueError):
|
||||
return False, "video_id 必须是数字"
|
||||
current_name = (current_name or '').strip()
|
||||
new_name = (new_name or '').strip()
|
||||
if not current_name or not new_name:
|
||||
return False, "缺少 current_name / new_name"
|
||||
try:
|
||||
resp = requests.post(
|
||||
f"{self.base_url}/api/oracle/identity/correct",
|
||||
json={"video_id": video_id, "current_name": current_name,
|
||||
"new_name": new_name, "token": self.token},
|
||||
timeout=(10, 30))
|
||||
except requests.RequestException as e:
|
||||
logger.error(f"人物纠错回推失败: {e}")
|
||||
return False, str(e)
|
||||
if resp.status_code == 200:
|
||||
return True, ""
|
||||
msg = f"HTTP {resp.status_code}: {resp.text[:200]}"
|
||||
logger.error(f"人物纠错回推失败: {msg}")
|
||||
return False, msg
|
||||
|
||||
def push_video_delete(self, video_id: int):
|
||||
"""回推事件时间轴"删除该视频"到 Oracle。返回 (success: bool, error: str)。"""
|
||||
try:
|
||||
video_id = int(video_id)
|
||||
except (TypeError, ValueError):
|
||||
return False, "video_id 必须是数字"
|
||||
try:
|
||||
resp = requests.post(
|
||||
f"{self.base_url}/api/oracle/video/delete",
|
||||
json={"video_id": video_id, "token": self.token},
|
||||
timeout=(10, 30))
|
||||
except requests.RequestException as e:
|
||||
logger.error(f"视频删除回推失败: {e}")
|
||||
return False, str(e)
|
||||
if resp.status_code == 200:
|
||||
return True, ""
|
||||
msg = f"HTTP {resp.status_code}: {resp.text[:200]}"
|
||||
logger.error(f"视频删除回推失败: {msg}")
|
||||
return False, msg
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
def _run(self):
|
||||
logger.info(f"OracleSync 线程启动，间隔 {self.interval_sec}s，目标 {self.base_url}")
|
||||
while self._running:
|
||||
try:
|
||||
self._pull_once()
|
||||
except Exception as e:
|
||||
self._last_error = str(e)
|
||||
logger.error(f"同步异常: {e}", exc_info=True)
|
||||
# 分段休眠，便于 stop 快速唤醒
|
||||
for _ in range(self.interval_sec):
|
||||
if not self._running:
|
||||
break
|
||||
time.sleep(1)
|
||||
|
||||
def start(self):
|
||||
if self._running:
|
||||
return
|
||||
self._running = True
|
||||
self._thread = threading.Thread(target=self._run, daemon=True, name='oracle-sync')
|
||||
self._thread.start()
|
||||
|
||||
def is_alive(self):
|
||||
return self._thread is not None and self._thread.is_alive()
|
||||
|
||||
def stop(self):
|
||||
self._running = False
|
||||
if self._thread:
|
||||
self._thread.join(timeout=5)
|
||||
|
||||
def trigger_now(self) -> bool:
|
||||
"""立即触发一次同步（命名后即时拉回 / 手动）。"""
|
||||
return self._pull_once()
|
||||
|
||||
def status(self) -> dict:
|
||||
return {
|
||||
"running": self.is_alive(),
|
||||
"last_sync_at": self._last_sync_at.isoformat() if self._last_sync_at else None,
|
||||
"last_error": self._last_error,
|
||||
"last_count": self._last_count,
|
||||
"cursor": db_layer.get_sync_cursor(),
|
||||
"interval_sec": self.interval_sec,
|
||||
}
|
||||
@@ -7,9 +7,8 @@ UI-API - Vue 前端数据接口（新架构 v3：Vue SPA 取代 Streamlit）
|
||||
<id>`（删除视频会话），因为语义上属于 videos 这个资源，比塞进 member_manager.py
|
||||
更清晰；写法沿用项目里"先回推 Oracle，成功后处理本地状态"的既有模式。
|
||||
|
||||
/api/ui/service-status 需要代理 Oracle 的 /api/oracle/activity（浏览器不直连
|
||||
Oracle，避免 token 暴露），写法照抄 img_proxy.py 的模式：复用 oracle_sync.get_sync()
|
||||
已解析好的 base_url/token，不再单独存一份配置。
|
||||
/api/ui/service-status 代理同机 fam-edge 的 /api/oracle/activity（浏览器不直连
|
||||
fam-edge，避免 token 暴露），走 edge_client 统一读 base_url/token。
|
||||
"""
|
||||
import re
|
||||
from datetime import datetime
|
||||
@@ -19,7 +18,7 @@ from flask import Blueprint, request, jsonify
|
||||
|
||||
from .logger import setup_logger
|
||||
from . import db_layer
|
||||
from .oracle_sync import get_sync
|
||||
from . import edge_client
|
||||
|
||||
logger = setup_logger('fam-core.ui_api')
|
||||
|
||||
@@ -50,7 +49,7 @@ def videos():
|
||||
date_filter = request.args.get('date') or None
|
||||
page = max(0, request.args.get('page', 0, type=int))
|
||||
page_size = 15
|
||||
rows = db_layer.get_sync_videos(limit=page_size, offset=page * page_size,
|
||||
rows = db_layer.get_videos(limit=page_size, offset=page * page_size,
|
||||
date_filter=date_filter)
|
||||
return jsonify({"videos": _ser(rows), "page": page, "page_size": page_size}), 200
|
||||
|
||||
@@ -58,10 +57,10 @@ def videos():
|
||||
@ui_bp.route('/api/ui/videos/<int:video_id>', methods=['GET'])
|
||||
def video_detail(video_id):
|
||||
"""单个视频会话详情 + 时间线事件列表（事件时间轴右侧）。"""
|
||||
video = db_layer.get_sync_video(video_id)
|
||||
video = db_layer.get_video(video_id)
|
||||
if not video:
|
||||
return jsonify({"error": "视频不存在"}), 404
|
||||
events = db_layer.get_sync_events_for_video(video_id)
|
||||
events = db_layer.get_events_for_video(video_id)
|
||||
return jsonify({"video": _ser(video), "events": _ser(events)}), 200
|
||||
|
||||
|
||||
@@ -69,16 +68,13 @@ def video_detail(video_id):
|
||||
def video_delete(video_id):
|
||||
"""删除视频会话（事件时间轴"删除"入口）。
|
||||
|
||||
先回推 Oracle 物理删除（events + videos 行 + 磁盘文件），成功后再清理本地
|
||||
MariaDB 镜像——增量同步（get_sync_delta）只做 upsert 感知不到删除，不能像
|
||||
命名纠错那样靠 trigger_now() 拉增量顺带清理，必须显式调 delete_sync_video。
|
||||
Oracle 回推失败时不清理本地镜像，避免"Oracle 还留着、NAS 却以为删了"的
|
||||
数据不一致，让用户看错误提示后重试。
|
||||
交给 fam-edge 处理：它删 events + videos 行，还要删磁盘上的运动片段文件。
|
||||
迁云前这里还要再清一次 NAS 本地镜像（增量同步只 upsert，感知不到删除），
|
||||
现在没有镜像了，fam-edge 删完就是最终状态。
|
||||
"""
|
||||
ok, err = get_sync().push_video_delete(video_id)
|
||||
ok, err = edge_client.push_video_delete(video_id)
|
||||
if not ok:
|
||||
return jsonify({"error": f"删除失败: {err}"}), 502
|
||||
db_layer.delete_sync_video(video_id)
|
||||
return jsonify({"status": "ok", "video_id": video_id}), 200
|
||||
|
||||
|
||||
@@ -86,7 +82,7 @@ def video_delete(video_id):
|
||||
def stats():
|
||||
"""统计卡：视频/事件/人物/关注数（可选按日期过滤）。"""
|
||||
date_filter = request.args.get('date') or None
|
||||
return jsonify(_ser(db_layer.get_sync_stats(date_filter))), 200
|
||||
return jsonify(_ser(db_layer.get_stats(date_filter))), 200
|
||||
|
||||
|
||||
def _clean_person(s: str) -> str:
|
||||
@@ -100,7 +96,7 @@ def people():
|
||||
|
||||
对应旧 Streamlit 版本 app.py 里的分组逻辑，原样搬到服务端。
|
||||
"""
|
||||
rows = db_layer.get_sync_people()
|
||||
rows = db_layer.get_people()
|
||||
groups = {}
|
||||
for p in rows:
|
||||
key = p.get('canonical_name') or p['label']
|
||||
@@ -128,7 +124,7 @@ def people():
|
||||
def people_clips():
|
||||
"""某人物出现过的运动片段列表（人物卡「运动片段」区块）。
|
||||
|
||||
按 label/canonical_name 匹配 sync_events.person_list_json → 关联视频
|
||||
按 label/canonical_name 匹配 events.person_list_json → 关联视频
|
||||
（运动片段优先）。返回片段 video_id/filename/event_start_time/duration/
|
||||
summary/camera_name + first_ts（缩略图定位）+ clip_events（片段内事件数）。
|
||||
"""
|
||||
@@ -136,7 +132,7 @@ def people_clips():
|
||||
if not label:
|
||||
return jsonify({"error": "缺少 label 参数"}), 400
|
||||
limit = min(20, request.args.get('limit', 10, type=int) or 10)
|
||||
clips = db_layer.get_sync_people_clips(label, limit)
|
||||
clips = db_layer.get_people_clips(label, limit)
|
||||
return jsonify({"label": label, "clips": _ser(clips)}), 200
|
||||
|
||||
|
||||
@@ -163,41 +159,25 @@ def attention_events():
|
||||
@ui_bp.route('/api/ui/named-members', methods=['GET'])
|
||||
def named_members():
|
||||
"""已命名成员真名列表（AI 对话页快捷选择）。"""
|
||||
return jsonify({"members": db_layer.get_sync_named_members()}), 200
|
||||
return jsonify({"members": db_layer.get_named_members()}), 200
|
||||
|
||||
|
||||
@ui_bp.route('/api/ui/model-stats', methods=['GET'])
|
||||
def model_stats():
|
||||
"""云端模型调用统计：按模型聚合 + 最近调用明细。"""
|
||||
agg = db_layer.get_sync_model_calls_stats()
|
||||
calls = db_layer.get_sync_model_calls(limit=100)
|
||||
agg = db_layer.get_model_calls_stats()
|
||||
calls = db_layer.get_model_calls(limit=100)
|
||||
return jsonify({"aggregate": _ser(agg), "recent_calls": _ser(calls)}), 200
|
||||
|
||||
|
||||
@ui_bp.route('/api/ui/service-status', methods=['GET'])
|
||||
def service_status():
|
||||
"""服务状态页：NAS 同步状态 + Oracle 实时活动（代理，token 不下发浏览器）。"""
|
||||
sync = get_sync()
|
||||
nas_status = sync.status()
|
||||
"""服务状态页：fam-edge 的队列/分割/模型活动（代理，token 不下发浏览器）。
|
||||
|
||||
oracle_data = None
|
||||
oracle_error = None
|
||||
if sync.base_url and sync.token:
|
||||
import requests
|
||||
try:
|
||||
r = requests.get(f"{sync.base_url}/api/oracle/activity",
|
||||
params={'token': sync.token}, timeout=15)
|
||||
if r.status_code == 200:
|
||||
oracle_data = r.json()
|
||||
else:
|
||||
oracle_error = f"Oracle activity HTTP {r.status_code}"
|
||||
except Exception as e:
|
||||
oracle_error = f"连接 Oracle 失败: {e}"
|
||||
else:
|
||||
oracle_error = "未配置 oracle_sync.base_url/token"
|
||||
|
||||
return jsonify({
|
||||
"nas_sync": nas_status,
|
||||
"oracle": oracle_data,
|
||||
"oracle_error": oracle_error,
|
||||
}), 200
|
||||
迁云前这里还有一块 "NAS 同步状态"（镜像拉取的游标/周期/上次条数），随镜像层
|
||||
一起删了——现在前端读的就是 fam-edge 写的那份库，没有"同步"这个中间状态。
|
||||
NAS 侧只剩运动事件推送，它的心跳在 fam-edge 的 service_activity 里，
|
||||
已经包含在 activity 快照中。
|
||||
"""
|
||||
data, err = edge_client.get_activity()
|
||||
return jsonify({"oracle": data, "oracle_error": err}), 200
|
||||
|
||||
@@ -1,299 +0,0 @@
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
|
||||
import jwt
|
||||
import pytest
|
||||
|
||||
from fam_core import auth
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_env_and_state(monkeypatch):
|
||||
"""每个用例前清掉环境变量和进程内状态，用例之间不互相污染。"""
|
||||
for var in ('AUTH_HUB_ISSUER', 'AUTH_HUB_CLIENT_ID',
|
||||
'AUTH_HUB_CLIENT_SECRET', 'AUTH_HUB_REDIRECT_URI'):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
auth._sessions.clear()
|
||||
auth._pending.clear()
|
||||
auth._warned_unconfigured = False
|
||||
auth._jwks_client = None
|
||||
yield
|
||||
auth._sessions.clear()
|
||||
auth._pending.clear()
|
||||
|
||||
|
||||
def _set_auth_hub_env(monkeypatch):
|
||||
monkeypatch.setenv('AUTH_HUB_ISSUER', 'http://auth-hub.example')
|
||||
monkeypatch.setenv('AUTH_HUB_CLIENT_ID', 'fam-core')
|
||||
monkeypatch.setenv('AUTH_HUB_CLIENT_SECRET', 'sekret')
|
||||
monkeypatch.setenv('AUTH_HUB_REDIRECT_URI', 'http://fam.example/api/auth/callback')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 配置齐全性校验（fail closed）
|
||||
# ---------------------------------------------------------------------------
|
||||
def test_require_auth_hub_config_fails_closed_when_unconfigured():
|
||||
assert auth._require_auth_hub_config() is None
|
||||
|
||||
|
||||
def test_require_auth_hub_config_fails_closed_when_partially_configured(monkeypatch):
|
||||
monkeypatch.setenv('AUTH_HUB_ISSUER', 'http://auth-hub.example')
|
||||
monkeypatch.setenv('AUTH_HUB_CLIENT_ID', 'fam-core')
|
||||
assert auth._require_auth_hub_config() is None
|
||||
|
||||
|
||||
def test_require_auth_hub_config_succeeds_when_fully_configured(monkeypatch):
|
||||
_set_auth_hub_env(monkeypatch)
|
||||
cfg = auth._require_auth_hub_config()
|
||||
assert cfg == {
|
||||
'issuer': 'http://auth-hub.example',
|
||||
'client_id': 'fam-core',
|
||||
'client_secret': 'sekret',
|
||||
'redirect_uri': 'http://fam.example/api/auth/callback',
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 本地会话（fam_session cookie）
|
||||
# ---------------------------------------------------------------------------
|
||||
def test_session_ttl_is_two_hours():
|
||||
assert auth._SESSION_TTL == 2 * 3600
|
||||
|
||||
|
||||
def test_is_authed_true_within_ttl():
|
||||
tok = 'sometoken'
|
||||
auth._sessions[tok] = time.time() + 3600
|
||||
|
||||
class _FakeRequest:
|
||||
cookies = {'fam_session': tok}
|
||||
monkeypatch_request = auth.request
|
||||
try:
|
||||
auth.request = _FakeRequest()
|
||||
assert auth.is_authed() is True
|
||||
finally:
|
||||
auth.request = monkeypatch_request
|
||||
|
||||
|
||||
def test_is_authed_false_after_expiry():
|
||||
tok = 'expiredtoken'
|
||||
auth._sessions[tok] = time.time() - 1 # 已过期
|
||||
|
||||
class _FakeRequest:
|
||||
cookies = {'fam_session': tok}
|
||||
monkeypatch_request = auth.request
|
||||
try:
|
||||
auth.request = _FakeRequest()
|
||||
assert auth.is_authed() is False
|
||||
# 过期后应该从 session 表里清掉，不留垃圾
|
||||
assert tok not in auth._sessions
|
||||
finally:
|
||||
auth.request = monkeypatch_request
|
||||
|
||||
|
||||
def test_is_authed_false_without_cookie():
|
||||
class _FakeRequest:
|
||||
cookies = {}
|
||||
monkeypatch_request = auth.request
|
||||
try:
|
||||
auth.request = _FakeRequest()
|
||||
assert auth.is_authed() is False
|
||||
finally:
|
||||
auth.request = monkeypatch_request
|
||||
|
||||
|
||||
def test_create_session_registers_token_with_ttl():
|
||||
before = time.time()
|
||||
tok = auth._create_session()
|
||||
assert tok in auth._sessions
|
||||
assert auth._sessions[tok] >= before + auth._SESSION_TTL
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 白名单
|
||||
# ---------------------------------------------------------------------------
|
||||
def test_whitelist_exact_paths():
|
||||
for path in ('/login', '/api/auth/callback', '/api/logout', '/api/auth/check',
|
||||
'/health', '/favicon.ico'):
|
||||
assert auth._is_whitelisted(path) is True
|
||||
|
||||
|
||||
def test_whitelist_assets_prefix():
|
||||
assert auth._is_whitelisted('/assets/index-abc123.js') is True
|
||||
|
||||
|
||||
def test_whitelist_rejects_protected_paths():
|
||||
for path in ('/', '/api/ui/people', '/api/chat/ask', '/api/ss/status'):
|
||||
assert auth._is_whitelisted(path) is False
|
||||
|
||||
|
||||
def test_whitelist_no_longer_includes_removed_password_login_endpoint():
|
||||
"""账号密码登录接口已随 SSO 改造下线，不应再出现在白名单里。"""
|
||||
assert auth._is_whitelisted('/api/login') is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 通过 Flask test_client 打完整流程
|
||||
# ---------------------------------------------------------------------------
|
||||
@pytest.fixture
|
||||
def app():
|
||||
from flask import Flask, jsonify as _jsonify
|
||||
app = Flask(__name__)
|
||||
app.register_blueprint(auth.auth_bp)
|
||||
|
||||
@app.route('/some-protected-page')
|
||||
def _protected_page():
|
||||
return 'ok'
|
||||
|
||||
@app.route('/api/protected')
|
||||
def _protected_api():
|
||||
return _jsonify({"ok": True})
|
||||
|
||||
auth.init_auth(app)
|
||||
return app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(app):
|
||||
return app.test_client()
|
||||
|
||||
|
||||
def test_login_redirects_to_auth_hub_authorize_with_pkce(client, monkeypatch):
|
||||
_set_auth_hub_env(monkeypatch)
|
||||
resp = client.get('/login')
|
||||
assert resp.status_code == 302
|
||||
location = resp.headers['Location']
|
||||
assert location.startswith('http://auth-hub.example/authorize?')
|
||||
assert 'code_challenge=' in location
|
||||
assert 'code_challenge_method=S256' in location
|
||||
assert 'client_id=fam-core' in location
|
||||
assert len(auth._pending) == 1
|
||||
|
||||
|
||||
def test_login_rejects_when_unconfigured(client):
|
||||
resp = client.get('/login')
|
||||
assert resp.status_code == 503
|
||||
|
||||
|
||||
def test_login_redirects_home_when_already_authed(client, monkeypatch):
|
||||
_set_auth_hub_env(monkeypatch)
|
||||
tok = auth._create_session()
|
||||
client.set_cookie('fam_session', tok)
|
||||
resp = client.get('/login')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/'
|
||||
|
||||
|
||||
def test_callback_rejects_unknown_or_expired_state(client, monkeypatch):
|
||||
_set_auth_hub_env(monkeypatch)
|
||||
resp = client.get('/api/auth/callback?state=nope&code=abc')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/login'
|
||||
assert 'fam_session' not in resp.headers.get('Set-Cookie', '')
|
||||
|
||||
|
||||
def test_callback_rejects_idp_error(client, monkeypatch):
|
||||
_set_auth_hub_env(monkeypatch)
|
||||
resp = client.get('/api/auth/callback?error=access_denied&state=x')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/login'
|
||||
|
||||
|
||||
def test_callback_exchanges_code_and_sets_session_cookie(client, monkeypatch):
|
||||
_set_auth_hub_env(monkeypatch)
|
||||
auth._pending['thestate'] = {'verifier': 'v', 'expires': time.time() + 600}
|
||||
|
||||
id_token = jwt.encode(
|
||||
{'iss': 'http://auth-hub.example', 'aud': 'fam-core', 'sub': '1',
|
||||
'preferred_username': 'ericwyuan', 'exp': time.time() + 300, 'iat': time.time()},
|
||||
'unused', algorithm='HS256') # 签名算法在这里不重要，被下面的 mock 绕过验签
|
||||
|
||||
class _FakeResp:
|
||||
status_code = 200
|
||||
text = ''
|
||||
def json(self):
|
||||
return {'id_token': id_token, 'access_token': 'x', 'token_type': 'Bearer'}
|
||||
|
||||
monkeypatch.setattr(auth.requests, 'post', lambda *a, **k: _FakeResp())
|
||||
|
||||
class _FakeSigningKey:
|
||||
key = 'unused'
|
||||
|
||||
class _FakeJwksClient:
|
||||
uri = 'http://auth-hub.example/.well-known/jwks.json'
|
||||
def get_signing_key_from_jwt(self, token):
|
||||
return _FakeSigningKey()
|
||||
|
||||
monkeypatch.setattr(auth, '_get_jwks_client', lambda uri: _FakeJwksClient())
|
||||
monkeypatch.setattr(auth.jwt, 'decode', lambda *a, **k: {
|
||||
'preferred_username': 'ericwyuan', 'sub': '1'})
|
||||
|
||||
resp = client.get('/api/auth/callback?state=thestate&code=abc')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/'
|
||||
assert 'fam_session=' in resp.headers['Set-Cookie']
|
||||
assert 'thestate' not in auth._pending
|
||||
assert len(auth._sessions) == 1
|
||||
|
||||
|
||||
def test_callback_rejects_when_token_exchange_fails(client, monkeypatch):
|
||||
_set_auth_hub_env(monkeypatch)
|
||||
auth._pending['thestate'] = {'verifier': 'v', 'expires': time.time() + 600}
|
||||
|
||||
class _FakeResp:
|
||||
status_code = 400
|
||||
text = 'invalid_grant'
|
||||
|
||||
monkeypatch.setattr(auth.requests, 'post', lambda *a, **k: _FakeResp())
|
||||
|
||||
resp = client.get('/api/auth/callback?state=thestate&code=abc')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/login'
|
||||
assert len(auth._sessions) == 0
|
||||
|
||||
|
||||
def test_callback_rejects_when_id_token_verification_fails(client, monkeypatch):
|
||||
_set_auth_hub_env(monkeypatch)
|
||||
auth._pending['thestate'] = {'verifier': 'v', 'expires': time.time() + 600}
|
||||
|
||||
class _FakeResp:
|
||||
status_code = 200
|
||||
text = ''
|
||||
def json(self):
|
||||
return {'id_token': 'bogus'}
|
||||
|
||||
monkeypatch.setattr(auth.requests, 'post', lambda *a, **k: _FakeResp())
|
||||
|
||||
def _boom(uri):
|
||||
raise jwt.PyJWTError("boom")
|
||||
monkeypatch.setattr(auth, '_get_jwks_client', _boom)
|
||||
|
||||
resp = client.get('/api/auth/callback?state=thestate&code=abc')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/login'
|
||||
assert len(auth._sessions) == 0
|
||||
|
||||
|
||||
def test_logout_clears_session(client):
|
||||
tok = auth._create_session()
|
||||
client.set_cookie('fam_session', tok)
|
||||
resp = client.post('/api/logout')
|
||||
assert resp.status_code == 200
|
||||
assert tok not in auth._sessions
|
||||
|
||||
|
||||
def test_before_request_guard_redirects_unauthed_page_to_login(client):
|
||||
resp = client.get('/some-protected-page')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/login'
|
||||
|
||||
|
||||
def test_before_request_guard_401s_unauthed_api(client):
|
||||
resp = client.get('/api/protected')
|
||||
assert resp.status_code == 401
|
||||
|
||||
|
||||
def test_before_request_guard_allows_authed_requests(client):
|
||||
tok = auth._create_session()
|
||||
client.set_cookie('fam_session', tok)
|
||||
assert client.get('/some-protected-page').status_code == 200
|
||||
assert client.get('/api/protected').status_code == 200
|
||||
155
fam-core/tests/test_db_layer.py
Normal file
155
fam-core/tests/test_db_layer.py
Normal file
@@ -0,0 +1,155 @@
|
||||
"""db_layer 直读 SQLite 的单测（2026-09-13 迁云重写后新增）。
|
||||
|
||||
重点验证从 MySQL 翻译过来的几处方言：json_each 取代 JSON_CONTAINS、
|
||||
substr 取代 LEFT、以及历史脏数据（person_list_json 不是合法 JSON）不能
|
||||
把整条查询搞崩——这在 MySQL 下 JSON_CONTAINS 会直接报错，SQLite 下
|
||||
json_each 同样会，所以查询里加了 json_valid 前置判断。
|
||||
"""
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
|
||||
from fam_core import db_layer
|
||||
|
||||
# fam-edge 建表语句的最小子集（列名与 oracle_db.py 保持一致）
|
||||
_SCHEMA = """
|
||||
CREATE TABLE videos (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT, filename TEXT UNIQUE, camera_name TEXT,
|
||||
duration_sec REAL, event_start_time TEXT, status TEXT DEFAULT 'pending',
|
||||
summary_json TEXT, events_json TEXT, people_json TEXT, compute_provider TEXT,
|
||||
created_at TEXT, updated_at TEXT, processed_at TEXT);
|
||||
CREATE TABLE events (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT, video_id INTEGER, ts TEXT, description TEXT,
|
||||
person_list_json TEXT, person_appearances_json TEXT, is_attention_event INTEGER DEFAULT 0);
|
||||
CREATE TABLE people (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT, label TEXT UNIQUE, canonical_name TEXT,
|
||||
first_seen TEXT, appearances INTEGER DEFAULT 0, source TEXT, features_json TEXT,
|
||||
display_uid TEXT, updated_at TEXT);
|
||||
CREATE TABLE model_calls (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT, provider TEXT, model TEXT, video_id INTEGER,
|
||||
filename TEXT, started_at TEXT, duration_sec REAL, success INTEGER, error TEXT,
|
||||
created_at TEXT);
|
||||
"""
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db(tmp_path, monkeypatch):
|
||||
path = tmp_path / "oracle.db"
|
||||
conn = sqlite3.connect(path)
|
||||
conn.executescript(_SCHEMA)
|
||||
conn.executescript("""
|
||||
INSERT INTO videos (id, filename, camera_name, event_start_time, status, processed_at)
|
||||
VALUES (1, 'motion_20260913_080000.mp4', '客厅', '2026-09-13 08:00:00', 'done', '2026-09-13 08:10:00'),
|
||||
(2, 'whole_20260912_090000.mp4', '客厅', '2026-09-12 09:00:00', 'done', '2026-09-12 09:10:00'),
|
||||
(3, 'whole_20260911_090000.mp4', '客厅', '2026-09-11 09:00:00', 'done', '2026-09-11 09:10:00'),
|
||||
(4, 'motion_20260910_070000.mp4', '客厅', '2026-09-10 07:00:00', 'pending', NULL);
|
||||
-- video 2 有事件（应展示），video 3 没有事件且不是 motion_（应被过滤掉）
|
||||
INSERT INTO events (video_id, ts, description, person_list_json, is_attention_event)
|
||||
VALUES (1, '00:00:05', '汤圆在客厅玩耍', '["汤圆"]', 0),
|
||||
(1, '00:00:20', '有人靠近门口', '["人物B"]', 1),
|
||||
(2, '00:01:00', '汤圆和奶奶', '["汤圆","奶奶"]', 0),
|
||||
(2, '00:02:00', '脏数据事件', '不是合法JSON', 0);
|
||||
INSERT INTO people (id, label, canonical_name, appearances, source)
|
||||
VALUES (1, '人物A', '汤圆', 12, 'manual'),
|
||||
(2, '人物B', NULL, 3, 'llm'),
|
||||
(3, '汤圆', '汤圆', 5, 'manual');
|
||||
INSERT INTO model_calls (provider, model, success, duration_sec, created_at)
|
||||
VALUES ('gemini', 'flash', 1, 10.0, '2026-09-13 08:10:00'),
|
||||
('gemini', 'flash', 0, 20.0, '2026-09-13 08:20:00'),
|
||||
('nvidia', 'vila', 1, 5.0, '2026-09-12 08:10:00');
|
||||
""")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
monkeypatch.setattr(db_layer, '_db_path', str(path))
|
||||
monkeypatch.setattr(db_layer, '_chat_schema_ready', False)
|
||||
return path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- videos
|
||||
def test_get_videos_only_returns_content_sessions(db):
|
||||
"""只展示运动片段或含事件的会话：分割 0 段的整段素材不该淹没时间轴。"""
|
||||
ids = [v['id'] for v in db_layer.get_videos()]
|
||||
assert ids == [1, 2] # 3 无事件且非 motion_，4 未完成
|
||||
assert db_layer.get_videos()[0]['event_count'] == 2
|
||||
|
||||
|
||||
def test_get_videos_date_filter_and_paging(db):
|
||||
assert [v['id'] for v in db_layer.get_videos(date_filter='2026-09-12')] == [2]
|
||||
assert [v['id'] for v in db_layer.get_videos(limit=1, offset=1)] == [2]
|
||||
|
||||
|
||||
def test_get_video_and_events(db):
|
||||
assert db_layer.get_video(1)['filename'].startswith('motion_')
|
||||
assert db_layer.get_video(999) is None
|
||||
evs = db_layer.get_events_for_video(1)
|
||||
assert [e['ts'] for e in evs] == ['00:00:05', '00:00:20']
|
||||
|
||||
|
||||
def test_get_attention_events(db):
|
||||
rows = db_layer.get_attention_events()
|
||||
assert len(rows) == 1 and rows[0]['ev_date'].startswith('2026-09-13')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 人物命中（JSON）
|
||||
def test_people_clips_matches_label_and_its_aliases(db):
|
||||
"""'人物A' 的规范名是"汤圆"，而事件里记的是"汤圆"——要能顺着别名找到。"""
|
||||
clips = db_layer.get_people_clips('人物A')
|
||||
assert {c['video_id'] for c in clips} == {1, 2}
|
||||
assert clips[0]['video_id'] == 1 # 按录制时间倒序
|
||||
assert clips[0]['first_ts'] == '00:00:05'
|
||||
assert clips[0]['clip_events'] == 1
|
||||
|
||||
|
||||
def test_person_hit_is_exact_not_substring(db):
|
||||
"""json_each 是精确匹配：查"人物"不该命中"人物B"。"""
|
||||
assert db_layer.get_people_clips('人物') == []
|
||||
assert {c['video_id'] for c in db_layer.get_people_clips('人物B')} == {1}
|
||||
|
||||
|
||||
def test_malformed_person_json_does_not_break_queries(db):
|
||||
"""video 2 里混了一条非 JSON 的脏数据，查询必须照常返回而不是整条报错。"""
|
||||
rows = db_layer.query_events_for_person_date('汤圆', '2026-09-12')
|
||||
assert len(rows) == 1 and rows[0]['ts'] == '00:01:00'
|
||||
|
||||
|
||||
def test_query_events_for_person_date(db):
|
||||
assert db_layer.query_events_for_person_date('汤圆', '2026-09-13')[0]['description'] == '汤圆在客厅玩耍'
|
||||
assert db_layer.query_events_for_person_date('汤圆', '2026-09-01') == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- people / model_calls
|
||||
def test_people_and_named_members(db):
|
||||
assert len(db_layer.get_people()) == 3
|
||||
assert db_layer.get_named_members() == ['汤圆'] # distinct 且非空
|
||||
ctx = db_layer.get_known_members_context()
|
||||
assert '- 汤圆（标识：人物A）' in ctx and '- 人物B' in ctx
|
||||
|
||||
|
||||
def test_model_calls_stats(db):
|
||||
stats = {s['model']: s for s in db_layer.get_model_calls_stats()}
|
||||
assert stats['flash']['ok_cnt'] == 1 and stats['flash']['fail_cnt'] == 1
|
||||
assert stats['flash']['avg_duration'] == 15.0
|
||||
assert len(db_layer.get_model_calls(limit=2)) == 2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 统计
|
||||
def test_stats_全量与按日(db):
|
||||
total = db_layer.get_stats()
|
||||
assert total['videos'] == 2 and total['events'] == 4 and total['attention'] == 1
|
||||
day = db_layer.get_stats('2026-09-13')
|
||||
assert day['videos'] == 1 and day['events'] == 2
|
||||
# 人物数：汤圆（含 label 人物A/汤圆两行都归一到"汤圆"）+ 人物B
|
||||
assert total['people'] >= 2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- chat_history
|
||||
def test_chat_history_roundtrip_creates_table_on_demand(db):
|
||||
"""chat_history 是本模块唯一写的表，建表是懒加载的（库文件属于 fam-edge）。"""
|
||||
chat_id = db_layer.insert_chat_history('今天有人来吗', '有，08:00 有人靠近门口',
|
||||
'上下文', '2026-09-13', '汤圆')
|
||||
assert chat_id == 1
|
||||
rows = db_layer.get_chat_history(limit=10)
|
||||
assert rows[0]['user_question'] == '今天有人来吗'
|
||||
assert rows[0]['created_at']
|
||||
assert db_layer.get_chat_history(person_filter='不存在的人') == []
|
||||
assert len(db_layer.get_chat_history(date_filter='2026-09-13')) == 1
|
||||
@@ -2,6 +2,9 @@ flask>=2.0.0
|
||||
gunicorn>=20.0.0
|
||||
requests>=2.28.0
|
||||
PyYAML>=6.0
|
||||
# 统一登录：验 auth-hub 的 id_token 签名（RS256）+ 签发会话 cookie（HS256）
|
||||
PyJWT>=2.8.0
|
||||
cryptography>=42.0.0
|
||||
# google-generativeai 和 openai 为可选依赖（代码用 requests 直接调 REST API）
|
||||
# 如需 SDK 方式调用，取消注释并在 Python 3.9+ 环境安装:
|
||||
# google-generativeai>=0.5.0
|
||||
|
||||
@@ -270,9 +270,16 @@ def activity():
|
||||
disk_info["free_gb"] = round(_shutil.disk_usage(watch_path).free / (1024 ** 3), 1)
|
||||
except Exception as e:
|
||||
logger.warning(f"磁盘空间查询异常: {e}")
|
||||
# NAS 推送链路（2026-09-13 起 NAS 上只剩 fam-notifier 这一个服务，
|
||||
# 服务状态页的「NAS 运动推送」卡片读这里；心跳由它定期空 POST 刷新）
|
||||
motion = {
|
||||
"heartbeat_age_sec": db.get_motion_heartbeat_age_sec(),
|
||||
"recent": db.get_recent_motion_events(5),
|
||||
}
|
||||
return jsonify({
|
||||
"queue": q_status,
|
||||
"db": db.get_queue_status(),
|
||||
"motion": motion,
|
||||
"segment": segment,
|
||||
"disk": disk_info,
|
||||
"rclone": _last_activity('rclone'),
|
||||
|
||||
@@ -6,6 +6,8 @@ FAM-Edge 主应用 - Flask 单进程（新架构 v2.1）
|
||||
- VideoQueue（生产-消费队列：同步落地新视频入队，独立消费者云端分析，模型超时 ×2）
|
||||
- PersonService（人物汇总合并，定时）
|
||||
- DiskGuard（磁盘空间守护，定时检查，剩余空间过低时清理最旧的已完成素材）
|
||||
- Auth（统一登录：/login + OIDC 回调 + 给 Caddy forward_auth 用的会话校验，
|
||||
2026-09-12 从 NAS fam-core 迁入，原因见 auth.py 开头）
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
@@ -16,6 +18,7 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from .config_loader import load_config
|
||||
from .logger import setup_logger
|
||||
from .api_gateway.api_gateway import api_bp
|
||||
from .auth import auth_bp
|
||||
from . import state
|
||||
from .video_queue import VideoQueue
|
||||
from .person_service import PersonService
|
||||
@@ -25,6 +28,7 @@ logger = setup_logger('fam-edge.app')
|
||||
|
||||
app = Flask(__name__)
|
||||
app.register_blueprint(api_bp)
|
||||
app.register_blueprint(auth_bp)
|
||||
|
||||
|
||||
@app.route('/', methods=['GET'])
|
||||
|
||||
285
fam-edge/src/fam_edge/auth.py
Normal file
285
fam-edge/src/fam_edge/auth.py
Normal file
@@ -0,0 +1,285 @@
|
||||
"""
|
||||
Auth - 摄像头系统统一登录（2026-09-12 从 NAS fam-core 迁到甲骨文 fam-edge）
|
||||
|
||||
背景：登录流程原先跑在 NAS 的 fam-core 里，NAS 一挂或 frp 隧道一断，
|
||||
`smart-camera.zichuan.xyz/login` 直接 502——连登录页都进不去。登录是入口，
|
||||
不该依赖家里那台机器，所以整体搬到甲骨文：前端静态文件和 auth-hub 本来就在这台，
|
||||
搬完之后 NAS 只剩纯数据接口。
|
||||
|
||||
跟旧实现（fam-core/auth.py，已删除）的三处关键差异：
|
||||
|
||||
1. **服务端调用 auth-hub 走本机**：换 token 和拉 JWKS 用 `AUTH_HUB_INTERNAL_BASE`
|
||||
（默认 http://127.0.0.1:5300），不再走公网 TLS。旧链路是 NAS 跨公网访问
|
||||
https://auth.zichuan.xyz，踩过两个坑：`jwt.PyJWKClient` 用 urllib + 系统 CA
|
||||
（不像 requests 自带 certifi），群晖上很容易 CERTIFICATE_VERIFY_FAILED；
|
||||
以及两台机器时钟偏差会让 id_token 的 iat 看起来"来自未来"。
|
||||
但 `iss` 校验和浏览器跳转仍然用公网 `AUTH_HUB_ISSUER`——id_token 里的 iss
|
||||
是 auth-hub 自己配的公网地址，浏览器也只能跳公网地址。
|
||||
|
||||
2. **登录态改成无状态签名 cookie**：HS256 签发，密钥 `FAM_SESSION_SECRET`。
|
||||
旧实现是进程内 token 表，服务一重启所有人被踢下线；fam-edge 是视频分析进程，
|
||||
重启比 fam-core 更频繁，进程内会话在这里完全不成立。
|
||||
|
||||
3. **回调失败渲染错误页，不再 302 回 /login**：旧实现每个失败分支都跳 /login，
|
||||
而 auth-hub 侧只要还有会话，/authorize 会立刻再签发一个 code 跳回来，
|
||||
两边对跳成死循环——浏览器只报"重定向次数过多"，既看不到登录页也看不到原因。
|
||||
|
||||
拦截不在本模块做：Caddy 用 forward_auth 打到 `/api/auth/verify`，
|
||||
校验通过才把 `/api/*` 反代到 NAS 的 fam-core。
|
||||
"""
|
||||
import os
|
||||
import secrets
|
||||
import time
|
||||
from base64 import urlsafe_b64encode
|
||||
from hashlib import sha256
|
||||
from urllib.parse import urlencode
|
||||
|
||||
import jwt
|
||||
import requests
|
||||
from flask import Blueprint, Response, jsonify, make_response, redirect, request
|
||||
|
||||
# 导入即加载 /opt/fam-edge/.env（config_loader 模块级 _load_env_file），
|
||||
# 下面所有 os.environ.get 才读得到 AUTH_HUB_* / FAM_SESSION_SECRET。
|
||||
from . import config_loader # noqa: F401
|
||||
from .logger import setup_logger
|
||||
|
||||
logger = setup_logger('fam-edge.auth')
|
||||
|
||||
auth_bp = Blueprint('auth', __name__)
|
||||
|
||||
COOKIE_NAME = 'fam_session'
|
||||
_SESSION_TTL = 2 * 3600 # 登录态有效期 2 小时
|
||||
_PENDING_TTL = 10 * 60 # /login -> /api/auth/callback 之间的等待上限
|
||||
_LEEWAY = 60 # 验 id_token 时容忍的跨机器时钟偏差（秒）
|
||||
|
||||
_pending = {} # state -> {verifier, expires}（进程内，PKCE 用）
|
||||
_warned_unconfigured = False # 只在第一次拒绝登录时打一条警告，别刷屏
|
||||
_jwks_client = None # jwt.PyJWKClient 单例，内建 JWKS 缓存
|
||||
|
||||
|
||||
def _config():
|
||||
"""接入参数全部来自环境变量（/opt/fam-edge/.env）。"""
|
||||
issuer = os.environ.get('AUTH_HUB_ISSUER', '').rstrip('/')
|
||||
return {
|
||||
'issuer': issuer,
|
||||
# 服务端直连 auth-hub 的地址；没配就退回公网 issuer（本地开发用）
|
||||
'internal_base': (os.environ.get('AUTH_HUB_INTERNAL_BASE', '') or issuer).rstrip('/'),
|
||||
'client_id': os.environ.get('AUTH_HUB_CLIENT_ID', ''),
|
||||
'client_secret': os.environ.get('AUTH_HUB_CLIENT_SECRET', ''),
|
||||
'redirect_uri': os.environ.get('AUTH_HUB_REDIRECT_URI', ''),
|
||||
'session_secret': os.environ.get('FAM_SESSION_SECRET', ''),
|
||||
}
|
||||
|
||||
|
||||
def _require_config():
|
||||
"""五个变量缺任何一个都拒绝登录（fail closed）。返回配置 dict 或 None。
|
||||
|
||||
本服务监听 0.0.0.0，没有"配置不全就退回某种默认放行"这种兜底。
|
||||
"""
|
||||
global _warned_unconfigured
|
||||
cfg = _config()
|
||||
if not all([cfg['issuer'], cfg['client_id'], cfg['client_secret'],
|
||||
cfg['redirect_uri'], cfg['session_secret']]):
|
||||
if not _warned_unconfigured:
|
||||
logger.error(
|
||||
"AUTH_HUB_ISSUER/CLIENT_ID/CLIENT_SECRET/REDIRECT_URI 或 "
|
||||
"FAM_SESSION_SECRET 未配置齐全，拒绝所有登录——"
|
||||
"请在 /opt/fam-edge/.env 里补齐后 systemctl restart fam-edge")
|
||||
_warned_unconfigured = True
|
||||
return None
|
||||
return cfg
|
||||
|
||||
|
||||
def _get_jwks_client(jwks_uri):
|
||||
global _jwks_client
|
||||
if _jwks_client is None or _jwks_client.uri != jwks_uri:
|
||||
_jwks_client = jwt.PyJWKClient(jwks_uri)
|
||||
return _jwks_client
|
||||
|
||||
|
||||
def _cleanup_pending():
|
||||
now = time.time()
|
||||
for state in [s for s, v in _pending.items() if v['expires'] < now]:
|
||||
_pending.pop(state, None)
|
||||
|
||||
|
||||
def _is_https():
|
||||
"""Caddy 终止 TLS 后回源是明文 HTTP，request.is_secure 恒为 False，
|
||||
所以 cookie 的 Secure 标志要看 X-Forwarded-Proto。"""
|
||||
return request.headers.get('X-Forwarded-Proto', '') == 'https' or request.is_secure
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 会话（无状态签名 cookie）
|
||||
# ---------------------------------------------------------------------------
|
||||
def _issue_session(secret, sub, username):
|
||||
now = int(time.time())
|
||||
return jwt.encode({'sub': sub, 'username': username,
|
||||
'iat': now, 'exp': now + _SESSION_TTL},
|
||||
secret, algorithm='HS256')
|
||||
|
||||
|
||||
def current_user():
|
||||
"""返回 cookie 里的用户信息（dict），未登录/过期/签名不对返回 None。"""
|
||||
cfg = _config()
|
||||
if not cfg['session_secret']:
|
||||
return None
|
||||
token = request.cookies.get(COOKIE_NAME, '')
|
||||
if not token:
|
||||
return None
|
||||
try:
|
||||
# 这里刻意不给 leeway：cookie 是本进程自签自验的，没有跨机器时钟偏差问题，
|
||||
# 加了只会让每个会话白白多活 60 秒。_LEEWAY 只用于 auth-hub 签发的 id_token。
|
||||
return jwt.decode(token, cfg['session_secret'], algorithms=['HS256'])
|
||||
except jwt.PyJWTError:
|
||||
return None
|
||||
|
||||
|
||||
def is_authed():
|
||||
return current_user() is not None
|
||||
|
||||
|
||||
def _error_page(reason, status=502):
|
||||
"""回调失败时给一个看得懂的页面。
|
||||
|
||||
这里刻意不做自动跳转：auth-hub 有会话时会立刻再发一个 code 回来，
|
||||
自动跳转等于把用户关进死循环。让用户自己点"重新登录"，最多再失败一次。
|
||||
"""
|
||||
html = (
|
||||
'<!doctype html><html lang="zh-CN"><head><meta charset="utf-8">'
|
||||
'<meta name="viewport" content="width=device-width,initial-scale=1">'
|
||||
'<title>登录失败</title></head>'
|
||||
'<body style="font-family:system-ui,-apple-system,sans-serif;background:#0b0e14;'
|
||||
'color:#e6ebf2;display:flex;min-height:100vh;align-items:center;justify-content:center;margin:0">'
|
||||
'<div style="max-width:34rem;padding:2rem">'
|
||||
'<h1 style="font-size:1.25rem;margin:0 0 .75rem">登录没能完成</h1>'
|
||||
'<p style="color:#9aa7b8;line-height:1.7;margin:0 0 1.5rem">原因：{reason}</p>'
|
||||
'<a href="/login" style="display:inline-block;background:#5b8cff;color:#fff;'
|
||||
'text-decoration:none;padding:.6rem 1.2rem;border-radius:.6rem">重新登录</a>'
|
||||
'</div></body></html>'
|
||||
).format(reason=reason)
|
||||
return Response(html, status=status, mimetype='text/html; charset=utf-8')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 端点
|
||||
# ---------------------------------------------------------------------------
|
||||
@auth_bp.route('/login', methods=['GET'])
|
||||
def login():
|
||||
"""不渲染登录表单，直接跳 auth-hub 走 Authorization Code + PKCE。"""
|
||||
if is_authed():
|
||||
return redirect('/')
|
||||
|
||||
cfg = _require_config()
|
||||
if not cfg:
|
||||
return _error_page('本服务的登录参数没配齐（管理员看 fam-edge 日志）', 503)
|
||||
|
||||
_cleanup_pending()
|
||||
verifier = secrets.token_urlsafe(48)
|
||||
challenge = urlsafe_b64encode(
|
||||
sha256(verifier.encode('ascii')).digest()).rstrip(b'=').decode('ascii')
|
||||
state = secrets.token_hex(24)
|
||||
_pending[state] = {'verifier': verifier, 'expires': time.time() + _PENDING_TTL}
|
||||
|
||||
params = {
|
||||
'response_type': 'code',
|
||||
'client_id': cfg['client_id'],
|
||||
'redirect_uri': cfg['redirect_uri'],
|
||||
'scope': 'openid profile',
|
||||
'state': state,
|
||||
'code_challenge': challenge,
|
||||
'code_challenge_method': 'S256',
|
||||
}
|
||||
# 浏览器要跳的是公网 issuer，不是 internal_base
|
||||
return redirect("{}/authorize?{}".format(cfg['issuer'], urlencode(params)))
|
||||
|
||||
|
||||
@auth_bp.route('/api/auth/callback', methods=['GET'])
|
||||
def callback():
|
||||
cfg = _require_config()
|
||||
if not cfg:
|
||||
return _error_page('本服务的登录参数没配齐（管理员看 fam-edge 日志）', 503)
|
||||
|
||||
if request.args.get('error'):
|
||||
logger.warning("auth-hub 拒绝授权: {}".format(request.args.get('error')))
|
||||
return _error_page('auth-hub 拒绝了这次授权：{}'.format(request.args.get('error')), 403)
|
||||
|
||||
state = request.args.get('state', '')
|
||||
code = request.args.get('code', '')
|
||||
pending = _pending.pop(state, None)
|
||||
if not pending or pending['expires'] < time.time() or not code:
|
||||
logger.warning("回调 state 缺失/过期/重放，拒绝")
|
||||
return _error_page('这次登录请求已过期或被重复使用，请重新登录', 400)
|
||||
|
||||
try:
|
||||
resp = requests.post(
|
||||
"{}/token".format(cfg['internal_base']),
|
||||
data={
|
||||
'grant_type': 'authorization_code',
|
||||
'code': code,
|
||||
'redirect_uri': cfg['redirect_uri'],
|
||||
'client_id': cfg['client_id'],
|
||||
'client_secret': cfg['client_secret'],
|
||||
'code_verifier': pending['verifier'],
|
||||
}, timeout=(5, 15))
|
||||
except requests.RequestException as e:
|
||||
logger.error("auth-hub /token 请求失败: {}".format(e))
|
||||
return _error_page('连不上 auth-hub 的 /token（{}）'.format(e))
|
||||
|
||||
if resp.status_code != 200:
|
||||
logger.warning("auth-hub /token 拒绝: {} {}".format(resp.status_code, resp.text[:200]))
|
||||
return _error_page('auth-hub 拒绝换发 token（HTTP {}）'.format(resp.status_code))
|
||||
|
||||
id_token = (resp.json() or {}).get('id_token', '')
|
||||
try:
|
||||
jwks_client = _get_jwks_client(
|
||||
"{}/.well-known/jwks.json".format(cfg['internal_base']))
|
||||
signing_key = jwks_client.get_signing_key_from_jwt(id_token)
|
||||
claims = jwt.decode(id_token, signing_key.key, algorithms=['RS256'],
|
||||
audience=cfg['client_id'], issuer=cfg['issuer'],
|
||||
leeway=_LEEWAY)
|
||||
except jwt.PyJWTError as e:
|
||||
logger.warning("id_token 验签/校验失败: {}".format(e))
|
||||
return _error_page('id_token 校验失败（{}）'.format(e))
|
||||
|
||||
username = claims.get('preferred_username', '')
|
||||
token = _issue_session(cfg['session_secret'], claims.get('sub', ''), username)
|
||||
resp2 = make_response(redirect('/'))
|
||||
resp2.set_cookie(COOKIE_NAME, token, max_age=_SESSION_TTL, httponly=True,
|
||||
samesite='Lax', secure=_is_https(), path='/')
|
||||
logger.info("登录成功 username={}".format(username))
|
||||
return resp2
|
||||
|
||||
|
||||
@auth_bp.route('/api/logout', methods=['POST'])
|
||||
def logout():
|
||||
"""只清本地 cookie，不动 auth-hub 上的登录态（那是全站 SSO 会话）。
|
||||
|
||||
cookie 是无状态签名的，服务端没法单独作废某一张——真要立刻全员下线，
|
||||
换掉 FAM_SESSION_SECRET 重启即可。
|
||||
"""
|
||||
resp = make_response(jsonify({"ok": True}))
|
||||
resp.delete_cookie(COOKIE_NAME, path='/')
|
||||
return resp
|
||||
|
||||
|
||||
@auth_bp.route('/api/auth/check', methods=['GET'])
|
||||
def check():
|
||||
user = current_user()
|
||||
return jsonify({"authed": user is not None,
|
||||
"username": (user or {}).get('username', '')})
|
||||
|
||||
|
||||
@auth_bp.route('/api/auth/verify', methods=['GET', 'POST', 'PUT', 'DELETE', 'PATCH'])
|
||||
def verify():
|
||||
"""给 Caddy forward_auth 用：2xx 放行，401 拦截。
|
||||
|
||||
Caddy 会把原始请求的 cookie 一起带过来，401 的响应体会原样回给浏览器，
|
||||
前端 api.js 看到 401 就会自动跳 /login。
|
||||
"""
|
||||
user = current_user()
|
||||
if not user:
|
||||
return jsonify({"error": "未登录", "code": 401}), 401
|
||||
resp = make_response('', 204)
|
||||
resp.headers['X-Auth-User'] = user.get('username', '')
|
||||
return resp
|
||||
@@ -33,13 +33,42 @@ class OracleDB:
|
||||
def __init__(self, db_path: str):
|
||||
os.makedirs(os.path.dirname(db_path), exist_ok=True)
|
||||
self.db_path = db_path
|
||||
self._conn = sqlite3.connect(db_path, check_same_thread=False)
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||
self._conn.execute("PRAGMA busy_timeout=10000")
|
||||
self._local = threading.local() # 每线程一条连接，见下面的 _conn
|
||||
self._all_conns = [] # 仅供 close() 收尾
|
||||
self._conns_lock = threading.Lock()
|
||||
self._write_lock = threading.Lock() # 复合写（如 DELETE+INSERT+commit）串行化
|
||||
self._init_schema()
|
||||
|
||||
@property
|
||||
def _conn(self) -> sqlite3.Connection:
|
||||
"""当前线程的连接（2026-09-13 从"全进程共用一条"改成每线程一条）。
|
||||
|
||||
原来是 __init__ 里建一条 `check_same_thread=False` 的连接给所有线程共用：
|
||||
VideoQueue / PersonService / DiskGuard 三个后台线程，加上 gunicorn 的请求
|
||||
线程，并发读写同一个连接对象，事务状态互相踩踏。线上长期刷三类报错，
|
||||
全是这一个根因：
|
||||
|
||||
- `cannot start a transaction within a transaction`：一个线程的事务还
|
||||
开着，另一个线程又要开——DiskGuard 的清理被打断 2837 次，磁盘守护基本
|
||||
靠运气生效（表现为剩余空间在 2.7G 和 16G 之间来回荡）
|
||||
- `no more rows available`：commit 时游标已被别的线程重置，87 次
|
||||
- `database is locked`：每小时上百次，NAS 推来的运动事件被 500 打回
|
||||
|
||||
sqlite3 的连接本来就不是可并发共享的对象。改成各线程各拿一条之后：WAL 下
|
||||
多连接读不互斥，写由 SQLite 自己排队（busy_timeout 兜底等 10 秒），而
|
||||
`_write_lock` 继续保证"复合写"在本进程内串行，语义不变。
|
||||
"""
|
||||
conn = getattr(self._local, 'conn', None)
|
||||
if conn is None:
|
||||
conn = sqlite3.connect(self.db_path, timeout=10, check_same_thread=False)
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
conn.execute("PRAGMA busy_timeout=10000")
|
||||
self._local.conn = conn
|
||||
with self._conns_lock:
|
||||
self._all_conns.append(conn)
|
||||
return conn
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
def _init_schema(self):
|
||||
c = self._conn
|
||||
@@ -1078,4 +1107,12 @@ class OracleDB:
|
||||
self._conn.commit()
|
||||
|
||||
def close(self):
|
||||
self._conn.close()
|
||||
"""关掉所有线程开过的连接（不只当前线程这一条）。"""
|
||||
with self._conns_lock:
|
||||
conns, self._all_conns = self._all_conns, []
|
||||
for c in conns:
|
||||
try:
|
||||
c.close()
|
||||
except sqlite3.Error:
|
||||
pass
|
||||
self._local = threading.local()
|
||||
|
||||
@@ -84,7 +84,15 @@ class VideoQueue:
|
||||
if row is None:
|
||||
# 新文件：先过 mtime 稳定窗口 + 可解码校验，通过才登记入队；失败标记 invalid
|
||||
if self.file_validate:
|
||||
if time.time() - os.path.getmtime(path) < self.stable_window_sec:
|
||||
try:
|
||||
mtime = os.path.getmtime(path)
|
||||
except OSError:
|
||||
# 列目录之后、读 mtime 之前，DiskGuard 可能刚好把这个文件清掉了
|
||||
# （两个后台线程的正常竞态）。跳过它就行——不 catch 的话整轮扫描
|
||||
# 会被这一个文件中断，后面的新素材本轮都登记不上。
|
||||
logger.info(f"文件已不在（可能刚被 DiskGuard 清理），跳过本轮: {fn}")
|
||||
continue
|
||||
if time.time() - mtime < self.stable_window_sec:
|
||||
logger.info(f"文件仍在写入（mtime 未稳定），跳过本轮: {fn}")
|
||||
continue
|
||||
ok, verr, vmeta = validate_video(path)
|
||||
|
||||
241
fam-edge/tests/test_auth.py
Normal file
241
fam-edge/tests/test_auth.py
Normal file
@@ -0,0 +1,241 @@
|
||||
"""统一登录（2026-09-12 从 NAS fam-core 迁来）的单测。
|
||||
|
||||
除了把原 fam-core/tests/test_auth.py 的用例搬过来，额外盯死三件迁移时的关键行为：
|
||||
回调失败不准再 302（防死循环复发）、服务端调 auth-hub 走内网地址但 iss 仍按公网校验、
|
||||
会话 cookie 必须是无状态的（不依赖任何进程内状态）。
|
||||
"""
|
||||
import time
|
||||
|
||||
import jwt
|
||||
import pytest
|
||||
from flask import Flask
|
||||
|
||||
from fam_edge import auth
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_env_and_state(monkeypatch):
|
||||
for var in ('AUTH_HUB_ISSUER', 'AUTH_HUB_INTERNAL_BASE', 'AUTH_HUB_CLIENT_ID',
|
||||
'AUTH_HUB_CLIENT_SECRET', 'AUTH_HUB_REDIRECT_URI', 'FAM_SESSION_SECRET'):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
auth._pending.clear()
|
||||
auth._warned_unconfigured = False
|
||||
auth._jwks_client = None
|
||||
yield
|
||||
auth._pending.clear()
|
||||
|
||||
|
||||
def _set_env(monkeypatch, internal_base=None):
|
||||
monkeypatch.setenv('AUTH_HUB_ISSUER', 'https://auth.example')
|
||||
monkeypatch.setenv('AUTH_HUB_CLIENT_ID', 'fam-core')
|
||||
monkeypatch.setenv('AUTH_HUB_CLIENT_SECRET', 'sekret')
|
||||
monkeypatch.setenv('AUTH_HUB_REDIRECT_URI', 'https://cam.example/api/auth/callback')
|
||||
monkeypatch.setenv('FAM_SESSION_SECRET', 'session-signing-secret')
|
||||
if internal_base:
|
||||
monkeypatch.setenv('AUTH_HUB_INTERNAL_BASE', internal_base)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client():
|
||||
app = Flask(__name__)
|
||||
app.register_blueprint(auth.auth_bp)
|
||||
return app.test_client()
|
||||
|
||||
|
||||
def _cookie(secret='session-signing-secret', username='ericwyuan', ttl=3600):
|
||||
now = int(time.time())
|
||||
return jwt.encode({'sub': '1', 'username': username, 'iat': now, 'exp': now + ttl},
|
||||
secret, algorithm='HS256')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 配置齐全性（fail closed）
|
||||
# ---------------------------------------------------------------------------
|
||||
def test_require_config_fails_closed_when_unconfigured():
|
||||
assert auth._require_config() is None
|
||||
|
||||
|
||||
def test_require_config_fails_closed_without_session_secret(monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
monkeypatch.delenv('FAM_SESSION_SECRET')
|
||||
assert auth._require_config() is None
|
||||
|
||||
|
||||
def test_login_rejects_when_unconfigured(client):
|
||||
resp = client.get('/login')
|
||||
assert resp.status_code == 503
|
||||
# 配置缺失也不能跳转，否则同样会跟 auth-hub 对跳
|
||||
assert 'Location' not in resp.headers
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /login
|
||||
# ---------------------------------------------------------------------------
|
||||
def test_login_redirects_to_authorize_with_pkce(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
resp = client.get('/login')
|
||||
assert resp.status_code == 302
|
||||
location = resp.headers['Location']
|
||||
assert location.startswith('https://auth.example/authorize?')
|
||||
assert 'code_challenge=' in location
|
||||
assert 'code_challenge_method=S256' in location
|
||||
assert 'client_id=fam-core' in location
|
||||
assert len(auth._pending) == 1
|
||||
|
||||
|
||||
def test_login_always_sends_browser_to_public_issuer(client, monkeypatch):
|
||||
"""内网地址只给服务端自己用，浏览器必须跳公网——跳 127.0.0.1 用户当然打不开。"""
|
||||
_set_env(monkeypatch, internal_base='http://127.0.0.1:5300')
|
||||
resp = client.get('/login')
|
||||
assert resp.headers['Location'].startswith('https://auth.example/authorize?')
|
||||
|
||||
|
||||
def test_login_redirects_home_when_already_authed(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
client.set_cookie('fam_session', _cookie())
|
||||
resp = client.get('/login')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/'
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 回调：失败分支一律错误页，绝不再跳 /login（旧实现死循环的根因）
|
||||
# ---------------------------------------------------------------------------
|
||||
def test_callback_unknown_state_shows_error_page_without_redirecting(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
resp = client.get('/api/auth/callback?state=nope&code=abc')
|
||||
assert resp.status_code == 400
|
||||
assert 'Location' not in resp.headers
|
||||
assert 'fam_session' not in resp.headers.get('Set-Cookie', '')
|
||||
|
||||
|
||||
def test_callback_idp_error_shows_error_page_without_redirecting(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
resp = client.get('/api/auth/callback?error=access_denied&state=x')
|
||||
assert resp.status_code == 403
|
||||
assert 'Location' not in resp.headers
|
||||
|
||||
|
||||
def test_callback_token_failure_shows_error_page_without_redirecting(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
auth._pending['thestate'] = {'verifier': 'v', 'expires': time.time() + 600}
|
||||
|
||||
class _Resp:
|
||||
status_code = 400
|
||||
text = 'invalid_grant'
|
||||
monkeypatch.setattr(auth.requests, 'post', lambda *a, **k: _Resp())
|
||||
|
||||
resp = client.get('/api/auth/callback?state=thestate&code=abc')
|
||||
assert resp.status_code == 502
|
||||
assert 'Location' not in resp.headers
|
||||
assert 'fam_session' not in resp.headers.get('Set-Cookie', '')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 回调：成功路径
|
||||
# ---------------------------------------------------------------------------
|
||||
def _stub_successful_exchange(monkeypatch, captured):
|
||||
class _Resp:
|
||||
status_code = 200
|
||||
text = ''
|
||||
def json(self):
|
||||
return {'id_token': 'the-id-token', 'access_token': 'x'}
|
||||
|
||||
def _post(url, **kwargs):
|
||||
captured['token_url'] = url
|
||||
return _Resp()
|
||||
monkeypatch.setattr(auth.requests, 'post', _post)
|
||||
|
||||
class _FakeKey:
|
||||
key = 'unused'
|
||||
|
||||
class _FakeJwks:
|
||||
uri = 'unused'
|
||||
def get_signing_key_from_jwt(self, token):
|
||||
return _FakeKey()
|
||||
|
||||
def _jwks(uri):
|
||||
captured['jwks_uri'] = uri
|
||||
return _FakeJwks()
|
||||
monkeypatch.setattr(auth, '_get_jwks_client', _jwks)
|
||||
|
||||
real_decode = auth.jwt.decode
|
||||
|
||||
def _decode(token, key, **kw):
|
||||
# id_token 走 RS256：记下校验参数并返回固定 claims；
|
||||
# 会话 cookie 走 HS256：交给真正的实现，别把验签也 mock 掉
|
||||
if kw.get('algorithms') == ['RS256']:
|
||||
captured['decode_kwargs'] = kw
|
||||
return {'sub': '1', 'preferred_username': 'ericwyuan'}
|
||||
return real_decode(token, key, **kw)
|
||||
monkeypatch.setattr(auth.jwt, 'decode', _decode)
|
||||
|
||||
|
||||
def test_callback_success_sets_cookie_and_goes_home(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
auth._pending['thestate'] = {'verifier': 'v', 'expires': time.time() + 600}
|
||||
captured = {}
|
||||
_stub_successful_exchange(monkeypatch, captured)
|
||||
|
||||
resp = client.get('/api/auth/callback?state=thestate&code=abc')
|
||||
assert resp.status_code == 302
|
||||
assert resp.headers['Location'] == '/'
|
||||
assert 'fam_session=' in resp.headers['Set-Cookie']
|
||||
assert 'HttpOnly' in resp.headers['Set-Cookie']
|
||||
assert 'thestate' not in auth._pending # 用过的 state 必须立刻作废
|
||||
|
||||
|
||||
def test_callback_talks_to_internal_base_but_validates_public_issuer(client, monkeypatch):
|
||||
"""换 token / 拉 JWKS 走本机，避免公网 TLS 那条链路上的 CA 和时钟坑；
|
||||
但 id_token 里的 iss 是 auth-hub 配置的公网地址，校验必须按公网来。"""
|
||||
_set_env(monkeypatch, internal_base='http://127.0.0.1:5300')
|
||||
auth._pending['thestate'] = {'verifier': 'v', 'expires': time.time() + 600}
|
||||
captured = {}
|
||||
_stub_successful_exchange(monkeypatch, captured)
|
||||
|
||||
client.get('/api/auth/callback?state=thestate&code=abc')
|
||||
assert captured['token_url'] == 'http://127.0.0.1:5300/token'
|
||||
assert captured['jwks_uri'] == 'http://127.0.0.1:5300/.well-known/jwks.json'
|
||||
assert captured['decode_kwargs']['issuer'] == 'https://auth.example'
|
||||
assert captured['decode_kwargs']['audience'] == 'fam-core'
|
||||
assert captured['decode_kwargs']['leeway'] == auth._LEEWAY
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 会话 cookie / check / verify / logout
|
||||
# ---------------------------------------------------------------------------
|
||||
def test_session_is_stateless(client, monkeypatch):
|
||||
"""cookie 自带签名，服务端不存任何东西——换个进程、重启服务照样认。"""
|
||||
_set_env(monkeypatch)
|
||||
assert auth._pending == {}
|
||||
client.set_cookie('fam_session', _cookie())
|
||||
assert client.get('/api/auth/verify').status_code == 204
|
||||
|
||||
|
||||
def test_verify_rejects_missing_expired_and_forged_cookies(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
assert client.get('/api/auth/verify').status_code == 401
|
||||
|
||||
client.set_cookie('fam_session', _cookie(ttl=-10))
|
||||
assert client.get('/api/auth/verify').status_code == 401
|
||||
|
||||
client.set_cookie('fam_session', _cookie(secret='wrong-secret'))
|
||||
resp = client.get('/api/auth/verify')
|
||||
assert resp.status_code == 401
|
||||
assert resp.get_json()['error'] == '未登录'
|
||||
|
||||
|
||||
def test_check_reports_auth_state(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
assert client.get('/api/auth/check').get_json() == {'authed': False, 'username': ''}
|
||||
|
||||
client.set_cookie('fam_session', _cookie(username='ericwyuan'))
|
||||
assert client.get('/api/auth/check').get_json() == {'authed': True, 'username': 'ericwyuan'}
|
||||
|
||||
|
||||
def test_logout_clears_cookie(client, monkeypatch):
|
||||
_set_env(monkeypatch)
|
||||
client.set_cookie('fam_session', _cookie())
|
||||
resp = client.post('/api/logout')
|
||||
assert resp.status_code == 200
|
||||
assert 'fam_session=;' in resp.headers['Set-Cookie']
|
||||
@@ -463,3 +463,63 @@ def test_delete_video_does_not_touch_ss_motion_events(tmp_path):
|
||||
assert db.get_video_by_motion_event_id(555) is None
|
||||
row = db._conn.execute("SELECT * FROM ss_motion_events WHERE event_id=?", (555,)).fetchone()
|
||||
assert row is not None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 并发（2026-09-13：修"全进程共用一条 sqlite 连接"的回归测试）
|
||||
# ---------------------------------------------------------------------------
|
||||
def test_concurrent_writes_from_many_threads(tmp_path):
|
||||
"""多线程同时读写不能互相踩踏。
|
||||
|
||||
改成每线程一条连接之前，这里会稳定炸出三类错误之一：
|
||||
`cannot start a transaction within a transaction` / `no more rows available`
|
||||
/ `database is locked`——线上 DiskGuard 的清理就是被第一种打断了 2837 次。
|
||||
"""
|
||||
import threading
|
||||
|
||||
db = _db(tmp_path)
|
||||
errors = []
|
||||
rounds = 25
|
||||
|
||||
def writer(tid):
|
||||
try:
|
||||
for i in range(rounds):
|
||||
db.record_activity('t%d' % tid, 'act%d' % i, 'detail')
|
||||
db.set_cursor('cursor_t%d' % tid, str(i))
|
||||
db.record_motion_events([{
|
||||
'event_id': tid * 1000 + i, 'camera_id': 2, 'event_type': 10,
|
||||
'start_time': 1789000000 + i, 'duration': 5,
|
||||
'thumbnail_url': '',
|
||||
}])
|
||||
db.get_recent_activities(5)
|
||||
db.get_motion_heartbeat_age_sec()
|
||||
except Exception as e: # noqa: BLE001 —— 要把原始异常带出来看
|
||||
errors.append(f"线程{tid}: {type(e).__name__}: {e}")
|
||||
|
||||
threads = [threading.Thread(target=writer, args=(t,)) for t in range(8)]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join(timeout=60)
|
||||
|
||||
assert not errors, "并发写出错：\n" + "\n".join(errors[:5])
|
||||
assert db._conn.execute("SELECT COUNT(*) FROM ss_motion_events").fetchone()[0] == 8 * rounds
|
||||
for tid in range(8):
|
||||
assert db.get_cursor('cursor_t%d' % tid) == str(rounds - 1)
|
||||
|
||||
|
||||
def test_close_releases_every_thread_connection(tmp_path):
|
||||
"""close() 要收掉所有线程开过的连接，不只当前线程那一条。"""
|
||||
import threading
|
||||
|
||||
db = _db(tmp_path)
|
||||
db.record_activity('main', 'x', '')
|
||||
|
||||
def other():
|
||||
db.record_activity('other', 'y', '')
|
||||
t = threading.Thread(target=other)
|
||||
t.start(); t.join()
|
||||
|
||||
assert len(db._all_conns) == 2 # 主线程 + 子线程各一条
|
||||
db.close()
|
||||
assert db._all_conns == []
|
||||
|
||||
73
fam-edge/tests/test_video_queue.py
Normal file
73
fam-edge/tests/test_video_queue.py
Normal file
@@ -0,0 +1,73 @@
|
||||
"""VideoQueue 生产者扫描的单测。
|
||||
|
||||
目前只覆盖一个点：生产者列目录之后、读 mtime 之前，DiskGuard 可能刚好把文件清掉
|
||||
（两个后台线程的正常竞态）。线上表现为 `生产者扫描异常: [Errno 2] No such file or
|
||||
directory`，代价是**整轮扫描中断**——排在后面的新素材本轮全都登记不上，要等下一轮。
|
||||
"""
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from fam_edge import video_queue as vq_mod
|
||||
from fam_edge.video_queue import VideoQueue
|
||||
|
||||
|
||||
class _FakeDB:
|
||||
"""只实现生产者路径上用到的方法。"""
|
||||
def __init__(self):
|
||||
self.registered = []
|
||||
self.activities = []
|
||||
|
||||
def get_video_by_filename(self, fn):
|
||||
return None
|
||||
|
||||
def ensure_video(self, fn, path, camera_name=None):
|
||||
self.registered.append(fn)
|
||||
return len(self.registered)
|
||||
|
||||
def set_video_file_status(self, vid, valid, err, meta=None):
|
||||
pass
|
||||
|
||||
def record_activity(self, service, action, detail=''):
|
||||
self.activities.append((service, action))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def queue_with_two_files(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(vq_mod, 'load_config', lambda: {
|
||||
'gdrive_sync': {'local_dir': str(tmp_path), 'camera_name': '客厅'},
|
||||
'video_processing': {'file_validate': True, 'stable_window_sec': 60},
|
||||
})
|
||||
(tmp_path / "a_被删掉的.mp4").write_bytes(b"x")
|
||||
(tmp_path / "b_正常的.mp4").write_bytes(b"y")
|
||||
db = _FakeDB()
|
||||
q = VideoQueue(db)
|
||||
monkeypatch.setattr(q, '_enqueue', lambda vid: None)
|
||||
monkeypatch.setattr(vq_mod, 'validate_video', lambda p: (True, '', {'fps': 25}))
|
||||
return q, db
|
||||
|
||||
|
||||
def test_produce_skips_file_deleted_mid_scan_and_keeps_going(queue_with_two_files, monkeypatch):
|
||||
"""被删掉的那个跳过，后面的照常登记——不能整轮中断。"""
|
||||
q, db = queue_with_two_files
|
||||
real_getmtime = os.path.getmtime
|
||||
|
||||
def fake_getmtime(path):
|
||||
if 'a_被删掉的' in path:
|
||||
raise FileNotFoundError(2, 'No such file or directory', path)
|
||||
return real_getmtime(path) - 3600 # 早于稳定窗口，视为写入完成
|
||||
monkeypatch.setattr(vq_mod.os.path, 'getmtime', fake_getmtime)
|
||||
|
||||
q._produce_once() # 不抛异常
|
||||
|
||||
assert db.registered == ["b_正常的.mp4"] # 被删的跳过，后面的没受影响
|
||||
|
||||
|
||||
def test_produce_still_skips_files_being_written(queue_with_two_files, monkeypatch):
|
||||
"""mtime 太新（rclone 还在写）仍然要跳过，别把半成品入队。"""
|
||||
q, db = queue_with_two_files
|
||||
monkeypatch.setattr(vq_mod.os.path, 'getmtime', lambda p: __import__('time').time())
|
||||
|
||||
q._produce_once()
|
||||
|
||||
assert db.registered == []
|
||||
27
fam-notifier/config/config.yaml.example
Normal file
27
fam-notifier/config/config.yaml.example
Normal file
@@ -0,0 +1,27 @@
|
||||
# fam-notifier 配置（NAS 上唯一保留的服务）
|
||||
# 复制为 config.yaml 后填实际值；${VAR} 会从环境变量解析（start_notifier.sh 会 source .env）
|
||||
|
||||
motion_notifier:
|
||||
enabled: true
|
||||
poll_enabled: true # 轮询主路径（默认开启）
|
||||
dsm_host: "127.0.0.1" # Surveillance Station 就在本机（NAS），走回环即可
|
||||
dsm_port: 5000
|
||||
dsm_account: "${DSM_ACCOUNT}"
|
||||
dsm_password: "${DSM_PASSWORD}"
|
||||
camera_ids: [2] # 轮询关注的摄像头（Generic_ONVIF-001）
|
||||
|
||||
# 推送目标：甲骨文 fam-edge。这是本服务唯一的出口，单向。
|
||||
oracle_base_url: "http://129.146.26.249:5000"
|
||||
oracle_token: "${ORACLE_SYNC_TOKEN}"
|
||||
timeout_sec: 10 # 单次 SS 请求超时
|
||||
|
||||
# 心跳：跟轮询 SS 无关，只是定期空 POST 一下甲骨文的 /api/ss/motion，证明
|
||||
# NAS->甲骨文这条推送链路本身还活着（enabled=true 就跑，不受 poll_enabled 影响）。
|
||||
# 甲骨文侧 dsm_motion_prefilter.max_heartbeat_age_sec（默认 900s）据此判断"无运动"
|
||||
# 结论是否可信——这个心跳间隔要明显小于那个阈值，否则会被误判成链路已死。
|
||||
heartbeat_interval_sec: 300
|
||||
|
||||
# 轮询参数
|
||||
poll_interval_sec: 60 # 轮询间隔
|
||||
poll_window_hours: 2 # 每轮回看窗口（小时），覆盖轮询间隔内的新事件
|
||||
batch_size: 100 # 单批推送上限
|
||||
3
fam-notifier/requirements.txt
Normal file
3
fam-notifier/requirements.txt
Normal file
@@ -0,0 +1,3 @@
|
||||
# NAS 上唯一要装的依赖：一个 HTTP 客户端 + YAML 配置解析
|
||||
requests>=2.31.0
|
||||
PyYAML>=6.0
|
||||
72
fam-notifier/scripts/S99fam-notifier.sh
Executable file
72
fam-notifier/scripts/S99fam-notifier.sh
Executable file
@@ -0,0 +1,72 @@
|
||||
#!/bin/sh
|
||||
# fam-notifier 开机自启（DSM 没有 systemd，/usr/local/etc/rc.d/ 是唯一的守护手段）
|
||||
#
|
||||
# 部署：sudo cp 到 /usr/local/etc/rc.d/S99fam-notifier.sh && sudo chmod 755 同路径
|
||||
#
|
||||
# 为什么非有不可：这个进程是摄像头系统在 NAS 上唯一保留的东西（轮询
|
||||
# Surveillance Station，把运动事件单向推给甲骨文）。它没起来的话事件就悄无声息
|
||||
# 地断流——2026-09-04 到 09-12 断档整整十天，就是因为当时它还在 fam-core 里、
|
||||
# fam-core 挂了没人重启，而且不打开网站根本发现不了。
|
||||
#
|
||||
# 以 ericwyuan 而不是 root 运行：保持 logs/ 和 data/cursor.json 的属主跟手动启动
|
||||
# 时一致，免得 root 建出来的文件之后普通用户改不动。
|
||||
RUN_USER=ericwyuan
|
||||
APP_DIR=/volume1/web/sentinel-home-ai/fam-notifier
|
||||
START="$APP_DIR/scripts/start_notifier.sh"
|
||||
LOG="$APP_DIR/logs/start.log"
|
||||
PIDFILE="$APP_DIR/fam-notifier.pid"
|
||||
|
||||
# DSM 上没有 pgrep，用 grep 字符类找 pid（[f] 写法避免匹配到 grep 自己）
|
||||
find_pid() {
|
||||
ps aux | grep "[f]am_notifier" | awk '{print $2}' | head -1
|
||||
}
|
||||
|
||||
start() {
|
||||
RUNNING=$(find_pid)
|
||||
if [ -n "$RUNNING" ]; then
|
||||
echo "fam-notifier 已在运行 (PID $RUNNING)"; echo "$RUNNING" > "$PIDFILE"; return 0
|
||||
fi
|
||||
# 等待数据卷 volume1 挂载就绪（最多 90 秒），避免开机时脚本早于卷挂载而静默失败
|
||||
n=0
|
||||
while [ ! -f "$START" ] && [ $n -lt 90 ]; do
|
||||
sleep 1; n=$((n+1))
|
||||
done
|
||||
if [ ! -f "$START" ]; then
|
||||
echo "等待 $START 超时（数据卷可能未挂载），启动放弃"; return 1
|
||||
fi
|
||||
# su 自身的 stdio 也要断开：否则通过 SSH 执行本脚本时，子进程继承了连接的管道，
|
||||
# 命令跑完了 SSH 会话却迟迟不退出（开机时无所谓，手动敲的时候很烦）
|
||||
su "$RUN_USER" -c "cd '$APP_DIR' && setsid nohup sh scripts/start_notifier.sh >> '$LOG' 2>&1 < /dev/null &" > /dev/null 2>&1 < /dev/null
|
||||
sleep 4
|
||||
PID=$(find_pid)
|
||||
if [ -n "$PID" ]; then
|
||||
echo "$PID" > "$PIDFILE"
|
||||
echo "fam-notifier 已启动 (PID $PID)"
|
||||
else
|
||||
echo "启动失败，请查看 $LOG 与 $APP_DIR/logs/fam-notifier.log"; return 1
|
||||
fi
|
||||
}
|
||||
|
||||
stop() {
|
||||
if [ -f "$PIDFILE" ]; then kill "$(cat "$PIDFILE")" 2>/dev/null; rm -f "$PIDFILE"; fi
|
||||
for p in $(ps aux | grep "[f]am_notifier" | awk '{print $2}'); do kill "$p" 2>/dev/null; done
|
||||
echo "fam-notifier 已停止"
|
||||
}
|
||||
|
||||
status() {
|
||||
PID=$(find_pid)
|
||||
if [ -n "$PID" ]; then
|
||||
echo "运行中 (PID $PID)"
|
||||
tail -3 "$APP_DIR/logs/fam-notifier.log" 2>/dev/null
|
||||
else
|
||||
echo "未运行"; return 1
|
||||
fi
|
||||
}
|
||||
|
||||
case "$1" in
|
||||
start) start ;;
|
||||
stop) stop ;;
|
||||
restart) stop; sleep 2; start ;;
|
||||
status) status ;;
|
||||
*) echo "用法: $0 {start|stop|restart|status}"; exit 1 ;;
|
||||
esac
|
||||
36
fam-notifier/scripts/start_notifier.sh
Executable file
36
fam-notifier/scripts/start_notifier.sh
Executable file
@@ -0,0 +1,36 @@
|
||||
#!/bin/bash
|
||||
# fam-notifier 启动脚本（NAS 端）
|
||||
# 用法: bash scripts/start_notifier.sh
|
||||
#
|
||||
# NAS 上没有 systemd，挂了不会自启——重启方式见 docs/DEPLOY.md §1.2。
|
||||
set -e
|
||||
|
||||
APP_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
CONFIG_FILE="$APP_DIR/config/config.yaml"
|
||||
|
||||
if [ ! -f "$CONFIG_FILE" ]; then
|
||||
echo "错误: 配置文件不存在: $CONFIG_FILE"
|
||||
echo "请复制 config/config.yaml.example 为 config.yaml 并填入实际值"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "$APP_DIR"
|
||||
|
||||
# 加载 DSM_ACCOUNT / DSM_PASSWORD / ORACLE_SYNC_TOKEN。
|
||||
# set -a 包裹：.env 里是裸赋值，不 export 的话子进程读不到（2026-09-01 踩过）。
|
||||
ENV_FILE="$APP_DIR/../.env"
|
||||
if [ -f "$ENV_FILE" ]; then
|
||||
set -a
|
||||
source "$ENV_FILE"
|
||||
set +a
|
||||
fi
|
||||
|
||||
if [ -d "venv" ]; then
|
||||
source venv/bin/activate
|
||||
fi
|
||||
|
||||
# 包在 src/ 下，不加这行 `python -m fam_notifier` 找不到模块
|
||||
export PYTHONPATH="$APP_DIR/src${PYTHONPATH:+:$PYTHONPATH}"
|
||||
|
||||
echo "启动 fam-notifier（轮询 Surveillance Station -> 推送甲骨文）..."
|
||||
exec python -m fam_notifier
|
||||
0
fam-notifier/src/fam_notifier/__init__.py
Normal file
0
fam-notifier/src/fam_notifier/__init__.py
Normal file
34
fam-notifier/src/fam_notifier/__main__.py
Normal file
34
fam-notifier/src/fam_notifier/__main__.py
Normal file
@@ -0,0 +1,34 @@
|
||||
"""
|
||||
fam-notifier 入口：`python -m fam_notifier`
|
||||
|
||||
没有 Flask，没有对外端口——这个进程只做一件事：轮询 Surveillance Station，
|
||||
把运动事件单向推给甲骨文的 fam-edge。状态查看去网站的「服务状态」页，
|
||||
心跳和事件都记在甲骨文那边（service_activity / ss_motion_events）。
|
||||
"""
|
||||
import sys
|
||||
import time
|
||||
|
||||
from .logger import setup_logger
|
||||
from .notifier import MotionNotifier
|
||||
|
||||
logger = setup_logger('fam-notifier')
|
||||
|
||||
|
||||
def main():
|
||||
notifier = MotionNotifier()
|
||||
if not notifier.enabled:
|
||||
logger.error("motion_notifier.enabled=false，没什么可做的，退出")
|
||||
return 1
|
||||
notifier.start()
|
||||
logger.info("fam-notifier 已启动（Ctrl+C 退出）")
|
||||
try:
|
||||
while notifier.is_alive():
|
||||
time.sleep(5)
|
||||
except KeyboardInterrupt:
|
||||
logger.info("收到中断，停止轮询")
|
||||
notifier.stop()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
32
fam-notifier/src/fam_notifier/config_loader.py
Normal file
32
fam-notifier/src/fam_notifier/config_loader.py
Normal file
@@ -0,0 +1,32 @@
|
||||
"""
|
||||
配置加载器 - 从 config.yaml 读取配置
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import yaml
|
||||
|
||||
|
||||
def _resolve_env_vars(value):
|
||||
"""递归解析字符串中的 ${ENV_VAR} 引用"""
|
||||
if isinstance(value, str):
|
||||
def replace_env(match):
|
||||
env_name = match.group(1)
|
||||
return os.environ.get(env_name, match.group(0))
|
||||
return re.sub(r'\$\{(\w+)\}', replace_env, value)
|
||||
elif isinstance(value, dict):
|
||||
return {k: _resolve_env_vars(v) for k, v in value.items()}
|
||||
elif isinstance(value, list):
|
||||
return [_resolve_env_vars(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def load_config(config_path=None):
|
||||
"""加载 YAML 配置文件，自动解析 ${ENV_VAR} 引用"""
|
||||
if config_path is None:
|
||||
config_path = os.path.join(
|
||||
os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))),
|
||||
'config', 'config.yaml'
|
||||
)
|
||||
with open(config_path, 'r', encoding='utf-8') as f:
|
||||
raw = yaml.safe_load(f)
|
||||
return _resolve_env_vars(raw)
|
||||
34
fam-notifier/src/fam_notifier/logger.py
Normal file
34
fam-notifier/src/fam_notifier/logger.py
Normal file
@@ -0,0 +1,34 @@
|
||||
"""
|
||||
日志工具 - 统一格式（stdout + 文件双写）
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
|
||||
# 日志目录：fam-notifier/logs/（相对 src 的上一级），失败则退化为仅 stdout
|
||||
_LOG_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), 'logs')
|
||||
|
||||
|
||||
def setup_logger(name='fam-notifier', level=logging.INFO):
|
||||
"""配置并返回 logger（stdout + 文件双写）"""
|
||||
logger = logging.getLogger(name)
|
||||
if logger.handlers:
|
||||
return logger
|
||||
logger.setLevel(level)
|
||||
formatter = logging.Formatter(
|
||||
'%(asctime)s [%(name)s] %(levelname)s %(message)s',
|
||||
datefmt='%Y-%m-%d %H:%M:%S'
|
||||
)
|
||||
handler = logging.StreamHandler(sys.stdout)
|
||||
handler.setFormatter(formatter)
|
||||
logger.addHandler(handler)
|
||||
# 文件输出（daemon 模式下 stdout 不可见，文件是唯一可追溯日志）
|
||||
try:
|
||||
os.makedirs(_LOG_DIR, exist_ok=True)
|
||||
file_handler = logging.FileHandler(
|
||||
os.path.join(_LOG_DIR, 'fam-notifier.log'), encoding='utf-8')
|
||||
file_handler.setFormatter(formatter)
|
||||
logger.addHandler(file_handler)
|
||||
except OSError:
|
||||
pass # 目录不可写时退化为仅 stdout
|
||||
return logger
|
||||
@@ -1,5 +1,11 @@
|
||||
"""
|
||||
MotionNotifier - NAS 端运动监测通知服务（轮询主路径，2026-08-22 定型）
|
||||
MotionNotifier - NAS 端运动监测通知服务（轮询主路径，2026-08-22 定型；
|
||||
2026-09-13 从 fam-core 拆出，成为 NAS 上唯一保留的服务）
|
||||
|
||||
拆出来的原因：摄像头插在 NAS 上（Surveillance Station 就是 NAS 本身），这部分
|
||||
搬不走；而其余所有东西——数据库、查询接口、AI 问答、登录——都已经迁到甲骨文。
|
||||
拆完之后 NAS 侧没有 Flask、没有 MariaDB、没有对外端口，只有这一个进程单向往
|
||||
甲骨文推事件，挂了重启即可，不影响网站。
|
||||
|
||||
职责:
|
||||
在 NAS 本机**轮询**群晖 Surveillance Station 的运动侦测事件
|
||||
@@ -12,13 +18,15 @@ MotionNotifier - NAS 端运动监测通知服务（轮询主路径，2026-08-22
|
||||
- 推送后由甲骨文本地落库 ss_motion_events，供 video_processor 本地预过滤使用。
|
||||
|
||||
可靠性设计:
|
||||
- 游标（MariaDB sync_cursor.motion_last_event_id）持久化；重启优先续用 DB 游标，
|
||||
- 游标持久化在本地 JSON 文件（拆出前存 NAS MariaDB 的 sync_cursor 表，
|
||||
现在 NAS 上已经没有数据库了）；重启优先续用游标，
|
||||
停机期间的事件由窗口回看补推；仅首次部署时才初始化为 SS 当前最大 id。
|
||||
- 推送失败的批次不推进游标，下一轮窗口回看重试（不丢事件）。
|
||||
- 心跳线程（enabled 即跑，与轮询无关）：定期空 POST /api/ss/motion，证明
|
||||
NAS->Oracle 推送链路存活，供甲骨文侧判断"无运动"结论是否可信。
|
||||
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
@@ -27,11 +35,33 @@ from datetime import datetime, timezone
|
||||
|
||||
import requests
|
||||
|
||||
from ..logger import setup_logger
|
||||
from ..config_loader import load_config
|
||||
from .. import db_layer
|
||||
from .logger import setup_logger
|
||||
from .config_loader import load_config
|
||||
|
||||
logger = setup_logger('fam-core.motion_notifier')
|
||||
logger = setup_logger('fam-notifier')
|
||||
|
||||
# 游标文件：拆出前这是 MariaDB 里的一行，现在 NAS 上没有数据库了。
|
||||
# 丢了也不致命——窗口回看会把最近的事件补推一遍，甲骨文侧按 event_id 幂等落库。
|
||||
_CURSOR_PATH = os.environ.get('FAM_NOTIFIER_CURSOR') or os.path.join(
|
||||
os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))),
|
||||
'data', 'cursor.json')
|
||||
|
||||
|
||||
def _read_cursor() -> str:
|
||||
try:
|
||||
with open(_CURSOR_PATH, encoding='utf-8') as f:
|
||||
return str(json.load(f).get('motion_last_event_id') or '')
|
||||
except (OSError, ValueError):
|
||||
return ''
|
||||
|
||||
|
||||
def _write_cursor(value: str):
|
||||
try:
|
||||
os.makedirs(os.path.dirname(_CURSOR_PATH), exist_ok=True)
|
||||
with open(_CURSOR_PATH, 'w', encoding='utf-8') as f:
|
||||
json.dump({'motion_last_event_id': str(value)}, f)
|
||||
except OSError as e:
|
||||
logger.warning(f"游标写入失败（下轮窗口回看会补推，不丢事件）: {e}")
|
||||
|
||||
_MOTION_NOTIFIER = None
|
||||
|
||||
@@ -235,15 +265,15 @@ class MotionNotifier:
|
||||
def _init_cursor(self):
|
||||
"""启动时初始化 last_event_id。
|
||||
|
||||
优先使用 DB 已保存的游标：服务停机/重启期间产生的新事件，会在重启后
|
||||
优先使用文件里保存的游标：服务停机/重启期间产生的新事件，会在重启后
|
||||
被下一轮窗口回看捞到并补推（不丢事件）。
|
||||
仅当 DB 无有效游标（首次部署）时，取 SS 当前最大事件 id 作为起点，
|
||||
仅当没有有效游标（首次部署）时，取 SS 当前最大事件 id 作为起点，
|
||||
避免把历史事件全部回灌一遍。
|
||||
"""
|
||||
saved = db_layer.get_motion_cursor()
|
||||
saved = _read_cursor()
|
||||
if saved and int(saved) > 0:
|
||||
self._last_event_id = int(saved)
|
||||
logger.info(f"运动通知游标初始化（DB）: last_event_id={self._last_event_id}")
|
||||
logger.info(f"运动通知游标初始化（本地文件）: last_event_id={self._last_event_id}")
|
||||
return
|
||||
now = int(datetime.now(timezone.utc).timestamp())
|
||||
events = self._fetch_events(now - 3600, now) # 最近 1h 用于定位最大 id
|
||||
@@ -295,7 +325,7 @@ class MotionNotifier:
|
||||
if int(e.get('duration') or 0) <= 0:
|
||||
self._zero_dur_ids.add(int(e.get('id')))
|
||||
self._last_event_id = cursor
|
||||
db_layer.set_motion_cursor(str(self._last_event_id))
|
||||
_write_cursor(str(self._last_event_id))
|
||||
|
||||
def _run(self):
|
||||
logger.info(f"MotionNotifier 启动，轮询间隔 {self.poll_interval_sec}s，"
|
||||
7
fam-notifier/tests/conftest.py
Normal file
7
fam-notifier/tests/conftest.py
Normal file
@@ -0,0 +1,7 @@
|
||||
import os
|
||||
import sys
|
||||
|
||||
# 让测试能直接 `from fam_notifier.xxx import yyy`，无需先 pip install -e .
|
||||
_SRC = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'src')
|
||||
if _SRC not in sys.path:
|
||||
sys.path.insert(0, _SRC)
|
||||
@@ -1,6 +1,6 @@
|
||||
import time
|
||||
|
||||
from fam_core.motion_notifier.motion_notifier import MotionNotifier
|
||||
from fam_notifier.notifier import MotionNotifier
|
||||
|
||||
|
||||
def _cfg(**overrides):
|
||||
@@ -22,7 +22,7 @@ def _cfg(**overrides):
|
||||
|
||||
def _make_notifier(monkeypatch, **cfg_overrides):
|
||||
monkeypatch.setattr(
|
||||
"fam_core.motion_notifier.motion_notifier.load_config",
|
||||
"fam_notifier.notifier.load_config",
|
||||
lambda: _cfg(**cfg_overrides))
|
||||
return MotionNotifier()
|
||||
|
||||
@@ -48,7 +48,7 @@ def test_send_heartbeat_success_updates_state(monkeypatch):
|
||||
def fake_post(url, json=None, timeout=None):
|
||||
calls.append((url, json))
|
||||
return _FakeResp(200, {"status": "ok", "received": 0, "stored": 0})
|
||||
monkeypatch.setattr("fam_core.motion_notifier.motion_notifier.requests.post", fake_post)
|
||||
monkeypatch.setattr("fam_notifier.notifier.requests.post", fake_post)
|
||||
ok = n.send_heartbeat()
|
||||
assert ok is True
|
||||
assert n._last_heartbeat_at is not None
|
||||
@@ -63,7 +63,7 @@ def test_send_heartbeat_http_error_records_failure(monkeypatch):
|
||||
|
||||
def fake_post(url, json=None, timeout=None):
|
||||
return _FakeResp(500, {}, text="boom")
|
||||
monkeypatch.setattr("fam_core.motion_notifier.motion_notifier.requests.post", fake_post)
|
||||
monkeypatch.setattr("fam_notifier.notifier.requests.post", fake_post)
|
||||
ok = n.send_heartbeat()
|
||||
assert ok is False
|
||||
assert n._last_heartbeat_error == "HTTP 500"
|
||||
@@ -75,7 +75,7 @@ def test_send_heartbeat_network_exception_records_failure(monkeypatch):
|
||||
|
||||
def fake_post(url, json=None, timeout=None):
|
||||
raise _requests.RequestException("connection refused")
|
||||
monkeypatch.setattr("fam_core.motion_notifier.motion_notifier.requests.post", fake_post)
|
||||
monkeypatch.setattr("fam_notifier.notifier.requests.post", fake_post)
|
||||
ok = n.send_heartbeat()
|
||||
assert ok is False
|
||||
assert "connection refused" in n._last_heartbeat_error
|
||||
@@ -87,7 +87,7 @@ def test_start_runs_heartbeat_thread_even_when_poll_disabled(monkeypatch):
|
||||
has_motion_in_range_local() 会一直 fail-open，省配额的效果就没了。"""
|
||||
n = _make_notifier(monkeypatch, poll_enabled=False, heartbeat_interval_sec=3600)
|
||||
monkeypatch.setattr(
|
||||
"fam_core.motion_notifier.motion_notifier.requests.post",
|
||||
"fam_notifier.notifier.requests.post",
|
||||
lambda url, json=None, timeout=None: _FakeResp(200, {"stored": 0}))
|
||||
try:
|
||||
n.start()
|
||||
@@ -1,53 +1,68 @@
|
||||
<script setup>
|
||||
import { onMounted, onUnmounted, ref } from 'vue'
|
||||
import { onMounted, ref } from 'vue'
|
||||
import { useRoute } from 'vue-router'
|
||||
import { api, fmtDateTime } from './api.js'
|
||||
import { api } from './api.js'
|
||||
import { navItems } from './router.js'
|
||||
|
||||
const route = useRoute()
|
||||
const sync = ref(null)
|
||||
const authed = ref(false)
|
||||
|
||||
async function refreshStatus() {
|
||||
async function checkAuth() {
|
||||
try {
|
||||
const data = await api.status()
|
||||
sync.value = data.sync
|
||||
const r = await api.authCheck()
|
||||
authed.value = !!r.authed
|
||||
} catch {
|
||||
sync.value = null
|
||||
authed.value = false
|
||||
}
|
||||
}
|
||||
|
||||
let timer = null
|
||||
onMounted(() => {
|
||||
refreshStatus()
|
||||
timer = setInterval(refreshStatus, 30000)
|
||||
})
|
||||
onUnmounted(() => clearInterval(timer))
|
||||
|
||||
async function doLogout() {
|
||||
try {
|
||||
await fetch('/api/logout', { method: 'POST' })
|
||||
} catch {
|
||||
// 忽略网络错误，下面强制跳转即可
|
||||
}
|
||||
// 清掉 cookie 后回根路径，未登录态会让后端 401 -> 自动跳 /login 重新登录
|
||||
window.location.href = '/'
|
||||
}
|
||||
|
||||
onMounted(checkAuth)
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="flex min-h-screen flex-col lg:flex-row">
|
||||
<!-- 桌面端左侧栏：品牌 + 同步状态。lg 以下隐藏，改用下面的移动端顶栏 -->
|
||||
<!-- 桌面端左侧栏：品牌 + 登录入口。lg 以下隐藏，改用下面的移动端顶栏 -->
|
||||
<aside class="hidden w-64 shrink-0 border-r border-border bg-[#090c12] px-4 py-6 lg:block">
|
||||
<div class="mb-6">
|
||||
<div class="text-[17px] font-bold text-[#f7f9fc]">🏠 家庭智能监控</div>
|
||||
<div class="mt-1 text-[11px] text-text-mute">SENTINEL HOME AI · 管理后台</div>
|
||||
</div>
|
||||
<div v-if="sync" class="rounded-xl border border-border bg-panel-2 px-4 py-3 text-xs leading-loose text-text-dim">
|
||||
<b class="text-[#f7f9fc]">同步状态</b><br />
|
||||
状态 <span :class="sync.running ? 'font-semibold text-ok' : 'font-semibold text-danger'">{{ sync.running ? '同步中' : '未运行' }}</span><br />
|
||||
最近 <b class="text-[#ccd5e1]">{{ sync.last_sync_at ? fmtDateTime(sync.last_sync_at) : '—' }}</b><br />
|
||||
本次增量 {{ sync.last_count ? `视频+${sync.last_count[0]} / 事件+${sync.last_count[1]} / 人物+${sync.last_count[2]}` : '—' }}<br />
|
||||
游标 {{ sync.cursor ? fmtDateTime(sync.cursor) : '（全量）' }}
|
||||
<div v-if="sync.last_error" class="font-semibold text-danger">⚠ {{ sync.last_error }}</div>
|
||||
<div class="mt-4">
|
||||
<a v-if="!authed" href="/login"
|
||||
class="block rounded-xl bg-gradient-to-br from-accent to-accent-2 px-4 py-2.5 text-center text-sm font-semibold text-white shadow-[0_4px_16px_-4px_rgba(91,140,255,.45)] transition-opacity hover:opacity-90">
|
||||
🔑 登录
|
||||
</a>
|
||||
<button v-else @click="doLogout"
|
||||
class="block w-full rounded-xl border border-border bg-panel-2 px-4 py-2.5 text-center text-sm font-medium text-text-dim transition-colors hover:text-text">
|
||||
👋 退出登录
|
||||
</button>
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
<!-- 移动端顶栏：品牌 + 简要同步指示灯，lg 以上隐藏（用左侧栏代替） -->
|
||||
<!-- 移动端顶栏：品牌 + 登录/退出，lg 以上隐藏（用左侧栏代替） -->
|
||||
<header class="flex items-center justify-between border-b border-border bg-[#090c12] px-4 py-3 lg:hidden">
|
||||
<div class="text-[15px] font-bold text-[#f7f9fc]">🏠 家庭智能监控</div>
|
||||
<span v-if="sync" class="flex items-center gap-1.5 text-xs font-medium" :class="sync.running ? 'text-ok' : 'text-danger'">
|
||||
<span class="h-1.5 w-1.5 rounded-full bg-current"></span>{{ sync.running ? '同步中' : '未运行' }}
|
||||
</span>
|
||||
<div class="flex items-center gap-2">
|
||||
<a v-if="!authed" href="/login"
|
||||
class="rounded-lg bg-gradient-to-br from-accent to-accent-2 px-3 py-1.5 text-xs font-semibold text-white shadow-[0_4px_16px_-4px_rgba(91,140,255,.45)]">
|
||||
🔑 登录
|
||||
</a>
|
||||
<button v-else @click="doLogout"
|
||||
class="rounded-lg border border-border bg-panel-2 px-3 py-1.5 text-xs font-medium text-text-dim">
|
||||
退出
|
||||
</button>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<div class="min-w-0 flex-1 px-4 py-4 sm:px-8 sm:py-6">
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
// API 薄封装：生产环境同源相对路径；开发环境走 vite.config.js 的 /api 代理。
|
||||
|
||||
// 全局：未登录时首个 401 直接跳转到 /login 走 OIDC/PKCE 登录流程。
|
||||
// 用一个模块级开关保证整次会话只触发一次跳转，避免多个并发 401 重复导航。
|
||||
let _redirectingToLogin = false
|
||||
|
||||
async function request(path, options = {}) {
|
||||
const res = await fetch(path, {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
@@ -12,6 +16,14 @@ async function request(path, options = {}) {
|
||||
// 非 JSON 响应（如 404 空 body），保持 data=null
|
||||
}
|
||||
if (!res.ok) {
|
||||
// 401 未登录：交给后端 /login（302 到 auth-hub）发起统一登录。
|
||||
// /api/auth/check 等白名单接口不会返回 401，所以这里只会命中真正的鉴权失败。
|
||||
if (res.status === 401 && !_redirectingToLogin) {
|
||||
_redirectingToLogin = true
|
||||
window.location.href = '/login'
|
||||
// 返回永挂起的 promise，阻止调用方继续渲染"未登录"错误态（页面即将跳转）
|
||||
return new Promise(() => {})
|
||||
}
|
||||
const msg = (data && (data.error || data.message)) || `HTTP ${res.status}`
|
||||
throw new Error(msg)
|
||||
}
|
||||
@@ -51,8 +63,7 @@ export const api = {
|
||||
mergeMember: (source, target) =>
|
||||
request('/api/member/merge', { method: 'POST', body: JSON.stringify({ source, target }) }),
|
||||
|
||||
status: () => request('/api/status'),
|
||||
syncTrigger: () => request('/api/sync/trigger', { method: 'POST' }),
|
||||
authCheck: () => request('/api/auth/check'),
|
||||
|
||||
identityCorrect: (video_id, current_name, new_name) =>
|
||||
request('/api/member/identity-correct', {
|
||||
|
||||
@@ -11,8 +11,6 @@ import Spinner from '../components/Spinner.vue'
|
||||
const data = ref(null)
|
||||
const oracleError = ref('')
|
||||
const loading = ref(false)
|
||||
const syncing = ref(false)
|
||||
const syncMsg = ref('')
|
||||
|
||||
async function load() {
|
||||
loading.value = true
|
||||
@@ -26,22 +24,6 @@ async function load() {
|
||||
}
|
||||
}
|
||||
|
||||
async function syncNow() {
|
||||
syncing.value = true
|
||||
syncMsg.value = ''
|
||||
try {
|
||||
const r = await api.syncTrigger()
|
||||
const c = r.last_count || []
|
||||
syncMsg.value = c.length
|
||||
? `同步完成：视频+${c[0] ?? 0} 事件+${c[1] ?? 0} 人物+${c[2] ?? 0} 模型调用+${c[3] ?? 0} 身份映射+${c[4] ?? 0}`
|
||||
: '同步完成'
|
||||
await load()
|
||||
} catch (e) {
|
||||
syncMsg.value = `同步失败：${e.message}`
|
||||
} finally {
|
||||
syncing.value = false
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(load)
|
||||
|
||||
@@ -59,7 +41,9 @@ const rclone = computed(() => oracle.value.rclone)
|
||||
const person = computed(() => oracle.value.person)
|
||||
const segment = computed(() => oracle.value.segment)
|
||||
const segCons = computed(() => segment.value?.consistency || {})
|
||||
const nasSync = computed(() => data.value?.nas_sync)
|
||||
const motion = computed(() => oracle.value.motion || {})
|
||||
// 心跳超过 15 分钟视为推送链路失联（与甲骨文侧 max_heartbeat_age_sec 默认值一致）
|
||||
const motionOk = computed(() => (motion.value.heartbeat_age_sec ?? 1e9) < 900)
|
||||
const modelCalls = computed(() => oracle.value.model_calls || [])
|
||||
const activities = computed(() => oracle.value.activities || [])
|
||||
|
||||
@@ -77,16 +61,13 @@ const SVC_BADGE = {
|
||||
<button :disabled="loading" @click="load" class="rounded-lg border border-border bg-panel-2 px-3.5 py-1.5 text-sm text-text-dim hover:border-accent/40 hover:text-white disabled:opacity-50">
|
||||
{{ loading && data ? '刷新中…' : '🔄 刷新' }}
|
||||
</button>
|
||||
<button :disabled="syncing" @click="syncNow" class="rounded-lg border border-accent/40 bg-accent/10 px-3.5 py-1.5 text-sm text-accent hover:bg-accent/20 disabled:opacity-50">
|
||||
{{ syncing ? '同步中…' : '⇄ 立即同步' }}
|
||||
</button>
|
||||
<span class="text-xs text-text-mute">{{ syncMsg || '点击刷新立即更新；立即同步会马上从甲骨文拉一次增量' }}</span>
|
||||
<span class="text-xs text-text-mute">数据直接来自甲骨文，无需同步</span>
|
||||
</div>
|
||||
|
||||
<Spinner v-if="loading && !data" text="加载服务状态…" />
|
||||
<template v-else>
|
||||
<p v-if="oracleError" class="mb-4 text-sm text-warn">{{ oracleError }}</p>
|
||||
<EmptyState v-if="!data?.oracle && !data?.nas_sync" text="暂时无法获取服务状态" />
|
||||
<EmptyState v-if="!data?.oracle" text="暂时无法获取服务状态" />
|
||||
|
||||
<template v-else>
|
||||
<div class="mb-3 text-[15px] font-bold text-[#f7f9fc]">各服务当前状态</div>
|
||||
@@ -112,10 +93,12 @@ const SVC_BADGE = {
|
||||
{{ person ? person.action : '暂无记录' }}
|
||||
<template #sub>{{ person ? `${(person.ts || '').slice(0, 19)} · ${(person.detail || '').slice(0, 46)}` : '—' }}</template>
|
||||
</ServiceCard>
|
||||
<ServiceCard icon="📡" name="NAS 同步" :value-color="nasSync ? '#fbbf24' : undefined">
|
||||
{{ nasSync ? `游标 ${(nasSync.cursor || '').slice(0, 19)}` : '不可达' }}
|
||||
<ServiceCard icon="📡" name="NAS 运动推送" :value-color="motionOk ? '#4ade80' : '#fbbf24'">
|
||||
{{ motion.heartbeat_age_sec == null ? '从未收到心跳'
|
||||
: motionOk ? `正常 · ${Math.round(motion.heartbeat_age_sec)}s 前`
|
||||
: `失联 ${Math.round(motion.heartbeat_age_sec / 60)} 分钟` }}
|
||||
<template #sub>
|
||||
{{ nasSync ? `最近 ${(nasSync.last_sync_at || '').slice(0, 19)} · 增量 V${nasSync.last_count?.[0] ?? 0} E${nasSync.last_count?.[1] ?? 0} P${nasSync.last_count?.[2] ?? 0} M${nasSync.last_count?.[3] ?? 0}` : '—' }}
|
||||
{{ motion.recent?.length ? `最近事件 #${motion.recent[0].event_id} · ${(motion.recent[0].received_at || '').slice(0, 19)}` : '暂无运动事件' }}
|
||||
</template>
|
||||
</ServiceCard>
|
||||
<ServiceCard icon="🧠" name="云端模型" :value-color="modelCalls.length ? '#4ade80' : undefined">
|
||||
|
||||
@@ -1,13 +1,12 @@
|
||||
<script setup>
|
||||
import { computed, onMounted, ref } from 'vue'
|
||||
import { api, fmtDateOnly, fmtDateTime } from '../api.js'
|
||||
import { api, fmtDateOnly } from '../api.js'
|
||||
import PageHeader from '../components/PageHeader.vue'
|
||||
import EmptyState from '../components/EmptyState.vue'
|
||||
import Spinner from '../components/Spinner.vue'
|
||||
|
||||
const modelChart = ref([]) // [{label, value}]
|
||||
const attention = ref([])
|
||||
const syncStatus = ref(null)
|
||||
const loadError = ref('')
|
||||
const loading = ref(true)
|
||||
|
||||
@@ -18,10 +17,9 @@ async function load() {
|
||||
loadError.value = ''
|
||||
loading.value = true
|
||||
try {
|
||||
const [ms, att, statusData] = await Promise.all([
|
||||
const [ms, att] = await Promise.all([
|
||||
api.modelStats(),
|
||||
api.attentionEvents(),
|
||||
api.status(),
|
||||
])
|
||||
const byProvider = {}
|
||||
for (const row of ms.aggregate) {
|
||||
@@ -30,7 +28,6 @@ async function load() {
|
||||
}
|
||||
modelChart.value = Object.entries(byProvider).map(([label, value]) => ({ label, value }))
|
||||
attention.value = att.events
|
||||
syncStatus.value = statusData.sync
|
||||
} catch (e) {
|
||||
loadError.value = e.message
|
||||
} finally {
|
||||
@@ -42,7 +39,7 @@ onMounted(load)
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<PageHeader icon="📈" title="统计图表" sub="模型来源 / 关注事件 / 同步状态" />
|
||||
<PageHeader icon="📈" title="统计图表" sub="模型来源 / 关注事件" />
|
||||
|
||||
<Spinner v-if="loading" text="加载统计数据…" />
|
||||
<template v-else>
|
||||
@@ -79,15 +76,5 @@ onMounted(load)
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<div class="mb-3 mt-6 text-[15px] font-bold text-[#f7f9fc]">同步状态</div>
|
||||
<div v-if="syncStatus" class="rounded-xl border border-border bg-panel-2 px-4 py-3 text-xs leading-loose text-text-dim">
|
||||
运行状态 <span :class="syncStatus.running ? 'font-semibold text-ok' : 'font-semibold text-danger'">{{ syncStatus.running ? '同步中' : '未运行' }}</span><br />
|
||||
最近同步 <b class="text-[#ccd5e1]">{{ syncStatus.last_sync_at ? fmtDateTime(syncStatus.last_sync_at) : '—' }}</b><br />
|
||||
本次增量 {{ syncStatus.last_count ? `视频+${syncStatus.last_count[0]} / 事件+${syncStatus.last_count[1]} / 人物+${syncStatus.last_count[2]}` : '—' }}<br />
|
||||
游标 <b class="text-[#ccd5e1]">{{ syncStatus.cursor ? fmtDateTime(syncStatus.cursor) : '（全量）' }}</b><br />
|
||||
周期 {{ syncStatus.interval_sec }}s
|
||||
<div v-if="syncStatus.last_error" class="font-semibold text-danger">⚠ {{ syncStatus.last_error }}</div>
|
||||
</div>
|
||||
<EmptyState v-else text="无法获取同步状态" />
|
||||
</template>
|
||||
</template>
|
||||
|
||||
@@ -128,8 +128,13 @@ CREATE TABLE IF NOT EXISTS family_members (
|
||||
-- ============================================================
|
||||
|
||||
-- 7.1 视频会话表（Oracle videos 镜像）
|
||||
-- v2 (2026-09-03)：id 改为 NAS 本地自增主键（不再直接等于 Oracle id，因 Oracle 库
|
||||
-- 重建/重排会复用 id，导致 filename UNIQUE 二次冲突 1062）；Oracle 的 videos.id 仅
|
||||
-- 落 oracle_id 列溯源。业务唯一键仍为 filename。子表 video_id 引用仍用 Oracle id
|
||||
-- （新视频插入时 id=oracle_id；旧视频原地更新保留原 id，外键不失效）。
|
||||
CREATE TABLE IF NOT EXISTS sync_videos (
|
||||
id INT PRIMARY KEY COMMENT 'Oracle videos.id',
|
||||
id INT AUTO_INCREMENT PRIMARY KEY COMMENT 'NAS 本地自增主键',
|
||||
oracle_id INT COMMENT 'Oracle videos.id（仅溯源参考）',
|
||||
drive_file_id VARCHAR(255) COMMENT 'Google 硬盘文件 ID',
|
||||
filename VARCHAR(500) NOT NULL UNIQUE COMMENT '视频文件名（唯一）',
|
||||
camera_name VARCHAR(50) COMMENT '摄像头名称/位置',
|
||||
@@ -164,8 +169,11 @@ CREATE TABLE IF NOT EXISTS sync_events (
|
||||
) ENGINE=InnoDB COMMENT='甲骨文事件镜像表';
|
||||
|
||||
-- 7.3 人物规范表（Oracle people 镜像）
|
||||
-- v2 (2026-09-03)：同 7.1，id 改为 NAS 本地自增主键，Oracle people.id 落 oracle_id
|
||||
-- 列溯源；业务唯一键仍为 label。people.id 无外键引用，仅作展示排序。
|
||||
CREATE TABLE IF NOT EXISTS sync_people (
|
||||
id INT PRIMARY KEY COMMENT 'Oracle people.id',
|
||||
id INT AUTO_INCREMENT PRIMARY KEY COMMENT 'NAS 本地自增主键',
|
||||
oracle_id INT COMMENT 'Oracle people.id（仅溯源参考）',
|
||||
label VARCHAR(100) NOT NULL UNIQUE COMMENT '抽象标识，如"人物A"',
|
||||
canonical_name VARCHAR(100) COMMENT '规范名（用户命名或 LLM 合并），NULL 表示未命名',
|
||||
first_seen VARCHAR(32) COMMENT '首次出现时间',
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
# 用法: ./scripts/start_core.sh
|
||||
set -e
|
||||
|
||||
# APP_DIR = <repo>/fam-core
|
||||
APP_DIR="$(cd "$(dirname "$0")/.." && pwd)/fam-core"
|
||||
CONFIG_FILE="$APP_DIR/config/config.yaml"
|
||||
|
||||
@@ -14,10 +15,16 @@ fi
|
||||
|
||||
cd "$APP_DIR"
|
||||
|
||||
# 加载项目级环境变量（ORACLE_SYNC_TOKEN 等，与甲骨文端保持一致）
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
if [ -f "$SCRIPT_DIR/../.env" ]; then
|
||||
source "$SCRIPT_DIR/../.env"
|
||||
# 加载项目级环境变量（ORACLE_SYNC_TOKEN / AUTH_HUB_* 等，与甲骨文端保持一致）
|
||||
# 注意：.env 里 AUTH_HUB_* 是“裸赋值”（无 export 前缀），必须用 set -a 包裹 source，
|
||||
# 否则这些变量只存在于 shell 内、不会 export 给 gunicorn 子进程，导致 fam-core
|
||||
# 启动后读不到 AUTH_HUB_ISSUER 等、登录被 fail-closed 拒绝（见 auth.py）。
|
||||
# 另外 .env 路径基于 $APP_DIR 推导，避免“先 cd 再算 SCRIPT_DIR”的相对路径错位。
|
||||
ENV_FILE="$APP_DIR/../.env"
|
||||
if [ -f "$ENV_FILE" ]; then
|
||||
set -a
|
||||
source "$ENV_FILE"
|
||||
set +a
|
||||
fi
|
||||
|
||||
# 检查虚拟环境
|
||||
|
||||
Reference in New Issue
Block a user