Commit Graph

132 Commits

Author SHA1 Message Date
ericwyuan
99d75a4bda feat(adapter): 云端视觉适配器 + role 角色区分
- base_adapter 增加 role 字段(vision/text)与 get_role()
- gemini_adapter 修复 v1beta 下模型名 404(gemini-1.5-flash→gemini-flash-latest), 改逐帧调用
- 新增 nvidia_adapter(openai SDK, 规避 NIM 单次限 1 图逐帧), 注册 adapter_factory
- 视觉分析仅 vision 角色参与, 文本融合交给 role=text 模型
2026-08-20 09:11:44 +08:00
0af541097f 更新 README.md 2026-08-20 01:00:53 +00:00
0045b074a2 更新 README.md 2026-08-20 01:00:05 +00:00
ericwyuan
3a5ea459a2 docs: 需求文档与 README 整合为单一文档
- 按当前代码实际状态重写:推送模式架构、自适应关键帧、实际 API 端点
- 补充实测性能基准(30min视频929s)、E2E 验证结果、可靠性加固机制
- 记录已知遗留问题(多图失效/吞吐不足)与 v1.1 计划
- 项目需求文档.md 内容并入后删除
2026-08-20 07:19:55 +08:00
ericwyuan
5adda97829 docs: add real video benchmark results to PROGRESS.md
- 30min 360MB video pipeline: 70s transfer + 39s keyframe + 820s AI = 929s total
- FFmpeg fast seek optimization: 180s+ → 33s (6-8x faster)
- AI analysis is the bottleneck: avg 68s/frame, 0.9 tokens/s on ARM CPU
2026-08-20 02:31:06 +08:00
ericwyuan
7ce8aff67c perf: use FFmpeg fast seek instead of fps filter for frame extraction
- Replace single `ffmpeg -vf fps=1/interval` call (full video decode) with
  per-frame `ffmpeg -ss <ts> -frames:v 1` calls (keyframe seek)
- 6-8x faster on ARM: 180s+ → 33s for 60 frames from 30min video
- Per-frame timeout 30s (was 120s single call), failed seeks logged and skipped
- Verified end-to-end: 30min 360MB video → 60 candidates → 12 keyframes in 39s
2026-08-20 02:30:17 +08:00
ericwyuan
6a9d1626fb fix(event-time): event_end_time/frame_timestamp NOT NULL 列兜底
- Edge push 模式: event_end_time 未提供时用 start+视频时长推算
- fam-core db_layer: insert_event 空值兜底(end→start→NOW)
- insert_event_detail: frame_timestamp 空值兜底 NOW
解决 1048 Column cannot be null 落库失败
2026-08-20 02:20:01 +08:00
ericwyuan
e7fb0c92bc docs(readme): 记录 Gemini API Key 与调用示例(gemini-flash-latest,已验证可用) 2026-08-20 02:18:42 +08:00
ericwyuan
c635dd69f1 fix(db): 空字符串datetime归一化为NULL,兼容MariaDB严格模式
monitor_events.event_end_time / event_details.frame_timestamp 收到空串时
插入 NULL 而非 '',避免 1292 Incorrect datetime value 报错
2026-08-20 02:10:37 +08:00
ericwyuan
e7c1641b72 fix(dispatcher): resp.json(silent=) 是 Flask API,requests 不支持会抛 TypeError
Edge 返回结果解析时 TypeError 逃逸到轮询循环,任务卡死 PROCESSING
2026-08-20 01:59:10 +08:00
ericwyuan
be30b5f875 fix(config): Ollama 超时与 num_predict 按 ARM 实测调整
- 实测 1024px 帧视觉编码 ~36s/帧 + ~5 tok/s 生成
- num_predict 500→60: 500 会单帧跑数分钟必触发 240s 超时
- vlm_visual/model timeout 240→600, vlm_fusion 120→300, overall 600→1800
2026-08-20 01:54:32 +08:00
ericwyuan
3a195d69b9 fix(dispatcher): 僵尸PROCESSING任务回收 + fam-core文件日志
- db_layer 新增 reclaim_stale_processing: PROCESSING 超过 push_timeout+120s 重置 PENDING
- dispatcher 轮询前先回收僵尸任务(进程重启/Edge重启导致 in-flight 请求丢失的场景)
- logger 增加 fam-core/logs/fam-core.log 文件输出(daemon 模式 stdout 不可见)
2026-08-20 01:42:59 +08:00
ericwyuan
727642d38b feat: adaptive keyframe count based on video duration
- Replace fixed 5-8 keyframe limit with duration-based adaptive sizing
- Candidate frames: clamp(duration_min × 2, 30, 120)
- Keyframe cap: clamp(duration / 150s, 8, 30)
- 30min video → 12 keyframes (was 8), 60min → 24, 120min → 30
- Short videos (<12min) still get floor of 8 keyframes
- Add Ollama keep-alive config doc to PROGRESS.md (OLLAMA_KEEP_ALIVE=-1)
- Update config.yaml and config.yaml.example with new video params
2026-08-20 01:29:14 +08:00
ericwyuan
40944428d1 feat: video analysis switched to push mode (upload whole video, sync response)
Rationale: Oracle cannot reach NAS (Tailscale userspace mode on NAS, no TUN),
the old pull+webhook design requires Edge to download video from NAS and
callback to NAS - both blocked. New design is one-way NAS -> Oracle:

- FAM-Edge: new POST /api/edge/video/push endpoint accepts multipart video
  upload, reuses existing OpenCV scene-change keyframe selection, analyzes
  synchronously and returns the result payload directly in the HTTP response
  (no webhook callback). Old /api/edge/video/analyze kept for compatibility.
- FAM-Edge: VideoPreprocessor.save_upload() saves the uploaded file
- FAM-Edge: AIOrchestrator.process_push_task() runs the full pipeline
  (health check -> extract -> select -> compress -> VLM -> fusion) and
  returns callback-style payload dict
- FAM-Core: Dispatcher rewritten to push mode - reads local video file,
  uploads with task metadata (camera_name, event_start_time from file mtime,
  known_members_context), applies the result to DB via shared
  event_receiver.apply_success_event()
- FAM-Core: event_receiver success logic extracted into reusable
  apply_success_event() (used by both webhook route and dispatcher)
- config: edge_url -> /api/edge/video/push, push_timeout 1800s, gunicorn
  Edge timeout raised to 1800s for long synchronous analysis
2026-08-20 01:03:48 +08:00
ericwyuan
c596bf7603 feat: FAM-UI deployed on NAS (Streamlit 1.61.1)
- Streamlit 1.61.1 + pandas 2.3.3 installed in fam-ui venv (Python 3.10)
- FAM-UI running headless on 0.0.0.0:8501, HTTP 200, health=ok
- config.yaml synced to NAS (core_url + MariaDB unix_socket)
- All 5 services now running: FAM-Core, FAM-Edge, FAM-UI, MariaDB, Ollama
- PROGRESS.md: task 27 complete
2026-08-20 00:36:37 +08:00
ericwyuan
06e1e5a307 docs: update PROGRESS.md with chat proxy, Tailscale status, E2E results 2026-08-20 00:03:57 +08:00
ericwyuan
94a8805045 feat: FAM-Edge chat proxy + PyMySQL migration for FAM-UI + config update
- FAM-Edge: add /api/edge/chat proxy endpoint forwarding to local Ollama
  (Ollama port 11434 not exposed externally, FAM-Edge acts as reverse proxy)
- FAM-Core config: edge_url and ollama_url switched from Tailscale IP to
  Oracle public IP (Tailscale firewall blocking between NAS and Oracle)
- FAM-UI: migrate mysql.connector to PyMySQL (same as FAM-Core)
- FAM-UI: cursor(dictionary=True) replaced with cursorclass=DictCursor
- End-to-end chat verified: FAM-Core -> FAM-Edge proxy -> Ollama -> response
  Answer: 今天没有观察到张三 (no events in DB yet, expected)
2026-08-19 23:59:28 +08:00
ericwyuan
c8cd2f2665 fix: MariaDB JSON path compat for name_member + FAM-Core API verified
- name_member(): replace MySQL-only JSON_REPLACE/JSON_CONTAINS with wildcard
  JSON paths (unsupported in MariaDB 10.11) with Python-side JSON parsing
- FAM-Core venv rebuilt with Python 3.10.20 on NAS
- All API endpoints tested: health, status, member list, unnamed, naming
- POST /api/member/name: 人物A successfully named to 张三
- PROGRESS.md updated with full test results and task completion
2026-08-19 23:48:35 +08:00
ericwyuan
66b499725d [deploy] FAM-Core + FAM-Edge 服务启动 + Tailscale 跨服务器验证
- FAM-Core (NAS:8000): gunicorn 启动成功, scheduler+dispatcher 运行中
- FAM-Edge (Oracle:5000): gunicorn 启动成功, ollama 健康检查通过
- Tailscale 跨服务器连通: Oracle→NAS FAM-Core health=ok
- config.yaml 同步到 Oracle, FAM-Edge /health 返回 healthy_models=[ollama]
- PROGRESS.md 更新服务运行状态表
2026-08-19 23:32:35 +08:00
ericwyuan
47658fa068 [deploy] PyMySQL 替换 + Tailscale 组网 + config.yaml + 路径修复
- db_layer.py: mysql-connector-python(19MB) → PyMySQL(45KB), 去掉连接池
- config_loader.py: 修复路径解析 (2级→3级 dirname), FAM-Core + FAM-Edge 均修复
- Tailscale 组网完成: Oracle=100.74.137.126, NAS=100.70.234.39, 互通验证通过
- NAS Python 依赖: flask/gunicorn/pymysql/PyYAML 全部安装成功
- NAS 代码部署: /volume1/web/sentinel-home-ai/, FAM-Core 7 模块导入 PASS
- NAS DB 连接验证: PyMySQL → MariaDB 10.11.11, 6 张表可见, PASS
- config.yaml 创建: FAM-Core + FAM-Edge + FAM-UI (Tailscale IP + 密码)
- requirements.txt: mysql-connector-python → PyMySQL
2026-08-19 23:27:28 +08:00
ericwyuan
e5ac667586 [deploy] NAS MariaDB DDL 执行 + Tailscale 状态 + README 更新
- NAS MariaDB 10.11.11 确认可用, root 密码 iLoveJava5!
- DDL 执行成功: sentinel_home_ai 库 + 6 张表 (process_tasks, monitor_events, daily_summaries, event_details, chat_history, family_members)
- NAS Tailscale 已运行, IP: 100.70.234.39
- Oracle Tailscale 1.102.2 已安装, 待用户认证
- README 新增 MariaDB 密码 + Tailscale IP + NAS 型号信息
- PROGRESS.md 更新全部环境状态
2026-08-19 23:08:21 +08:00
ericwyuan
50d83539f6 [deploy] Oracle 环境部署 + 模型基准测试 + OllamaAdapter 优化
- Oracle 依赖安装: flask/gunicorn/requests/PyYAML/opencv/numpy 全部成功
- FAM-Edge 全部 8 个模块导入验证 PASS
- llava-phi3 基准测试: 冷启动 109s, 预热无限制 13.35s, 预热+num_predict=30 仅 4.34s PASS
- OllamaAdapter 新增 num_predict 可配置参数 (默认 500)
- config.yaml.example 新增 num_predict 配置项
- 创建 PROGRESS.md 进度追踪文档
2026-08-19 23:01:21 +08:00
ericwyuan
c1bfac59df fix(requirements): 移除 google-generativeai/openai 硬依赖 - 代码用 requests 直接调 REST API,兼容 Python 3.8 2026-08-19 22:50:30 +08:00
ericwyuan
ba905d2f84 [1.1] 服务器访问信息 - NAS/Oracle SSH 凭证 + Gitea 账号记录到 README 2026-08-19 22:46:31 +08:00
ericwyuan
c45464c3e2 [补全] 各模块 __init__.py 导出 - 确保import链路完整 2026-08-19 22:37:47 +08:00
ericwyuan
5e4587bdc2 [3.5补充] Storage-Cleaner - finally清理逻辑 + 超期残留目录清理 2026-08-19 22:35:50 +08:00
ericwyuan
d741ade9b7 [6.0] 部署脚本与文档 - start_core/edge/ui.sh + DEPLOY.md 部署指南 2026-08-19 22:27:47 +08:00
ericwyuan
4cfb327455 [4.1-5.4] FAM-UI Streamlit - 事件列表/筛选/明细/AI对话/对话历史/成员命名/统计图表 + 配置 2026-08-19 22:26:56 +08:00
ericwyuan
cdd1f21d4c [3.1-3.5] FAM-Edge 全链路 - API-Gateway/Video-Preprocessor/AI-Orchestrator/模型适配器(基类+Ollama+Gemini)/熔断器/JSON解析容错 + 配置 2026-08-19 22:25:38 +08:00
ericwyuan
da6b1c8d39 [2.1-2.5] FAM-Core 六个子模块 - Scheduler/Dispatcher/Event-Receiver/Chat-Handler/Member-Manager/Video-Server + DB层 + 配置 2026-08-19 22:21:31 +08:00
ericwyuan
d6d294bbbe [1.2] 数据库 DDL - 六张表完整建表 + 初始化脚本 2026-08-19 22:18:22 +08:00
ericwyuan
3a315c7129 [1.0] 项目初始化 - 目录结构 + README + .gitignore 2026-08-19 22:17:23 +08:00