feat: adaptive keyframe count based on video duration
- Replace fixed 5-8 keyframe limit with duration-based adaptive sizing - Candidate frames: clamp(duration_min × 2, 30, 120) - Keyframe cap: clamp(duration / 150s, 8, 30) - 30min video → 12 keyframes (was 8), 60min → 24, 120min → 30 - Short videos (<12min) still get floor of 8 keyframes - Add Ollama keep-alive config doc to PROGRESS.md (OLLAMA_KEEP_ALIVE=-1) - Update config.yaml and config.yaml.example with new video params
This commit is contained in:
12
PROGRESS.md
12
PROGRESS.md
@@ -1,6 +1,6 @@
|
|||||||
# 项目进度追踪
|
# 项目进度追踪
|
||||||
|
|
||||||
> 最后更新: 2026-08-20 00:35
|
> 最后更新: 2026-08-20 01:25
|
||||||
|
|
||||||
## 服务运行状态
|
## 服务运行状态
|
||||||
|
|
||||||
@@ -105,14 +105,16 @@
|
|||||||
| 25 | FAM-Core 配置切换至 Oracle 公网 IP | ✅ 完成 | 2026-08-20 |
|
| 25 | FAM-Core 配置切换至 Oracle 公网 IP | ✅ 完成 | 2026-08-20 |
|
||||||
| 26 | 端到端聊天验证 (Core→Edge→Ollama) | ✅ 完成 | 2026-08-20 |
|
| 26 | 端到端聊天验证 (Core→Edge→Ollama) | ✅ 完成 | 2026-08-20 |
|
||||||
| 27 | NAS 安装 Streamlit + pandas + 部署 FAM-UI | ✅ 完成 | 2026-08-20 |
|
| 27 | NAS 安装 Streamlit + pandas + 部署 FAM-UI | ✅ 完成 | 2026-08-20 |
|
||||||
|
| 28 | Ollama 模型常驻内存 (OLLAMA_KEEP_ALIVE=-1) | ✅ 完成 | 2026-08-20 |
|
||||||
|
| 29 | 关键帧提取改为自适应帧数 (随视频时长动态计算) | ✅ 完成 | 2026-08-20 |
|
||||||
|
|
||||||
### 待完成
|
### 待完成
|
||||||
|
|
||||||
| # | 任务 | 依赖 | 优先级 |
|
| # | 任务 | 依赖 | 优先级 |
|
||||||
|---|------|------|--------|
|
|---|------|------|--------|
|
||||||
| 28 | 视频端到端集成测试 | FAM-UI ✅ | 高 |
|
| 30 | 视频端到端集成测试 | FAM-UI ✅ | 高 |
|
||||||
| 29 | 单元测试 (JSON parser, circuit breaker, schema) | - | 中 |
|
| 31 | 单元测试 (JSON parser, circuit breaker, schema) | - | 中 |
|
||||||
| 30 | Tailscale 防火墙修复 (NAS↔Oracle) | - | 低 |
|
| 32 | Tailscale 防火墙修复 (NAS↔Oracle) | - | 低 |
|
||||||
|
|
||||||
## 技术决策记录
|
## 技术决策记录
|
||||||
|
|
||||||
@@ -124,6 +126,8 @@
|
|||||||
6. **Python 3.10 venv** - NAS 系统 Python 3.8 过旧,用 Synology Python3.10 包创建 venv
|
6. **Python 3.10 venv** - NAS 系统 Python 3.8 过旧,用 Synology Python3.10 包创建 venv
|
||||||
7. **FAM-Edge 聊天代理** - Ollama 端口 11434 未对外暴露,FAM-Edge 新增 /api/edge/chat 代理转发至本地 Ollama
|
7. **FAM-Edge 聊天代理** - Ollama 端口 11434 未对外暴露,FAM-Edge 新增 /api/edge/chat 代理转发至本地 Ollama
|
||||||
8. **Oracle 公网 IP 替代 Tailscale** - Tailscale 两节点在线但端口不通(防火墙),edge_url 和 ollama_url 改用 Oracle 公网 IP 129.146.203.203
|
8. **Oracle 公网 IP 替代 Tailscale** - Tailscale 两节点在线但端口不通(防火墙),edge_url 和 ollama_url 改用 Oracle 公网 IP 129.146.203.203
|
||||||
|
9. **Ollama 模型常驻内存** - systemd 加 `OLLAMA_KEEP_ALIVE=-1`,模型加载后永不卸载,消除 55s 冷启动延迟,常驻占用 4.3GB 内存(系统 12GB 够用)
|
||||||
|
10. **关键帧自适应帧数** - 原固定 5-8 帧对长视频太稀疏(30分钟仅8帧=每3.75分钟1帧),改为随视频时长自适应:候选帧 `clamp(duration_min×2, 30, 120)`,关键帧上限 `clamp(duration/150s, 8, 30)`。30分钟→12帧,60分钟→24帧,封顶30帧
|
||||||
|
|
||||||
## FAM-Core API 测试结果 (2026-08-20)
|
## FAM-Core API 测试结果 (2026-08-20)
|
||||||
|
|
||||||
|
|||||||
@@ -18,7 +18,9 @@ database:
|
|||||||
|
|
||||||
scheduler:
|
scheduler:
|
||||||
scan_interval: 60
|
scan_interval: 60
|
||||||
video_dir: "/volume1/surveillance"
|
# E2E 测试期间指向独立测试目录(正式目录 /volume1/surveillance 有 285 个历史视频,
|
||||||
|
# 全量建任务会导致 100GB 跨公网上传,待与用户确认回补策略后再切回)
|
||||||
|
video_dir: "/volume1/web/sentinel-home-ai/e2e-test"
|
||||||
video_extensions: [".mp4", ".mkv", ".avi"]
|
video_extensions: [".mp4", ".mkv", ".avi"]
|
||||||
file_stable_seconds: 60
|
file_stable_seconds: 60
|
||||||
camera_name: "客厅"
|
camera_name: "客厅"
|
||||||
|
|||||||
@@ -13,11 +13,15 @@ server:
|
|||||||
port: 5000
|
port: 5000
|
||||||
max_concurrent_tasks: 1
|
max_concurrent_tasks: 1
|
||||||
|
|
||||||
# 关键帧筛选参数
|
# 关键帧筛选参数(自适应:帧数随视频时长动态计算)
|
||||||
video:
|
video:
|
||||||
candidate_frames: 30
|
candidate_per_minute: 2 # 每分钟粗抽候选帧数
|
||||||
min_key_frames: 5
|
candidate_min: 30 # 候选帧下限(短视频保底)
|
||||||
max_key_frames: 8
|
candidate_max: 120 # 候选帧上限(超长视频截断)
|
||||||
|
key_frame_interval_sec: 150 # 关键帧间隔(秒),每2.5分钟1张
|
||||||
|
min_key_frames: 5 # 关键帧下限(帧差不足时补足到此数)
|
||||||
|
max_key_frames_floor: 8 # 关键帧上限的下限(短视频保底)
|
||||||
|
max_key_frames_cap: 30 # 关键帧上限(超长视频截断)
|
||||||
mse_threshold: 500
|
mse_threshold: 500
|
||||||
jpeg_quality: 80
|
jpeg_quality: 80
|
||||||
max_long_edge: 1024
|
max_long_edge: 1024
|
||||||
|
|||||||
@@ -13,11 +13,15 @@ server:
|
|||||||
port: 5000
|
port: 5000
|
||||||
max_concurrent_tasks: 1
|
max_concurrent_tasks: 1
|
||||||
|
|
||||||
# 关键帧筛选参数
|
# 关键帧筛选参数(自适应:帧数随视频时长动态计算)
|
||||||
video:
|
video:
|
||||||
candidate_frames: 30 # 粗抽候选帧数
|
candidate_per_minute: 2 # 每分钟粗抽候选帧数
|
||||||
min_key_frames: 5 # 最少关键帧
|
candidate_min: 30 # 候选帧下限(短视频保底)
|
||||||
max_key_frames: 8 # 最多关键帧
|
candidate_max: 120 # 候选帧上限(超长视频截断)
|
||||||
|
key_frame_interval_sec: 150 # 关键帧间隔(秒),每2.5分钟1张
|
||||||
|
min_key_frames: 5 # 关键帧下限(帧差不足时补足到此数)
|
||||||
|
max_key_frames_floor: 8 # 关键帧上限的下限(短视频保底)
|
||||||
|
max_key_frames_cap: 30 # 关键帧上限(超长视频截断)
|
||||||
mse_threshold: 500 # 帧差阈值
|
mse_threshold: 500 # 帧差阈值
|
||||||
jpeg_quality: 80
|
jpeg_quality: 80
|
||||||
max_long_edge: 1024
|
max_long_edge: 1024
|
||||||
|
|||||||
@@ -3,13 +3,19 @@ Video-Preprocessor - 视频预处理
|
|||||||
|
|
||||||
流程:
|
流程:
|
||||||
1. 下载视频(超时 60s)
|
1. 下载视频(超时 60s)
|
||||||
2. FFmpeg 等距粗抽 30 张候选帧
|
2. 根据视频时长自适应计算候选帧数,FFmpeg 等距粗抽
|
||||||
3. OpenCV 帧差分析筛选 5-8 张关键帧(MSE 阈值)
|
3. 根据视频时长自适应计算关键帧数,OpenCV 帧差分析筛选(MSE 阈值)
|
||||||
4. 压缩(长边 ≤ 1024px,JPEG 质量 80)
|
4. 压缩(长边 ≤ 1024px,JPEG 质量 80)
|
||||||
|
|
||||||
|
自适应规则:
|
||||||
|
- 候选帧: max(candidate_min, duration_min * candidate_per_minute), 上限 candidate_max
|
||||||
|
- 关键帧: max(min_key_frames, duration / key_frame_interval_sec), 上限 max_key_frames_cap
|
||||||
|
例: 30分钟视频 → 候选60张 → 关键帧12张(每2.5分钟1张)
|
||||||
|
例: 3分钟视频 → 候选30张 → 关键帧8张(保底)
|
||||||
|
|
||||||
异常兜底:
|
异常兜底:
|
||||||
- ffprobe 失败 -> 退化为按 60s 间隔抽帧
|
- ffprobe 失败 -> 退化为按 60s 间隔抽帧
|
||||||
- 帧差分析异常 -> 退化为等距抽 5 帧
|
- 帧差分析异常 -> 退化为等距抽 min_key_frames 帧
|
||||||
- OpenCV 压缩失败 -> 跳过该帧,记录 WARN
|
- OpenCV 压缩失败 -> 跳过该帧,记录 WARN
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
@@ -33,9 +39,13 @@ class VideoPreprocessor:
|
|||||||
self.task_id = task_id
|
self.task_id = task_id
|
||||||
cfg = load_config()
|
cfg = load_config()
|
||||||
video_cfg = cfg.get('video', {})
|
video_cfg = cfg.get('video', {})
|
||||||
self.candidate_frames = video_cfg.get('candidate_frames', 30)
|
self.candidate_per_minute = video_cfg.get('candidate_per_minute', 2)
|
||||||
|
self.candidate_min = video_cfg.get('candidate_min', 30)
|
||||||
|
self.candidate_max = video_cfg.get('candidate_max', 120)
|
||||||
|
self.key_frame_interval_sec = video_cfg.get('key_frame_interval_sec', 150)
|
||||||
self.min_key_frames = video_cfg.get('min_key_frames', 5)
|
self.min_key_frames = video_cfg.get('min_key_frames', 5)
|
||||||
self.max_key_frames = video_cfg.get('max_key_frames', 8)
|
self.max_key_frames_floor = video_cfg.get('max_key_frames_floor', 8)
|
||||||
|
self.max_key_frames_cap = video_cfg.get('max_key_frames_cap', 30)
|
||||||
self.mse_threshold = video_cfg.get('mse_threshold', 500)
|
self.mse_threshold = video_cfg.get('mse_threshold', 500)
|
||||||
self.jpeg_quality = video_cfg.get('jpeg_quality', 80)
|
self.jpeg_quality = video_cfg.get('jpeg_quality', 80)
|
||||||
self.max_long_edge = video_cfg.get('max_long_edge', 1024)
|
self.max_long_edge = video_cfg.get('max_long_edge', 1024)
|
||||||
@@ -43,6 +53,9 @@ class VideoPreprocessor:
|
|||||||
timeout_cfg = cfg.get('timeout', {})
|
timeout_cfg = cfg.get('timeout', {})
|
||||||
self.download_timeout = timeout_cfg.get('download', 60)
|
self.download_timeout = timeout_cfg.get('download', 60)
|
||||||
|
|
||||||
|
# 视频时长(秒),在 extract_candidate_frames 中填充
|
||||||
|
self.video_duration = 0.0
|
||||||
|
|
||||||
# 临时目录
|
# 临时目录
|
||||||
self.work_dir = f"/tmp/fam_media/task_{task_id}"
|
self.work_dir = f"/tmp/fam_media/task_{task_id}"
|
||||||
self.video_path = os.path.join(self.work_dir, f"video_{task_id}.mp4")
|
self.video_path = os.path.join(self.work_dir, f"video_{task_id}.mp4")
|
||||||
@@ -96,15 +109,23 @@ class VideoPreprocessor:
|
|||||||
return 0.0
|
return 0.0
|
||||||
|
|
||||||
def extract_candidate_frames(self, video_path: str) -> List[str]:
|
def extract_candidate_frames(self, video_path: str) -> List[str]:
|
||||||
"""等距粗抽候选帧"""
|
"""等距粗抽候选帧(数量随视频时长自适应)"""
|
||||||
os.makedirs(self.frames_dir, exist_ok=True)
|
os.makedirs(self.frames_dir, exist_ok=True)
|
||||||
duration = self._get_video_duration(video_path)
|
duration = self._get_video_duration(video_path)
|
||||||
|
self.video_duration = duration
|
||||||
|
|
||||||
if duration > 0:
|
if duration > 0:
|
||||||
interval = duration / self.candidate_frames
|
duration_min = duration / 60
|
||||||
|
# 自适应候选帧数:每分钟 candidate_per_minute 张,保底 candidate_min,上限 candidate_max
|
||||||
|
candidate_count = min(
|
||||||
|
max(self.candidate_min, int(duration_min * self.candidate_per_minute)),
|
||||||
|
self.candidate_max
|
||||||
|
)
|
||||||
|
interval = duration / candidate_count
|
||||||
else:
|
else:
|
||||||
# 兜底: 每 60s 抽一帧
|
# 兜底: 每 60s 抽一帧
|
||||||
interval = 60
|
interval = 60
|
||||||
|
candidate_count = 0
|
||||||
logger.warning(f"[task_id={self.task_id}] ffprobe 失败,退化为 60s 间隔抽帧")
|
logger.warning(f"[task_id={self.task_id}] ffprobe 失败,退化为 60s 间隔抽帧")
|
||||||
|
|
||||||
cmd = [
|
cmd = [
|
||||||
@@ -125,13 +146,29 @@ class VideoPreprocessor:
|
|||||||
for f in os.listdir(self.frames_dir)
|
for f in os.listdir(self.frames_dir)
|
||||||
if f.endswith('.jpg')
|
if f.endswith('.jpg')
|
||||||
])
|
])
|
||||||
log_task(logger, self.task_id, 'extract', f'粗抽 {len(frames)} 张候选帧')
|
log_task(logger, self.task_id, 'extract',
|
||||||
|
f'视频时长 {duration:.0f}s, 粗抽 {len(frames)} 张候选帧 (目标 {candidate_count})')
|
||||||
return frames
|
return frames
|
||||||
|
|
||||||
|
def _compute_adaptive_key_frame_counts(self) -> Tuple[int, int]:
|
||||||
|
"""根据视频时长自适应计算关键帧下限和上限"""
|
||||||
|
if self.video_duration > 0:
|
||||||
|
# 每隔 key_frame_interval_sec 秒 1 张关键帧
|
||||||
|
adaptive = int(self.video_duration / self.key_frame_interval_sec)
|
||||||
|
max_kf = min(max(self.max_key_frames_floor, adaptive), self.max_key_frames_cap)
|
||||||
|
else:
|
||||||
|
max_kf = self.max_key_frames_floor
|
||||||
|
min_kf = max(self.min_key_frames, max_kf // 2)
|
||||||
|
return min_kf, max_kf
|
||||||
|
|
||||||
def select_key_frames(self, candidate_frames: List[str]) -> List[str]:
|
def select_key_frames(self, candidate_frames: List[str]) -> List[str]:
|
||||||
"""帧差分析筛选关键帧"""
|
"""帧差分析筛选关键帧(数量随视频时长自适应)"""
|
||||||
if len(candidate_frames) <= self.min_key_frames:
|
min_kf, max_kf = self._compute_adaptive_key_frame_counts()
|
||||||
return candidate_frames[:self.max_key_frames]
|
log_task(logger, self.task_id, 'select_keyframes',
|
||||||
|
f'自适应关键帧: min={min_kf}, max={max_kf} (视频时长 {self.video_duration:.0f}s)')
|
||||||
|
|
||||||
|
if len(candidate_frames) <= min_kf:
|
||||||
|
return candidate_frames[:max_kf]
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# 加载所有候选帧
|
# 加载所有候选帧
|
||||||
@@ -142,7 +179,7 @@ class VideoPreprocessor:
|
|||||||
images.append((path, img))
|
images.append((path, img))
|
||||||
|
|
||||||
if len(images) < 2:
|
if len(images) < 2:
|
||||||
return candidate_frames[:self.max_key_frames]
|
return candidate_frames[:max_kf]
|
||||||
|
|
||||||
# 计算每帧与前一关键帧的 MSE
|
# 计算每帧与前一关键帧的 MSE
|
||||||
key_indices = [0] # 首帧必选
|
key_indices = [0] # 首帧必选
|
||||||
@@ -158,18 +195,18 @@ class VideoPreprocessor:
|
|||||||
if key_indices[-1] != len(images) - 1:
|
if key_indices[-1] != len(images) - 1:
|
||||||
key_indices.append(len(images) - 1)
|
key_indices.append(len(images) - 1)
|
||||||
|
|
||||||
# 若 < min_key_frames,从剩余中均匀补足
|
# 若 < min_kf,从剩余中均匀补足
|
||||||
if len(key_indices) < self.min_key_frames:
|
if len(key_indices) < min_kf:
|
||||||
remaining = [i for i in range(len(images)) if i not in key_indices]
|
remaining = [i for i in range(len(images)) if i not in key_indices]
|
||||||
step = max(1, len(remaining) // (self.min_key_frames - len(key_indices)))
|
step = max(1, len(remaining) // (min_kf - len(key_indices)))
|
||||||
for i in range(0, len(remaining), step):
|
for i in range(0, len(remaining), step):
|
||||||
if len(key_indices) >= self.min_key_frames:
|
if len(key_indices) >= min_kf:
|
||||||
break
|
break
|
||||||
key_indices.append(remaining[i])
|
key_indices.append(remaining[i])
|
||||||
key_indices.sort()
|
key_indices.sort()
|
||||||
|
|
||||||
# 若 > max_key_frames,按差异值降序取前 N
|
# 若 > max_kf,按差异值降序取前 N
|
||||||
if len(key_indices) > self.max_key_frames:
|
if len(key_indices) > max_kf:
|
||||||
# 计算每个关键帧与前一帧的差异
|
# 计算每个关键帧与前一帧的差异
|
||||||
diffs = []
|
diffs = []
|
||||||
for idx in key_indices[1:-1]: # 不含首末帧
|
for idx in key_indices[1:-1]: # 不含首末帧
|
||||||
@@ -178,7 +215,7 @@ class VideoPreprocessor:
|
|||||||
diffs.sort(key=lambda x: x[1], reverse=True)
|
diffs.sort(key=lambda x: x[1], reverse=True)
|
||||||
# 保留首末帧 + 差异最大的
|
# 保留首末帧 + 差异最大的
|
||||||
keep = {0, len(images)-1}
|
keep = {0, len(images)-1}
|
||||||
for idx, _ in diffs[:self.max_key_frames - 2]:
|
for idx, _ in diffs[:max_kf - 2]:
|
||||||
keep.add(idx)
|
keep.add(idx)
|
||||||
key_indices = sorted(keep)
|
key_indices = sorted(keep)
|
||||||
|
|
||||||
@@ -187,9 +224,9 @@ class VideoPreprocessor:
|
|||||||
return key_frames
|
return key_frames
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"[task_id={self.task_id}] 帧差分析异常: {e},退化为等距抽 5 帧")
|
logger.warning(f"[task_id={self.task_id}] 帧差分析异常: {e},退化为等距抽 {min_kf} 帧")
|
||||||
step = max(1, len(candidate_frames) // self.min_key_frames)
|
step = max(1, len(candidate_frames) // min_kf)
|
||||||
return candidate_frames[::step][:self.min_key_frames]
|
return candidate_frames[::step][:min_kf]
|
||||||
|
|
||||||
def _compute_mse(self, img1, img2) -> float:
|
def _compute_mse(self, img1, img2) -> float:
|
||||||
"""计算两帧的 MSE"""
|
"""计算两帧的 MSE"""
|
||||||
|
|||||||
Reference in New Issue
Block a user