feat: NAS端FFmpeg预压缩 — 根治360MB视频跨境上传超时

根因诊断(网络问题,非代码逻辑):
- NAS(中国移动) → Oracle(美国Phoenix) 跨境上行带宽实测仅 0.5-0.9MB/s 且波动大
- 下载方向 3.0MB/s(非对称,典型国际出口拥塞)
- Tailscale P2P: ping 可通(224ms)但持续数据流被阻断(0 B/s, 127s超时)
- 360MB 原始视频上传需 10-20min,任何大分块都会超时

NAS dispatcher:
- 新增 _compress_video: 480p/CRF28/veryfast,静态监控场景实测 ~36x 压缩比
  (360MB → ~12MB,上传时间 20min → ~30s)
- ffmpeg 自动探测: CodecPack ffmpeg41(带libx264) > /usr/local/bin > PATH
  (Synology 系统 ffmpeg 被裁剪,无 h264 编解码)
- scale 两级滤镜: force_original_aspect_ratio=decrease + trunc(iw/2)*2
  (h264 要求偶数尺寸,853x480 会报错)
- 压缩缓存 /tmp/fam_compressed/task_{id}/,源文件未变则重试复用,TTL 24h
- 压缩完成后重置 PROCESSING 状态(重置 stale 回收计时基准)
- 压缩失败回退原始文件分块上传
- _query_uploaded_chunks 失败时记录日志(原静默失败导致全量重传无感知)

Edge api_gateway:
- fix: total_chunks 变更清理旧分块后同步重写 meta.json
  (原 bug: meta.json 不更新导致每块上传都触发清理,删除同批新分块死循环)

config:
- stale_timeout 600→1800(覆盖压缩+上传+Edge队列积压总时长)
- 新增 ffmpeg_path / compress_timeout 配置项
This commit is contained in:
ericwyuan
2026-08-20 14:59:50 +08:00
parent a9a1b1dda5
commit 203b076783
3 changed files with 139 additions and 18 deletions

View File

@@ -168,10 +168,10 @@ def upload_chunk():
chunk_path = os.path.join(d, f"chunk_{chunk_index:04d}")
try:
# 检查 total_chunks 是否变化chunk_size 变更导致),自动清理旧分块
# 检查 total_chunks 是否变化chunk_size 变更导致),自动清理旧分块并更新元数据
import json
meta_path = os.path.join(d, "meta.json")
if total_chunks and os.path.exists(meta_path):
import json
try:
with open(meta_path) as mf:
old_meta = json.load(mf)
@@ -181,15 +181,16 @@ def upload_chunk():
for fn in os.listdir(d):
if fn.startswith('chunk_'):
os.remove(os.path.join(d, fn))
with open(meta_path, 'w') as f:
json.dump({"filename": filename, "total_chunks": total_chunks}, f)
except (json.JSONDecodeError, IOError):
pass
chunk_file.save(chunk_path)
size_kb = os.path.getsize(chunk_path) / 1024
# 写/更新元数据
# 写元数据(首次上传时)
if not os.path.exists(meta_path):
import json
meta = {"filename": filename, "total_chunks": total_chunks}
with open(meta_path, 'w') as f:
json.dump(meta, f)