feat: NAS端FFmpeg预压缩 — 根治360MB视频跨境上传超时
根因诊断(网络问题,非代码逻辑):
- NAS(中国移动) → Oracle(美国Phoenix) 跨境上行带宽实测仅 0.5-0.9MB/s 且波动大
- 下载方向 3.0MB/s(非对称,典型国际出口拥塞)
- Tailscale P2P: ping 可通(224ms)但持续数据流被阻断(0 B/s, 127s超时)
- 360MB 原始视频上传需 10-20min,任何大分块都会超时
NAS dispatcher:
- 新增 _compress_video: 480p/CRF28/veryfast,静态监控场景实测 ~36x 压缩比
(360MB → ~12MB,上传时间 20min → ~30s)
- ffmpeg 自动探测: CodecPack ffmpeg41(带libx264) > /usr/local/bin > PATH
(Synology 系统 ffmpeg 被裁剪,无 h264 编解码)
- scale 两级滤镜: force_original_aspect_ratio=decrease + trunc(iw/2)*2
(h264 要求偶数尺寸,853x480 会报错)
- 压缩缓存 /tmp/fam_compressed/task_{id}/,源文件未变则重试复用,TTL 24h
- 压缩完成后重置 PROCESSING 状态(重置 stale 回收计时基准)
- 压缩失败回退原始文件分块上传
- _query_uploaded_chunks 失败时记录日志(原静默失败导致全量重传无感知)
Edge api_gateway:
- fix: total_chunks 变更清理旧分块后同步重写 meta.json
(原 bug: meta.json 不更新导致每块上传都触发清理,删除同批新分块死循环)
config:
- stale_timeout 600→1800(覆盖压缩+上传+Edge队列积压总时长)
- 新增 ffmpeg_path / compress_timeout 配置项
This commit is contained in:
@@ -168,10 +168,10 @@ def upload_chunk():
|
||||
chunk_path = os.path.join(d, f"chunk_{chunk_index:04d}")
|
||||
|
||||
try:
|
||||
# 检查 total_chunks 是否变化(chunk_size 变更导致),自动清理旧分块
|
||||
# 检查 total_chunks 是否变化(chunk_size 变更导致),自动清理旧分块并更新元数据
|
||||
import json
|
||||
meta_path = os.path.join(d, "meta.json")
|
||||
if total_chunks and os.path.exists(meta_path):
|
||||
import json
|
||||
try:
|
||||
with open(meta_path) as mf:
|
||||
old_meta = json.load(mf)
|
||||
@@ -181,15 +181,16 @@ def upload_chunk():
|
||||
for fn in os.listdir(d):
|
||||
if fn.startswith('chunk_'):
|
||||
os.remove(os.path.join(d, fn))
|
||||
with open(meta_path, 'w') as f:
|
||||
json.dump({"filename": filename, "total_chunks": total_chunks}, f)
|
||||
except (json.JSONDecodeError, IOError):
|
||||
pass
|
||||
|
||||
chunk_file.save(chunk_path)
|
||||
size_kb = os.path.getsize(chunk_path) / 1024
|
||||
|
||||
# 写/更新元数据
|
||||
# 写元数据(首次上传时)
|
||||
if not os.path.exists(meta_path):
|
||||
import json
|
||||
meta = {"filename": filename, "total_chunks": total_chunks}
|
||||
with open(meta_path, 'w') as f:
|
||||
json.dump(meta, f)
|
||||
|
||||
Reference in New Issue
Block a user