fix: 分块大小减至5MB + Edge端chunk_size变更自动清理 + 断点续传防护

NAS dispatcher:
- CHUNK_SIZE 10MB→5MB(~1Mbps上行带宽下可靠传输)
- chunk上传timeout (30,120)→(60,180)(增加连接和读取余量)
- _query_uploaded_chunks 返回 (set, edge_total) 元组
- expected_total != edge_total 时跳过断点续传(防止chunk_size变更导致文件损坏)

Edge api_gateway:
- upload_chunk 检测 total_chunks 变更,自动清理旧分块
- 防止不同chunk_size的旧分块与新分块混合导致assemble后文件损坏
This commit is contained in:
ericwyuan
2026-08-20 14:18:29 +08:00
parent d6054de060
commit 5915cf4be3
2 changed files with 37 additions and 10 deletions

View File

@@ -168,11 +168,26 @@ def upload_chunk():
chunk_path = os.path.join(d, f"chunk_{chunk_index:04d}")
try:
# 检查 total_chunks 是否变化chunk_size 变更导致),自动清理旧分块
meta_path = os.path.join(d, "meta.json")
if total_chunks and os.path.exists(meta_path):
import json
try:
with open(meta_path) as mf:
old_meta = json.load(mf)
if old_meta.get('total_chunks') and old_meta['total_chunks'] != total_chunks:
logger.warning(f"[task_id={task_id}] total_chunks 变更 "
f"({old_meta['total_chunks']}{total_chunks}),清理旧分块")
for fn in os.listdir(d):
if fn.startswith('chunk_'):
os.remove(os.path.join(d, fn))
except (json.JSONDecodeError, IOError):
pass
chunk_file.save(chunk_path)
size_kb = os.path.getsize(chunk_path) / 1024
# 写元数据(首次上传时)
meta_path = os.path.join(d, "meta.json")
# 写/更新元数据
if not os.path.exists(meta_path):
import json
meta = {"filename": filename, "total_chunks": total_chunks}