ericwyuan/GarminHealthLab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[阶段6] 同步 Garmin 全量数据：31 项日指标 + 奖励 + 个人纪录

Browse Source 
原来每天只存 7 个指标，而 get_user_summary 一次就返回 60+ 字段，
另有睡眠分期、训练准备度、耐力分等独立端点从未被调用。

db.py:
- health_data 新增 31 列（距离/活动卡路里/基础代谢/爬楼/强度分钟/
  久坐时长/最高最低心率/最大压力/身体电量四项/血氧/呼吸/
  睡眠深浅REM清醒分期/睡眠血氧/睡眠呼吸/睡眠压力/训练准备度/
  VO2max/耐力分）
- 新增 badges 与 personal_records 两张表，均以 (user_id, garmin_id)
  为主键，重复同步更新而非累积
- 新增增量迁移: CREATE TABLE IF NOT EXISTS 对已存在的表不生效，
  新列必须显式 ALTER，否则生产库上永远不会出现。按列名比对后
  逐个补齐，SQLite 与 MariaDB 都幂等

services/garmin.py:
- _extract_daily 改为汇总 user_summary + sleep + hrv +
  training_readiness + training_status + endurance_score 五个端点
- 每个可选端点用 _safe 包裹：某项设备不记录时留 NULL，不影响当天其余数据
- 新增 sync_badges / sync_personal_records（账号级，每次同步取一次）

fix(garmin): 个人纪录整批写入失败
- Garmin 在同一份数据里混用 ISO 字符串和 Unix 毫秒时间戳，
  prStartTimeGmt 是 1570961412000，写进 DATETIME 列被 MariaDB
  以 1292 拒绝，导致 11 项个人纪录一条都没存进去
- 新增 _to_datetime 统一处理 ISO / 毫秒 / 秒三种形状，并优先取
  Garmin 自己提供的 *Formatted 字段

services/ai.py:
- 送给模型的 CSV 从 7 列扩到 23 列，纳入身体电量、血氧、呼吸、
  训练准备度、耐力分和睡眠分期

接口: GET /api/health/badges、/api/health/personal-records

tests (+13, 共 292):
- 徽章/纪录的往返、重复同步不累积、按用户隔离
- 两个用户可持有同一个 Garmin 徽章 id 而不冲突
- 时间戳三种形状的归一化及无效值不抛异常

NAS 实测: 7 天数据每天 31 项指标、65 个奖励、11 项个人纪录

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
ericwyuan
2026-08-23 20:48:49 +08:00
parent bb774c332b
commit cbbff61082
10 changed files with 696 additions and 96 deletions
Download Patch File Download Diff File Expand all files Collapse all files

255
backend/services/garmin.py
View File

@@ -69,6 +69,21 @@ class MFARequired(RuntimeError):
"""Raised when a password login needs a code this process cannot obtain."""
# garth sends a browser User-Agent, which its SSO flow needs. The data API
# treats that same UA as a browser hitting it directly and answers every
# request with HTTP 200 and an empty array — no error, just no data. The
# official app's UA (and in fact any non-browser one) returns real data, so
# the header is swapped after login, before any API call.
API_USER_AGENT = "com.garmin.android.apps.connectmobile"
def _use_api_user_agent(client):
try:
client.garth.sess.headers["User-Agent"] = API_USER_AGENT
except AttributeError:
pass # a stubbed client in tests has no session
def _is_cn():
# Selects Garmin's China service, a separate backend with separate
# accounts. This project tracks an international account.
@@ -124,8 +139,11 @@ def _connect(creds, user_id=None):
token = load_token(user_id) if user_id else None
if token:
client.garth.loads(token)
# Populates display_name/unit_system and proves the token still works.
_use_api_user_agent(client)
# Proves the token still works, and refreshes it if near expiry.
client.garth.refresh_oauth2()
# garminconnect builds most of its URLs from display_name, so leaving
# it unset sends every request to ".../None".
client.display_name = client.garth.profile["displayName"]
return client
@@ -140,12 +158,24 @@ def _connect(creds, user_id=None):
# garth's default MFA prompt calls input(); under gunicorn stdin is
# closed, so it raises EOFError rather than anything descriptive.
raise MFARequired(
"该 Garmin 账号开启了两步验证，无法在服务端直接登录。"
"请在 NAS 上执行一次 `python garmin_login.py` 完成验证并保存令牌。"
"该 Garmin 账号开启了两步验证。请在「数据同步」页面用密码重新绑定，"
"系统会提示你输入验证码。"
) from e
_use_api_user_agent(client)
return client
def describe(e):
"""A message that is never empty.
Some exceptions carry no text at all — a bare `assert` raises
AssertionError with str(e) == "" — and storing that produced a failed
sync whose recorded reason was blank, which is undiagnosable.
"""
text = str(e).strip()
return f"{type(e).__name__}: {text}" if text else type(e).__name__
def _num(*values):
"""First value that is a usable number."""
for v in values:
@@ -154,46 +184,179 @@ def _num(*values):
return None
def _extract_daily(client, date_str):
"""One day of metrics, assembled from the endpoints that carry them.
def _safe(fn, default=None):
"""Call an optional endpoint; a metric the device does not record must not
abort the whole day."""
try:
return fn()
except Exception:
return default
Sleep and HRV are separate endpoints in this library — they are not part
of the daily summary — so a sync that only read the summary would record
every night as "no sleep data".
def _first(seq):
return seq[0] if isinstance(seq, list) and seq else {}
def _to_datetime(*values):
"""Normalise Garmin's several timestamp shapes into an ISO string.
The same payload mixes ISO strings ("2019-10-13T10:10:12.0") with epoch
milliseconds (1570961412000); handing the latter to a DATETIME column is
rejected outright, so a personal record whose only timestamp was numeric
failed the whole batch.
"""
summary = client.get_user_summary(date_str) or {}
for v in values:
if v is None or v == "":
continue
if isinstance(v, str):
return v[:26]
if isinstance(v, (int, float)) and not isinstance(v, bool):
# Values past ~1e11 are milliseconds, below that seconds.
seconds = v / 1000 if v > 1e11 else v
try:
return datetime.datetime.utcfromtimestamp(seconds).isoformat(
timespec="seconds"
)
except (ValueError, OverflowError, OSError):
continue
return None
sleep_seconds = None
sleep_quality = None
try:
sleep = (client.get_sleep_data(date_str) or {}).get("dailySleepDTO") or {}
sleep_seconds = _num(sleep.get("sleepTimeSeconds"))
sleep_quality = _num(sleep.get("sleepScores", {}).get("overall", {}).get("value")
if isinstance(sleep.get("sleepScores"), dict) else None)
except Exception:
pass # a missing night must not abort the whole day
hrv = None
try:
hrv_body = client.get_hrv_data(date_str) or {}
summary_block = hrv_body.get("hrvSummary") or {}
hrv = _num(summary_block.get("lastNightAvg"), summary_block.get("weeklyAvg"))
except Exception:
pass
def _extract_daily(client, date_str):
"""Everything Garmin exposes for one day.
The daily summary is the bulk of it, but sleep, HRV, training readiness
and endurance each live behind their own endpoint — none of them appear in
get_user_summary. Each is fetched defensively so a metric this device does
not record leaves a NULL instead of failing the day.
"""
s = client.get_user_summary(date_str) or {}
sleep_dto = (_safe(lambda: client.get_sleep_data(date_str)) or {}).get(
"dailySleepDTO"
) or {}
scores = sleep_dto.get("sleepScores") if isinstance(
sleep_dto.get("sleepScores"), dict
) else {}
sleep_seconds = _num(sleep_dto.get("sleepTimeSeconds"))
hrv_summary = (_safe(lambda: client.get_hrv_data(date_str)) or {}).get(
"hrvSummary"
) or {}
readiness = _first(_safe(lambda: client.get_training_readiness(date_str), []))
training = _safe(lambda: client.get_training_status(date_str), {}) or {}
vo2 = (training.get("mostRecentVO2Max") or {}).get("generic") or {}
endurance = _safe(lambda: client.get_endurance_score(date_str), {}) or {}
def secs(key):
return _num(sleep_dto.get(key))
return {
"date": date_str,
"steps": _num(summary.get("totalSteps")),
"heartRate": _num(summary.get("restingHeartRate"),
summary.get("averageHeartRate")),
"heartRateVariability": hrv,
# --- activity / energy ---
"steps": _num(s.get("totalSteps")),
"stepGoal": _num(s.get("dailyStepGoal")),
"distanceMeters": _num(s.get("totalDistanceMeters")),
"caloriesBurned": _num(s.get("totalKilocalories")),
"activeCalories": _num(s.get("activeKilocalories")),
"bmrCalories": _num(s.get("bmrKilocalories")),
"floorsAscended": _num(s.get("floorsAscended")),
"floorsDescended": _num(s.get("floorsDescended")),
"intensityMinutes": (
(_num(s.get("moderateIntensityMinutes")) or 0)
+ (_num(s.get("vigorousIntensityMinutes")) or 0)
) or None,
"sedentarySeconds": _num(s.get("sedentarySeconds")),
"activeSeconds": _num(s.get("activeSeconds")),
# --- heart / stress ---
"heartRate": _num(s.get("restingHeartRate"), s.get("averageHeartRate")),
"heartRateMax": _num(s.get("maxHeartRate")),
"heartRateMin": _num(s.get("minHeartRate")),
"heartRateVariability": _num(
hrv_summary.get("lastNightAvg"), hrv_summary.get("weeklyAvg")
),
"stress": _num(s.get("averageStressLevel")),
"stressMax": _num(s.get("maxStressLevel")),
# --- body battery ---
"bodyBatteryHigh": _num(s.get("bodyBatteryHighestValue")),
"bodyBatteryLow": _num(s.get("bodyBatteryLowestValue")),
"bodyBatteryCharged": _num(s.get("bodyBatteryChargedValue")),
"bodyBatteryDrained": _num(s.get("bodyBatteryDrainedValue")),
# --- breathing / blood oxygen ---
"spo2Avg": _num(s.get("averageSpo2")),
"spo2Min": _num(s.get("lowestSpo2")),
"respirationAvg": _num(
s.get("avgWakingRespirationValue"), s.get("latestRespirationValue")
),
"respirationMin": _num(s.get("lowestRespirationValue")),
"respirationMax": _num(s.get("highestRespirationValue")),
# --- sleep ---
"sleepDuration": round(sleep_seconds / 3600, 1) if sleep_seconds else None,
"sleepQuality": sleep_quality,
"stress": _num(summary.get("averageStressLevel")),
"caloriesBurned": _num(summary.get("totalKilocalories")),
"sleepQuality": _num((scores.get("overall") or {}).get("value")),
"sleepDeepSeconds": secs("deepSleepSeconds"),
"sleepLightSeconds": secs("lightSleepSeconds"),
"sleepRemSeconds": secs("remSleepSeconds"),
"sleepAwakeSeconds": secs("awakeSleepSeconds"),
"sleepSpo2Avg": secs("averageSpO2Value"),
"sleepRespirationAvg": secs("averageRespirationValue"),
"sleepStressAvg": secs("avgSleepStress"),
# --- training ---
"trainingReadiness": _num(readiness.get("score")),
"vo2max": _num(vo2.get("vo2MaxValue")),
"enduranceScore": _num(endurance.get("overallScore")),
}
def sync_badges(client, user_id):
"""Earned badges. Keyed by Garmin's badge id, so re-syncing updates."""
badges = _safe(lambda: client.get_earned_badges(), []) or []
stored = 0
for b in badges:
bid = b.get("badgeId")
if bid is None:
continue
health.upsert_badge(user_id, {
"id": str(bid),
"badgeKey": b.get("badgeKey"),
"name": b.get("badgeName"),
"categoryId": _num(b.get("badgeCategoryId")),
"difficultyId": _num(b.get("badgeDifficultyId")),
"earnedDate": _to_datetime(b.get("badgeEarnedDate")),
"earnedCount": _num(b.get("badgeEarnedNumber")),
"points": _num(b.get("badgePoints")),
})
stored += 1
return stored
def sync_personal_records(client, user_id):
records = _safe(lambda: client.get_personal_record(), []) or []
stored = 0
for r in records:
rid = r.get("id")
if rid is None:
continue
health.upsert_personal_record(user_id, {
"id": str(rid),
"typeId": _num(r.get("typeId")),
"activityId": r.get("activityId"),
"activityName": r.get("activityName"),
"activityType": r.get("activityType"),
"value": _num(r.get("value")),
# Prefer the pre-formatted strings; the bare fields are epoch ms.
"achievedAt": _to_datetime(
r.get("prStartTimeLocalFormatted"),
r.get("prStartTimeGmtFormatted"),
r.get("activityStartDateTimeLocalFormatted"),
r.get("prStartTimeLocal"),
r.get("prStartTimeGmt"),
),
})
stored += 1
return stored
def _activity_end(start, duration_seconds):
if not start or not duration_seconds:
return start
@@ -260,7 +423,7 @@ def sync_data(user_id, creds, days=None, client=None):
try:
client = client or _connect(creds, user_id)
except Exception as e:
message = str(e)
message = describe(e)
_set_sync_status(user_id, "error", now, records_synced=0, last_error=message)
return {
"status": "error",
@@ -280,7 +443,7 @@ def sync_data(user_id, creds, days=None, client=None):
try:
record = _extract_daily(client, date_str)
except Exception as e:
day_errors.append(f"{date_str}: {e}")
day_errors.append(f"{date_str}: {describe(e)}")
continue
# A day Garmin has no data for comes back all-None; storing it would
# create an empty row that the metric endpoints then have to filter.
@@ -294,7 +457,20 @@ def sync_data(user_id, creds, days=None, client=None):
client, user_id, start_date, today.isoformat()
)
except Exception as e:
day_errors.append(f"activities: {e}")
day_errors.append(f"activities: {describe(e)}")
# Badges and personal records are account-wide rather than per-day, so
# they are fetched once per sync rather than inside the day loop.
badges_synced = 0
records_synced_pr = 0
try:
badges_synced = sync_badges(client, user_id)
except Exception as e:
day_errors.append(f"badges: {describe(e)}")
try:
records_synced_pr = sync_personal_records(client, user_id)
except Exception as e:
day_errors.append(f"personal_records: {describe(e)}")
# Every single day failing means something systemic (expired session,
# API change) — reporting that as a clean success would hide it.
@@ -308,13 +484,18 @@ def sync_data(user_id, creds, days=None, client=None):
user_id, "idle", now, records_synced=days_synced,
last_error="; ".join(day_errors[:3]) if day_errors else None,
)
message = f"同步完成，更新 {days_synced} 天数据、{activities_synced} 条运动记录"
message = (
f"同步完成，更新 {days_synced} 天数据、{activities_synced} 条运动记录、"
f"{badges_synced} 个奖励、{records_synced_pr} 项个人纪录"
)
if day_errors:
message += f"（{len(day_errors)} 天跳过）"
message += f"（{len(day_errors)} 项跳过）"
return {
"status": "success",
"recordsSynced": days_synced,
"activitiesSynced": activities_synced,
"badgesSynced": badges_synced,
"personalRecordsSynced": records_synced_pr,
"message": message,
"lastSyncTime": now,
}