「又被限流了」的根因找到了,不是请求量,是令牌。 `_connect` 里 `refresh_oauth2()` 换来的新 OAuth2 令牌只活在进程内存里—— `save_token` 只在绑定账号时调用过一次。于是每次客户端缓存过期(15 分钟)、 每个 gunicorn worker、每次部署重启,都从库里读回**同一个已过期的令牌**, 然后再做一次真实 SSO 换令牌。而 SSO 端点是按账号限流最狠的那个,社区报告能 封 48 小时(garth #217、python-garminconnect #337)。我今天为了部署重启了 八次服务,每次都清掉缓存。 - `refresh_oauth2()` 成功后 `_persist_token()` 写回。拆出这个函数是因为它和 `save_token` 想要的正好相反:重新绑定要作废现有会话,持久化刷新结果必须 保住刚刚产出它的那个会话 - 写回时不带 garmin_email,否则 upsert 会把绑定邮箱刷成 NULL,数据同步页会 忘记绑的是哪个账号 - 刷新加进程内锁,并在拿到锁后重读一次库:另一个线程刚换过就直接用它的, 不再自己去换一次 - 五条测试盯住这个不变量,包括「冷缓存不该再换一次」(这条如果回归,就是同一 个 bug 再来一遍) 顺带把数据端点也节流了——那是另外一半问题,不是这次的病因,但一天历史要 9 次 调用,730 天全历史 6600 个请求全速打出去,不该指望佳明一直容忍: - services/garmin_throttle.py:代理包住 client,所有调用(含以后新加的)都经 同一个收口,按间隔排队并计数 - 0.5s 是查过的:garmin-data-export 默认 0.15s、garmin-connect-scraper 默认 3s、官方合作方 API 100 次/分钟(0.6s)。依据写在文件顶部 - 单次同步 1200 个请求预算,跑满就干净收尾、下次接着跑(已存的天数本来就跳过) - 运动详情每次最多补 40 条——新账号几百条,不限量就是一次性打光预算 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
203 lines
7.3 KiB
Python
203 lines
7.3 KiB
Python
"""
|
|
Client-side pacing for Garmin Connect.
|
|
|
|
Why this exists
|
|
---------------
|
|
A sync is not a handful of requests. Each day costs **nine** API calls
|
|
(`_extract_daily` makes six, `daily_extras` three), short syncs add five more
|
|
for the within-day curves, every activity without a stored detail costs one,
|
|
and five account-wide calls finish the run. So:
|
|
|
|
7 days ~ 100 requests
|
|
30 days ~ 300 requests
|
|
730 days ~ 6600 requests
|
|
|
|
and until now they were fired as fast as the network allowed. That is what
|
|
walks the account into a 429 — repeatedly, because the only defence was a 24h
|
|
cooldown *after* Garmin had already punished us. Backing off after the fact
|
|
does not stop it happening again; spacing the requests does.
|
|
|
|
Garmin publishes no limit for these endpoints, so the default comes from what
|
|
other consumers actually get away with (checked 2026-09-03):
|
|
|
|
* `sirredbeard/garmin-data-export` paces at **0.15s** by default, backs off on
|
|
a 429, ramps back up, and pauses every 250 calls. Its README suggests
|
|
`--delay 1.0` "if you want to play it safe".
|
|
* `evg656e/garmin-connect-scraper` defaults to **3s**, warning that smaller
|
|
values "may result in timeout penalties from Garmin".
|
|
* Garmin's *official* (partner) API allows **100 requests/minute** — 0.6s
|
|
each — which is the closest thing to a number Garmin itself stands behind.
|
|
|
|
0.5s sits between them: slower than the official allowance, three times more
|
|
cautious than the aggressive scraper, six times faster than the timid one.
|
|
|
|
Worth being clear about what this does and does not fix. The brutal
|
|
multi-hour blocks reported across those projects are on the **SSO/login**
|
|
endpoint and are keyed per account; this app's own lockout came from there,
|
|
not from data volume (it was re-minting a token on every connect — see
|
|
`_connect` in garmin.py). Pacing the data calls is the other half: a full
|
|
backfill firing 6600 requests flat out is not something to rely on Garmin
|
|
tolerating.
|
|
|
|
What it does
|
|
------------
|
|
* **Spacing.** Every call waits until at least `min_interval` has passed since
|
|
the previous one, per account.
|
|
* **A per-run budget.** A run may issue at most N requests; the sync then stops
|
|
cleanly and resumes next time (it already skips days it has stored). Without
|
|
this, "全部历史" is a single unbounded burst no matter how well spaced.
|
|
|
|
Both apply at one choke point — a proxy around the client object — so every
|
|
call site is covered, including ones added later. Anything that reaches Garmin
|
|
without going through the client is not paced, which today means only the
|
|
login/token calls on `client.garth`: a couple per sync, not thousands.
|
|
"""
|
|
import functools
|
|
import os
|
|
import threading
|
|
import time
|
|
|
|
|
|
class BudgetExhausted(Exception):
|
|
"""This run has issued as many requests as it is allowed.
|
|
|
|
Not an error: the caller keeps what it stored and continues next run.
|
|
"""
|
|
|
|
|
|
def min_interval():
|
|
"""Seconds between requests. Read per call so it is tunable without a
|
|
restart, and so tests can set it to 0."""
|
|
return float(os.environ.get("GARMIN_MIN_INTERVAL_SECONDS") or 0.5)
|
|
|
|
|
|
def default_budget():
|
|
"""Requests one sync run may issue.
|
|
|
|
1200 at 0.5s is about 10 minutes of wall clock — long enough to cover four
|
|
months of history in one go (9 requests per day), short enough that a full
|
|
backfill is a handful of background runs rather than one long hammering.
|
|
"""
|
|
return int(os.environ.get("GARMIN_REQUEST_BUDGET") or 1200)
|
|
|
|
|
|
class Limiter:
|
|
"""Per-account spacing and per-run request accounting."""
|
|
|
|
def __init__(self, interval=None):
|
|
self._interval = interval
|
|
self._lock = threading.Lock()
|
|
self._next_at = 0.0
|
|
self.used = 0
|
|
self.budget = None
|
|
|
|
@property
|
|
def interval(self):
|
|
return min_interval() if self._interval is None else self._interval
|
|
|
|
def start_run(self, budget=None):
|
|
"""Begin a run. `budget` None means unlimited (a short, known-size job)."""
|
|
with self._lock:
|
|
self.used = 0
|
|
self.budget = budget
|
|
|
|
@property
|
|
def remaining(self):
|
|
if self.budget is None:
|
|
return None
|
|
return max(0, self.budget - self.used)
|
|
|
|
def acquire(self):
|
|
"""Claim a slot, waiting for it if necessary."""
|
|
with self._lock:
|
|
if self.budget is not None and self.used >= self.budget:
|
|
raise BudgetExhausted(
|
|
f"本次同步已达请求预算上限({self.budget} 次)"
|
|
)
|
|
self.used += 1
|
|
now = time.monotonic()
|
|
wait = self._next_at - now
|
|
# Reserve this call's slot before releasing the lock, so two
|
|
# threads get consecutive slots rather than the same one.
|
|
self._next_at = max(now, self._next_at) + self.interval
|
|
# Slept outside the lock: two callers should wait out their own
|
|
# (non-overlapping) slots concurrently, not queue behind each other's.
|
|
if wait > 0:
|
|
time.sleep(wait)
|
|
|
|
|
|
_limiters = {}
|
|
_limiters_lock = threading.Lock()
|
|
|
|
|
|
def limiter_for(user_id):
|
|
"""The limiter for one account, created on first use.
|
|
|
|
Per account and per process. Two Gunicorn workers therefore pace
|
|
independently — acceptable because a given account syncs from one place at
|
|
a time (the scheduler claims its tick through the database, and a manual
|
|
sync sets the status to `syncing`), and the budget bounds the damage if
|
|
that ever stops holding.
|
|
"""
|
|
key = user_id or "_anon"
|
|
with _limiters_lock:
|
|
found = _limiters.get(key)
|
|
if found is None:
|
|
found = Limiter()
|
|
_limiters[key] = found
|
|
return found
|
|
|
|
|
|
class PacedClient:
|
|
"""Attribute proxy that paces every public method call.
|
|
|
|
Non-callables (`display_name`, `garth`) pass through untouched, as do
|
|
private names, and attribute writes reach the wrapped client — `_connect`
|
|
sets `client.display_name` and the library reads it back on every request.
|
|
"""
|
|
|
|
def __init__(self, inner, limiter):
|
|
object.__setattr__(self, "_inner", inner)
|
|
object.__setattr__(self, "_limiter", limiter)
|
|
|
|
@property
|
|
def limiter(self):
|
|
return object.__getattribute__(self, "_limiter")
|
|
|
|
@property
|
|
def inner(self):
|
|
return object.__getattribute__(self, "_inner")
|
|
|
|
def __getattr__(self, name):
|
|
attr = getattr(object.__getattribute__(self, "_inner"), name)
|
|
if name.startswith("_") or not callable(attr):
|
|
return attr
|
|
limiter = object.__getattribute__(self, "_limiter")
|
|
|
|
@functools.wraps(attr)
|
|
def paced(*args, **kwargs):
|
|
limiter.acquire()
|
|
return attr(*args, **kwargs)
|
|
|
|
return paced
|
|
|
|
def __setattr__(self, name, value):
|
|
setattr(object.__getattribute__(self, "_inner"), name, value)
|
|
|
|
def __repr__(self):
|
|
inner = object.__getattribute__(self, "_inner")
|
|
return f"<PacedClient {inner!r}>"
|
|
|
|
|
|
def pace(client, user_id, budget=None):
|
|
"""Wrap `client` so its calls are spaced, and start a run.
|
|
|
|
Idempotent: wrapping an already-wrapped client returns it (with the run
|
|
restarted) rather than stacking two layers of waiting.
|
|
"""
|
|
limiter = limiter_for(user_id)
|
|
limiter.start_run(default_budget() if budget is None else budget)
|
|
if isinstance(client, PacedClient):
|
|
return client
|
|
return PacedClient(client, limiter)
|