tootsies

a discord bot for the tootsies server. ask, recap, discuss, ship features by typing.

Project maintained by mejasonmejason Hosted on GitHub Pages — Theme by mattgraham

Billboard alerts: when we fire, and what it looks like

The plan for the alert lanes on top of utils/billboard.py + the HITS forecast feed. Two jobs, deliberately separated:

Parity — report what everyone has. The weekly chart, the #1, the top-10 debuts. Table stakes. If we don’t have this we’re not a chart account.
Edge — the things the accounts copying each other structurally cannot post. This is where the sources research pays off, and it is most of this document.

Every timing claim below is measured, not assumed; the probes are in docs/BILLBOARD_SOURCES.md.

1. The race we cannot win — and shouldn’t enter

Measured Monday 2026-08-03 22:00 UTC:

Source	Chart it was serving
`@PopBase` on X	Aug 8, 2026 (posted ~5h earlier)
`billboard.com/charts/hot-100/`	Aug 1, 2026
GitHub JSON mirror	Aug 1, 2026

The wire has the top 10 about a day before any feed does. Billboard hands its headline chart news to press/socials before the chart pages flip, so a poll-only lane is structurally last. No amount of polling fixes this — it is a publishing decision, not a latency problem.

So: do not build a lane whose job is to beat PopBase to “here’s the top 10.” We would lose every week, and lose visibly. The wire is the trigger; the feed is the truth.

What the copiers all share is that they post the top 10, on Sunday/Monday, after the chart exists. Every edge below is a different axis: earlier, deeper, or reconciled.

2. The lanes

Lane A — Chart drop (PARITY)

Trigger: billboard.chart() returns a week_of newer than the last one posted.
Timing: Tuesday-ish, when the pages flip. Deliberately not racing the wire.
Fires: once per chart per week.
Edge: none. This is the floor.

Because we’re second by design here, the post should not pretend to be breaking. It should carry what a repost can’t: full context per row (peak, weeks on, the streak).

PREVIEW (mock — illustrative, not model output):
  hot 100, week of aug 1.
  ella langley's 40th week on the chart and 16th at #1. taylor holds at 2.
  lil baby debuts at 18. BTS re-enters at 33.

Lane B — The called shot (EDGE: ~5 days early) ⭐

This is the highest-value lane and the reason the HITS source matters.

HITS Daily Double’s building chart projects first-week units for the chart that has not published yet. We can call the #1 — with a number — days before Billboard prints it, and then be publicly right.

Trigger: a new hits-top-50 or midweek-20 doc, with a NEW entry in the top 5 (or a projected #1 change).
Timing: Tuesday (midweek) and Thursday/Friday (building) — ~5 days before the chart page flips.
Fires: ~1–2/week.
Edge: we are first by days, not seconds, and it is a falsifiable prediction rather than a repost. Nobody screenshots a chart repost; people screenshot a called shot.

This is validated on real data, both directions:

Retrospective (it works): HITS on Jul 25 projected GRACIE ABRAMS | DAUGHTER FROM HELL — NEW at #1, 123,975 units. The Billboard 200 dated Aug 1 — which is what billboard.com serves right now — printed #1, Daughter From Hell, DEBUT. Called correctly, days ahead.

Live and unresolved as of writing: HITS on Jul 31 projects CHARLI XCX | MUSIC, FASHION, FILM — NEW at #3, 74,403 units (52,027 pure album sales, 22,272 streaming). The Billboard 200 dated Aug 8 has not published. That is a callable shot right now, scoreable within ~48h — and the natural first live test of this lane.

PREVIEW (mock):
  charli xcx is tracking for a #3 debut. ~74k first week, 52k of it pure sales.
  chart doesn't print till friday. we'll see.

The honesty rule matters here: a projection is stated as a projection, and the lane is worthless if it hedges into meaninglessness or overclaims into being wrong. Fail CLOSED — no HITS doc, no called shot.

As built. A lane in cogs/music_desk.py, ranked second in _LANE_PRIORITY — behind the printed chart (a harder fact) and ahead of everything else, because it is the most perishable thing the desk carries: its whole value is the lead over Billboard’s print, and it is worth nothing once the chart lands.

Callable = a projected DEBUT (last_week is None) inside the top 10. A climber is real data but not a called shot; the claim we can be publicly right about is “this lands at #N”.
One call per release per projected chart week. The dedup key carries the chart date, not the doc, so the Tuesday midweek wins the post — being first, mechanically — and the Thursday building chart can only add releases the midweek’s top 20 didn’t carry.
One per slot, and the cap counts units that SHIP, not attempts, so a row held under the self-gate cannot starve the rows behind it.
Its own rollout stage (music_called_shot, default STAGING). It is the one desk lane that makes a falsifiable public claim, so a mod auditions the calls in #bot-logs before they reach the room without muting the rest of the desk. OFF skips the lane entirely — no HITS read, no compose.
Card: the unified big-number card with the projected rank as the figure, rendered projected=True so it wears the forecast treatment, credited to HITS, no market link.

Live dry run (2026-08-04, real Sanity pull + real compose + the real 0.6 gate), on the Charli XCX call above: 6/6 ship at 0.92, every number matching the source, no career or trajectory claims. The framing needed two rounds to get there, both caught by the dry run and invisible to unit tests:

the first version produced wire copy — “…is tracking a #3 debut on the Billboard 200, per Hits Daily Double, with ~74.4K projected units, 52K of it pure sales” — four near-identical headlines with the trailing press-release tag that _CONTEXT already bans for Kalshi;
the block and the framing were fighting: the block said “credit them by name” while the framing banned the trailing tag, so the model kept reaching for it.

Aligning both on “credit HITS inside the sentence, in your own words” fixed it: 0/6 used the banned tag afterwards. The self-gate could not have caught this — it grades grounding and accuracy, and the copy was accurate the whole time.

Lane C — Deep cuts (EDGE: depth)

Everyone posts the top 10. We parse all 100 (and all 200). Rows 11–100 are uncontested territory and they’re where the actual movement is.

Trigger: movements(week, min_notability=...) on rows outside the top 10.
Timing: with Lane A.
Fires: 2–4/week at a sensible floor.
Edge: not speed — coverage. Structurally unavailable to anyone reposting a top-10 graphic.

Real examples off the Aug 1 Hot 100, none of which any top-10 post contains: a 31-spot climb to #26 (Gracie Abrams, “Hit The Wall”), a 25-spot climb to #17 (Shakira, the World Cup song), a re-entry at #33 (BTS), an 11-spot fall at #42 (Beyoncé).

Lane D — Reconciliation (EDGE: nobody else has both numbers)

We hold three numbers for the same release that rarely agree:

HITS projection (before)
Kalshi/Luminate implied or settled units (utils/luminate.py, already integrated)
Billboard published chart position

The gaps between them are a story only someone holding all three can tell — and it’s inherently accountable, which is the tone that earns trust.

Trigger: a projected release resolves on the published chart.
Timing: Sunday/Tuesday, after Lane B’s call.
Fires: ~1/week.
Edge: unique. Also the mechanism that keeps Lane B honest — we score our own calls in public.

PREVIEW (mock):
  called charli at 74k. luminate settled 81.2k, billboard printed 76k.
  the filter took 5k. projection was light but the debut spot was right.

Lane E — Radio early warning (EDGE: leading indicator)

HITS’ Mediabase add_chart / building_chart update multiple times daily (4 minutes stale at probe). Radio adds today become airplay chart movement in ~2 weeks and Hot 100 movement after that. It is the only genuinely leading signal in the stack.

Trigger: an adds spike or a large plays_this_week jump for a song not yet in the Hot 100 top 40.
Timing: daily; weeks ahead of the chart.
Fires: ~1/week, tight threshold.
Edge: predictive, and essentially uncovered on X.

This lane is the most speculative of the five — treat it as staging-first until the correlation is checked against a few weeks of real outcomes. Ship it dark.

3. When we fire: thresholds

Movement.notability collapses rank × kind × move-size into one number so a lane thresholds on a single value. Measured on the real Hot 100 week of 2026-08-01 (reproduce with python -m scripts.dryrun_billboard):

Floor	Alerts that week	Use
0	45	the weekly report (Lane A) — everything that moved
5	6	Lane C default — a few real stories
10	4	tighter
20	2	only the big ones

Calibration points: a #1 debut scores 200, a top-10 debut ~22, a new peak at #3 42, a 25-spot climb to #17 13, a new peak at #32 4.

Always fires regardless of score: a #1 change, any top-10 debut. Never fires: a new peak outside the top 40 (true of nearly every climbing song), a sub-10-spot move, anything already posted this week (the existing post_dedup_history spine), and any week whose rows are flagged stale (no degraded source exists today; the guard stays).

Not a trigger, deliberately: “an artist’s first career #1.” It reads like the best alert on the list, and we cannot detect it — a chart row carries position, movement, peak and weeks, never career history. That is the same gap the compose fence closes (§2 status notes): asserting it would be guessing. It becomes available only if we add a career-history source, and it should not be faked from a single chart page.

3b. Reconciling with the surfaces we already have

Billboard/HITS is the fourth music-numbers source in a room that already had three, so the real risk isn’t a missing feature — it’s the same fact posted twice from different sources. Mapped out explicitly:

Surface	What it says about a chart move	Collides?
`music_news` (newsroom)	RELAYS the move off `@billboardcharts` / `@chartdata`	Yes — directly
`music_desk` reveal/tracking	Luminate UNITS behind the release (Kalshi-settled)	Adjacent, same subject
`music_desk` projection	Kalshi-IMPLIED first-week number, mid-week	Overlaps HITS (Lane B)
`market_alert` / betting	Price moves on a market the chart settles	Adjacent, same subject

The one live collision, now fixed. The newsroom posts “X debuts at #1 on the Hot 100” relayed from the wire; the chart lane posts the same move off the first-party feed. Before this, WIRE_DEDUP_SURFACES covered the wire desks but not music_desk — the desk only ever posted market numbers, which rarely collided with a headline — and the desk read only its own history. So the two surfaces could restate each other with neither able to see it. The chart lane is what made that live, so music_desk now both appears in the set (the newsroom sees the desk’s posts) and reads it (the desk sees the newsroom’s). Both directions are needed; either alone leaves one surface blind. The Haiku topic-dedup is what actually catches it, since a relay and a first-party read of one move are worded differently by construction.

The adjacent-but-distinct ones are fine, and shouldn’t be merged. A Luminate units reveal (“140.5K first-week units”) and a chart position (“#1 on the Billboard 200”) are genuinely different facts about the same release — Billboard’s published number is filtered down from Luminate’s, which is the whole reason the desk can report a figure nobody else has. Suppressing one for the other would lose information. The 36h topic window plus the judge’s “genuine development vs restatement” call is the right tool here, not a hard rule.

Where the overlap was real — projections. DECIDED: HITS wins, Kalshi yields. The desk’s existing projection lane posts a Kalshi-implied first-week number; Lane B posts the HITS one for the same release in the same week. Those are two estimates of one quantity, and posting both as separate stories reads as the desk repeating itself.

The rule, implemented in MusicDesk._hits_covered_albums: any release HITS is projecting is spoken for by the called-shot lane, and the Kalshi projection lane skips it. HITS is the industry’s own tracking estimate; Kalshi’s is a market’s guess at what that estimate will say. When we hold the real projection, the market read is the weaker of the two and adds no fact. Nothing is lost — the market number still drives market_drop and market_alert, which are about the market itself rather than the release. The suppression fails open: if HITS is unreadable, the Kalshi lane behaves exactly as it did before Lane B existed.

The richer “both numbers in one sentence” shape (“HITS says 74K, the market’s at 68K”) is Lane D, and it stays Lane D. It needs a matcher from a HITS album to a Kalshi event plus its own honesty rules about disagreeing sources, which is a bigger change than choosing which of two lanes speaks.

The newsroom’s chart gate is the other open piece — and the biggest. cogs/music_news.py:_live_chart_position verifies a wire chart claim against live rows, but charts_named() returns () for a Billboard claim (its comment says so outright: “A chart we can’t read (Billboard/ARIA) → web verify, as before”), so those fall through to a web search. That was correct when we had no Billboard feed. Now we do, and it is the cheapest remaining win in the whole epic: 30 of 118 claim lookups unresolved over 30 days, largely because Billboard claims were being checked against Spotify rows. Wiring bot.billboard into that gate is deliberately NOT in this PR — it changes how wire claims get verified and dropped, which is its own behavioural change with its own risk surface, and it deserves its own diff and dry run rather than riding along on a lane.

4. Where this rides

Nothing here needs new infrastructure. Per CLAUDE.md’s cross-cutting spines:

Scheduled lanes ride ScheduledPoster (master switch, mood, /menu calendar, slot pacing, dedup, 429 retry) — not a new scheduler.
Post-text dedup uses the one post_dedup_history table — no new *_history table.
Lane B and Lane E ship behind the experiments.py registry (staging = #bot-logs audition) because they make falsifiable public claims. Lane A/C can go to production.
Numeric floors (min_notability, the adds threshold) are Tunables in utils/tunables.py, so they’re tuned from /menu rather than by redeploy.
Every lane fails CLOSED on missing data: no chart, no post. Prefer absent over invented.

5. Status

Piece	State
Billboard chart read + parse + guards + loud blocked telemetry	built (`utils/billboard.py`)
`movements` / `diff_weeks` / `notability` derivations	built, calibrated on a real week
Dry run over every chart, and over the real lane compose	built (`scripts/dryrun_billboard.py`, `--compose`)
Lanes A + C as a music-desk lane, with cards	built — `cogs/music_desk.py`, the desk’s first-priority lane
Chart standing in the prediction/market context	built — `billboard.chart_context` → the desk’s compose
Being first (two-speed cache)	built — `expected_chart_date` + a ~20min re-read while a flip is due, + the read-time `superseded` check that makes it actually work
HITS Sanity client (Lane B/E source)	built — `utils/hits.py`; robots question resolved, see `BILLBOARD_SOURCES.md` §5b
Lane B (called shot), with cards	built — `cogs/music_desk.py`, the desk’s second-priority lane, own `music_called_shot` stage (ships STAGING)
Being first on the projection (publish-window cache)	built — `hits.expected_tracking_end` + `superseded`, the `utils.billboard` pattern
Lane D (reconciliation), E (radio)	not built — the data is in place for both
Newsroom chart-gate switched to Billboard for Billboard claims	not built — the cheapest remaining win (25% of claim lookups unresolved)

How the built lanes ride the desk

Owner steer: “this should ride music desk, and have its dedicated prioritized lane.” Same call as the streaming-milestone lane (#1495, “it should be on music desk not its own”), so it is a lane in cogs/music_desk.py, not a cog.

Priority is positional. _sched_compose_units returns an ordered list the base delivers in order, and the chart lane is built first. On a week where the chart moved, the chart leads the slot; on a quiet week the lane returns [] and the desk behaves exactly as it did before. No new scheduler, no new kill switch, no new dedup table — it reuses music_desk_events under a bb:<chart>:<week>:<title> key.

Being first is a cache property, not a scheduler one. expected_chart_date knows which Saturday-dated chart should be current (verified against the live flip). Two things use it, and both are needed:

the read path (superseded) — a cached chart is served only while it is the current week. Once a flip is due, the copy we hold is out of date, so the client goes and looks.
the write path — a chart that is behind the due week is cached for ~20 minutes; the current week is held for hours.

The first slot after a flip already has the new week, most of the week is one read per chart, and the extra reads are bounded to the flip window.

Why the read check exists, and why the TTL alone was not enough. DurableCache’s L1 tier is a plain LRU with no expiry: utils/lru.py stores no timestamps, and get() returns an L1 hit without checking any deadline. A short TTL therefore only shortens the life of the durable row — the in-process copy is served until the bot restarts. Shipped that way, the whole mechanism above was decorative: the first read of the day pinned last week’s chart in memory and the lane would never have seen the flip. Fixed by checking the chart DATE at read time, which is also the stronger question — it asks whether what we hold is the current week, not how long ago we asked. Regression test: test_a_superseded_chart_is_refetched_not_served_from_cache. The same L1 hole affects every other DurableCache consumer (see the tracking issue); this fix is Billboard-local on purpose, because changing the shared cache changes upstream call volume on metered APIs.

Same voice, same floor. The lane runs the identical compose_market_drop + music_desk_score path as every other desk story and the same 0.6 ship floor — it gets priority, not a lower bar. Live dry run: a #1 debut scored 0.95 and a new peak 0.92.

One fence the dry run forced. Given a #1 debut the model wrote “Gracie Abrams’ first chart-topper” — plausible, ungrounded, and not a number, so the number-only fence missed it. A chart row carries position, movement, peak and weeks and nothing else, so any career or record claim is invented by construction. The framing now forbids them explicitly: 0/6 samples carried one afterward, against a hit on the very first sample before.

What the HITS client makes possible now

projected_chart_date() is the piece that turns a projection into a scoreable call: a doc’s date is the tracking-week end (always a Thursday) and the Billboard chart it feeds is dated 9 days later. That rule was verified against a known outcome — the doc dated 2026-07-23 projected Gracie Abrams at #1 as a NEW entry, and the Billboard 200 dated 2026-08-01 printed exactly that — so “were we right” is now billboard.chart(week=proj.chart_date), a lookup rather than a judgement. It is also what makes Lane D (reconciliation) mechanical rather than manual.

Verified live at build time: the current building chart calls the chart dated 2026-08-08, billboard.chart("billboard200", week=2026-08-08) returns nothing (unpublished — so the call is live), and the top projected debut is CHARLI XCX | MUSIC, FASHION, FILM at #3, ~74.4K units (52K pure).

Two properties of the rendered block are deliberate and belong to the block rather than to a prompt, because a caller might not pass one: the “PROJECTED, not final” framing (a projection stated as a fact is the one way this lane actively misleads) and the HITS credit (an uncredited number is the one way it’s unfair to them).

Order of remaining work: the newsroom gate fix, then Lane B — the called shot, still the highest-value lane and now fully sourced.