# BFM apps2 — v2 Architecture

> The decoupling / cross-asset crystal-ball engine. Drafted 2026-06-25.
> Supersedes the ~40-app v1 sprawl (`apps/`). Read this before building anything here.

---

## Mission

Surface the **hottest alpha** by detecting **relationships breaking** — not levels moving.
The edge is in **extremes, decouplings, and decorrelations** across assets and across
timescales (micro → macro), and in projecting the **chain reactions** one unusual event
sets off. v1 computed levels and threw away time; v2 records everything, computes the
leading-edge transforms centrally, and ranks *what's turning right now*.

The north star example (the gold thesis): gold rose while real yields rose — the normal
gold↔real-yield link **broke**, revealing a price-insensitive official-sector bid. The
**volume** on that decoupled move scaled the coming USD suppression, which projected the
NAS100 repricing. v2 must catch that *as it begins*, at the smallest timescale it first
appears, and project the snowball with honest, widening uncertainty.

---

## 0. What changed from v1 (the whole point)

v1 = ~40 apps that each re-fetch the same chain/bars, compute one **level**, render a
**snapshot**, and forget it. No history → no velocity, no z-score, no divergence. The
fusion layer recorded a goldmine but only ever showed the latest snapshot; the real
early-warning engines (scanner/harbinger/forecast) were siloed from the ensemble.

v2 inverts it: **fetch once → record every signal → compute leading-edge transforms in
one shared library → fuse → rank heat.** Fewer screens, strictly more alpha, because the
leading edge is structural.

---

## 1. Design laws (the constitution)

1. **Record-first. No ephemeral path.** If a number is computed, it's written the same call. Un-recorded data is lost alpha — v1's #1 failure.
2. **Compute the chain/bars once per tick.** One snapshot per symbol feeds *all* signals → cross-signal consistency (same instant) + ~10× less I/O.
3. **Decoupling-first.** The primary object of study is the **relationship**, not the level. Every asset and pair is monitored for EXTREME / DECOUPLE / DECORRELATE at multiple timescales.
4. **Everything is a voter.** Every predictor emits `{dir, conf, target}` × 9 horizons. No siloed detectors; the chain projection and the radar are voters too.
5. **The leading edge is a transform, not an app.** velocity / accel / z-score / percentile / residual / corr-delta / regime-flip-distance is ONE shared library, parameterized by window.
6. **Multi-scale by default.** Every transform runs across a window-set (micro→macro). A signal climbing the timescales (micro decoupling → macro regime break) is the earliest tell.
7. **Default screen ranks heat, not a catalog.** You open to "what's turning right now," not 40 cards.
8. **Hazy by design.** Outputs are probabilistic with sample sizes and widening cones. Honest uncertainty beats false precision.
9. **The UI flashes state-changes and is fully tunable.** Breaks (decorrelation incl. inverse, decoupling, unusual trend, accel/decel) actively *flash the moment they fire* — not just sit in a heatmap. And every detector parameter (date range, candle/timeframe, algo-curve windows/thresholds) is live-adjustable so the user can tune the radar to match their own eyeball read of a turn (§9b).

---

## 2. The pipeline (strict one-direction flow)

```
 INGEST           STORE             DETECT / SIGNAL          FUSE             SURFACE
 ┌────────┐  ┌──────────────┐  ┌────────────────────┐  ┌──────────┐  ┌──────────────────┐
 │snapshot│─▶│ series store │─▶│ radar (decoupling) │─▶│ ensemble │─▶│ heat board +     │
 │ once   │  │ append-only  │  │ + voters emit      │  │ + calib  │  │ radar/chain/...  │
 │ /tick  │  │ one schema   │  │   dir/conf/target  │  │ + chain  │  │ lenses (render-  │
 └────────┘  └──────────────┘  └────────────────────┘  │ projection│  │  only, 0 compute)│
  chain+bars   every metric,     transforms READ the     └──────────┘  └──────────────────┘
  +fut+fx+      every tick,        store (not re-fetch);   P(up), flip,
  yields+perp   one flat schema    radar = the funnel      dispersion,
                                   front                   snowball cones
```

Key change vs v1: **signals read the store, not the API** — so every signal automatically
has history → velocity/z/residual/corr-delta are free.

---

## 3. Data sources (NO new paid subscriptions; add FRED, free)

| Need | Source | Status |
|---|---|---|
| Futures price **+ real volume** (GC, CL, NQ, ZN/ZB, DX) | **Databento GLBX** (daily/hourly batch; cPanel can't run live ws, and we don't need it — chain is multi-day) | ✅ have key; budget a small one-time daily backfill |
| Options chains+greeks+OI+IV (the options voters) | **Massive/Polygon** (`_common/massive.php`) | ✅ have key |
| Indices (^NDX, ^HSI), FX (DXY, USDCNH, EURUSD), continuous-future prices | **Yahoo** (free) | ✅ |
| Near-real-time US equity bars | **Alpaca** | ✅ have creds |
| **Real yields / breakevens** (`DFII10`, `T10YIE`, `DGS10`) — the gold-decoupling anchor | **FRED** (free API key, daily, back to ~2003) | ❌ **ADD — free, do first** |
| Crypto perp funding / OI / basis | **Hyperliquid** (free, unauthenticated) | ✅ free; currently unused |

Volume must come from the **futures** (Databento), never ETF proxies (GC volume ≠ GLD).
History depth: Yahoo prices + FRED yields go back decades; GLBX volume ~2010+ (covers the
key analogs: 2011 gold, 2015 yuan deval, 2018–19 trade war, 2020, 2022, 2024–25).

---

## 4. Schemas (three canonical stores)

**1. Snapshot (raw, per tick)** — one pull, stored verbatim so any signal is re-derivable/backtestable:
```
data/snapshots/<sym>/YYYY-MM-DD.jsonl
  {ts, spot, volume, atm_iv?, chain?:[{k,exp,type,iv,delta,gamma,vega,oi,vol,bid,ask}], bars:{1m,1h,1d}}
```

**2. Series (the product)** — every signal value, one flat schema; the thing every leading view reads:
```
data/series/<sym>/<signal>.jsonl
  {ts, h:"1h", dir:0.6, conf:0.55, value:1.1e9, target:750}
```

**3. Radar (the decoupling tensor, per tick)** — extreme/decouple/decorrelate × scale:
```
data/radar/<sym>.jsonl
  {ts, scale:"meso", extreme_z:2.4, residual_z:3.1, decorr:[{pair:"gold:real_yield", corr:0.05, corr_norm:-0.62, break:0.67}], heat:0.88}
```

**4. Fusion (the recorded ensemble, per tick)** — so the fusion itself has history:
```
data/fusion/<sym>.jsonl
  {ts, spot, horizons:[{h,p_up,dir,target,lo,hi,conf,contrib:{...}}], family:{hedging,vol,flow,chain}}
```

---

## 5. The transform library (`core/transform.php`) — multi-scale leading edge

Every transform takes a recorded series + a **window** and returns a comparable value.
The radar and fusion call these; no app re-implements them.

| Transform | What it surfaces |
|---|---|
| `level` | the raw value |
| `velocity` / `accel` | 1st / 2nd derivative — momentum & its change (the "pulse", now free for every signal) |
| `zscore(window)` / `percentile` | how extreme vs its own history |
| `residual(basket, window)` | actual − expected-from-drivers (rolling multi-regression) = **decoupling** |
| `corr(b, window)` / `corr_delta` | pairwise correlation and its **change** = **decorrelation** |
| `regime_flip_distance` | signed distance to a sign-flip line (zero-gamma, delta-flip, max-pain, corr=0) |
| `divergence(price)` | signal vs price disagreement |

**Window-set (the micro→macro spine):**
`micro = {1h, 4h}` · `meso = {1d, 5d, 20d}` · `macro = {60d, 120d, 252d}`.
Every transform runs across the set so the same break can be flagged at multiple scales.

---

## 6. The Decoupling Radar (`core/radar.php` + `lenses/radar`) — the universal anomaly front

Generalizes the gold insight into a radar over **every asset and every pair**, monitoring
three break-types at three scales. This is the funnel that feeds the chain engine and the
heat board.

**The three break-types**
- **EXTREME** — one series at an unusual level/velocity vs its own history (`zscore`, `velocity`, `accel`). *"How unusual is this asset right now?"*
- **DECOUPLE** — one asset moving against what its drivers predict (`residual` from the rolling cross-asset regression). *"It's doing something its drivers can't explain."* (gold↑ while real-yield↑)
- **DECORRELATE** — a pair's rolling correlation breaking from its norm (`corr_delta`, corr regime flip). *"Two things that always move together just stopped."* (gold/real-yield −0.7 → 0)

**The radar tensor:** `[asset | pair] × [extreme | decouple | decorrelate] × [micro | meso | macro]`.
Any hot cell = a candidate trigger. `heat = |residual_z| · |corr_break| · velocity_z` (per cell).

**Cross-timescale climbing (the earliest tell):** the radar shows the *same* relationship
across micro/meso/macro side by side. A break that appears at **micro first and propagates
up** to meso then macro is the earliest possible warning that a structural regime change is
forming — catch it on the 1h scale before it's a 20d trend. The radar flags "climbing"
cells explicitly.

**Radar lens UI:** a tensor heatmap (rows = assets/pairs, columns = scale, cell = break-type
heat, with a small EXTREME/DECOUPLE/DECORR glyph), a "climbing now" rail (breaks ascending
the timescales), and click → the asset's multi-scale residual + corr-delta time series.

---

## 7. The Cross-Asset Chain crystal ball (`core/{graph,propagate,analog}.php` + `data/graph.json` + `lenses/chain`)

When the radar fires a trigger, the chain engine answers *"what snowball does this start, how
big, and when?"* Four parts:

1. **Causal graph** (`data/graph.json`) — editable map of cross-asset edges `{from, to, sign, lag, mechanism, trigger_condition, beta, calibrate}`. Domain knowledge as data. The gold→USD→NAS100 chain is encoded there.
2. **Anomaly/Residual** = the radar (§6) — supplies the trigger (which asset, direction, magnitude, which relationship broke).
3. **Propagation** (`propagate.php`) — walk the graph forward from the trigger: projected downstream move = `trigger_magnitude × calibrated_edge_beta`, lagged by `edge.lag`, with **confidence decaying per hop** (the haze). Only edges whose recent calibrated r² is strong are "armed"; weak edges grey out.
4. **Analog** (`analog.php`) — the honest crystal ball. Find historical days where this *same* anomaly+decoupling occurred (same asset, same broken relationship, similar magnitude); aggregate the realized forward paths of every downstream asset → an **empirical forward cone** (median + quantiles + sample size + hit-rate). *"Last 8 times gold spiked on volume while decoupling from real yields: DXY −3% med /3wk, NDX +6% med /6wk, n=8, 6/8."*

**Chain lens UI (the crystal ball screen):** (1) normalized z-scored overlay of all chain
assets on one axis so 0.3σ divergences show, trigger marked, projected cones drawn; (2) the
rolling **correlation-delta matrix** (earliest tell); (3) the **residual heatmap** (trigger
detector); (4) the **snowball panel** — trigger node → downstream nodes with projected
magnitude/lag (propagate) + analog cone overlaid, confidence widening per hop.

**It's also a voter:** `signals/chain/chain_projection.php` emits the downstream call (e.g.
NAS100 `{dir:+0.7, conf, target}`) into the fusion engine — so the macro snowball shows up on
the heat board and votes in Oracle alongside the options engine, weighted by its calibrated
hit-rate.

---

## 8. Signal families (replaces ~40 apps with ~20 small files)

Each signal = one file: read snapshot → compute → emit `{dir,conf,target}×9` + record its series.

- **`signals/hedging/`** — gex, dex, gamma_term, vanna_charm, oi_delta (per-strike ΔOI), futures_basis, perp_funding (Hyperliquid). *Dealer/bank forced flows.*
- **`signals/vol/`** — iv_surface, skew, vrp, term_structure, zerodte. *Fear pricing / protection demand.*
- **`signals/flow/`** — premium_cohort (retail vs inst), breadth, cross_asset, ratios, momentum, price_structure (squeeze/divergence). *Where capital actually is.*
- **`signals/chain/`** — cross_asset_resid (per-asset residual + decoupling as a voter), chain_projection (the snowball voter). *The macro crystal ball.*

`greeks-pulse` disappears as an app — "pulse" = `transform.velocity()` on every signal, for free.

---

## 9. Lenses (~6 thin frontends, render the store, ZERO compute)

| Lens | Purpose |
|---|---|
| **heat** | THE default screen. Every signal×symbol ranked by `heat = \|velocity_z\| · \|level_z\| · cross_signal_agreement · calibration_weight`. Tags: FLIP / ACCEL / DIVERGE / EXTREME / CLIMBING. |
| **radar** | the decoupling tensor (§6) — extreme/decouple/decorrelate × micro/meso/macro, climbing rail. |
| **chain** | the cross-asset crystal ball (§7) — overlay + corr-delta matrix + residual heatmap + snowball cones. |
| **fuse** | Oracle+Mosaic merged: P(up) ribbon over time, voter-flip timeline, dispersion oscillator, cross-horizon P(up) inversion, family-flip lead. |
| **evolve** | any signal over time: level ↔ velocity ↔ z-score ↔ divergence-vs-price toggle. |
| **surface** | any signal as strike×expiry×time surface with Δ/z toggle. |
| **scan** | multi-symbol board ranked by heat / flips / family-flip-lead. |

Shared `shared/chart.js` = one canvas engine carrying the full R-UI-20 contract (Y-scale
slider+gutter, pan, wheel-zoom, click→dateline x-synced, fullscreen, save-PNG) so no lens
re-implements charts.

---

## 9b. Frontend: live alerting (flash) + the eyeball simulator (`shared/alert.js`, `shared/controls.js`)

The lenses are not passive heatmaps. They actively **flash the moment a break fires** so the
eye catches the *beginning* of a turn, and they expose every detector knob so the user can
tune the radar to match their own eyeball read.

### A. Live alerting — flash on state-change (`shared/alert.js`)
A break **flashes (a transient pulse animation) at the instant it crosses its threshold**, then
settles into a static highlight while the condition persists (with a "since HH:MM" stamp), so a
persistent condition never drowns out a *new* one. Distinct visual language per break-type,
intensity scaled by `heat`:

| Break | Flash treatment |
|---|---|
| **DECORRELATION** | the pair's link pulses; corr-delta cell rings |
| **INVERSE correlation** (corr crosses 0 / flips sign) | stronger alarm — red ring + `INVERSE` tag (the highest-value tell: things that moved together now oppose) |
| **DECOUPLE** (residual spike) | asset cell flashes amber, `⊗` glyph |
| **UNUSUAL TREND / EXTREME** (z-extreme, new regime) | flash + `EXTREME` tag |
| **ACCELERATION** | up-chevron, brightening pulse |
| **DECELERATION** | down-chevron, dimming pulse |
| **CLIMBING** (break ascending micro→meso→macro) | a rising arrow across the scale columns — the earliest warning |

Plus a top **alert ticker / event log** rail: a chronological "what just fired" feed
("10:32 gold:real_yield DECORRELATED → INVERSE · meso · heat 0.88"), click → jumps to it, so
nothing is missed off-screen. Severity = heat; flash intensity scales with it. Palette follows
the house order (R/B/Y/G before O/P); honors a global **calm mode** / reduced-motion toggle and
an optional subtle audio/badge cue (off by default — minimal aesthetic).

### B. The eyeball simulator — live fine-tune controls (`shared/controls.js`)
A persisted control rail on every analytical lens that **re-drives the charts instantly from the
store** (no re-fetch — transforms are parameterized and recompute on read; this is *why* the
record-first architecture makes tuning cheap):

- **Date range** — start/end + quick windows; drives the whole view.
- **Candle / timeframe range** — resolution (1m/5m/15m/1h/4h/1d/1w) + candle width (per the house candle rules: shrink-to-fit then scroll).
- **Algo-curve params (every transform's knobs, as live sliders):**
  - z-score / percentile **lookback window**
  - correlation **window** + the "normal corr" **baseline window**
  - residual-regression **basket membership** (toggle which drivers each asset is regressed against)
  - **smoothing** (EMA span) on overlays
  - **break thresholds** (z, corr-break, accel) — these directly control *what flashes*
  - overlay toggles: moving averages, regression channel, bands, parabolic/curvature
- **Tuning presets** — save/load named parameter sets ("my swing eyeball", "fast intraday").

The goal: the user tunes window/threshold/basket until the flashing matches their own read of
"a trend is changing" — effectively **calibrating the detector to their intuition.** Because the
radar's transforms and the UI's sliders are *the same parameterized functions* over the recorded
series, the eyeball simulator is a first-class property of the architecture, not a bolt-on.

---

## 10. Directory layout

```
apps2/
  ARCHITECTURE.md          ← this file (the map)
  core/
    ingest.php             snapshot once → snapshots/  (cron, RTH-gated)
    store.php              the ONLY I/O module (append/read series, snapshots, radar, fusion)
    transform.php          ★ multi-scale leading-edge library (§5)
    radar.php              ★ extreme/decouple/decorrelate tensor (§6)
    graph.php              load + calibrate causal edges
    propagate.php          forward chain projection
    analog.php             historical analog matcher (the empirical cone)
    fuse.php               ensemble → fusion/ ; calibrate.php → weights.json
    contract.md            the voter + schema spec (single source of truth)
  signals/  hedging/  vol/  flow/  chain/
  lenses/   heat/  radar/  chain/  fuse/  evolve/  surface/  scan/
  shared/   chart.js  alert.js (flash-on-fire, §9b.A)  controls.js (eyeball simulator, §9b.B)  ui.js  swiss.css
  data/     snapshots/  series/  radar/  fusion/  graph.json  weights.json
```

---

## 11. Build order (each phase ships something usable)

1. **Spine** — `core/store.php` + `transform.php` + `contract.md`. Nothing works without it.
2. **Chain ingest** — pull the 7 chain assets (Databento daily GLBX + Yahoo + **FRED real yields**) with volume into `snapshots/`. Start recording immediately — every unrecorded day is lost history.
3. **Radar detector + flash + controls** — `radar.php` + the `radar`/`chain` lens panels 1–3 (normalized overlay, correlation-delta matrix, residual heatmap), wired to `shared/alert.js` (flash-on-fire, incl. INVERSE) and `shared/controls.js` (live date/candle/algo-curve tuning). *This alone is a working, tunable decoupling radar across micro→macro that flashes turns as they begin.*
4. **Crystal ball** — encode `graph.json`, add `analog.php` (empirical cone) + `propagate.php` (projection) + the snowball panel.
5. **Fusion + heat board** — `fuse.php` + `calibrate.php`, wire `chain_projection` + radar as voters; build the `heat` default screen + `fuse` meta-views.
6. **Backfill signal families** — hedging/vol/flow signals into the store; add `surface`/`evolve`/`scan` lenses.

---

## 12. Keep / rebuild / drop from v1

- **Keep & port:** the voter contract, Black-Scholes (`bs.php`), market-hours gating, Massive/Yahoo/Databento/Hyperliquid fetchers, Swiss CSS skin, the calibration concept.
- **Rebuild:** one unified store (not scattered JSONL types), one chart engine, fusion as a recorded time-series, the radar as a first-class detector.
- **Drop on purpose:** per-app re-fetching, per-app re-implemented z/Δ, the "ephemeral" path, greeks-pulse-as-app, the 40-card hub (→ heat board), seasonality-as-app (keep as a static note).

---

## 13. Honesty & limits

- The chain engine detects **broken relationships and matches analogs** — it does not assert causation. Output is always "this pattern historically preceded X, n=K, hit R/K," with cones widening per hop.
- Edge betas are **regime-dependent**; `graph.php` recalibrates on a rolling window and greys out edges whose recent r² is weak. Confident chains only when the recent relationship is stable.
- Rare macro events → small analog samples → wide cones. That haziness is honest and intended.
- Cadence: the chain is **multi-day**, so daily/hourly resolution is the right altitude. Pay for live/tick futures only if intraday chain detection is later proven necessary — not now.

> Companion specs: `data/graph.json` (the causal map), `core/contract.md` (voter + schema spec, written in Phase 1).