031-peer-source-mix

architecture
accepted

Peer source mix and connection malus

Context

Peer selection maintains four outbound candidate sources: static, shared (peer-sharing), snapshot (big-ledger file), and ledger (live registrations), plus inbound Using slots filled by promoting duplex inbound connections. A hard-coded waterfall (static → shared → snapshot → ledger) starves later sources whenever earlier ones can fill target_upstream_peers (#1180).

EDR-030 stores connection failure_count and a sticky adversarial flag for peer-sharing filters, but regulate_peers only skipped cool-downs and already-outbound peers. Failed dials were immediately eligible again; maintenance outages had no time-based rehab for outbound selection.

Admins need a small formula for the desired source mix that:

  • does not lock the set of source names forever;
  • handles peers that appear in multiple sources;
  • composes with a forbidden cool-down (hard) and a quality signal (soft);
  • lets connection failures fade even when the peer is not re-tried for a while.

Decision

Mix formula (peer-mix)

One string configures floors, proportional weights, and optional per-source malus half-lives:

peer-mix = item ("," item)*
item     = entry | naked_decay
entry    = name floor? weight? decay?
naked_decay = "@" duration                        # sets default half-life for *following* entries
name     = static | shared | snapshot | ledger | inbound   # registry; unknown ⇒ config error
                                                 # inbound is Using slots filled by promoting
                                                 # duplex inbound connections, not a dial pool
floor    = "!" uint                               # minimum slots if eligible peers exist
weight   = "~" uint                               # proportionality (not numeric equality)
decay    = "@" duration                           # malus half-life for this source only
duration = N"s" | N"m" | N"h" | N"d"

Examples:

static!2@2h, shared~6@6h, snapshot~8@12h, ledger~4@24h
@12h, static!2, shared~6, snapshot~8, ledger~4@48h
ledger~1@48h, static~0, shared~0, snapshot~0

A naked @duration (no source name) updates the running default half-life for subsequent entries that omit @…. A per-entry @… overrides that default for that source only and does not change the running default. The initial running default is 6h.

Defaults when a field is omitted:

FieldDefault
!floor0
~weight1 if the entry is present (including floor-only static!2); use ~0 to exclude a source from proportional fill
@decay on an entrycurrent running default (from the last naked @…, else 6h)

No = for weights (reads as equality). No arithmetic, conditionals, or nested groups.

Allotment and spill

For open = target_upstream_peers − using_occupancy eligible peers (using_occupancy is pending Host/SRV resolution plus Using outbound plus Using inbound):

  1. Assign floors in declaration order, each capped by eligible count and remaining open slots.
  2. Distribute any remainder proportionally by ~weight (largest-remainder), only among sources with weight > 0 and remaining eligible peers.
  3. Spill: any still-unfilled open slots are filled in declaration order from sources that still have eligible peers (empty / short buckets never leave permanent holes when another source can fill).

Multi-source membership

Each peer has a single canonical origin for mix accounting (fixed priority, independent of formula order):

static > shared > snapshot > ledger

A peer counts only toward its canonical source’s allotment.

Hard forbid vs soft quality

RuleKind
Active cool-downHard — never dial
Already in outbound_peersHard
Connection / protocol failureNot offered while evolved malus is at least the share threshold (0.05); below that, soft malus still lowers sampling weight
AdversarialCool-down is hard for the ban window; after that, dial is allowed. Sticky adversarial remains only for peer-sharing filters (EDR-030)
Observed scores (lag, fetch, …)Soft — goodness for ranking

Where sources and mix live

Candidate pools (static, shared, snapshot, ledger) and the admin PeerMix live in the Performance resource, not in the peer-selection stage state.

Reasons:

  1. Consistent malus physics — every evolve (impulse, share check, outbound ranking) uses the half-life of the peer’s canonical source from the mix (@… or default 6h). Performance must know source membership to pick τ.
  2. TraceBuffer / message size — large peer sets must not ride pure-stage stage state or high-frequency messages; the resource is outside the stage graph’s serialized footprint.

Peer selection keeps only connection maps, cool-downs, and targets. It calls Performance for select_outbound, select_share_peers, ingest_shared_peers, set_ledger_candidates, etc.

Static/snapshot pools and the mix formula are fixed at Performance::with_peer_sources construction (node resource registration). They are never reconfigured on the live resource. Live ledger candidates and shared peers still update via effects (set_ledger_candidates, ingest_shared_peers).

Lazy-decay connection malus

Performance stores (malus, malus_as_of) per peer (plus optional telemetry counters).

malus(now, τ) = malus_as_of × 0.5^((now − as_of) / τ)

where τ = half-life for the peer’s canonical source (from peer-mix), same for:

  • connection-failure / adversarial impulses (evolve then add, store at now);
  • outbound quality scoring;
  • share eligibility (malus(now, τ) < share_threshold).

Decay proceeds even if the peer is never selected again until the next touch/event.

Impulses (policy constants, not admin surface v1):

EventImpulse
Connect failure+8
Adversarial (with cool-down)+12

Peer-sharing also requires advertisable and !adversarial (sticky adversarial is permanent for sharing only).

Within-bucket selection

After allotting n slots to a source:

  1. Eligible = canonical members of that source, not outbound, not cooling.
  2. For each candidate compute
    score = goodness(scores) − λ · malus(now, τ) [+ never_connected_bonus]
    Never-connected / no Performance record: small positive bonus (exploration of fresh addresses).
  3. Convert to sampling weights w ∝ exp(score / T) (temperature T fixed policy).
  4. Fill up to n peers from best score to worse. Ties are a weighted draw. A worse score is used once every better candidate in the bucket has a slot, so a connect failure is not preferred while a healthier candidate remains, but it is dialled when the bucket would otherwise stay short. A peer is not dialled again within 2 seconds of a connect attempt or a connection failure; that hold-off lives in peer selection, not in the score.

Ownership

ConcernOwner
Source pools, mix, allotment, quality sample, malus τPerformance resource (EDR-030)
Cool-downs, outbound/inbound connection maps, targetsPeer selection
Formula parse / registryPerformance (PeerMix)

Consequences

  • Config grows a peer-mix string (CLI / env); invalid syntax fails at startup.
  • regulate_peers becomes mix → quality-weighted sample, not a waterfall.
  • Sharing rehab follows malus decay; adversarial peers stay non-shareable until a future explicit clear (not part of this decision).
  • Adding a source means a new registry name + pool wiring; old formulas remain valid if they omit the new name (weight 0 for absent names).

Default formula

static!2@15m, inbound~6, shared~6, snapshot~3@1h, ledger~3@24h

Future work

  • Admin knobs for impulses, λ, T, share threshold (if operators need them).
  • Align churn demotion (EDR-030) with the same malus/goodness axes.