Перейти к основному содержимому

Retry DLQ Replay Matrix By Schema Family — Исполнительная матрица доставки, повторов и восстановления

Версия: 1.0
Дата: 24.04.2026
Статус: Готов к обсуждению

Назначение документа​

Этот документ фиксирует первую execution-level матрицу retry, DLQ, replay и recovery expectations по основным async schema families платформы.

Его задача — определить:

  • какие families требуют aggressive retry, а какие нет;
  • где DLQ является обязательным safety boundary;
  • где replay является нормальной planned operation, а где exceptional path;
  • как delivery discipline зависит от domain meaning schema family;
  • как эти правила соотносятся с observability, incident response, compatibility и release discipline.

Документ не является broker-specific operational handbook. Он является policy matrix для execution contours платформы.

Опорные документы​

Почему Этот Документ Нужен Отдельно​

После фиксации:

  • schema families;
  • channel catalog;
  • version evolution policy;

уже недостаточно просто знать, как выглядит событие и как его можно версионировать.

Платформе нужен следующий execution-layer, который отвечает:

  • сколько раз пытаемся повторять обработку;
  • когда переводим сообщение в DLQ;
  • какие families требуют replay-first recovery;
  • где silent retry недопустим;
  • где нужен operator escalation instead of endless retry.

Без этого even well-designed event contracts будут operationally unsafe.

Главный Принцип​

Delivery policy должна определяться не технической симметрией, а доменным риском.

То есть:

  • не всем schema families нужен одинаковый retry pattern;
  • не всё нужно endlessly retry;
  • не всё нужно replay automatically;
  • financial, booking and governance critical paths должны быть строже, чем low-risk integration signals.

Matrix Dimensions​

Для каждой schema family ниже фиксируются:

  • delivery criticality;
  • retry stance;
  • backoff stance;
  • DLQ stance;
  • replay stance;
  • operator escalation stance;
  • observability minimum.

Family 1. canonical-change​

Delivery Criticality​

Высокая. Эти события двигают downstream materialization and projection truth.

Retry Stance​

  • retry allowed;
  • bounded retries mandatory;
  • silent drop forbidden.

Backoff Stance​

  • exponential backoff preferred;
  • upper bound required.

DLQ Stance​

  • mandatory for non-repeatable failures;
  • quarantine path should be explicit.

Replay Stance​

  • first-class;
  • replay is normal recovery and regeneration tool.

Operator Escalation​

  • required when DLQ or quarantine volume exceeds baseline;
  • required when replay changes canonical outcomes materially.

Observability Minimum​

  • lag by producer/consumer;
  • replay volume;
  • DLQ count;
  • canonical change processing failures.

Family 2. offer-publication​

Delivery Criticality​

Высокая. Ошибка здесь напрямую влияет на external truth exposure.

Retry Stance​

  • retry allowed for projection and downstream publication consumers;
  • bounded retries only;
  • no infinite retry loops.

Backoff Stance​

  • short-to-medium backoff;
  • should avoid publication storm loops.

DLQ Stance​

  • mandatory when publication suppression/allowance signals cannot be applied safely;
  • operator visibility required.

Replay Stance​

  • strong replay support for projection rebuilds and publication recalculation;
  • replay should be scoped by entity/surface where possible.

Operator Escalation​

  • required when suppressed/allowed transitions fail to propagate;
  • required when stale publishable state may remain externally visible.

Observability Minimum​

  • stale publication lag;
  • projection rebuild failures;
  • suppressed-but-still-visible anomalies;
  • publication DLQ count.

Family 3. quote-lifecycle​

Delivery Criticality​

Очень высокая. Ошибки здесь могут directly distort commercial promise and booking admissibility.

Retry Stance​

  • retry allowed only in bounded form;
  • no unbounded repeat on invalidation/repricing-sensitive consumers;
  • consumer idempotency mandatory.

Backoff Stance​

  • short bounded backoff;
  • should converge quickly or escalate.

DLQ Stance​

  • mandatory for repeated processing failures;
  • quote invalidation/expiry processing must never disappear silently.

Replay Stance​

  • replay allowed but must be controlled;
  • replay may trigger repricing or UI/booking guard consequences.

Operator Escalation​

  • required when quote invalidation signals are not applied;
  • required when quote events diverge from booking guard behavior;
  • required when multiple versions coexist unexpectedly.

Observability Minimum​

  • quote event lag;
  • invalidation processing errors;
  • expiry job failure rate;
  • consumer mismatch by version.

Family 4. booking-transitions​

Delivery Criticality​

Максимальная. Это one of the most operationally and financially sensitive families.

Retry Stance​

  • retry only with explicit bounded policy;
  • recovery jobs separate from domain event reprocessing;
  • no endless retry on uncertain external state.

Backoff Stance​

  • short first retries, then escalated spaced retries;
  • delayed follow-up path preferred over tight loops.

DLQ Stance​

  • mandatory;
  • booking transition processing failures must be visible as incidents, not hidden queue noise.

Replay Stance​

  • replay extremely sensitive;
  • historical replay must be controlled and audited;
  • replay should not silently re-trigger operational promises without gating.

Operator Escalation​

  • mandatory for unknown_external_state;
  • mandatory for repeated consumer failures on booking.confirmed;
  • mandatory when support-facing state diverges from event flow.

Observability Minimum​

  • per-state lag;
  • unknown-state backlog;
  • recovery job age;
  • booking event DLQ count;
  • incident correlation with booking ids.

Family 5. post-booking-case​

Delivery Criticality​

Высокая. Это service-capability family.

Retry Stance​

  • retry allowed with bounded policy;
  • case-routing failures should escalate early.

Backoff Stance​

  • moderate backoff;
  • must respect SLA clocks.

DLQ Stance​

  • mandatory for unresolved processing failures;
  • post-booking support signals cannot disappear.

Replay Stance​

  • replay possible;
  • should preserve case ownership and historical audit trail.

Operator Escalation​

  • required when service owner or customer-impact state becomes ambiguous;
  • required when case events age beyond SLA without downstream handling.

Observability Minimum​

  • case event lag;
  • support-case routing failures;
  • SLA breach risk by queue age;
  • post-booking DLQ count.

Family 6. settlement-reconciliation​

Delivery Criticality​

Максимальная. Это financial-domain family.

Retry Stance​

  • retry allowed only with strict bounded policy;
  • correction/recompute jobs preferred over blind reprocessing when financial ambiguity appears.

Backoff Stance​

  • cautious bounded backoff;
  • must avoid duplicate financial side effects.

DLQ Stance​

  • mandatory;
  • financial processing failures must create explicit operator/finance visibility.

Replay Stance​

  • replay possible but highly controlled;
  • replay should be traceable and finance-reviewed when it can change discrepancy outcomes.

Operator Escalation​

  • mandatory on settlement posting failure;
  • mandatory on repeated reconciliation processing mismatch;
  • mandatory when clearing effect may diverge from booking reality.

Observability Minimum​

  • settlement event lag;
  • reconciliation queue growth;
  • duplicate processing suspicion signals;
  • financial DLQ count;
  • replay audit trail visibility.

Family 7. usage-governance​

Delivery Criticality​

Средне-высокая. Ошибки здесь опасны для partner control and enforcement, но обычно не так directly irreversible, как booking or settlement.

Retry Stance​

  • retry allowed with bounded policy;
  • enforcement signals should converge quickly.

Backoff Stance​

  • short-to-medium backoff;
  • no long silent drift for quota exhaustion signals.

DLQ Stance​

  • mandatory for enforcement-processing failures;
  • operator visibility required when tenant restrictions fail to apply.

Replay Stance​

  • replay possible;
  • usually scoped and less dangerous than financial or booking replay;
  • should not silently recalculate old governance decisions without note.

Operator Escalation​

  • required when exhausted/degraded states do not reach enforcement layer;
  • required when partner-facing surface contradicts governance state.

Observability Minimum​

  • usage governance lag;
  • enforcement mismatch counters;
  • throttling signal failures;
  • governance DLQ count.

Cross-Family Retry Rules​

Для всех first-wave schema families действуют:

  • bounded retry count;
  • no infinite poison cycling;
  • explicit transition to DLQ/quarantine;
  • idempotent consumer behavior where retry exists;
  • operator-visible terminal failure state.

Cross-Family Replay Rules​

Replay допустим только если:

  • family replay semantics documented;
  • replay actor and reason traceable;
  • replay does not silently distort historical interpretation;
  • downstream consequences understood;
  • observability can distinguish replay traffic from live traffic.

Cross-Family DLQ Rules​

DLQ is mandatory when:

  • message cannot be safely processed after bounded retries;
  • semantic mismatch or unsupported version detected;
  • payload missing critical required fields;
  • downstream effect could become unsafe if retried blindly.

DLQ is not a trash bin. It is an operator-visible holding zone.

Version-Evolution Interaction​

Если schema family changes version, retry/DLQ/replay rules must be reviewed explicitly.

Особенно:

  • new optional fields may be safe for retry, but not always safe for replay assumptions;
  • version overlap may increase DLQ risk for old consumers;
  • semantic drift may turn “safe retry” into unsafe repeated processing.

Incident-Response Interaction​

The following should open incident attention quickly:

  • DLQ growth on booking-transitions;
  • DLQ growth on settlement-reconciliation;
  • replay surge on canonical-change or offer-publication;
  • consumer version mismatch on high-value families;
  • unknown-state backlog growth in booking flows.

What Must Stay Out Of Scope​

Пока не нужно:

  • фиксировать exact retry numbers for every runtime;
  • описывать vendor-specific broker commands;
  • путать schema-family matrix с low-level ops runbook;
  • делать this matrix final for all future maturity stages.

Что Должно Появиться Следом​

Следующий практический слой после этого документа:

Этот документ связывает async contracts с execution discipline и закрывает важный operational gap между schema evolution и реальной обработкой событий.

Уточнение под Фазы 4–6 (28.04.2026)​

Retry / DLQ / replay matrix per schema family должна учитывать 6 классов событий (см. eventing-and-queue-baseline.md) с разными guarantees:

Класс eventRetry policyDLQ retentionReplay capability
Domainexponential backoff, max 5 retries30 днейfull replay из event log
Analyticalbest-effort retry7 днейreplay не критичен
Sagaguaranteed retry до compensation90 днейsaga state recovery
Audit / securityguaranteed retry, immutable storage7 летtamper-evident replay
Operational (DR/SLA/incidents)guaranteed retry1+ летreplay для analysis
Complianceguaranteed retry, immutable7+ летreplay для audit

Каноничные имена событий per домен — в специализированных документах (см. initial-event-taxonomy.md).

DR-совместимость:

  • domain events — Tier 2 (rebuildable из event log);
  • saga events — Tier 1 (transactional integrity);
  • audit log — Tier 1 + WORM;
  • compliance events — Tier 1.

См. disaster-recovery-and-capacity.md для recovery class mapping.

Уточнение выполнено через no-destruction.