When the Link Blinks: Flapping, Optical Margin, and the Metro Inflection

September 29, 2026

Link flapping is not a new phenomenon. In classical telecommunications a flapping port raised alarms, generated a truck roll and occasionally cost an SLA credit. AI infrastructure has changed that calculus, moving a failure mode from the operations backlog onto the critical path. This paper sets out what a flap is, what it costs in a training cluster, what causes it, and where the detection scheme changes the odds.

The Challenge in Brief

Engineers have managed unstable interfaces for as long as they have existed, and the response was proportionate: it was rarely existential, because the traffic above it was asynchronous and the applications retried. Flapping takes two forms.

1. Link/port flapping — an interface that repeatedly transitions between up and down in short intervals. On an optical link it surfaces as FEC loss-of-lock, a torn-down BFD session, re-convergence — and, in the harder cases, as a link that stays administratively up while quietly discarding frames.
2. Route flapping — a router repeatedly advertising and withdrawing a path, or oscillating between paths, in BGP or an IGP.

The two are connected: an unstable link is a common trigger for an unstable route. They are not the same failure and not fixed in the same place.

This paper addresses link/port flapping. That is the layer the optical interface governs: error-correction headroom, link budget and impairment tolerance are properties of the transceiver and the fiber. Route flapping sits downstream, and its remedies — dampening, timer tuning, topology design — belong to the control plane.

What changed is the damage. Collective operations are synchronous, so one stalled rank blocks every other. The unit of damage is not the port and not the rack — it is the training job. Inside an NVL72-class rack the scale-up NVLink domain is copper and untouched by an optical event; the damage propagates through the scale-out fabric. Alibaba’s term for the metric — GPUs degraded per fault event — is the fault explosion radius.

A flap is not a broken link. It is a link operating too close to the edge of its error-correction window.

Most Persistent Flaps Are Margin Events, Not Hardware Failures

The root causes look heterogeneous. Sorted cleanly, they fall into five buckets.

• Physical layer — end-face contamination, return loss, macrobend, connector misalignment.
• Optoelectronic — laser and TEC degradation, thermal excursion at high module power.
• Protocol and software — FEC lock behavior, link training, firmware mismatch, re-convergence.
• Host electrical interface — SerDes equalization, pre-emphasis set for the wrong trace length, channel variation between port positions in the same chassis.
• Operational — install quality, multi-vendor interoperability, cable churn during expansion.

The list is diverse. The mechanism behind the persistent cases is not. Physical-layer degradation, optoelectronic ageing and thermal excursion progressively consume margin — optical on the line side, electrical on the host side — until an otherwise recoverable disturbance crosses the FEC threshold. A contaminated connector costs a fraction of a decibel; a macrobend adds loss. None of them breaks anything. Once the operating point sits near the threshold, ordinary fluctuation carries the link across and back. That is a flap, and it explains why flaps cluster on specific links rather than distributing randomly: margin is a property of a path, not a fleet. Software, firmware and routing failures also produce flaps. They have different root causes and are not the subject here. It also explains the standard remedy. Flap suppression applies hysteresis — a lower threshold to declare the link down than to bring it back up. It repairs nothing; it widens the band the signal must cross twice.

The electrical bucket is the most underestimated. Port positions in a switch or router do not have equal trace lengths, so host channel loss differs port to port. A SerDes equalization and pre-emphasis setting that closes the eye on one port can leave another marginal. That link comes up, passes turn-up, and flaps later under thermal drift. Host-side PRBS generation and checking is how per-port equalization gets verified rather than assumed.

Figure 1: Interface and channel count per link: multi-lane IM-DD versus single-channel coherent. Each interface is a place where margin can be lost. Counts only — no per-device reliability claim is implied. Arycs Technologies analysis.

Coherent Detection Moves the Operating Point Away From the Cliff

Two margins are in play, and they are not interchangeable. Optical margin is the distance from the received signal to the receiver’s sensitivity boundary. FEC margin is the distance from the measured pre-FEC error rate to the decoder’s failure threshold. A link can hold adequate optical power and still run with little impairment margin, so for flapping the question is whether a transient pushes the instantaneous pre-FEC error rate across the FEC boundary. Three mechanisms move that boundary further away.

Headroom instead of a cliff. The standardized FEC families are not close together. KP4 hard-decision FEC, RS(544,514), corrects to a pre-FEC BER of 2.3E-4; a staircase code at 6.7% overhead reaches 4.5E-3; a concatenated code pairing a soft-decision inner code with a hard-decision outer code reaches 1.25E-2, at 3.9 dB more net coding gain (IEEE 802.3 B10K contribution, 2017). The honest comparison is the near-matched one: 5.8% to 6.7% overhead buys 20× the raw-error tolerance. Spending 14.8% buys 54×. The code does not change the receiver’s error curve; it moves the threshold under it.

Optical budget. OIF specifies the unamplified, single-wavelength coherent link at an 11 dB loss budget (OIF, 2020). A 1.5 dB penalty from a contaminated connector consumes about one seventh of it. On an IM-DD link engineered to a low-single-digit-decibel budget, the same 1.5 dB can be most of the margin.

Fewer places to lose it. One channel, one duplex LC, two end faces, against a multi-lane module with an MPO and 2N end faces. The coherent DSP also tracks chromatic dispersion, polarization rotation and carrier phase. Direct detect has electrical equalization of its own, but without access to the optical field it cannot offer the same general-purpose compensation of polarization and phase-dependent impairments, and its dispersion tolerance is more constrained.

One honest friction point: this headroom is selectable, not inherited. Coherent line interfaces are commonly specified with more than one FEC mode, and low-latency modes trade correction strength for latency. Practical headroom is a property of the whole transceiver architecture — receiver, DSP, modulation, FEC and optical design — not of the code alone. Margin is engineered, not inherited.

Figure 2: Pre-FEC BER versus received optical power. The curves are receivers; the horizontal lines are FEC modes. Thresholds: KP4 2.3E-4, staircase 4.5E-3, concatenated soft-decision 1.25E-2, at 5.8%, 6.7% and 14.8% overhead (IEEE 802.3 B10K, 2017). Loss budget is the OIF 400ZR unamplified case, shown as the standardized example (OIF, 2020). Receiver curves and operating points are illustrative. Arycs Technologies analysis.

The Standards Record Already Drew This Line — at 40 km

The industry has already run this evaluation once, formally. In 2017 IEEE 802.3 chartered a study group on optical PHYs beyond 10 km, and the question in front of it was precisely how far direct detect could be pushed before coherent became the better answer. Stronger FEC was the lever under discussion: the same contribution that supplies this paper’s threshold figures proposed borrowing the 400G ZR FEC to extend direct-detect PAM4 to 40 km.The line landed where the margin ran out. IEEE 802.3cn standardized direct-detect PHYs at 50, 200 and 400 Gb/s for reaches of at least 40 km. IEEE 802.3ct standardized 100 Gb/s over DWDM using phase and amplitude modulation with coherent detection, for reaches of at least 80 km. Direct detect did not fail at 80 km. It needed a transport layer beneath it — amplification and dispersion management — and that line system is itself a flap source, through gain transients, ripple and channel-count changes.Where does that squeeze land next? At 1.6T with 200G per lane, TDECQ, dispersion tolerance and reflection sensitivity all tighten at 2 km and 10 km, bringing the same margin problem down to campus reach. That is the opening now discussed for coherent lite. It is a forecast and should be read as one.
Margin Is a Design Parameter, Not an Inherited PropertyFlapping is a margin event: five common causes, one mechanism. Coherent detection shifts the operating point away from the FEC cliff through code headroom, link budget and a smaller interface count. IEEE 802.3 ran this trade once already, and drew the line at 40 km. The useful output is not a verdict on detection schemes. It is a short list of what a link designer actually controls, and where flapping is won or lost.

• FEC mode — how much pre-FEC margin the selected mode leaves at the worst-case operating point, and what latency that headroom costs.
• Loss budget — whether the reserve absorbs a contaminated connector, or whether 1.5 dB consumes it.
• Host equalization — whether per-port pre-emphasis is verified with PRBS or inherited from a reference design that assumed a different trace length.
• Telemetry — whether pre-FEC BER is trended, so a link drifting toward threshold is found before it starts flapping.

Two boundaries. Coherent is not universal: at 500 m inside the fabric IM-DD is adequate and cheaper, coherent carries its own flap modes on amplified spans, and no detection scheme touches host-side electrical margin. And the comparison made here is margin, not flap rate. Margin is specified — FEC thresholds and loss budgets are published numbers, and the gap between them is wide. Flap rate is the downstream outcome, and no controlled public comparison of it exists in either direction. Anyone quoting a figure for it cannot show the data either.The failure mode follows the margin. As 1.6T compresses the campus budget, flapping follows it there — and so do the four questions above.
References

• Alibaba HPN: A Data Center Network for Large Language Model Training. ACM SIGCOMM 2024. Link-flapping rate of 5K–60K events per day; the term fault explosion radius.
• A. Kokolis, M. Kuchnik et al. (FAIR at Meta). Revisiting Reliability in Large-Scale Machine Learning Research Clusters. arXiv:2410.21680, 2024; IEEE HPCA 2025.
• Meta. The Llama 3 Herd of Models. arXiv:2407.21783, 2024. 419 unexpected interruptions in 54 days on 16,384 GPUs.
• B. Smith, I. Lyubomirsky and S. Bhoja. Leveraging 400G ZR FEC Technology. IEEE 802.3 Beyond 10 km Optical PHYs Study Group, Orlando, 8–9 November 2017. FEC thresholds — KP4 2.3E-4 at 5.8% overhead, staircase 4.5E-3 at 6.7%, concatenated 1.25E-2 at 14.8%; net coding gains of 6.9, 9.76 and 10.8 dB. Also the proposal to extend direct-detect PAM4 to 40 km using 400G ZR FEC.
• OIF. 400ZR Implementation Agreement, 2020 (announced 29 April 2020). Unamplified single-wavelength loss budget of 11 dB.
• IEEE Std 802.3ct-2021. 100 Gb/s Physical Layer specifications for operation over DWDM systems using phase and amplitude modulation with coherent detection, for reaches of at least 80 km.
• IEEE Std 802.3cn-2019. Physical Layer specifications for 50, 200 and 400 Gb/s operation over single-mode fibre with reaches of at least 40 km.

About Arycs TechnologiesArycs delivers power-efficient, coherent-class optical connectivity based on silicon photonics, coherent DSP, and advanced optical architectures. Our solutions provide industry-leading bandwidth per watt, deterministic performance, and flexible network evolution for AI, cloud, telecom, and edge infrastructure. Designed for real-world deployment, Arycs Technologies enables networks to scale with growing AI demand without disruptive redesign or hardware replacement.arycs-tech.com | LinkedIn: Arycs TechnologiesWhen the Link Blinks: Flapping, Optical Margin, and the Metro Inflection