Mitigation of nonlinear impairments for advanced optical modulation formats

Optical fibre networks form the backbone of the global communication infrastructure but are currently experiencing an unprecedented level of stress due to more and more bandwidth-hungry applications. In an effort to address this and avoid a so-called capacity crunch, research groups around the world have focused their attention on more spectrally-efficient modulation formats, to increase available capacity at a competitive cost. However, the drive towards higher- order modulation formats leads to greater transmission impairments, reducing the maximum distance over which increased capacity can be provided. The thesis describes the research work carried out to investigate the achievable transmission distances when using higher order modulation formats together with digital backpropagation (DBP). DBP is a digital signal processing (DSP) algorithm, capable of compensating for deterministic nonlinear impairments by inverting the fibre channel. Single-channel and wavelength-division-multiplexed (WDM) transmission has been investigated in experiment and simulation for a variety of polarisation-division-multiplexed (PDM) modulation formats: binary-phase-shift-keying (PDM-BPSK), quadrature-phase-shift-keying (PDM-QPSK), 8-phase-shift-keying (PDM-8PSK), 8-quadrature amplitude modulation (PDM-8QAM), 16-quadrature amplitude modulation (PDM-16QAM) and polarisation-switched QPSK (PS-QPSK). Record transmission distances were achieved in WDM transmission experiments with PDM-BPSK, PS-QPSK and PDM-QPSK at 42.9Gbit/s as well as for PDM-8PSK and PDM-8QAM at 112Gbit/s, over the most common fibre type: standard single mode fibre (SSMF) and the most common amplification solution: erbium doped fibre amplifiers (EDFA). For the first time, nonlinear compensation has been compared experimentally for different modulation formats and a fixed-complexity DBP algorithm. Its use led to increased benefit for more spectrally efficient modulation formats. Computer simulations were used to explore the upper bounds of achievable performance improvement with DBP, using an algorithm with unconstrained complexity. Furthermore, DBP was investigated for varying symbol rates and channel spacings to investigate trade-offs with respect to the digital receiver bandwidth. It was shown that even though DBP is computationally expensive, it can achieve significant improvements in transmission reach and BER performance. The results presented in this thesis, can be applied to the design of future optical transmission systems.