soundofheaven.info Personal Growth INFORMATION THEORY PDF

Information theory pdf

Thursday, May 16, 2019 admin Comments(0)

Most scientists agree that information theory began in with Shannon's famous Since then, information theory has kept on designing devices that reach or. PDF | Shannon's mathematical theory of communication defines fundamental limits on how much information can be transmitted between the different. remain viewable on-screen on the above website, in postscript, djvu, and pdf formats. . Conventional courses on information theory cover not only the beauti- .


Author: ARIANA JANUSZ
Language: English, Spanish, Dutch
Country: Jamaica
Genre: Art
Pages: 290
Published (Last): 17.08.2016
ISBN: 861-7-25874-722-1
ePub File Size: 17.89 MB
PDF File Size: 14.70 MB
Distribution: Free* [*Regsitration Required]
Downloads: 25159
Uploaded by: KALLIE

This chapter introduces some of the basic concepts of information theory, as well We shall often use the shorthand pdf for the probability density func-. This book is devoted to the theory of probabilistic information measures and their application to coding theorems for information sources and noisy channels. Information Theory is one of the few scientific fields fortunate enough to have an Information Theory was not just a product of the work of Claude Shannon.

We consider two different inner receivers, the coherent and non-coherent receiver, which are both based on the matched filter output 4. For example, the Kerr nonlinearity implies that if the signal level of the input is doubled, unlike many models for copper-wired and wireless channels, the statistical distribution of the output changes in a more intricate way than by pure scaling. First, it accurately describes the propagation on wired and wireless links under certain conditions; and second, it is one of the few waveform channels whose capacity is known exactly. You have access. For practical reasons, the transmitter in figure 1 is often divided into two parts, as illustrated in figure 1 b. While models of fibre-optic channels have so far defied exact information-theoretic analyses, substantial progress continues to be made, and the insights obtained are likely to inform system designs for many years to come.

Thus, the differential entropy h X can be interpreted as the growth-rate factor of the volume of the typical set of sequences of length n associated with a PDF p X. Without attempting to be absolutely rigorous, using only these rough properties of typical sets, we will now consider information transmission over a discrete memoryless channel.

We assume the channel has input alphabet , output alphabet and channel law p Y X y x , giving the probability of observing symbol at the channel output when symbol is transmitted. We will assign a PMF p X to the transmitted symbols, and assume that the transmitter is constrained to the transmission of typical sequences only. When a sequence x n is transmitted, and assuming n is sufficiently large, the set of output sequences observed by the receiver are confined to a typical set of size about 2 nH Y X , where H Y X denotes the so-called conditional entropy of Y given X , given as 3.

The set of probable received sequences corresponding to all possible transmitted sequences is itself a typical set containing about 2 nH Y elements, where H Y is the entropy associated with the PMF. Our goal is to design a codebook of transmitted sequences with the property that there is little probability of overlap between the noise balls corresponding to different codewords.

In this case, each received sequence is highly likely to fall within the noise ball associated with just one codeword the transmitted one , and thus a low probability of error can be achieved if the decoder simply produces the associated codeword.

The relationships among these various sets are illustrated in figure 3. Figure 3. The relationships between typical sets, noise balls and transmitted and received codewords. To maximize the transmission rate, we would like to design a codebook with as many codewords of possible, yet with the property that the noise balls corresponding to different codewords are essentially non-overlapping.

Ideally, we might hope that the noise balls completely partition the set of typical output sequences. Assuming that the transmitted and received sequences are of length n , because the set of typical channel output sequences contains about 2 nH Y elements, and each noise ball contains about 2 nH Y X elements, we would certainly not hope for more than about codewords. In this case, the ratio of the volume of the typical output sequences to the volume of a noise ball gives a bound on the size of a decodable codebook expressed in terms of the mutual information I X ; Y between continuous random variables X and Y , defined as 3.

Indeed, it is possible to show that if a codebook of more than 2 nR codewords of length n is used, then the probability of error of any decoder cannot be made arbitrarily small, i.

As , this fraction approaches zero, suggesting that a placement of essentially non-overlapping noise balls becomes increasingly more feasible with increasing block length. By expurgating a constant fraction of the codewords with the worst individual probability of error which has a negligible impact on the rate , the worst-case probability of error can also be made to approach zero. In summary, we see that at transmission rates R greater than the mutual information I X ; Y , the probability of error cannot be made to approach zero, while at transmission rates R less than I X ; Y , any arbitrarily small error probability can, in principle, be achieved by choosing the block length n to be sufficiently large.

Thus for discrete memoryless channels, when the channel input symbols are chosen according to a probability mass or density function p X x , the maximum achievable rate of reliable information transmission is the mutual information I X ; Y. Because the mutual information depends on p X x , the channel capacity for discrete memoryless channels is equal to the maximum mutual information that can be achieved over all possible input distributions, i.

It is important to note that 3. In particular, this so-called single-letter expression does not hold for general channels with memory. The multiletter generalization holds for so-called information stable channels with memory. While this latter formula gives the capacity of many channels of practical interest, it is possible to create mathematical models for channels for which even this formula fails to hold; see [ 10 ] for a discussion and development of a capacity formula that applies to even more exotic channel models with memory.

All of these capacity formulae apply to waveform channels whenever such channels are completely equivalent to some discrete-time channel model for example, via projection on a countable orthogonal basis. For nonlinear waveform channels such as the optical-fibre channel, it appears difficult to achieve such an equivalence, and, therefore, one typically must resort to making approximations and assumptions e.

The most well-studied channel in information theory is the AWGN channel. This is for two reasons. First, it accurately describes the propagation on wired and wireless links under certain conditions; and second, it is one of the few waveform channels whose capacity is known exactly.

In the AWGN channel, the relationship between the input x t and output y t , which are both complex processes, is 4. Moreover, in optical communications, white Gaussian noise models well the amplified spontaneous emission noise introduced by optical amplifiers. Assume that the complex waveform x t is limited to a bandwidth W , i. The expression in 4. Other channels may have a larger or smaller capacity, depending on their particular characteristics.

The capacity C P , W in 4. To increase the capacity, one can increase the bandwidth W , the power P , or both. If the bandwidth is fixed and the transmitted power is increased, then the capacity C P , W tends to infinity, but it grows only logarithmically with power.

On the other hand, if the power is fixed and the bandwidth increases, the capacity will never exceed. These two cases highlight the fact that when bandwidth is available, it is a good idea to spread the power over the whole bandwidth rather than only using a small part of it.

We will now use the band-limited AWGN channel in 4. First, there exist multiple discrete-time channels that correspond to the same waveform channel, depending on the choices for the inner transmitter and receiver. These discrete-time channels can have different capacities, of which the highest is equal to the capacity of the underlying waveform channel.

Second, there exist multiple transmission schemes for the same discrete-time channel figure 1 c , depending on the choices for the outer transmitter and receiver. These schemes can have different mutual information, of which the highest is equal to the channel capacity of the discrete-time channel. To elaborate on the first point, we design a waveform x t from a sequence of complex numbers x n as 4. This is a linear modulator, which we use as the inner transmitter in figure 1 b.

We consider two different inner receivers, the coherent and non-coherent receiver, which are both based on the matched filter output 4. It can be shown that the coherent receiver in combination with the inner transmitter 4. The variance of the transmitted symbol x i is for all i. The capacities of these two discrete-time channels are shown in figure 4 a , where the shaded region indicates that the capacity of the non-coherent channel is known by means of upper and lower bounds see appendix but not exactly.

Despite the fact that they both communicate over the same waveform channel, their capacities are quite different. This is exactly the same expression as 4. For general channels, however, there exists no such equivalence between continuous- and discrete-time models. Figure 4. The highest of these, which gives the capacity of 4. If a suboptimal non-coherent detector is applied, the capacity in the shaded region is less. The highest of these represents the capacity of the channel. We now turn to the second principle, namely that different transmission schemes for the same channel have different maximum achievable rates.

Using the discrete-time AWGN channel 4. For the finite input alphabets , which all correspond to well-known digital modulation formats, the mutual information converges to as the SNR increases, whereas the it grows unboundedly in the case of continuous input distributions.

The highest mutual information is at all SNRs obtained for the circular Gaussian distribution, and this mutual information is, indeed, equal to the highest capacity in figure 4 a. This shows that the capacity-achieving input distribution for this discrete-time channel is circular Gaussian. Because information theory is a mathematical science, it needs a mathematical description of the channel.

Information theory and reliable communication

The term W t , z represents random noise, uncorrelated in t and z , which is added in optical amplifiers and detectors. The model has been demonstrated to be very accurate and it can be adapted to a wide range of scenarios, including different amplification schemes and dual-polarization transmission. Unfortunately, no explicit relation between X t and Y t is known.

The equation cannot be solved analytically except in a few special cases. The capacity of the optical fibre channel is not known. This is a property that it shares with most other real-world channels.

Elements Of Information Theory 2nd Ed

The standard approach in such cases is to sandwich the capacity between lower and upper bounds. If such bounds can be derived, and if they are reasonably close to each other, then reliable conclusions can be drawn about the capacity, and the results often give insights into how to design efficient transmission schemes for the channel in question. Lower and upper bounds have different nature and are derived by different mathematical techniques.

A lower bound describes what is possible, whereas an upper bound describes what is impossible. Any transmission scheme i. Other lower bounds can be obtained by studying other transmission schemes. For example, it is sufficient to study any single pair of inner transmitter and receiver in figure 1 b. Therefore, there exists a rich literature about lower bounds on the capacity of optical channels [ 4 , 7 , 11 , 12 ]. Without any ambitions to plot any of these bounds exactly, which would confine the study to a specific system set-up, the general behaviour of these lower bounds is illustrated by the coloured curves in figure 5.

Even though most of these lower bounds have a peak at some power, their envelope does not: Figure 5. General behaviour of upper and lower bounds on the capacity of fibre-optic channels. An upper bound on the average transmit power is also shown, which confines the capacity to a triangular region shaded. It is not known where in this region the true capacity lies.

In contrast, upper bounds cannot be based on isolated transmission schemes—they predict that rates above a certain value can never be achieved for the considered channel, with neither present methods nor any methods that may devised in the future.

Thus, an upper bound derived for a specific discrete-time channel is not necessarily valid for the underlying waveform channel. Figure 5 includes the only upper bound on the capacity of the fibre-optic channel that we are aware of, which was proved only recently [ 13 ].

In plain words, it states that the capacity is no larger than the capacity of an AWGN channel 4. There is a sizeable gap between the lower and upper bounds in figure 5.

The gap increases with power, which may give the impression that capacity might, indeed, be infinite in some cases. Unfortunately, there is a third bound, which comes not from information theory but from physics: If the transmit power is increased too high, the fibre will heat up and eventually melt.

Hence, the capacity is not only bounded from above and below, but also from the right. It is still an open question where in the grey region the true capacity lies. The picture gets more complicated when one considers a whole network of connections rather than a single point-to-point link.

An optical network consists of a mesh of nodes connected with fibres. Each fibre carries many parallel signals on different wavelengths, which interfere with each other during propagation. The connections are routed from source to destination via several intermediate nodes, where the signals are separated in wavelength-selective switches. This set-up creates a topology that connects geographically separated nodes with each other in the same configurable network.

The maximum achievable rates in optical networks have been analysed only under certain simplifying assumptions. The most common approach is to consider the capacity of a single point-to-point link in the network, assuming certain behavioural models for the transmission on the interfering channels [ 12 , 15 , 16 ].

Another approach, which is better aligned with conventional information theory, is to study the set of rates that can be simultaneously achieved over a set of interfering connections in the network [ 17 — 19 ]. Although we have so far argued that information theory is an indispensable tool for the analysis of communication systems, we now argue that it is also eminently suitable for system design.

By understanding the fundamental limits associated with any given transmission strategy, system builders can allocate development effort appropriately, converging to a design with the most suitable trade-off between data transmission performance and implementation complexity in any given application. As our understanding of the fundamental limits and the transmission schemes that approach them matures, we expect—or, more accurately, we speculate—that new approaches for information transmission over the fibre-optic channel will emerge.

Widely used linear matched-filter receivers—which are known to provide a sufficient statistic for detection at the output of the AWGN channel—are not necessarily the optimal processor at the output of a nonlinear channel. In AWGN channels, capturing and processing energy outside the frequency band of the transmitted signal is useless; in fibre channels, owing to the nonlinearity, such processing may be helpful as out-of-band signals are correlated with the in-band signal of interest.

Information theory is expected to help system designers understand the impact on achievable transmission rates of replacing expensive optical components and devices with less efficient and less expensive alternatives.

We would expect that the potentially significant cost reductions that may be achievable using this approach will drive research and development in this direction. WDM creates user subchannels centred at different carrier frequencies wavelengths , analogously to the method by which the signals from different radio or television broadcasters are kept separated.

Unfortunately, this linear multiplexing technique inevitably suffers from interference crosstalk between the channels owing to channel nonlinearity, which appears to limit the data transmission rates that can be achieved in such systems. At least in principle, it may be possible to multiplex the signals of different users in different bands in the nonlinear spectral domain.

In contrast with WDM systems, in which crosstalk occurs even in ideal noise-free systems, because nonlinearly multiplexed subchannels are completely noninteracting in the absence of noise, we speculate that the amount of interaction between channels will be smaller than in WDM systems in the low-noise regime.

At present, this idea is far from practical, because there are no known physical devices apart from synthesis by digital algorithms that can achieve such multiplexing with the same convenience as the linear superposition of multiple modulated laser sources operating at different wavelengths. Furthermore, deviations from ideality in particular, loss, noise, imperfections in waveform synthesis will have a deleterious effect that is at present only poorly understood.

Certainly, further investigation of NFDM is warranted. In applying the results of information theory, system designers must be cautious to ensure that all constraints and costs are reflected in the information-theoretic model. For example, the usual analysis of the capacity of the AWGN channel considers only the cost of the transmitter power P and the bandwidth W , neglecting, for example, the power expended in the operation of the receiver.

While such a model is certainly appropriate in situations like long-distance wireless communication where the transmitter power is the dominant component in the total system power budget, this may not be the regime of greatest interest in long-haul optical communications, where significant power is consumed in the operation of the receiver.

To operate near the channel capacity requires long codes and complicated power-hungry decoding algorithms; thus, as suggested by recent information-theoretic analyses [ 26 ], when minimizing total power consumption it may be beneficial to operate a system at some gap from the channel capacity.

In K -user linear multiuser interference channels, the recently proposed concept of interference alignment [ 27 ] is an intriguing concept with a potential for application to optical-fibre systems. Here, each user partitions its set of available degrees-of-freedom into two bins: As optical fibre system designs have increasingly come to afford ever more sophisticated digital signal-processing and error-correcting decoding algorithms in the receiver, design choices made previously to simplify processing are being revisited.

Information Theory

A primary example is the development of single-mode fibre, which greatly simplifies equalization and signal processing in long-haul systems. On the other hand, multimode or few-mode fibre gives the possibility of achieving a substantial enhancement in information-carrying capacity, at the expense of devices that allow for coupling of different signals simultaneously into several modes at the transmitter, and detecting and processing these modes at the receiver. Unlike the similar conventional linear multiantenna transmission systems used in wireless communications, nonlinearity is expected to be a dominant consideration in such multimode systems.

Establishing fundamental information-theoretic limits on such systems remains an open problem. Information theory builds upon mathematical models of communication systems to establish fundamental limits on their information-carrying capability. Information theory guides system designers to find efficient strategies with which to exploit a given set of transmission resources.

Even though the exact channel capacity, which is the maximum achievable data rate for a given channel, is not known exactly, bounds and estimates are available, which give important insights into system designs. While models of fibre-optic channels have so far defied exact information-theoretic analyses, substantial progress continues to be made, and the insights obtained are likely to inform system designs for many years to come. We declare we have no competing interests. The research was supported by the Swedish Research Council under grant nos.

This appendix provides mathematical expressions for some of the capacity estimates and bounds that are illustrated in this paper. The non-coherent AWGN capacity in figure 4 a was characterized by the upper bound in [ 29 ], eqn 42 and the lower bound in [ 30 ], eqn These bounds are not very tight. Stronger bounds exist but are in general more complex to evaluate [ 29 , 30 ].

The continuous input distributions in figure 4 b are a complex circular Gaussian distribution and a complex uniform distributions over a square. In all three cases, the constellations are scaled to the desired power and probabilities are uniform on the constellation points.

Similarly, y 1 ,…, y n is denoted as y n , X 1 ,…, X n is denoted as X n , etc. Random variables are uppercase e. X and a single realization thereof lowercase x. Login to your account. Forgot password? Keep me logged in. New User. Change Password. Old Password. New Password. Create a new account. Returning user. Can't sign in? Forgot your password? Enter your email address below and we will send you the reset instructions. If the address matches an existing account you will receive an email with instructions to reset your password Close.

Request Username. Forgot your username? Enter your email address below and we will send you your username. You have access. View PDF. Implications of information theory in optical fibre communications Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences http: As , this fraction approaches zero, suggesting that a placement of essentially non-overlapping noise balls becomes increasingly more feasible with increasing block length.

By expurgating a constant fraction of the codewords with the worst individual probability of error which has a negligible impact on the rate , the worst-case probability of error can also be made to approach zero.

In summary, we see that at transmission rates R greater than the mutual information I X ; Y , the probability of error cannot be made to approach zero, while at transmission rates R less than I X ; Y , any arbitrarily small error probability can, in principle, be achieved by choosing the block length n to be sufficiently large.

Thus for discrete memoryless channels, when the channel input symbols are chosen according to a probability mass or density function p X x , the maximum achievable rate of reliable information transmission is the mutual information I X ; Y.

Because the mutual information depends on p X x , the channel capacity for discrete memoryless channels is equal to the maximum mutual information that can be achieved over all possible input distributions, i. It is important to note that 3. In particular, this so-called single-letter expression does not hold for general channels with memory. The multiletter generalization holds for so-called information stable channels with memory. While this latter formula gives the capacity of many channels of practical interest, it is possible to create mathematical models for channels for which even this formula fails to hold; see [ 10 ] for a discussion and development of a capacity formula that applies to even more exotic channel models with memory.

All of these capacity formulae apply to waveform channels whenever such channels are completely equivalent to some discrete-time channel model for example, via projection on a countable orthogonal basis.

For nonlinear waveform channels such as the optical-fibre channel, it appears difficult to achieve such an equivalence, and, therefore, one typically must resort to making approximations and assumptions e. The most well-studied channel in information theory is the AWGN channel. This is for two reasons. First, it accurately describes the propagation on wired and wireless links under certain conditions; and second, it is one of the few waveform channels whose capacity is known exactly.

In the AWGN channel, the relationship between the input x t and output y t , which are both complex processes, is 4. Moreover, in optical communications, white Gaussian noise models well the amplified spontaneous emission noise introduced by optical amplifiers.

Assume that the complex waveform x t is limited to a bandwidth W , i. The expression in 4. Other channels may have a larger or smaller capacity, depending on their particular characteristics.

The capacity C P , W in 4. To increase the capacity, one can increase the bandwidth W , the power P , or both. If the bandwidth is fixed and the transmitted power is increased, then the capacity C P , W tends to infinity, but it grows only logarithmically with power. On the other hand, if the power is fixed and the bandwidth increases, the capacity will never exceed. These two cases highlight the fact that when bandwidth is available, it is a good idea to spread the power over the whole bandwidth rather than only using a small part of it.

We will now use the band-limited AWGN channel in 4. First, there exist multiple discrete-time channels that correspond to the same waveform channel, depending on the choices for the inner transmitter and receiver.

These discrete-time channels can have different capacities, of which the highest is equal to the capacity of the underlying waveform channel. Second, there exist multiple transmission schemes for the same discrete-time channel figure 1 c , depending on the choices for the outer transmitter and receiver.

These schemes can have different mutual information, of which the highest is equal to the channel capacity of the discrete-time channel.

Elements of Information Theory 2nd Edition PDF

To elaborate on the first point, we design a waveform x t from a sequence of complex numbers x n as 4. This is a linear modulator, which we use as the inner transmitter in figure 1 b. We consider two different inner receivers, the coherent and non-coherent receiver, which are both based on the matched filter output 4.

It can be shown that the coherent receiver in combination with the inner transmitter 4. The variance of the transmitted symbol x i is for all i. The capacities of these two discrete-time channels are shown in figure 4 a , where the shaded region indicates that the capacity of the non-coherent channel is known by means of upper and lower bounds see appendix but not exactly.

Despite the fact that they both communicate over the same waveform channel, their capacities are quite different. This is exactly the same expression as 4. For general channels, however, there exists no such equivalence between continuous- and discrete-time models.

Figure 4. The highest of these, which gives the capacity of 4. If a suboptimal non-coherent detector is applied, the capacity in the shaded region is less. The highest of these represents the capacity of the channel. We now turn to the second principle, namely that different transmission schemes for the same channel have different maximum achievable rates.

Using the discrete-time AWGN channel 4. For the finite input alphabets , which all correspond to well-known digital modulation formats, the mutual information converges to as the SNR increases, whereas the it grows unboundedly in the case of continuous input distributions. The highest mutual information is at all SNRs obtained for the circular Gaussian distribution, and this mutual information is, indeed, equal to the highest capacity in figure 4 a.

This shows that the capacity-achieving input distribution for this discrete-time channel is circular Gaussian.

Theory pdf information

Because information theory is a mathematical science, it needs a mathematical description of the channel. The term W t , z represents random noise, uncorrelated in t and z , which is added in optical amplifiers and detectors. The model has been demonstrated to be very accurate and it can be adapted to a wide range of scenarios, including different amplification schemes and dual-polarization transmission.

Unfortunately, no explicit relation between X t and Y t is known. The equation cannot be solved analytically except in a few special cases. The capacity of the optical fibre channel is not known. This is a property that it shares with most other real-world channels. The standard approach in such cases is to sandwich the capacity between lower and upper bounds.

If such bounds can be derived, and if they are reasonably close to each other, then reliable conclusions can be drawn about the capacity, and the results often give insights into how to design efficient transmission schemes for the channel in question. Lower and upper bounds have different nature and are derived by different mathematical techniques.

A lower bound describes what is possible, whereas an upper bound describes what is impossible. Any transmission scheme i. Other lower bounds can be obtained by studying other transmission schemes. For example, it is sufficient to study any single pair of inner transmitter and receiver in figure 1 b. Therefore, there exists a rich literature about lower bounds on the capacity of optical channels [ 4 , 7 , 11 , 12 ].

Without any ambitions to plot any of these bounds exactly, which would confine the study to a specific system set-up, the general behaviour of these lower bounds is illustrated by the coloured curves in figure 5. Even though most of these lower bounds have a peak at some power, their envelope does not: Figure 5. General behaviour of upper and lower bounds on the capacity of fibre-optic channels. An upper bound on the average transmit power is also shown, which confines the capacity to a triangular region shaded.

It is not known where in this region the true capacity lies. In contrast, upper bounds cannot be based on isolated transmission schemes—they predict that rates above a certain value can never be achieved for the considered channel, with neither present methods nor any methods that may devised in the future. Thus, an upper bound derived for a specific discrete-time channel is not necessarily valid for the underlying waveform channel.

Figure 5 includes the only upper bound on the capacity of the fibre-optic channel that we are aware of, which was proved only recently [ 13 ]. In plain words, it states that the capacity is no larger than the capacity of an AWGN channel 4.

There is a sizeable gap between the lower and upper bounds in figure 5. The gap increases with power, which may give the impression that capacity might, indeed, be infinite in some cases.

Unfortunately, there is a third bound, which comes not from information theory but from physics: If the transmit power is increased too high, the fibre will heat up and eventually melt. Hence, the capacity is not only bounded from above and below, but also from the right. It is still an open question where in the grey region the true capacity lies. The picture gets more complicated when one considers a whole network of connections rather than a single point-to-point link.

An optical network consists of a mesh of nodes connected with fibres. Each fibre carries many parallel signals on different wavelengths, which interfere with each other during propagation.

The connections are routed from source to destination via several intermediate nodes, where the signals are separated in wavelength-selective switches. This set-up creates a topology that connects geographically separated nodes with each other in the same configurable network. The maximum achievable rates in optical networks have been analysed only under certain simplifying assumptions. The most common approach is to consider the capacity of a single point-to-point link in the network, assuming certain behavioural models for the transmission on the interfering channels [ 12 , 15 , 16 ].

Pdf information theory

Another approach, which is better aligned with conventional information theory, is to study the set of rates that can be simultaneously achieved over a set of interfering connections in the network [ 17 — 19 ].

Although we have so far argued that information theory is an indispensable tool for the analysis of communication systems, we now argue that it is also eminently suitable for system design. By understanding the fundamental limits associated with any given transmission strategy, system builders can allocate development effort appropriately, converging to a design with the most suitable trade-off between data transmission performance and implementation complexity in any given application.

As our understanding of the fundamental limits and the transmission schemes that approach them matures, we expect—or, more accurately, we speculate—that new approaches for information transmission over the fibre-optic channel will emerge.

Widely used linear matched-filter receivers—which are known to provide a sufficient statistic for detection at the output of the AWGN channel—are not necessarily the optimal processor at the output of a nonlinear channel. In AWGN channels, capturing and processing energy outside the frequency band of the transmitted signal is useless; in fibre channels, owing to the nonlinearity, such processing may be helpful as out-of-band signals are correlated with the in-band signal of interest.

Information theory is expected to help system designers understand the impact on achievable transmission rates of replacing expensive optical components and devices with less efficient and less expensive alternatives. We would expect that the potentially significant cost reductions that may be achievable using this approach will drive research and development in this direction.

WDM creates user subchannels centred at different carrier frequencies wavelengths , analogously to the method by which the signals from different radio or television broadcasters are kept separated. Unfortunately, this linear multiplexing technique inevitably suffers from interference crosstalk between the channels owing to channel nonlinearity, which appears to limit the data transmission rates that can be achieved in such systems.

At least in principle, it may be possible to multiplex the signals of different users in different bands in the nonlinear spectral domain. In contrast with WDM systems, in which crosstalk occurs even in ideal noise-free systems, because nonlinearly multiplexed subchannels are completely noninteracting in the absence of noise, we speculate that the amount of interaction between channels will be smaller than in WDM systems in the low-noise regime.

At present, this idea is far from practical, because there are no known physical devices apart from synthesis by digital algorithms that can achieve such multiplexing with the same convenience as the linear superposition of multiple modulated laser sources operating at different wavelengths.

Furthermore, deviations from ideality in particular, loss, noise, imperfections in waveform synthesis will have a deleterious effect that is at present only poorly understood. Certainly, further investigation of NFDM is warranted. In applying the results of information theory, system designers must be cautious to ensure that all constraints and costs are reflected in the information-theoretic model. For example, the usual analysis of the capacity of the AWGN channel considers only the cost of the transmitter power P and the bandwidth W , neglecting, for example, the power expended in the operation of the receiver.

While such a model is certainly appropriate in situations like long-distance wireless communication where the transmitter power is the dominant component in the total system power budget, this may not be the regime of greatest interest in long-haul optical communications, where significant power is consumed in the operation of the receiver.

To operate near the channel capacity requires long codes and complicated power-hungry decoding algorithms; thus, as suggested by recent information-theoretic analyses [ 26 ], when minimizing total power consumption it may be beneficial to operate a system at some gap from the channel capacity.

In K -user linear multiuser interference channels, the recently proposed concept of interference alignment [ 27 ] is an intriguing concept with a potential for application to optical-fibre systems. Here, each user partitions its set of available degrees-of-freedom into two bins: As optical fibre system designs have increasingly come to afford ever more sophisticated digital signal-processing and error-correcting decoding algorithms in the receiver, design choices made previously to simplify processing are being revisited.

A primary example is the development of single-mode fibre, which greatly simplifies equalization and signal processing in long-haul systems. On the other hand, multimode or few-mode fibre gives the possibility of achieving a substantial enhancement in information-carrying capacity, at the expense of devices that allow for coupling of different signals simultaneously into several modes at the transmitter, and detecting and processing these modes at the receiver.

Unlike the similar conventional linear multiantenna transmission systems used in wireless communications, nonlinearity is expected to be a dominant consideration in such multimode systems. Establishing fundamental information-theoretic limits on such systems remains an open problem.

Information theory builds upon mathematical models of communication systems to establish fundamental limits on their information-carrying capability. Information theory guides system designers to find efficient strategies with which to exploit a given set of transmission resources.

Even though the exact channel capacity, which is the maximum achievable data rate for a given channel, is not known exactly, bounds and estimates are available, which give important insights into system designs. While models of fibre-optic channels have so far defied exact information-theoretic analyses, substantial progress continues to be made, and the insights obtained are likely to inform system designs for many years to come.

We declare we have no competing interests. The research was supported by the Swedish Research Council under grant nos. This appendix provides mathematical expressions for some of the capacity estimates and bounds that are illustrated in this paper.

The non-coherent AWGN capacity in figure 4 a was characterized by the upper bound in [ 29 ], eqn 42 and the lower bound in [ 30 ], eqn These bounds are not very tight. Stronger bounds exist but are in general more complex to evaluate [ 29 , 30 ]. The continuous input distributions in figure 4 b are a complex circular Gaussian distribution and a complex uniform distributions over a square. In all three cases, the constellations are scaled to the desired power and probabilities are uniform on the constellation points.

Similarly, y 1 ,…, y n is denoted as y n , X 1 ,…, X n is denoted as X n , etc. Random variables are uppercase e. X and a single realization thereof lowercase x. Login to your account. Forgot password?

Keep me logged in. New User. Change Password. Old Password. New Password. Create a new account. Returning user. Can't sign in? Forgot your password? Enter your email address below and we will send you the reset instructions. If the address matches an existing account you will receive an email with instructions to reset your password Close. Request Username. Forgot your username? Enter your email address below and we will send you your username.

You have access. View PDF. Implications of information theory in optical fibre communications Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences http: You have access Review article.

Erik Agrell Erik Agrell http: Alex Alvarado Alex Alvarado http: Frank R. Kschischang Frank R. Kschischang http: Abstract Recent decades have witnessed steady improvements in our ability to harness the information-carrying capability of optical fibres.

Download figure Open in new tab Download PowerPoint. References 1 Hecht J. New York, NY: Oxford University Press. Shannon CE. Bell Syst. IEEE J. Quantum Electron. Agrell E.