Unified design of iterative receivers using factor graphs

Unified Design of Iterative Receivers Using FactorGraphsAndrew P.Worthen and Wayne E.Stark Abstract—Iterative algorithms are an attractive approach to approximating optimal,but high-complexity,joint chan-nel estimation and decoding receivers for communication systems.We present a unified approach based on factor graphs for deriving iterative message-passing receiver al-gorithms for channel estimation and decoding.For many common channels,it is easy tofind simple graphical models that lead directly to implementable algorithms.Canonical distributions provide a new,general framework for handling continuous variables.Example receiver designs for Rayleigh fading channels with block or Markov memory,and multi-path fading channels withfixed unknown coefficients illus-trate the effectiveness of our approach.Keywords—channel estimation;fading channels;iterative decoding;low-density parity check(LDPC)codesI.IntroductionFor many communication systems it is well-known that joint demodulation and decoding is required for optimum performance.Typically,this processing is too complex for practical implementation and some sort of serial process-ing is employed.Iterative algorithms which approximate optimal joint decoding for a variety of concatenated codes [1–3]are known to have excellent performance.This has led to an interest in iterative algorithms for approximat-ing joint channel estimation,demodulation,and decod-ing,which we call iterative receivers.The usual paradigm for these designs is the interconnection of soft-input,soft-output(SISO)modules[4].Graphical models for codes[5–7]lead to iterative algo-rithms for decoding,including the turbo decoding algo-rithm[8].Factor graphs[9–12],so named because they graphically represent function factorizations,are general-izations of Wiberg’s work[6]that lead to particularly clean derivations of iterative algorithms.The sum-product al-gorithm computes the marginals of the function that the graph represents using message passing on the graph.If the graph is cycle-free,the results are exact;however,empirical results show that the algorithm yields excellent decoding decisions even when the graph contains cycles.The possi-bility of building graphical models for joint decoding and demodulation has been suggested without elaboration in [5,6,13],and we have studied a simple system using a low-density parity check(LDPC)code on the block interference channel in[14].Our goal in this paper is to present a unified approach to designing iterative receivers using factor graphs.Factor The authors are with the Department of Electrical Engineering and Computer Science at the University of Michigan,Ann Arbor,MI, USA.e-mail:{worthena,stark}@.This research was supported by the Department of Defense Research&Engineering (DDR&E)Multidisciplinary University Research Initiative(MURI) on“Low-Energy Electronics Design for Mobile Platforms”and man-aged by the Army Research Office(ARO)under grant DAAH04-96-1-0377.A.Worthen was supported by a National Science Foundation Graduate Research Fellowship.graph-based descriptions of iterative receivers turn out to be extremely compact and intuitively attractive.Most ex-isting iterative receivers,including the internal algorithms of SISO modules,are easily derived in the factor graph framework.Because the factor graph theory completely defines the message update equations,maintaining soft in-formation throughout the algorithm and properly treating extrinsic information are handled automatically.Theflexi-bility of our approach,including our canonical distributions framework for handling continuous variables,makes it easy to derive alternative receivers of varying complexity and performance.In Section II,we describe our notation and very briefly summarize the theory of factor graphs follow-ing[10].The design approach is outlined in Section III.In Section IV,we present several simple design examples il-lustrating the approach and demonstrating its convenience andflexibility.Section V summarizes our conclusions.II.Notation and Preliminaries Consider a multiplicative factorization of a global func-tiong(x1,x2,...,x n)=jf j(x Qj)(1)where x Qjare sets of elements from{x1,x2,...,x n}.The factor graph corresponding to the factorization consists of a variable node for each variable x i,a factor node for each function f j,and edges connecting each factor node to the variable nodes associated with its arguments.Thus,the factor graph is a bipartite graph where edges only connect variable nodes to factor nodes.In drawing factor graphs, we typically usefilled circles for factor nodes and open circles for variable nodes.Let uppercase letters specify nodes in the factor graph and use lowercase letters for their associated variables.The indicator function I{statement}is1if the statement is true and0otherwise.Messages are functions passed on the edges of the graph.Denote the message from node X to node Y byµX→Y(·)and note that,since either X or Y is a variable node,it is unambiguous to say thatµX→Y(·)is a function of its associated variable(either X or Y).The sum-product algorithm approximates the marginals ˆg(x k)of the global function with respect to all variables x k,k=1,2,...,n,whereˆg(x k)=x j:j=kg(x1,x2,...,x n).(2)The algorithm is described as message passing on the factor graph and proceeds as follows.All the messages are initial-ized to the constant function1.Then,the messages leav-ing each node are updated according to a schedule which specifies the sequence in which the nodes are to be pro-cessed.General formulas for computing the new outgoing messages in terms of the incoming messages as well as some recursions for efficiently computing the updates are given in[9,10].If the factor graph isfinite and has no cycles,then there exists a schedule that computes the exact marginalsin afinite number of node updates,and such a schedule is usually easy tofind.III.Design of Iterative Receivers Consider a typical communication system where the mes-sage to be transmitted s=q(s,corrupted by the chan-nel,is then processed by the receiver to produce an es-timate of the transmitted messageˆsthat evolves randomly with time.The channel is described by a conditional density for the channel output given the current channel state,the current input,and,for inter-symbol interference(ISI)channels,several past in-puts.Extensions to multiple-access channels;multi-input, multi-output channels;and recording channels are also pos-sible.The optimal receiver for this model,with respect to bit error probability isˆs i arg maxs i p(s i|y:s i=ξu,u).(3)Because of the systematic encoding assumption,the s i vari-ables are equal to a subset of the x i’s,so optimal decod-ing of the x i’s yields the optimal s i’s as well.Supposing that the messages s:x i=ξu|x)p(u∈},(4)but this is usually too complex to implement directly. Our approach is to factor the objective function p(y,u)I{x∈}are known[11].Our assumptions about the chan-nel structure imply that the channel transition density p(y,u|x)=N−1i=L+1p(y i|u,x i, (x0)(5)where L is the length of the ually,a natural fac-torization of the channel state density p(u∈}and updating C should be taken as performing a single update iteration for the decoder.Any interleaving between the code and the channel can also be absorbed into the code definition.We assume binary phase shift keying (BPSK)signaling with the mapping{0,1}⇒{+1,−1}.A.Block Fading ChannelThe coherent block fading channel[16],a popular model for frequency-hopping and time-division multiple accessWORTHEN AND STARK:UNIFIED DESIGN OF ITERATIVE RECEIVERS 3X N-1T N-1Fig.1.Factor graph for block fading channel with m =3.CI {x m);Πkp(u k ).(TDMA)systems,divides the transmitted codeword into hops of length m symbols.The fading level within each hop is assumed to be constant and the fading levels are inde-pendent Rayleigh random variables.The received value is further corrupted by additive white Gaussian noise.Thus,if the fading level in the k th hop is u k ,the received values are y i =u i|x )p(u ∈}=I {xm)·Nm)is a Gaussian density with mean u iis included in the func-tions for the T nodes.We elect to update the nodes in the following order:T ,U ,T ,X ,C ,X ;and repeat until a stopping criterion,such as the current decisions forming a codeword,is satisfied.We discretize the channel state messages µT i →U k (u k )with a canonical distribution corre-sponding to a weighted sum of impulsesµT i →U k (u k )=6 l =1a l δ(u k −ˆu l )(7)where ˆu l are -12,-9,-6,-3,0,and 3dB respectively andthe coefficients are found by sampling the message func-tion derived from the update equations for T i .We use the usual variable update equation for the U nodes assuming δ(x )δ(y )=0if x =y and δ(x )δ(y )=δ(x )if x =y ,so the outgoing messages µU k →T j (u k )are also of the form in (7).Because the channel state only affects the reliability of the bit estimate and because the channel state estimate is not very sensitive to the bit likelihoods,the error introduced by this simple quantization does not seem to degrade the performance very much.Kang and Stark [17]derive a similar receiver for a finite-state block interference channel,where the noise variance instead of the fading level depends on the channel state.They divide the channel state estimation into separate components for the two component codes,but the over-Fig.2.Performance of (1024,512)rate 1/2LDPC code on blockfading channel as a function of the hop length.channel state information,×channel estimation,◦no channel estimation.all effect is the same as the receiver above with an alter-nate update schedule that captures the separate state ing a regular LDPC code instead of a turbocode,we found results,shown in Fig.2,similar to [17].As the memory increases,the performance gap between the channel estimation iterative receiver and an iterative de-coder using perfect knowledge of the channel state narrows.However,for fixed codeword length the reduced number of hops per codeword increases the SNR required for a given bit error rate.Operating closer to capacity seems to make channel memory more valuable,forcing the optimal hop length to be strictly greater than one [18].B.Rayleigh Fading ChannelThe frequency-flat,time-selective,Rayleigh fading chan-nel,which has been modeled by first-order and second-order Markov models [19,20],is particularly important for wireless communication systems.Coherent models with finite-states have been treated in [21–23].In the non-coherent case,pilot symbol-assisted modulation (PSAM)or differentially-encoded phase-shift keying (DPSK)mod-ulation is often used for robustness against the phase un-certainty.The non-coherent Rayleigh channel and its close relatives have been extensively studied in the iterative de-tection literature [24–30].We consider a discrete-time first-order model where the continuous-valued channel state u k represents the zero-mean complex Gaussian fading level and the fading pro-cess is stationary and Markov with conditionally Gaus-sian transition pdf p(u k |u k −1).The channel output y i =u i (−1)x i +n i is conditionally Gaussian.In order to help re-solve the phase ambiguity,we insert L known pilot symbols at regular intervals within the codeword x C ∈is the part corresponding to the error-control code andxr r s n s h il symb ls hi h ar all Th4IEEE TRANSACTIONS ON INFORMATION THEORY,VOL.XX,NO.Y,MONTH2001U......U U UP CU......U U UCa)b)Fig.3.Factor graphs for non-coherent Rayleigh channel.a)PSAM with pilots every third symbol,b)DPSK.C I{x P=0 |x)p(u∈ }=I{x P=0according to x i+1=x i+x i mod2withx 0=0and transmit x)indicatesthe differential encoding of x|x)p(u∈}·I{x)}=I{xWORTHEN AND STARK:UNIFIED DESIGN OF ITERATIVE RECEIVERS5......U UΠΠΠΠ......U Ua)b)Fig.4.Alternate factor graphs for channel portion.a)amplitude-phase:Πk p(uα,k,uθ,k|uα,k−1,uθ,k−1);T i p(y i|x i,uα,i,uθ,i);b) real-imaginary:ΠI,k p(u I,k|u I,k−1);ΠQ,k p(u Q,k|u Q,k−1);T i p(y i|x i,u I,i,u Q,i).TX...C...a)b)C......c)Fig. 5.Factor graphs for the multi-path fading channel.a)direct,b)modulation state,c)reduced state.C I{x ,x i,x i−1,...,x i−L)or p(y i|u,x i,h i,x i−J,...,x i−L);Πp(uare known[36–38].However,be-cause the wireless channel is time-varying,we often cannotassume that a perfect estimate of the channel response isavailable.We consider the case where the coefficients arezero-mean,independent,complex Gaussian random vari-ables that are constant for the duration of the codeword,but change between ing(5)and our assump-tions about the channel structure,we can factor the likeli-hood function asp(y,u)I{x∈}·N−1i=0p(y i|u|x)p(u∈}=I{x,x i,h i)I{h i+1=[x i,˜h i]}·Lk=0p(u k)(15)where˜h i is the vector containing thefirst L−1elements ofh i.When we consider p(y i|u6IEEE TRANSACTIONS ON INFORMATION THEORY,VOL.XX,NO.Y,MONTH2001...C...Fig.6.Factor graph for multi-path fading channel using outputestimates.C I{x);Λ0I{h0all zero}.hybrid,reduced state approach,reminiscent of decision-feedback sequence estimation[39],has the factor graphshown in Fig.5c.The actual distribution for the messagesµTi→U turns outto be a Gaussian mixture with2L+1terms corresponding to the2L+1possible values for[x i,h i].We might consider canonical distributions similar to those used in the previous section.However,even if the complete distribution is used with the structure in(15),there is still a two-fold phase ambiguity in each coefficient due to the unknown data.By comparison,a joint equalizer and channel estimator can use the shift properties of the data sequence to determine the channel u,we can reduce the ambiguity to a single common phase factor.In our approach,this leads to the factorization shown in Fig.6,p(y,v∈}=I{x,x i,h i)I{h i+1=[x i,˜h i]}(16) where each element of v, and note that,for given x i and h i,p(y i|v.From the factor graph update equations and a little algebra,wefind thatµVk→T i (v k)=N−1l=0l=i[p(y l|v k,x l(k),h l(k))P l,k+1](17)whereP l,k=1l=iP l,k(18)which matches the form of the Baum-Welch algorithm-based channel estimator in[41]except that we do not useinformation from stage i in the estimate of v k for stage i.This could be implemented by having each node T l passP l,k and y l in its message to node V k which could thencompute theˆv k,i for its outgoing messages.Clearly,othercanonical distributions forµVk→T iare possible and mightachieve better performance.It is also possible to furtherconstrain the vWORTHEN AND STARK:UNIFIED DESIGN OF ITERATIVE RECEIVERS7the International Communications Conference,May1993,pp.1064–1070.[2]S.Benedetto, D.Divsalar,G.Montorsi,and F.Pollara,“Se-rial concatenation of interleaved codes:Performance analysis, design,and iterative decoding,”Telecommunications and Data Acquisition Progress Report,vol.42-126,Aug.1996.[3]R.G.Gallager,“Low-density parity-check codes,”IRE Trans-actions on Information Theory,vol.IT-8,no.1,pp.21–28,Jan.1962.[4]J.Hagenauer,“The Turbo principle:Tutorial introduction andstate of the art,”in Proceedings of the International Symposium on Turbo Codes,Brest,France,Sept.1997,pp.1–11.[5]R.M.Tanner,“A recursive approach to low complexity codes,”IEEE Transactions on Information Theory,vol.IT-27,no.5, pp.533–547,Sept.1981.[6]N.Wiberg,Codes and Decoding on General Graphs,Ph.D.thesis,Link¨o ping University,Link¨o ping,Sweden,1996.[7]N.Wiberg,H.-A.Loeliger,and R.Koetter,“Codes and iter-ative decoding on general graphs,”European Transactions on Telecommunications,vol.6,no.5,pp.513–525,Sept.–Oct.1995.[8]R.J.McEliece,D.J.C.MacKay,and J.-F.Cheng,“Turbo de-coding as an instance of Pearl’s“belief propagation”algorithm,”IEEE Journal on Selected Areas in Communications,vol.16,no.2,pp.140–152,Feb.1998.[9] B.J.Frey,F.R.Kschischang,H.-A.Loeliger,and N.Wiberg,“Factor graphs and algorithms,”in Proceedings of the35th Annual Allerton Conference on Communication,Control and Computing,Sept.1997,pp.666–680.[10] F.R.Kschischang, B.J.Frey,and H.-A.Loeliger,“Factor graphs and the sum-product algorithm,”m.utoronto.ca/frank/factor/,submitted to IEEE Transactions on Information Theory,1998.[11] F.R.Kschischang and B.J.Frey,“Iterative decoding of com-pound codes by probability propagation in graphical models,”IEEE Journal on Selected Areas in Communications,vol.16, no.2,pp.219–230,Feb.1998.[12]R.K¨o tter and A.Vardy,“Factor graphs:Constructions,classi-fication,and bounds,”in Proceedings of the1998International Symposium on Information Theory ISIT’98,Cambridge,MA, Aug.1998,p.14.[13] A.Jimenez,Soft Iterative Decoding of Low-Density Convolu-tional Codes,Engineering licenciate thesis,Lund University, Oct.1997.[14] A.P.Worthen and W. E.Stark,“Low-density parity checkcodes for fading channels with memory,”in Proceedings of the 36th Annual Allerton Conference on Communication,Control, and Computing,Sept.1998,pp.117–125.[15]S.M.Aji and R.J.McEliece,“The generalized distributive law,”IEEE Transactions on Information Theory,vol.46,no.2,pp.325–349,Mar.2000.[16]R.J.McEliece and W.E.Stark,“Channels with block interfer-ence,”IEEE Transactions on Information Theory,vol.IT-30, no.1,pp.44–53,Jan.1984.[17]J.H.Kang and W.E.Stark,“Turbo codes for noncoherentFH-SS with partial band interference,”IEEE Transactions on Communications,vol.46,no.11,pp.1451–1458,Nov.1998. [18] A.P.Worthen and W.E.Stark,“Interference mitigation infrequency-hopped spread-spectrum systems,”in Proceedings of the IEEE6th International Symposium on Spread-Spectrum Techniques and Applications ISSSTA’00,New Jersey,USA, Sept.2000,vol.1,pp.58–62.[19]H.S.Wang and N.Moayeri,“Finite-state Markov channel—auseful model for radio communication channels,”IEEE Trans-actions on Vehicular Technology,vol.44,no.1,pp.163–171, Feb.1995.[20]M.J.Chu,D.L.Goeckel,and W.E.Stark,“On the designof Markov models for fading channels,”in Proceedings of the IEEE Vehicular Technology Conference Fall,Amsterdam,The Netherlands,Sept.1999.[21]J.H.Kang,W.E.Stark,and A.O.Hero,“Turbo codes forfading and burst channels,”in Proceedings of the IEEE Global Communications Conference GLOBECOM’98:Communica-tion Theory Mini-Conference,Nov.1998,pp.40–45.[22]J.Garcia-Frias and J.D.Villasenor,“Simplified methods forcombining hidden Markov models and turbo codes,”in Proceed-ings of the50th IEEE Vehicular Technology Conference VTC ’99,Amsterdam,The Netherlands,Sept.1999,pp.1580–1584.[23]J.Garcia-Frias and J.D.Villasenor,“Exploiting binary Markovchannels with unknown parameters in turbo decoding,”in Proceedings of the IEEE Global Communications Conference GLOBECOM’98,Sydney,Australia,Nov.1998,pp.3244–3249.[24] E.K.Hall and S.G.Wilson,“Turbo codes for noncoherentchannels,”in Proceedings of the IEEE Global Communications Conference GLOBECOM’97:Communication Theory Mini-Conference,Phoenix,AZ,1997.[25]M.Peleg and S.Shamai(Shitz),“Iterative decoding of codedand interleaved noncoherent multiple symbol detected DPSK,”Electronics Letters,vol.33,no.12,pp.1018–1020,June1997.[26]P.Hoeher and J.Lodge,“‘Turbo DPSK’:Iterative differentialPSK demodulation and channel decoding,”IEEE Transactions on Communications,vol.47,no.6,pp.837–843,June1999. [27]M.C.Valenti and B.D.Woerner,“A bandwidth efficient pi-lot symbol technique for coherent detection of turbo codes over fading channels,”in Proceedings of the1999Military Commu-nications Conference MILCOM’99,Oct.1999,paper3.4. [28]H.-J.Su and E.Geraniotis,“Improved performance of a PSAMsystem with iterativefiltering and decoding,”in Proceedings of the36th Annual Allerton Conference on Communication,Con-trol,and Computing,Sept.1998,pp.156–166.[29]Y.Xu,H.-J.Su,and E.Geraniotis,“Pilot symbol assisted QAMwith iterativefiltering and turbo decoding over Raleighflat fad-ing channels(sic),”in Proceedings of the1999Military Com-munications Conference MILCOM’99,Oct.1999,paper3.5.[30] C.Komninakis and R.D.Wesel,“Pilot-aided joint data andchannel estimation inflat correlated Rayleigh fading,”in Proceedings of the IEEE Global Communications Conference GLOBECOM’99,1999.[31] A.P.Worthen and W.E.Stark,“On iterative receivers fornon-coherent channels,”To appear in Proceedings of the IEEE International Symposium on Information Theory and Its Appli-cations ISITA’00,Honolulu,HI,Nov.2000.[32]I.D.Marsland and P.T.Mathiopoulos,“Multiple differentialdetection of parallel concatenated convolutional(turbo)codes in correlated fast Rayleigh fading,”IEEE Journal on Selected Areas in Communications,vol.16,no.2,pp.265–274,Feb.1998.[33]K.R.Narayanan,“Iterative demodulation and decoding of trel-lis coded CPM,”in Proceedings of the1999Military Communi-cations Conference MILCOM’99,Oct.1999,paper42.1. [34]I.D.Marsland and P.T.Mathiopoulos,“On the performance ofiterative noncoherent detection of coded M-PSK signals,”IEEE Transactions on Communications,vol.48,no.4,pp.588–596, Apr.2000.[35] C.Komninakis and R.D.Wesel,“Iterative joint channel es-timation and decoding inflat correlated Rayleigh fading,”in Proceedings of the7th International Conference on Advances in Communications and Control,Athens,Greece,June1999.[36] C.Douillard,M.J´e z´e quel,C.Berrou,A.Picart,P.Didier,andA.Glavieux,“Iterative correction of intersymbol interference:Turbo-equalization,”European Transactions on Telecommuni-cations,vol.6,no.5,pp.507–511,Sept.-Oct.1995.[37] D.Raphaeli and Y.Zarai,“Combined turbo equalization andturbo decoding,”in Proceedings of the International Symposium on Turbo Codes,Brest,France,Sept.1997,pp.180–183. [38] A.Glavieux,ot,and bat,“Turbo equalization over afrequency selective channel,”in Proceedings of the International Symposium on Turbo Codes,Brest,France,Sept.1997,pp.96–102.[39] A.Duel-Hallen and C.Heegard,“Delayed decision-feedback se-quence estimation,”IEEE Transactions on Communications, vol.37,no.5,pp.428–436,May1989.[40] C.Ant´o n-Haro,J.A.R.Fonollosa,and J.R.Fonollosa,“Blindchannel estimation and data detection using hidden Markov models,”IEEE Transactions on Signal Processing,vol.45,no.1,pp.241–247,Jan.1997.[41]J.Garcia-Frias and J.D.Villasenor,“Blind turbo decoding andequalization,”in Proceedings of the IEEE Vehicular Technology Conference VTC Spring’99,Houston,TX,May1999,vol.3, pp.1881–1885.[42]Z.-N.Wu and J.M.Cioffi,“Turbo decision aided equalizationfor magnetic recording channels,”in Proceedings of the IEEE Global Telecommunications Conference GLOBECOM’99,1999, pp.733–738.Andrew Worthen received the B.S.(with highest honors)and M.S.degrees from the University of Michigan,Ann Arbor,MI,in8IEEE TRANSACTIONS ON INFORMATION THEORY,VOL.XX,NO.Y,MONTH20011997and1999respectively.In1997,he was an intern at TellabsResearch Center,Mishawaka,IN.His research interests include low-density parity check codes,iterative decoding,and iterative receiversfor fading channels.He is currently pursuing his Ph.D.at the Uni-versity of Michigan.Wayne Stark received the B.S.(with highest honors),M.S.,andPh.D.degrees in electrical engineering from the University of Illinois,Urbana in1978,1979,and1982respectively.Since September1982he has been a faculty member in the Department of Electrical En-gineering and Computer Science at the University of Michigan,AnnArbor where he is currently Professor.From1984-1989he was Editorfor Communication Theory of the IEEE Transactions on Communi-cation in the area of Spread-Spectrum Communications.He wasinvolved in the planning and organization of the1986InternationalSymposium on Information Theory which was held in Ann Arbor,Michigan.He was selected by the National Science Foundation asa1985Presidential Young Investigator.He is principal investiga-tor of a Army Research Office Multidisciplinary University ResearchInitiative(MURI)project on Low Energy Mobile Communications.His research interests are in the areas of coding and communicationtheory,especially for spread-spectrum and wireless communicationnetworks.Dr.Stark is a member of Eta Kappa Nu,Phi Kappa Phiand Tau Beta Pi and a Fellow of the IEEE.。