Bayesian Bounds for Parameter Estimation and Nonlinear Filtering/tracking

The first comprehensive development of Bayesian Bounds for parameter estimation and nonlinear filtering/tracking Bayesian estimation plays a central role in many signal processing problems encountered in radar, sonar, communications, seismology, and medical diagnosis. There are often highly nonlinear problems for which analytic evaluation of the exact performance is intractable. A widely used technique is to find bounds on the performance of any estimator and compare the performance of various estimators to these bounds. This book provides a comprehensive overview of the state of the art in Bayesian Bounds. It addresses two related problems: the estimation of multiple parameters based on noisy measurements and the estimation of random processes, either continuous or discrete, based on noisy measurements. An extensive introductory chapter provides an overview of Bayesian estimation and the interrelationship and applicability of the various Bayesian Bounds for both static parameters and random processes. It provides the context for the collection of papers that are included. This book will serve as a comprehensive reference for engineers and statisticians interested in both theory and application. It is also suitable as a text for a graduate seminar or as a supplementary reference for an estimation theory course.Table of ContentPreface. Introduction (Harry L. Van Trees and Kristine L. Bell). 1 Bayesian Estimation: Static Parameters. 1.1 Maximum Likelihood and Maximum a Posteriori Estimation. 1.1.1 Nonrandom Parameters. 1.1.2 Random Parameters. 1.1.3 Hybrid Parameters. 1.1.4 Examples. 1.2 Covariance Inequality Bounds. 1.2.1 Covariance Inequality. 1.2.2 Bayesian Bounds. 1.2.3 Scalar Parameters. 1.2.3.1 Bayesian Cramer-Rao Bound. 1.2.3.2 Weighted Bayesian Cramer-Rao Bound. 1.2.3.3 Bayesian Bhattacharyya Bound. 1.2.3.4 Bobrovsky-Zakai Bound. 1.2.3.5 Weiss-Weinstein Bound. 1.2.4 Vector Parameters. 1.2.4.1 Bayesian Cramer-Rao Bound. 1.2.4.2 Weighted Bayesian CRB. 1.2.4.3 Bayesian Bhattacharyya Bound. 1.2.4.4 Bobrovsky-Zakai Bound. 1.2.4.5 Weiss-Weinstein Bound. 1.2.5 Combined Bayesian Bounds. 1.2.6 Nuisance Parameters. 1.2.6.1 Nonrandom Unwanted Parameters. 1.2.6.2 Random Parameters. 1.2.7 Hybrid Parameters. 1.2.8 Functions of the Parameter Vector. 1.2.8.1 Scalar Parameters. 1.2.8.2 Vector Parameters. 1.2.9 Summary: Covariance Inequality Bounds. 1.3 Ziv-Zakai Bounds. 1.3.1 Scalar Parameters. 1.3.2 Equally Likely Hypotheses. 1.3.3 Vector Parameters. 1.4 Method of Interval Estimation. 1.5 Summary. 2 Bayesian Estimation: Random Processes. 2.1 Continuous-Time Processes and Continuous-Time Observations. 2.1.1 Nonlinear Models. 2.1.1.1 Linear AWGN Process and Observations. 2.1.1.2 Linear AWGN Process, Nonlinear AWGN Observations. 2.1.1.3 Nonlinear AWGN Process and Observations. 2.1.1.4 Nonlinear Process and Observations. 2.1.2 Bayesian Cramer-Rao Bounds: Continuous-Time. 2.2 Continuous-Time Processes and Discrete-Time Observations. 2.2.1 Extended Kalman Filter. 2.2.2 Bayesian Cramer-Rao Bound. 2.2.3 Discretizing the Continuous-Time State Equation. 2.3 Discrete-Time Processes and Discrete-Time Observations. 2.3.1 Linear AWGN Process and Observations. 2.3.2 General Nonlinear Model. 2.3.2.1 MMSE and MAP Estimation. 2.3.2.2 Extended Kalman Filter. 2.3.3 Recursive Bayesian Cramer-Rao Bounds. 2.4 Global Recursive Bayesian Bounds. 2.5 Summary. 3 Outline of the Book. Part I Bayesian Cramer-Rao Bounds. 1.1 H. L. Van Trees, Excerpts from Part I of Detection, Estimation, and Modulation Theory, pp. 66-86, Wiley, New York, 1968 (reprinted Wiley 2001). 1.2 M. P. Shutzenberger," A generalization of the Frechet-Cramer inequality in the case of Bayes estimation," Bulletin of the American Mathematical Society, vol. 63, no. 142, 1957. Part II Global Bayesian Bounds. 2.1 B. Z. Bobrovsky, E. Mayer-Wolf, and M. Zakai, "Some classes of global Cramer-Rao bounds," Ann. Stat., vol. 15, pp. 1421-1438, 1987. 2.2 H. L. Van Trees, Excerpts from Part I of Detection, Estimation, and Modulation Theory, pp. 273-286, Wiley, New York, 1968 (reprinted 2001). 2.3 D. Rife and R. Boorstyn, "Single-tone parameter estimation from discrete-time observations," IEEE Trans. Inform. Theory, vol. IT-20, no. 5, pp. 591-598, September 1974. 2.4 R. J. McAulay and E. M. Hostetter, "Barankin bounds on parameter estimation," IEEE Trans. Info. Theory, vol. IT-17, no. 6, pp. 669-676, November 1971. 2.5 R. Miller and C. Chang, "A modified Cramer-Rao bound and its applications, IEEE Trans. Info. Theory, vol. 24, no. 3, pp. 398-400, May 1978. 2.6 A. Weiss and E. Weinstein, "A lower bound on the mean-square error in random parameter estimation," IEEE. Trans. Info. Theory, vol. 31, no. 5, pp. 680-682, September 1985. 2.7 E. Weinstein and A. J. Weiss, "Lower bounds on the mean square estimation error," Proceedings of the IEEE, vol. 73, no. 9, pp. 1433-1434, September 1985. 2.8 E. Weinstein and A. J. Weiss, "A general class of lower bounds in parameter estimation," IEEE Trans. Info. Theory, vol. 34, no. 2, pp. 338-342, March 1988. 2.9 J. S. Abel, "A bound on mean-square-estimate error," IEEE. Trans. Info. Theory, vol. 39, no. 5, pp. 1675-1680, September 1993. 2.10 A. Renaux, P. Forster, P. Larzabal, and C. Richmond, "The Bayesian Abel bound on the mean square error," ICASSP 2006, vol. 3, pp. III-9-12, Toulouse, France. 2.11 J. Ziv and M. Zakai, "Some lower bounds on signal parameter estimation," IEEE. Trans. Info. Theory, vol. IT-15, no. 3, pp. 386-391, May 1969. 2.12 L. P. Seidman, "Performance limitations and error calculations for parameter estimation," Proc. IEEE, vol. 58, no. 5, pp. 644-652, May 1970. 2.13 D. Chazan, M. Zakai, and J. Ziv, "Improved lower bounds on signal parameter estimation," IEEE Trans. Info. Theory, vol. IT-21, no. 1, pp. 90-93, Jan. 1975. 2.14 S. Bellini and G. Tartara, "Bounds on error in signal parameter estimation," IEEE. Trans. Commun., vol. COM-22, pp. 340-342, March 1974. 2.14a S. Bellini and G. Tartara, "Corrections to 'Bounds on error in signal parameter estimation,' " IEEE Trans. Commun., vol. 23, no. 4, p. 486, April 1975. 2.15 M. Wax and J. Ziv, "Improved bounds on the local mean-square error and the bias of parameter estimators," IEEE. Trans. Info. Theory, vol. 23, no. 4, pp. 529-530, July 1977. 2.16 E. Weinstein, "Relations between Belini-Tartara, Chazan-Zakai-Ziv, and Wax-Ziv lower bounds," IEEE. Trans. Info. Theory, vol. 34, no. 2, pp. 342-343, March, 1988. 2.17 K. L. Bell, Y. Steinberg, Y. Ephraim, and H. L. Van Trees, "Extended Ziv-Zakai lower bound for vector parameter estimation," IEEE. Trans. Info. Theory, vol. 43, no. 2, pp. 624-637, March 1997. 2.18 K. L. Bell, Y. Ephraim, and H. L. Van Trees, "Explicit Ziv-Zakai lower bound for bearing estimation," IEEE. Trans. Signal Process., vol. 44, no. 11, pp. 2810-2814, November 1996. 2.19 S. Basu and Y. Bresler, "A global lower bound on parameter estimation error with periodic distortion functions," IEEE. Trans. Info. Theory, vol. 46, no. 3, pp. 1145-1150, May 2000. 2.20 H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part III, Chapter 10, pp. 275-308, Wiley, New York, 1971 (reprinted Wiley 2001). 2.21 F. Athley, "Threshold region performance of maximum likelihood direction of arrival estimators," IEEE Trans. on Signal Process., vol. 53, no. 4, pp. 1359-1373, April 2005. 2.22 C. D. Richmond, "Capon algorithm mean-squared error threshold SNR prediction and probability of resolution," IEEE Trans. on Signal Process., vol. 53, no. 8, pp. 2748-2764, August 2005. 2.23 C. Richmond, "Mean-squared error and threshold SNR prediction of maximum-likelihood signal parameter estimation with estimated colored noise covariances," IEEE. Trans. Info. Theory, vol. 52, no. 5, pp. 2146-2164, May 2006. 2.24 L. Najjar-Atallah, P. Larzabal, and P. Forster, "Threshold region determination of ML estimation in known phase data-aided frequency synchronization," IEEE Signal Process. Letters, vol. 12, no. 9, pp. 605-608, September 2005. 2.25 L. D. Brown and R. C. Liu, "Bounds on the Bayes and minimax risk for signal parameter estimation," IEEE. Trans. Info. Theory, vol. 39, no. 4, pp. 1386-1394, July 1993. 2.26 J. K. Thomas, L. L. Scharf, and D. W. Tufts, "The probability of a subspace swap in the SVD," IEEE Trans. Signal Process., vol. 43, no. 3, pp. 730-736, March 1995. Part III Hybrid Bayesian Bounds. 3.1 Y. Rockah and P. M. Schultheiss, "Array shape calibration using sources in unknown locations-Part I: far-field sources," IEEE Trans. Acoust., Speech Signal Process., vol. 35, no. 3, pp. 286-299, March 1987. 3.2 I. Reuven and H. Messer, "A Barankin-type lower bound on the estimation error of a hybrid parameter vector," IEEE. Trans. Info. Theory, vol. 43, no. 3, pp. 1084-1093, May 1997. 3.3 J. Tabrikian and J. Krolik, "Efficient computation of the Bayesian Cramer-Rao bound on estimating parameters of Markov models," IEEE Conf. Acoustics, Speech, and Sig. Process., ICASSP'99, pp. 1761-1764, 1999. 3.4 S. Buzzi, M. Lops, and S. Sardellitti, "Further results on Cramer-Rao bounds for parameter estimation in long-code DS/CDMA systems," IEEE Trans. Sig. Process., vol. 53, no. 3, pp. 1216-1221, March 2005. 3.5 P. Tichavsky' and K. Wong, "Quasi-fluid-mechanics-based quasi-Bayesian Cramer-Rao bounds for deformed towed-array direction finding," IEEE Trans. Signal Process., vol. 52, no. 1, pp. 36-47, Jan. 2004. Part IV Constrained Cramer-Rao Bounds. 4.1 J. D. Gorman and A. O. Hero, "Lower bounds for parametric estimation with constraints," IEEE. Trans. Info. Theory, vol. 36, no. 6, pp. 1285-1301, November 1990. 4.2 T. L. Marzetta, "A simple derivation of the constrained multiple parameter Cramer-Rao bound," IEEE. Trans. Signal Process., vol. 41, no. 6, pp. 2247-2249, June 1993. 4.3 P. Stoica and B. C. Ng, "On the Cramer-Rao bound under parametric constraints," IEEE Signal Process Letters, vol. 5, no. 7, pp. 177-179, July 1998. 4.4 T. J. Moore, B. M. Sadler, and R. J. Kozick, "Regularity and strict identifiability in MIMO