|
|
|
|
|
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Two econometric textbooks are used as references in this course. They are
Other major sources:
A major emphasis is on nonparametric identification and robust Bayesian computation. For background see Bayesian Methods of Inference and Computation Berger, J.O., (1985). Statistical Decision Theory and Bayesian Analysis , Springer. Gelman, A., Carlin, J.B., Stern, H.S., and Rubin, D.B. (1995). Bayesian Data Analysis , Chapman and Hall. Jaynes, E.T., (1994). Probability: The logic of science , unpublished book. Chapters may be downloaded from http://bayes.wustl.edu/etj/prob.html Jeffreys, H., (1961). Theory of Probability , Oxford University Press. Liu, J.S. (2001). Monte Carlo Strategies in Scientific Computing , Springer-Verlag, New York. Robert, C.P. and G. Casella (1999). Monte Carlo Statistical Methods , Springer-Verlag, New York. Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics , Wiley.
1. Classical Discrete Choice Models A. Theory of Classical Parametric Discrete Choice and Econometric Methods Ben-Akiva, M., D. Bolduc, and J. Walker (2001), "Specification, Identification, & Estimation of the Logit Kernel (or Continuous Mixed Logit) Model", unpublished ms, MIT. Dagsvik, J. (1994), "The Continuous Generalized Extreme Value Model with Special Reference To Static Models of Labor Supply", Econometrica . Domencich, T. and D. McFadden (1975). Urban Travel Demand: A Behavioral Analysis , North Holland. Hausman, Jerry and D. Wise (1978), "Conditional Probit Model For Qualitative Choice: Discrete Decisions Recognizing Interdependence and Heterogeneous Preferences", Econometrica , 46: pp. 403-426. McFadden, D. (1973), "Conditional Logit Analysis of Qualitative Choice Behavior", in Frontiers of Econometrics , ed. by P. Zarembka, Academic Press, N.Y., pp. 105-42. ____ (1981), "Econometric Models of Probabilistic Choice", in D. McFadden and C. Manski, Structural Analysis of Discrete Choice with Econometric Applications , MIT Press. ____ (1984), "Qualitative Response Models", in Z. Griliches and M. Intriligator, eds., Handbook of Econometrics , North Holland: Amsterdam. McFadden, D. and C. Manski, "Alternative Estimators and Sample Designs for Discrete Choice Analysis", in D. McFadden and C. Manski, op.cit. Yellot, J. (1977), "The Relationship Between Luce's Choice Axiom, Thurstone's Theory of Comparative Judgement and the Double Exponential Distribution", Journal of Mathematical Psychology , Vol. 15, pp 109-144.
B. Andersen, E. B. (1980), Discrete Statistical Models with Social Science Applications , North Holland, chapters 2, 3, 6. Bishop, E., S. Fienberg and P. Holland (1975). Discrete Multivariate Analysis , MIT Press. McCullagh, P. and J. Nelder (1990), Generalized Linear Models , Second Edition, Chapman Hall. C. Computation in Discrete Choice Models Chib, S. (2001), "Markov Chain Monte Carlo Methods: Computation and Inference", Chapter 57 in J. Heckman and E. Leamer eds., Handbook of Econometrics , Vol. 5. Geweke, J. and M. Keane (2001), "Computationally Intensive Methods for Integration in Econometrics'', Chapter 56 in Handbook of Econometrics, Vol. 5 McCulloch, R.E. and P. E. Rossi, (1996), "Bayesian Analysis of the Multinomial Probit Model", Working paper, GSB, The University of Chicago. D. Nonparametric and Semiparametric Discrete Choice Cosslett, S. (1983), "Distribution Free Maximum Likelihood Estimator of The Binary Choice Model", Econometrica , Vol. 51, pp. 765-782. Han, A. (1987), "Nonparametric Analysis of A Generalized Regression Model: The Maximum Rank Correlation Function", Journal of Econometrics , Vol. 35, pp. 303-316. Manski, C. (1985), "Semiparametric Analysis of Discrete Choice: Asymptotic Properties of The Maximum Score Estimator", Journal of Econometrics , 27, pp. 313-333. ____ (1988), "The Identification of Binary Response Models", J ournal of the American Statistical Association , September. Matzkin, R. (1992), "Nonparametric and Distribution Free Estimation of Threshold Crossing and Binary Choice Models", Econometrica (60) 2: 239-270. ____ (1993), "Nonparametric Identification and Estimation of Polychotomous Choice Models," Journal of Econometrics , 58, 137-68. ____ (1994), "Restrictions of Economic Theory in Nonparametric Methods", Handbook of Econometrics , Vol. IV, Chapter 42. Thompson, T.S. (1989). "Identification of Semiparametric Discrete Choice Models", Discussion Paper 249, University of Minnesota.
2. Background in Nonparametric Regression Methods Ichimura, H. (1995), "Semiparametric Least Squares Estimation of Single Index Models", Journal of Econometrics. Ichimura, H. and S.T. Thompson (1999), "Nonparametric Identification of Random Coefficient Binary Choice Model", Journal of Econometrics. Powell, J., J.H. Stock and T.M. Stoker (1986), "Semiparametric Estimation of Index Coefficients", Econometrica. Robinson, P.J. (1988), "Root-N-Consistent Semiparametric Regression," Econometrica .
A. Tobit Models and Extensions and Cross Section Bias Models Ahn, C. and J. Powell (1993), "Semiparametric Estimation of Censored Selection Models with a Nonparametric Selection Mechanism", Journal of Econometrics. Amemiya, T. (1985), Advanced Econometrics , Chapter 10. Andrews, D. (1993), "Asymptotic Normality of Series Estimators For Nonparametric and Semiparametric Regression Models", June, Econometrica . Gallant, R. and D. Nychka, "Seminonparametric Maximum Likelihood Estimation", Econometrica, March, 1987, pp. 363-390. Goldberger, A., "Abnormal Selection Bias", in S. Karlin, T. Amemiya and L. Goodman, Studies in Econometrics, Time Series and Multivariate Statistics , pp. 67-84, Academic Press. Gourieroux, C., J. Laffont and A. Monfort (1980), "Coherence Conditions in Simultaneous Linear Equation Models with Endogenous Switching Regimes", Econometrica , pp. 675-695. Hanemann, M. (1984), "Discrete Continuous Models of Consumer Demand", Econometrica , Vol. 52(3): 541-61. Heckman, J. (1978), "Dummy Endogenous Variables In A Simultaneous Linear Equation System", Econometrica , Vol. 46, pp. 931-959. ____ (1979), "Sample Selection Bias As A Specification Error", Econometrica , 953-161. ____ (1990), "Varieties of Selection Bias," American Economic Review , May. Heckman, J. and B. Honoré (1990), "The Empirical Content of the Roy Model", Econometrica, September. Heckman, J. and T. MaCurdy (1985), "A Simultaneous Equations Linear Probability Model", Canadian Journal of Economics . ____ (1986), "Labor Econometrics", Handbook of Econometrics , Vol. 3, Ed. by Z. Griliches and M. Intriligator, pp. 1919-1977, Amsterdam, North Holland. Heckman, J. and R. Robb (1985), "Alternative Methods For Estimating The Impact of Interventions", in J. Heckman and B. Singer, eds., Longitudinal Analysis of Labor Market Data, Cambridge University Press. Lee, L. F. (1978), "Unionism and Wage Rates: A Simultaneous Equations Models with Qualitative and Limited Dependent Variables", International Economic Review , Vol. 19, pp. 415-433. Newey, W. K. (1990), "Series Estimation of Regression Functionals", Econometric Theory , 10, 1-28. Newey, W. (1999), "Two Step Series Estimation of Sample Selection Models", MIT working paper, 99-104, Econometrica. Powell, J. (1979), "Least Absolute Deviations Estimator for Censored Regression Models," Journal of Econometrics , 25, No. 3, 303-326. ____ (1986), "Systematically Trimmed Least Squares Estimation for Tobit Models," Econometrica , 54, 1435-1460. ____ (2000), "Semiparametric Estimation of Censored Selection Models", in Nonlinear Econometric Inference , C. Hsiao, K. Morimune, and J. Powell, eds., Cambridge: Cambridge University Press.
B. Application: Taxes, Labor Supply and Consumption Blundell, R., T. MaCurdy and C. Meghir (2002), "Estimation and Specification of Labor Supply Models," forthcoming in J. Heckman and E. Leamer, eds., Handbook of Econometrics , Vol. 6, Amsterdam: North Holland. Blomquist, S. (1983), "The Effect of Income Taxation on the Labor Supply of Married Men in Sweden", J ournal of Public Economics , Vol. 22, pp. 169-197. Hausman, J. (1987), "Taxes and Labor Supply," in A. Auerbach and M. Feldstein, eds., Handbook of Public Finance , North Holland. Heckman, J. (1974), "Shadow Prices, Market Wages and Labor Supply", Econometrica , July. ____ (1975), "The Effects of Child Care Programs on Women's Work Effort", in T.W. Shultz, ed., The Economics of the Family , University of Chicago Press. ____ (1983), "Comment" in Behavioral Simulation Methods in Tax Policy Analysis , ed., by M. Feldstein, University of Chicago Press. Heckman, J. and T. MaCurdy (1981), "New Methods For Estimating Labor Supply Functions: A Survey", Research in Labor Economics , Vol. 4. Killingsworth, M., and J. Heckman, "Female Labor Supply: A Survey", in O. Ashenfelter and R. Layard, Chapter 2. C. Application: The Effects of Union on Wages Duncan, G., and D. Leigh (1985), "The Endogeneity of Union Status: An Empirical Test", Journal of Labor Economics , Vol. 3, July, pp. 385-402. Heckman, J., "Varieties of Selection Bias", op.cit. Lee, L. (1978), op.cit, IER. Lewis, H. (1986), Union Relative Wage Effects: A Survey , Chicago. Robinson, C. (1989), "The Joint Determination of Union Status and Union Wage Effects", Journal of Political Economy , June, 639-667.
D. Application: Self-Selection, Wages and Hedonic Models Card, D. and A. Krueger (1992), "Does School Quality Matter? Returns to Education and the Characteristics of Public Schools in the United States," Journal of Political Economy , 100:1; 1-40. Card, D. (1995), "Earnings, Schooling and Ability Revisited," Research in Labor Economics , 14, 23-48. _____(1999), "Estimating the Causal Effect of Schooling," In O. Ashenfelter and D. Cards eds., Handbook of Labor Economics . Carneiro, P., K. Hansen and J. Heckman (2001), "Educational Attainment and Labor Market Outcomes: Estimating Distributions of The Returns to Educational Interventions,'' unpublished paper presented as Klein Lecture, University of Pennsylvania; First Draft presented at Midwest Econometrics Group (MEG), October, 2000, forthcoming, International Economic Review , 2002. ____ (2002), "Removing the Veil of Ignorance in Assessing The Distributional Impacts of Social Policies", forthcoming SEPR and NBER. Carneiro, P., J. Heckman and E. Vytlacil (2001), "Estimating the Rate of Return to Schooling When It Varies Among Individuals", unpublished manuscript, U of Chicago, April, 2001. Ekeland, I., J. Heckman and L. Nesheim, "Identification and Estimation of Hedonic Models", May , 2001, U of Chicago, Revised November, 2001. Gorman, W. M. (1980), "A Possible Procedure For Analyzing Quality Differentials in the Egg Market," Review of Economic Studies , 47, 843-856. Griliches, Z., Price Indexes and Quality Change, Harvard University Press. (Introduction; Chapter 3, Griliches; Chapter 8, Hall). Griliches, Z. (1977), "Estimating The Returns To Schooling: Some Econometric Problems," Econometrica , 45, 1-22. Heckman, J. and B. Honoré (1990), "The Empirical Content of the Roy Model," Econometrica , 58(5), 1121-1149. Heckman, J. and R. Robb (1985), "Using Longitudinal Data to Estimate Age, Period and Cohort Effects in Earnings Equations," in William M. Mason and Stephen E. Feinberg, eds., Cohort Analysis in Social Research Beyond the Identification Problem , Springer-Verlag, New York. Heckman, J. and G. Sedlacek (1985), "Heterogeneity, Aggregation and Market Wage Functions: An Empirical Model of Self-Selection in the Labor Market," Journal of Political Economy , 93(6), 1077-1125. ____ (1990), "Self-Selection and The Distribution of Hourly Wages", Journal of Labor Economics . Heckman, J. and J. Scheinkman (1987), "The Importance of Bundling in A Gorman-Lancaster Model of Demand For Characteristics: A Theoretical Analysis With Evidence From the Labor Market", Review of Economic Studies , April, 54, 323-339. Heckman, J. and E. Vytlacil (1998), "Cognitive Ability and The Rising Return to Education", Review of Economics and Statistics , February, 2001, 1-12. Lancaster, K. (1971). Consumer Demand: A New Approach , (New York: Columbia University Press). Sattinger, M. (1991), "Self-Selection in The Labor Market: A Survey", Journal of Economic Literature. Tinbergen, J. (1956), "On The Theory of Income Distribution", Welteirtsohaftliohes Archiv, 77, pp. 155-73. Willis, R., and S. Rosen (1979), "Education and Self Selection", Journal of Political Economy , October.
3. Treatment Effects and Instrumental Variables Clements, N., J. Heckman and J. Smith (1997), "Making The Most Out of Programme Evaluations and Social Experiments: Accounting for Heterogeneity in Programme Impacts," Review of Economic Studies , 64, 487-535. Darolles, S., J.P. Florens and E. Renault (2001), "Nonparametric Instrumental Regression", unpublished manuscript, University of Toulouse. Florens, J. P. and J. Heckman (2001), "Three Notions of Causality", unpublished manuscript, presented at ECEC Conference, Louvain, Belgium, December. Florens, J. P., J. Heckman, C. Meghir and E. Vytlacil (2001), "Instrumental Variables, Local Instrumental Variables and Control Functions", unpublished manuscript, University of Chicago. Heckman, J. (1997), "Instrumental Variables Methods For the Correlated Random Coefficient Model: Estimating The Average Rate of Return to Schooling When The Return Is Correlated With Schooling," Journal of Human Resources , 33(4), 974-987, (see also discussion in Heckman, LaLonde and Smith; Heckman and Vytlacil surveys). ____ (2001), "Microdata, Heterogeneity and the Evaluation of Public Policy," Journal of Political Economy, 109(4): 673-748. Heckman, J., R. LaLonde and J. Smith (1999), "The Economics and Econometrics of Active Labor Market Programs," in O. Ashenfelter and D. Card, Chapter 31, Handbook of Labor Economics , Vol. IV, 1865-2073. Heckman, J., H. Ichimura, J. Smith, P. Todd (1998), "Characterizing Selection Bias Using Experimental Data," Econometrica , 66(5), 1017-1098. Heckman, J., H. Ichimura, J. Smith, P. Todd (1996), "Sources of Selection Bias in Evaluating Social Programs: An Interpretation of Conventional Measures and Evidence on The Effectiveness of Matching As A Program Evaluation Method," Proceedings of The National Academy of Sciences , 93, 13416-13420. Heckman, J. and R. Robb (1985), "Alternative Methods For Evaluating The Impact of Interventions," in J. Heckman and B. Singer, Longitudinal Analysis of Labor Market Data , Econometric Society Monograph 10, (Cambridge University Press). Heckman, J. and J. Smith (1995), "Assessing The Case for Social Experiments," Journal of Economic Perspectives , 9(2), 85-110. Heckman, J. and E. Vytlacil (2000), "Structural Equations, Treatment Effects and Econometric Policy Evaluation," Fisher Schultz Lecture World Congress of The Econometric Society, forthcoming, Econometrica . Heckman, J. and E. Vytlacil (2000), "Local Instrumental Variables" in Cheng Hsiao, Kimio Morimune, and James Powell, eds., Nonlinear Statistical Modeling: Proceedings of the Thirteenth International Symposium in Economic Theory and Econometrics , (Cambridge, Cambridge University Press). Heckman, J. and E. Vytlacil (2001), "Local Instrumental Variables,'' with J. Heckman, in Nonlinear Statistical Inference: Essays in Honor of Takeshi Amemiya , C. Hsiao, K. Morimune, and J. Powell eds., Cambridge: Cambridge University Press, 2000. Holland, P. (1986), "Statistics and Causal Inference", Journal of the American Statistical Association, Vol. 81, pp. 945-960. Imbens, G. and J. Angrist (1994), "Identification and Estimation of Local Average Treatment Effects," Econometrica , 62(4), 467-476. Little, R. and D. Rubin (1983), "The Central Role of the Propensity Score in Observational Studies For Causal Effects", Biometrika , Vol, 70, pp. 41-55. ____ (1985), "Constructing A Control Group Using Multivariate Matched Sampling Incorporating the Propensity Score", American Statistician , Vol. 39, pp. 33-38. Manski, C. (1994), "The Selection Problem", in C. Sims, Advances in Econometrics , Proceedings of the World Congress of the Econometric Society, Barcelona. ____ (1995). The Identification Problem In The Social Sciences, Cambridge: Harvard University Press.
4. Panel Data Models For Discrete and Continuous Time A. Basic Theory Arellano, M. and B. Honoré (2001), "Panel Data" in J. Heckman and E. Leamer, Handbook of Econometrics , Vol. V, (North Holland). Baltagi, B. (1996), Econometric Analysis of Panel Data , (New York, Wiley). Chamberlain, G. (1982), "Multivariate Regression Models For Panel Data", Journal of Econometrics , Vol. 18, 1982, 5-46. ____ (1986), "Panel Data" in Z. Griliches and M. Intriligator, eds., Handbook of Econometrics , North Holland. ____, "Heterogeneity, Omitted Variable Bias and Duration Dependence" in J. Heckman and B. Singer, Longitudinal Analysis of Labor Market Data , Cambridge Press. Elbers, C. and G. Ridder (1982), "True vs. Spurious Duration Dependence", Review of Economic Studies , Vol. 49, 403-410. Flinn C. and J. Heckman (1982), "Models For the Analysis of Labor Force Dynamics," in R. Bassman and G. Rhodes, Advances in Econometrics , Vol. 1, (JAI Press) 65-69. ____(1983), "The Likelihood Function," in Advances in Econometrics , 2, 225-231, ed. By R. Bassman and G. Rhodes, (JAI Press). Hansen, K., J. Heckman and K. Mullen (2001), "Nonparametric Identification of Single Spelled Duration Models with Time-Varying Regressors", working paper, Department of Economics, University of Chicago. Heckman, J. (1981), "Statistical Models For Discrete Panel Data", in C. Manski and D. McFadden, eds., Structural Analysis of Discrete Data With Econometrics Applications , 114-178, MIT, Cambridge Press. ____, "The Incidental Parameters Problem and the Problem of Initial Conditions" in C. Manksi and D. McFadden, op.cit. Heckman, J. and B. Honoré (1989), "The Identifiability of The Competing Risks Model", Biometrika , July. Heckman, J., R. Robb and J. Walker (1990), "Testing the Mixture of Exponential Hypothesis and Estimating the Mixing Distribution By The Method of Moments", Journal of The American Statistical Association , June. ____ (1990), "Estimating Fecundability Using Data on Waiting Times To Conception", Journal of the American Statistical Association , June. Heckman, J. and B. Singer (1985), "Social Science Duration Analysis" in J. Heckman and B. Singers, eds., Longitudinal Analysis of Labor Market Data , Cambridge University Press. Heckman, J. and J. Walker (1990), "The Impact of Wages and Income on The Timing and Spacing of Births in Sweden", Econometrica , November. Hsiao, C. (1986), Panel Data, Chapters 1, 2, 3 , Cambridge: Cambridge University Press. Judge, G. W. Griffiths, R. Hill and T. Lee (1980), The Theory and Practice of Econometrics , (Wiley), Chapter 13, "Unobserved Variables". Kalbfleisch, J. and R. Prentice (1980), The Statistical Analysis of Failure Time Data , Wiley. Lancaster, T. (1990), The Analysis of Transition Data, Cambridge University Press . Lancaster, T. (1991), The Econometric Analysis of Transition Data , Chapters 1-4, Cambridge: Cambridge University Press.
B. Structural Methods Blundell, R., A. Duncan and C. Meghir (1998), "Estimating Labor Supply Responses Using Tax Policy Reforms," Econometrica , 66, 827-801. Cameron, S. and J. Heckman (1998), "Life Cycle Schooling and Dynamic Selection Bias: Models and Evidence for Five Cohorts of American Males," Journal of Political Economy , 106(2), 262-311. _____ (1999), "Dynamic Models of Schooling Attainment for Blacks, Whites and Hispanics," 1992, revised 1998, Journal of Political Economy , June, 2001. Dagsvik, J. (1983), "Discrete Dynamic Choice: An Extension of the Choice Models of Thurstone and Luce", Journal of Mathematical Psychology , Vol. 27, 1-43. Eckstein, Z. and K. Wolpin (1989), "The Specification and Estimation of Dynamic Discrete Choice Models," Journal of Human Resources , Vol. 24, Fall, 562-598. _____ (1999), "Why Youths Drop out of High School: The Impact of Preferences, Opportunities, and Abilities," Econometrica , 67(6): 1295-1339. Flinn, C. (1986), "Wages and Job Mobility of Young Workers", Journal of Political Economy , June. Flinn, C. and J. Heckman (1982), "New Methods For Analyzing Structural Models of Labor Force Dynamics", Journal of Econometrics , January, 18, 115-168. Heckman, J. (1981), "Statistical Models for Discrete Panel Data," in C. Manski and D. McFadden eds., Structural Analysis of Discrete Data With Econometric Applications , (M.I.T. Press), Chapter 3 and 4. Heckman, J. and T. MaCurdy (1980), "A Life Cycle Model of Female Labor Supply", Review of Economic Studies , January. Keane, M. and K. Wolpin (1997), "The Career Decisions of Young Men", Journal of Political Economy , 105(3), 473-522. ____ (2001), The Effect of Parental Transfers and Borrowing Constraints on Educational Attainment", International Economic Review , 42(4): 1051-1103. Miller, R. (1984), "Matching and Occupational Choice", Journal of Political Economy , December. Rust, J. (1987), "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher", Econometrica , Vol. 55, 999-1033. ____ (1994), "Estimation of Dynamic Structural Models: Problems and Prospects", in C. Sims, ed., Advances in Econometrics , Cambridge: Cambridge University Press. C. Application: Earnings MaCurdy, T. (1982), "The Use of Time Series Processes To Model The Error Structure of Earnings in A Longitudinal Data Analysis", Journal of Econometrics , Vol. 18, 83-114. ____ (2001), "A Practitioner's Approach for Modeling Wage Dynamics: Micro-Longitudinal and Pooled Cross-Section/Time Series Analyses", forthcoming in J. Heckman and E. Leamer, Handbook of Econometrics , Vol. 6, Amsterdam: North Holland.
D: Application: Labor Supply Blundell, R., T. MaCurdy and C. Meghir (2001), op.cit. forthcoming in Handbook of Econometrics , Vol. 6, Amsterdam: North Holland. Heckman, J. (1978), "Simple Statistical Models For Discrete Panel Data", Annals De Insee' , Vol. 30-31, 227-270. ____ (1981), "Heterogeneity and State Dependence", in S. Rosen, ed., Studies in Labor Markets , University of Chicago Press. MaCurdy, T. (1981), "An Empirical Model of Labor Supply in A Life Cycle Setting", Journal of Political Economy , December, 1059-1085. ____ (1985), "Interpreting Empirical Models of Labor Supply in An Intertemporal Framework with Uncertainty", in J. Heckman and B. Singer, ed., Longitudinal Analysis of Labor Market Data , Cambridge. 5. Models For Families Bound, J., and G. Solon (1999), "Double Trouble," Economics of Education Review , Fall Issue. Griliches, Z. (1979), "Sibling Models and Data in Economics: Beginning of A Survey," Journal of Political Economy . Solon, G. (1999), "Survey of Sibling Models," in O. Ashenfelter and D. Card, Handbook of Labor Economics , Vol. IV. (North Holland).
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||