Categorical data analysis is common in such disciplines as landscape ecology and environmental history. A motivating example is a study conducted to assess the influence of social, economic, and historical factors on forest covers. In this thesis, I consider the statistical analysis of this type of problem in a spatial binary setting and a multinomial regression setting, and develop new methodology and theory for this purpose. Autologistic regression models are proposed for relating spatial binary responses to spatial ownership characteristics. A penalized estimation method is developed under pseudolikelihood and an approximation is derived for assessing the variation of pseudolikelihood estimates. A simulation study is conducted to evaluate the performance of the proposed method and algorithm, followed by a data example. Under the multinomial regression setting, I propose a group Lasso type of regularization method for multinomial regression models that can shrink some or all of the regression coefficients to zero simultaneously. Since the existing theorems cannot be directly applied to group Lasso for multinomial regression models, we establish a framework for selection consistency under suitable regularity conditions. Further, we devise an algorithm to compute the group Lasso estimates. A simulation study shows that our method outperforms the traditional Lasso in terms of sum of the squared bias, Kullback-Leibler divergence, specificity, and correct model selection frequency. For illustration, our method is applied the data example.