Evaluasi Kinerja Regresi Lokal pada Pemodelan Hubungan Nonlinier: Studi Simulasi Monte Carlo

Linda Rassiyanti; Rohimatul Anwar; Sella Nofriska

doi:10.61132/arjuna.v4i5.3185

Authors

Linda Rassiyanti Institut Teknologi Sumatera
Rohimatul Anwar Universitas Lampung
Sella Nofriska Institut Agama Islam Negeri Sorong

DOI:

https://doi.org/10.61132/arjuna.v4i5.3185

Keywords:

Bias–Variance, Local Regression, LOESS, Monte Carlo Simulation, Nonlinear Relationships

Abstract

LOESS (Locally Estimated Scatterplot Smoothing) is a nonparametric approach for modeling nonlinear relationships without explicitly specifying a functional form. Its performance depends on the span, sample size, and error variability. This study evaluates local regression against classical linear regression and cubic polynomial regression across three types of nonlinear relationships (sinusoidal, local spike, and Runge function) through a Monte Carlo simulation with 300 replications per scenario. The factors varied include sample size (n = 30, 50, 100, 200), sigma (0.15 and 0.40), and LOESS span (0.3, 0.5, and 0.7). Model performance is evaluated using the Mean Squared Error (MSE), along with its decomposition into squared bias and variance. The results show that a narrow span (0.3) consistently yields the lowest MSE for functions with sharp curvature and local spikes, with an average MSE reduction of up to 91.7% compared with linear regression for the Runge function. For relatively smooth functions, cubic polynomial regression slightly outperforms LOESS. Larger sample sizes reduce the MSE of nonparametric methods by decreasing variance, whereas excessively wide spans increase bias due to oversmoothing. These findings highlight the importance of adaptive span selection and demonstrate the advantages of local regression in modeling complex and heterogeneous nonlinear relationships.

Downloads

Download data is not yet available.

References

Araveeporn, A. (2019). An estimating parameter of nonparametric regression model based on smoothing techniques. Statistical Journal of the IAOS, 35(2), 269–276. https://doi.org/10.3233/SJI-180477

Barreiro-Ures, D., Cao, R., Francisco-Fernández, M., & Fernández-Casal, R. (2026). Bagging cross-validated bandwidth selection in nonparametric regression estimation with applications to large-sized samples. Computational Statistics & Data Analysis, 213, Article 108257. https://doi.org/10.1016/j.csda.2025.108257

Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019). Reconciling modern machine-learning practice and the classical bias-variance trade-off. Proceedings of the National Academy of Sciences, 116(32), 15849–15854. https://doi.org/10.1073/pnas.1903070116

Boulesteix, A.-L., Groenwold, R. H. H., Abrahamowicz, M., Binder, H., Briel, M., Hornung, R., Morris, T. P., Rahnenführer, J., & Sauerbrei, W. (2020). Introduction to statistical simulations in health research. BMJ Open, 10(12), Article e039921. https://doi.org/10.1136/bmjopen-2020-039921

Calonico, S., Cattaneo, M. D., & Farrell, M. H. (2018). On the effect of bias estimation on coverage accuracy in nonparametric inference. Journal of the American Statistical Association, 113(522), 767–779. https://doi.org/10.1080/01621459.2017.1285776

Cattaneo, M. D., & Titiunik, R. (2022). Regression discontinuity designs. Annual Review of Economics, 14, 821–851. https://doi.org/10.1146/annurev-economics-051520-021409

Cattaneo, M. D., Jansson, M., & Ma, X. (2024). Local regression distribution estimators. Journal of Econometrics, 240(2). https://doi.org/10.1016/j.jeconom.2021.01.006

Efron, B., & Hastie, T. (2016). Computer age statistical inference: Algorithms, evidence, and data science. Cambridge University Press. https://doi.org/10.1017/CBO9781316576533

Eilers, P. H. C., & Marx, B. D. (2021). Practical smoothing: The joys of P-splines. Cambridge University Press. https://doi.org/10.1017/9781108610247

Fox, J., & Weisberg, S. (2019). An R companion to applied regression (3rd ed.). SAGE Publications.

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). An introduction to statistical learning: With applications in R (2nd ed.). Springer. https://doi.org/10.32614/CRAN.package.ISLR2

Kharoua, A., Mezhoud, K. A., & Mohdeb, Z. (2024). On asymptotic properties of local linear regression predictor. SN Operations Research Forum, 5(4). https://doi.org/10.1007/s43069-024-00384-y

Montgomery, D. C., Peck, E. A., & Vining, G. G. (2021). Introduction to linear regression analysis (6th ed.). Wiley.

Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074–2102. https://doi.org/10.1002/sim.8086

Wood, S. N. (2017). Generalized additive models: An introduction with R (2nd ed.). Chapman and Hall/CRC. https://doi.org/10.1201/9781315370279