Smartification of the Design and Validation of an Administrative Automation Evaluation Model in Government Organizations: An Exploratory Mixed-Methods Approach

Abstract

This study designed and validated a multidimensional model for evaluating administrative automation in government organizations. An exploratory mixed-methods design was used. In the qualitative phase, content analysis of documents and expert interviews was used to identify evaluation indicators; the resulting pool was refined through a two-round Delphi procedure with 20 experts. In the quantitative phase, a 41-item questionnaire was administered to 127 public-sector managers and experts familiar with administrative automation systems. Partial least squares structural equation modeling (PLS-SEM) was used to assess the measurement and structural models, and an adaptive neuro-fuzzy inference system (ANFIS) was trained to capture nonlinear prediction patterns. The qualitative phase yielded 100 operational indicators, 30 components, and 10 final dimensions. Delphi consensus exceeded 80% for all dimensions, with the highest consensus for process performance and efficiency (93%) and the lowest for sustainability and environmental impacts (80%). PLS-SEM showed statistically significant effects for all ten predictors of the automation evaluation construct, although negative coefficients for selected constructs require cautious methodological interpretation. Model fit was acceptable (SRMR = 0.041; NFI = 0.91). The ANFIS test results indicated moderate-to-good predictive performance (RMSE = 0.464; R² = .626), with strategic alignment and public value, technology maturity and scalability, user experience and human resources, and financial outcomes emerging as the most influential inputs. The findings indicate that successful administrative automation depends on the simultaneous alignment of strategy, technology maturity, process redesign, human capacity, financial value, and governance safeguards.