Identification of High-Risk Patterns in Automobile Insurance Claims Using Deep Representation Learning and Evolutionary Clustering
Abstract
Detecting fraud in automobile insurance claims is challenging because reliable labels are scarce, the classes are highly imbalanced, and fraudulent behavior is often concealed. Supervised methods depend on accurately labeled training data, whereas unsupervised methods can identify unusual patterns but may offer limited support for prioritizing high-risk claims. This study proposes a hybrid framework that combines deep representation learning and evolutionary clustering to identify and prioritize potentially high-risk automobile insurance claims. The framework was evaluated using a real-world dataset of 179,514 automobile insurance claims. A deep autoencoder learned compact latent representations, and a genetic algorithm optimized initial cluster centroids in the latent space. K-means was then used for final cluster assignment. The main evaluation configuration used a 16-dimensional latent space, 40 training epochs, the Adam optimizer, and five clusters. A composite risk score combined reconstruction error, distance from the assigned cluster centroid, and insurance-specific profile indicators. Evaluation used a limited set of verified labels and synthetically injected fraudulent cases, with Precision@k, Recall@k, and the area under the precision–recall curve (PR-AUC) as the main performance measures. The proposed method produced more compact and better-separated clusters than the baseline methods. The silhouette score increased from 0.124 for K-means in the raw feature space to 0.3953 for evolutionary clustering in the autoencoder latent space. Precision@100 reached 0.68, compared with 0.18 for the K-means baseline, and PR-AUC increased from 0.142 to 0.562. The model identified groups with unusual financial and temporal characteristics, supporting the prioritization of suspicious claims for further investigation. Combining deep representation learning with evolutionary clustering offers a practical decision-support approach for ranking automobile insurance claims and directing audit resources toward potentially high-risk cases. The limited availability of verified fraud labels and the use of synthetic evaluation cases constrain conclusions about real-world fraud detection performance. The framework should therefore support expert review rather than determine whether a claim is fraudulent.
Introduction
Insurance fraud is a major challenge for the insurance industry, imposing substantial costs on insurers and, ultimately, policyholders. Increasingly complex fraud patterns, diverse forms of abuse, and large volumes of claims data make the identification of unusual behavior a difficult, multidimensional problem (Seidi Aghil Abadi et al., 2017). Many claims lack definitive fraud labels, while human assessments can be affected by uncertainty and bias. Data-driven claims analysis can therefore support expert decision-making and improve audit procedures (Schrijver et al., 2024). Machine learning has attracted considerable attention in insurance fraud analysis. Much of the existing research, however, relies on supervised methods that require labeled data. In practice, generating accurate and reliable labels is difficult because labeling is expensive, fraud patterns change over time, and legitimate claims greatly outnumber fraudulent ones (Schrijver et al., 2024). These constraints can limit model performance and generalizability in operational settings (Esna-Ashari et al., 2022). Unsupervised methods, particularly clustering, can reveal hidden structures and behavioral patterns in large claims datasets without predefined labels. Grouping similar records may help identify subgroups with unusual or high-risk characteristics and provide an analytical complement to expert judgment. Classical clustering methods such as K-means nevertheless remain sensitive to centroid initialization, cannot directly capture complex nonlinear relationships, and may converge to local minima (Aytekin et al., 2018; Krishna & Murty, 1999). This study presents a data-driven hybrid framework for identifying suspicious patterns in automobile insurance claims by combining deep representation learning, evolutionary optimization, and clustering. A deep autoencoder first reduces dimensionality and captures nonlinear structure in the claims data. The framework has four principal components. First, clustering is transferred from the raw feature space to a compact autoencoder latent space, where a genetic algorithm (GA) optimizes the initial centroids. This combination is intended to reduce noise and represent complex dependencies among claim attributes more effectively than conventional clustering. Second, a synthetic fraud injection strategy addresses the scarcity of verified labels. Artificial cases are generated from known insurance fraud patterns to provide a weakly labeled basis for evaluating anomaly identification. As described in the methods, these cases are intended for evaluation rather than for supervised training of the core representation model. Third, a composite risk score combines reconstruction error, distance from the assigned cluster centroid, and insurance-specific profile information. This allows claims to be ranked by relative risk rather than assigned a definitive binary fraud label. Limited labels are used in validation and in optimizing the risk-scoring weights. Fourth, evaluation extends beyond internal clustering indices to include PR-AUC, Precision@k, and Recall@k, alongside analyses of cluster structure. The framework was applied to 179,514 automobile insurance claims. The main reported implementation used a 16-dimensional latent space, 40 epochs of autoencoder training with Adam, and K-means with k = 5. The results suggest that the framework can distinguish potentially suspicious behavioral patterns and support the prioritization of claims for detailed audit.
Theoretical Background
Approaches to fraud detection
Fraud management encompasses prevention, detection, investigation, recovery, assessment, and continuous monitoring. Figure 1 summarizes these activities and the principal learning approaches used for detection.
Supervised learning
Supervised models learn from labeled examples in which fraudulent and nonfraudulent cases are identified in advance. Common approaches include decision trees, support vector machines, neural networks, random forests, and ensemble methods. Their applicability depends on the quality and representativeness of the available labels (Schrijver et al., 2024).
Their main limitation in fraud detection is their dependence on valid labels. In many insurance applications, labeled data are scarce, imbalanced, and potentially subject to labeling error. Changes in fraudulent behavior also mean that models trained on historical data may fail to detect emerging fraud patterns.
Unsupervised learning
Unsupervised methods seek structure in data without outcome labels. In anomaly detection, statistical, distance-based, and representation-learning methods identify observations that deviate from dominant patterns (Ruff et al., 2021).
Their principal advantage is that they do not require labeled data, making them relevant to label-scarce settings such as health insurance. However, the boundary between legitimate and fraudulent behavior is often difficult to establish, and results can depend strongly on similarity measures and model parameters (Vosseler, 2022).
Semi supervised learning
Semi-supervised methods combine a small labeled dataset with a larger unlabeled dataset. Information in the unlabeled observations is used to improve learning and predictive performance. Such methods are particularly relevant when labeling is costly or time-consuming.
Semi-supervised approaches can make use of limited labeled information, but their suitability depends on how labels enter model development. In the present framework, representation learning and clustering are unsupervised, whereas labels are used for risk-score calibration and evaluation. These roles should be distinguished from label-supervised training of the encoder.
Anomaly detection
Anomaly detection identifies observations that deviate from a reference pattern. Reconstruction-based models use discrepancies between observations and their reconstructions, whereas methods such as Isolation Forest identify anomalies through different mechanisms. Unusual observations can be candidates for investigation, but an anomaly score does not establish fraud (Ruff et al., 2021).
These approaches can accommodate unlabeled data. Selecting thresholds and interpreting alerts remain important practical considerations. Comparative evaluations also show that performance depends on the dataset and detector, supporting the use of relevant baselines rather than assuming that one method is universally superior (Goldstein & Uchida, 2016).
Clustering in data analysis
Clustering groups observations according to similarity without predefined outcome labels. Deep clustering combines this task with learned representations intended to expose useful structure in complex data (Min et al., 2018). In the present study, clusters describe groups of claims; fraud status requires separate evidence.
For the present insurance application, clustering is treated as an exploratory component of risk analysis. The resulting groups guide further examination rather than determine whether individual claims are fraudulent. Unsupervised insurance fraud analysis similarly requires careful interpretation of anomaly indicators (Vosseler, 2022).
Main clustering approaches
Clustering approaches include centroid-based, hierarchical, density-based, and model-based methods. In deep clustering, the representation and clustering procedure are interdependent design choices (Min et al., 2018). Here, K-means is used to assign latent observations to a specified number of groups.
K means and its limitations
K-means begins with a specified number of clusters and initial centroids. Each observation is assigned to its nearest centroid, usually according to Euclidean distance, and the centroids are updated iteratively until convergence. Its objective is to minimize the within-cluster sum of squared distances (Aytekin et al., 2018).
K-means depends on its initialization, the number of clusters, and the geometry of the feature representation. Learning a suitable representation and improving initialization are therefore distinct strategies that can be investigated separately (Aytekin et al., 2018; Krishna & Murty, 1999).
Clustering for insurance fraud and risk analysis
In this study, clustering identifies groups with similar behavioral and financial characteristics. Cluster membership is not interpreted as a verified fraud label. The groups are used to support subsequent examination of potentially unusual claims (Nian et al., 2016; Vosseler, 2022).
When reliable labels are limited, unsupervised scores can help rank claims for further investigation. Their operational value depends on how well the ranking concentrates independently verified cases of interest, rather than on cluster separation alone (Nian et al., 2016).
Evolutionary optimization of clustering
Genetic algorithms have been combined with K-means to improve the search for clustering solutions. Krishna and Murty (1999) proposed a hybrid genetic K-means algorithm with clustering-specific search operators, providing an established precedent for evolutionary clustering.
By emulating evolutionary processes, a GA can explore a broad search space and identify solutions that improve on local search procedures. Studies have used GAs to optimize cluster centroids or assignments and have reported improvements over classical K-means (Yousefimehr et al., 2025).
Representation learning and evolutionary optimization address different aspects of the analysis. Autoencoders provide compact representations, while a genetic algorithm searches over candidate solutions. Existing work has combined these ideas for outlier analysis, motivating a precise account of the contribution of the present insurance application (Li et al., 2022).
Genetic algorithms for clustering optimization
A genetic algorithm is an evolutionary optimization method based on selection, mutation, and recombination. Its ability to search complex nonlinear spaces has led to its use in difficult optimization problems. Each candidate solution is encoded as a chromosome and evaluated using a fitness function. Derivatives and linearity assumptions are not required.
Evolutionary clustering predates the present study. Krishna and Murty (1999) developed genetic K-means by combining genetic search with a K-means operator. The present framework instead searches for initial centroids in an autoencoder representation before final K-means assignment.
Evolutionary clustering in fraud detection
Unsupervised approaches have been investigated for insurance fraud detection, including deep-learning methods (Gomes et al., 2021). In the present application, small or statistically distinctive groups are treated as candidates for review. Their unusual characteristics do not constitute independent confirmation of fraudulent behavior.
Genetic algorithms in insurance fraud analysis
Yousefimehr et al. (2025) investigated automobile insurance fraud detection using clustering improved by a genetic algorithm. This work provides an insurance-specific comparator for the present approach. Claims about differences in performance or implementation require a comparison under clearly specified evaluation conditions.
Operational fraud detection also faces challenges beyond cluster construction. Work on adversarial learning discusses the difficulties of developing fraud detection methods under real-world conditions (Lunghi et al., 2023). Such considerations motivate evaluating the framework on independently verified cases as well as synthetic anomalies.
Limitations of existing approaches
High-dimensional data can make abnormal patterns difficult to identify and interpret. Li et al. (2022) combined a variational autoencoder with a genetic algorithm for outlier detection and interpretation. Their approach is relevant prior work, although its optimization target differs from centroid initialization in the present study.
Another limitation is the interpretation of evolutionary clustering as direct fraud detection. In an unsupervised setting, this interpretation can overstate what the results establish. A more defensible framework should perform clustering in an informative representation space and define its output as analytical support for expert review.
The present study examines the combination of latent representations, evolutionary centroid initialization, and claim-level risk scoring. The intended output is a ranking for investigation. Its usefulness must be evaluated independently of internal clustering validity (Vosseler, 2022).
Latent representation learning with autoencoders
Representation learning aims to extract compact and informative representations from raw data (Ruff et al., 2021). Operational insurance datasets often contain many attributes, complex correlations, noise, and nonlinear distributions. These characteristics can adversely affect classical methods, including clustering.
A latent representation is a lower-dimensional space intended to preserve essential structure while reducing redundancy and noise. Learning such a representation can improve subsequent analysis (Ruff et al., 2021).
Autoencoders
An autoencoder is an unsupervised representation-learning model that uses a neural network to learn a compact mapping of its inputs (Audibert et al., 2020). It has two main components: an encoder, which maps the input to a lower-dimensional latent space, and a decoder, which reconstructs the original input from that representation.
Training minimizes the discrepancy between the input and its reconstruction. Constraints such as a low-dimensional bottleneck or regularization encourage the network to retain the most informative patterns and structures (Pang et al., 2022).
Nonlinear dimensionality reduction
Unlike linear dimensionality-reduction methods such as principal component analysis, autoencoders can model complex nonlinear relationships. This makes them potentially useful for claims data with heterogeneous and nonlinear distributions (Audibert et al., 2020).
Latent representations learned by autoencoders may improve downstream tasks such as clustering, customer segmentation, and behavioral analysis without requiring supervisory labels (Min et al., 2018).
Unsupervised analysis of unusual behavior
Autoencoders are often trained to represent the dominant patterns in unlabeled data. Observations that are less consistent with those patterns may have larger reconstruction errors. In insurance applications, reconstruction error is best interpreted as an analytical indicator rather than a definitive fraud decision (Li et al., 2022).
Accordingly, reconstruction error is used as one indicator of deviation from the learned representation, rather than as a calibrated probability or definitive label of fraud (Ruff et al., 2021).
Combining autoencoders and clustering
Autoencoder representations can be supplied to clustering algorithms. Aytekin et al. (2018) showed that an explicit normalization constraint on learned representations can improve their compactness and separation for K-means. Such findings motivate representation learning, but do not guarantee that the present architecture will achieve the same benefit without direct evaluation (Min et al., 2018).
Why a hybrid framework is needed
An autoencoder alone does not guarantee an optimal clustering structure. The quality of the latent representation depends on the network architecture, loss function, and training procedure. Moreover, reconstruction training does not directly optimize cluster centroids.
Autoencoder-based outlier analysis has already been combined with genetic optimization (Li et al., 2022). In the present framework, the specific role of the GA is to optimize candidate centroids in the learned representation; the autoencoder itself is trained using reconstruction loss.
Related Research
Insurance fraud detection has been studied using statistical, supervised, unsupervised, and other data-driven approaches. A systematic review of automobile insurance fraud detection describes the methods and datasets used in this literature and identifies directions for further investigation (Schrijver et al., 2024).
Supervised and unsupervised approaches differ in their dependence on outcome labels. For automobile insurance applications, label availability, dataset characteristics, and the evaluation design influence which methods can be assessed credibly (Schrijver et al., 2024).
Unsupervised insurance fraud analysis seeks patterns that warrant further investigation. Observations that receive high anomaly scores should not be treated as confirmed fraud solely because they differ from common behavior (Nian et al., 2016; Vosseler, 2022).
Min et al. (2018) reviewed the relationship between representation learning and clustering. In the present framework, latent representation learning and centroid optimization constitute distinct components whose contributions warrant separate evaluation.
Krishna and Murty (1999) established an early hybrid genetic K-means approach. Building on evolutionary clustering, the present study examines centroid initialization in an autoencoder representation and its application to insurance claim prioritization.
Insurance-specific work has investigated genetic optimization of clustering (Yousefimehr et al., 2025). In the present framework, cluster and risk-score outputs are intended to assist review, with case-level verification remaining separate from algorithmic ranking.
Yousefimehr et al. (2025) provide relevant prior work on genetic-algorithm-enhanced clustering for automobile insurance fraud analysis. A direct comparison with that method would require matching the data, labels, and evaluation protocol; findings from different datasets should not be treated as directly comparable.
Deep clustering research examines how learned representations can facilitate grouping observations (Min et al., 2018; Alturki et al., 2021). Autoencoder–GA approaches to outlier analysis also exist (Li et al., 2022). These studies establish relevant methodological precedents for the proposed framework.
The present study evaluates a particular integration of deep representation learning, evolutionary centroid initialization, and domain-informed claim prioritization. Its contribution is therefore framed as an application and evaluation of this integration in automobile insurance claims, rather than the first combination of autoencoders and genetic algorithms. The principal questions concern the added value of each component and the validity of the resulting risk ranking.
The framework combines unsupervised representation learning and clustering with label-informed risk-score calibration and evaluation. Its purpose is to prioritize potentially high-risk claims for investigation, rather than establish fraud from cluster membership alone.
Proposed Method
Overview
The framework first learns the dominant structure of the data using an autoencoder. Poor reconstruction may indicate unusual behavior and provides one signal for subsequent risk analysis. Clustering is performed in the learned latent space rather than directly in the raw feature space.
A GA optimizes the initial centroids to reduce sensitivity to random initialization and mitigate convergence to poor local solutions. Each chromosome represents a candidate set of centroids. The best chromosome is used to initialize the final K-means procedure.
After clustering, each claim receives a risk score based on reconstruction error, distance from its assigned centroid, and insurance-specific characteristics such as claim amount, claim frequency, and reporting delay. A GA is also used to optimize the weights of the risk components. Small or otherwise unusual clusters with high mean risk scores are selected for further investigation. Limited verified labels and synthetically injected cases provide the basis for evaluation. Figure 2 summarizes the five stages.
Stage 1 Data cleaning and preparation
The study used a real-world dataset of 179,514 automobile insurance claims. Each record represented a separate claim and contained numerical, temporal, and categorical information about the policy, policyholder, and accident. Variable types and missingness were examined before modeling.
Unique identifiers can encourage memorization without providing generalizable analytical information. The claim, policy, customer, and police-report identifiers were therefore excluded. Categorical attributes, including claim type, accident province, and policy class, were converted to numerical representations. One-hot encoding was used for categorical variables, and relative-frequency encoding was applied to high-cardinality variables to limit dimensional expansion.
Jalali dates for the accident, claim submission, and claim-file issuance were converted to numerical ordinal values. Reporting delay and processing delay were then calculated, and the raw date fields were removed. These interval variables were intended to capture temporal patterns in claims behavior.
Missing numerical values were imputed using the median rather than removing records, helping preserve rare events and limiting sensitivity to extreme observations. Numerical features were transformed using robust scaling based on distributional quantiles to reduce the influence of large monetary values and outliers.
The dataset was divided chronologically into training and test subsets according to accident date. More recent claims were reserved for testing, allowing evaluation on later observations and reducing temporal information leakage.
Because verified fraud labels were scarce, synthetic cases were generated by modifying a proportion of otherwise normal claims using known fraud-related patterns. These included unusually large claim amounts, sharply reduced reporting delays, and repeated claims over short periods. These modifications were applied to a proportion of normal claims in the training subset. The resulting synthetic cases were used only for evaluation and did not enter the unsupervised training of the core model. The synthetic labels provided weak supervision for evaluation.
Stage 2 Latent representation learning
A symmetric deep autoencoder was trained without fraud labels to capture nonlinear structure and reduce dimensionality. Its encoder compressed the input into a bottleneck representation, and its decoder reconstructed the input from that representation. The network used fully connected layers and rectified linear unit (ReLU) activations.
Training minimized mean squared reconstruction error. The objective was to represent common patterns accurately; unusual claims could then exhibit larger reconstruction errors. This behavior was used as an anomaly signal, without assuming that every poorly reconstructed claim was fraudulent.
Stage 3 Evolutionary clustering
The learned latent vectors were clustered to distinguish prevalent patterns from rare or potentially high-risk behavior. The working assumption was that common behavior would tend to form compact groups, whereas unusual behavior might occur in smaller, more dispersed groups or farther from cluster centroids.
Rather than initialize K-means randomly, a GA searched for suitable initial centroids. Each chromosome encoded a complete set of centroids in the latent space. Candidate solutions were assessed using the silhouette score, an internal measure that favors within-cluster compactness and between-cluster separation without requiring fraud labels.
The evolutionary procedure consisted of random initialization from latent observations, tournament selection, single-point crossover, and Gaussian-noise mutation. Mutation maintained population diversity and helped reduce premature convergence. After the final generation, the best chromosome supplied the initial centroids for K-means.
The resulting behavioral clusters were subsequently interpreted together with reconstruction errors and insurance-specific attributes to calculate risk scores and support label-informed evaluation.
Stage 4 Evaluation
Evaluation extended beyond internal clustering validity. Injected fraud patterns served as weak labels, and the limited verified labels provided additional information for assessing the prioritization of suspicious claims.
PR-AUC was used because fraud detection involves severe class imbalance. Precision@k and Recall@k measured the concentration and coverage of labeled positive cases among the k highest-ranked claims. These measures reflect the practical limitation that insurers can investigate only a restricted number of cases.
Cluster structure was assessed using the silhouette score and Davies–Bouldin index. The former measures compactness and separation; the latter evaluates the relative similarity of clusters, with lower values indicating better separation. The results also report the Calinski–Harabasz index.
Reconstruction error was defined as ||x − x̂||². Its distribution was compared between normal and synthetically modified cases to examine whether the autoencoder reconstructed unusual observations less accurately.
For interpretability, each cluster was profiled using key insurance variables, including claim amount, claim frequency, and reporting delay. These profiles helped relate high-risk clusters to domain-relevant patterns, such as unusual claims or reporting behavior.
Stage 5 Risk scoring and high risk cluster identification
A composite risk score converted the model outputs into an operational ranking of claims. The score combined structural information from clustering with the behavioral deviation represented by reconstruction error:
Rᵢ = α RecErrᵢ + β Distᵢ + γ ProfileScoreᵢ
Here, RecErrᵢ denotes the reconstruction error for claim i; Distᵢ is its distance from the centroid of the assigned cluster; and ProfileScoreᵢ is a weighted insurance-domain indicator incorporating factors such as an unusual claim amount, repeated claims over a short interval, and reporting delay. The coefficients α, β, and γ were optimized by a GA to maximize PR-AUC on the injected cases.
After claim-level scores were calculated, clusters were ranked by their mean risk score. Clusters with relatively low density and high mean risk were identified as high-risk groups. The score represents relative priority for review; it is not established as a calibrated probability of fraud.
Incorporating domain information was intended to reduce false alerts for large but legitimate claims. Each referral could also include the main reasons for its elevated score, such as high reconstruction error or unusual reporting delay, making the output easier for claims specialists to interpret.
Model outputs
The framework produced several complementary outputs. Preprocessing and temporal feature extraction supplied the input to the autoencoder, which generated a compact representation of claims behavior. Evolutionary clustering then grouped the latent vectors into behavioral clusters, whose structure was assessed using internal validity indices.
Cluster size, centroid distance, and reconstruction error were combined to identify rare or unusual patterns. Smaller groups with higher mean reconstruction error were treated as candidates for closer review. Statistical profiles of key insurance attributes provided further interpretation of these groups. The following section presents the implementation, representative outputs, and quantitative comparisons.
Implementation and Results
Dataset characteristics
Table 1 describes selected claim attributes in the dataset. These include financial, temporal, behavioral, and geographical information. Financial attributes comprise claim amount, vehicle value, and premium; temporal attributes include reporting and processing delays; behavioral information includes previous claims and claim status; and geographical attributes include accident and driving-license locations.
Attribute | Data type | Description | Example |
|---|---|---|---|
prdid | Numerical | Claim record identifier | 435822557 |
PolicyId | Numerical | Policy number | 37017165 |
PrdDate | Jalali date | Claim-file issuance date | 1404/09/06 |
SubmitDate | Jalali date | Claim submission date | 1404/09/05 |
AccidentDate | Jalali date | Accident date | 1404/09/02 |
AccidentCityId | Numerical | Accident city code | 701 |
DrvLicenseCityId | Numerical | Driving-license city code | 701 |
DrvLicenseTypeId | Numerical | Driving-license type | 17 |
RecEstimateAmt | Numerical | Estimated claim amount | 0 |
PrdAmt | Numerical | Claim-file amount | 2,500,000 |
PolicyPrm | Numerical | Insurance premium | 3,0277 |
CustomerId | Numerical | Policyholder identifier | 701 |
CityId | Numerical | Policyholder city code | 53 |
age | Numerical | Policyholder age | 26 |
GenderId | Numerical | Gender | 1 = male; 2 = female |
VehicleVerId | Numerical | Vehicle type | 6576056 |
BuiltYear | Numerical | Vehicle manufacture year | 1396 |
PolicyVehicleValueAmt | Numerical | Vehicle value | 10,000,000,000 |
DiffDays | Numerical | Difference in days | 62 |
ClaimStateId | Numerical | Overall claim status | 2 |
ClaimSubstateId | Numerical | Claim substatus | 7 |
Note. Dates and vehicle manufacture years use the Jalali calendar. Field names and example values are shown as recorded in the dataset.
Categorical variables were converted into numerical inputs before neural-network training. Label encoding was used, together with one-hot encoding for variables with few categories.
Data cleaning
Nonanalytical identifiers—prdid, PolicyId, CustomerId, and PoliceReportSerial—were excluded to reduce the risk of memorization and improve generalizability.
Accident, submission, and processing-related dates were converted from the Jalali calendar to ordinal values. Two interval features were then derived:
Delay_Report = SubmitDate − AccidentDate
Delay_Process = PrdDate − SubmitDate
These variables capture reporting and administrative delays that may be relevant to risk analysis. The raw date columns were subsequently removed, during preprocessing.
Median imputation was used because financial variables were skewed and the median is less sensitive than the mean to extreme values. Robust scaling was applied to limit the influence of widely dispersed claim amounts and improve numerical stability during network training.
All 179,514 records were retained during preprocessing. Table 2 summarizes the changes in feature count at each stage.
Stage | Variables | Count | Description |
|---|---|---|---|
Raw data | All variables | 36 | Includes identifiers and raw dates |
Identifier removal | prdid, PolicyId, CustomerId, PoliceReportSerial | 32 | Four nonanalytical columns removed |
Date processing | PrdDate, SubmitDate, AccidentDate → Delay_Report, Delay_Process | 34 | Ordinal conversion and two interval features added |
Missing-value imputation | All variables | 34 | Median imputation |
Robust scaling | All numerical variables | 34 | Scaling to reduce the influence of outliers |
Autoencoder implementation
The unsupervised autoencoder consisted of an encoder and a decoder. The encoder progressively reduced dimensionality, while the decoder expanded the latent representation to reconstruct the inputs. Reconstruction loss was minimized using Adam. The encoder output was then used as the input to evolutionary clustering, and observation-level reconstruction errors were retained as anomaly indicators.
For a preprocessed dataset X = {xᵢ ∈ ℝᵈ | i = 1, …, N}, N = 179,514 and d denotes the number of numerical input features. The encoder maps each input to a lower-dimensional vector:
zᵢ = fθ(xᵢ), fθ: ℝᵈ → ℝᵖ
The reconstruction objective is:
L = (1/N) Σᵢ ||xᵢ − x̂ᵢ||²
The autoencoder architecture was input → 128 → 64 → 16 → 64 → 128 → output, with ReLU activations. After 30 epochs of training, 16-dimensional latent vectors were extracted for each claim, providing a compact representation of its behavioral and financial characteristics.
Table 3 presents an illustrative latent-coordinate output with 12 displayed coordinates. Table 4 shows illustrative observation-level reconstruction errors, and Table 5 summarizes reconstruction error across the analyzed observations.
Sample | Z1 | Z2 | Z3 | Z4 | Z5 | Z6 | Z7 | Z8 | Z9 | Z10 | Z11 | Z12 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 0.12 | -0.05 | 0.34 | 0.08 | -0.11 | 0.05 | -0.02 | 0.07 | 0.01 | -0.03 | 0.04 | 0.02 |
2 | -0.08 | 0.21 | 0.12 | -0.04 | 0.07 | -0.03 | 0.05 | 0.02 | -0.01 | 0.01 | -0.02 | 0.03 |
3 | 0.05 | -0.12 | 0.29 | 0.11 | -0.08 | 0.04 | 0.00 | -0.02 | 0.03 | -0.01 | 0.01 | -0.01 |
4 | 0.03 | -0.07 | 0.21 | 0.05 | -0.05 | 0.02 | 0.01 | 0.01 | -0.02 | 0.00 | 0.01 | 0.00 |
5 | -0.02 | 0.09 | 0.18 | -0.01 | 0.03 | -0.01 | 0.02 | 0.00 | 0.01 | -0.01 | 0.00 | 0.01 |
6 | 0.15 | -0.11 | 0.30 | 0.10 | -0.12 | 0.05 | -0.03 | 0.06 | 0.02 | -0.02 | 0.03 | 0.01 |
7 | 0.01 | 0.05 | 0.12 | 0.02 | -0.03 | 0.01 | 0.00 | 0.00 | 0.01 | 0.00 | -0.01 | 0.01 |
8 | -0.03 | 0.07 | 0.15 | -0.02 | 0.04 | -0.01 | 0.01 | 0.01 | -0.01 | 0.00 | 0.01 | 0.01 |
9 | 0.04 | -0.08 | 0.22 | 0.07 | -0.06 | 0.03 | 0.00 | 0.01 | 0.01 | -0.01 | 0.01 | 0.00 |
10 | -0.01 | 0.03 | 0.10 | -0.01 | 0.02 | 0.00 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
Sample | Reconstruction error |
|---|---|
1 | 0.0023 |
2 | 0.0031 |
3 | 0.0018 |
4 | 0.0020 |
5 | 0.0019 |
6 | 0.0028 |
7 | 0.0015 |
8 | 0.0021 |
9 | 0.0024 |
10 | 0.0017 |
Statistic | Value |
|---|---|
Number of observations | 179,380 |
Mean | 0.0045 |
Standard deviation | 0.0021 |
Minimum | 0.0012 |
Maximum | 0.0152 |
Chromosome representation and fitness
Each chromosome contained K cluster centroids in the latent space:
Chromosome = [c₁, c₂, …, cK]
For a latent dimension p, a flattened chromosome contains K × p genes. Initial chromosomes were generated by sampling candidate centroids from the latent observations. For each candidate solution, observations were assigned to their nearest centroid using Euclidean distance, and fitness was calculated as:
Fitness = SilhouetteScore(Z, cluster labels)
The fitness measure requires no fraud labels. Selection, crossover, and mutation generated successive populations until the stopping condition was reached. The chromosome with the highest silhouette score was selected for final K-means initialization.
In the first generation, the best fitness was 0.3013. This provided the starting point for evolutionary refinement. Figure 3 reproduces the generation-by-generation output, showing an increase to 0.3953 over 30 generations.
Selecting and reshaping the best chromosome
Each member of the population was a one-dimensional vector containing all centroid coordinates in sequence. To evaluate a chromosome, the vector was reshaped into K centroids, observations in Z_latent were assigned to them, and the silhouette score was calculated.
After all chromosomes had been evaluated, the candidate with the highest fitness was selected. The selected vector was reshaped into a K × latent_dim matrix: each row represented a centroid and each column a latent coordinate. This representation enabled the optimized centroids to be passed directly to K-means.
As shown in Figure 3, the best silhouette score increased progressively from 0.3013 to 0.3953 over 30 generations, indicating improvement in the clustering objective. The final chromosome supplied the centroids for the subsequent assignment stage. The trace demonstrates improvement over the evaluated generations, but does not by itself establish a globally optimal solution.
Final K means assignment
K-means assigned latent observations to clusters using the GA-derived centroids as its initial values. This differs from conventional random initialization and is intended to reduce sensitivity to starting conditions and the risk of poor local solutions.
Each claim received a cluster label representing a group with similar behavioral and financial characteristics. These labels formed the basis for cluster profiling, the identification of potentially high-risk groups, and the analysis of unusual claims. Figure 4 presents a sample of the latent vectors and their assigned cluster labels.
Table 6 presents the reshaped best-chromosome centroid output, with a reported best-fitness value of 0.6124.
Cluster | Z1 | Z2 | Z3 | Z4 | Z5 | Z6 | Z7 | Z8 |
|---|---|---|---|---|---|---|---|---|
1 | -0.182896 | 1.374876 | -0.645964 | -0.799192 | -0.482744 | -0.953329 | 0.122670 | 1.624678 |
2 | 0.298753 | -0.751791 | -0.426358 | 1.148446 | 0.113270 | -1.438278 | 0.919229 | -0.668144 |
3 | -0.643550 | 1.029161 | -0.334775 | -0.403648 | -0.955125 | 0.423599 | 2.062525 | -1.067533 |
4 | 1.187386 | 2.589564 | 0.579363 | 0.325796 | 0.194384 | -0.353166 | 0.338484 | -0.295401 |
5 | 1.296995 | -0.022868 | -0.999302 | -0.504775 | 0.840620 | 0.546734 | -0.238394 | -0.366824 |
Cluster | Z9 | Z10 | Z11 | Z12 | Z13 | Z14 | Z15 | Z16 |
|---|---|---|---|---|---|---|---|---|
1 | 0.323079 | -0.252354 | -0.291811 | -1.563191 | 0.883110 | -0.078737 | -0.180480 | 3.193108 |
2 | 1.873298 | 1.080048 | -0.447322 | 1.281016 | 0.067856 | 0.852774 | 0.484733 | -0.846357 |
3 | 0.024219 | 1.422211 | -0.079641 | 0.452372 | -0.662394 | 0.428307 | -0.187144 | 0.985730 |
4 | 0.168461 | 1.317598 | -1.006543 | 1.139879 | 1.317115 | -0.118069 | -2.121855 | -0.607822 |
5 | -0.391758 | -0.922241 | 1.615376 | -0.322320 | 1.217159 | 1.521316 | 0.998311 | -0.431620 |
Figure 5. Distribution of observations across five clusters
The division of responsibilities in the hybrid method is straightforward: the GA searches for suitable latent-space centroids, and K-means performs the final assignments. This combines evolutionary search with the computational efficiency of K-means while retaining interpretable cluster membership.
Evaluation configuration
The main evaluation configuration was five clusters, a latent dimension of 16, an autoencoder architecture of 36 → 32 → 16 → 32 → 36, 40 training epochs, and a batch size of 256. The dataset contained 179,514 records.
Internal clustering validity
Table 7 compares K-means in the raw feature space, autoencoder representations followed by K-means, and the proposed autoencoder–GA method.
Method | Silhouette ↑ | Davies–Bouldin ↓ | Calinski–Harabasz ↑ |
|---|---|---|---|
K-means in raw space | 0.124 | 2.451 | 1,240 |
Autoencoder + K-means | 0.312 | 1.120 | 4,580 |
Proposed method (AE + GA) | 0.3953 | 0.842 | 6,820 |
The best GA fitness increased from 0.3013 to 0.3953 over 30 generations, an improvement of approximately 31%. This indicates that evolutionary refinement improved the evaluated clustering objective. For this convergence result, fitness represented cluster separation in a 12-dimensional latent space.
The silhouette score increased from 0.124 for raw-space K-means to 0.312 after representation learning and to 0.3953 with GA optimization. The Davies–Bouldin index decreased from 2.451 to 0.842, while the Calinski–Harabasz index increased from 1,240 to 6,820. Together, these results indicate more compact and better-separated clusters under the reported evaluation.
Better structural separation can assist claims specialists in identifying groups for review. Internal validity indices alone, however, do not establish a boundary between fraudulent and legitimate claims.
Prioritization performance
The next comparison used the limited available labels and synthetically injected cases. Table 8 reports Precision@100, Recall@100, and PR-AUC for the three methods.
Method | Precision@100 | Recall@100 | PR-AUC |
|---|---|---|---|
K-means baseline | 0.18 | 0.05 | 0.142 |
Autoencoder + K-means | 0.42 | 0.28 | 0.385 |
Proposed method (AE + GA) | 0.68 | 0.45 | 0.562 |
The proposed method achieved a PR-AUC of 0.562, compared with 0.385 for autoencoder plus K-means and 0.142 for the raw-space K-means baseline. Precision@100 was 0.68, meaning that 68 of the 100 highest-ranked observations were positive according to the evaluation labels, compared with 18 for baseline K-means. This corresponds to an approximately 3.78-fold increase in Precision@100. Recall@100 increased from 0.05 to 0.45.
These results suggest that the framework concentrated more labeled positive cases near the top of the review list. Because the evaluation includes synthetic cases and only limited verified labels, the positive cases should not all be interpreted as independently confirmed real-world fraud. The findings support prioritization under the reported evaluation conditions.
Reconstruction error analysis
The overall mean reconstruction error was reported as 0.0045, with a standard deviation of 0.0021 and a maximum of 0.0152. The largest errors were mainly associated with unusual or suspicious observations. Table 9 compares reconstruction errors for normal and synthetically injected cases.
Sample type | Mean reconstruction error | Standard deviation |
|---|---|---|
Normal | 0.0038 | 0.0011 |
Synthetic fraud (injected) | 0.0124 | 0.0035 |
All observations | 0.0045 | 0.0021 |
The mean error for injected cases was 0.0124, compared with 0.0038 for normal cases, approximately a 3.26-fold difference. It was also about 2.76 times the reported overall mean. This pattern is consistent with the autoencoder reconstructing common behavior more accurately than the injected anomalies. The difference supports the use of reconstruction error as a risk indicator, although no inferential significance test is reported.
Behavioral profiles of clusters
Table 10 presents five clusters selected from ten clusters for detailed behavioral interpretation. The displayed profiles account for 112,570 claims.
Cluster | Claims, n | Mean claim amount (tomans) | Mean reporting delay (days) | Mean reconstruction error | Mean risk score | Suspicious claims (%) |
|---|---|---|---|---|---|---|
C1 | 24,150 | 12,000,000 | 2.1 | 0.0031 | 0.24 | 1.2 |
C2 | 18,420 | 45,500,000 | 14.5 | 0.0082 | 0.78 | 12.5 |
C3 | 35,600 | 8,200,000 | 1.5 | 0.0028 | 0.18 | 0.8 |
C4 | 12,100 | 95,000,000 | 28.2 | 0.0115 | 0.92 | 24.6 |
C5 | 22,300 | 15,100,000 | 3.4 | 0.0042 | 0.35 | 2.1 |
Cluster C4 had the highest mean risk score (0.92), the longest mean reporting delay (28.2 days), the largest mean claim amount (95,000,000 tomans), and a suspicious-claim percentage of 24.6%. These characteristics identify C4 as a priority for investigation, without independently proving fraud.
Cluster C2 also had a relatively high mean claim amount (45,500,000 tomans), a mean reconstruction error of 0.0082, and a mean risk score of 0.78. Its financial and temporal profile warranted further expert review. This pattern may warrant investigation for potentially organized, high-value fraud; independent case-level evidence would be needed to establish that interpretation.
By contrast, C3 had the lowest mean reconstruction error and risk score, consistent with comparatively typical behavior. Such groups may be candidates for streamlined claims handling, subject to the insurer's review procedures. C2 and C4 were identified as high-risk clusters because of their elevated reconstruction errors, risk scores, and reporting delays.
Optimized risk score weights
Table 11 presents the optimized component weights for the composite risk score.
Component | Weight | Interpretation |
|---|---|---|
Reconstruction error | α = 0.55 | Deviation from the learned dominant patterns |
Distance from cluster centroid | β = 0.30 | Deviation from the assigned cluster |
Insurance profile score | γ = 0.15 | Domain indicators, including policyholder history |
Reconstruction error received the largest weight (0.55), followed by distance from the cluster centroid (0.30) and the insurance profile score (0.15). The score therefore captured both deviation from the overall learned representation and deviation from the assigned behavioral group. Insurance-domain indicators supplied additional context. This combination was intended to improve the relevance of alerts and reduce unnecessary investigation, although no separate false-positive-rate analysis is reported.
Limitations
The evaluation includes synthetically modified cases and a limited set of verified labels. Performance on injected anomalies may not generalize to independently confirmed fraud, particularly when the variables used to construct anomalies also contribute to the risk score. The reported internal clustering indices describe structural separation rather than fraud status. Repeated-run uncertainty estimates, independent temporal validation, and fully specified component-level comparisons are needed to establish the robustness and operational value of the framework.
Conclusion
This study proposed a hybrid framework for identifying potentially high-risk automobile insurance claims when verified fraud labels are limited. A deep autoencoder extracted compact representations, and a genetic algorithm optimized cluster centroids before final K-means assignment. The framework was applied to a dataset of 179,514 claims.
The main evaluation configuration reduced the data to a 16-dimensional latent space. The silhouette score increased from 0.124 for K-means in the raw feature space to 0.3953 for the proposed method. Evolutionary optimization improved the best silhouette score from 0.3013 to 0.3953 over 30 generations, suggesting that centroid refinement contributed to the final clustering structure.
Reconstruction error provided an interpretable indication of unusual behavior. Injected cases had a mean error of 0.0124, compared with 0.0038 for normal cases. This difference suggests that reconstruction-based information can complement cluster structure in ranking claims for review.
The proposed method achieved Precision@100 of 0.68, compared with 0.18 for the baseline. Under the study's labeling and injection scheme, 68 of the 100 highest-ranked observations were positive evaluation cases. High-risk groups such as C4, with a mean risk score of 0.92, illustrate how the framework could help focus limited audit resources.
The results support combining unsupervised deep representation learning with evolutionary optimization as an interpretable decision-support approach. Conclusions about confirmed fraud remain constrained by limited verified labels, synthetic evaluation cases, and the need for further validation of the reported configurations and output summaries. Future research could use graph neural networks to investigate relationships among policyholders, claimants, and other involved parties and to identify potential fraud networks.
Funding
This research received no external funding.
Conflict of Interest
The authors declare no conflicts of interest.
Author Contributions
M.P.: Conceptualization, methodology, software, formal analysis, data curation, and writing—original draft. Z.B.F.: Conceptualization, methodology, supervision, and writing—review and editing. M.H.: Validation, formal analysis, and writing—review and editing. A.B.: Methodology, supervision, and writing—review and editing. All authors reviewed and approved the final manuscript.
References
Alturki, A., Bchir, O., & Ismail, M. M. B. (2021). Joint Deep Clustering: Classification and Review. International Journal of Advanced Computer Science and Applications, 12(10). https://doi.org/10.14569/IJACSA.2021.0121096
Audibert, J., Michiardi, P., Guyard, F., Marti, S., & Zuluaga, M. A. (2020). USAD: UnSupervised Anomaly Detection on Multivariate Time Series. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 3395–3404. https://doi.org/10.1145/3394486.3403392
Aytekin, C., Ni, X., Cricri, F., & Aksu, E. (2018). Clustering and Unsupervised Anomaly Detection with L2 Normalized Deep Auto-Encoder Representations (arXiv:1802.00187). arXiv. https://doi.org/10.48550/arXiv.1802.00187
Esna-Ashari, M., Khamesian, F., & Khanizadeh, F. (2022). Using local outlier factor to detect fraudulent claims in auto insurance. Journal of Mathematics and Modeling in Finance, (Online First). https://doi.org/10.22054/jmmf.2022.68662.1058
Goldstein, M., & Uchida, S. (2016). A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data. PLOS ONE, 11(4), e0152173. https://doi.org/10.1371/journal.pone.0152173
Gomes, C., Jin, Z., & Yang, H. (2021). Insurance fraud detection with unsupervised deep learning. Journal of Risk and Insurance, 88(3), 591–624. https://doi.org/10.1111/jori.12359
Krishna, K., & Murty, M. N. (1999). Genetic K-means algorithm. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 29(3), 433–439. https://doi.org/10.1109/3477.764879
Li, J., Zhang, J., Bah, M. J., Wang, J., Zhu, Y., Yang, G., Li, L., & Zhang, K. (2022). An Auto-Encoder with Genetic Algorithm for High Dimensional Data: Towards Accurate and Interpretable Outlier Detection. Algorithms, 15(11), 429. https://doi.org/10.3390/a15110429
Lunghi, D., Simitsis, A., Caelen, O., & Bontempi, G. (2023). Adversarial Learning in Real-World Fraud Detection: Challenges and Perspectives. Proceedings of the Second ACM Data Economy Workshop, 27–33. https://doi.org/10.1145/3600046.3600051
Min, E., Guo, X., Liu, Q., Zhang, G., Cui, J., & Long, J. (2018). A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture. IEEE Access, 6, 39501–39514. https://doi.org/10.1109/ACCESS.2018.2855437
Nian, K., Zhang, H., Tayal, A., Coleman, T., & Li, Y. (2016). Auto insurance fraud detection using unsupervised spectral ranking for anomaly. The Journal of Finance and Data Science, 2(1), 58–75. https://doi.org/10.1016/j.jfds.2016.03.001
Pang, G., Shen, C., Cao, L., & Van Den Hengel, A. (2022). Deep Learning for Anomaly Detection: A Review. ACM Computing Surveys, 54(2), 1–38. https://doi.org/10.1145/3439950
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Müller, K.-R. (2021). A Unifying Review of Deep and Shallow Anomaly Detection. Proceedings of the IEEE, 109(5), 756–795. https://doi.org/10.1109/JPROC.2021.3052449
Schrijver, G., Sarmah, D. K., & El-hajj, M. (2024). Automobile insurance fraud detection using data mining: A systematic literature review. Intelligent Systems with Applications, 21, 200340. https://doi.org/10.1016/j.iswa.2024.200340
Seidi Aghil Abadi, Z., Sehhat, S., & Salehi, R. (2017). Investigation and analysis of fraudulent factors in the third-party civil liability car Insurance (Third-party insurance-physical damage). Iranian Journal of Insurance Research, 7(1), 13–26. https://doi.org/10.22056/ijir.2018.01.02 [In Persian]
Vosseler, A. (2022). Unsupervised Insurance Fraud Prediction Based on Anomaly Detector Ensembles. Risks, 10(7), 132. https://doi.org/10.3390/risks10070132
Yousefimehr, B., Ghatee, M., Moradi, S., Tafakor, Y., & Tavakoli, S. (2025). Detecting car insurance fraud using improved clustering with genetic algorithm. Iranian Journal of Insurance Research, 14(2), 109–118. https://doi.org/10.22056/ijir.2025.02.02 [In Persian]