Skip to content

PaySim

PaySim is a synthetic dataset that simulates mobile-money transactions for fraud detection. The task is to predict whether a transaction is fraudulent.

This dataset is large. It has 6,362,620 transactions, of which only 8,213 are fraudulent and 6,354,407 are not. The strong class imbalance makes it a good test for fraud detection, and the size makes it suitable for large scale boolean rules mining.

Data preparation

Download PaySim.csv from Kaggle and place it in the ./data folder. Then run the preprocessing script at scripts/preprocess/paysim_preprocess.py.

python scripts/preprocess/paysim_preprocess.py --data_path data

The script renames the isFraud column to target, adds two boolean features for external origin and destination accounts, and writes data/PaySim.hdf.

Running a benchmark

Benchmark the Boolean GP on the preprocessed dataset with scripts/run_benchmark.py.

python scripts/run_benchmark.py --data_path data/PaySim.hdf

Because the dataset is large, a quick run with fewer runs, folds, and epochs is useful for a first check.

python scripts/run_benchmark.py \
    --data_path data/PaySim.hdf \
    --num_runs 5 \
    --n_folds 3 \
    --num_epochs 500

See python scripts/run_benchmark.py --help for the full list of options.

Hyperparameter tuning

Tune the dataset with the Optuna script.

python scripts/optuna_hypertuning.py \
    --data_path data/PaySim.hdf \
    --study_name PaySim \
    --hp_config hyperparameter_configs/default.yaml \
    --n_trials 100 \
    --artifact_dir ./artifacts

Results

The table below reports the F1 score on the validation set for black-box and explainable classifiers on PaySim.1

Type Algorithm F1 score
black-box k-NN 0.16
black-box SVM 0.47
black-box Random Forest 0.81
black-box Autoencoder + MLP 0.82
black-box XGBoost 0.84
explainable DSC 0.78
explainable DSC + Fuzzy logic 0.19
explainable hgp-lib (depth=0)1 0.81
explainable hgp-lib + hypertuning 0.82

References

  1. Ramona-Georgiana Albert, George Stoica, and Mihaela Elena Breabăn. Evolving Boolean Rule-Based Classifiers for Fraud Detection via Genetic Programming. In 2025 27th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), 524–531. 2025. URL: https://ieeexplore.ieee.org/abstract/document/11479552, doi:10.1109/SYNASC69064.2025.00076. ↩↩