Perturbed Model Validation: A New Framework to Validate Model Relevance - INRIA - Institut National de Recherche en Informatique et en Automatique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2019

Perturbed Model Validation: A New Framework to Validate Model Relevance

Résumé

This paper introduces PMV (Perturbed Model Validation), a new technique to validate model relevance and detect overfitting or underfitting. PMV operates by injecting noise to the training data, retraining the model against the perturbed data, then using the training accuracy decrease rate to assess model relevance. A larger decrease rate indicates better concept-hypothesis fit. We realise PMV by using label flipping to inject noise, and evaluate PMV on four real-world datasets (breast cancer, adult, connect-4, and MNIST) and three synthetic datasets in the binary classification setting. The results reveal that PMV selects models more precisely and in a more stable way than cross-validation, and effectively detects both overfitting and underfitting.
Fichier principal
Vignette du fichier
main.pdf (1.12 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02139208 , version 1 (24-05-2019)
hal-02139208 , version 2 (27-05-2019)

Identifiants

  • HAL Id : hal-02139208 , version 1

Citer

Jie Zhang, Earl T Barr, Benjamin Guedj, Mark Harman, John Shawe-Taylor. Perturbed Model Validation: A New Framework to Validate Model Relevance. 2019. ⟨hal-02139208v1⟩
289 Consultations
728 Téléchargements

Partager

Gmail Facebook X LinkedIn More