The pitfalls of overfitting in optimization of a manufacturing ouality control procedure

  • Tea Tušar, Klemen Gantar, and Bogdan Filipič. The pitfalls of overfitting in optimization of a manufacturing quality control procedure. In Proceedings of the 7th International Conference on Bioinspired Optimization Methods and their Applications, BIOMA 2016, pages 241-253, Bled, Slovenia, 2016.
    [BibTeX] [Abstract] [Download PDF]

    We are concerned with the estimation of copper-graphite joints quality in commutator manufacturing—a classification problem in which we wish to detect whether the joints are soldered well or have any of the four known defects. This quality control procedure can be automated by means of an on-line classifier that can assess the quality of commutators as they are being manufactured. A classifier suitable for this task can be constructed by combining computer vision, machine learning and evolutionary optimization techniques. While previous work has shown the validity of this approach, this paper demonstrates that the search for an accurate classifier can lead to overfitting despite cross-validation being used for assessing the classifier performance. We inspect several aspects of this phenomenon and propose to use repeated cross-validation in order to amend it.

    @InProceedings{2016-05-TUSAR,
    title = {{The pitfalls of overfitting in optimization of a manufacturing quality control procedure}},
    author = {Tea Tu\v{s}ar and Klemen Gantar and Bogdan Filipi\v{c}},
    booktitle = {{Proceedings of the 7th International Conference on Bioinspired Optimization Methods and their Applications, BIOMA 2016}},
    pages = {241-253},
    date = {2016-05-18/2016-05-20},
    year = {2016},
    address = {Bled, Slovenia},
    url = {http://bioma.ijs.si/proceedings/2016/17%20-%20The%20Pitfalls%20of%20Overfitting%20in%20Optimization%20of%20a%20Manufacturing%20Quality%20Control%20Procedure.pdf},
    abstract = {We are concerned with the estimation of copper-graphite joints quality in commutator manufacturing---a classification problem in which we wish to detect whether the joints are soldered well or have any of the four known defects. This quality control procedure can be automated by means of an on-line classifier that can assess the quality of commutators as they are being manufactured. A classifier suitable for this task can be constructed by combining computer vision, machine learning and evolutionary optimization techniques. While previous work has shown the validity of this approach, this paper demonstrates that the search for an accurate classifier can lead to overfitting despite cross-validation being used for assessing the classifier performance. We inspect several aspects of this phenomenon and propose to use repeated cross-validation in order to amend it.}
    }

This entry was posted in Dissemination. Bookmark the permalink.

Comments are closed.