Visual analysis and authentication of artworks are challenging tasks central to art history and criticism. This preliminary study presents a computational tool for scholars examining and authenticating a restricted class of paintings, with a specific focus on the paintings of Raffaello Sanzio da Urbino, more popularly known as Raphael. We applied transfer learning to the ResNet50 deep neural network for feature extraction and used a support vector machine (SVM) binary classifier in support of authentication. Edge detection and analysis algorithms, considered to be crucial for capturing the essence of Raphael’s artistic style, including the brushwork signatures, were also integrated and are used as an authentication tool. The machine learning approach we have developed demonstrates an accuracy of 98% in image-based classification tasks during validation using a test set of well known and authentic paintings by Raphael. Of course, a full authentication protocol relies on provenance, history, material studies, iconography, studies of a work’s condition, and more. Our work, then, contributes to just a portion of a full authentication protocol. Our findings suggest that machine learning methods, properly employed by experts aware of context, may enhance and expand traditional visual analysis for problems in art authentication.
For such ancient paintings, often repair work has been done, sometimes over considerable portions of the surface.
It's entirely possible that Joseph's face had to be redone (water damage, smoke, a tear) a century later and of course was done by a different painter.
I'd expect most old paintings would 'suffer' from this. Not unlikely some of the training data (49 paintings 'by Raphael') suffered from this, muddying the conclusion further.
https://heritagesciencejournal.springeropen.com/articles/10....
Abstract
Visual analysis and authentication of artworks are challenging tasks central to art history and criticism. This preliminary study presents a computational tool for scholars examining and authenticating a restricted class of paintings, with a specific focus on the paintings of Raffaello Sanzio da Urbino, more popularly known as Raphael. We applied transfer learning to the ResNet50 deep neural network for feature extraction and used a support vector machine (SVM) binary classifier in support of authentication. Edge detection and analysis algorithms, considered to be crucial for capturing the essence of Raphael’s artistic style, including the brushwork signatures, were also integrated and are used as an authentication tool. The machine learning approach we have developed demonstrates an accuracy of 98% in image-based classification tasks during validation using a test set of well known and authentic paintings by Raphael. Of course, a full authentication protocol relies on provenance, history, material studies, iconography, studies of a work’s condition, and more. Our work, then, contributes to just a portion of a full authentication protocol. Our findings suggest that machine learning methods, properly employed by experts aware of context, may enhance and expand traditional visual analysis for problems in art authentication.