Real counts for the contemporary-trained YOLOv11s on the annotated historical images, none of which it ever trained on, at the default confidence threshold of 0.25. The detector doesn't mislabel the past; it mostly doesn't see it.
Problem
Object detectors are trained on today's streets and quietly assumed to work everywhere. Prior research has shown that image classifiers lose accuracy as the years pass. I asked the same question of object detection over 35 years: how much does a modern detector degrade on street photography from the 1980s and 1990s, and is the cause poorer image quality, or the changing look of cars, buildings and people?
What I did
I unified four datasets spanning 35 years, from 1980s New York archives to 2017+ crowdsourced street imagery, into one five-class taxonomy and fine-tuned four YOLOv11 variants. I then ran four controlled experiments: normalising resolution across eras, synthetically ageing modern images to separate image quality from morphology, mapping where errors fall in the frame, and diagnosing each class's error type. Partway through I found that my first scores included images the models had trained on, which flattered the modern benchmark. I rebuilt the evaluation on held-out images and re-ran every experiment, and the results here are the corrected ones. The study was planned with a research proposal, an ethics application and a data management plan.
Result
Scored on images they never saw, contemporary-trained models lose 61 to 63% of their mAP on historical images. Morphology dominates: ageing modern images explains only 11 to 22% of the drop, and normalising resolution more than doubled the gap instead of closing it. At the default confidence threshold the failure is silent: precision holds while recall collapses, and errors on historical frames reach the image edges about twice as often. The modern benchmarks use 30 held-out images each, so their scores are noisy.
Why it fails
Ageing modern photos only gets you part of the way. Adding film grain, faded colour, blur and compression to modern images costs some accuracy, but real 1980s photos lose far more. The remainder comes from the objects themselves: boxier cars, different buildings and clothing. The aged images stay slightly sharper than real archives, so the quality share is a conservative estimate.
Historical errors reach the frame edges about twice as often. About 63% of them involve a box that reaches into the outer 15% of the frame, against 33% on modern dashcam images. Measured by box centre instead, errors sit toward the middle of every dataset, so the gap shows boxes stretching to the borders, not errors piling up there. I have not tested why.
[email protected] on images the models never saw: all of the 1980s and 1990s sets, 30 held-out images for each modern set. Edge rate: share of all errors whose box reaches into the outer 15% of the frame.
What I'd do now
The study isolates what drives the gap; the next steps are about closing it without losing the present:
- Separate colour from age. The 1990s images are greyscale, which mixes two effects. Testing greyscale modern images on their own would isolate it.
- Collect more of the rare classes. Cyclists and trucks barely appear in the archives, which makes their results unreliable. Targeted sampling would give each class a fair test.
- Raise the resolution floor. The 1980s images are around 150 pixels across. Higher-resolution scans would show how much of the remaining gap is truly morphological.
- Recalibrate confidence for archive photos. Cars keep an AP of about 0.5 on the 1980s photos, so the model still ranks them sensibly; it just is not confident enough to report them at the default threshold. Calibrating the threshold on archive images would recover detections, at the cost of more false positives.
- Test mixing eras fairly. Training on all eras lifted held-out 1980s mAP from 0.15 to 0.46, and the modern scores did not visibly drop. With only 30 modern images per set I cannot say whether anything is lost, so a larger shared test set comes first.
- Score on bigger held-out sets. My modern benchmarks use 30 images each. A larger held-out set would tighten every modern number and the per-class results.
Skills gained
- Research design: framing hypotheses and designing controlled experiments that separate competing explanations
- Dataset engineering: unifying four sources from 35 years into a single five-class annotation taxonomy
- Object detection: fine-tuning and evaluating YOLOv11 variants of different sizes and training regimes
- Error analysis: precision-recall diagnostics, per-class failure types and spatial error mapping
- Synthetic degradation: building an image-ageing pipeline calibrated against real historical photos
- Academic practice: literature review, a research proposal, an ethics application, a data management plan and a poster
Contact
Looking for a data science or AI intern?
I'm open to internships in machine learning, computer vision and applied research for February 2027. Email is the quickest way to reach me.