July 2026 | Released “Fine-Tuning an Image Classification Model” for Machine Learning Engineers
AccountWe have added a new problem for machine learning engineers, “Fine-Tuning an Image Classification Model,” to the AI Collaboration format.
Problem Overview
In this problem, candidates fine-tune an ImageNet-pretrained ResNet-50 on a biological species image dataset sourced from GBIF to build a classification model with strong generalization performance.
The training data is a small dataset containing 10 classes with 100 images per class. Candidates use PyTorch to implement the model training process while mitigating overfitting, which is common when working with limited data. The time limit is 90 minutes.
Skills You Can Evaluate
This problem primarily evaluates the following skills:
- Designing a fine-tuning strategy, including replacing the head of a pretrained model, freezing layers, and applying discriminative learning rates
- Applying data augmentation, regularization, and learning-rate scheduling suited to limited data
- Estimating generalization performance and tuning hyperparameters
- Implementing a training loop in PyTorch
The final model is evaluated on hidden validation data using macro-F1 and ECE (Expected Calibration Error). This makes it possible to assess not only classification performance but also the reliability of predicted probabilities.
Recommended Hiring Profile
This problem is suitable for evaluating machine learning engineers who have experience implementing deep learning models with PyTorch and foundational knowledge of fine-tuning pretrained models and image processing.