September 2026 | Added New System Design, Practical Development, and Machine Learning Questions to AI Collaboration

In September 2026, we added new questions to the AI Collaboration format: ten system design questions in which candidates work out the design through conversation, along with practical development questions and questions for machine learning engineers.

You can add any of them from the AI Collaboration question list on the test creation screen.

Questions added

Theme and Question Mode are filters in the question list. Candidates take all of these questions on the editor screen.

System design questions

We added ten questions in the same format as "Social Network Home Timeline Design," which we added in August 2026.

The problem statement does not provide the required features, expected scale, quality targets, or development constraints upfront. Candidates ask an AI acting as the product manager to draw out the conditions, then write up the requirements, an architecture diagram, and their design decisions and trade-offs in DESIGN.md. Diagrams are drawn in Mermaid, with an AI agent available to help with the notation.

• URL Shortener Service Design: short IDs that resist collisions and guessing, expiration, low-latency redirects, handling short URLs that draw heavy traffic, and keeping redirects working when analytics fails.
• Accommodation Booking Service Design: preventing double booking by handling availability and reservations atomically, while accounting for the different load profiles of search, listing management, and booking.
• Business Card Management Service Design: digitizing photographed business cards, with asynchronous OCR and human review, a boundary for handling personal data, reprocessing, and backlog monitoring.
• C2C Ticket Marketplace Design: exclusive control so the same listing is never sold twice, state transitions from a temporary hold to a completed purchase, isolating load concentrated on popular events, and consistency between listings and search results.
• Large-Scale Web Crawling Service Design: a service that periodically crawls company websites and analyzes changes, covering job splitting, per-host access limits, duplicates and redelivery, failures and retries, and operations a small team can sustain.
• Content Publishing Service Design: reflecting drafts, publication, updates, and deletion consistently across read paths such as search and recent-content listings, while handling read-heavy traffic and image delivery.
• Video Streaming Service Design: a consistent design from video upload, pre-publication review, and multi-format conversion through streaming delivery and a home feed based on watch history.
• Push Notification Delivery Service Design: serving time-sensitive notifications and large broadcasts on one platform, covering external provider failures and rate limits, duplicate prevention, and delivery state tracking.
• Real-Time Chat Service Design: combining low-latency delivery to online members with storage that never loses an accepted message, including ordering, duplicates, reconnects, and degradation during failures.
• Designing a Gradual Migration from a Monolith to Microservices: planning a step-by-step migration of a team task-management app without interrupting users, covering service boundaries, data ownership and consistency, and cutover without downtime.

Skills you can evaluate

• Problem-solving: estimating load from the scale and quality targets they elicit, and judging a feasible design and its alternatives
• Communication: confirming the conditions that drive the design through conversation with a non-engineering stakeholder, without gaps or excess
• Design: designing the data model, read and write paths, and failure behavior consistently and with clear reasoning
• AI collaboration: using an AI agent for requirements and diagrams while making the architectural and trade-off decisions themselves

Who they're for

Mid-level to senior engineers who can design the backend of a web service. "Designing a Gradual Migration from a Monolith to Microservices" is aimed at engineers who can own designs that include data migration for an existing product.

Practical development questions

Legacy Order Management System Insourcing Migration & Inventory PoC (C# / ASP.NET Core)

This is the C# / ASP.NET Core version of the existing "Legacy Order Management System Insourcing Migration & Inventory PoC."

The scenario is a project to bring a manufacturer's order management system in-house from an outside vendor. Candidates ask an AI acting as the insourcing lead to draw out the current state and constraints, plan the migration and choose the technology, and then build an inventory management PoC. The PoC is implemented in C# / ASP.NET Core; candidates choose everything else, such as the architecture and data store.

It is aimed at tech leads and senior engineers with .NET experience who can lead technology selection and migration design.

Employee Data Synchronization from HR CSV Files

Each time the HR system's CSV is imported, an organization member management service ends up with duplicate members, missed updates, and invitations and access that don't match. Candidates improve it so that it synchronizes safely on every import. They investigate and handle not only the reported symptoms but also other cases that could occur, then explain the cause, design decisions, and what they verified in SOLUTION.md.

Available in six variants: Django / Express TypeScript / Go / Laravel / Ruby on Rails / Spring Boot.

Preventing Double Charges and Reconciling with a Payment Provider

An order service is double-charging and losing charges in its integration with an external payment provider, and candidates repair it. The integration document the company still holds disagrees with how the provider behaves today, so candidates investigate the supplied code and documents together with the provider sandbox's actual behavior. They design and implement reconciliation of unsettled charges, handling of data that is already inconsistent, and the line between automatic repair and human decisions, then explain what they investigated and what they can guarantee.

Available in six variants: Django / Express TypeScript / Go / Laravel / Ruby on Rails / Java / Spring Boot.

Triage and recover a server with high CPU usage

An operations question in which candidates log in to a production Linux server whose CPU usage stays high, find the cause, and resolve it. It checks whether they can identify the process causing the load and stop it for good without taking down the business services running on the server. Candidates write their investigation steps, cause, actions, how they confirmed business impact, and recurrence prevention in report.md, which is evaluated together with the server's final state.

Because the scenario is a production server, the AI agents in the test environment have no permission to inspect it. Candidates investigate on their own with basic commands such as top, netstat, and systemctl. No specialized knowledge is required, so it works for junior infrastructure and operations engineers.

Machine learning questions

Following "Fine-tuning an image classification model," added in July 2026, we added two more questions for machine learning engineers. In both, candidates implement training in PyTorch and can use a GPU in the test environment. The final model is evaluated on hidden validation data.

Building an image classifier that holds up in the deployment setting

Candidates fine-tune an ImageNet-pretrained ResNet-50 on an image dataset of four butterfly species. In the training data, species and capture setting are strongly correlated, so a model that simply learns that correlation breaks down in the deployment setting, where capture settings are evenly distributed. The question evaluates whether candidates can inspect the data, identify the correlation, and redesign how the training data itself is composed. Accuracy on the weakest group is evaluated, not just average accuracy.

Building a surface inspector that shows its evidence

Set in visual inspection on a production line, this question asks candidates to build a model that reports not only the defect type but also where in the image it looked to decide. The training data has only image-level labels, with no defect locations. Candidates must show the location for each of six defect types whose evidence appears differently, from a thin scratch to scattered tiny pits to peeling that shows only at its outline, and the weakest class weighs most in scoring.

Who they're for

Machine learning engineers with hands-on PyTorch experience. Familiarity with fine-tuning pretrained models and a working knowledge of distribution shift and generalization make these questions easier to approach.

How to use them

Add any of these questions from the AI Collaboration question list on the test creation screen. You can filter the question list by Theme, Question Mode, and Screen Layout.

Related pages

• Understanding the AI Collaboration Format
• How to Select Test Questions
• Create a Coding Test