Standard AI Labeling Tool
TeammatesRyan Cook - PM
Igo Moiseev - Senior Director, Engineering
Vallab Karanam - Front End Eng
Timeline2022 - 2024
*Some visuals and UI elements have been modified to comply with NDA requirements.
Overview
Behind every “just walk out” shopping experience is a mountain of data—and someone labeling it. At Standard AI, our computer vision systems rely on human reviewers to verify and annotate retail activity, like detecting when a shopper picks an item from the shelf. But the internal tool our contractors used was slow, confusing, and frustrating.
My Role
As the Lead Product Designer, I set out to reimagine the labeling experience with a clear goal: make the work of human data labelers faster, simpler, and more intuitive—because their role is essential to training smarter AI. I streamlined what was once a complex, fragmented process into a clean, two-step workflow powered by two key features: Smart Blink and Dynamic Camogram.
Beyond rethinking core interactions, I also led a series of quality-of-life improvements—from a refreshed visual design and faster hotkey inputs to a more intuitive comment menu—all designed to reduce friction and support the people behind the data.
For a long time, data labelers had to manually guess product identities and rely on semantic search to label “take” actions—a process that was both slow and unreliable.
Our contractors were labeling “take” actions in video footage—one frame at a time. To tag a product, they had to manually locate it in a planogram, search, compare thumbnails, and match it with what they saw. It was tedious, error-prone, and mentally exhausting.
Key pain points:
- Average time to label one “take” action: ~2 minutes
- Learning curve too steep—most contractors quit before mastering it
- 85% of reviewers found labeling difficult or confusing
- Human inefficiency slowed down AI training and drove up ops costs
“I find it easy to label a take action.” — 0% agreement in reviewer survey
AI Labeling Tool 2.0
After multiple in-depth sessions with data labelers and QA, we redesigned the AI Labeling Tool with a cleaner, more focused interface. We removed non-essential features, keeping only the most-used—Smart Blink—and rebuilt the labeling experience around it, integrating the newly developed Dynamic Camogram for a more streamlined and intuitive workflow.
Smart Blink
Labeling a “take” action often begins with confirming whether a product was actually removed from the shelf. The Smart Blink feature simplifies this by showing a quick ping-pong animation of ±5 seconds around the playhead—highlighting any items that were taken, so reviewers can spot changes instantly and label with confidence.
Dynamic Camogram
By analyzing store planograms, our machine learning algorithm learns to make educated guesses about which products occupy specific shelf locations. We integrated this visual search capability directly into the UI—transforming the shelf itself into an interactive interface, where data labelers can perform precise, direct manipulation with speed and clarity.
A Simple Yet Powerful Combo
Combining the 2 main features, the data labeler gets a more streamlined journey when labeling an action. Reducing the average labeling time from 2min to less than 30 second.
Comment Menu
In real-world scenarios, interactions are often more complex than simply taking or placing a product on a shelf. We redesigned the comment menu to give data labelers an easy way to “bail out” of frustrating cases, while also enabling the system to capture, sort, and learn from a wide range of edge cases.
Keyboard Shortcut
After closely observing the interaction patterns of dozens of data labelers with QA experts, we developed an intuitive and easy-to-learn hotkey system—and tested it ourselves to ensure usability. The shortcuts were inspired by traditional video editing tools and, surprisingly, the classic control schemes of video games.
Impact
We soft-launched the new interaction in a simulated “Shadow Mode” and gradually rolled it out:- 📉 Review time dropped from ~2 mins to <1 minute
- 💸 Cost per store decreased by 83%
- 🤖 Accuracy of product detection rose to 96%
- 🚀 The feature shipped in production across 21+ stores
Dynamic Camogram turned a tedious task into a guided, intuitive interaction—training better AI while creating a better experience for the humans behind it.
Reflection
This project reinforced a core belief: great AI starts with great human tools. By turning data labeling into a collaborative loop between people and machine, we made Standard AI’s models not just smarter—but more scalable and sustainable.
"Designing for people behind the curtain is what makes the magic on stage possible."
© 2025 Ke Ding. All rights reserved.