Enriching the retail experience with pixel-perfect semantic segmentation

The mission: Make virtual try-ons feel real

ZERO10 is a tech company specializing in providing AR and AI solutions with expertise in multiple industries including retail and wholesale, sports, entertainment, and gaming. The company aimed to utilize augmented reality for online footwear shopping, enabling customers to obtain a realistic picture before making a purchase. They required data that was not only accurate but also intricately segmented to represent real-world fit and finish. Their existing pipeline, however, fell short on both quality and speed.

The challenge: Achieving pixel perfection at scale

While details are critical to the AR experience, ZERO10's existing segmentation tool could not provide detailed accuracy. Adding to that, their previous vendor was unable to match precision, scale, and speed required.

Key obstacles

• Inadequate precision from the current Segment Anything Model (SAM) tool. As an automated segmentation tool, SAM struggles to cleanly segment shoes from backgrounds or feet, resulting in visual artifacts and unnatural overlays
• Inconsistent output and detail across image datasets from the previous vendor affected the AR model's performance
• Pipeline bottleneck with more time needed to retrain models due to poor segmentation, resulting in the model struggling to scale with new footwear, styles, and textures
• Unmemorable AR try-on experience due to misalignment and segmentation leaks impacting realism and decreasing customer confidence

The goal

Deliver a detail-rich segmentation pipeline to power AR in footwear.

The solution: A dynamic of expertise and infrastructure

We fronted this exercise by bringing in annotators with experience in semantic segmentation and leveraged our data annotation infrastructure for task completion and scalability.

Our approach

1. Engaged a team of annotators familiar with the fine details of semantic segmentation
2. Familiarized annotators with the scope-of-work and the use of our proprietary annotation platform—streamlining the data labeling process to ensure adherence
3. Set up a collaborative QC and feedback workflow within a scalable infrastructure to enable error mitigation and correction, reducing potential delays in a pipeline that processes high-volume, complex data

The results: Pixel-precise segmentations in rapid succession

• Completed the task in 5 weeks — beating the 6–9 week timeline by 37.5%
• Delivered 15,000 high-quality segmentation data points
• Hit 99% segmentation accuracy, surpassing the 90% target
• Overcame the limitations of the existing automated segmentation tool
• Accelerated AR product development and deployment, enabling a more immersive and precise try-on experience

Through focused talent engagement and ready tools to forward AI, we quickly propelled ZERO10's goals to impact customer engagement and satisfaction on a higher level.