In my previous Advisor, we discussed how technology vendors and end-user companies are applying the advanced natural language processing capabilities of generative artificial intelligence (GenAI) to develop new solutions and applications designed to facilitate conversational commerce. This Advisor continues our retail industry focus by examining the use of GenAI combined with augmented reality (AR), virtual reality (VR), computer vision, digital twins, and social collaboration technologies to further enhance customers’ online and in-store shopping experiences.
Key Developments
Retailers have been applying AR, VR, and computer vision to enhance their digital commerce platforms for some time now. The most popular use is for “virtual try-on” applications, which, as suggested, provide a virtual environment for shoppers to see how clothes, makeup, accessories, and other products look and fit before making a purchase. Although most popular in clothing and fashion applications, retailers also use these same technologies to allow customers to see how furniture, rugs, artwork, lighting, and other home furnishings would look in their own homes.
The latest trend has retailers expanding their AR- and VR-powered virtual shopping experiences with the addition of GenAI, digital twins, and collaboration technologies. This is significant: the convergence of AR and VR with GenAI, digital twins, and social collaboration is reshaping both e-commerce and in-store shopping. In effect, it is allowing retailers to offer more personalized and immersive experiences and enabling them to engage with customers in new, innovative ways. Benefits include a better customer experience (CX), increased shopping carts, and reduced product returns. The following are some real-world examples of companies following this trend.
Walmart Optical Try-On
Retail giant Walmart has been using AR and VR to help customers envision how fashion, beauty, and home items would look virtually for several years. In 2021, it acquired virtual fitting room vendor Zeekit. Walmart subsequently used this technology to develop several virtual try-on applications, including Choose My Model and Be Your Own Model. These were implemented on its mobile app and website in 2022. Last year, Walmart introduced a virtual makeup try-on feature on its mobile app.
In February, the company expanded its virtual try-on offerings to include a new prescription eyewear application that allows customers to virtually try on and purchase frames (via mobile app or website). This application applies 3D visualization, computer vision, and other advanced algorithms to create a digital twin of eyewear frames. Company reps claim that the digital twin technology offers the customer a more realistic virtual try-on experience.
After finding a pair of frames they like, a shopper clicks on the “Try Them On” button on the product page. A facial scan is performed using the camera on the customer’s mobile device, allowing them to see how the frames look on their face. Once the customer uploads their prescription, another face scan is performed to capture their pupillary distance measurements (i.e., the distance in millimeters between the centers of each pupil.) After making the purchase, should the customer require further adjustments to their glasses, they can bring them into any Walmart store for service.
Shopping in VR: Wayfair’s Decorify App
Retailers are also developing applications specifically designed to take advantage of the highly interactive functionality (and growing popularity) of VR headsets. European homeware retailer Wayfair has developed a virtual room styler (called “Decorify”) that implements GenAI and 3D imaging to allow Apple Vision Pro users to reimagine their living spaces and purchase products from Wayfair’s catalog of home furnishings.
Decorify provides a highly immersive and visually interactive environment for users to redesign their living spaces. To create their virtual room, users take a photo of the room using the Vision Pro VR headset or select one of the sample images provided by the app. Next, they choose a particular style — modern, mid-century, industrial, and so forth — and Decorify, using GenAI, will generate and present images of the shopper’s room redesigned to reflect the requested look and feel.
Decorify leverages VR’s spatial computing capabilities; users can view and select Wayfair items in 3D directly from their reimagined virtual room to see how they look and fit. Moreover, 3D items are visualized accurately and appear at scale wherever placed in the virtual room. Users can also move the products around in their virtual rooms to suit their decorating tastes. This highly interactive functionality lets shoppers accurately view products in any room or space in their home and offers them the additional confidence that their purchases will suit their tastes and needs. Although a pilot application, Decorify is available in the visionOS App Store for Apple Vision Pro.
Social Commerce: Walmart’s Shop with Friends
Retailers are integrating collaboration and social networking capabilities with their virtual try-on technology to create more interactive and engaging social shopping platforms. A good example is Walmart’s “Shop with Friends” application, which builds on the company’s expertise in AR and more recent developments involving GenAI search.
The Shop with Friends experience proceeds as follows. Using virtual try-ons, shoppers mix and match apparel items virtually in an AR environment, allowing them to view how clothing items will look on a model similar in size and shape to themselves. A GenAI search capability lets them ask for and view products by expressing what they’re looking for using natural language. They can search for products and advice by use case or theme.
For example, they could enter, “I want to be a pirate for Halloween” and would be presented with various pirate-themed items: a puffy shirt or blouse, toy sword, eye patch, boots, scarf, skull-and-cross bones flag, and so forth. These items can be selected from across different product categories (the platform integrates with Walmart’s catalog of products available online and in physical stores.) Once they have created their virtual outfit, shoppers can share them with friends. This includes inviting friends via popular mobile sharing apps where they can interact by “hearting” looks and commenting on their friend’s fashion choices.
Savage X Fenty’s Virtual In-Store Shopping Experience
Some retailers are turning to AR and AI-powered platforms to recreate the convenience of online virtual try-on applications in their physical stores. For example, Savage X Fenty is testing a virtual in-store fitting room technology developed in collaboration with FIT:MATCH.ai and Intel in its store in Atlanta, Georgia, USA.
The application, called Fit Xperience, employs AR, computer vision, AI, and digital twins to digitize the in-store fitting room experience. The fitting room–based technology works by scanning shoppers and matching them with the best-fitting clothes based on their body shape. It uses Intel RealSense depth sensors to create an anonymized avatar based on the shopper’s exact body shape. Fit Xperience then compares the shopper’s “body map” to a database of digital twins (bodies that have been scanned previously) and applies AI algorithms to recommend the best-fitting products and accommodating styles to the customer.
In addition to personalization, Fit Xperience offers other benefits. It speeds up the shopping experience by eliminating guesswork, allowing customers to shop for only what fits. It also helps ensure customer privacy: by using depth sensors, not cameras, to generate a mapping of a customer’s shape, no videos or pictures are taken or stored. The process is also fast; company reps claim it takes less than 30 seconds from beginning to end.
Conclusion
Retailers are optimizing their AR- and VR-powered virtual shopping experiences with the addition of GenAI, digital twins, and social collaboration technologies. The convergence of these technologies is reshaping both e-commerce and in-store shopping by allowing retailers to offer more personalized and immersive experiences and enabling them to engage with customers in new, innovative ways.
Finally, I’d like to get your opinion on the use of AR and VR with GenAI, digital twins, collaboration, and other advanced technologies. As always, your comments will be held in strict confidence. You can email me at experts@cutter.com or call +1 510 356 7299 with your comments.
