Designing a New Way for Shoppers and Customers to Get Help
CASE STUDY INSTACART
THE CHALLENGE
Design a solution to significantly reduce agent calls while also providing shoppers and customers with excellent service.
ROLE
Lead Product Designer
THE OUTCOME
An AI-powered chat assistant that provides 24/7 assistance to shoppers and customers for instant help, and hands off to an agent only when necessary. Saved Instacart $3M over a year.
TOOLS
Voiceflow, FigJam, Excel, Miro, third-party software (for publishing, intent management, and analytics)
Being an Instacart shopper is rewarding in several ways: additional income, flexible hours, and a lovely way to serve your community. Families depend on you to shop for them so they can spend more quality time with each other.
And using Instacart as a customer is incredibly helpful in that it saves a ton of time grocery shopping. Time that can be spent with family and friends, learning a new hobby, setting up for a party, or a million other things.
But whether you’re a shopper or a customer, you’ll encounter issues and call the help center for assistance (which, face it, nobody likes to do, no matter how good the customer service team is).
Instacart, especially after the influx of users that joined the platform after COVID hit, began to receive an enormous number of calls to their help center.
Shoppers and customers were reaching out so often that Instacart didn't have enough agents to meet the demand and still provide excellent service.
That’s why our small, but mighty team of three (1 design manager, 2 designers) built an end-to-end chat experience for shoppers and customers to get instant help from an AI assistant.
DESIGN PROCESS
Our team developed our very own conversation design process. We carefully documented it and shared it with stakeholders to evangelize our practice internally.
We researched the most common questions users reach out about and designed 42 workflows to address the top 80% of shopper contact drivers and 62% of customer contact drivers.
AI ASSISTANT IN ACTION
CUSTOMER CHATBOT
Here’s our customer chatbot in action, available on web only.
SHOPPER CHATBOT
Here’s our shopper chat experience, available on mobile and mobile web only. Please note that I didn’t design the UI. We were restricted with how our UI could look due to the third-party platform we were using to engineer our workflows.
DESIGNING & PROTOTYPING IN VOICEFLOW
Voiceflow is the conversation design tool we use to wireframe and prototype our experiences. We used a third-party software (that I can’t name) to publish our workflows, manage intents, and track our analytics. Voiceflow is known as “the Figma of conversation design.”
Here’s what the tool looks like:
Here’s the customer chatbot example above wireframed in Voiceflow:
SUCCESS METRICS
In order to measure the success of our chatbot experiences, we monitored several user, business, and technology metrics. Some are listed below:
User Metrics
Customer satisfaction (CSAT): How satisfied the user is with the experience. Based on a 5-star rating the user provides at the end of a chat
Self-service rate (SSR): The percent of users who complete a flow and get their question answered
Drop-off rate: The percent of users who leave a chat at a particular point
User feedback: Typed feedback users provide at the end of a chat
Volume: The number of users who go through a flow
Business Metrics
Agent call deflection: The percent of agent calls deflected due to users using the chatbot to self-serve
Agent cost savings: The amount of money saved by deflecting calls to agents
Revenue generated: The amount of money generated as a result of a user going through the chatbot
Cost per order (CpO): The amount of money Instacart spends to fulfill a single order
Technology Metrics
Intent accuracy: How accurate the NLU model is in categorizing the user’s intention and mapping it to the correct workflow
Confidence score: The confidence level (in a percent) the model has in predicting the user’s intention
ACCESSIBILITY
Advocating for accessibility features was an uphill battle, and our team was only able to achieve minimal gains, like changing the chat colors to a brand-aligned, accessible green and an easily readable font. What I would’ve liked to see in-production include:
multimodality, specifically voice capabilities that wouldn’t require a user to type
more concise messaging (often we were limited by legal + content requirements)
testing integration with screen readers, voice control, magnification, and other assistive technologies
labelling interactive elements
notifying the user when a new message arrives
using sounds when sending and receiving messages
including alt-text for images and videos
IMPACT
Since the deployment of the customer and shopper help chatbots in 2021, our team has saved the company more than $3 million alone through over 10 million user sessions.
INDIVIDUAL IMPACT
Building a large-scale AI solution is a big team effort (especially when you have a little team, ha), but it’s great to work on a small team because each person can see how large their independent impact is.
Here are a couple of high-impact experiences I led and designed end-to-end:
Customer Use Case: Instacart Membership
Context
I noticed high cancellation rates for our Instacart membership program and wanted to find ways to save costs in membership refunds and decrease agent contacts.
Optimization
Due to low engineering resources, I made content-only changes to our membership workflow. I added additional information about membership benefits, gave customers as easy way to cancel, but did not offer the option for a refund until later in the flow so users don't immediately select that option.
Impact
Our self-service rate increased from 6% to 30%, meaning that more users went through this new flow. The changes to the flow also saved us an additional $1,500/week ($78,000/year) in customer experience costs by deflecting agent contacts, which equals to $0.05 in savings per global orders.
Shopper Use Case: Marqeta Card Declined
Context
Our highest reason for agent contact from our shoppers is when their Marqeta card declines at checkout. A Marqeta card is an Instacart-issued debit card that shoppers use to buy items. Sometimes the card gets declined at checkout for various reasons, causing shoppers to reach out, often stressed and frustrated.
Optimization
I shortened the flow and edited the content to be short and easy-to-understand for a more seamless and natural experience. I also worked with engineers to incorporate an API to authenticate the shopper's Marqeta card and introduced new variables to differentiate between shoppers' physical and digital cards.
Impact
Our self-service rate increased by ~4% and our agent cost savings increased by an additional $4922/month (~$60,000/year).
WAYS TO IMPROVE
Improve accessibility features (see above '“ACCESSIBILITY” section)
Make UI improvements
Make UI consistent across shopper and customer apps
Make UI of agent handoff page consistent with the rest of the chatbot experience
Include voice capabilities for a multimodal experience
Improve NLU capabilities to identify more utterances accurately
Increase the number of automations
Re-think priorities
Focus on optimizing and improving a few flows instead of focusing on covering the most use cases
Focus on added value the chatbot can bring rather than recreating experiences already in the app
Create more personalized flows that remember user information (name, returning or first-time user, past orders, etc.)
Introduce proactivity to suggest offers, help, or anything else to the user exactly when they need it