Lifecycle Email Testing for Lyft Driver Onboarding
Checking whether an earnings email confused new drivers before cutting it
TL;DR
- Problem
- A day-six earnings email might have been confusing new drivers who were still learning how pay worked.
- What we did
- We designed a four-week holdout test comparing drivers who got the email with drivers who didn't, tracking support activity alongside driving.
- Result
- A test with a decision agreed in advance for either outcome; the final readout isn't in the available records.
- Industry
- Rideshare
- Timeline
- Four-week test, 2022
- Engagement
- Prior in-house work at Lyft
- Tools & channels
- Lifecycle emailHoldout test
An earnings email might be confusing new drivers
New drivers received an earnings email six days after joining Lyft. The team suspected the numbers could confuse people still learning how earnings worked, but the email went to every new driver, so removing it on a hunch was risky.
Test the email before changing the onboarding flow
We split new drivers into two equal groups: one received the email, the other didn’t. The plan tracked help-page visits and support tickets as signs of confusion, alongside driver hours, rides and earnings to catch any harm to activity.
What each group received and what we measured
| Group | Day-six earnings email | Measures |
|---|---|---|
| Holdout | Not sent | Help-page visits, support tickets, driver hours, rides, earnings |
| Treatment | Sent | The same measures, compared with the holdout |
A decision agreed before the data came in
The value was in deciding the next step before seeing the data. If dropping the email caused no harm, early earnings messages would stop and new education would be built. If it did cause harm, the messages would be filtered and rewritten. That kept a high-volume change from resting on opinion.
How to read these numbers
This case documents the test design. The final results are not in the available records, so we don't claim the email was removed or that onboarding improved.
Services used:First-use onboarding
