AI Copy Rotation at Lyft: Testing Message Fatigue
A randomized push experiment against static copy and no message
During his work at Lyft, Anthony Sledge tested rotating AI-assisted copy against a static daily push and a no-message control. The July 2024 record reports +0.44% driver hours for the rotating variant versus control, with a more sustained response than the static message.
At a glance
+0.44%
Driver hours vs. control
Rotating treatment; p=0.05
July 2024
Observation window
July 2–30
3
Experiment arms
No message, static, rotation
Capabilities used
The problem
Daily reminders repeated the same information. The team wanted to learn whether changing the wording could sustain engagement without changing the underlying offer.
The approach
The work and the decision behind it
A three-way split compared no communication, a single static daily message and randomly selected titles and bodies from a reviewed variant pool. The test ran July 2–30, 2024. Keeping the offer consistent helped isolate the communication approach.
Results and limitations
The reporting window and what it supports
The rotating variant recorded +0.44% driver hours versus control (p=0.05); the static variant recorded +0.22% (p=0.33). The retained account describes fatigue in the static treatment around week two and a more consistent response in the rotating treatment. Those separate comparisons with control do not establish a statistically significant difference between the two treatments. The recorded decision was to ship the rotating variant.
Why this matters
This is evidence for testing copy variety in a frequent lifecycle message. It does not establish that AI copy universally beats human copy: the experiment tested a rotation strategy, with AI helping create the variants.