Hi, the current A/B auto optimizing is causing us issues.
I would love to request if either
A) We can have the option to turn this off, and optimize our A/B test results manually like it used to be
B) Or, make this feature better. Here’s two problems I’m seeing and have heard from other members as well
Issue #1 - It only optimizes for vanity metrics - open rate, reply rate. IT should be able to optimize for “Opportunities”
Issue #2 - It is not gathering a large enough sample size to accurately optimize A/B test results.
I have many campaigns where after a variant sends 20 emails, it is shut off and instantly is allocating volume to other variants.
This is nowhere near enough data to make these conclusions, and my copy is being wasted.
For an A/B test to be run effectively, we need statistical significance in the data.
We would typically wait for each variant to send 300+ emails each before making conclusions.
I would love if this can be adjusted, it’s really disrupting our workflows.
Please authenticate to join the conversation.
Completed
💡 Feature Request

AJ Cassata
Get notified by email when there are changes.
Completed
💡 Feature Request

AJ Cassata
Get notified by email when there are changes.