Grok Bot and Brevo: run a campaign experiment

A hypothetical trial campaign has 120 eligible contacts and two proposed subject lines. The trigger is the decision before launch. Extra clicks after two extra days are no win.

The team wants to know whether a practical subject line produces more completed trial activations than a curiosity-led one. The question is about activations, not opens. xAI’s Marketing Analyst example insists on sourced numbers and matching windows; use the same discipline when Grok Bot drafts a test plan. We have not run this Grok Bot and Brevo connection in a live account.

Freeze the question and the people

Write one sentence before building a campaign: “Among eligible, subscribed trial contacts, does subject B change completed activation within seven days of the cohort’s recorded scheduled start compared with subject A?” Hold the body, offer, URL and send time fixed. Record one scheduled start time for the whole cohort. Define the activation event in the product system, its contact identity key, and how delayed events are handled. If that source cannot be joined to Brevo recipients, the experiment can measure clicks, but it cannot answer the activation question.

Use the current subscription and suppression records, not CRM membership or a company domain, to define eligibility. Remove duplicates, bounces, unsubscribes, already activated contacts and contacts in a conflicting test. Freeze the eligible IDs and assign them once with a reproducible random method and an experiment ID. Save the assignment ledger before any send. Recheck consent and suppression immediately before sending. Log anyone newly ineligible as an unsent dropout in their original arm; never reassign them or silently remove them from the primary analysis.

That hypothetical group might assign 48 to A, 48 to B and 24 to an unsent holdout. Those are proposed groups, not results or a sample-size recommendation. Calculate the sample needed for your actual baseline and useful effect before launch. A small test may only prove that measurement works. See the A/B testing guide for planning the sample.

Record	Needed fields	Deterministic check
Eligibility	Contact ID, subscription source, suppression status, freeze time	Every assigned contact was eligible at freeze
Assignment	Experiment ID, contact ID, arm, assignment time	One arm per contact; holdout receives no test send
Outcome	Event ID, contact ID, event time, source	First qualifying activation per contact in the same cohort-anchored seven-day window for all arms

Configure a send that respects the review gate

Brevo’s native A/B campaign tests two subject lines or content versions on equal test samples, chooses a winner by opens or clicks, then automatically sends it to the remainder. That is useful only if the automatic rollout is approved before launch. It does not provide a pause for a human to inspect the first result.

For this experiment, map the frozen arms to explicit Brevo recipient lists or segments through an approved, verified operator. Create two unscheduled campaign drafts, one for A and one for B, with the same sender, body, destination and scheduled window. Leave the holdout out of both. Check each recipient list or segment against the ledger, recheck suppression, and make sure the mapping will not trigger an unrelated automation. Compare actual recipient counts and previews, then authorize each send. If Brevo cannot prove the mapping, stop before sending. The starter connection guide explains why a focused MCP server does not itself restrict permissions.

Predeclare a complaint threshold and stop pending sends if it is crossed. Preserve the assignment and any partial delivery; a stopped test is not a victory for whichever arm happened to send first. Keep unsubscribe and bounce counts alongside the primary outcome. Our double opt-in guide covers the subscription boundary.

Ask for a report, then decide the next experiment

Copy this prompt after supplying the frozen ledger and dated reports:

Analyze only experiment ID and sources I provide. Reconcile eligible contacts, A/B/holdout assignment, actual recipients, delivery, bounces, unsubscribes and complaints by arm. Use the single recorded scheduled cohort start as day zero for all A, B and holdout contacts; measure the same seven-day window and divide by originally assigned contacts in each arm. Deduplicate repeated event IDs, then count distinct contacts with their first qualifying activation at most once each from the named product event source. Keep unsent dropouts in their assigned arm and label delivery-only rates as secondary diagnostics. Keep missing events distinct from an unavailable source. Show numerators, original assigned denominators, unsent dropouts, protocol deviations and report retrieval times. State whether the planned sample and decision rule were met. Do not infer sales from clicks or use opens as a primary outcome. Do not reassign contacts, declare an early winner, create or send a campaign, or change a contact. Propose one next experiment as a draft, with its hypothesis and required approval.

Check: compare Brevo delivery and complaint totals with the campaign report at the same retrieval time. Document whether its Apple Mail Privacy Protection and bot-activity filters were applied; Brevo includes such activity by default. The email metrics guide explains why opens can mislead. Report test-phase and any later rollout separately.

A valid result names the eligible population, assignment, actual exposure, outcome source, window and uncertainty. If the predeclared decision rule is unmet, record “inconclusive” and draft the next test. Measure seven-day distinct activations per originally assigned contact in A, B and holdout from the same scheduled cohort start. Show delivery, complaints and unsent dropouts separately. Delivered-only rates are secondary diagnostics because delivery occurs after randomization. Without valid random assignment or an outcome source, a gap is descriptive, not causal. The decision must be reproducible from the ledger. Compare the separate product feedback follow-up loop.