[question] How to do A/B Testing?

[Blankeos]

Hi this is more of a question for learning... I read from here that OpenPanel can do "A/B Testing":

• https://openpanel.dev/docs#-analytics-platform-comparison
• https://openpanel.dev/compare/posthog-alternative
• Though it also says OpenPanel can't do A/B Testing here: https://openpanel.dev/compare/crazy-egg-alternative

Since I'm a little new to this, I think a comparison would be nice to look at. This is my little mental model of how A/B Testing works...

My assumptions on A/B Testing w/ 🟦 OpenPanel vs 🐗 PostHog

1. Campaign Setup - how to start an A/B test campaign? Rolling out strategy (setup feature flags? variables? rollout percentages?)
   • 🟦 I assume none
   • 🐗
     • Define 1 new Feature Flag i.e. key _ab_campaign and possible values (i.e. test | control).
     • Create a new Experiment w/ rollout settings i.e. filters (based on the .identify() traits) OR percentages".
2. Variant Assignment - when user loads in the app, how are they assigned as test or control.
   • 🟦 Manually, in code like generating a random value or checking identity. And also make sure to op.identify({ '_ab_campaign': 'test' }) or op.identify({ '_ab_campaign': 'control' })
   • 🐗 Automatic when using the SDK based on Experiment.
3. Variant Rendering - how to show what the user should see.
   • 🟦 Since (2) is manual, just reuse the code that assigns 'test' or 'control' and conditionally render.
   • 🐗 Conditionally render w/

       posthog.onFeatureFlags(() => { 
          if (posthog.getFeatureFlag("_ab_campaign") === 'control') {...} 
          else if (posthog.getFeatureFlag("_ab_campaign") === 'test') { ... } }
       )

4. Event emitting - the conversion event
   • 🟦 I assume op.track('_ab_campaign_success', { ... })
   • 🐗 I assume just posthog.capture("_ab_campaign_success", { ... })
5. Monitoring - how to see results of A/B test
   • 🟦 You can create a report: funnel
     • Then add 1 event & 1 breakdown:
       • event conversion event _ab_campaign_success
       • breakdown first is a profile property: _ab_campaign
   • 🐗 Handled automatically in the related Experiment.

I think that's how it would work, but let me know if I'm off. Would appreciate any redirection.