Why we won't give you an attendance ROI number

Every tool in this category eventually promises a number: your community was worth this much. We looked hard at whether we could produce one honestly for a single host. We cannot, and the reason is arithmetic rather than effort.

What a causal number would require

To claim that attending caused someone to buy, you need two groups that differ only in whether they attended, and enough people in each to see an effect through the noise.

Run that calculation at the size of the rooms we sell into. With about 50 people per group, the smallest effect you could reliably detect is roughly 27 percentage points. An effect that large would be visible without measuring it.

Go the other way and ask what it would take to detect a 10-point effect through randomised invitations, at a realistic take-up rate. You need somewhere near 820 attendees. A single studio's recurring format does not produce that in a year.

The comparison that looks rigorous and is not

The obvious workaround is to compare people who were invited and came against people who were invited and did not.

It reads like a control group. It is not one. The people who came are the people who were already more inclined — more free that evening, more attached, more likely to buy anyway. The comparison is descriptive, and it is biased in the direction of the claim we would be selling.

That is the same failure the loyalty-programme literature already documented: Leenheer and colleagues (2007) found about 86% of the raw effect disappears once self-selection is corrected for.

If we shipped that comparison as ROI, we would be selling you your own selection bias with our logo on it.

What we will do instead

Three things, none of which require pretending.

A coverage-rated report. What we can see is who came and how often. We will show that against your own records, and we will state plainly what share of the room we could actually identify. A number without its denominator is a testimonial.

One real test, when you want it. A perk given to a randomly chosen half of an eligible group is a genuine experiment, because we assign it rather than the person self-selecting. It measures one small thing properly instead of one big thing badly.

A labelled comparison, never an unlabelled one. Anything that is an association gets called an association, in the report, in the same size type as the result.

The part that only works later

Pooling randomised tests across many hosts eventually produces something a single host never can. That is a benchmark, and it would be presented as one — never as your number.

We are not there. We have no pilots running and no results to show. When we do, the prediction goes up before the result, and the misses go up with the hits.