Is your marketing mix model (MMM) a waste of money?

If you don’t know how to build a model yourself, it’s almost impossible to spot a firm that’s just blowing smoke. But you don’t need to know how to build a model to know how to buy one.

Here are 5 brutal questions to ask an MMM firm before you sign the contract:

1. “How will you handle our uneven data precision?”

The Trap: Firms want to jump straight into modeling.

The Risk: Precise data (like digital) gets more credit than lagging data (like direct mail invoices that come weeks after the fact). The model biases toward digital just because the signal is cleaner.

The Fix: Demand they match spend to the exact date the media dropped before touching a line of code. Bad data in, bad model out.

2. “Can your model explain our biggest historic spend shifts?”

The Trap: A bad model can “predict” sales perfectly just by riding basic seasonality and trends.

The Test: Look at a period where you cut TV by 40% or turned off a channel. If their model can’t accurately replicate what happened to sales during that specific shock, it doesn’t understand cause and effect.

3. “Can you explain this to me in plain English?”

Red Flag A: They lead the pitch with R-squared or prediction accuracy. (MMM is a causal model, not a prediction model.)

Red Flag B: They don’t ask about your data quality upfront.

Red Flag C: They hide behind jargon or machine learning buzzwords that can’t isolate individual channel impacts.

4. “Will you hand over an interactive Excel tool?”

The Demand: Ask for an Excel file embedded with the final model parameters.

The Reason: You should be able to type in new spend numbers and see the predicted sales change yourself. If they say they can’t build this, they are building a black-box prediction model, not a causal model. Walk away.

5. “Are you willing to tie success to a live spend test?”

The Play: Tell them upfront that you will immediately execute their first major recommendation (e.g., shifting 20% from Channel A to Channel B).

The Reality: Real validation is about real-world revenue. Good firms won’t flinch at this. The ones blowing smoke will.

Experience is just the name we give our past bad judgment. Avoid my mistakes.