Issue 01: Does Creative Testing Methodology Actually Even Matter?

The core of my argument is this: don’t build creative testing campaigns.

Instead, launch every new piece of ad creative in the same campaigns as your evergreen content using Bid Caps (or other Cost Controls).

Why? 

Because this leverage’s Meta’s machine learning to scale your winners and suppress your losers, which produces these three benefits (among others):

Jess is a very talented creative strategist. I think so highly of him that I interviewed him a while ago here (it's one of my most popular podcast episodes ever—you should listen).

Since his creative agency doesn’t always do the client’s media buying he’s seen his creative get tested lots of different ways. So he has a strong opinion. He's also a gracious and thoughtful interlocutor. 

But I still think he’s wrong.

I won’t relitigate the argument here. I responded to lots of his tweets directly, so you can get into the full conversation there if you’re so inclined. That’s not the point of this essay.

The point of this essay is the part that Jess and I agree on:

This conversation matters.

I wasn’t at all bothered by Jess responding to me so forcefully. Quite the opposite. Even though I think Jess is wrong (wildly, wildly wrong), I love that he thinks it’s important enough to put real time and effort into making his case.

That’s because he sees what I see: if high performing ads really are crucial to success on Meta Ads, then how you go about identifying winning (and losing) ads has a meaningful effect on your business’s top and bottom lines.

You’d think that idea wouldn’t be controversial amongst DTC marketers and media buyers. But apparently it is.

One of the responses I saw repeated was some version of, “Who really cares that much about this?” Or, similarly, “See: there are multiple good ways to do it.”

Here’s a tweet from David Herrmann to that effect. Here’s Cody Plofker (also a previous podcast guest here and here) at least partially agreeing with David. And here’s an agency operator named Nacho Vanzini to similar effect.

There are plenty more, as there always are. This response is a common part of the DTC-tactics conversation cycle. It goes like this:

• Person 1: This is the right way to do it.
• Person 2: No, that’s the wrong way to do it. This is the right way to do it.
• Person 3: See, there are multiple good ways to do it. Also this conversation is less important than this, this, and that. Stop talking about it.

Now the folks I’m quoting here are experienced Meta Ads practitioners and my point isn’t to pick on them specifically. So let’s try to hear the steelman version of their arguments. I’d state it like this:

While media buying tactics have some value, they’re drastically less important than the creative itself.

And look, this isn’t totally wrong. If you’ve ever gotten the impression from me that Bid Caps are going to completely change the performance of your creative: sorry. That’s not what I meant. Not remotely.

Here's what I'm actually saying:

Your Meta Ads spend is probably the largest or 2nd largest (behind inventory) line item on your DTC brand’s books. To suggest that how you leverage the tool to allocate that spend doesn’t really matter strikes me as deeply wrong if you really care about profit.

Let me put it more simply: 

Don’t let anyone convince you that how you manage 6 or 7-figure line items doesn’t really matter. It does. And you should do everything in your power to handle every part of them with excellence. Doing so has a direct impact on your profits almost by definition.

Saying, "Creative testing methodology isn't what matters; creative quality is" is a bit like saying to your Director of Supply Chain, "Your manufacturer isn't what matters; your raw materials supplier is."

The both matter! Of course they do! And so do your lead time, your payment terms, your MOQs, your per unit costs, and on and on and on. Great operators pursue excellence (expressed in profit-impact) in all of these areas and more.

Let me restate it one more time in as stark of terms as I can:

Don’t let anyone convince you that a $142,000 line item doesn’t matter.

It does. It definitely does.

Again, my point isn’t to hammer the guys I quoted above. They clearly agree that you should care a lot about profit. They all also invest plenty of time creating public content about media buying tactics and strategies.

My point is to highlight that operators and their teams have to fight the temptation to slip into a kind of malaise about what seem like the esoteric details of a business’s operations.

That's what actually led me to write this in the first place. The best operators are relentless about eliminating cost centers and expanding profit centers.

I’ve seen these operators and their businesses up close. They generate huge bottom line margins (both as raw numbers and as percentages), very often at earlier stages of business than most people think is possible in DTC.

I reflected on this at podcast length last week. The same themes are present in my conversation with Drew Fallon from two weeks before. I can’t stop thinking about it, talking about it with my clients, and seeing it lurking in the background (if you just know where to look) of public conversations about DTC.

So Jess and I agree: your creative testing methodology matters. You should think carefully about it. Go appreciate the time and effort he put into convincing you of his position.

Just don’t listen to his advice about how to actually do it.

----

If you want an even deeper dive on my creative testing methodology, start with this podcast episode and then go to this one..

If you want to transition your ad account to financially-driven media-buying that leverages Cost Controls (i.e. the stuff I’m talking about here and in those podcast episodes), there’s no better structured guide for how to do it than the courses, community, and coaching that make up Common Thread Collective’s ADmission program.

Because you're on my email list, you also get a special offer for a free coaching call for new ADmission members (on which, full disclosure, I have a small kickback) that you can get here.

If I was in that position, I’d take the offer, dive into the content, make the ad account changes, then use the free call with one of the super experienced media buyers from CTC/ADmission to pulse check the whole thing. It’ll pay off in droves.

----

One last thing, since this is my first newsletter…

You’re on my email list because you signed up for free resources from me or because of an Opening The Books episode. If you’re not into this (first ever) newsletter installment, no problem: you can always unsubscribe.

But if you’re still with me, would you do me a favor and tell me what you think? Was it helpful to you? Would you change anything? I’d be glad for the feedback.

Thanks a ton.

- Andrew