Evidenza: The future of AI-driven market research

Jon Busby: Welcome back to another episode of the Tech Marketing Podcast. I'm joined by Peter Weinberg, founder of Evidenza.ai.

Peter Weinberg: Co founder, but yes.

Jon Busby: I've got founder written here, so you're now founder.

Peter Weinberg: I've written, maybe I wrote them out of the story, but technically we do have two co founders in addition to myself.

Jon Busby: And you used to be at the LinkedIn B2B Institute as well before you left to found this. So what's the story there? What's the How did you end up leaving to found Evidenza?

Peter Weinberg: Yeah, it's a long sordid tale. Just kidding. It's not. It's not. I'm not sure it's long or sordid, but we were at LinkedIn for 10 years, me and my co founder, John, where we helped start the B2B Institute.

Worked with a lot of these world leading experts, Burnett and Thiel, the Ehrenberg Bass Institute, to really just try to discover new, more profitable approaches to B2B marketing. And it was fantastic. It was incredible. We loved it. That's why we stayed for a decade. But in our last quarter at LinkedIn, we saw something that just completely shocked us and we promptly quit our jobs to start the company.

That's here we are. Here we are. And am I allowed to know what that was? Oh yeah, of course. I guess that's the natural follow up question. So basically, if you want to get a company to implement These new approaches to marketing, like the Ehrenberg Bass way of marketing. There's a really key input, which is market research.

So we would work with, let's say, Salesforce. And we would say, OK, now we're going to do a survey of thousands of CRM buyers to try to measure your mental availability, understand all the buying situations for the category. So on and so forth. The problem with market research is it's very valuable if you know what you're doing, but it just does not scale very well.

So we were having a lot of trouble bringing these reports, these research reports and marketing strategies to clients. And then we discovered this new space that we're in now, which is called Synthetic Research. So the idea is this, instead of surveying, let's say, a thousand CEOs, you survey AI generated copies of those CEOs.

So you create a thousand synthetic CEOs, a synthetic sample, or a silicon sample. And you can then interview these people, you can interview a synthetic CMO, I've done that myself, you might want to have a synthetic CMO on your podcast someday. Or you can actually get them to take a quantitative survey.

And needless to say, it's about a thousand times faster than traditional market research. Things that used to take six months can now be done in six minutes. Needless to say, it's a lot more cost effective, because you don't have to somehow bribe CEOs to take a 60 question survey to the degree you ever could.

And then the question is really just accuracy. People are like, okay, I buy it. I'm sure it's faster. I'm sure it's cheaper, but is it as accurate? And that is what ultimately led to us quitting to LinkedIn, is we started to just compare traditional research results to synthetic research results.

And we found they were more or less identical. In some cases we found 95 percent match, 90 percent match. Since starting the company in January, we've run about 60 of these head to head tests. test where you just ask the synthetic sample what you ask the real sample, and then you just see how well they line up.

And it is startling to me every time that they line up almost perfectly. And so that's our business. We've actually been in stealth mode until this week. This is our launch week. So we've been working in secret for the past five months with a lot of the biggest B2B brands in the world to perfect this synthetic research platform.

And that is the tale. That is the tale of how we got here.

Jon Busby: And it's something we're also experimenting with it together. So we recently did a piece of research with the financial times where we What the Financial Times broke down the IT Decision maker into six personas that we call them things like a short alpha and so on, and we've been turning those into synthetic personas.

I love it. But the question we've always come back to is exactly what you've highlighted. That is accuracy. Have you addressed the accuracy challenge? Did, was it just always spot on, or have you found there to be a few bumps in the road up to now then?

Peter Weinberg: Yeah, I think right from the beginning we found very high match rates, but after every single test we can totally do to calibrate the model, tweak it, see what worked, what didn't work.

And this I think is the other very interesting thing about the space is that synthetic research is as bad as it will ever be, it's going to get better and better. It's going to get more and more accurate, whereas traditional research is, potentially even getting worse and worse as people learn to opt out of surveys and become harder and harder to reach.

So I'd say we started from a very high point of accuracy, certainly enough that we had the confidence to quit our wonderful jobs despite my wife being seven months pregnant at the time, which was a lot. Our confidence has grown as we've run more and more tests and more and more categories.

Jon Busby: So you've not just had a baby as a company, but you've also had a baby during the founding as well. Yeah,

Peter Weinberg: my, my grandmother always told me babies bring good luck. I think she's been right. It's, we've had a great start, but yes, I crazily quit my full time stable job with paternity leave with a seven month pregnant wife.

Now I have a. Wonderful seven week old girl, Lola. She probably watching right now. She loves marketing technology podcasts. She just loves it. That's how we put her to sleep every single night, not your podcast, but the more boring podcast sometimes play for her to put her to sleep. Yeah.

Jon Busby: That just, for me, that just shows the huge amount of confidence you have in this direction in what this could mean for the business.

Peter Weinberg: Yeah. People tell me they're like, that's crazy. Do you really believe that's real? I'm like, how much more could I believe it? I believe John and I have put our entire careers on the line. So yeah, we have, again, great confidence in where it's at today, but also an understanding that it's just going to continue to get better and better.

There are things that can't do well, but the things it can't do well are just like temporary obstacles as far as we're concerned.

Jon Busby: It's like the Will Smith eating spaghetti, right? As an example, right? A year ago, it looked. Wrong. And now it's just going to get better and better.

Peter Weinberg: Yeah, you see this.

So it's a great example of, there was some research. Part of the premise of this is that AI is able to impersonate people. That's the whole premise of what we're talking about here. So then there's been multiple tests now where they've seen how good there's 20 academic papers on how well they simulate human responses.

And if you look at the kind of charts with each model, They start strong and then they just get closer and closer. So the gap, which I think was already overcomable, as I said, it's just shrinking. They're getting more and more human like in their behaviour and their responses.

Jon Busby: Do you think we're, one thing that I've always talked about in marketing being a technologist is this idea of a Turing test.

Is there, there's a point where It'd be indistinguishable between a human and a, an AI bot. Now, I think we're fairly close to that now.

Peter Weinberg: I think we've blown right past it. I think the Turing test is completely in the rearview mirror. I don't know how you feel about it.

Jon Busby: Two years ago, I would have said, Oh, we're a few years away, and now I'm starting to agree with you.

It is, as I just try and think this through for a second, The challenge with any B2B market research has always been, as you say, scale. I'd also say it's cost. You could run a panel or a focus group every time you want to redo your brand, but that's going to be prohibitively expensive.

Exactly. And incredibly accurate.

Peter Weinberg: Exactly. And what you've invented here

Jon Busby: is essentially Focus group that you can run on tap whenever you want to.

Peter Weinberg: Customer on demand and you hit the really key point, which is we have 19 customers. 17 of them are B2B. We do have a big B2C customer Mars and another one also very big, but almost all of them are B2B.

And that's because the pain point, the market research pain point. It's a thousand times more acute in B2B. Trying to get a CIO to take a survey, whether we're talking about chief investment officer, chief, information security officer, it was just really not possible. To the degree it was possible, yeah, you had to spend hundreds of thousands of dollars and wait six months for results.

So our clients are saying, okay, finally we can really have the voice of the customer in B2B. Which is also interesting just because I think, there's always been this challenge, which we've written about a lot at the B2B Institute, where B2B marketers just don't always have as much authority or power as their B2C counterparts.

And a big reason why is sales, because in theory, marketing is the voice of the customer. In B2B, it's harder to argue that because you've got the sales force. talking to customers every day, so it gives them a kind of asymmetrical advantage over the marketers. Now where you have marketers able to talk 24/7 to synthetic customers in different segments representing different personas, you really have an ability for B2B marketers to act as that voice of the customer internally, which I think is just hugely exciting for B2B marketing. I really do.

Jon Busby: One thing we found on our AI journey, and as And even the debate when we've been looking at synthetic personas. So we kicked off a project a year ago now to try and turn survey responses into it. Very similar to probably what you've done. Not going to say we've developed it to as much as Evidenza, and I'm very excited to see what you've developed.

So there's going to be more players in this space. Like we're developing our own thing. Yes. You mentioned before there's a lot of space. to have, there's going to be a lot more providers in this market. Like, how do you think that's going to work over the next few years as this technology develops?

Peter Weinberg: Yeah, I think what you're going to find, first of all, with any technology, there's going to be a lot of skepticism. There's going to be some early adopters, who are mostly our clients, and then people who need a lot of work to accept the idea that a synthetic CFO will answer the same way as a human CFO.

So that's natural. Eventually, I believe all research will be synthetic research. So all the naysayers, all the critics, eventually, I believe the Evidence, no pun intended, will become overwhelming, and people will start to do more synthetic research. Everyone will start to do synthetic research. So then you have to game that out.

And what really happens, it becomes very interesting. It's already very interesting, which is, the technology is only as good as the marketers. The answers are only as good as the questions. So there's a very famous quote. This quote from Picasso, John and I love this quote, where he says, The problem with computers is they can only give you answers, right?

They don't know what to ask. For example, when we have a synthetic sample, John and I, because we're trained in the Ehrenberg Bass School, we ask very particular questions. We ask about category entry points. We ask about brand distinctiveness. We ask about mental availability. Other synthetic companies are either just going to be, ask Whatever you want, and we do a bit of that as well.

But they're going to look at other metrics. They might try to measure brand purpose. I don't think brand purpose is a good metric at all. But in theory, you could talk to a synthetic panel about that. So in a world where answers become commoditized, the real expertise of the real competitive edge is the ability to ask great questions.

And that's why ultimately, even though I do believe synthetic research will become a very crowded category, we're time. I think there will be space for different companies that have different approaches or ways of marketing, let's call it.

Jon Busby: And actually you've hit on the point I was going to try and make it, which is we found through all of our experimentation and research that with anything AI based, the value is when you go down to a very very niche use case.

You've mentioned there that you ask questions about mentor availability and, brand saliency and probably for certain elements. Is that the only use case for Evidenza or are you expecting it to expand over time?

Peter Weinberg: No, really our goal, and that's why, Jonathan Knowles brought us up, who has been a mentor of ours.

Our goal is actually to build a marketing finance interface. In other words, we want to build a brand. CFO friendly marketing plans powered by synthetic research. So the synthetic research angle is obviously very new and exciting and novel and what allows you to do it at incredible speed and at lower cost, but we believe the real opportunity is, again, What questions do you ask?

And so everything we do, and we're helping clients across the whole go to markets, we're helping them with segmentation, with positioning, creative testing, media planning, measurement, all aspects of a marketing plan. But crucially, we are financializing everything. So in other words, when we look at a segment, We are assigning it a financial value when we look at a category entry point, we are assigning it a financial value and we're working with our clients to build plans that say we're going to make X marketing decision because we think it's going to have Y commercial application.

And that I think forget about AI for a second. That has just been missing in marketing for far too long, especially in B2B marketing.

Jon Busby: Completely agree. And I think it's been one of the themes of this conference has been talking around understanding how the business makes money and then being able to, as a CMO, articulate that and put it into the CFO's language.

In fact, I was interviewing Dina earlier, who was Not just the CMO of Xerox, but the chief growth and disruption officer. And she talked entirely in financial metrics like EBITDAs, profit margins, like nothing. No, not, we're not, we're talking about reach and click through rates and all the other stuff.

Peter Weinberg: Yeah. But you don't see that often enough. Again, it's a difference in B2B and B2C and B2C. A lot of B2C firms, like Mars, or Procter Gamble, these are marketing led companies. The CMO often becomes the CEO. The marketers own a P& L. It's very different in B2B. In B2B, they often don't own a P& L.

Sales and product are the center of the organization. So they're less close to the money and the commercial. conversation. One of the first things I do, it used to both make me laugh and make me cry is like when clients would send me their marketing strategy, I just hit apple F and search for a dollar sign to see if there's a single dollar sign in their marketing strategy.

And there almost never was. And if there was, it was the last slide asking for a budget, right? It was not, here's the decision we're going to make and here's the commercial outcome. It was. Like a mood board of feelings and brand attributes and nonsense with a multimillion dollar budget ask at the end.

So this ability to be more financial, I think you have seen it as a theme in this conference, and it's the right theme. The question is, how do you do it? And that's actually where I think the real opportunity is with AI and synthetic research. It's to build more financial marketing plans, things that can be backed up by AI.

Jon Busby: One challenge when it comes to generating content or using artificial intelligence, especially in marketing, has been this concern about sea of same if we all start generating articles using chat, GPT, it all becomes very similar. And I think search, for instance, is in a lot of hot water at the moment over how they're going to approach this.

How, as more synthetic persona companies come to market, as you just mentioned earlier, More. You reckon all research we synthetic like how you what's your strategy for avoiding that sea of sameness in your own models?

Peter Weinberg: Yes I will say there is one part of the we are on the planning side not the execution side Because I think you're already seeing it's getting very crowded in AI Execution so one thing we are absolutely not doing is AI generated creative.

Not because we think it's a bad idea. In fact, there's already good research saying AI generated creative is better, in many cases, than human generated creative. But just because the category seems to be already quite crowded and well taken care of. The sea of sameness thing it's funny, I'll give you a super obnoxious contrarian take, but, I think it's important for people to keep in mind, it's we, at the B2B Institute, we did this research where we tested 5,000 B2B ads on a 1 to 5 effectiveness metric with system 1, so 5 star, very good, 1 star, bad.

77 percent of B2B ads scored a 1. Which means they are essentially terrible and are expected to generate no market share growth at all. So I think there is this kind of funny part where people are like, Oh, is creative gonna get bad because it's all AI generated? It pretty much couldn't be worse.

Even if AI just generates bad creative, at least it will be able to do it much faster. more cost efficiently than what we currently have. We are already drowning in a sea of sameness and it's humans who have chosen to make all of their brands blue and have the exact same value propositions and all look and sound the same.

It's humans who have done that. AI, actually, this has been probably the biggest surprise for me, is it actually has a very good understanding of distinctiveness. So what I mean by that is, If you talk to a synthetic sample and you're like, What are some colours that you rarely see in cloud computing when it comes to brands?

They'll tell you like, we rarely see hot pink brands. We rarely see mustard yellow brands. So in other words, AI can spot the pattern, it can tell you what that sameness looks like, and then it can identify the opportunities to be different, to be distinctive. So if anything, I think it could rescue us from the sea of sameness, because AI understands how things are the same and different, that's what it is.

It's the same. It's a statistical model looking at the similarities between things and next word prediction, etc. So I'm not worried about it at all because I don't think it could possibly be worse. And I think it has the potential to be better.

Jon Busby: Now that you put it that way, I disagree. It's cynical,

Peter Weinberg: but I think it's, I think the evidence would support it.

Jon Busby: Yeah. How many red and when I started in B2B, especially in tech, all of our brands, I think were red or gray. Yeah.

Peter Weinberg: Everything. It all looks the same. It's all terrible. Like you think an AI can't write like code. 11 percent better ROI, which is what like 90 percent of B2B ads say, yeah, it can do that.

Yeah, I think it's an opportunity actually. So you mentioned Evidenza mainly sits on the planning side. Yes.

Jon Busby: Like who are you expecting to be the main users of your platform? Is it because we talk about B2B marketers, it's a very general term. Yeah. Is it the strategists? Is it the people generating the campaigns?

What level are you expecting it to be utilized?

Peter Weinberg: I'm sure it will be used at all levels, and different companies will be set up to service different levels. We work with CMOs. Those are our clients. We work with CMOs because the CMO is the one who needs to build the plan that gets approval from finance.

Which is the angle that we're going for. So that's really who we're working at. We're working at the executive levels of these marketing organizations. I'm sure tools will emerge that are more like tactical, practical, day to day. And I think that'll be a very exciting space. I could imagine us joining planning and execution at some point to create a kind of flywheel.

But yeah, I think that's the opportunity is. Everyone's talking about AI in this super tactical way, everyone, the discourse around it in the marketing industry is inane, because everyone's just talking about creative generation, and it's a proper marketer knows that creative generation is a very small part of the job of marketing, it's like at most 2%, so where is all the conversation about how AI can inform strategy, not just execution around creativity, and that is the kind of, White space that we are trying to play in which is upstream of creative generation forget about is the AI generated creative good or bad How about is it on strategy?

Are you saying the right thing? Are you targeting the right audience? That's what the CMO needs to figure out

Jon Busby: I don't know if you were here for David's yesterday when he was talking about personalization

Peter Weinberg: Sadly, I was not

Jon Busby: he I mean we often say this it together as well one of the worst things you can do with AI is Yeah, like it's now such it's such a part of our day to day, whether you're generating content that you say, or looking to generate strategy.

So what would you do? What advice would you give a B2B marketer just to get started?

Peter Weinberg: Yeah personalization is a great example. I'll answer your question and then I'll come back to personalization. I believe the key thing is that marketers need to master the fundamentals of marketing. They need to know how to do marketing before they can figure out how to do marketing with AI.

Because as I said, AI is only as good as the marketer. It's a tool. Personalization. Everyone is now talking about how you can use AI to do personalization. I believe, and I've always believed, that personalization is one of the worst ideas in the history of the marketing industry. I don't believe that creative should be personalized.

I think the most effective creative is actually generalized. You find a generalizable need and you talk to a big audience, a big segment worth a lot of money. Disney does not make personalized movies, they make movies about lost animals trying to find their way home that appeal to all children in every country.

Jon Busby: You just basically upset every ABM marketeer.

Peter Weinberg: Of course, and I have for many years. But my point is, If people think personalization is a good idea, which I don't, but many people think that, then they're going to use AI to do personalization. And to be fair to all those people, just like we were saying with bad creative, at least now personalization will be cheaper and faster.

So in that sense the efficiency gain is good whether or not you think it's a good or bad idea. But I actually think the marketers who really win will be the ones who know that personalization is a bad idea, and actually what they need to use AI for is not to create a million little segments and a million little messages, but actually AI is able to aggregate segments together.

I'll talk about this a little tomorrow, but you can have an AI subdivide a market into a hundred niche personas or ICPs, but more impressively and more importantly, you can have it pull together a hundred ICPs across a hundred categories into one massive market. market and then figure out what the one biggest need is for that mass market and then design one creative execution for that one big segment.

So again, it all comes back to, do you have a way of marketing? Do you know how to do marketing correctly? And then do you use AI to do the right things?

Jon Busby: You've made a lot of ABM marketers probably quite sweat. I always do. They never liked me message, but like that is a powerful point of you.

But let's talk about some of the clients you have. You may not be able to talk about specifics, but you've mentioned Mars a few times. What successes have you seen already whilst you've been in stealth mode? Is there a single point where you've looked at something and gone this is just going to keep going?

What successes or case studies can you really talk about? Yeah, listen, we have 19 customers. A lot of them are on our website. They're listening. That I could tell you them now, like EY, ServiceNow ZoomInfo, very big companies, and almost all of our customers have renewed, about 90 percent of them have come back and said they like the first report, they want a lot more of it on subscription.

Peter Weinberg: So I consider that success for our business, but success for customers, that's where you get success as a business and I think it's in all those areas I was talking about, like ServiceNow is a great example of what we're just talking about, where we worked with ServiceNow on segmentation and we worked on it in both directions yes, we can show all the different segments that buy ServiceNow, which is a very complicated business that sits a lot of cross a lot of categories, but because it's a very good marketing team, They also want to know, how can we simplify things?

How can we use AI to actually collapse segments, so that we can design bigger messages for bigger segments? We've got clients who, we've helped them identify what their key category entry points are, which has brought, Again, not complexity to their marketing, but actually simplicity.

Where, sure, we can show you the 79 different messages that could drive commercial value, but critically, we can show you the top 5, and then say, forget about the other 70, focus on these. Put all of your resources towards these to get better commercial outcomes. We really are working with clients across this portfolio, and again, it's a contrarian take, but I think it's actually, I'm impressed at the ability of AI to simplify things, not to make things more complicated, but people who are in the complexity business, and there are a lot of them in the marketing industry, that's how they will use it.

And, I wish them the best, of course, but, we're trying to do something a little different.

Jon Busby: This concept of just one master campaign, whatever you want to call it, I think Is a powerful, quite different stance toe. How many people are looking to use it? Extremely different. We've been talking around first part, the use of first party data and how you can utilize it to create your custom banner ads down to the one to one level.

And maybe on the e commerce side, if we're talking about B2C, that works well. But in the use of first party data is extremely important. You're right, in B2B, it is about trying to create a simplified message that cuts through.

Peter Weinberg: Yeah, listen, if you want a nice answer that brings everyone together, is there are different types of marketing and there's different types of objectives.

Les Bonnett and Peter Field, I think, who are great mentors of mine, would tell you brand messaging, top of funnel brand marketing, is about that, five big messages or one big message for that one master. market. As you go lower down the funnel, there are more opportunities to have more specific messaging for specific segments.

Aaron Berg Bass, I'm not sure, really does believe that. I think they truly believe in mass marketing and just going after the five biggest messages. But there's room for many different ways of marketing, and I do believe, as we've been saying, AI is an enormous efficiency gain no matter what you believe.

The question The effectiveness gain will be different for different marketing organizations, depending on how well they've chosen their strategy.

Jon Busby: So of those 19 customers, is there, and you mentioned ServiceNow, so I'm relatively familiar with the ServiceNow pillars. We also do stuff with them at the moment.

Great brand. Fantastic brand. Is there, have you got to the point of that simplifying their message into one master campaign yet? Or?

Peter Weinberg: Yeah, we've helped inform. I don't want to speak in too much detail about client work, but I can tell you we've been working with them closely on segmentation and messaging.

And when we had our launch this week in ad week, Colin Fleming, the CMO of ServiceNow happily went on record to say, How much value he was getting out of the Evidenza product. So I like to think that we have we have helped there. But again, that's a very good marketing organization. So when you give them AI, they know how to do good marketing better.

Things are going to be different depending on how well trained the marketing organization is, whether they know what they should be doing in the first place.

Jon Busby: Let's dig into that just for a second then. So is there a risk? If they're not a mature marketing organization for using AI in the wrong direction or using your product in the wrong direction.

Peter Weinberg: Yeah, I think there is. I do. I think the risk is because let's stick with synthetic research. Because it is infinitely granular, you can quickly start to make things really complicated. Let's put it this way. Generating all of this market research data used to have a cost associated with it, an extremely high cost.

So people were very selective about what they asked. Now, because synthetic research is going to democratize that, ostensibly that's a great thing, because now more companies of more sizes will have access to market research. Big companies can ask more questions. That's all good. But there is a risk, and the risk is that if it becomes so easy to generate customer data, And insights, all of a sudden you could end up drowning in a, you talk about, we've been talking about big data for the past 20 years this is big data.

This is exponential. This is infinite data. So I do think there is a risk that, if you don't really know what you're trying to do, You can end up really increasing your complexity. And although people in marketing often don't talk about it enough, complexity has a cost, more complexity generally increases your cost in all kinds of ways.

It's not, you look smart by saying complicated things, but, genius is the ability to simplify. And I think that's going to become a real really important factor in marketing. A. I. marketing effectiveness,

Jon Busby: Peter. It has been a fantastic conversation. I wish you all the best of luck.

Peter Weinberg: Thank you so much

Jon Busby: Now you're out of stealth mode. check evidenza out.

Peter Weinberg: Please do. Please do.

Jon Busby: Is there anything you want to say to our listeners? All of course being aspiring tech CMOs?

Peter Weinberg: No, I would just again, I'd say make sure you've mastered the fundamental ways of marketing. Make sure you have a way of marketing before you look into having an AI way of marketing. That I think is the big. Opportunity ahead of all of us.

Jon Busby: We have one question I like to ask all of our guests. Great, what is it?

Peter Weinberg: Is it a fun one? Like my last meal or something?

Jon Busby: No, I wish. No it's something we ask all of our guests. We asked the same question last year as well.

So in five words or less, If you were to give one piece of advice to a future CMO, what would it be?

Peter Weinberg: Five words or less. Learn the principles of marketing.

Jon Busby: There we go.

Peter Weinberg: Can we do it? That's what I like.

Jon Busby: Perfect. Thanks very much, Peter. It's been a pleasure having you on the podcast.

Peter Weinberg: Thank you so much for having me.