How Australian Home Lender Lendi Measures The Invisible Work Of AI
Most brands and businesses are trying to figure out how to improve their metrics when it comes to AI – and also what those metrics even are.
Problem is, measuring how chatbots shape brand discovery is mostly still a mystery.
But “you can’t improve what you can’t measure,” said Michael McCash, GM of growth at Australian home loan and property sales company Lendi Group.
McCash joined Lendi two years ago with a plan to sharpen the company’s more than $10 million in annual paid media, which has meant developing an in-house model for multi-touch attribution and bringing in Mutinex as its new marketing mix modeling (MMM) vendor. Lendi is also a pilot brand customer of Mutinex’s new AI measurement solution called AI Visibility.
Incorporating MMM has helped iron out wrinkles Lendi didn’t even know where there. For example, McCash said, the brand had been a major buyer of streaming TV and data-driven video on demand. Makes sense. For years, the industry has been betting on and investing in streaming as a major channel for growth.
But when Lendi dug in, what it found through its MMM reports is that old-school, free-to-air television had been “underweighted” and was actually a very efficient channel.
“We [started] diverting money from elsewhere in less efficient channels,” McCash said. “[And TV] just swallowed it up and kept increasing ROI.”
Media buyers often favor the channels they use themselves, he said, and many are young and consume a lot of streaming. Meanwhile, marketers trust the data they can easily see, meaning clicks, impressions, search term rates and other readily available metrics.
Which leads us to AI, the shiniest new marketing channel there is.
“With more people moving towards the Claudes and Geminis and ChatGPTs of the world, [the] initial discovery phase has become somewhat hidden again,” McCash said, like it was when Google and online search were new.
With little visibility into how AI platforms respond to prompts or surface brands, he said, many marketers are turning to information they find on forums like Reddit and X to try and piece together how people use LLMs and whether their brands show up. The platforms have made it “invisible or quite a hard task” to see how certain keywords or types of searches surface products, he added.
The Mutinex AI Visibility tool, like other AEO or GEO analytics offerings, generates insights by mass prompting LLMs with pertinent searches, like “find home loans in Western Australia,” for example, and then analyzes the results. Because there’s no reliable analytics dashboard for tracking those types of searches, the only option is to run thousands of prompts and observe the AI output.
Lendi also works with another vendor called Tracksuit that measures brand metrics and tracks how AI treats a brand compared to its competitive set. On top of that, Lendi partners with Askable for customer and mass market surveys. These tools combined help the company keep a finger on the pulse of its performance in new AI search and discovery markets.
But visibility is only part of the equation. Lendi is also testing whether advertising in AI chatbots is worth the investment.
According to McCash, the AI chatbots that include ads, such as ChatGPT and Google’s Gemini, haven’t been major ROI contributors, at least not yet.
“We’ve dipped our toes in the water,” McCash said of AI chatbot advertising. So far, though, Lendi’s MMM shows middling performance. The CPMs and customer acquisition rates are too high to justify much investment.
But what Lendi does see is a new type of customer, one who shows up to the Google search box or a Lendi-owned site having already done considerable homework on the brand. These potential home loan buyers are more knowledgeable and further down the funnel, because they’ve already done a bunch of research by prompting LLMs.
And so, for now, Lendi’s AI investment is focused less on buying chatbot ads and more on measurement and understanding how chatbots impact discovery.
McCash said he expects that the AI platforms themselves will provide more tools and help as they grow their own ad businesses, including providing data, insights and dashboards with anonymized information on the types of questions people ask and how brands show up.
Those insights are important, he said, because AI chatbots are now “the first port of call” for millions of people looking for a home or a loan. That type of data is critical for helping marketers understand how research through LLMs during the early stages can change the mindset of the customers who eventually reach them.
“All of that pre-work is being done early and in-platform now,” McCash said of AI chatbot users researching home loans. “And I think [getting access to] that knowledge is the next frontier” for marketers.