2.First party customer databases – Historically, respondents for market research studies
were recruited only through third party panel providers. In an attempt to get better
respondent feedback while simultaneously also reducing panel costs, most large brands
are now building first party customer databases of their own. By engaging customers
(past or present) in short, intuitive surveys, brand teams can get higher quality feedback
and insights from surveys.
3.Behavioral Science – Insights teams are replacing “stated” research techniques with
behavioral science inspired methodologies to get more human insights from research.
Behavioral science is a 3-time Nobel Prize winning field of research that has shed light
on how humans make decisions using mental shortcuts like heuristics and biases.
Insights teams have embraced the use of behavioral science in research and are now
accepting the fact that customers don’t DO what they SAY and often they don’t even say
what they THINK or FEEL. Behavioral science helps bridge the gaps between
FEEL/THINK/SAY/DO and helps brands teams get deeper, more actionable insights
from every research project.
4.Data reporting and visualization – For decades, survey findings from market research
projects were delivered through Powerpoint decks filled with boring bar charts and cross-
tables. Innovation in data visualization technologies has made a big impact on how
insights from market research studies are reported and shared within brand teams.
Real-time reporting of research data is also changing how brand teams act upon
research findings. In the past, teams had to at least wait for the preliminary read out
presentation, which took weeks to prepare from the last day of fieldwork. Today, real-
time reporting of survey data through online dashboards makes it possible for insights
teams to intervene early in research projects and get the right questions in the survey
based on early data.
Lack of innovation in message testing research
Despite all the innovation seen in market research over the past decade, one type of market
research that is still operating in the old model is message testing research. There has been
little methodological innovation in message testing and standard survey methodologies
conventionally used for message testing still don’t meet some basic needs of brand teams:
- Can’t test a lot - Researchers still cannot test a lot of messages in one survey. It limits
the ability of brand teams from exploring a wide range of messaging ideas and platforms
or researchers have to conduct rounds of iterative research to test a large number of
- Sea of sameness – Conventional message testing methodologies often produce similar
scores across all messages because they are not sensitive enough to pick up nuanced
differences in messages. Marketing teams struggle to make campaign decisions if there
isn’t enough separation between Good/Better/Best message scores from surveys and
end up making decisions based on their judgment more than research.
- Message drivers of appeal – There is still no way for researchers to understand the
driver of appeal for a message without asking stated diagnostic questions like, “What do
you like/not like about this message?”
- Campaign readiness – The main deliverable of most conventional message testing
methodologies is a rank order/hierarchy of messages, followed by a TURF type analysis
to estimate the optimal number of messages to use. In order to be campaign ready,
marketing teams need message testing research to ideally deliver a segment-level,
channel specific message map, which is not what they are getting from research today.
With increasing use of digital marketing channels, message testing has also been shifting out of
market research/insights teams to marketing teams because they are conducting in-market A/B
testing as part of their marketing campaigns. While A/B testing is easy to conduct and uses real-
world results, data shows that 80% of A/B tests don't produce any results and are canceled
before they reach statistical significance.
A/B testing has many of the same constraints that market research-based message testing has.
Most brand teams can only afford (time and money) to test a few (6-8) messages in A/B tests.
A/B tests don’t explain why one option is better than the other and attribution analysis is difficult
if the A/B tests are run on bundles of messages instead of single messages like email subject
Future of message testing: Decision heuristics science + AI
Message testing research needs a reboot. Currently, it is not helping insights teams deliver what
their marketing partners really need to execute winning messaging campaigns:
- What insights teams deliver - message hierarchy and TURF analysis
- What marketing teams need – Optimal messages bundles and storyflow, personalized
for every segment and channel.
An ideal message testing system would require a fundamentally different methodological
approach that uses science and algorithms to test more messages in market research and
produces deliverables that directly lead to messaging campaigns.
In the past few years, latest advances in decision heuristics science and AI have created new
opportunities to build a better message testing system. Decision heuristics science allows
messaging choices to be set up as behavioral experiments in research. Artificial intelligence
offers the ability to do more with research data and translate it into winning messaging
Decision heuristics science can transform how messages are tested in surveys with
- Test 100s of messages in one survey so that we can improve our chances of finding
winning messaging campaigns
- Get better data on each message so that we can find better message bundles out of
billions of possibilities
- Get true message appeal without asking so that we can create even more successful
messaging campaigns in the future
Artificial Intelligence can transform how data from message testing surveys is used to drive
- Be more campaign ready straight out of the research by identifying optimal message
bundles/storyflow for every channel
- Reduce marketing waste by personalizing messaging campaigns for each customer
segment based on their decision heuristics
- Train predictive models so that we can refresh campaigns more often without more
CMO (Choose Message Optimizer) from Newristics combines the power of decision heuristics
science and AI to test messages in a better way. CMO has been proven superior in industries
with the most complex messaging needs, like the pharmaceutical industry. In a meta-analysis of
50+ studies in the pharmaceutical industry, CMO was consistently proven to be superior:
- 100% success rate – CMO-identified message bundles outperformed
benchmarks/controls in ALL studies
- 1.7x improvement – CMO-identified message bundles had 1.7x times higher preference
share than ALL benchmarks
- Competitive position – CMO-identified message bundles helped 7 out of 10 brands beat
the market leader or extend their leadership over the nearest competitor.
Transforming message testing in banking industry
To demonstrate the value of innovation in message testing in the banking industry, Newristics
conducted a large-scale, head-to-head, challenger type study using control messages from the
#1 bank in the country - JP Morgan Chase. The study was designed to be the ultimate
Can a different approach to developing and testing messages outperform the messaging
efficacy of the country's largest bank...without the help of their marketing department?
- Collect marketing messages for a leading retail bank (50 messages selected for Chase)
- Create alternative versions of Chase messages using decision heuristics science (350
new messages developed)
- Test >400 messages with consumers (N=1,000) in a large-scale message testing survey
using Choose Message Optimizer methodology.
- Use AI to analyze survey data and identify winning message bundles from >2.5 trillion
- Compare CMO-generated optimal message bundles to Chase in-market campaigns
using Preference Share metric
- Using decision heuristics science and AI in message testing resulted in 100% success
rate – every message bundle generated by CMO was superior to the in-market
messaging campaign used by Chase.
- New message bundles identified through CMO had 30%-50% higher preference share
than Chase in-market messaging across all campaign types.
So, how was the bank messaging improved by 30-50%?
- Increased and Improved Messaging Ideas - 350 new messages developed in 5 days
using decision heuristics science
- Turbo Message Testing - 400+ messages tested in only one survey with only 1,000
banking customers using a methodology powered by decision heuristics science
- Optimal Channel & Campaign Execution - AI successfully identified winning message
bundles for each channel out of 2.5 trillion possibilities.
- Actionable Messaging Insights - Decision heuristics science identified precise
language drivers of message appeal for different audience segments (e.g. Gen X vs Gen
Since customer acquisition in banking is driven mostly by trigger events (e.g. moving to a new city), banks need an always-on messaging model so that they can be in the consideration set whenever a consumer is ready to switch. Being always-on is an expensive value proposition in messaging and can be cost prohibitive unless the messaging playbook is optimized for segment-level and channel-level messaging.
Retail banks have the opportunity to make their messaging campaigns significantly more effective by utilizing decision heuristics science + AI in the development and testing of messaging before the campaign launch. Download the full study report here and get up to speed on the latest scientific and algorithmic advances in messaging optimization.
Newristics is famous for helping brands optimize messaging using a combination of behavioral
science and machine learning algorithms. In the past ten years, Newristics has optimized
messaging for 100s of world-leading brands generating $100s billion in revenue every year.