• LinkedIn
• Twitter
• Instagram
• YouTube
• RSS Feed

Apple •Spotify• Pocket Casts •Youtube •Overcast •RSS

What’s up everyone, today I have the pleasure of sitting down with Pranav Piyush, Co-Founder and CEO at Paramark.

Summary: Pranav guides us out of the labyrinth of multi-touch attribution under the clear sky of incrementality and causality, urging marketers to focus on whether their efforts genuinely drive sales that wouldn’t have happened otherwise. Early-stage startups can benefit by prioritizing simple methods like geo-based testing over complex attribution models, allowing intuition to guide resourceful experimentation. By understanding the underlying motivations and true causality behind customer actions, marketers can craft campaigns that resonate deeply and drive real results. As businesses grow, balancing intuition with structured analytics becomes crucial. Holdout tests and marketing mix modeling provide actionable insights, ensuring strategies remain effective in a competitive landscape. This approach transforms marketing from a cost into an investment in sustainable growth, making each dollar count.

Jump to a Section

• What’s More Valuable? Analytical Skills or Creative Taste?
• Understanding Incrementality in Marketing
• Multi-Touch Attribution Overlooks Causality
• The Limited Utility of Attribution for Early-Stage Startups
• Experimenting with Geo and Time-Based Tests
• When ROI Analysis Isn’t Necessary
• Embracing Holdout Tests in Marketing
• The Role of Marketing Mix Modeling in Measurement
• Balancing Career and Personal Life

About Pranav

• Pranav started his career at well known brands like PayPal and Dropbox
• He co-founded Padlet, the popular collaboration app to make school less boring
• He’s former Head of Growth at Magento and Pilot.com before becoming VP of Marketing at BILL
• He’s also a Reforge Instructor for a new marketing measurement course
• And in March of last year, Pranav co-founded Paramark to help marketers measure and forecast the impact of their investments

What’s More Valuable? Analytical Skills or Creative Taste?

Marketing’s creative nature often gets overshadowed by the obsession with data. Recently, HubSpot’s co-founder Brian Halligan suggested that marketers with good taste are undervalued compared to those with analytical skills.

Pranav agrees, arguing that creativity now drives the most significant impact in marketing. We often question the overuse of the term “data-driven” in marketing, suggesting a shift towards being more “creatively driven.” Pranav responds, arguing that data-driven and data-informed are all kind of bullshit. Relying solely on being “data-informed” is not sufficient. He emphasizes that without the ability to discern the success of a creative idea through data, creativity alone falls short.

Marketers face the challenge of making memorable impressions on people they’ve never met, and this requires innovation and creativity. While data is essential, Pranav notes that many marketers don’t truly understand the depth of analytical skills. True data literacy involves grasping complex concepts like correlation and causation, which are often missing in marketers’ education. 

Pranav points out that the dichotomy between creativity and analytics is overly simplistic. Marketers need to integrate both skills. This blend is crucial not only in marketing but in other business functions like product development. He uses the example of launching a feature and gauging its success. If only 10% of the customer base uses it, understanding the broader impact on adoption, revenue, and retention is essential.

Despite recognizing the importance of analytical skills, Pranav emphasizes that good taste in marketing offers a unique advantage. Creativity leads to building compelling campaigns that resonate more profoundly with audiences. This insight suggests that while data provides valuable insights, it is creativity that ultimately distinguishes successful marketing efforts. Pranav further highlights the importance of rigorous testing and measurement. A successful feature or campaign isn’t just about positive feedback; it needs to contribute to tangible business outcomes, such as increased revenue or cost savings. Without proper measurement, the value of creative initiatives remains unclear.

Key takeaway: To truly excel in marketing, you need to embrace a harmonious balance between analytical skills and creative taste. This means honing your ability to interpret data while also nurturing your creative instincts to craft memorable campaigns. Instead of relying solely on data or creativity, focus on integrating these skills. Use data to measure the success of your creative ideas, ensuring they lead to meaningful business outcomes like increased revenue or customer retention. By blending data literacy with creative insight, you’ll develop campaigns that resonate deeply and drive tangible results.

Back to the top ⬆️

Understanding Incrementality in Marketing

We often hear marketers claiming they understand ROI and reporting, yet the concept of incrementality often eludes them. Pranav sheds light on this by differentiating between attribution and incrementality. Attribution, as he explains, is rooted in the idea of cause and effect. However, its usage has been diluted over time, losing its original meaning.

Pranav appreciates our provided definition of incrementality: business results from marketing campaigns or channels that wouldn’t have occurred otherwise. He elaborates that if a prospect would have purchased a product without the influence of marketing, then that marketing effort isn’t incremental. Conversely, if a prospect’s decision to buy is directly influenced by marketing, then that effort is incremental. It’s like constructing a skyscraper floor by floor, with each level representing a marketing effort that elevates the building to new heights, transforming the skyline one piece at a time.

He emphasizes the importance of understanding incrementality beyond traditional marketing channels, especially in B2B contexts. This involves considering scaled sales channels, partner channels, and affiliate channels. The essence of incrementality lies in recognizing the true impact of marketing efforts on sales and other business outcomes.

Pranav’s insights underscore the need for marketers to move beyond surface-level metrics and understand the deeper implications of their strategies. By focusing on incrementality, they can more accurately measure the effectiveness of their campaigns and make informed decisions that drive real business growth.

Key takeaway: Focus on incrementality to truly gauge your marketing impact. Instead of just relying on attribution metrics, assess whether your efforts genuinely drive sales that wouldn’t have happened otherwise. By understanding and applying incrementality across all channels, you can refine your strategies and foster real business growth.

Back to the top ⬆️

Multi-Touch Attribution Overlooks Causality

Multi-touch attribution (MTA) often gets hailed as the holy grail of marketing measurement. Many believe it’s essential to solve attribution by capturing all touchpoints. However, Pranav argues that the obsession with MTA overlooks fundamental issues, particularly around causality.

When discussing attribution, we need to understand cause and effect. Pranav illustrates this with a simple example: if someone clicks on a Google link and converts, did that click cause the conversion? Sometimes it does, but other times it doesn’t. He emphasizes the need to ask, “What prompted the search in the first place?” Without knowing this, we aren’t truly understanding causality. We’re merely observing sequences of actions without grasping their underlying motivations.

Pranav criticizes the current approach to MTA, which often amounts to behavioral analytics. This method logs sequences like A led to B led to C, but it doesn’t clarify if A caused B. This lack of clarity is compounded by pressures on marketing and analytics teams to produce quick results, pushing them towards convenient but superficial solutions.

The martech industry, according to Pranav, has profited from building easy, superficial tools rather than delving into the complex but necessary task of understanding true causality. He believes this approach must change for the industry to advance meaningfully. By focusing on more robust methodologies, marketers can gain genuine insights into the effectiveness of their campaigns.

Key takeaway: Instead of fixating on capturing every touchpoint with multi-touch attribution, concentrate on understanding the underlying motivations behind each interaction. Challenge yourself to ask what prompts customer actions and seek to understand the true causality in your marketing efforts. By moving beyond mere sequences of actions and focusing on genuine cause-and-effect relationships, you can develop deeper insights and create more effective campaigns that drive meaningful results.

Back to the top ⬆️

The Limited Utility of Attribution for Early-Stage Startups

When discussing ROI reporting, the focus often shifts to justifying campaign value to higher-ups. However, for early-stage startups, this approach may be misaligned. Pranav shares insights on how startups should prioritize their efforts, emphasizing the limited utility of traditional attribution methods at this stage.

For startups, Pranav argues that focusing on attribution is less critical. Instead, founders should concentrate on scaling revenue from $100K to $1 million, then to $2 million. In such scenarios, a founder’s intuition about what’s working is more valuable than precise software-generated data. The primary goal is to identify and master a few key channels. When you have only three channels, the need for intricate attribution diminishes significantly.

As the business grows and significant funding is raised, a more structured approach to marketing spend becomes necessary. Pranav suggests a straightforward method: geo-based testing. By targeting specific geographic areas and comparing the results with non-targeted regions, startups can gauge the effectiveness of their campaigns without requiring sophisticated tools. This method is particularly useful for B2B companies, allowing them to focus efforts where their target accounts are most concentrated.

Pranav recommends running targeted ads in one region while monitoring organic visits and lead volumes from both targeted and non-targeted areas. This simple yet effective approach helps in understanding if the marketing spend is yielding the desired results. If there is no noticeable bump in the targeted region, it’s a clear signal to stop spending money on those campaigns.

The essence of Pranav’s advice is to embrace experimentation early. For startups, having historical data may not be possible, but running controlled tests, like geo-based ones, can provide actionable insights. These experiments help refine marketing strategies and ensure that limited resources are invested wisely.

Key takeaway: For early-stage startups, prioritize scaling revenue over complex attribution models. Rely on intuition and simple methods like geo-based testing to assess campaign effectiveness. By comparing targeted and non-targeted regions, you can identify successful strategies without sophisticated tools, allowing you to experiment and invest resources wisely.

Back to the top ⬆️

Experimenting with Geo and Time-Based Tests

When discussing marketing experiments, Pranav brings up the effectiveness of geo and time-based tests. These methods, while straightforward, come with potential limitations that marketers must consider. He delves into the nuances of both testing approaches and offers practical advice for early-stage startups.

Pranav explains that user-based tests, such as email or in-product experiments, offer more control. By splitting users into control and test groups, marketers can mitigate ambiguity, provided there’s a sufficient volume of users. For startups with a small user base, it’s essential to focus on growth rather than testing.

Geo testing, Pranav suggests, is valuable for channels beyond a marketer’s control, like Google or LinkedIn ads. Although geographic differences can pose challenges, analyzing traffic trends over six months can reveal consistent patterns across regions. For instance, traffic seasonality might be similar across states, debunking assumptions about regional variability.

Time-based tests involve intensifying efforts for short periods and observing the results. Pranav offers a LinkedIn content strategy as an example. Increasing post frequency or shifting content formats can provide insights into what drives engagement. However, time-based tests require significant changes to detect meaningful differences, making them less suitable for minor tweaks.

Key takeaway: Focus on Control and Test Groups for more precise insights if your startup has a sizable user base. For external channels like Google Ads, use geo testing to find consistent patterns across regions. Implement time-based tests by making significant changes to strategies, such as increasing content frequency, to gain meaningful insights. Prioritize growth over minor tweaks to drive impactful results.

Back to the top ⬆️

Does Everything Need to be a Test?

We often grapple with the question: does everything need to be tested? This debate is especially relevant in marketing, where testing can consume time and resources. Pranav offers a nuanced perspective, highlighting that while testing is valuable, it’s not always necessary.

Pranav references a meta-analysis from companies like Booking.com, Amazon, Uber, and Airbnb, which found that 70-80% of tests do not yield significant results. This statistic underscores that not every test will contribute incrementally to business outcomes. Accepting this failure rate as part of the learning process is crucial for fostering a positive testing culture.

He suggests that testing should always start with a clear hypothesis. For instance, if onboarding emails drive a 3% increase in trial conversions, the next step is to determine if enhancing these emails can push that number higher. The goal of the test—whether increasing conversions or raising feature awareness—dictates the approach and whether testing is worthwhile.

Pranav emphasizes the importance of prioritizing tests based on their potential impact. Not every test should be run, especially in environments with limited capacity. He provides a tactical example: when relaunching their website, they chose not to run an A/B test. Instead, they conducted extensive qualitative research, gathering feedback from advisors, customers, and marketing experts. The overwhelmingly positive feedback from respected figures like Rand Fishkin validated their approach without the need for statistical testing.

This approach highlights that qualitative insights can sometimes substitute for formal tests, particularly when the feedback comes from trusted sources. By balancing the need for quantitative data with qualitative insights, marketers can make informed decisions without overburdening their teams with unnecessary tests.

Key takeaway: Not every marketing initiative requires formal testing. Prioritize tests with clear hypotheses and significant potential impact, and consider qualitative feedback as a valuable tool for validation. This balanced approach ensures effective use of resources and actionable insights.

Back to the top ⬆️

When ROI Analysis Isn’t Necessary

Amazing companies, like Wistia and Ahrefs, intentionally avoid detailed ROI analysis and attribution discussions. Instead, they focus solely on revenue as their primary metric. Just revenue, not revenue attributed to certain efforts or campaigns. This countercultural approach raises an intriguing question: Is it always necessary to obsess over analytics and data when you’re already seeing success?

Like a moth circling a bright porch light on a summer night, one can become drawn irresistibly to the light of data, fluttering around it over and over, unable to break away. No matter what else happens around it, the moth’s entire focus is on that single light, representing the all-consuming nature of obsession, where one’s thoughts and energy are fixated on a singular point of interest.

Pranav acknowledges that this strategy can work, particularly for product-led growth (PLG) companies. He mentions that strong products and effective in-product marketing can sustain and grow a business without the need for granular attribution. Companies like Ahrefs maintain a visible presence through content, videos, and social media, relying on leading indicators like impressions and engagement to gauge their success.

However, Pranav points out that this approach has its limitations. It works well for companies at a certain scale, such as Ahrefs and Wistia, which are both significant players in their industries. But as these companies grow and face increased competition, they may need to adopt more traditional marketing strategies. The case of Tesla, which started advertising as competition in the electric vehicle market intensified, exemplifies this shift.

Pranav argues that while intuition and leading indicators are valuable, they don’t scale indefinitely. As companies expand and diversify their marketing efforts, relying solely on intuition becomes impractical. Larger teams and more complex campaigns necessitate a more structured approach to marketing and analytics.

Ultimately, Pranav believes that even companies currently thriving without detailed ROI analysis will eventually need to invest in more robust marketing strategies as they scale. The competitive landscape demands continuous adaptation and refinement of marketing approaches to maintain growth and market share.

Key takeaway: While relying on intuition and leading indicators can be effective for early success, especially in product-led growth companies, don’t neglect the need for structured marketing strategies as your company scales. As you grow and face more competition, begin incorporating more robust analytics to refine your marketing efforts. This ensures you can adapt to the evolving competitive landscape and sustain long-term growth without solely depending on intuition.

Back to the top ⬆️

Embracing Holdout Tests in Marketing

Holdout tests, though not commonly discussed among marketers, are essential for understanding the true impact of marketing efforts. Pranav dives into this topic, highlighting its importance and offering practical advice on how to implement these tests effectively.

Customer.io, our favorite email automation tool and sponsors of the show, includes holdout testing as a feature. This approach involves excluding a small percentage of your audience from receiving certain campaigns to measure the incremental impact accurately. Pranav emphasizes that holdout testing should be a standard practice, particularly for email and in-app communications, where you have control over the surface.

One common concern is the perceived opportunity cost—fear of losing out on potential conversions by excluding a segment of your audience. Pranav addresses this by suggesting that holdout tests don’t have to be permanent. Running a holdout for a few months can provide valuable insights into the incremental contribution of your campaigns. Once the hypothesis is proven or disproven, you can revert to full audience engagement and revisit holdout tests periodically.

Pranav also gives credit to Meta and Google for their advanced holdout testing capabilities. Meta’s conversion lift and Google’s solutions for YouTube and display ads offer true randomized controlled trials, setting a benchmark for other platforms. He expresses hope that in the coming years, more marketing platforms will integrate holdout testing features, making it a standard practice across the industry.

In summary, holdout testing allows marketers to measure the real impact of their campaigns, free from assumptions and biases. By systematically excluding a portion of the audience and comparing results, marketers can gain a clearer understanding of their efforts’ effectiveness.

Key takeaway: Implement holdout tests in your marketing strategy to accurately assess the true impact of your campaigns. By excluding a small portion of your audience temporarily, you can measure the incremental contribution of your efforts without biases. Don’t let fear of opportunity cost deter you; use short-term holdouts to gather insights, then revert to full engagement. Revisit these tests periodically to refine your strategies and ensure your marketing efforts are genuinely effective.

Back to the top ⬆️

The Role of Marketing Mix Modeling in Measurement

Marketing mix modeling (MMM) is an essential tool for understanding the broader impact of a company’s entire marketing strategy. When discussing the differences between MMM, multi-touch attribution (MTA), and incrementality testing, Pranav offers valuable insights into their unique focus areas and optimal usage scenarios.

Pranav explains that MTA, focuses on optimizing the bottom-of-the-funnel conversion journey. It captures the interactions that lead directly to conversions, providing insight into how different touchpoints contribute to final sales. This method is crucial for refining the steps that convert leads into customers, although Pranav prefers not to use the term “attribution” due to its broader connotations.

MMM, on the other hand, provides a correlational analysis of historical data, identifying which channels and strategies have the strongest association with business outcomes. This analysis helps companies understand the effectiveness of various marketing efforts and serves as a starting point for strategic planning. However, MMM alone does not establish causation, which is where incrementality testing comes into play.

Incrementality testing and other forms of experimentation, such as A/B and holdout tests, are designed to establish causality. These tests involve real-world experiments to determine the actual impact of specific marketing actions. Pranav highlights the symbiotic relationship between MMM and incrementality testing: MMM can inform the types of tests to run, while the results from these tests can refine MMM models by providing prior insights.

For early-stage companies, incrementality testing is more practical since it does not require extensive historical data. These companies can use experimentation to validate their marketing efforts and optimize their strategies based on real-world results. As companies grow and accumulate more data, MMM becomes increasingly valuable, allowing them to leverage historical insights to inform future strategies.

Pranav emphasizes that sophisticated companies should invest in both MMM and incrementality testing. By combining these approaches, businesses can gain a comprehensive understanding of their marketing effectiveness and continuously improve their strategies.

Key takeaway: Begin with incrementality testing to validate your marketing strategies through real-world results, especially if you’re just starting out. As you gather more data, integrate marketing mix modeling (MMM) to use historical insights for strategic planning. This combined approach helps you optimize your marketing efforts and enhance business growth by understanding what truly works.

Back to the top ⬆️

Does Email Require as Much Testing as Other Channels?

Email often gets sidelined in marketing experiments due to the misconception that it’s a free resource. Many enterprise teams believe that since sending emails doesn’t have an immediate cost, it doesn’t require as much testing as other channels. However, this perception overlooks the significant investments in email development, deliverability, and martech tools to avoid spam filters. Pranav stresses the importance of recognizing these costs and prioritizing email experiments.

Pranav highlights the undervaluation of organic and owned channels. Many marketers focus heavily on paid advertising, assuming it brings the most leverage. Yet, there are powerful examples of successful organic and earned media campaigns. For instance, the National Park Services’ social media team garners millions of views and likes with their engaging content. Their success exemplifies the potential of creative organic strategies.

Arby’s provided another example when Jon Stewart left The Daily Show. Their humorous tweet offering him a job was featured on his show, showcasing the impact of clever, timely content in earned media. Such instances illustrate that innovative, risk-taking approaches can yield significant returns, even in channels traditionally seen as less impactful.

Pranav emphasizes the need for creativity in email marketing. Sticking to generic, predictable formats leads to a “sea of sameness” where messages get lost. Instead, he advocates for bold, unconventional ideas that stand out and capture attention. This approach requires a cultural shift within marketing teams to embrace creativity and risk-taking.

Key takeaway: Don’t overlook email as a powerful channel for marketing experimentation. Recognize the investments in email development and prioritize creative and bold experiments to stand out from the “sea of sameness.” Shift your focus from over-relying on paid advertising to leveraging organic and owned channels, like email, by embracing innovative and risk-taking strategies that capture attention and drive engagement.

Back to the top ⬆️

Balancing Career and Personal Life

Managing the demands of being a co-founder, CEO, and active parent is no small feat. Pranav, who also juggles weightlifting and exploring foundational marketing techniques, offers an insightful perspective on maintaining balance amidst these challenges. When asked about finding happiness and success while managing a busy schedule, Pranav shared his personal strategies.

Pranav emphasizes the importance of engaging in activities that provide a mental break. For him, weightlifting serves as a challenging yet refreshing task that allows his brain to focus on something entirely different for an hour. This practice not only boosts his physical strength but also reinforces his belief that overcoming difficult tasks in one area can inspire confidence in others.

A philosophical approach also plays a significant role in Pranav’s life. Drawing from Buddhism and Stoicism, he views the pursuit of happiness as an ongoing journey. Embracing the notion that life is a mix of suffering and joy, he finds solace in small moments and practices self-compassion. Recently, he has incorporated meditation into his routine using the Waking Up app, which he highly recommends for its benefits in fostering mindfulness and presence.

Maintaining connections with friends and family is another cornerstone of Pranav’s approach. Being vulnerable and open with loved ones helps ground him and provides perspective on life’s inevitable challenges. He humorously acknowledges that despite his zen-like philosophy, he is not always calm and stable, emphasizing the human aspect of his journey.

When it comes to parenting, Pranav finds joy in teaching his daughters about subjects like math and science. He cherishes moments when they understand his work in branding and marketing, finding pride in their curiosity and growth. Balancing work and family time requires constant adjustment, but prioritizing these interactions enriches both his personal and professional life.

Key takeaway: Achieve balance by integrating mental breaks like exercise to boost confidence and resilience. Embrace a mindset that accepts life’s ups and downs, and practice mindfulness through meditation. Stay connected with loved ones to gain perspective, and prioritize quality family time to enrich your personal and professional life.

Back to the top ⬆️

Episode Recap

In marketing, understanding incrementality is key to real growth. It’s not just about knowing which channels drive sales, but about determining whether those sales would occur without your marketing efforts. This focus ensures every dollar spent truly contributes to business growth, allowing for more refined and effective strategies.

Early-stage startups, in particular, can benefit from this mindset. Rather than getting lost in complex attribution models, these businesses should rely on intuition and simple methods like geo-based testing. By comparing results across different regions, startups can identify successful strategies without the need for sophisticated tools, conserving resources for more experimentation and innovation.

The real insight comes from moving beyond multi-touch attribution to uncover the motivations behind customer actions. Instead of just mapping interactions, the focus should be on understanding why customers act. By digging deeper into these motivations, marketers can craft campaigns that resonate more deeply, driving meaningful results.

This is where marketing experiments come in. For startups with a sizable user base, using control and test groups can provide precise insights, especially for external channels like Google Ads. Techniques like geo testing and time-based tests, such as altering content frequency, reveal consistent patterns and provide significant insights. The goal is to drive growth with impactful changes, avoiding the trap of minor tweaks that yield little.

As businesses scale, balancing intuition with structured analytics becomes crucial. While early successes might rely on gut feelings, growth requires robust analytics to navigate competition. Implementing holdout tests can accurately measure campaign impacts, providing insights into incremental contributions without bias. Revisiting these tests ensures strategies remain effective and aligned with market dynamics.

Ultimately, the journey begins with incrementality testing to validate strategies through real-world results. As more data is gathered, integrating marketing mix modeling (MMM) helps leverage historical insights for strategic planning. This combined approach propels marketing efforts from mere expenditures to strategic investments, navigating the vast martech galaxy with sustainable growth as the destination. It ensures that strategies remain impactful and forward-thinking as they chart new courses through this ever-expanding universe.

Listen to the full episode ⬇️ or Back to the top ⬆️

Follow Pranav👇

• LinkedIn
• Twitter
• Paramark

✌️

—
Intro music by Wowa via Unminus
Cover art created with Midjourney (check out how)

Apple •Spotify• Pocket Casts •Youtube •Overcast •RSS

Related tags

See all episodes

Future-proofing the humans behind the tech

Apple •Pocket Casts•Google •Overcast •Spotify •Breaker •Castro •RSS

• LinkedIn
• Twitter
• Instagram
• YouTube
• RSS Feed