{"id":10330,"date":"2026-09-14T01:14:27","date_gmt":"2026-09-14T08:14:27","guid":{"rendered":"https:\/\/www.kaushik.net\/avinash\/?p=10330"},"modified":"2026-09-11T17:10:52","modified_gmt":"2026-09-12T00:10:52","slug":"googles-statistics-significance-replication-risk","status":"publish","type":"post","link":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/","title":{"rendered":"Google’s Statistics: Heads They Win, Tails You Lose."},"content":{"rendered":"

When Google reports lift results\u2026 Who is it solving for?<\/p>\n

Let’s start with a phrase that causes me to cringe every time I hear it: “directional results.”<\/p>\n

It sounds sophisticated. Nuanced. Even reasonable. But what does it actually mean?<\/p>\n

Answer: Just about anything you want it to mean!<\/p>\n

I was reminded of this while reading Google’s official guidance<\/a> on how to interpret Google\/YouTube’s Brand Lift, Search Lift, and Conversion Lift studies.<\/p>\n

\"Google's<\/center><\/p>\n

It made me pause. Then sad. Read the last one carefully.<\/p>\n

Google says: \u226590% is “very good chance.” 70%-90% is “good chance.” 50%-70% is “moderate chance.” <50% is \"no lift.\"\n\nAcross three layers the sentiment emphasized is, paraphrasing: Your ads probably <\/strong>did something good.<\/em><\/p>\n

The last layer is, paraphrasing: Hey, the ads might not have sucked, spend some more and let’s see what happens.<\/em><\/p>\n

Google also says in the article: Studies at 50%+ can provide valuable, directional insights, and explicitly tells advertisers in the 50%-70% band to use the results directionally.<\/p>\n

If you pause and reflect on this carefully caveated article, you’ll see this is a case of: Heads Google wins, tails you lose.<\/strong><\/em><\/p>\n

This post was first published as TMAI Premium #523<\/strong>. My weekly newsletter cuts through the noise: Strategic frameworks, actionable advice, zero fluff – built on decades of doing, not just advising. For serious Marketing and Analytics professionals. Subscribe today.<\/a><\/div>\n

 
\nWhose Success Matters More?<\/strong><\/span><\/p>\n

To bring this lesson home, let’s assume I ran a campaign on YouTube.<\/p>\n

\n
  • Investment: $5 million<\/strong>\n
  • Brand KPI: Consideration<\/strong>\n
  • YouTube-reported BLS result: +3-point lift<\/strong>\n
  • YouTube-reported Certainty: 70%<\/strong>\n<\/div>\n

     
    \nGoogle calls 70%-90% Certainty a “good chance.<\/em>” It also says studies at 50%+ can provide “valuable, directional insights.<\/em>”<\/p>\n

    What. The. Heck. Does. Directionally<\/em> mean?<\/p>\n

    \n
  • Do I spend another $5 million on another YouTube campaign?\n
  • Do I spend less and do something different, in a different place?\n
  • Do I change the creative or media tactics?\n
  • Do I shift the money to Meta that presented a new exciting idea yesterday?\n
  • Do I run another “test” (and to test what)?\n<\/div>\n

     
    \nFrom Google’s view, the interpretation is that there is a “good chance the results were caused by your ads.<\/em>” But as a Senior Director of Brand Media, is a “good chance” good enough for me to report to my CFO and ask for $5 mil more? <\/p>\n

    Two Parties. Two VERY Different Consequences.<\/strong><\/span><\/p>\n

    Party One: Google:<\/strong> It sells advertising. If a study contains weak evidence that advertising could possibly have worked, Google has every incentive not to throw it away. Perhaps, and that’s the operative word, perhaps there is something <\/em>useful to learn. Fair enough.<\/p>\n

    Party Two:<\/strong> You: You own the Marketing budget. Your job is not to find something interesting in the data.<\/em> Your job is to answer: IS the evidence in YouTube’s BLS strong enough that I should risk another $5 million of my company’s money on the conclusion<\/strong> that the first $5 million worked?<\/p>\n

    Here, I made a little table to outline incentives and implications: <\/p>\n

    \"Google<\/center><\/p>\n

    It is impossible to know if Google’s wording on its sites, in its client decks, is intentionally self-serving. It does not matter. The outcome is self-serving.<\/strong><\/p>\n

    Google does not<\/strong> bear the consequences of an optimistic interpretation. Your company does.<\/strong> Google also, to their credit, has a drive-by line that says: Guys, interpret results based on your business needs and risk tolerance.<\/em><\/p>\n

    Since I am bearing the risk, for consequential business decisions I use 90% Certainty as the floor<\/u>. It isn’t magic; it is simply the minimum level of evidence I’m willing to accept before telling a CMO or CFO: “This worked. Spend again.<\/em>”<\/p>\n

    [<\/font><\/strong>Premium Subscribers: Refer to TMAI #298: Smarter Statistical Significance Reporting.<\/em> It includes a one-pager to standardize stat sig reporting across your team, agency, vendors. If you can’t find it, please email me.]<\/font><\/strong><\/p>\n

    Google’s emphasis is on “moderate chance,” “good chance,” “directionally.” These are comforting, decision-sounding words that obfuscate evidence more than illuminate it. The language is for sure carefully caveated, but it ignores the immense risk and cost it intentionally or unintentionally shifts onto Google’s clients.<\/p>\n

    Let me prove it to you.<\/p>\n

    CMO, Meet Replication Risk.<\/strong><\/span><\/p>\n

    Rather than discuss p-values associated with “good chance,” “moderate chance,” “use these results directionally,” let’s focus on the question your CMO and CFO care about:<\/p>\n

    If we ran this campaign again with another $5m budget, how likely is it that we will get a result strong enough to confidently say: “Round two worked. Spend again.”<\/strong><\/p>\n

    You’ll recall YouTube reported a +3-point lift in Consideration at 70% Certainty.<\/p>\n

    While they might not use the technical phrase, your CMO and CFO are essentially asking: What’s the Replication Risk?<\/em><\/em><\/p>\n

    [<\/strong>For this thought experiment, we’ll assume, the second $5m will face the same underlying conditions as the first $5m. Same platform (doh!), audience, seasonal factors, promotions, creative, sample size, etc.]<\/strong><\/p>\n

    To compute Replication Risk, I’ll use a simple Bayesian replication model inspired by biostatistician Steven Goodman<\/a>. It helps translate Google’s abstract “Certainty” into a much more practical question: What might happen if we measure the same thing again?<\/em><\/p>\n

    YouTube’s BLS estimated a +3-point lift in Consideration. But +3 is an estimate; rather than pretending it is precise, the Bayesian approach carries that uncertainty forward into the next measurement.<\/p>\n

    As business owners, it is our money on the line. We are applying this simple logic: <\/p>\n

    (Uncertainty from what the true effect might be)
    \ncombined with
    \n(Uncertainty from measuring it again)
    \n= How confident should we be that we will see similar results?<\/p>\n

    Let’s do some math!<\/p>\n

    Three computations for the Business Owner lens: <\/p>\n

    \n

    1.<\/strong> Repeat Shows Positive Lift: The probability thta an equivalent repeat produces any positive lift, even a tiny, tiny, tiny +0.01 points – an extremely low bar.<\/p>\n

    2.<\/strong> Repeat Reaches \u226590% Certainty: The probability that an equivalent repeat produces \u226590% Certainty – my minimum standard for evidence strong enough for us to spend again.<\/p>\n

    3.<\/strong> Replication Risk: Simply the opposite of #1. The probability that the repeat falls short of the \u226590% standard.<\/p>\n

    Human language: What are the chances I spend another $5m, and still don’t get a result with certainty for me tell my CMO or CFO, “This worked. Spend again.”<\/p>\n

    Punch to the gut: If your Replication Risk is 70%… There is only a 30% chance that spending another $5m will result in you telling your CMO “This worked.” #killmefirst<\/p>\n

    Google vs. You: Google calls 70% Certainty a “good chance.” The replication math shows the “much, much less than good chance” you’ll take in round two.\n<\/p><\/div>\n

    Here’s the model using a simple Bayesian replication model inspired by Mr. Goodman:<\/p>\n

    \"Results<\/center><\/p>\n

    [<\/strong>Pedantic Note: Google says displayed Certainty is rounded down in 5-point increments. So a displayed 70% can represent 70% to just under 75% and hence the exact replication % can move a few points. For simplicity, I treat Google’s displayed Certainty as exact.]<\/strong><\/p>\n

    YouTube had reported 70% “Certainty,” follow the row above… That “good chance” Google label translates into:<\/p>\n

    \n

    1. Just a 65% (!) chance of any Positive Lift, with a <\/p>\n

    2. A puny 30% chance of hitting high certainty, <\/p>\n

    3. when repeating the same spend ($5m) under the same conditions.<\/strong>\n<\/div>\n

    As a business owner, would you feel confident saying yes, let’s spend on YT again?<\/em><\/p>\n

    Forget Google for a second, take me: I recommend a minimum of 90% Certainty<\/strong> for a business decision. That comes with a Replication Risk of 50% (!!) and Repeat Shows Positive Lift of just 82%. Too uncomfortable in many cases (what if it was a $10m question, would you take 50% and 82%?)<\/p>\n

    Now Alex and Juliette understand why I so often push for 95% Certainty<\/strong>. When you spend as much money as we do on YouTube\u2026 At least get the Replication Risk down to 40%, increase the Repeat Shows Positive Lift to 88% and increase the odds more in the company’s favor.<\/p>\n

    So\u2026 Are results below 90% Certainty useless? <\/p>\n

    If you have real money riding on it, look at column 3, look at column 5 in the table, see if you want those odds. If you have little to no money riding on it, sure – go try a new hypothesis, design a different\/better test, give additional money to Google\/YT\/Meta\/TT, remeasure.<\/p>\n

    A bonus relevant lesson from “Beyond Power Calculations: Assessing Type S (sign) and Type M (magnitude) Errors”: Weak evidence can hurt twice: The evidence may not survive (70% in our case), and the apparent impact might be overstated (+3 in our case). <\/p>\n

    You’ve been warned. <\/p>\n

    Premium Subscriber Bonus: A Working Model.<\/strong><\/span><\/p>\n

    I want you to be able to see all the formulas and math I’ve used in my computation of Replication Risk.<\/p>\n

    I want you to be able to test lift’s Certainty AND Magnitude (Repeat Shows Positive).<\/p>\n

    I want you to be able to type in the results of your Brand Lift, Search Lift, and Conversion Lift studies and calculate your own truth (before you put your career on the line).<\/p>\n

    To enable all this, I’ve built a working model with Read Me + Methodology, Calculator, Risk Table, and Calculate Your Truth tabs. It is a bonus for TMAI Premium subscribers<\/a>; please just email me for lift_replication_risk_model.xlsx.<\/p>\n

    Bottom line.<\/strong><\/span><\/p>\n

    Google and you are sitting on opposite sides of a consequential decision.<\/p>\n

    Google sells advertising. You are spending money.<\/p>\n

    Google might <\/strong>reasonably find value in evidence that is merely suggestive. See the first table. You have to decide whether that evidence is sufficient and measured credibly enough to risk the next tranche of your budget.<\/p>\n

    Don’t let an advertising platform’s definition of useful information<\/em> become your company’s definition of sufficient evidence<\/em>. <\/p>\n

    Stop outsourcing your tolerance for risk to the company selling you ads or the Agency whose fees are tied to the size of your spend.<\/p>\n

    Carpe diem.<\/p>\n

    PS:<\/strong> We have used Google’s specific table and specific guidance today. Replication risk applies just as much to Meta, TikTok, and confident calcs your Agency is sending you. Please check just as carefully if they are solving for themselves or you.<\/p>\n","protected":false},"excerpt":{"rendered":"

    When Google reports lift results\u2026 Who is it solving for? Let’s start with a phrase that causes me to cringe […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"default","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"set","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[11,2],"tags":[299,171],"class_list":["post-10330","post","type-post","status-publish","format-standard","hentry","category-advanced-analytics","category-analytics","tag-actionable-analytics","tag-key-performance-indicators"],"acf":[],"yoast_head":"\nGoogle's Statistics: Heads They Win, Tails You Lose. - Occam's Razor by Avinash Kaushik<\/title>\n<meta name=\"description\" content=\"Google recommends a 70% statistical significance lift result as a "good chance caused by your ads." The problem? It's just a 65% chance a repeat would show a positive lift. Should you spend $5m again?\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Google's Statistics: Heads They Win, Tails You Lose. - Occam's Razor by Avinash Kaushik\" \/>\n<meta property=\"og:description\" content=\"Google recommends a 70% statistical significance lift result as a "good chance caused by your ads." The problem? It's just a 65% chance a repeat would show a positive lift. Should you spend $5m again?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/\" \/>\n<meta property=\"og:site_name\" content=\"Occam's Razor by Avinash Kaushik\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-14T08:14:27+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.kaushik.net\/avinash\/wp-content\/uploads\/2026\/09\/google_stats_guidance4.png\" \/>\n<meta name=\"author\" content=\"Avinash Kaushik\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Avinash Kaushik\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/\"},\"author\":{\"name\":\"Avinash Kaushik\",\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/#\\\/schema\\\/person\\\/4833e40c5e82d40b6e6d0cc37d026ad6\"},\"headline\":\"Google’s Statistics: Heads They Win, Tails You Lose.\",\"datePublished\":\"2026-09-14T08:14:27+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/\"},\"wordCount\":109,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/google_stats_guidance4.png\",\"keywords\":[\"actionable analytics\",\"key performance indicators\"],\"articleSection\":[\"Advanced Analytics\",\"Analytics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/\",\"url\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/\",\"name\":\"Google's Statistics: Heads They Win, Tails You Lose. - Occam's Razor by Avinash Kaushik\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/google_stats_guidance4.png\",\"datePublished\":\"2026-09-14T08:14:27+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/#\\\/schema\\\/person\\\/4833e40c5e82d40b6e6d0cc37d026ad6\"},\"description\":\"Google recommends a 70% statistical significance lift result as a \\\"good chance caused by your ads.\\\" The problem? It's just a 65% chance a repeat would show a positive lift. Should you spend $5m again?\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/google_stats_guidance4.png\",\"contentUrl\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/google_stats_guidance4.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/googles-statistics-significance-replication-risk\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Google’s Statistics: Heads They Win, Tails You Lose.\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/#website\",\"url\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/\",\"name\":\"Occam's Razor by Avinash Kaushik\",\"description\":\"Marketing | Strategy | Analytics\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.kaushik.net\\\/avinash\\\/#\\\/schema\\\/person\\\/4833e40c5e82d40b6e6d0cc37d026ad6\",\"name\":\"Avinash Kaushik\",\"description\":\"Author: Web Analytics 2.0 & Web Analytics: An Hour A Day | Chief Strategy Officer: Human Made Machine\",\"sameAs\":[\"https:\\\/\\\/www.kaushik.net\\\/avinash\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/akaushik\\\/\",\"https:\\\/\\\/x.com\\\/https:\\\/\\\/twitter.com\\\/avinash\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Google's Statistics: Heads They Win, Tails You Lose. - Occam's Razor by Avinash Kaushik","description":"Google recommends a 70% statistical significance lift result as a \"good chance caused by your ads.\" The problem? It's just a 65% chance a repeat would show a positive lift. Should you spend $5m again?","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/","og_locale":"en_US","og_type":"article","og_title":"Google's Statistics: Heads They Win, Tails You Lose. - Occam's Razor by Avinash Kaushik","og_description":"Google recommends a 70% statistical significance lift result as a \"good chance caused by your ads.\" The problem? It's just a 65% chance a repeat would show a positive lift. Should you spend $5m again?","og_url":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/","og_site_name":"Occam's Razor by Avinash Kaushik","article_published_time":"2026-09-14T08:14:27+00:00","og_image":[{"url":"https:\/\/www.kaushik.net\/avinash\/wp-content\/uploads\/2026\/09\/google_stats_guidance4.png","type":"","width":"","height":""}],"author":"Avinash Kaushik","twitter_misc":{"Written by":"Avinash Kaushik","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/#article","isPartOf":{"@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/"},"author":{"name":"Avinash Kaushik","@id":"https:\/\/www.kaushik.net\/avinash\/#\/schema\/person\/4833e40c5e82d40b6e6d0cc37d026ad6"},"headline":"Google’s Statistics: Heads They Win, Tails You Lose.","datePublished":"2026-09-14T08:14:27+00:00","mainEntityOfPage":{"@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/"},"wordCount":109,"commentCount":0,"image":{"@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/#primaryimage"},"thumbnailUrl":"https:\/\/www.kaushik.net\/avinash\/wp-content\/uploads\/2026\/09\/google_stats_guidance4.png","keywords":["actionable analytics","key performance indicators"],"articleSection":["Advanced Analytics","Analytics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/","url":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/","name":"Google's Statistics: Heads They Win, Tails You Lose. - Occam's Razor by Avinash Kaushik","isPartOf":{"@id":"https:\/\/www.kaushik.net\/avinash\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/#primaryimage"},"image":{"@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/#primaryimage"},"thumbnailUrl":"https:\/\/www.kaushik.net\/avinash\/wp-content\/uploads\/2026\/09\/google_stats_guidance4.png","datePublished":"2026-09-14T08:14:27+00:00","author":{"@id":"https:\/\/www.kaushik.net\/avinash\/#\/schema\/person\/4833e40c5e82d40b6e6d0cc37d026ad6"},"description":"Google recommends a 70% statistical significance lift result as a \"good chance caused by your ads.\" The problem? It's just a 65% chance a repeat would show a positive lift. Should you spend $5m again?","breadcrumb":{"@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/#primaryimage","url":"https:\/\/www.kaushik.net\/avinash\/wp-content\/uploads\/2026\/09\/google_stats_guidance4.png","contentUrl":"https:\/\/www.kaushik.net\/avinash\/wp-content\/uploads\/2026\/09\/google_stats_guidance4.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.kaushik.net\/avinash\/googles-statistics-significance-replication-risk\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.kaushik.net\/avinash\/"},{"@type":"ListItem","position":2,"name":"Google’s Statistics: Heads They Win, Tails You Lose."}]},{"@type":"WebSite","@id":"https:\/\/www.kaushik.net\/avinash\/#website","url":"https:\/\/www.kaushik.net\/avinash\/","name":"Occam's Razor by Avinash Kaushik","description":"Marketing | Strategy | Analytics","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.kaushik.net\/avinash\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.kaushik.net\/avinash\/#\/schema\/person\/4833e40c5e82d40b6e6d0cc37d026ad6","name":"Avinash Kaushik","description":"Author: Web Analytics 2.0 & Web Analytics: An Hour A Day | Chief Strategy Officer: Human Made Machine","sameAs":["https:\/\/www.kaushik.net\/avinash","https:\/\/www.linkedin.com\/in\/akaushik\/","https:\/\/x.com\/https:\/\/twitter.com\/avinash"]}]}},"_links":{"self":[{"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/posts\/10330","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/comments?post=10330"}],"version-history":[{"count":11,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/posts\/10330\/revisions"}],"predecessor-version":[{"id":10364,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/posts\/10330\/revisions\/10364"}],"wp:attachment":[{"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/media?parent=10330"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/categories?post=10330"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kaushik.net\/avinash\/wp-json\/wp\/v2\/tags?post=10330"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}