As marketing professionals, you're no stranger to the exhilarating dance of A/B testing. You've celebrated victories — the soaring conversion rates, the triumphant headlines. And you've mourned defeats — the flatlining metrics, the dashed hopes. But what about those murky, in-between moments? The tests that neither shout "Eureka!" nor whisper "Back to the drawing board." The ones that leave us suspended in uncertainty. Today, let's explore the five stages of grief that accompany these often-forgotten experiments. 1. Denial Picture this: Your latest test has been running for weeks. It's like a marathon runner stuck in a perpetual sprint. You cling to optimism. "There's still time," you tell yourself. "Statistical significance is just around the corner." Denial wraps you in a comforting cocoon. You scrutinize the test setup, check the tags, tweak the parameters. Maybe if you squint hard enough, the results will magically align with your expectations. But deep down, you know—the sands of reality have shifted. Think about the development impact of waiting. If you keep that test running and just not getting any results that are telling you whether it’s a good idea or a bad idea, how will that impact your next Sprint? How will that impact your next release and how far will that push out other possibly winning ideas? 2. Anger The numbers mock you. The inconclusive graphs taunt your patience. Anger flares—a primal response to emotional discomfort. You're not vulnerable; you're furious. "Why did we launch this test?" you demand. "Whose idea was it? Wow, we've wasted so much time." But pause. Breathe. You didn't fail; you explored. Every inconclusive test is a breadcrumb—a lesson in disguise. Consider the value of knowing what doesn't move the needle. Those areas where you can unleash creativity without fear. Conversations with designers become liberating: "Go wild; this part isn't critical." 3. Bargaining The fog of uncertainty thickens. You’re no longer in denial, but you’re not ready to surrender either. Bargaining sets in. “Maybe we can relaunch the test with a slight tweak,” you propose. “Perhaps our statistical significance requirements are too high. Maybe we can be confident with being less confident.” If the potential impact of this test idea was truly substantial for the business, consider relaunching it with a slightly different experience. Tweak it, iterate, and ground your decisions in data—not wishful thinking. Remember, we don’t live our lives by statistical significance levels and p-values. If someone tells us that a decision provides more than a 50% chance of success in real life, we probably would make that decision. So, do we have to make every business decision based solely on rigid statistical rules? Perhaps not. Consider this: If your company wasn’t testing at all, they might have made a decision one way or the other without the safety net of experimentation. Maybe good enough is better than perfect. Maybe perfect is getting in your way of making a decision that’s probably correct. These are important reflections. If you have learnings from this test, if you arrive at an informed decision, you may be able to leap right into the fifth stage—acceptance 4. Depression The fog thickens further. You’ve moved beyond anger; now you’re adrift in the abyss of doubt. “Maybe we should stop testing,” echoes in your mind. “Maybe we have just hit a wall. Maybe it just doesn’t make sense to do it anymore.” The weight of indecision presses down. But pause. Take a breath. You didn’t fail; you explored. Every inconclusive test is a breadcrumb—a lesson in disguise. It’s time to reexamine our roadmap. Let’s dissect every assumption we’ve made. Pull it apart, scrutinize it, and figure out why we’re wrong. But remember, you’re not alone in this journey. Gather your team. Seek input. Hold a retrospective. What went well in this test? What did we learn, if anything? And most importantly, what would we have done differently if we ran it again? As a group, come to a consensus. 5. Acceptance The final stage beckons—the phoenix rising from the ashes. You've shared findings, dissected failures, and learned. Acceptance isn't resignation; it's wisdom. Turn off the test; let the data rest. Analyze, segment, and glean insights. Consider the development impact of waiting. Then, as the bargaining phase fades, make an informed call. Maybe it's not perfect, but good enough. Share your learnings across the organization. Remember, credibility lies in transparency. And as we emerge from the five stages, we emerge wiser. At ZIlker Trail, we've weathered the storms of experimentation. Let's chart new courses together!

AI. Continuous Improvement. Democratizing Analytics Data. All are important topics that business leaders were talking about at the 2024 QuantumLEAP conference in March. In this interview, Director of Growth Strategy Erik Sheehan gives his thoughts on all three topics. On the topic of AI, Quantum Metric announced a new AI product named "Felix" at the conference. The goal is to utilize AI to boost efficiency, allowing teams to focus more on fixing problems instead of going through mountains of data to identify them. This will free up resources to be more strategic, but what will that mean for entry-level employee training? How will our future leaders learn how to identify problems? Regarding Continuous Improvement, Erik talks about how tools have evolved to help companies quickly identify areas for improvement, marrying data with session replay to allow for rapid product improvement. But how do companies address the needs for continuous improvement against requirements for tech stack speed and stability? The interview then moves to data democratization. Erik discusses how increasing access to data across companies will improve efficiency while eliminating churn and confusion. Watch the video below to get insights on how best in class companies are actually using AI today, how organizations are balancing continuous improvement with the challenges of platform stability, and how wider access to consistent sources of data aligns goals, enhances decision-making, and ultimately improves the customer experience.

Brian Cahak, the founder of Zilker Trail Consulting, recently returned from South by Southwest and sat down with Keith Swiderski for a recap. In Part 1 of our interview, Brian discusses the uniqueness of South by Southwest. He also talks about why Amplitude was an ideal partner for the Zilker Trail event and why you should consider their product today. "As we were out in the market looking for products that we could build managed services around, we were instantly attracted to Amplitude...they've evolved their product suite from just analytics to now include an experimentation suite as well as audience capturing and management capabilities...as you zoom out, what they have in one integrated platform is actionable insights, personalized activation through optimization, and then data enrichment through the audience stream...and because it's all in one cohesive platform, it makes it easier for clients to be able to implement all of this without a bunch of extra integration work." See part one of the interview below, and check out part 2 of the interview here.

In Part 2 of our SXSW 2024 recap, Zilker Trail Founder Brian Cahak shares recommendations for companies looking to optimize their success when introducing new tools and technologies. These recommendations include: - Setting clear objectives for the tool - Identifying key questions that the tool can help answer - Ensuring stakeholders receive clear communication about the product’s value - Celebrating achievements and milestones collectively to maintain enthusiasm Brian also introduces his "Hall of Justice" concept. A "Hall of Justice" includes business leaders around the company. Bringing these leaders together for a workshop to discuss which business questions are critical for a tool to answer helps focus resources on building outputs to answer these questions, and get leaders excited about the tool and what it can offer. Building this excitement early, and celebrating early wins, can be key in ensuring that a new tool provides value for a company versus sitting on a shelf. In case you missed part 1 of this interview, check it out here. Part 3 of the interview can be found here.

In Part 3 of our SXSW 2024 recap, founder Brian Cahak discusses the common phenomenon where companies hire teams of brilliant data scientists only to assign them mundane and repetitive tasks, such as creating dashboards. Brian explores methods to calculate the revenue impact of not utilizing these experts to their full potential and suggests ways companies could engage these resources more effectively. "The tooling by itself isn't the savior, but the tooling wrapped with a certain alignment, with clarity of task and purpose, with clear action plans, is how you really get the outcomes that you're aiming for." We conclude the interview by discussing our guest artist, Zach Person, and explaining why he was the perfect partner for the event. Links: Check our Zach Person on Spotify: https://lnkd.in/eqYZx5XU Learn more about Amplitude: https://amplitude.com/ Missed parts 1 and 2 of the interview? Part 1 and Part 2 can be found here.

Experimentation isn't for everyone. It's world where numbers dance and insights hide, where experiments fail over 70% of the time, where you're proven wrong much more than you're proven right. But for those that practice it, it can be a magical key to unlock growth in your company. In this video, Keith dives into the five traits that he has found can transform you from an analyst into an Experimentation whiz, and move your company into a growth mindset with a culture of Experimentation. 1. Pattern Whisperer Your eyes scan spreadsheets like a seasoned detective. Rows and columns reveal their secrets—the subtle correlations, the outliers, the hidden symmetries. You're not content with surface-level summaries; you dive deep, tracing the threads that weave the fabric of truth. Whether it's sales data or user behavior, you spot patterns others miss. Your superpower? Seeing the invisible stitches that bind the data universe. 2. Embracing the Wrongness Ah, the thrill of being wrong! Wait, what? While others cringe at their missteps, you embrace them. Why? Because each wrong turn is a breadcrumb leading to enlightenment. A hypothesis falls flat? Fantastic! You've ruled out one path, and the next attempt inches you closer to the bullseye. Being wrong isn't failure; it's progress in disguise. 3. The Uncharted Questioner While colleagues tread familiar ground, you venture into uncharted territories. Your questions defy convention. "What if we gamify the checkout process?" you ponder. Or perhaps, "Why does the bounce rate spike during full moons?" These queries raise eyebrows, but they also ignite innovation. You're the compass pointing north when everyone else clings to the map. Keep asking—the unasked questions hold the keys to breakthroughs. 4. Problem Whisperer Problems aren't roadblocks; they're puzzles waiting to be solved. Problems fuel your adrenaline. They're your playground, your proving ground. And when you crack them open, the rush of victory is sweeter than the finest chocolate. Keep loving those problems—they're your stepping stones. 5. Decoding the Human Equation Data isn't lifeless; it pulses with human choices. Why did Sarah abandon her cart? What nudged Alex to click "Subscribe"? You're not satisfied with mere metrics; you seek the human heartbeat behind the numbers. Dive into psychology, sociology, and empathy. Understand motivations, fears, and desires. Because when you unravel the human equation, you unlock decisions. And decisions shape destinies. Wear these traits proudly. When companies seek wizards who blend science with intuition, be ready. And remember: We're here, cheering you on. Reach out. Let's solve problems together.

Zilker’s GPS Growth Model is a repeatable framework built on compounding feedback loops that incorporates brand, customer and business value. ​ Learn from the founder of Zilker Trail about how our proven approach aligns expertise, muscle and a repeatable framework to deliver consistent insights and growth.

Experimentation—the very word conjures up images of lab coats, beakers, and late-night hypotheses. For businesses, it’s often seen as a labyrinth of complexity, a costly venture that requires meticulous planning and execution. But what if I told you that experimentation doesn’t have to be a headache? What if there’s a way to embrace its spirit without traditional testing? Keith Swiderski, our Experimentation Lead, sheds light on this paradox. In a recent video, he challenged the status quo, urging companies to think beyond the confines of A/B tests and statistical significance. Here’s the secret: experimentation isn’t just about testing—it’s about mindset. Google Optimize may be waving goodbye, but that’s no reason to abandon experimentation. Creating an experimentation culture isn’t rocket science—it’s behavioral science. Start by: Leadership Buy-In: Leaders set the tone. When they champion experimentation, teams follow suit. Cross-Functional Collaboration: Break silos. Marketing, product, and engineering collaborate to design experiments that span the customer journey. Celebrate Learning: Whether it’s a win or a “not-quite-yet,” celebrate insights. Learning fuels progress. So, why bother with experimentation? Because it’s the heartbeat of innovation. It’s the spark that ignites breakthroughs. And when you’re ready to embark on this scientific adventure, remember: We’re here to guide you. Reach out. Let’s experiment together.