For a decade, the management canon swung like a pendulum. First by declaring the supremacy of gut feelings, then the inevitability of ‘big data’. In practice, the best leaders do neither. They pair fast, comprehensive analytics with adaptable human heuristics and simple rules honed by context to make sharper, faster, and more resilient choices.
This is especially true in today’s startup ecosystem, where we are drowning in information yet starved for wisdom. We have real-time dashboards for user acquisition, churn rates, and burn rates. We track every click, scroll, and impression. This firehose of ‘big data’ analytics, we are told, holds the key to comprehensive insights and rapid, effective strategy.
And it does. Data analytics is spectacularly fast and ruthlessly efficient at identifying the what. It can tell you that 29 per cent of your users drop off at the payment screen. It can flag that your new feature has a 0.9 per cent adoption rate.
But data is usually silent on the why. The biggest challenge in an organisation’s application of big data lies in the fact that there is a lot of data, but very few insights. Abundant data does not necessarily lead to smart decisions.
And this is exactly where the company leader’s most undervalued asset comes into play: human intuition. This is not necessarily referring to blind gut feel, but rather the collection of highly adaptable, experience-driven heuristics that allow a leader to see around the corner, not just at the graph on the screen.
The best leaders I have worked with are not data-purists or instinct-driven cowboys. They are hybrids. They combine the comprehensive, high-speed processing of big data with the nuanced, adaptive sensemaking of human heuristics. This integration is what leads to truly effective strategic decisions.
The SME’s dilemma: “We don’t have a data science team”
This is where I often hear the objection, for example, from a startup that I am currently working with at Singapore’s LaunchPad. They raised a concern that the hybrid model sounds great for larger organisations like Grab or Google, but they are running an AI startup with only 50 employees. They questioned their ability to achieve this without having, for example, a team of PhDs to run regressions.
My answer is unequivocal. This hybrid method is not only achievable for startups and SMEs, but it is also possible to do it better because there are fewer organisational structure layers, faster feedback, and less political noise. The constraint is not capital; it’s clarity and cadence.
Large corporations use big data to optimise an existing, proven machine. The benefit of a startup is that it is still building the machine. Its greatest advantages are speed, agility, and a deep, intuitive connection to customers. These advantages are often lost at scale.
For an SME, ‘big data’ is a misleading term. A data lake is not always necessary; you just need the right data. Instead of hoarding data just in case, you collect fit-for-decision data, which is the smallest, fastest, most reliable signals that inform this decision at this moment. Similarly, what’s not always needed is a costly platform or tool to get enterprise-grade visibility of your data. The goal is time-to-insight, not tool sophistication.
Also Read: Why Generative AI requires a paradigm shift in technology and culture
I normally have the following suggestions for startups or SMEs that I work with in terms of the hybrid model, marrying intuition with big data:
Data tells you What, intuition asks Why
For a startup, your ‘big data’ is simply your Google Analytics, your Mixpanel dashboard, your CRM, or even your Stripe payment history. The goal is not to analyse everything, but to find the critical few metrics that matter.
Returning to the data point mentioned earlier, “29 per cent of users drop off at the payment screen”, there are two trains of thought.
- The data-only response: “The page must be broken. Let’s refactor the code. Let’s A/B test the button colour.”
- The hybrid response: The leader looks at the data, but then their intuition (heuristics) kicks in. “I wonder if this isn’t a technical bug, but a trust bug. We ask for a credit card right after they’ve seen only one feature. It feels too aggressive. What if we move the paywall after the user sees the ‘Aha!’ moment?”
Data shows what happened. Human intuition, built from a deep understanding of customer psychology, provides the hypothesis as to why.
Use intuition to form the hypothesis, use data to validate it
This is the most resource-efficient way to operate. Instead of using limited engineering resources to test every possibility, company leaders can utilise their intuition to make an educated bet.
- Intuitive hypothesis: “My gut tells me our best customers aren’t the ‘enterprise’ clients we’re chasing, but the small agencies who use the tool daily.”
- Data-driven test: “Let’s pause our expensive enterprise outreach for two weeks. Let’s take that small budget and run a hyper-targeted campaign aimed at 100 small agencies. We will measure the conversion rate and, more importantly, the 30-day engagement.”
This is a simple, inexpensive experiment that uses a human heuristic to set the strategy and direction, and a clean data set to validate (or invalidate) it. This method focuses efforts on a single, high-leverage target instead.
Also Read: Generative AI in daily life: A practical guide
Resources that focus on sensemaking, not just reporting
With limited resources, your most valuable meetings are not data reporting meetings. They are data sensemaking meetings to figure out the why and what.
- A reporting meeting says: “User sign-ups were down 15 per cent.”
- A sensemaking meeting asks: “User sign-ups were down 15 per cent. What else was true last week? Was it a holiday in a key market? Did a competitor launch a new campaign? Did our blog post on a technical topic drive away non-technical users? What does this mean?”
This is a cultural shift. It empowers your team to be data-informed, not data-imprisoned. It gives them permission to bring their human insights, their conversations with customers, their frustration with the product, and their sense of the market into the conversation alongside the dashboard.
The achievable hybrid powered by GenAI
This hybrid approach is now more accessible than ever, thanks to the rise of generative AI. For resource-constrained startups and SMEs, generative AI models can significantly lower the fixed cost of analysis (data cleaning, correlation and pattern detection, forecasting), translate founders’ tacit rules-of-thumb into testable prompts or lightweight decision checklists, and run rapid what-if simulations that managers validate with contextual judgment.
AI, that’s used with a human-in-the-loop, with clear guardrails on data quality, privacy, and bias, doesn’t replace intuition. Instead, it amplifies disciplined intuition by making evidence easier to assemble, assumptions more explicit, and decisions faster, cheaper, and more consistent for startups and SMEs.
—
Editor’s note: e27 aims to foster thought leadership by publishing views from the community. Share your opinion by submitting an article, video, podcast, or infographic.
Enjoyed this read? Don’t miss out on the next insight. Join our WhatsApp channel for real-time drops.
Image generated using AI.
The post Data-driven or gut-led? Why the best startups do both appeared first on e27.