Agile, Scrum, Kanban and the Lies We Tell Ourselves About Creating Value

But often, the excitement dies down. Companies feel like Agile isn’t living up to its promises. Teams feel trapped in rituals like standups, retrospectives, and Kanban boards, with little real improvement. Meanwhile, organizations often keep using the same leadership styles, the same lack of psychological safety, and the same outdated metrics. People do the actions of sprints or continuous flow, but see only a little benefit from the big changes.

For more than 20 years, Agile frameworks like Scrum, Kanban, and Extreme Programming (XP) have been praised as the solution to ongoing problems in product delivery, project management, and value creation. Consultants have said that with daily stand-up meetings, sprints, user stories, and visually appealing boards, organizations will release better products faster, align teams seamlessly, and impress customers on an ongoing basis.

Why does this happen? One major oversight is our collective assumption that “Agile,” as defined by frameworks like Scrum or Kanban, must be a universal solution for every context and every product challenge. Second, we assume that “linear” project management methods are no longer useful. The truth is more complicated. Agile and linear approaches can be very powerful when used in the right way and supported by the right mindset.

In this article, we will explore the deeper, often unspoken challenges that get in the way of both Agile and traditional linear methods. We will discuss how the most important factor in creating value is not a particular framework but rather the organizational and cultural context. This context includes your leadership approach, your technical discipline, and, most importantly, your commitment to continuous learning and adapting. If we see Agile frameworks (like Scrum, Kanban, and XP) and linear methods (like waterfall and stage-gate) as just a few tools in a bigger toolbox, we can finally deliver value to our customers.

The Underlying Tension: Why “Agile vs. Linear” Is a False Dichotomy

For years, project managers have been debating whether Agile and linear (or predictive) methods are at odds with each other. Traditional linear methods, also called “Waterfall,” follow a step-by-step sequence:

1. Requirements analysis
2. Design
3. Implementation
4. Verification
5. Maintenance

Each step is done before moving on to the next one. This is usually done with formal approvals. This approach works well when the requirements are clear from the start and the environment is stable. Some examples of this are construction projects, regulated industries, or highly standardized manufacturing. When used in these situations, linear frameworks can be very good. They provide clarity, (ideally) predictable steps, and and a detailed roadmap that aligns well with regulatory or compliance demands.

Agile frameworks were created to deal with situations where things are always changing and not many things are known for sure. One example of this is the development of software in markets where things are changing quickly. In these situations, it is very important to be able to make quick changes, test ideas, and change course based on feedback. Scrum, Kanban, XP, and other Agile methods focus on frequent collaboration, where teams can organize themselves and deliver small improvements over time.

There’s a lot of debate about this, especially when companies try to apply the same approach to everything. The truth is, the best approach depends on the situation. Some jobs can be handled well by a more organized approach, while other jobs need to be more flexible and can’t be done with a set plan.

In today’s world, we understand that things can be complicated. Sometimes, simple tasks can be done with simple methods. But when things get complicated and change a lot, it’s better to use different methods. Agile methods are often better for complicated problems that change often. Companies often mix these two types of methods to deal with the complexity of their products, projects, and plans.

A Growth Mindset as the Baseline for Success

No matter if you use Agile frameworks or more traditional, linear approaches, the key to delivering real value is having a growth mindset. Psychologist Carol Dweck introduced the term “Growth Mindset,” which means believing that abilities and intelligence can be developed through dedication and hard work. This mindset helps people overcome challenges and stay eager to learn.

1. Willingness to Experiment
   Agile is all about trying new things, and the same is true of effective project management. In project management, managing risk can be a kind of testing. Whether you’re testing a hypothesis in a Scrum sprint or running a pilot study in a phase-gate process, having a growth mindset means learning from both successes and failures.
2. Openness to Feedback
   Frequent feedback loops are a key part of Agile. But even in a more traditional approach, effective teams still do stage reviews that include feedback from stakeholders and customers. If the project team and leadership are open to new ideas, they’ll embrace feedback that might require changes to the plan.
3. Continuous Learning and Adaptation
   When a growth mindset is common in an organization, people there are always looking for ways to improve. They don’t just do things the way they’ve always done them. They always want to get better at planning (whether they plan in a step-by-step way or in a way that involves trying out different ideas) and what they deliver (the things that really matter). They see frameworks as tools that can be improved over time, not as rules that should be followed without question.

An environment lacking a growth mindset will fail to maximize either Agile or linear frameworks. Instead, it will stagnate, resisting change and punishing failure, which ultimately cripples innovation and adaptation regardless of the method in use.

The Roots of “Agile” and Its Limitations

The Agile Manifesto (2001) called for new ways of working, emphasizing:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

Since then, frameworks  like Scrum, Kanban, and Extreme Programming (XP) have been created to put these values into action. The idea was (and still is) appealing: if teams focus on delivering small amounts of value regularly, constantly check and adjust their work, and work together, they can respond quickly to changing customer needs.

But many organizations have adopted Agile methodologies without fully understanding them.

1. Ritualized Ceremonies without Real Change
   Standups, sprints, and retrospectives become meaningless. People “go through the motions” but rarely use these ceremonies to deal with big problems (like micromanagement, incentives that are not aligned, and poor technical quality).
2. Partial or Shallow Technical Practices
   Many people do not pay enough attention to technical issues and working off tech debt often leading to the underlying code quality or systems architecture issues remaining unresolved.
3. Leadership Misalignment
   Sometimes, executives say they want “faster output” using Agile methods, but they still use the old top-down way of managing teams, which destroys autonomy and empowerment.
4. Cultural Mismatch
   Many organizations don’t have the right environment for people to work together. They need to have psychological safety and trust. Without these, no framework will work.

Agile frameworks can enhance strong foundations, allowing for quick feedback loops, experimentation, and close collaboration. However, they don’t replace the need for clarity of purpose, clear goals, a strategy, a healthy organizational culture, and supportive leadership..

The Validity of Linear Methods

Today, many people are excited about Agile transformations. Because of this, it’s easy to think that linear methods are old-fashioned. But linear or predictive project management can actually be very effective in certain situations.

• Requirements are relatively stable and well-understood at the outset and can not be negotiated – especially in a highly regulated environment
• The regulatory or compliance environment demands formal documentation of each phase
• Significant portion of the work is repetitive or standardized, such as manufacturing a known product in large quantities

For example, think about making medicines. In this case, it’s very important to follow the rules set by the FDA’s Current Good Manufacturing Practices (cGMP). A step-by-step process makes sure that each important step (like creating the medicine, testing it in animals, testing it in humans, and increasing the amount of medicine made) is finished and documented with few mistakes.

Another example is when a government agency is working on a civil engineering or construction project. In these cases, there are often very specific design details, budgets, and deadlines set in contracts. While agile methods can be useful in the early stages of designing a project, once construction begins, a more traditional, step-by-step approach is necessary to make sure that building codes, safety standards, and engineering tolerances are followed.

It’s important to understand that even in these areas, having the ability to adapt and think in a growth mindset can still lead to positive results. For example, many large construction companies are starting to use Lean Construction methods. These methods involve teams using visual task boards (similar to Kanban) and daily meetings to identify and address issues. They still follow the usual project phases and approval procedures.

The Common Dysfunctions That Undermine Both Approaches

Whether an organization chooses Agile frameworks or more linear methods, they often stumble for the same underlying reasons:

1. Lack of Clear Vision and Goals
   • Teams don’t understand why they’re building a product or completing a project.
   • In a linear context, this leads to missed specifications or misaligned deliverables.
   • In Agile, it manifests as constant pivoting without any coherent strategic direction.
2. Siloed Departments and Poor Collaboration
   • Traditional organizations often separate Development, QA, Operations, and Marketing, each with its own objectives.
   • No matter the method, these silos create bottlenecks and communication gaps.
   • The transition from one phase to the next (in linear projects) or sprint to sprint (in Agile) becomes difficult.
   • Actually delivering something and getting it into a productive state is a huge pain with a lot dependencies involved that hinder the market-readiness
3. Command-and-Control Leadership
   • Executives who micromanage every step along the way don’t let the team make decisions for themselves.
   • In a linear approach, this fosters mindless compliance rather than engagement.
   • In Agile settings, it destroys the self-organizing dynamic and growth mindset essential for iterative improvement.
4. Misaligned Incentives and Metrics
   • Organizations that reward meeting deadlines instead of achieving real results end up with surface-level success indicators.
   • Agile teams that are measured purely on “story points” rather than value delivered to the customer suffer the same fate.
5. Lack of Psychological Safety
   • Individuals are afraid to share concerns, admit mistakes, or propose innovative ideas.
   • Agile ceremonies such as retrospectives lose their meaning because no one feels safe enough to speak openly.
   • In linear projects, issues remain hidden sometimes even in a formal “phase review” which leads to them being uncovered far too late.
6. Insufficient Technical Practices
   • Whether writing code or executing a manufacturing process, subpar technical discipline leads to rework, delays, and poor quality.
   • Agile demands robust engineering practices to release iteratively. Linear demands rigorous documentation and planning. Either can fail if technical excellence isn’t pursued.

These problems don’t care what framework you’re using. The big mistake is thinking that a method like Agile will suddenly fix deeper problems in how your organization and culture work.

The Real Key: Value Creation Over Ritual

Value is the ultimate currency of any work system. Value isn’t about how many features you build or tasks you complete; it’s about how well you solve real customer problems, generate revenue, and deliver results that matter. Both agile and linear methods can help you get there, but only if you always keep value in mind.

1. Value Definition
   • In an Agile context, teams often define value in user stories, focusing on what the end-user or customer actually needs.
   • In a linear context, the scope statement or requirements document sets the baseline for what value looks like.
   • In both cases, clarity on why a feature (or project requirement) is valuable to the customer and user of our work is most important thing to define.
2. Measurement and Feedback Loops
   • Agile teams embed short feedback cycles – demos, reviews, and iterative releases – to evaluate customer value quickly.
   • Linear projects rely on phase gates and milestones to evaluate progress.
   • The difference is frequency and adaptability. However, if you focus on the result and are willing to change your approach if feedback indicates that you are not on the right track, a linear project can also include elements of adaptation.
3. Outcome vs. Output
   • “Output” is the volume of work done: tasks completed, lines of code written, story points burned.
   • “Outcome” is the impact of that work: customer satisfaction, cost savings, revenue growth, time-to-market reduction.
   • Both Agile and linear processes can fall into the trap of measuring output (e.g., sprint velocity or Gantt chart progress) instead of real outcomes.
4. Inspect and Adapt
   • The Agile principle of “inspect and adapt” isn’t just for Scrum or Kanban. Other methods like linear approaches also have reviews, but they’re less often.
   • The question is: do you have the courage to act on the findings? Too often, organizations only go through the ceremony (retrospective or phase review) and ignore its implications.

When you anchor your work in value creation and maintain the agility to respond to real feedback, you’re practicing the essence of agile value creation – no matter how formally “Agile” or “linear” your framework appears on paper.

The Stacey Matrix: A Key to Choosing Your Method

One of the most useful models for deciding how to approach project management – whether agile, linear, or a hybrid – is the Stacey Matrix. Developed by Ralph Douglas Stacey, this framework helps categorize work along two dimensions:

1. Clarity of Requirements – How well do they know where to go?
2. Clarity of Methodology and Technology – How much predictability or clarity exists in the technology, requirements, and context?

By plotting these on a matrix, organizations can distinguish between different levels of complexity:

1. Simple (Clear)
   • Here, both requirements and methods are well understood. Processes are repeatable, and few unknowns exist.
   • Traditional, linear management excels in this space because the path from start to finish is obvious and uncontroversial.
   • Example: routine manufacturing tasks with standardized operating procedures.
2. Complicated
   • The objectives are understood, but the methods require expert analysis and may not be obvious to non-specialists.
   • While a linear approach can still work – often guided by experienced professionals – some elements of Agile or iterative processes (like prototyping or repeated verification) can help manage risks and improve solutions.
   • Example: designing complex but well-defined systems, such as an engine, where specialists must carefully plan each step but the end goal is relatively stable.
3. Complex
   • Requirements and solutions are unclear, evolving, or uncertain. Multiple variables interact in unpredictable ways.
   • Agile approaches often thrive here because short feedback loops allow teams to adapt as they learn.
   • Example: software product development in a competitive, fast-changing market, where you discover ideal features only through user feedback.
4. Chaotic
   • Extreme instability and ambiguity dominate. Objectives or methods might be unclear, and urgent intervention is necessary.
   • The priority is to stabilize the situation quickly – often through short experiments or triage.
   • As soon as possible, teams should move the work out of the chaotic domain into a more manageable (complex or complicated) state.
   • Example: crisis management (e.g., responding to a major cybersecurity breach).

The Stacey Matrix helps to avoid the pitfall of using the same management style for all projects. If you’re dealing with Complex issues, an Agile approach might work well. For Simple or Complicated projects, a more traditional or even a hybrid approach might be better.

Where Scrum, Kanban, XP, and Linear Methods Shine

Scrum

• Best for teams developing a product in an environment where requirements evolve quickly.
• Time-boxed sprints provide focus and structure, while roles like Product Owner and Scrum Master create clarity.
• Suited to complex work with uncertainty, where rapid feedback cycles are critical to course-correct.

Kanban

• Ideal for steady flows of incoming requests (e.g., support tickets, minor enhancements) or teams needing flexible scheduling.
• Visualizing work-in-progress (WIP) and setting WIP limits helps expose bottlenecks and optimize flow.
• Continuous adaptation rather than fixed sprints fosters incremental improvement.

Extreme Programming (XP)

• Strengthens teams with rigorous technical practices: test-driven development, pair programming, continuous integration.
• Highly effective when high code quality and frequent releases are non-negotiable.
• Can be combined with Scrum or Kanban to add engineering discipline.

Linear / Predictive Approaches

• Staged approach ensures each phase meets strict quality or regulatory criteria.
• Useful when the scope is well-defined upfront, and the cost of rework is prohibitive.

It’s perfectly legitimate – and often wise – to mix these methods. For example, many organizations use a phase-gate process at the portfolio level to greenlight projects based on strategic alignment, and then delegate how teams work (Scrum, Kanban, or a hybrid) based on the complexity of the project. In the end the goal should be to create customer value at the lowest possible cost while maintaining a high level of quality and adherence to all applicable regulations.

Cultivating a Growth Mindset Across the Organization

No matter if you are leading teams that work in an agile way or running a program that follows a more traditional path, it is important to make sure that a growth mindset is present at all levels. Here is how you can do that:

1. Leadership Modeling
   • Leaders set the tone by encouraging experimentation, talking about their own learnings and be open about new ideas.
   • This signals that it’s safe to fail small and learn fast – essential for both rapid iteration and stage-gate innovation.
2. Rewarding Learning, Not Just Success
   • Reframe “failure” as a learning opportunity, not a personal shortcoming.
   • Recognize team members who propose improvements, run experiments, or take the initiative to solve tough problems – even if the outcomes are imperfect.
3. Inclusive Decision-Making
   • Collaboration should not be confined to Agile ceremonies or linear reviews.
   • When teams see their input valued, they develop a shared sense of ownership and accountability for the successful value delivery.
4. Ongoing Coaching and Support
   • Provide resources for professional development and skill-building, whether technical (e.g., TDD in software) or conceptual (e.g., risk analysis in linear planning).
   • Offer coaching at all levels, from executives to frontline teams, to reinforce continuous improvement.

A growth mindset changes retrospectives and phase reviews from simple checks of boxes into opportunities for a meaningful change. It brings people together to solve problems and deliver a finished product to the customer instead of blaming each other. This basic idea is the foundation of any successful method, whether it is Agile or linear.

Common Dysfunctional Patterns to Avoid

1. “We’re Doing Scrum, but…”
   …we never act on retrospective findings.
   • …our teams have no real autonomy.
   • …our increments never actually reach end-users for feedback.
   • This superficial implementation leads to cynicism and stagnation.
2. “We Put Up a Kanban Board, but…”
   • …we ignore WIP limits and never visualize the actual bottlenecks.
   • …our managers still use the board solely for micromanagement.
   • …we do zero system-level improvements, making Kanban a fancy to-do list.
3. “We’re Lean, but…”
   • …we focus solely on cost-cutting rather than building quality in.
   • …our leaders still operate in a top-down manner, stifling the continuous improvement spirit.
   • …employees fear retribution for surfacing problems.
4. “We’re Linear, but…”
   • …we keep forcing stage gates even when the requirements are unclear and keep shifting.
   • …we skip necessary reviews or cut corners on compliance to meet arbitrary deadlines.
   • …we miss the chance to incorporate incremental learnings in each phase.

In all these situations, the problem isn’t the framework itself, but the way people act and think in the organization that makes it ineffective.

Busting the Myths

• Myth #1: “Agile will fix our broken culture.”
  Reality: Culture is shaped by shared values, behaviors, and leadership actions. Agile can highlight issues, but it cannot fix them on its own.
• Myth #2: “Linear methods are outdated and irrelevant.”
  Reality: In regulated or standardized environments linear methods offer efficiency and clarity unmatched by iterative approaches. They remain the go-to for many standardized or heavily regulated industries.
• Myth #3: “We’ll succeed if we follow the [Scrum / PMBOK / Kanban] Guide to the letter.”
  Reality: Framework guides provide structures and best practices, but real success depends on how well you address deeper issues—vision, trust, technical discipline, leadership style.
• Myth #4: “Our teams can be Agile even if the rest of the organization isn’t.”
  Reality: You can pilot change in small pockets, but true transformation requires cross-functional alignment, executive support, and often a change in organizational design.
• Myth #5: “Agile is only for software; linear is only for manufacturing.”
  Reality: Agile principles such as iterative learning and collaboration can benefit marketing campaigns, sales workflows, and even parts of large-scale engineering. Conversely, linear methods have a place in certain software scenarios where scope and risk are highly constrained.

Strategies for Achieving Real, Sustainable Value

1. Contextual Assessment
   • Before adopting or overhauling any methodology, assess the complexity, volatility, and regulatory constraints of your projects.
   • Use frameworks like the Cynefin framework or a simple risk/complexity matrix to determine if Agile, linear, or hybrid methods make sense.
2. Strong, Aligned Leadership
   • Leadership must articulate a clear vision and be willing to change their own behaviors.
   • The shift to an outcome-focused culture often requires new incentives, metrics, and management practices.
3. Invest in Coaching and Facilitation
   • Skilled Agile ciaches can help teams adopt Scrum or Kanban effectively, but you may also need leadership coaches to foster cultural change at the executive level.
   • Similarly, project management professionals (PMPs) or Lean Six Sigma experts can refine linear processes and optimize organizational workflows.
4. Start with a High-Impact Pilot
   • Target a strategic project that can benefit from either Agile or improved linear methods.
   • Dedicate resources, secure executive buy-in, and ensure cross-functional collaboration.
   • Publicize successes (and lessons) to build momentum for broader organizational shifts.
5. Address Technical or Process Debt
   • In software teams, tackle code quality and automate testing. In manufacturing or construction, standardize procedures and ensure safety and compliance.
   • Remove the friction that impedes rapid iteration or disciplined linear checkpoints.
6. Measure What Matters
   • Shift from output-based metrics (story points, tasks completed) to outcome-based metrics (customer satisfaction, time-to-market, ROI, defect rates, risk reduction).
   • Align rewards and recognition with these outcome metrics to drive the right behaviors.
7. Continually Reflect and Adapt
   • Retrospectives or phase reviews should lead to concrete action items, not just discussion.
   • Encourage transparency: celebrate the success of improvements, and learn openly from failures.
   • Scale effective practices to new teams or departments once they are proven in pilots.

Measuring True Value in Any Approach

Ultimately, whether you’re sprinting every two weeks or marching through a linear plan, value must be your goal. A growth mindset and a supportive culture enable teams to constantly asking questions:

• What customer or business problem are we solving?
• How do we measure success in solving that problem?
• What feedback loops confirm whether we are on the right track?
• Are we prepared to pivot if the data suggests a change in direction?

By shifting the focus from “How many features did we ship?” to “What measurable impact did we deliver?” and “What problem did the customer solve with our product or service?” you are creating real value for your customers. Agile’s frequent feedback cycles make this easier in dynamic environments, while linear’s structured checkpoints make it easier in more predictable environments.

Conclusion: A Call for Contextual, Value-Driven Approaches

Agile – in the form of Scrum, Kanban, XP, or other frameworks – can be immensely powerful. Linear methods can be equally effective in the right environment. But both approaches can fail spectacularly if organizations ignore the core building blocks: leadership support, psychological safety, technical discipline, cultural alignment, and a relentless focus on value.

Instead of framing your delivery strategy as a either “Agile or Waterfall”, recognize that both are merely means to an end. The real question is: What is your organizational context and how can you most effectively create value for your customers?

Companies must move beyond the superficial adoption of ceremonies or checkpoints. They must aim for a growth mindset that empowers teams to continuously learn, adapt to new information, and refine their practices-whether iterative or linear-based on what truly drives results. When the entire organization aligns behind a clear vision and purpose, trust replaces micromanagement, and frameworks become enablers.

So the next time someone says, “We need more agility,” or “Agile doesn’t work,” take a step back. Look at your culture, your structures, your goals, and your mindset. Assess whether your situation calls for an iterative loop of inspect-and-adapt or the clarity and rigor of a well-defined project plan-or perhaps a mix of the two. Above all, keep asking: Where is the customer value, and how do we know we’re delivering it?

That’s the only metric that really matters.