The problem with Scrum

If you were asked this simple question

— Do you expect to be doing Scrum in 20 years from now?

your answer would necessarily be no. Because you know how much the world, and in particular the industry, changes in time.
You are quite sure that, if you were doing Scrum in 2045, something would be very wrong: i) either the world had frozen until then, or ii) you would be completely at odds with reality.
Now lets go for the fairly logical assumption that you will not be operating a psychosis 20 years from now. Its straightforward.
Therefore, you admit that you would be at odds with reality in 2045.
This leads you to the conclusion that you will surely want to move away from Scrum somewhere between today and 2045.
So you have to admit that, between today and 2045, you will be merely postponing the inevitable until you do away with Scrum.
Therefore, not only do you not have a reason to keep doing Scrum, but also the reason that made you do something else by 2045 is not affected by the timing. Besides this, Scrum was created more than 20 years ago.
Therefore, you arrive at the logical conclusion that, even if you stopped doing Scrum today, you would already be late.
Take from this that there is today no reason at all why anyone would be managing teams with Scrum.

The heading image of this article (from this source) is an illustration of how I see the daily Scrum ceremonies. For some surely remarkable reason, never have I heard a single highly qualified engineer question the following. The daily stand-up is performed, they believe, because it is important, that is, because the team does something in those ceremonies that is important enough for them to do it everyday as clockwork. Now if this was the case, then one should also admit that this very important thing, whatever it is, would be performed many times a day, because it is important, that is, surely not just one single time each day for just about 15 minutes. We don’t do important things just 15 minutes a day at a pre-set time, we do them whenever circumstance justifies so, precisely because they are more important than the other things, so they take the time of the other things when they need to happen. Surely that importance is incompatible with delaying an important matter discussion until the next day huddle. But if we assume this is the case, we should ask why then do we need a daily stand-up?
We have to accept there is some other reason that distinguishes this Scrum ceremony from the supposedly same things that are done during the workday. To understand what that is, we need to look at the specifics of the interaction in the daily stand-up that distinguish it from the other interactions in which the same activities are performed. Firstly, in the daily stand-up all the team is to be present. Second, in Scrum (but unlike the more evolved Disciplined Agile), the team lead is a necessary participant. Can you guess now what is the single thing that justifies the daily stand-up? Its status reporting. Its control. And to this they call “agile”.
This “agile” is more waterfall than Waterfall itself, because in Waterfall we only prescribe daily control over a project during critical periods. Otherwise the timing for the heartbeat is weekly. And to this they call “agile”.
In Lean management you can understand better the abomination of this practice. In Lean, practices follow their own cadences and they happen when they need to instead of merely because the calendar says so. We do not block the flow of work and value with a rock in the middle of the stream every 24 yards.
But as I said, never have I heard a single highly qualified engineer raise this issue, much less solve it. This is of course, because empirical control processes are designed to suppress conflicts about the governing variables. A form of functional lobotomy. And to this they call “agile”.

My inquiries into the industry “agile” standards are grey and old, and that is why you can also read about it in this article.

The rest of this article is based on Al Shalloway’s approach to what is now called Flow for Enterprise Transformation, the state-of-the-art knowledge system that his consulting company developed during the most part of the Agile experiment since its inception, and that was integrated by PMI in its approach for Agile management called Disciplined Agile (DA FLEX). I had three certifications in this method and that is where I come from in this regard. I took the freedom to change very little of Al’s words because they are so perfectly stated, while also adding a few of my own.

Let’s say you were going to buy a car. You walked into the showroom and saw amazing cars and were really excited. You buy the car, have a lot of troubles driving it and when you take it back they respond with:

— Yes, the instructions are simple, but driving the car is difficult to master.
— The car is so sophisticated, it’s not possible to have full predictability.
— Well, we thought this aspect of the car was good enough.
— Most people don’t have trouble with this, but since you do, we have this extra education for you.

How would you react to that? These four attitudes are prevalent in many Agile approaches. If you are a practitioner and wouldn’t accept this for a car I suggest not accepting it for the approach you’re taking to improvement. If you are a consultant, please stop doing it and please help others stop.
Saying something is difficult to master is insidious in that it gives those promoting it permission not to try to get better.
Instead of resign ourselves to a lack of predictability, we should see how much we can get and always get feedback. Looking at a complex problem in a different way often results in a solution that is predictably good. But we’ll never find it if we don’t look for it.
Good enough and too much are not the only alternatives; the use of minimum business increments (MBIs) proves it.
When people don’t understand something about Agile (e.g. tying the big picture to the small task) perhaps it’s not an indication of the person’s inability as much as it’s an indication that something is missing. Scrum has a different education and framework definition approach than Lean, Kanban, Theory of constraints and Flow have, and these are built in DA FLEX but not in Scrum.

The deadly synergy of the four attitudes above results in the following:

i) When someone doesn’t understand something, this attitude will make it easier to just think it’s good enough and just try to explain it better instead of trying to improve it.
ii) When people have difficulty with an approach it’s easier to just shrug our shoulders and explain that it’s difficult to master.
iii) While we can accept that predictability is difficult we should not accept less predictability than we can because our understanding is good enough.

The focus must always be on improving methods and not on following a framework.
The goal is not to be agile at the team level but to achieve business agility across the organization. This manifests both individual and organizational purposes and creates a better place to work. Optimal teams without improvements to the organization is not that useful.

According to the Scrum guide, Scrum is founded on empirical process control theory, or empiricism. Surprisingly, no one seems to challenge that empiricism may not be the best way to base a framework on. On the other hand, DA FLEX is based on the scientific method to overcome the inherent possible cognitive biases. The scientific method is a combination of empiricism and rationalism. It uses empiricism to support or invalidate hypothesis while using rationalism to enable it to create conjectures (i.e., new hypotheses) about how the world works. This combination is more powerful than empiricism alone while being less risky than relying just on academic reasoning.
This mirrors the following paraphrase

Theory without experience is useless, experience without theory is expensive.

Edwards Deming

Several Agile methods, particularly Scrum and most of its derivatives, are based on empiricism, which, while simple to explain, miss many opportunities for a deeper understanding of what is happening. This is not to say we cannot apply the scientific method to Scrum, but doing so requires examining its immutable roles, events, artifacts and rules, possibly leading us to move away from Scrum.

There are a few significant implications of a framework taking on a scientific approach to itself. These include:

i) everything we put forth in the framework is a hypothesis
ii) instead of defending our methods, we welcome, even request, dissent
iii) there cannot be any immutable aspects to the framework

The same way one does not need a complex approach to address a complex problem, as posited by Scrum, one also does not need an empirical approach to address an empirical problem.
DA FLEX is based on the scientific method of making hypotheses, observations and adjusting the hypotheses to fit the data. But no belief or practice is taken as sacrosanct. And deductive logic is encouraged while still requiring the logic to be validated by experience. In addition to this, while Agile was designed and intended to be applied to teams only, DA FLEX takes a systems-thinking point of view which is needed to address organization-wide transformation.

Scrum is a framework. While you are encouraged to add to the framework, certain practices of the framework are sacrosanct – and you are not supposed to change them. As with all frameworks this is to enable a well-defined starting point, however starting with a set framework with core practices that you are not supposed to change tends to have you stick with them longer than you should. Also, not all of Scrum’s practices are applicable everywhere.
While frameworks allow you to add things to them, you often need to change the framework itself. Not all of the prescribed practices in Scrum, which one must do in order to be “doing Scrum” are optimal, or even possible, in all situations.
When a team is being led by someone with only a Scrum certification, they may be reluctant to go away from Scrum because it has them go into territory in which they are not certified and may feel uncomfortable going outside of what they are certified in.

It is difficult to get a man to understand something, when his salary depends upon his not understanding it.

Upton Sinclair

When and how to transcend Scrum is left unanswered by the Scrum Guide which repeatedly encourages following Scrum practices. As the empirical process that it is, Scrum is a particular case of the more general “single loop learning” approaches as PMI termed them. These processes are designed to achieve intended outcomes and suppress conflicts about the governing variables. This means that, while you can make adjustments based on the results you get, you are not allowed to inquire about the method itself, therefore you are left managing a project with the same intelligence as a thermostat manages temperature. You cannot develop a new method even if one would lead you quicker to a solution to the problem you are facing.

We cannot solve our problems with the same thinking we used when we created them.

Albert Einstein

That lack of options to transcend the framework is justified by the seemingly logical statement that “we need a well-defined set of practices; adding options will cause confusion.”

But this is based on the unproven assumption that options have necessarily to be confusing. This is especially relevant to note in the cases when we are managing knowledge workers, because then it equates to a disenfranchisement of the most important assets of an organization — its people doing the work and creating value. You do surely not fear giving options to highly qualified people doing their jobs. In fact, you should fear the opposite, especially if you are paying their wages or waiting for them to solve your problem.
The answer to the claim above is that giving options is not the problem; the challenge is knowing when to give the option. Too soon and people may not have the opportunity to exploit their current approach, but too late means they are stuck with the same method for longer than they need while other alternatives may be better suited for the challenges at hand.
This becomes ever more impactful the more a team develops maturity in Scrum practices. A simple starting point when it needs to be is good, but we have to enable better learning as people get more advanced.
The key after the start is to provide alternatives now that the people know what the objectives of the Scrum practices are. This requires that the people adopting the practices have been given the objective of the practice. At this point, with a little experience under their belt, a deeper set of options can be provided. The key here is not to abandon practices, but to find better ways of doing what you are doing, getting to the root cause of the issue or take on a new practice if needed.
One doesn’t have to blindly follow Scrum but can use Lean-Thinking to provide better, more cost-effective practices to accomplish the intentions of Scrum.
Regarding navigating complexity, Goldratt and the Theory of Constrains identified the term “inherent simplicity”:

If we dive deep enough we’ll find that there are very few elements at the base – the root causes – which through cause-and-effect connections are governing the whole system.

Eli Goldratt

Scrum is also based on the premise that if you change your organization to follow the Scrum framework, Scrum theory, and Scrum values, then your organization will improve. We believe instead that the right approach is to make a decision as to when it’s more effective to change the organization to fit a framework or when to follow different practices that meet the same objectives of the framework more effectively. Of course, this may take you out of the Scrum framework, so you’d no longer be doing Scrum. However, we don’t think this matters. We should remember that we should focus on our objective and not worry about the particular method of getting there.

Struggling with breaking down stories into small chunks? Scrum intentionally provides no guidance here – it is a framework. Unfortunately, standard decomposition methods don’t take advantage of what’s been learned in the last decade with Behavior Driven Development (BDD). BDD provides a structure for decomposition that enables virtually any problem domain to be decomposed into small stories.

If we see people consistently make the same mistakes we should not attribute the failure to the people, but more to the system. This was established 80 years ago by Deming when he stated that the eco-system people find themselves in causes most of the challenges they experience. Our responsibility therefore is to improve the system, more than managing the people.
This is the anti-thesis of what we hear from many Scrum consultants: “Scrum would work if people would just do it.”

That may be true, but Scrum may not be the best approach for the problem a team faces. This approach retains the problem while modifying Scrum to make it invisible so that the dysfunction is no longer a thorn in the side of the team. There is an implication that if the team would put more effort into it, or take the time to figure Scrum out better, we could avoid these. This is a big, assumption.
Our attitude shift would be to ask “why is it so hard to fix this impediment? Is there a better approach, even if it means going beyond Scrum?”

In particular, consultants and promoters of frameworks must take responsibility when practitioners have trouble. This means to instead keep improving our offerings so our clients have fewer challenges.

Neither the Agile Manifesto nor the Scrum Guide mention the role of management once. The role of management is critical, however. It is ironic that Agile is about changing culture but either ignores or vilifies those in the best position to do it.

To contrast with all of the above, we can say the following regarding DA FLEX.

DA FLEX takes the attitude that the overwhelming majority of people are good and, when put in a good system, will achieve good results.
DA FLEX embraces the mindsets of Theory of Constraints, Lean and Flow, seeing no inconsistency between these. While these may be focused more on what we do, DA FLEX also incorporates models of personal and organizational development, which go well beyond the Agile mindset.
DA FLEX is not based on how to take Agile and scale it across the organization. It starts with a holistic view of the organization to begin with.
DA FLEX takes a pragmatic approach, meaning it focuses on what works, which stems from its scientific nature.

Another source for consideration is impartial and unrelated with the PMI. The BPM CBOK v3.0 [1] cites the concept of “Big Process” by Forrester, as a transformation by means of business processes that changes the organization’s focus from isolated BPM actions and incremental improvement initiatives, towards a transformation program that includes the entire organization with the support of executive leadership. This is the vision of the value stream. It is the performance of cross-functional processes, and not of functional areas or assets, that drives the achievement of true results. The broader the BPM initiative in the organization, the more effective it will be in delivering value. This is what DA FLEX is about.

Now for how long will you still be doing Scrum?

_________

[1] the Business Process Management Common Body of Knowledge, by the Association of Business Process Management Professionals