The ‘What’ of Scientific Development

The key is that our hypotheses are not deliverables. They are ideas seeking validation. This twist has important ramifications for how we end and enter sprints. At the end of a sprint, we can look at what we’ve done and for each hypothesis we can say: “yes” or “no”. Our planning is about what we will learn, rather than what we will do. “No” – We Didn’t Fail, the Hypothesis was Wrong By focusing on what was learned, scientific development tries to make “failing” much safer by changing what it means to fail. Today, failure is not delivering a solution. It sounds and feels like the team didn’t work hard enough, didn’t estimate accurately, or otherwise failed. This becomes a feedback loop that discourages risk taking and discovery. Ramifications for this feedback are obvious and very likely all around you. Concerns are about planning, estimation accuracy, faster execution, and more specialization. With scientific development, ending a sprint where a hypothesis results in a “no” is merely disproving the hypothesis. The hypothesis was wrong. Our technique was likely fine. Our work ethic was fine. Our idea was the thing that was wrong. In fact, in many ways this is a highly desirable outcome. For every “no”, we should have an in depth review of how we arrived at the hypothesis, how we tested it, and our results (code and all). This review will offer innumerable learning opportunities for those exploring the hypothesis while building the coaching skills of the team. Despite that, when we arrive at a “no”, we should use our learning to create a new hypothesis, which we can test in a subsequent sprint. The only outcome that does reflect poorly on the team’s performance, is when they don’t know the answer at the end of the sprint. “Yes” – Doing to Learn > Learning to Do With all this talk of proving a hypothesis and repeated hypothesis refining, you may be concerned that the team will spend all of it’s time thinking rather than doing. Those are fair concerns and can happen. They can also happen when writing stories or even in a classic waterfall project. Ultimately, it becomes a matter of facilitation. Proving a hypothesis is not a matter of citing a question on Stack Overflow and saying “yup, I saw it on there”. Proving a hypothesis means providing a working solution that definitively proves the hypothesis in the proper environment. If it’s starting to look as though proving your “yes” or “no” can involve a great deal of work, then good. It can. That’s the point. Scientific development is not about doing less, it’s about optimizing the people rather than the work they’re doing. Planning – A Different Conversation Sprint planning would also take a subtle, yet important shift. Most sprint planning looks at what needs to be done and ask who is best suited to do it. When we’re looking at which hypotheses we want to test, the question is less about “who wants to get this done?”, and is more like “who wants to learn this?”. That is a small, yet critical shift. The very nature of sprint planning could be pivoted from being a conversation about what we will do and turn it into a forum for helping people develop the skills they want to acquire. It could be the first step in unifying the act of work planning and team member growth. Of course, the convention alone won’t be enough to completely unify these things, but that is not the point. It’s a tool to assist you in becoming professional learners and experimenters rather than coders. It also takes learning and growth out of one-on-one and annual reviews and makes it part of the sprint process. Because planning is a team activity, this also means that everyone on the team will know what each other are learning and how each other are growing. This makes it possible for the entire team to support each others’ professional growth, rather than leaving that to leaders. In some ways, the team can’t help from doing that. If the team deeply reviews and offers feedback on all “no” results, then there is no escaping team members being perhaps the greatest asset to each others’ continued learning and development. Read On The ‘Why’ of Scientific Development The ‘How’ of Scientific Development The ‘Who’ of Scientific Development

The ‘Why’ of Scientific Development

Can we use our backlog for professional growth by tracking and talking about what we want to learn? Stories are nice, but… As an agile coach I’ve spent a lot of time teaching and helping people practice writing stories in the classic format “As a […] I Want […] So That […]”. The virtues of this format are well known, so I won’t waste words there. But, what about the shortcomings? Keeping Busy Because stories represent conversations, they’re intended to defer discovery until the last possible moment. However, classical thinking still demands that we think of them as deliverables. Once we begin down that road, it can be very easy to begin treating stories just like a queue of work to keep people busy. We’re not comfortable with uncertainty, so we research everything then write and estimate stories for projects upfront. Roles and teams get put in place to ensure that the stories are written, vetted, and prioritized with enough of a buffer that the engineering team never stops coding. In planning, we question whose best suited to get work done so the most value is being delivered. After all, we want our velocity and utilization to be high. We optimize ourselves for speed and turn ourselves into a small team of specialists. You Get What You Optimize Those outcomes aren’t accidental. They are a direct outcome of trying to optimize a system to maximize the utilization of a limited resource (team member time). We want to keep them busy. Unfortunately, optimizing to keep everyone busy, makes it very hard to manage other aspects of our work. Are we doing the right things? Are we doing them the right way? Are people growing? Of those, concern about team member growth is the biggest miss. Most companies know this already, and to fix it they add team member development as a separate activity. Every year there is a review. In many companies, every few weeks there’s a one-on-one meeting between the team member and leader. These are the forums for facilitating growth. Ironically, as we make headway with iterative delivery and agile practices for our work itself, our people development is still firmly rooted in a waterfall-like annual cycle with few feedback loops. A Backlog of Growth & Learning Opportunities What would it look like if our backlog wasn’t a list of things to do? What would it look like if it was a list of things to learn and ways to grow? What would it mean to our organization if leaders were prioritizing and discussing opportunities for team member growth rather than deliverables? These are the questions that led me to propose the idea of scientific development. Read On The ‘What’ of Scientific Development The ‘How’ of Scientific Development The ‘Who’ of Scientific Development

Jidoka and Captured Knowledge

In manufacturing, you likely know the problems to look for; broken threads, misshapen parts, etc.. In software, you may have no idea what kinds of problems you may create by changing code. In this way, practicing jidoka is invaluable. It’s how later developers know what can break, what is broken, and offers wisdom about how to fix bugs without breaking something else. How? Captured Knowledge. Each automated test represents knowledge captured from previous engineers about how the system was expected to operate in various scenarios. This is actionable knowledge that all future engineers can make use of. It lets future engineers test their changes and immediately learn what (if anything) has been broken without first needing to learn all the nuisances of the software. It is the period when learning how the software operates that greatly extends engineer ramp up, because too often even simple debugging requires direct use of the application and guessing through which scenarios may invoke the altered code. Without that captured knowledge, the future engineer has to re-learn everything that was known in the past because the raw code does not capture the nuances of what is expected or required. In this way, the knowledge in the tests acts as a safety net. It’s also knowledge the engineer does not need to fill her brain with by memorizing, because he test can be relied upon to act as an institutional memory. Actionable Knowledge This is not documentation for documentations sake. It’s a used and actionable part of the process that cannot be ignored. If you fail a test, the build should fail and block your progress. Unlike an external document, which simply becomes out of date. In some ways, that build break acts as an andon call, alerting you to having missed a standard step (documenting the behavior of your code). Flexibility Creates Choices (and agility) By offering the wisdom of all engineers who previously worked on the application, we greatly reduce the learning curve needed for an engineer to become a productive team member. This means that we have much more flexibility in terms of how many add to (or take from) an application. If we add engineers, the drain on those already on the app will be more modest. The tests provide a safety net which can enable a bit more autonomy win confidence. We can also add more than just a few since the bank of knowledge from which they can draw isn’t limited to the brains and mouths of the senior engineer on the team. If we remove engineers, we shouldn’t need to lose too much sleep over having them forget their current application. The test bank should go a long way in preserving the most critical information. By making this investment in our code today, we gain flexibility around where to invest our effort for the biggest wins tomorrow. That is the heart of agility. Pursing our greatest opportunities today in a way that won’t stop us from pursuing better opportunities tomorrow. Foreshadowing There is still more to say about how they enable speed. We will look at that next time.

Jidoka (automation with a human touch)

The two pillars of lean are just-in-time, and jidoka. Today we’ll look for jidoka in software development. What is jidoka Often translated as “automation with a human touch”, some people also use the word autonomation. The principle is rather simple. When designing a process, find opportunities for the process to self identify errors and stop so that the problems can be fixed at the source. With this in place, human attention can be focused on either finding and eliminating root cause for the error or improving the task itself. This means we’re investing the human mind where it will do the most good. Example Perhaps the most common example of jidoka is the looms made by Sakichi Toyoda. These had the ability to detect thread breakages and stop. This meant that humans could then intervene to fix the issue and let the loom resume. Until there was a break, the advanced looms could be monitored, with many looms sharing one operator. This was a vast departure for it’s time, when any one loom required a great deal of energy from a single dedicated operator, whom may fail to notice thread breakages due to the many other aspects of weaving that required their attention. Software & Agile The parallel in software delivery is hopefully rather obvious; automated testing. This can be in the form of unit tests, integration tests, or most anything else that will allow human creativity to be applied to novel situations and the automation can be left to handle the monotonous work of simple testing. If our QA team members automate a common test, they can now spend their time looking for more subtle errors, problematic interfaces for humans, or other validation. Just like the loom operator, that test gives them the ability to use their human mind on more valuable things that a machine cannot so easily do (like building more automated tests).

Muri, Kaizen & Retros

When do you most need a retro? When you feel as though you don’t have the time for it. Muri In lean, the feeling of stress (or the impossibility of your task) is called “Muri”. It should serve as a powerful indicator that something is terribly wrong. If you feel this as a member of the team, it is your duty to stop what you’re doing (pull the metaphorical andon cord) and have a serious conversation. In common agile parlance, that is a retro, but it could just as easily be called kaizen. There may be many reasons for such a feeling. Perhaps your deadline is unreasonable with the current scope. Perhaps you are trying to centralize something on a small team that cannot support the load being put upon them. Perhaps you feel like you’ve nearly solved a problem, but have felt the same way for days and the solution continues to be elusive. Problems Just as your problems don’t nicely conform to a script such as “what went well” and “what didn’t”, your retro does not need to be so narrow. Be willing to reflect on what you’re doing, how you’re doing it, how you feel, and “why”. Be willing to have an open conversation about all these topics whenever necessary. Problems may not always arise on a two-week schedule and if they can be solved sooner then everyone is better off. Remember that the retro is also about finding solutions (or starting toward them). If you consistently talk about the same issues but take no action, that can be a source of muri! Beware of Justification Metrics and intrinsic motivation can deceive you. Your desire to “just finish it” can undermine your long term success. Your drive to do things “this way” despite lacking the time or team can make you sound like a victim. Don’t allow these justifications to enable muri. Listen to how you feel. If you feel that something is wrong, you may be telling yourself something your brain cannot yet comprehend. When tackling problems, be careful of “solutions” that justify your stress or avoid unpleasant conversations. You must be willing to accept the possibility that your perspective may be ill suited to this specific context. You must be comfortable being uncomfortable so that you can engage in the conversations that must be had. A great many problems can be solved by merely achieving a common understanding. Conversations, especially those that are unpleasant, can be the best tools in achieving this. Mirages & Solutions The solution may not be to bring in more people help the project get back on track. That can make things worse. You may be better off having the hard conversation about scope or dates. At the least, this can help you understand the relative costs of delay between the deliverables so that wise trade offs can be made. The solution may not be to expand the centralized team substantially so that they can manage all deployments for an enterprise. You may not want to staff a team of release engineers like a call center so that every deployment request can be handled quickly. You may be better off having the centralized team act as consultants to the de-centralized doers rather than doing the work themselves. This can empower the team and accelerate the progress they were aspiring to. The solution may not be to hire a consultant to solve the problem for you. Rather, all the talent you need is probably already there and you simply need to discover how to find it. This does not inspire confidence quite like rented credentials, but it gives you a chance to better learn about the team, it gives someone a chance to showcase their ability, and it gives someone within your team a growing and learning opportunity that you had been giving to the consultant. When do you most need a retro? When you feel a pit in your stomach whenever you think about it. When you cannot find the time due to the amount of work that needs to be done. When you’re dreading the conversation and just want to avoid it. When you’ve tried a dozen quick fixes and none of them have fixed it. These aren’t signs to delay. These are signs at you’ve delayed too long.

Takt Time & Waste Elimination

How does the use of sprints (takt) eliminate waste? By reducing Mura & Muri. Mura – Variability Imagine a lean auto plant with a 60 second takt. Expectations for everyone are clear. You will be done in 60 seconds. If you cannot consistently do your job in less than that, there is a problem. In 60 seconds, the next car will be ready for you. Imagine if my job takes 60 seconds, but the next car arrives at random. Sometimes it arrives in 20 seconds. Sometimes 300 seconds. I will need to worry about this. My attention will be split between doing the job and tracking incoming cars. I may even build special tools or modify the line to include staging areas. Doing this will help me manage the work, but this is an infrastructure I’m putting in place to help me manage the variability. It isn’t adding any client value. Unfortunately, I need to do this because I cannot be assured that I have enough time to do my job. If the next car is coming quickly, I may need to cut corners and just move on to be ready for it. It’s similar for creative teams. If we simply take work as it comes with no process to control it, we will invest a lot of energy into knowing and tracking the various due dates, conflicts, and needs. We may build waiting queues and layers of vetting to “speed things up” once we’re ready to start the work (if you think this will work, look at the system optimization that is lost in a production cell). A lot of mind power will be invested into just keeping everything in motion so that nothing appears to be failing. When deadlines conflict, we sacrifice testing, unit tests, or code reviews (or worse) to keep up. If we create a predictable pattern to enable delivery, people can start investing their mental energy into client problems rather than tracking the variations in our planning processes. There will be more about this tracking (managing work variation) in other articles. However, this takes a great deal of discipline. Not only must you respect your takt, but you must balance the work so that you are able to consistently deliver to your quality standards (testing, unit testing, etc.). There will be more about this balancing (managing skill variation) in another article. Muri – Stress It is that propensity to take on too much work, which creates so much stress for white collar workers. In manufacturing, the line controls flow. You can’t physically have two cars in your work station. For software engineers, you can have the entire backlog in a “doing” state and nothing will stop you. Juggling between what’s “on deck”, “getting started”, “wrapping up”, and “being supported” can spread your attention thin. The context switching alone can wear you down, let alone the greatly reduced ability to get things done due to the sheer volume of things you’re trying to do. Sprints offer a way to overcome this. At the sprint’s beginning, we plan. We plan as much as we can reasonably complete and focus on the most valuable things we can do in our limited time. This protects us from having the entire backlog in process. It focuses us onto a body of work we can achieve. This can greatly reduce the stress on team members while ironically increasing throughput by eliminating much of the context switching. In software development, the formal spring planning limits the “in process” work so it can’t overwhelm the team. If that seems overly simple, it is. That’s also why it’s powerful. Sprint plans help protect white collar workers from themselves. It makes sure their commitments are reasonable targets for completion, and this gives them both focus and satisfaction every sprint. Challenge Does your sprint process control variability in a way that enables focus? Does the team limit he work in each sprint to that which can be completed?

Takt Time & Sprints

Takt is a German word that has been adopted by lean. The original meeting is “beat” or “rhythm”. In an auto plant we may see a one minute takt. This means that every minute a finished car rolls off the assembly line, a dashboard is installed, a car is painted, etc.. Software development cannot possibly resemble his repetitive highly orchestrated pattern, or can it? We’re not a factory It is exactly this image that most creative and white-collar professionals abhor, and for good reason. Each request in their environment is unique and will require an unknown mix of skills to get it done. It is not like building the 109,345th Camry. The reality of what it will take will also not match our initial estimates. Toyota knows exactly how long it takes to make a Camry. How can we create a repetitive process if our work is inherently not repetitive? Rather easily if we accept that the work itself is unique, but our process for handling it doesn’t need to be. Timeboxes, Planning, & PDCA Although being agile doesn’t require “sprints”, it does require planning, frequent delivery, quick feedback and reflection. If you simply put consistent timeboxes around those things, you get sprints. A sprint is a PDCA cycle. We get an idea for how we can deliver the most value in the timebox. We execute, get feedback, reflect, make changes for how we’ll execute the next cycle based on our learnings and repeat. We use sprints to accept that the work itself is unique and cannot be standardized like an assembly line, while saying that the way we plan, execute, track, and reflect on that work can be standardized. In this way, we improve the way we do the work and establish our takt. Sprints as Takt Takt sets the cadence in lean – people need to fit within it and improve. Agile sprints do the same. You know the duration, every time. You improve how you execute within them, regardless of the type of work it is.  The need to juggle multiple competing priorities is greatly reduced. The priorities for the sprint were made clear in planning. In lean manufacturing, there is only one line. Which car do I work on? The one in my station. When will the next car arrive? In one minute. When is this one due? In one minute. Does this ever change? No. Your sprint process should offer similar confidence to team members. We should be able to feel the takt and all our activities should align to it. In agile, there is only one sprint. Which request do I work on? Those committed in sprint planning. When will the next request begin? At sprint planning. When is this one due? When the sprint ends. Does this ever change? No. Challenge What is your takt? Do you respect your takt? Does everything fit within the cadence? Is your takt the heartbeat of your team, or that of the organization?

Knowing why OVER knowing what

What does it mean to know why over what? What I do A focus on “what” we do creates many problems. It’s easy, though. It’s the first thing that comes to mind. I write code. I test. I gather requirements. I manage projects. Unfortunately, when we do this, it’s easy to lose sight of why we do it. You’ll notice this when project managers take people off the floor for hours (or days) to get the plan perfected. They’re so focused on what they do (manage the plan), that they inadvertently take too much time and attention away from value delivery. The client doesn’t want a plan. We will use plans, but the value we’re delivering is software. 7. Working software is the primary measure of progress. This isn’t a project manager problem; it’s a human problem. Engineers may try to sequester themselves for weeks to perfect something because “what” they do is write code. They see all those client conversations as waste and don’t want their skill to be called into question if there’s a defect. Clients want problems solved and value delivered today, not something perfect years from now. 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. We could harass analysts and testers as well, but hopefully you get the point. Thinking about what I do is a distraction from our real purpose. What To Do Too often teams have a backlog of solutions awaiting engineers to simply write the code. This is a symptom of an organization built around “what we do”, but the practice can exist in other environments. A focus on “what needs to be done” is destructive. It makes the assumption that we understood the problem when we wrote down an answer. It assumes that the need and the tools won’t change between writing the answer and implementing it. It assumes, that those designing the solution know the right architecture better than those implementing it. It assumes that the engineer has no creativity to bring to bear. It assumes we don’t need to work with the client anymore, we already did. It assumes we won’t learn more about the solution as we go. We should. 11. The best architectures, requirements, and designs emerge from self-organizing teams. The key concept here is emergence. If we focus on what to do, we will be anchored on a particular solution. Our goal should be tapping into our collective talents and creativity to arrive at the best possible minimum solution. No matter how smart or creative the person who wrote the solution, a team of skilled and creative people working together will create a better solution every time. By focusing on why we are doing the work (the problem that needs to be solved), we invite collaboration. We invite creativity. We invite better solutions. Using Why It can be hard to focus on “why” over “what”. It’s easier to “just write code” or “just manage projects”. Worst of all, the constant focus on “why” can turn into a tireless process of always questioning everything. To be successful, you need to get more comfortable with this. But there are some simple steps you can take. The easiest, is to start practicing writing and working with user stories. These are a good way to think about why we’re doing things to invite a discussion. As to what we do (our actual role) and how we invest our time, there is no easy answer. Always ask yourself if what you’re doing is adding he most value to the client of all the options in front of you. The horizon for this measurement should also be near term; today, this week, this sprint. That’s it. If what you’re doing is adding the most value for six months from now, you’re doing it wrong. Challenge Is what you’re doing today the most valuable thing you can do for the client? Are you doing the work you have because it’s your role, or because what you’re doing is contributing client value? Is there some other role you could take on, some assistance you could offer, or something else you can do that would deliver more value to the client than you are today? Why do you do what you do?

Working with OVER working for

What does it mean to work with a client as opposed to working for one? Handling Unknowns Consider a situation when there is a question or some uncertainty about how to solve a client issue, what does the conversation look like? Is it just you? Does the delivery team have a meeting? Do a few analysts have a debate? If you’re answering “yes”, then you’re working for a client. Rather than talking amongst yourselves, you should be talking with the client. Bring them to you to show them the actual problem. Have a conversation and decide together how to proceed. 4. Business people and developers must work together daily throughout the project. By discussing it without your clients, you’re making their decisions. You’re assuming you know the trade offs they want made. You’re assuming you deeply understand their problems. Stop assuming. They know their needs better than you do. If you disagree, try talking less. When you talk, you are only repeating what you already know, but if you listen, you may learn something new. J. P. McEvoy Scope Creep If you’re anticipating things your clients might need someday, then you’re working for a client. Why not spend time with the client to explain the problem you see and the solution you propose? It may not be a problem to the client. They may simply not share your concern. Alternatively, they may see the problem now, and appreciate your candor. 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Regardless of the outcome, you shouldn’t be deciding to add anything, especially not because the client may want it later or because “clients don’t know what they want”. For your own sake, always keep in mind that your time is limited. Make sure that whatever you’re doing will have the biggest impact possible. It may be that the problem you see, the client has already seen. They may have made the conscious choice to not do it because your time is precious and there are other things they want more. By talking, you can learn these things and they can learn from you. Together you can make sure that the right things (and only the right things) get done. 10. Simplicity–the art of maximizing the amount of work not done–is essential. Show & Tell When you have a show and tell, are you showing and telling or showing and talking? There’s an important difference. You’re telling if you’re using a rehearsed script, glazing over rough spots, embellishing the good things, and generally trying to deliver a message. You’re taking if you’re forthright about the gaps, seeking feedback, and trying to make sure the client understands where you are and you understand what they think of that. Our goal with the show and tell is more than just a demo. It’s making sure that the client’s needs are getting met and if not, that we’re adjusting. 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Keep in mind, if you’re really working with your client, there will be few (if any) surprises at the show and tell. If you’re accustomed to seeing surprised expressions, rethink the relationship you share. Try working with them. Challenge Do you reach out to your clients often, or do you try to find ways to delay or get out of client conversations? Do you think the client is an important part of the process, or do you think the client needs to get out of your way and let you do your job? Do you think you know best and your client needs to follow your lead? Do you think that the best knowledge comes when considering your expertise and the client’s perspective?

Why Manifesto

Because people in software like “manifestos”, try this as a litmus test for the decisions to make sure you’re thinking about value and not just staying busy.  As in the agile manifesto, the things on the right have value, but not as much as the things on the left. Working WITH clients OVER working FOR clients Knowing why OVER knowing what