Archive for the ‘Software Development’ Category

Agile Practices: Retrospectives

June 19, 2019

Looking today at the other side of the timebox, after the timebox finishes, does your team take a moment to reflect, review, or celebrate?  If you are not, you’re really missing out on a great opportunity to allow the team to see their successes, to self-improve and for management to actively work for the team.  Further, you’re not following the Agile Manifesto’s 12th principle:

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Allowing the team to see their successes is vitally important.  Engineers are all too often focused on the current and next sets of problems.  They rarely see what it is that they’ve actually accomplished.  It is even more rare for an engineer to meet with a real end user, to see what they’ve built being actively used.  I personally have found those opportunities to be hugely energizing, but have only had them occur occasionally.  The retrospective gives the team a chance to identify things, people, situations that worked well, exceeded expectations, or were all around good.

The reasons for not conducting retrospectives usually involve team negatives and timelines.

There isn’t time in the schedule

Your timelines are so tight that a 1 hour team meeting puts the schedule at risk?  It is too invasive to have the team come together and find ways to work better?  You really think that how things are done today is the absolute best way?  There’s no possible improvements?  The team is too untrustworthy to constructively find improvements in themselves, or their processes or the environments?

But, there is time for the weekly leadership/status meeting where the team is told they are behind schedule and need to find ways to get this whole thing done more quickly?  There’s time for the quarterly townhall where they are told how awesome the company is doing and how important their work is to the continued success of said company, Rah! Rah! Rah!?  I would suggest that you can afford to squeeze an actually useful meeting into your schedule, especially one that can reveal useful actions to positively affect the development effort.

We’ll do it after each release

This is a step in the right direction.  However, waiting until the end of a release cycle (sometimes 3 or 4 2-week sprints, yes? So, 6- 8 weeks?) is fraught with near term bias issues as well as delays in affecting change.  It really needs to be done after each timebox.  That said, depending on your situation and how directly the team is involved with production deployments, turning the release activities into a celebratory conclusion of many weeks of hard effort can count for a fair amount of team building.  It can also help create a culture where everyone on the team becomes willing to pitch in to ensure everything goes well with the deployments.

The team is so hyper critical of each other

They likely need guidance on how to be constructive during this process.  Too often retrospectives become opportunities for upset team members to call out others.  Each team culture is going to be different, but we can never allow anyone to single another out negatively.  Criticize the tools, the process, or the environments, but not each other.  It just isn’t productive or useful.  If this is all new to the team then some grace and forgiveness should be allowed, but perpetual disregard for this should not be tolerated and the offending team member should be removed from the meeting.

Management tears down the team

This is one of the main reasons why I prefer retrospectives to primarily be the immediate development team, the scrum master and the product manager.  The rest of the stakeholders can read the summary notes and recommendations.  If you have excellent relationships and communication with the other stakeholders, then ok – that’s actually how it is supposed to be.  But most teams don’t have this ideal.  Having the management team criticize the team directly and consistently typically doesn’t improve anything.  Worse, in these situations the team’s recommendations and advisements are usually dismissed or ignored.

What you have here is an adversarial situation where the team isn’t trusted.  If the team isn’t trusted we’ve also lost track of the 5th principle:

Build projects around motivated individuals.  Give them the environment and support they need, and trust them to get the job done.

We need to trust the team

And we need to help them constructively get through the retrospective.  We need to first encourage the identification of successes during the increment.  I don’t care how badly the sprint went, some things went well or were done as planned.  Secondly, we need to keep the team’s identified improvements focused on tools, processes, environment, etc., and away from personal attacks.  This perspective is critical.

I like to see retrospectives in a closed meeting room with a lot of 3″x3″ post-it notes, preferably in two different colors.  The size of the note restricts most people to simple phrases or a sentence or two and the colors are used to identify successes and improvements.  Each team member must put one up for each, but no more than 3 for each.  YMMV here but the idea is to require the team to think in both areas, but limit the totality of things that need to be categorized.  The scrum master then reviews them all, talking through each with the team to ensure that it is understood, and categorizes it with the team’s blessing.  Finally, we re-walk through the categories and identify ways of capitalizing on the successes and improvements making recommendations to the team leadership and to themselves as specific as practical.

Reflection is good

We generally don’t do enough of it as individual developers, and definitely not as teams.  Retrospectives provide that opportunity for the team to think about what just happened, celebrating the good, self-correcting what it can and find ways for management to intercede on its behalf.  Not doing a retrospective relegates feedback and reflection to ad-hoc processes that rarely results in the kinds of overall improvements the team at large can achieve when working together.


Agile Practices: Timebox Planning

June 17, 2019

With timeboxes in general reviewed, I’m going to next look at the edges of the timebox starting with the process by which the team accepts work into a timebox.  As discussed as part of timeboxes, the team itself must be doing the commitment.  However, even if they are not, the two ideas presented below will still be applicable.

How much can we get done during a timebox?

This is a common pain point for teams.  This is especially true when the team itself isn’t making the commitment.  Without some empirical data aiding the selection of work most anything chosen is likely a guess, or worst, simply forced on the team.  The Agile Manifesto’s Principle 8 is all about sustainable development for the team and stakeholders.  Timebox planning is critical to achieving that sustainability.

Team Velocity

Team velocity allows the team and stakeholders to estimate what amount of work may be achievable during a timebox.  It isn’t a guarantee of the commitment but it is an interesting estimate that’s more refined than t-shirt sizing.  After the team plans a few times (say 3-4 timeboxes) the number of story points that are actually completed in each timebox can be averaged – this average becomes the team’s velocity.  For timebox planning, the velocity allows us to look at the prioritized backlog and draw a line at where the team can PROBABLY get to (sum of story points above the line being approximately the current value of the velocity).  What specifically the team can commit to is going to depend on how the story tasking goes and the general story point values above that line.

The team’s velocity can, and will, change over time.  The team will get better, improving common development tasks and designs, create optimizations such that the previous story sizes baselines now are misaligned.  The team will also change composition over time.  Adding and removing team members will create jitter in the velocity value.  Company holidays will do that as well.  If a team works together long enough a moving average of the velocity can smooth out some of that jitter.

Story Tasking

Story tasking is also something I don’t often see during the planning ceremony.  More often I’ll see a set of stories dropped in front of the team that they’ve never seen before and then specific team members asked to take on certain stories.  Some of the remainder of the team tries to figure out however they can contribute by volunteering for what they can.  Eventually the leads realize that not everyone has an assignment so they dole them out to people ill prepared to handle them.

What I prefer to see is taking the highest priority story off the backlog and then breaking it down into the individually necessary tasks for the story.  The team should do this together.  Those with direct knowledge/responsibility should be the principle contributors to the task list, but those who are not should also listen in, learning, asking questions, even suggesting some tasks that were overlooked (e.g. “So, after you’ve finished with that new API, who updates the client facing documentation with the new entry point details?”)

Task sequencing and/or dependencies are noted and then these tasks are estimated in terms of the number of hours necessary to complete for a single person.  I prefer to see individual tasks being no more one logical business day worth of effort (there’s good arguments that 6 hours is the most hours per day a developer should commit to due to obligatory overhead: scrum meetings, grooming, weekly team meetings, random training, etc.).  If a task is two business days I’d prefer to see two tasks created.  This allows the team to see daily progress on the story board and it better allows for the identification of stalling stories.

The tasks are then assigned to members of the team (preferably voluntarily but sometimes they just have to be assigned).  At some point a developer will reach the maximum number of work hours in the timebox and is no longer eligible to pick up new work without putting an equivalent number of hours back onto the table for the team to pick up.  How this is negotiated is going to be specific to the dynamics of each team.

Once all of the developers have maxed out their hours of assignable tasks, no more stories can be added to the increment.  In an ideal world, the tasking works out perfectly with an exact right amount of work for every member of the team.   Of course this never happens, so when there’s more hours of tasks than available developers, that story is put back down and not included in the timebox commitment.  The team then looks down the priority list to see if we can find one that can be pulled in instead.

If there is none that can be reasonably pulled in, then we don’t worry about it and we proceed with the stories that have been committed to.  If everything goes well, and there’s time left in the timebox at the end, the team can review the backlog again to see if anything new has come up that can be pulled in easily.  But, we all know that the development process is non-linear with circumstances and events causing delays or problems that can easily use the available time.

The Team’s Commitment

All of this comes back to ways of helping the team own the commitment of the timebox.  Velocity helps the team itself directly understand what recent historical has said they can accomplish.  Story tasking helps ensures that neither the team, nor any individual, is over-committed.  Together, the team should become comfortable with its ability to commit to work within a timebox and be able to operate at a sustainable pace.  Further, management can use the measurable values to aid the short and near term planning efforts.  To not use these practices removes useful tools from which both the team and management can benefit.

Agile Practices: Timeboxes

June 13, 2019

The first of my key agile practices are timeboxes/sprints/increments.  They are common to most agile development methodologies.  The purpose is to set a scope for the development team to achieve during a fixed period of time.  Traditional recommendations are for timeboxes are 1 to 4 weeks, but YMMV.  The intent is to allow the team to focus on a (relatively) fixed set of deliverables ideally that can be put into production at the conclusion of the timebox.

Timeboxes by themselves are a neutral practice.  Their effectivity is determined by a few things that go around them, the most influential is who is doing the commitment for the timebox.  Ideally, the commitment is done by the team.  They pick the top-most stories from a prioritized, and groomed, backlog until they are uncomfortable with making any additional commitment.  I’ll go into timebox planning in a later post.

In reality, the backlog is often not groomed or prioritized, and the team often isn’t who is making the commitment.  Skipping over the issue of the backlog for now, if the team isn’t making the commitment, then who is?  Typically, it is going to be the team/technical lead or the product manager.

When I see this circumstance (and I personally have been in this situation, both as the lead and as a team member), we have to look at why.  It is often the case that someone, possibly in an overseeing management role, or the lead or product managers themselves, considers the person doing the commitment to be better than the team at-large at estimating what the team can do.

This is a possible red flag (yes, I’m going to red flag my past self).  Let’s say the commitments are perfect… the person making the commitment for the team is getting it perfect every time – each feature is completed on schedule, is fully tested and works perfectly in production just as it was imagined it would.  In this perfect scenario, the team is still losing out.  We never provide safe ways to fail and learn.  We never develop new leaders or new experts.  We have traded improving the team for the repeated successful delivery of the product.

But, the sprints are not actually successful, are they? 

More typically, what we have is Agile’s version of a death march – the increment is not completed, and instead of learning from the failure, we double up on the commitment for the next timebox… we might even start suggesting that we need an integration or stabilization sprint to account for the software not being ready for delivery at the end of the timebox.  Management starts to think that the team is incapable of executing properly… they must need some more training on how to be a high performing team…

But, this is fixable. 

Allowing the team to make the commitment both fixes a number of things and brings to light real team problems for management to consider.  Side-stepping problems related to the team being self-destructive, when the team makes a commitment, the team then owns the delivery of the commitment.  People tend to try to meet their obligations when they make them for themselves.  People tend to disregard obligations when others make them for others…

What if they are wrong and:

  1. They completed everything early?  If we’re before the completion of the sprint take the team back to the backlog and find something they thing they can finish in the remaining time.  During the next sprint planning, bump the commitment level a bit and see how it goes.  Having a few extra prioritized stories ready to go as stretch goals or for contingency purposes is “a good thing”.
  2. They tried to accomplish more than they could?  Assuming they did successfully deliver some stories, lower the commitment level for the next increment a bit.
  3. They didn’t complete anything?  There are many causes for this: weak stories, weak tasking, weak team knowledge, etc.   Looking at how the team analyzes the successes and failures of the timebox and their recommendations via the retrospective will be critical here.
  4. They won’t commit to anything?  There are no easy answers here, but you’ve probably created a culture that precludes failure or are offering stories that the team doesn’t have enough detail or knowledge to bring to completion (ignoring the analysis paralysis sorts of situations).  The fear culture is something that’s going to require a lot of work to improve, but in both cases, trying to break up the existing stories into smaller stories that have less risk, ambiguity, or failure paths, is going to move the team in the right direction.

After allowing the team to make the timebox commitment we can’t expect them to get it correct the first few times – this is an experimentally learned process.  The value is in the team’s ownership of the commitment.

What if they don’t build enough fast enough?

Well, that’s something different entirely.  Agile doesn’t build code faster, it builds what is currently important.  The common problem here is where there’s a whole number of equally super urgently high priority that must be done by next Thursday…. I’ve seen in both agile and waterfall where management individually assigns tasks/stories/whatever to team members in an effort to push important/pet/special/not-prioritized things forward.  These efforts often fail.  If they do work, then it is because of the hero work of some small number (one?) of team members.  Everything about this is wrong.  Management comes to single out these hero workers as the examples of “the best” employees when in fact the rest of the team is trying to pick up the obligations of those who were “re-prioritized” increasing the pressure and probability of failure on the remainder of the team.

Oh, but the production support obligations!

If you’re dealing with a couple of high priority issues per timebox, then understand that some minor impact (read: stories that won’t be completed) will occur to the commitment (scenario b above).  If you’re dealing with a lot of high priority issues per timebox, have the team commit to an appropriately lesser effort per timebox, BUT (this is where the prioritized backlog shines) ensure there are extra stories ready to go in case there’s time left in the timebox to pick up some extra work.  In either case make sure all issues completed during the timebox after commitment are reported and identified as accomplishments.  Over time the team will naturally adjust the commitment to be what it needs to be.

Timeboxes are a critical component of most agile systems

But timeboxes alone don’t solve anything.  If the team isn’t making the commitment then the team won’t own the commitment.  We need to look also at the timebox planning, retrospectives and team composition as they all feedback directly into the success of the team and what can be done during the timebox.

Agile Practices: What Works, What Doesn’t, and Why IMO

June 13, 2019

I’m not a certified agile coach or scrum master.  I’m an experienced technical/team lead/architect who has built high performing teams and delivered successful projects in many industries using different technology sets.  I’m an agile-ist at heart, but have worked and lead through variations of waterfall.  I’ve seen teams (mine in some cases) struggle with agile.

However, I think I’ve narrowed down some key practices that when done well, can significantly enhance the productivity of the team.  The interesting thing is that most of these practices are facilitating, each helps the others, so when one of them is substantially broken, the whole process falls apart on itself.

That said, it is all fixable.  Even with an imperfect team, and in imperfect circumstances (read: most people’s situations),  we can make improvements that can start making the team’s life better.  However, there are some things that can’t be easily overcome.  When seen, these are red flags that may need significant effort by the management team to influence.

Key Practices

The Start of a New Quarter

March 7, 2010

This week starts a new quarter at RIT and so I’ll be teaching another edition of SE 361. I love teaching 361. The class moves fast; every lecture is on a different topic.  The course concepts are taught “just in time” for their use by the class.  This week I’ll be teaching on teams, processes and planning.  By this time next week, newly formed teams will have their roles ironed out, team wiki set up and be thinking about the first set of assignments due next Wednesday.  In four weeks there’s an exam, two weeks later the first team presentation, two weeks later another exam, and two weeks later another presentation.  We’ll have covered process, requirements, design, acceptance testing, unit testing, integration testing and UI design issues in the five weeks leading up to the first presentation.  The class moves fast. I feel blessed and honored to be able to influence the next generation of developers.

While I love teaching, I would never give up my day job as a software developer.  I love writing software. I love making these annoying simple, and yet so wickedly complex, pieces of electronics do useful things. I like having a customer come ask me what they think is an impossible or unreasonable task and telling them “sure thing, no problem, I can do that”.  I come from an embedded systems background where we really do make electronics do interesting things, like broadcast and decode radio waves, or spin motors for conveyors…  These days as a software architect my focus is on business efficiencies; integrating various data systems to achieve previously unrealizable operational goals by navigating through the insanely complicated and disconnected operational rules of multiple organizations. While its easy to get lost in the details of the insanity, quality is paramount.  Balancing security with operational requirements and achieving reliable and trustworthy systems… that is what I love to do on a daily basis and that, in a nutshell, is problem solving.

Teaching is not problem solving.  It is a means of sharpening my problem solving skills by trying to be better prepared, more up-to-date… anticipating the next question.  It pushes me to know more about my craft, to reflect on how I perform  it, how do I want to improve it, how can I apply my craft to better achieve my business’ needs. I love teaching, but I’m hard pressed to walk away from the rush of resolving a problem that seemed impossible yesterday and yet now sets the bar a little bit (or a lot) higher for the next problem being sent my way.

The Agile Manifesto

March 3, 2010

I just love the Agile Manifesto. It contains excellent principles for how we should be developing software regardless of what type of development process we are using.  Indeed, I believe that despite its name (agile) we can use its principles in a waterfall situation.  I’m not going to advocate specifically for waterfall, but there are some places that cannot use the more aggressive forms of agile that are so loudly advocated these days.  Waterfall, even with all its weaknesses, will still be a heavily used methodology for many years to come.

I am going to start a series of posts where I am going to discuss each individual principle of the Agile Manifesto.  I’m looking forward to digging into each more directly to see how they fit into my personal philosophies and career experiences and how I think they should be applied in the future.