We had Agile Training the other day. All layers of the business are going through it. Engineering already does scrum each morning with cross functional groups, we have continuous integration, unit testing, functional testing, acceptance testing with a business representative before it goes to QA, etc. All those have been grass roots and because of developers wanting to ensure high quality products are delivered to QA.

We do Sprints, but what we do is not agile, it is waterfall with our use cases chopped up into sprints. It works ok, as we can keep a finger on the button as to where we are, but it is not agile. Essentially our system is not feature complete at the end of a sprint, it is at the end of the project (2 months dev cycle).

We have use cases as our product backlog, but they are not managed in an agile manner. The business hands them off to us at the start of the project and there is little resorting/re-ordering etc during the project. We tried using JIRA for our product backlog and chopping the use cases up into day long tasks. Surprisingly that did not work at all. Use cases were a more accepted level of granularity.

At a previous place I put all the requirement sheets up on a wall at the end of the cube farm and we moved them around when they were done from one part to another. The spec sheets had names in thick marker on them and were ticked with a green marker when complete. Despite the visual authority it gave, it wasn't that useful either.

Since the entire company is going to Agile Training there is a strong chance it will stick and the non-Agile parts of the production process will become more agile. It is an easy sell with Engineering, we have implemented and maintained the discipline on most of those processes already.

Use Cases come from the Unified Process and are the main mechanism for communicating the requirements of a project as a system of interactions between users, systems and the features that need to be implemented. They are usually dominated by happy paths and unhappy paths. One of the areas where they are good is that they identify prior system state, the changes on the system and the end state of the system after the action has occurred. Unfortunately they tend to be large in length, growing exponentially and cause all manner of other use cases to be written in support that they grow beyond the ability of developers to manage or control them.

User Stories are kind of equivalent to a one sentence summary of a use case with one path as the outcome; such as a happy path. They are also done from a, or multiple, user perspectives. Some advocates of agile methodology have argued that a user story in conjunction with the acceptance tests constitutes a Use Case; though this assumes that the Use Cases in many organizations do actually think and write out the alternate paths to the happy path.

The other difference between the two is that a Use Case is intended as a history of the system and used to describe the system in the functional documentation at the end of a project. User stories are far more throw-away and don't survive the scrum board.

Use Cases tend to be long and grow cruft. Once they obtain a certain length they become unusable and short sentences are usually used to convey what a use case should do rather than constantly referring to the use case itself. User stories are easier to move around the scrum board, or hand around. They are also smaller.

Another method, and one I prefer, is the semiotics of BDD which is another permutation of the user story, called scenarios, in such a way that the task is instantly testable. We already spot our unit tests with;

//given

//when

//then

We put our mocked objects and expectations in the given, perform the operation after the when, and then test with assertions the change in state of the system after the operation through assertions.

The BDD style of scenarios are unambiguously testable at all parts of the project, from automated engineering tests, to hand run QA tests, to users in production that will be doing the same tasks on the system.

At the previous place I worked we had a particularly large project with detailed spec crossing multiple powerpoint presentations of UI design. We ended up pasting it on the wall at the end of the cubicles - about 60 pages or so. I don't think I should post it here on this blog, I have photos of it, but I am not sure if the previous company would come after me or not for posting company sensitive information. I am guessing not as it is out in production now, but who knows.

Anyway, the point of the story is that the spec pages didn't move nicely from left to right, they grew out. As people finished a page of spec they posted it to the out edge of the scrum wall, so as the project continued we ended up with a hole in the middle of the wall defined by the absence of spec pages. I thought it was a nice emergent behavior from making the spec a physical item in the project.

Recently read Lean Software Development by Mary and Tom Poppendick. For software it is of limited use, most of the examples are from heavy industry such as the car industry in arguing that up front design doesn't work. This is a well known issue in software already. Probably the best maxim is the delay decision making.

But then it is competing with the short, sharp absolutes of Extreme Programming which are not as long winded and directly related to software engineering. As an agile book it is probably best for a manager who has come heavy industry and is now working in software and is trying to understand why all their projects are coming in late, poorly done, and incomplete.

For software engineers who read these types of books for fun; meh. We know it already and are aware that much of the problem stems from procedural issues that software engineering has no control over.

Weak book, would avoid it if you are a software engineer. -1.

I have had this URL and idea hanging around for several years now. I recently started working on it again in both gwt and turbogears. Time to jot down some user stories for it rather than blindly coding.

The impetus for it was my dissatisfaction with Ameritrade. I was annoyed that there was a $10 fee on every transaction so I had to invest $1,000 a time to make that fee minimal. I wanted to be able to invest as little as $50 if I had it.

I also didn't want the complexity of choosing single winners and losers, I largely do coffee house investing and would like that spread across a similar structure.

Additionally I would like to be able to do transactions - swap shares with others - rather than the clunky pull it out, wait a day, then transfer it to my checking account etc. If I needed to pay $1000 for something, I would like to be able to trade shares directly into their account. Sometimes investing is like a second checking account.

Starting off with user stories can be paralyzing, in large teams it is better if someone just writes one and then see if people agree with it or not. The business analysts will make it overly complicated, and engineers will make it overly simple.

Personally I don't think it is important to argue over one word in a user story as the tasks the flow from it end up being the authority in the work and the user story becomes more of a placeholder for completeness than any hard and explicit requirement.

One of the goals of a sprint is to product production quality code and features. From Mike Cohn's Agile Estimate and Planning;

An agile team has the goal of fixing all bugs in the iteration in which they are discovered. They become able to achieve this as they become more proficient in working in short iterations, especially through relying on automated testing.

When a programmer gives an estimate for coding something, that estimate includes time for fixing any bugs found in the implementation, or a separate task is identified and estimated.

A couple of things flow from that; one the development environment that is used for publishing working code, automated testing and quality assurance, must be as close an environment as possible to the production environment. A lot of companies have QA as just something outside of a dev environment and nothing else. Pushing this way is bringing it close to continuous deployment; especially in data heavy applications.

The departure in systems and data between development and production must be minimal.

One of the issues we are having is certainty when code is deployed to an environment. The infrastructure side of things in terms of source change managements encompasses database changes, firewall rules, code moving in sync with webservice contracts etc. One of the areas where we fall down is that we don't have explicit tests rolled into our continuous integration to provide certainty of the state of the system at any one time.

We don't use Behavorial Driven Development (BDD) - I wish we did - we use Scrum's form of User Stories which mimics the given, when, then style of requirement writing. Our unit tests are commented with given, when, then though; however user stories are close enough for our needs at this stage. One of the tenets of BDD is that everything needs to be testable and the language has to match that need. From here;

When you write code, you always use unit testing & integration testing to verify that the application is working as expected, but why don't we use that when we install a system?

What are you using to verify that your system is correctly configured and behaves the way you want?

Which is fascinating, since infrastructure is starting to adopt BDD; or Behavioral Driven Infrastructure to ensure certainty of the environment at any one time. There are software packages to support this approach as well, such as puppet. I would love to roll into the continuous integration those functional and unit tests that infrastructure perform.

I re-read Lean Software Development by Mary and Tom Poppendieck. My opinions haven't changed much of the book. It is late to the cause, and unlike the agile and XP methodologies this has vague type ideas, rather than concrete solutions and implementations. Again, I don't see that great a value in it. The ideology has been covered earlier and better in the scrum and agile books that have come before it.

One of their arguments is for the reduction of waste in the process. It is probably the most compelling part of the book. To the authors, waste is anything that does not add value, basically anything that does not go toward the final product.

The authors state that management does not add value to the product or the customer but they can reduce waste in an organization and have a high impact their. Often agile methodologies are introduced into a company or organization but there is no re-organization and the positions that existed under a heavy waterfall method; such as project managers and business analysts just get plunked into an agile. So there is no reduction of waste in that respect.

One of the purposes of agile is to have the product owners and engineers directly in communication with each other. The reduction of waste as that there are no layers of organization, no conduits of other people or positions for the goals of the product owner to get Chinese whispered before the engineers get it. I think one of the big failings of agile transitions is this. The Poppendieck's state;

Project tracking and control systems do not add value, and further, they may be an indication of too much work in the system. In a just in time manufacturing system, work moves so quickly through the factory so quickly that sophisticated tracking is unnecessary. If work moved through a development organization in a just in time manner, it would not need a sophisticated tracking system either.

One of the benefits of a continuous deployment system is that there is no wasted time going from dev, to QA, to stage and to production; it goes from check in, to automated tests, to production. The Poppendieck's continue;

Learning to see waste is an ongoing process of changing the way you think about what is really necessary. One way to discover waste is to think about what would jettison if had to get rid of all the excess baggage on a troubled project. It is usually easier to see waste in a crisis.

Since I have been in a troubled project in the last year one of the first things we did was drop the jump from development integration to QA and went straight to a second stage environment. There are other things I would have changed, we should have spent two weeks and automated the entire delivery mechanism from dev to prod.

What we haven't dropped, which we should have, was the vendor. The vendor has been unable to produce the quality output required, they have been unresponsive and this project was beyond their capabilities, more than their product could deliver, and was an entirely new methodology of delivery for them. Unfortunately we could not, we were trapped essentially, and an in house solution would have been better, faster and more astute as it turns out.

One area of constant waste is bugs being opened against environments that require software developers to debug the environment rather than their code. Worse, it is when a bug is closed in one environment, and then re-opened in another because it is not working in another environment, and that being tracked down to the environment itself.

It is tiring and morale sapping as well. I reckon it is on average about four hours a bug. It takes that long to track it through the system and then isolate the issue. We have a tonne of functional tests that are for the purpose of quickly isolating issues that are environmental, but they still seem to occur.

Reducing the number of environments is a bonus, as this minimizes the number of environments that need to be debugged. Having one development environment that mimics production is another bonus. Then development is being done directly against data that is the same as production (or scrubbed as need be) and since it matches production should have the same environmental issues as production that are easily translatable - and fixable in production.