Measuring Running Tests
I was coaching in Des Moines around middle 2000's. Another gig with John Goodsen.
I was helping a small team adopt agile. There was tension in the company about this whole agile transformation. Fortunately, the team was small and under the radar enough that we could storm ahead.
As the team started mastering TDD, I decided to post on the outside of our area how many tests passed that day. This encouraged us to increase that number each day.
We were approaching a particular feature the team was very nervous about. The list of business rules was long and complicated. Whenever they expressed their worries, one of us usually said something like "we'll deal with that when we get to it." Probably in a more snarky way, as we'd grown close enough to be able to push each others' buttons in a fun safe way.
As we finally dove into those business rules, the practice we'd put into fast running small microtests paid off. I think by the time we were done we'd nearly doubled the number of running tests.
The team was very confident in the solution and that all the business rules were correct.
I left before they finished, but I was a happy and proud coach to learn even though there piece was low priority, their system was the first to go into production.
Finding the right metric takes time and patience. It's important the metric encourages good behaviors especially when gamed. Number of running tests met that criteria. They gamed it by writing more tests for each feature to cover all possible ways it could break. They wrote smaller faster tests because we didn't want to wait long to know they all passed. Pair programming kept us all honest in that each test was valuable and not a dreaded assert(true).
This is why I am a huge fan of cycle time. Gaming the time a story takes from backlog to delivery produces good behaviors. Teams write smaller stories, they swarm a story if the pair gets stuck, and within a Kanban team, they swarm lanes outside there job when stories are blocked by WIP limits. Counting time on blocked stories gets them unblocked pretty quickly.
I didn't mean this to be a post on metrics. I'm no expert on metrics and others have more to say about them, but if you imagine how a metric will be gamed, and it will be gamed, and if said gaming will produce beneficial or negative behaviors, then you'll have taken a big step forward in determining what to measure.