Here's how to use a tracking system like JitterBug
Anyone associated with the project, including its end users, are permitted
to add bug entries. You cannot "get in trouble" by adding the wrong one.
But, contrarily, anyone permitted to add bugs must have a unique account
(their e-mail address), and instructions to use the Query system to seek
their bug before adding it. A Bug can be marked as a Duplicate of another
Bug, by number.
All initial entries must contain three sections, without exception: The
"Steps to Reproduce", "Observed Behavior", and "Expected Behavior". A
successful bug entry helps programmers reproduce the exact problem as
closely as possible, and this format forces the bug report author to focus
on "just the facts" and not say things like "well, the engine makes a funny
noise under load..."
The bug author sets the "Severity" rating, from "Blocker" thru "Critical" to
Use this life-and-limb scale:
- Enhancement - a feature request
- Trivial - the bug only makes us feel bad
- Minor - the bug has an obvious work-around
- Normal - the bug prevents access to a feature
- Major - the bug damages the end-user's property
- Critical - the bug damages the end-user
- Blocker - you risk losing a customer
Each bug comes with a set of attributes, such as the product and owner
fields. Set them as best you can - on a large modularized project with a
conjoined team there should be much overlap.
Executives review the bug stream, adjust the details, owner & severity, and
then set the Priority, from P1 at highest, to P5 meaning "after the Sun
In a supermarket, eggs may come in "Medium", "Large" and "Jumbo", but your
bugs' average Priority should be "P3".
It cannot be emphasized enough that the Priority rating is the executive's
single most important "knob" on the team's activity. Do not direct
programmers via screaming e-mails, and do not make them try to rate who is
screaming loudest about what. Set the Priority such that each team of
programmers sees a collated stream of feature requests, in order by BusinessValue
For best results, give that knob to a real customer. Don't make >them<
scream loudest at executives just to get things done.
Executives should not play god and add a "time estimate" to any bug report. There is simply no benefit whatsoever to this uniquely insulting and damaging practice. Anyone who estimates the time a bug should take must be prepared to fix the bug within that time estimate. And proper use of a bug tracker makes real project time estimates very easy.
Each bug has a Resolution and a Status. A bug goes from NEW and VALID to
ACCEPTED to FIXED or INVALID. Programmers start by trying to reproduce the
bug; if they can't they mark it WORKSFORME and take no further action until
At whim a programmer may add a note to the system and set the ownership to
another programmer. You cannot "get in trouble" by sending the bug to
someone else, because they are expected to know how to at least send it
Your default Query into the Bugzilla system should display your current
assigned and non-CLOSED bugs sorted by Priority, sub-sorted by Severity.
Your job under this system is to walk the list from top to bottom. Don't
pull items at whim, or based on what looks most fun, and never do more than
one item at the same time.
After reproducing the bug and witnessing its symptoms, set the state to
ASSIGNED, then research each symptom until you can write an automated test
that will only fail if the symptom is alive, and only pass if the symptom is
dead. See CaptureBugsWithTests
For each symptom, and afterwards for the whole bug, ask each question in
- could better automation have prevented it?
- is it a member of a category of similar issues?
- is my fix robust against similar bugs?
- did my fix increase any risks?
Collaborate and work the system until good answers are built-into the code
There's an old joke about a man sawing on a tree. A passer-by comments the
saw's blades are very dull. "Yessir, they are dull." "Maybe you should take
time to sharpen the saw before.." "No, sir! I don't have time to SharpenTheSaw
. I'm too busy cutting down this tree."
Take time to ensure your tools are adequate for the job, and as you fix make
sure you leave code quality and clarity better than when you found it. Do
not accept any module's current bad state as an excuse for a fix that
violates good design rules, such as "short methods". That bad design is
among the forces that created the bug. Try to avoid the need to ever visit
this module again!
Comment the new tests and code with the bug's URL. Then integrate the fix
with your team's code repository, and comment that check-in with the bug's
Add a commentary to the bug report including the fix's technical details,
modules affected, and answers to its question list.
Run all your modules' tests every 1 to 10 edits, and run your entire
project's tests at each critical juncture during the process.
Mark the bug FIXED, and return it to either its owner or a Quality Control
The new owner will pull each daily build, then automatically and manually
test that the bug is dead. If they are satisfied, they mark it CLOSED.
The bug tracking system can name and store queries. Executives should build
a small suite of queries that examine things like each programmer's current
load, or any orphaned or abandoned bugs.
To track, executives should count the number of bugs and feature requests
fixed in each iteration (1 to 3 calendar weeks). Don't change iteration
length in the middle of a project just to provide the illusion of velocity.
But that number of bugs fixed is a very accurate way to predict future
performance. Nobody should enter whopper bugs such as "play chess", so an
even granularity of bug sizes should help provide a very stable ProjectVelocity
In a discussion last week about the importance of defect tracking, I was referring back to the point that GeraldWeinberg
makes in QualitySoftwareManagement
about the distinction between failures
(observable problems with the software's behaviour) and faults
(the underlying problems in the code). A single fault can certainly cause multiple failures; likewise, I suppose, an observed failure might be tied to multiple faults.
I think Weinberg goes further in that he distinguishes a System Incident (an unexpected piece of behaviour in the system) that may or may not have its source in a Fault (i.e. it may be a feature and training might be the solution).
In the ideal case, your DefectTracking
system would actually correlate the two, so that as you went through and marked faults resolved, it would map back to the failures that should have been resolved.
The question is, does such a DefectTracking
system exist in the real world? -- BillBarnett
It would probably be sufficient to have a system that allows to mark a problem report as a duplicate of another. Less administrative overhead (i.e. you don't have to write a separate "fault description"
Perhaps we should separate these two ideas:
- Change Control / Problem Tracking / Request Tracking = keeping track of and resolving customer / tester issues.
- DefectTracking = historic tracking of errors / defects / limitations / misdesigns / etc. in code, over time -- to identify "problem areas" / common errors / "bad practices" etc.
It makes sense that a development team might use two separate systems, one (issue tracking?) to track the status of (and record information about) failures/system incidents. Ideally, multiple incidents of the same basic issue would be logged as such (rather than as a bunch of separate incidents). External users might need visibility into this database. Once a system incident has been diagnosed as related to a fault in the code, it moves into defect tracking, whether in the same or a separate system. Defect tracking is almost an internal workflow for the development team, tracking the state of a fault (who has it been assigned to, has a fix been applied, has it been tested, approved, implemented, etc).
The trick would be to allow the system(s) to correlate between faults and their associated system incidents.
Has anyone seen a system (or set thereof) that provide this functionality? -- bb
Bugzilla allows bugs to be marked as duplicates of other bugs, putting a reference to the "first" bug in any duplicates. Often in the Mozilla project's bugzilla base you'll see entries saying "This bug has been marked as a duplicate of 65121". You can also mark bugs as "dependent" on others, such that you can't close a bug until all its dependences have been closed. Using the terminology of failure and fault from above, with diligence, it would be possible to create a "fault" bug as dependent on the bugs for each tracks "failure" incident. Not the other way around, though it might seem logical at first glance, because you want to ensure that the fault doesn't get marked resolved until all the failures are addressed, or you might discover that a failure isn't caused by the fault you thought it was, so you can move the dependency to the correct fault. --StevenNewton