Bugs in software are inevitable and unavoidable probably because most software today is written by humans. Its not that humans are sloppy information systems creators, its because programming is both an art and a science which means there is a lot of variation in how to do it. This means every programmer has to address the issue of bugs in software. This blog post discusses the rationale for bug tracking and How to track bugs.
Why Track bugs.
The one article that every programmer should read on Bug Tracking is Painless Bug Tracking by Joel Spolsky. In the article he covers why you need a bug tracking solution with a real database.
The Reasons for having a bugs database:
- Nobody can keep track of all the bugs on pieces of paper or in their head.
- You often need to prioritize what bugs to fix.
- You often need to be able to reproduce the bug while fixing it.
- You often need to be able to track the history of the bug
- You need all kinds of metrics that most good bug databases provide to determine the quality of your software or development effort.
There are probably a few other reasons you should manage bugs, but the above should enable you to convince anyone that might think that bug tracking is another unnecessary effort in the software development effort.
If you are developing code, even on a team of one, without an organized database listing all known bugs in the code, you are simply going to ship low quality code. On good software teams, not only is the bug database used universally, but people get into the habit of using the bug database to make their own "to-do" lists, they set their default page in their web browser to the list of bugs assigned to them, and they start wishing that they could assign bugs to the office manager to stock more Mountain Dew. Painless Bug Tracking by Joel Spolsky.
How to track (and quash) the bugs
Testers usually capture the bug in testing. Software should always be tested by someone that didn’t write it. Always remember that Programmers will inevitably create bugs not because they are sloppy but because they are human.
When the bug is captured, the steps that caused the bug should be captured in the bug database. Having the list of reproduction steps is crucial for the programmer that has to fix it.Always reduce the number of steps when possible to enable the person doing the fixes to work faster.
Work hard to make sure that you create good bug reports because bad bug reports waste everyone’s time. Every bug needs to be assigned to exactly one person at all times, until it is closed.
When a bug is resolved, it gets assigned back to the person who opened it. Only the person who opened the bug can close the bug.
Three Parts To Every Good Bug Report
And the Lord spake, saying, "First shalt thou take out the Holy Pin. Then, shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shalt be three. Four shalt thou not count, nor either count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thou foe, who being naughty in my sight, shall snuff it."
-- Monty Python and the Holy Grail
It's pretty easy to remember the rule for a good bug report. Every good bug report needs exactly three things.
Steps to reproduce,
What you expected to see, and
What you saw instead.
Seems easy, right? Maybe not. As a programmer, people regularly assign me bugs where they left out one piece or another.
If you don't tell me how to repro the bug, I probably will have no idea what you are talking about. "The program crashed and left a smelly turd-like object on the desk." That's nice, honey. I can't do anything about it unless you tell me what you were doing. Now, I admit that there are two cases where it's hard to get exact steps to repro. Sometimes you just don't remember, or you're just transcribing a bug from "the field." (By the way, why do they call it "the field"? Is it, like, a field of rye or something? Anyway...) The other time it's OK not to have repro steps is when the bug happens sometimes but not all the time, but you should still provide repro steps, with a little annotation that says that it doesn't happen too often. In these cases, it's going to be really hard to find the bug, but we can try.
If you don't specify what you expected to see, I may not understand why this is a bug. The splash screen has blood on it. So what? I cut my fingers when I was coding it. What did you expect? Ah, you say that the spec required no blood! Now I understand why you consider this a bug.
Part three. What you saw instead. If you don't tell me this, I don't know what the bug is. That one is kind of obvious.(Painless Bug Tracking by Joel Spolsky.)
Top Ten Tips for Bug Tracking (Painless Bug Tracking by Joel Spolsky.)
- A good tester will always try to reduce the repro steps to the minimal steps to reproduce; this is extremely helpful for the programmer who has to find the bug.
- Remember that the only person who can close a bug is the person who opened it in the first place. Anyone can resolve it, but only the person who saw the bug can really be sure that what they saw is fixed.
- There are many ways to resolve a bug. FogBUGZ allows you to resolve a bug as fixed, won't fix, postponed, not repro, duplicate, or by design.
- Not Repro means that nobody could ever reproduce the bug. Programmers often use this when the bug report is missing the repro steps.
- You'll want to keep careful track of versions. Every build of the software that you give to testers should have a build ID number so that the poor tester doesn't have to retest the bug on a version of the software where it wasn't even supposed to be fixed.
- If you're a programmer, and you're having trouble getting testers to use the bug database, just don't accept bug reports by any other method. If your testers are used to sending you email with bug reports, just bounce the emails back to them with a brief message: "please put this in the bug database. I can't keep track of emails."
- If you're a tester, and you're having trouble getting programmers to use the bug database, just don't tell them about bugs - put them in the database and let the database email them.
- If you're a programmer, and only some of your colleagues use the bug database, just start assigning them bugs in the database. Eventually they'll get the hint.
- If you're a manager, and nobody seems to be using the bug database that you installed at great expense, start assigning new features to people using bugs. A bug database is also a great "unimplemented feature" database, too.
- Avoid the temptation to add new fields to the bug database. Every month or so, somebody will come up with a great idea for a new field to put in the database. You get all kinds of clever ideas, for example, keeping track of the file where the bug was found; keeping track of what % of the time the bug is reproducible; keeping track of how many times the bug occurred; keeping track of which exact versions of which DLLs were installed on the machine where the bug happened. It's very important not to give in to these ideas. If you do, your new bug entry screen will end up with a thousand fields that you need to supply, and nobody will want to input bug reports any more. For the bug database to work, everybody needs to use it, and if entering bugs "formally" is too much work, people will go around the bug database.
Related quotes and Links
Painless Bug Tracking by Joel Spolsky.
Keeping a database of bugs is one of the hallmarks of a good software team. I never cease to be amazed at how few teams are actually doing this. One of the biggest incorrect facts that programmers consistently seem to believe is that they can remember all their bugs or keep them on post-it notes.
If I can bend your ear a moment, I'd like to explain a pretty painless way to do bug tracking, in the spirit of my previous articles on painless schedules and painless specs.
First of all, you need a real database. On teams of 2 people writing a little bit of code over the course of a long weekend, it's probably OK to use a text file as the database. Anything larger, and you're going to need a real bug tracking database. There are zillions of bug tracking databases you can buy. (Blatant self-promotion: the one we wrote at Fog Creek Software, called FogBUGZ, is web based, pretty easy to use, and quite powerful, if I may say so myself.)
Programming - art, a craft or an engineering discipline
There is an ongoing debate on the extent to which the writing of programs is an art, a craft or an engineering discipline.[1] Good programming is generally considered to be the measured application of all three, with the goal of producing an efficient and evolvable software solution (the criteria for "efficient" and "evolvable" vary considerably). The discipline differs from many other technical professions in that programmers generally do not need to be licensed or pass any standardized (or governmentally regulated) certification tests in order to call themselves "programmers" or even "software engineers." However, representing oneself as a "Professional Software Engineer" without a license from an accredited institution is illegal in many parts of the world. (from Wikipedia’s Computer_programming)
- Advanced Defect Tracking - Designed for small and large software companies to simplify their feature development, bug tracking and helpdesk support. (Borderwave Software)
- BugAware - Web based bug tracking system and help desk solution for the company intranet. Features include email alerts, advanced reporting, team management and custom fields.
- Bugopolis Bug Station - Easy to install and use standalone server appliances based on open source software.
- BUGtrack - Web based defect tracking system.
- Bugzero - Web-based cross-platform bug tracking system supporting multiple projects, role-based access, automatic bug-assignment, file attachment, e-mail notification, metric reports and workflow.
- CustomerFirst - Bug tracking and CRM software enables customer-service agents to provide support via phone, Web, or email. Tracks customer profiles.
- Defect Agent - Defect Agent is a defect tracking software for software development teams. (Inborne Technology Corporation)
- ExtraView - Web-based bug tracking with Web API, command line and development workflow engine.
- Fast BugTrack - A web-based platform independent bug tracking system.
- FogBUGZ - Web-based bug tracking system.
- GNATS: GNU Bug Tracking System home page - A portable incident/bug report/help request-tracking system which runs on UNIX-like operating systems. It easily handles thousands of problem reports, has been in wide use since the early 90s, and can do most of its operations over e-mail. Several front end interfaces exist, including command line, emacs, and Tcl/Tk interfaces. There are also a number of Web (CGI) interfaces written in scripting languages like Perl and Python.
- IBM Rational ClearQuest - A defect and change tracking system that captures and manages all types of change requests throughout the development lifecycle.
- icTracker - Web-based bug tracking, task management, and project management software from IC Soft, Inc.
- Issue Organizer - Easy to use web-based bug tracking software. Pricing is not based on number of users. (Solid Graphics)
- IssueNet Intercept - A customizable bug tracking system and help desk with integrations to Visual Studio, VSS, Outlook, and MS Project.
- IssueView - Configurable Windows software for tracking defects, bugs, helpdesk calls, and knowledge base articles.
- JIRA - Browser-based bug, issue, task and defect tracking system, and project management software solution used for open source and enterprise projects. (Atlassian Software Systems)
- Legendsoft Spots - A web-based service order and defect tracking system for software development organizations and IT help desks.
- MetaQuest Software, Inc. - Census - A customizable bug tracking tool. Includes VSS integration, notifications, workflow, reporting and change history.
- New Fire Software - Defect Tracking software and recruiting software.
- OmniTracker - Customizable workflow tool for bug tracking, quality assurance, help desks and call centers.
- PointInSight: - Issue management system with collaborative teamwork and integrated communication.
- ProblemTracker - A Web-based tool for bug tracking, defect tracking, issue tracking, help desk, and automated support. By NetResults Corporation.
- PR-Tracker - Bug tracking system for software project management, advanced features, easy setup, integrated network and web versions.
- QuickBugs - A tool for tracking bugs, issues, changes and new features involved in product development.
- Radar - Web based solution that helps an organization to track, communicate, resolve and analyze defects in a software lifecycle, employing the Review Based Model that follows the CMM principles.
- Razor - Configuration management system providing process management, issue/problem tracking, version control, and release management.
- RMTrack Bug Tracking - A web-based bug tracking, issue tracking, and defect tracking and management application. Downloadable. Free 30-day trial.
- Software Planner - Manages all phases of the software lifecycle. It includes all the Defect Tracker features in addition to managing discussions, appointments / to-do lists, project tasks and contacts. (Pragmatic Software Company, Inc.)
- SWBTracker - Bug tracking and defect management software for Windows. By Software with Brains, Inc.
- TestTrack Pro - Bug and defect tracking software for Windows, Mac OS X, Linux and Solaris.(Seapine Software, Inc.)
- Tiera Software Defect Manager - Software development and support suite (bug tracking, call tracking, time tracking) for software and consulting companies. Windows and web browser based.
- TrackWise - Customizable bug and defect tracking software. (Sparta Systems, Inc.)
- WebPTS - Online software error tracking system. Features include - knowledge-base, attachments, customization. Encrypted access available.
Posts
