A count of the Pokemon in my local park yesterday!

In today’s complex software development environment, it is easy to see every data collection issue as a complex automation problem. For example, I recently had a discussion with a team that is trying to determine how they would know whether a coding standard change they were contemplating would be effective. The team proposed capturing defects the coder/developer pairs found in TFS. The plan was to enter each defect found into the tool. After a discussion, the team decided to adopt a much simpler data collection solution with very low overhead. One of the classic quality tools is a tally sheet or check sheet. A check sheet is a form that used to collect information as it happens as a set of checkmarks or tally marks on the form. Check sheets are a very simple data collection tool. They are often a great way to pilot data collection before spending time and effort on automation or adding overhead to work. A check sheet is a form of the histogram where data where data collection and categorization occurs at the same time as visualization.

As an example of check sheets in action, a few years ago, I met a team that counted buzzwords during their all-hands department meetings. They even had gone as far as to create a list of words. During their team meetings, they would tick off for every utterance of a specific buzzword by one of the team and department leaders. The winner had to buy the first round of drinks at happy hour.

Creating a Check Sheet:

  1.  Decide what to observe and then the categories to capture the data. The list of buzzwords became the categories used for the checklist.
  2. Decide on the time period to collect the data.  Rarely does it make sense to collect data continuously forever! In the example, the time period was the beginning and the end of the departmental meeting. 
  3. Design a simple form that is easy to use. The data collection method must fit the data collection environment. In the buzzword data collection game, the form was a simple 3×5 sticky note the players could keep in their notebook, away from prying eyes. Simplicity is key to getting a high level of compliance with data collection and improves data quality. Tick marks to collect failed tests is easy, keying them into TFS is less easy.
  4. Observe and collect the data. In the example, the observation was in the form of listening and the sample used included the whole team meeting. In the example of the team gathering data to validate the potential coding standard change, the team decided they would evaluate a representative sample of unit test events rather than every possible event.

An aside to the buzzword check sheet game, the leadership of the department were aware of what was going on but turned a blind eye to the practice because it meant people actually had to pay attention during the presentations.

The goal of a check sheet is to quickly generate and collect data; to turn opinion into a data or fact-based discussion. Check sheets can be a simple as a notepad next to a laptop with a name and time and date on it to aid in analysis. Consider the complications of automation once the if the data is valuable and collection needs to be done for an extended period of time.