Testing


Listen Now
Subscribe on iTunes
Check out the podcast on Google Play Music

This week’s Software Process and Measurement Cast features our essay revisiting the product owner role. The product owner role is hard, often messed up and a great opportunity for improvement.

The second column features the return of Steve Tendon talking about Tame The Flow: Hyper-Productive Knowledge-Work Performance, The TameFlow Approach and Its Application to Scrum and Kanban published J Ross (buy a copy here). We tackle Chapter 17 which is titled Challenges of Work-State Work in Process Limits. WIP limits have their plusses and minuses when discussing hyper-productivity.  

Our third column this week is from Jeremy Berriault. Jeremy discusses how to show the value of QA and why knowing and showing value is important!   Jeremy  blogs at https://jberria.wordpress.com/  

 

Re-Read Saturday News

This week we tackle Chapter 6 of Holacracy: The New Management System for a Rapidly Changing World by Brian J. Robertson published by Henry Holt and Company in 2015.   Chapter 6, Facilitating Governance, puts the ideas and processes defined in governance to work. (more…)

Listen Now
Subscribe on iTunes
Check out the podcast on Google Play MusicListen Now

The Software Process and Measurement Cast 440 features our essay on two storytelling techniques: premortems and business obituaries.  Almost all work that takes more than a few days is subject to risks that are not immediately obvious without some form of structured process to focus the team’s thought process. Teams often use storytelling techniques to generate a big picture/vision to guide a project or to help people frame their thoughts. A story provides a deeper and more nuanced connection between the team and information than most lists of PowerPoint bullets or a structured requirements documents. The same storytelling skill can be used as a risk management tool. Premortums and business obituaries are structured techniques for using storytelling for risk management.

Our second column is from Jeremy Berriault. Jeremy discusses the importance of conferences for learning new ideas and for networking.  Jeremy suggests that if you are have not learned new ways to test and you are testing the same way you were last year then you are falling behind. Jeremy  blogs at https://jberria.wordpress.com/  

Jon M Quigley brings his column, The Alpha and Omega of Product Development, to the Cast. In this installment, Jon discusses mental models and their impact on how you develop and deliver value.  One of the places you can find Jon is at Value Transformation LLC.

Re-Read Saturday News

Chapter 3 of Holacracy completes Part 1 by laying out the structure needed for an organization to be able to quickly and continuously evolve how authority is distributed.  An organization’s structure needs to be conducive to the processes needed to distribute authority.  This chapter provides an alternative to the classic pyramid structure of organization design which is typically out of date, irrelevant and difficult to change.

Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.

A Call To Action

I need your help. I have observed that most podcasts and speakers at conferences over-represent people from Europe and North America.  I would like to work on changing that exposure. I would like to develop a feature featuring alternate software development voices beginning with Africa and Southeast Asia. If this feature works we will extend it to other areas.   If you can introduce me to practitioners that would be willing to share their observations (short interviews) I would be appreciative!

Next SPaMCAST

The next Software Process and Measurement Cast will feature our interview with John Le Drew.  John and I discussed the concept of safety at work and how safety, or the lack of it, affects software teams.  John is the host of the Agile Path Podcast I recommend you check out his podcast but make sure you are back here for our interview next week!

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

Listen Now
Subscribe on iTunes
Check out the podcast on Google Play MusicListen Now

The Software Process and Measurement Cast 438 features our essay on leveraging sizing in testing. Size can be a useful tool for budgeting and planning both at the portfolio level and the team level.

Gene Hughson brings his Form Follows Function Blog to the cast this week to discuss his recent blog entry titled, Organizations as Systems and Innovation. One of the highlights of the conversation is whether emergence is a primary factor driving change in a complex system.

Our third column is from the Software Sensei, Kim Pries.  Kim discusses why blindly accepting canned solutions does not negate the need for active troubleshooting of for problems in software development.

Re-Read Saturday News

This week, we tackle chapter 1 of Holacracy: The New Management System for a Rapidly Changing World by Brian J. Robertson published by Henry Holt and Company in 2015. Chapter 1 is titled, Evolving Organization.  Holacracy is an approach to address shortcomings that have appeared as organizations evolve. Holacracy is not a silver bullet, but rather provides a stable platform for identifying and addressing problems efficiently.

Visit the Software Process and Measurement Cast blog to participate in this and previous re-reads.

Next SPaMCAST

The next Software Process and Measurement Cast will feature our interview with Alex Yakyma.  Our discussion focused on the industry’s broken mindset that prevents it from being Lean and Agile.  A powerful and possibly controversial interview.

Shameless Ad for my book!

Mastering Software Project Management: Best Practices, Tools and Techniques co-authored by Murali Chematuri and myself and published by J. Ross Publishing. We have received unsolicited reviews like the following: “This book will prove that software projects should not be a tedious process, for you or your team.” Support SPaMCAST by buying the book here. Available in English and Chinese.

33668249915_7a8d73072b_k.jpg

Which size metric makes sense for a testing organization is influenced by how testing is organized, where testing is incorporated into the value delivery chain, and whether the work is being done for a fee. (more…)

33510320232_40376a5052_k

Test case points is only one approach to determining the size of work that needs to be tested. The other measures fall into three broad categories.  The categories are: (more…)

Strengths and Weaknesses are up in the air!

Jeremy Berriault provided an example from this presentation at QAI Quest 2017 for us to count test case points.  Jeremy, QA Corner,  indicated baseline data was required to effectively run the three test cases in his example

The logon, transaction

, reports and expected output blocks represent verification points.  The arrows from one test case to another represent interfaces and steps are. . .steps.  The results of the count is as follows:

Test Case Number of
Steps
Interfaces Verification
Points
Baseline
Test Data
Complexity

test case 1

4 0

4

required medium
test case 2

4

0

4

required medium
test case 3 3 2 3 required

medium

Deriving the complexity leverages the following chart: (more…)

 

How do you measure out of the ordinary packages?

Independent testing groups are often asked how long and how much effort is required to test a piece of work.  Several size estimation techniques are actively in use in many organizations.  Each of these techniques begins by deriving size either based on a set of rules or through relative sizing.  Size, once derived, is used to estimate effort.  Effort is then used to generate cost, staffing and duration estimates.  The first sizing technique is“Test Case Points”

Test Case Points are a unit of measurement generated from the the testable requirements based on a set of rules.  The process is straightforward:

  1. Identify the testable requirements in a piece of work.  Use Cases or technical requirements documents are used for identifying testable requirements.  
  1. Identify the complexity of each testable requirement.  Test case points evaluate four factors to determine complexity:
    1. The number of test steps. The number of execution steps needed to arrive at an expected (or unexpected) outcome after all preconditions have been satisfied.
    2. The number of interfaces to the other requirements. A simple count of the number of interfaces in the test case.
    3. The number of verification points. A simple count of the points in the test case that the results are evaluated for correctness.
    4. Need for baseline test data. An evaluation of whether data needs to be created to execute the test case.  


Once all of the simple, medium and complex test cases are identified, they are summed by category.

  1. Weight each category.  

  1. Sum the weighted categories together to yield the total test case points

The goal of test case points is to use size to generate an estimate.  Every version of test case points I have worked with uses a set of factors to adjust the size as part of the sizing process.

  1. Develop an estimation adjustment weighting based on a set of factors (for those familiar with IFPUG Function Points this adjustment is a similar process to the one for determining the value adjustment factor). The factors are:
  1. Count or Single Factor Adjustment Factors
    Factor 14 – Operating System Combinations (simple count)
    Factor 15 – Browser Combinations (simple count)
    Factor 16 – Productivity Improvement from Second Iteration Onwards (percentage)
  2. Factors that leverage a combination of fixed factor and complexity weighting
    Factor 1 – Domain Knowledge & Complexity
    Factor 2 – Technical Know How
    Factor 3 – Integration with other Hardware Devices such as Handheld Devices, Scanners, Printers
    Factor 4 – Multi-lingual Support
    Factor 5 – Software/Hardware Setup
    Factor 6 – Environment Setup
    Factor 7 – Build Management
    Factor 8 – Configuration Management
    Factor 9 – Preparation of Test Bed
    Factor 10 – Stable Requirements
    Factor 11 – Offshore/Onsite Coordination
    Factor 12 – Test Data Preparation
    Factor 13 – Network Latency

 

  1. Generate an estimate using the following formula

Weighted Test Case Points X Adjustment Factor X Historical Productivity Rate

In many cases, organizations generate estimates for types of work separately using the adjustment factors that that would affect the type of work.  An example of a type of work is test case generation.  Factor 5, software/hardware setup, would not be predictive of the effort for setting up test cases.

The process for deriving test case points is fairly straightforward (steps 1 – 4).  The process of turning the test case points into an estimate is more complicated. Next, we will develop a short example and examine the strengths and weaknesses of the process – some which are very apparent and other are not.  

Next Page »