The overarching goal of quality assurance is to build constituent satisfaction by providing uniformly excellent goods and/or services. QA relies on a suite of tools to realize this goal, and testing is one of the most flexible and powerful. We usually think of testing as something done by a specific testing professional after a product is manufactured to ascertain whether or not it functions according to requirements. But as Momentum’s Brian Mills points out, early testing is the most effective means of catching product and service flaws before they wreak too much damage. The best early testing is an all-hands-on-deck proposition. In other words, everyone working on the project, from design through implementation, should be thinking like a tester to ensure quality results.

What’s the Big Deal About Testing?

Why do we care so much about testing in QA? Simply stated, it costs less to fix something earlier than fix it later. The 1-10-100 rule illustrates why this is. Also called the “cost of quality,” 1-10-100 describes the escalating costs that accompany progressive failures – and the correlating cost savings that happen when those failures are caught earlier. For every $1 spent on prevention, an organization can save $10 on correcting minor issues and avoid spending $100 on major failures.

A good example from a consumer standpoint is imagining what would happen if sparkling water ended up in a package labeled for flat spring water or vice versa. If a defect makes it all the way to the end-user, not only do you have the cost of the fix (refunding purchases, pulling stock, shipping new inventory), but you also have the inestimable cost to your reputation. If your customer doesn’t know if they can trust you to give them the correct beverage, how likely are they to buy from you again?

Perhaps that example feels too straightforward. After all, your organization’s reputation is probably based on more complex issues and a variety of services. However, suppose you think like a tester of that hypothetical bottle and envision all the steps that really go into making sure a water bottle meets consumer requirements. In that case, you realize how complicated the process can be – a lot like your enterprise.

A water bottle’s requirements include that it contains precisely 16.9 fluid ounces, or 500g, of water. Because an empty bottle with a cap weighs 10g, a full one must weigh 510g. A printed label must be wrapped around the bottle completely and adhered to itself. A barcode on the label must be imprinted with a specific value. The bottle must be made of recyclable, BPA-Free plastic that does not fail when frozen. And the cap must be secured to the bottle with a tamper-evident seal intact. That’s a lot of particulars to get right even before the bottle is filled! By testing the bottle at each point in the assembly process to ascertain if these conditions have been met, the manufacturer is inherently more likely to catch other flaws, up to and including what type of water is packed in a case to be shipped to a consumer.

You prevent failures at their source by thinking like a tester throughout your manufacturing or project process. It helps to understand the testing mindset. No one person will have all the desirable traits of a good tester. Still, generally, testers are both creative and technical, possessing several of the following attributes:

• Good at Listening
• Observant
• Inquisitive and Questioning
• Committed to Continuous Learning
• Practical
• Analytical
• Good Problem-Solving Skills

Categories and Types of Testing

Testing is an iterative process, meaning it is most successful when completed in phases. Before we get into those specific phases, let’s talk about the two categories of testing: functional testing and non-functional testing.

Functional testing is performed using specifications/requirements. Completed before non-functional testing, it can be manual or automated. Functional testing tests what a product or system does. When you press that red button, what happens? How about that blue one?

Non-functional testing checks the performance, reliability, scalability, and related attributes of a product, program, or project and is performed after functional testing. Non-functional testing can also be manual or automated and requires specific tools to complete. It tests how well a product or system works.

For example, you might test what could happen when all 1,000 of your employees use the same system simultaneously. If they all start work at 8:00, can the system handle 1,000 people logging in at the same time? What if you merge with another company and your thousand employees become 10,000? Can the system handle that load as well?

In addition to the testing categories, there are several types of testing that might apply to your project. Here are some of the most common:

• Unit Testing: The most basic type of functional testing, unit testing verifies a specific part (or unit). Software developers, for instance, might check to ensure that a unit of code works as intended. This happens throughout the development project, so the developer essentially has to think like a quality assurance tester throughout the process.
• Integration Testing: When one unit/component is integrated with another, it must be tested. This also applies to larger components of a system.
• System Testing: All components of a system are tested as a whole to ensure that the product meets the requirements. This is typically done in an environment separate from the development environment.
• Negative Testing: This is an essential subset of system testing. Negative testing is the practice of forcing an error into the system that you are testing with the intent of determining how the system will handle the error condition. For instance, to conduct negative tests on your water bottle product, you would:
  • Drop the bottle from a reasonable height (maybe a second-story window) and see if it breaks
  • Squeeze or apply crushing pressure to the bottle and see at what point it breaks
  • Test how gracefully it handles use – can it survive a fall of 4 feet or withstand pressure of x amount, such as when the bottle is placed in a bag with something on top of it?
•  Regression Testing: Regression testing is used for changes to existing systems. Regression tests ensure that newly introduced functionality has not adversely affected existing functionality. This is most easily done using previously executed tests. However, it’s important to note that “existing” doesn’t necessarily mean that the system is in production. It could be changes made to a system in development due to changes in requirements, defect resolution, and so on.

• Accessibility Testing: Accessibility testing is a specific form of usability testing to ensure that the system can be used by people with disabilities

• Performance Testing: Measuring how a system performs under specific conditions, performance testing is done with specific tools. This type of testing includes:
  • Load testing: Testing the system under “normal” conditions
  • Stress testing: Taking the system to the upper limits of its capacity. Similar to negative system testing, you are looking at how the system recovers from failures in stress testing
• Acceptance Testing: Usually known as User Acceptance Testing, this protocol tests that the system complies with end-user requirements and is ready for deployment. The focus is on the ability of users to perform their business processes.

Which of these testing types would help you best apply the 1-10-100 rule to your enterprise? How can you apply the precepts of functional and non-functional testing to your program at various stages in its development?

What to Know Before You Test

Once you’ve identified the testing types that best suit your project, a bit more thought and preparation are needed before running the tests. Some considerations:

• Who will be executing the test, and at which level?
• Are the tests being written by one party and executed by another? What does each party need to know to successfully run the test?
• Will there be data set up before testing begins, or will it be created with the test?
• Are all requirements covered by at least one test?
• Have you balanced the number of items being tested with the number of tests written? Multiple requirements can be covered in one test.
• Has a timeline been established? Where will testing take place?
• How will status and results be reported?
• Has a defect management process been enacted? How and when will fixes be applied?

Of course, one of the most important questions is: Do you need help determining what tests you need and how to run them to make sure your program successfully meets all requirements? A QA consultant can be an invaluable ally in helping you think like a tester on your next project.

Because everyone wins with thorough QA, every organization would do well to apply quality assurance tools to their enterprise. To learn more, subscribe to Momentum’s news and blog portal.