The Nature of Building Software: Build, Test, Repeat

Author: Brandon Frost, Director of PMO

Software engineering is more than just writing code. At a high level, the software development life cycle includes planning, analysis and design, building, testing, and support.  At all stages of this life cycle, most software projects include many smaller iterations; each of which includes some aspects of:

  • Design 
  • Building
  • Testing

This is done at both micro and macro levels throughout the project. In order to understand an important aspect of software development, let’s focus on one of the most important components of the life cycle: testing.  

Testing: Many Names, One Meaning

All software engineering projects, whether they are system integration projects, new coding projects, or configuration and implementation projects, go through many levels of testing.  Testing occurs at all levels of the project and comes in many varieties.  It’s funny how many different names the industry has produced to describe different types of testing; unit testing, smoke testing, integration testing, user acceptance testing, regression testing, load testing, beta testing, security testing, bug testing...the list goes on.

Why are there so many types of testing?  Why is testing so important?  Different test types serve different purposes.  But, regardless of the type, testing exists because the following statement is true; at a fundamental level, software developers don’t know if what they’ve built is actually going to work.  It must be tested to verify correct functionality.

A simple analogy can be drawn between software testing and pharmaceutical drug testing. Drug manufacturers invent a new serum to cure a disease.  The serum is injected into a test specimen (think mouse).  The results of the interaction between the serum and the specimen are observed. Failures (mouse dies) or inadequacies (mouse lives, but the disease isn’t cured) inform the manufacturer and allows them to mix up a new batch of serum.

Software developers do the exact same thing.  They invent a new batch of code.  The code is injected into a test system.  (This could be a local development environment or a company’s multi-node QA environment.)  Results of the code’s execution are observed.  Failures and inadequacies tell the developer how to re-write the code to obtain a more positive result.  This trial-and-error process is repeated until all desirables outcomes are met.

Testing in Real Life Scenarios

This process was experienced first-hand on a recent project at Zirous. Zirous has been working on a security implementation of significant size; more than a 2 year effort. The testing team has been iterating through test cycles for about a year and a half.  Typically, a test cycle began with the implementation of a significant piece of functionality.  It was unit tested, then incorporated into the main system.  All system components were regression tested to make sure everything built up to that point worked correctly with the new functionality.  Any issues found were fixed, the fix was implemented and tested, and itself fixed, if necessary.  Once the functional component was proven acceptable, the team moved to the next major functional piece.  This whole process was repeated about seven to ten times. By the time the project was complete, they had incorporated about 25 significant functional components and documented over 700 test cases.

How Testing Fits into the Big Picture

When a software engineering project is started, technical staff, no matter how knowledgeable, will be required to figure things out as they build.  In other words, at the beginning of a project software developers don’t really know how to build the software.  It doesn't matter the type of software engineering; writing code, integrating systems, or configuring applications, all software work requires some measure of art to figure out how things will ultimately be done.  Why?  Because software engineering is a trial-and-error process.

At their core, all software engineering activities are a Research & Development effort.  The iterative nature of the trial and error process and the fact that nobody knows how many times you will have to iterate through the process, determines that software engineers invent something new every time they exercise their profession.  This truth explains the nature of building software and why software engineering projects can be so complex!