CS615
– Software Engineering I
|
Lecture 7
|
Strategic
Approach to Software Testing
- Testing begins at the component level and works outward toward the
integration of the entire computer-based system.
- Different testing techniques are appropriate at different points in
time.
- The developer of the software conducts testing and may be assisted
by independent test groups for large projects.
- The role of the independent tester is to remove the conflict of
interest inherent when the builder is testing his or her own product.
- Testing and debugging are different activities.
- Debugging must be accommodated in any testing strategy.
- Make a distinction between:
- verification - are we building the product
right?
- validation - are we building the right product?
Criteria for
Completion of Testing

- Version of the failure model, called a logarithmic Poisson
execution-time model, takes the form
f(t) = (1/p) ln [ lopt + 1]
where
f(t)
= cumulative number of
failures that are expected to occur once the software has been tested for a
certain amount of execution time,
t,
lo
= the initial software
failure intensity (failures per time unit) at the beginning of testing,
p
= the exponential reduction
in failure intensity as errors are uncovered and repairs are made.
- The instantaneous failure intensity, l(t)
can be derived by taking the derivative of f(t)
l(t)
= 1o/( lopt + 1)
- Testers can predict the drop-off of errors as testing progresses
- Actual error intensity can be plotted against the predicted curve
- If the actual data gathered during testing and the logarithmic
Poisson execution time model are reasonably close to one another over a
number of data points, the model can be used to predict total testing time
required to achieve an acceptably low failure intensity
Strategic
Testing Issues
- Specify product requirements in a quantifiable manner before
testing starts.
- Specify testing objectives explicitly.
- Identify the user classes of the software and develop a profile for
each.
- Develop a test plan that emphasizes rapid cycle testing.
- Build robust software that is designed to test itself (e.g. uses
anitbugging).
- Use effective formal reviews as a filter prior to testing.
- Conduct formal technical reviews to assess the test strategy and
test cases.
Unit
Testing
Focuses verification effort on
the smallest unit of software design—the software component or module
- Black box and white box testing.
- Module interfaces are tested for proper information flow.
- Local data are examined to ensure that integrity is maintained.
- Boundary conditions are tested.
- Basis path testing should be used.
- All error handling paths should be tested.
- Drivers and/or stubs need to be developed to test incomplete
software.

(1)
misunderstood or incorrect arithmetic precedence
(2) mixed mode operations
(3) incorrect initialization
(4) precision inaccuracy
(5) incorrect symbolic representation of an
expression
- Test cases should uncover errors such as:
(1)comparison
of different data types
(2) incorrect logical operators or precedence
(3) expectation of equality when precision
error makes equality unlikely
(4) incorrect comparison of variables
(5) improper or nonexistent loop termination
(6) failure to exit when divergent iteration is
encountered
(7) improperly modified loop variables
Integration
Testing
- Top-down integration testing - Incremental approach to construction of program structure
- Depth-first integration would integrate all components on a major control
path of the structure
- Breadth-first integration incorporates all components directly subordinate at
each level, moving across the structure horizontally

1. Main control module used as a test driver and stubs
are substitutes for components directly subordinate to it.
2. Subordinate stubs are replaced one at a time with
real components (following the depth-first or breadth-first approach).
3. Tests are conducted as each component is integrated.
4. On completion of each set of tests and other stub is
replaced with a real component.
5. Regression testing may be used to ensure that new
errors not introduced.
- Bottom-up integration testing

0. Low level components are combined in clusters that
perform a specific software function.
1. A driver (control program) is written to coordinate
test case input and output.
2. The cluster is tested.
3. Drivers are removed and clusters are combined moving
upward in the program structure.
- Regression testing (check
for defects propagated to other modules by changes made to existing
program)
0. Representative sample of existing test cases is used
to exercise all software functions.
1. Additional test cases focusing software functions
likely to be affected by the change.
2. Tests cases that focus on the changed software
components.
- Conducted Daily
- Benefits:
- Integration risk is minimized
- Because smoke tests are conducted daily,
incompatibilities and other show-stopper errors are uncovered early
- Quality of the end-product is improved
- Because the approach is construction
(integration) oriented, likely to uncover both functional errors and
architectural and component-level design defects
- Error diagnosis and correction are simplified
- Errors uncovered during smoke testing are
likely to be associated with "new software increments"
- Progress is easier to assess
With
each passing day, more of the software has been integrated and more has been
demonstrated to work
1. Software components already translated into code are
integrated into a build.
2. A series of tests designed to expose errors that will
keep the build from performing its functions are created.
3. The build is integrated with the other builds and the
entire product is smoke tested daily (either top-down or bottom integration may
be used).
Integration
Test Documentation
- Test Specification - overall plan for integration of the software
and a description of specific tests
- Contains:
- test plan
- test procedure
- schedule for integration, the development of
overhead software
- test environment and resources are described
- records a history of actual test results,
problems, or peculiarities
- Becomes part of the software configuration
- Testing is divided into phases and builds that address specific
functional and behavioral characteristics of the software
- General Software Test Criteria
- Interface integrity (internal and external module interfaces are
tested as each module or cluster is added to the software)
- Functional validity (test to uncover functional defects in the
software)
- Information content (test for errors in local or global data
structures)
- Performance (verify specified performance bounds are tested)
Object-Oriented Testing
- begins by evaluating
the correctness and consistency of the OOA and OOD models