SE 616 – Introduction to Software Engineering |
Lecture 12 |
Quality –
o characteristic or attribute of
something
o measurable characteristics — compare
to known standards such as length, color, electrical properties, and
malleability
·
Two
kinds of quality:
o quality of design characteristics that designers
specify for an item
(e.g. grade of materials, tolerances, and performance
specifications)
o quality of conformance - degree to which the design
specifications are followed during manufacturing
·
Software
development –
o quality of design - requirements,
specifications, and the design of the system.
o quality of conformance - focused on
implementation.
Quality
control –
Quality assurance
–
·
The
auditing and reporting functions of management.
·
Goal
of quality assurance - provide management with the data necessary to be
informed about product quality, thereby gaining insight and confidence that
product quality is meeting its goals.
Cost of quality-
·
includes all costs incurred in the pursuit of
quality or in performing quality-related activities.
·
divided into costs associated with
prevention, appraisal, and failure.
o Prevention costs include
§ quality planning
§ formal technical reviews
§ test equipment
§ training
o Appraisal costs include
§ in-process and interprocess
inspection
§ equipment calibration and maintenance
§ testing
o Failure costs - those that would disappear if no defects
appeared before shipping a product to customers.
§ Internal failure costs - incurred when we detect a defect in
our product prior to shipment.
·
rework
·
repair
·
failure
mode analysis
§ External failure costs - associated with defects found after
the product has been shipped to the customer.
·
complaint
resolution
·
product
return and replacement
·
help
line support
·
warranty
work
Software
quality -
Conformance
to:
SQA group
-
·
Does
the software adequately meet the quality factors?
·
Has
software development been conducted according to pre-established standards?
·
Have
technical disciplines properly performed their roles as part of the SQA
activity?
Activities
performed by an independent SQA group:
Software
reviews - applied at
various points during software development and serve to uncover errors and
defects that can then be removed.
o
Point out needed improvements in the product of
a single person or team;
Primary
objective of formal technical reviews
FTR
· serves as a training ground
·
promotes backup and continuity because a number
of people become familiar with parts of the software that they may not have
otherwise seen.
·
class
of reviews that includes:
o walkthroughs
o inspections
o round-robin reviews
o other small group technical
assessments of software.
·
Each
FTR is conducted as a meeting that is planned, controlled, and attended.
Review
meeting constraints:
NOTE: FTR
focuses on a specific (and small) part of the overall software.
·
Walkthroughs
are conducted for each component or small group of components.
·
FTR
focuses on a work product (e.g., a portion of a requirements specification, a
detailed component design, a source code listing for a component).
·
Individual
who has developed the work product informs the project leader that the work
product is complete and that a review is required.
·
The
project leader contacts a review leader, who evaluates the product for
readiness, generates copies of product materials, and distributes them to two
or three reviewers for advance preparation.
·
Each
reviewer is expected to spend between one and two hours reviewing the product,
making notes, and otherwise becoming familiar with the work. Concurrently, the
review leader also reviews the product and establishes an agenda for the review
meeting, which is typically scheduled for the next day.
·
Review
meeting attended by
·
review
leader
·
all
reviewers
·
the
producer.
·
One
reviewer takes on the role of the recorder; that is, the individual who
records (in writing) all important issues raised during the review.
1. The FTR begins with an introduction of
the agenda and a brief introduction by the producer.
2. The producer then proceeds to
"walk through" the work product, explaining the material, while
reviewers raise issues based on their advance preparation.
3. When valid problems or errors are
discovered, the recorder notes each.
4. At the end of the review, all
attendees of the FTR must decide whether to
1. accept the product without further
modification
2. reject the product due to severe
errors (once corrected, another review must be performed)
3. accept the product provisionally
(minor errors have been encountered and must be corrected, but no additional
review will be required).
·
The decision made, all FTR attendees complete a
sign-off, indicating their participation in the review and their concurrence
with the review team's findings
·
All
issues that have been raised are summarized at the end of the review meeting
and a review issues list is produced.
·
In
addition, a formal technical review summary report is completed.
·
Review
summary report
·
answers
three questions:
·
What
was reviewed?
·
Who
reviewed it?
·
What
were the findings and conclusions?
·
single
page form (with possible attachments).
·
Becomes
part of the project historical record and may be distributed to the project
leader and other interested parties.
·
Review
issues list
·
serves
two purposes:
1. identify problem areas within the
product
2. serve as an action item checklist that
guides the producer as corrections are made.
·
attached
to the summary report.
Minimum set
of guidelines for formal technical reviews:
Statistical
quality assurance implies the following steps:
Example:
To apply
statistical SQA, a table is built.
·
Software
reliability can be measured, directed and estimated using historical and developmental
data.
·
Software
reliability is
defined in statistical terms as –
the probability of failure-free operation of a computer program in a specified environment for a specified time
·
Simple
measure of reliability - meantime-between-failure (MTBF)
MTBF
= MTTF + MTTR
o MTTF - mean-time-to-failure
o MTTR - mean-time-to-repair
·
Software
availability - the
probability that a program is operating according to requirements at a given
point in time and is defined as
Availability
= [MTTF/(MTTF + MTTR)] x
100%
·
MTBF
reliability measure is equally sensitive to MTTF and MTTR.
·
Availability
measure is more sensitive to MTTR, an indirect measure of the maintainability
of software.
·
Software
safety - software
quality assurance activity that focuses on the identification and assessment of
potential hazards that may affect software negatively and cause an entire
system to fail.
·
Modeling
and analysis process is conducted as part of software safety.
o Initially, hazards are identified and
categorized by criticality and risk.
o E.g., some of the hazards associated
with a computer-based cruise control for an automobile :
§ causes uncontrolled acceleration that
cannot be stopped
§ does not respond to depression of
brake pedal (by turning off)
§ does not engage when switch is
activated
§ slowly loses or gains speed
o Once identified, analysis techniques
are used to assign severity and probability of occurrence
o
Once
hazards are identified and analyzed, safety-related requirements can be
specified for the software.
§
Specification
can contain a list of undesirable events and the desired system responses to
these events.
o
The
role of software in managing undesirable events is then indicated
§ Quality assurance system - defined
as the organizational structure, responsibilities, procedures, processes, and
resources for implementing quality management.
§ Created to help organizations ensure
their products and services satisfy customer expectations by meeting their
specifications.
§ Cover wide variety of activities
encompassing a product's entire life cycle including planning, controlling,
measuring, testing and reporting, and improving quality levels throughout the
development and manufacturing process.
§ ISO 9000 describes quality assurance
elements in generic terms that can be applied to any business regardless of the
products or services offered.
§ The ISO 9000 standards adopted by many
countries including all members of the European Community, Canada, Mexico, the
United States, Australia, New Zealand, and the Pacific Rim.
§ After adopting the standards, a
country typically permits only ISO registered companies to supply goods and
services to government agencies and public utilities.
§ To become registered to one of the
quality assurance system models contained in ISO 9000, a company's quality
system and operations are scrutinized by third party auditors for compliance to
the standard and for effective operation.
§ Upon successful registration, a company
is issued a certificate from a registration body represented by the auditors.
Semi-annual surveillance audits ensure continued compliance to the standard.
§ ISO 9000 quality assurance models
treat an enterprise as a network of interconnected processes.
§ ISO 9000 describes the elements of a
quality assurance system in general terms.
§ Elements needed to implement:
o quality planning
o quality control
o quality assurance
o quality improvement
are
o organizational structure
o procedures
o processes
o resources
§ ISO 9000 does not describe how an
organization should implement quality system elements.
§ ISO 9001 is the quality assurance
standard that applies to software engineering.
§ Contains 20 requirements that must be
present for an effective quality assurance system.
§ Because the ISO 9001 standard is
applicable to all engineering disciplines, a special set of ISO guidelines (ISO
9000-3) have been developed to help interpret the standard for use in the
software process.
§ Requirements delineated by ISO 9001
address topics such as:
o management responsibility
o quality system
o contract review
o design control
o document and data control
o product identification and
traceability,
o process control
o inspection and testing
o corrective and preventive action
o control of quality records
o internal quality audits, training
o servicing
o statistical techniques.
§ For a software organization to become
registered to ISO 9001, it must establish policies and procedures to address
each of the requirements just noted (and others) and then be able to
demonstrate that these policies and procedures are being followed.
§ SQA Plan provides a road map for instituting software quality
assurance.
§ Serves as a template for SQA
activities that are instituted for each software project.
§ Standard for SQA plans has been
recommended by the IEEE
§ Standards, practices, and conventions
section
o lists all applicable standards and
practices that are applied during the software process (e.g., document
standards, coding standards, and review guidelines).
o all project, process, and (in some
instances) product metrics that are to be collected as part of software
engineering work are listed.
§ Reviews and audits section
o identifies the reviews and audits to
be conducted by the software engineering team, the SQA group, and the customer.
o provides an overview of the approach
for each review and audit.
§ Test section references the Software
Test Plan and Procedure
o defines test record-keeping
requirements.
o problem reporting and corrective
action defines procedures for reporting, tracking, and resolving errors and
defects, and identifies the organizational responsibilities for these
activities.
§ Tools and methods section
o Tools that support SQA activities and
tasks
o software configuration management
procedures for controlling change
o contract management approach
o establishes methods for assembling,
safeguarding, and maintaining all records; identifies training required to meet
the needs of the plan;
o defines methods for identifying,
assessing, monitoring, and controlling risk
·
Output
of the software process:
·
Number
of software configuration items (SCIs) grows rapidly as software process
progresses.
·
Change
may occur at any time, for any reason.
·
First
Law of System Engineering:
"No matter where you are in the system life cycle, the system will change, and the desire to change it will persist throughout the life cycle."
·
Four
sources of change:
·
Software
configuration management:
o Set of activities that have been
developed to manage change throughout the life cycle of computer software.
o Viewed as a software quality assurance
activity that is applied throughout the software process.
·
Software
configuration management concept that helps control change without seriously
impeding justifiable change.
·
IEEE
Std. No. 610.12-1990 defines a baseline as:
A specification or product that has been formally reviewed and agreed upon, that thereafter serves as the basis for further development, and that can be changed only through formal change control procedures.
·
Software
engineering context:
A baseline is a milestone in the development of software that is marked by the delivery of one or more software configuration items and the approval of these SCIs that is obtained through a formal technical review.
·
After
SCIs are reviewed and approved, they are placed in a project database (also
called a project library or software repository).
·
When
a member of a software engineering team wants to make a modification to a
baselined SCI:
o it is copied from the project database
into the engineer's private work space.
o this extracted SCI can be modified
only if SCM controls are followed
o
A
SCI is a document, an entire suite of test cases, or a named program component
(e.g., a C++ function or Java class).
o
SCIs
are organized to form configuration objects
o
cataloged
in the project database with a single name.
o
Has
a name, attributes, and is "connected" to other objects by
relationships.
e.g.
o
Configuration
objects, Design Specification, data model, component N, source code and Test
Specification are each defined separately.
o
Each
of object is related to others by the arrows.
§
Curved
arrow indicates a compositional relation.
·
data
model and component
N are part of the object Design Specification.
§
Double-headed
straight arrow indicates an interrelationship.
SCM
introduces a set of complex questions:
Must define
five SCM tasks:
Two types of
objects can be identified:
A "unit of text" that has been created by a software engineer during analysis, design, code, or test.
A collection of basic objects and other aggregate objects.
Each object
has a set of distinct features that identify it uniquely:
a
character string that identifies the object unambiguously
list of data items that identify
o the SCI type (e.g., document, program, data) represented by the object
o change and/or version information
entities
that are provided, processed, referenced or otherwise required by the object
a
pointer to the "unit of text" for a basic object and null for an
aggregate object
·
Identification
scheme must recognize that objects evolve throughout the software process.
o Before an object is baselined, it may
change many times
o After a baseline has been established,
changes may be quite frequent.
o Possible to create an evolution
graph.
o Describes the change history of an
object
§ Configuration object 1.0 undergoes
revision and becomes object 1.1.
§ Minor corrections and changes result
in versions 1.1.1 and 1.1.2
§ major update that is object 1.2.
·
Version
control combines
procedures and tools to manage different versions of configuration objects that
are created during the software process.
·
Change
control combines
human procedures and automated tools to provide a mechanism for the control of
change.
·
Change
Control Process:
o A change request is submitted
and evaluated to assess technical merit, potential side effects, overall impact
on other configuration objects and system functions, and the projected cost of
the change.
o The results of the evaluation are
presented as a change report, which is used by a change control
authority (CCA) — a person or group who makes a final decision on the
status and priority of the change.
o An engineering change order (ECO)
is generated for each approved change.
The
ECO describes the change to be made, the constraints that must be respected,
and the criteria for review and audit.
o The object to be changed is
"checked out" of the project database, the change is made, and
appropriate SQA activities are applied.
o The object is then "checked
in" to the database and appropriate version control mechanisms are used to
create the next version of the software.
NOTE:
·
"check-in"
and "check-out" process implements two elements of change control
o Access control - governs which software engineers have
the authority to access and modify a particular configuration object.
o Synchronization control - helps to ensure that parallel changes,
performed by two different people, don't overwrite one another
How to ensure
that the change has been properly implemented:
The formal
technical review focuses on the technical correctness of the configuration
object that has been modified.
A software
configuration audit complements the formal technical review by assessing a
configuration object for characteristics that are generally not considered
during review.
·
The
audit asks and answers the following questions:
Configuration
status reporting is
an SCM task that answers the following questions:
(1) What happened?
(2) Who did it?
(3) When did it happen?
(4) What else will be affected?
·
Each
time an SCI is assigned new or updated identification, a CSR entry is made.
·
Each
time a change is approved by the CCA (i.e., an ECO is issued), a CSR entry is
made.
·
Each
time a configuration audit is conducted, the results are reported as part of
the CSR task.
·
Output
from CSR may be placed in an on-line database so that software developers or
maintainers can access change information by keyword category.
·
CSR
report is generated on a regular basis and is intended to keep management and
practitioners appraised of important changes.
Number of
software configuration management standards:
·
Military:
o MIL-STD-483
o DOD-STD-480A
o MIL-STD-1521A
·
ANSI/IEEE
standards (Non-Military)
o ANSI/IEEE Stds. No. 828-1983, No.
1042-1987, Std. No. 1028-1988