CS615 – Software Engineering I
o characteristic or attribute of something
o measurable characteristics — compare to known standards such as length, color, electrical properties, and malleability
· Two kinds of quality:
o quality of design characteristics that designers specify for an item
(e.g. grade of materials, tolerances, and performance specifications)
o quality of conformance - degree to which the design specifications are followed during manufacturing
· Software development –
o quality of design - requirements, specifications, and the design of the system.
o quality of conformance - focused on implementation.
Quality control –
Quality assurance –
· The auditing and reporting functions of management.
· Goal of quality assurance - provide management with the data necessary to be informed about product quality, thereby gaining insight and confidence that product quality is meeting its goals.
Cost of quality-
· includes all costs incurred in the pursuit of quality or in performing quality-related activities.
· divided into costs associated with prevention, appraisal, and failure.
o Prevention costs include
§ quality planning
§ formal technical reviews
§ test equipment
o Appraisal costs include
§ in-process and interprocess inspection
§ equipment calibration and maintenance
o Failure costs - those that would disappear if no defects appeared before shipping a product to customers.
§ Internal failure costs - incurred when we detect a defect in our product prior to shipment.
· failure mode analysis
§ External failure costs - associated with defects found after the product has been shipped to the customer.
· complaint resolution
· product return and replacement
· help line support
· warranty work
Software quality -
SQA group -
· Does the software adequately meet the quality factors?
· Has software development been conducted according to pre-established standards?
· Have technical disciplines properly performed their roles as part of the SQA activity?
Activities performed by an independent SQA group:
Software reviews - applied at various points during software development and serve to uncover errors and defects that can then be removed.
o Point out needed improvements in the product of a single person or team;
Primary objective of formal technical reviews
· serves as a training ground
· promotes backup and continuity because a number of people become familiar with parts of the software that they may not have otherwise seen.
· class of reviews that includes:
o round-robin reviews
o other small group technical assessments of software.
· Each FTR is conducted as a meeting that is planned, controlled, and attended.
Review meeting constraints:
NOTE: FTR focuses on a specific (and small) part of the overall software.
· Walkthroughs are conducted for each component or small group of components.
· FTR focuses on a work product (e.g., a portion of a requirements specification, a detailed component design, a source code listing for a component).
· Individual who has developed the work product informs the project leader that the work product is complete and that a review is required.
· The project leader contacts a review leader, who evaluates the product for readiness, generates copies of product materials, and distributes them to two or three reviewers for advance preparation.
· Each reviewer is expected to spend between one and two hours reviewing the product, making notes, and otherwise becoming familiar with the work. Concurrently, the review leader also reviews the product and establishes an agenda for the review meeting, which is typically scheduled for the next day.
· Review meeting attended by
· review leader
· all reviewers
· the producer.
· One reviewer takes on the role of the recorder; that is, the individual who records (in writing) all important issues raised during the review.
1. The FTR begins with an introduction of the agenda and a brief introduction by the producer.
2. The producer then proceeds to "walk through" the work product, explaining the material, while reviewers raise issues based on their advance preparation.
3. When valid problems or errors are discovered, the recorder notes each.
4. At the end of the review, all attendees of the FTR must decide whether to
1. accept the product without further modification
2. reject the product due to severe errors (once corrected, another review must be performed)
3. accept the product provisionally (minor errors have been encountered and must be corrected, but no additional review will be required).
· The decision made, all FTR attendees complete a sign-off, indicating their participation in the review and their concurrence with the review team's findings
· All issues that have been raised are summarized at the end of the review meeting and a review issues list is produced.
· In addition, a formal technical review summary report is completed.
· Review summary report
· answers three questions:
· What was reviewed?
· Who reviewed it?
· What were the findings and conclusions?
· single page form (with possible attachments).
· Becomes part of the project historical record and may be distributed to the project leader and other interested parties.
· Review issues list
· serves two purposes:
1. identify problem areas within the product
2. serve as an action item checklist that guides the producer as corrections are made.
· attached to the summary report.
Minimum set of guidelines for formal technical reviews:
Statistical quality assurance implies the following steps:
To apply statistical SQA, a table is built.
· Software reliability can be measured, directed and estimated using historical and developmental data.
· Software reliability is defined in statistical terms as –
the probability of failure-free operation of a computer program in a specified environment for a specified time
· Simple measure of reliability - meantime-between-failure (MTBF)
MTBF = MTTF + MTTR
o MTTF - mean-time-to-failure
o MTTR - mean-time-to-repair
· Software availability - the probability that a program is operating according to requirements at a given point in time and is defined as
Availability = [MTTF/(MTTF + MTTR)] x 100%
· MTBF reliability measure is equally sensitive to MTTF and MTTR.
· Availability measure is more sensitive to MTTR, an indirect measure of the maintainability of software.
· Software safety - software quality assurance activity that focuses on the identification and assessment of potential hazards that may affect software negatively and cause an entire system to fail.
· Modeling and analysis process is conducted as part of software safety.
o Initially, hazards are identified and categorized by criticality and risk.
o E.g., some of the hazards associated with a computer-based cruise control for an automobile :
§ causes uncontrolled acceleration that cannot be stopped
§ does not respond to depression of brake pedal (by turning off)
§ does not engage when switch is activated
§ slowly loses or gains speed
o Once identified, analysis techniques are used to assign severity and probability of occurrence
o Once hazards are identified and analyzed, safety-related requirements can be specified for the software.
§ Specification can contain a list of undesirable events and the desired system responses to these events.
o The role of software in managing undesirable events is then indicated
§ Quality assurance system - defined as the organizational structure, responsibilities, procedures, processes, and resources for implementing quality management.
§ Created to help organizations ensure their products and services satisfy customer expectations by meeting their specifications.
§ Cover wide variety of activities encompassing a product's entire life cycle including planning, controlling, measuring, testing and reporting, and improving quality levels throughout the development and manufacturing process.
§ ISO 9000 describes quality assurance elements in generic terms that can be applied to any business regardless of the products or services offered.
§ The ISO 9000 standards adopted by many countries including all members of the European Community, Canada, Mexico, the United States, Australia, New Zealand, and the Pacific Rim.
§ After adopting the standards, a country typically permits only ISO registered companies to supply goods and services to government agencies and public utilities.
§ To become registered to one of the quality assurance system models contained in ISO 9000, a company's quality system and operations are scrutinized by third party auditors for compliance to the standard and for effective operation.
§ Upon successful registration, a company is issued a certificate from a registration body represented by the auditors. Semi-annual surveillance audits ensure continued compliance to the standard.
§ ISO 9000 quality assurance models treat an enterprise as a network of interconnected processes.
§ ISO 9000 describes the elements of a quality assurance system in general terms.
§ Elements needed to implement:
o quality planning
o quality control
o quality assurance
o quality improvement
o organizational structure
§ ISO 9000 does not describe how an organization should implement quality system elements.
§ ISO 9001 is the quality assurance standard that applies to software engineering.
§ Contains 20 requirements that must be present for an effective quality assurance system.
§ Because the ISO 9001 standard is applicable to all engineering disciplines, a special set of ISO guidelines (ISO 9000-3) have been developed to help interpret the standard for use in the software process.
§ Requirements delineated by ISO 9001 address topics such as:
o management responsibility
o quality system
o contract review
o design control
o document and data control
o product identification and traceability,
o process control
o inspection and testing
o corrective and preventive action
o control of quality records
o internal quality audits, training
o statistical techniques.
§ For a software organization to become registered to ISO 9001, it must establish policies and procedures to address each of the requirements just noted (and others) and then be able to demonstrate that these policies and procedures are being followed.
§ SQA Plan provides a road map for instituting software quality assurance.
§ Serves as a template for SQA activities that are instituted for each software project.
§ Standard for SQA plans has been recommended by the IEEE
§ Standards, practices, and conventions section
o lists all applicable standards and practices that are applied during the software process (e.g., document standards, coding standards, and review guidelines).
o all project, process, and (in some instances) product metrics that are to be collected as part of software engineering work are listed.
§ Reviews and audits section
o identifies the reviews and audits to be conducted by the software engineering team, the SQA group, and the customer.
o provides an overview of the approach for each review and audit.
§ Test section references the Software Test Plan and Procedure
o defines test record-keeping requirements.
o problem reporting and corrective action defines procedures for reporting, tracking, and resolving errors and defects, and identifies the organizational responsibilities for these activities.
§ Tools and methods section
o Tools that support SQA activities and tasks
o software configuration management procedures for controlling change
o contract management approach
o establishes methods for assembling, safeguarding, and maintaining all records; identifies training required to meet the needs of the plan;
o defines methods for identifying, assessing, monitoring, and controlling risk
· Output of the software process:
· Number of software configuration items (SCIs) grows rapidly as software process progresses.
· Change may occur at any time, for any reason.
· First Law of System Engineering:
"No matter where you are in the system life cycle, the system will change, and the desire to change it will persist throughout the life cycle."
· Four sources of change:
· Software configuration management:
o Set of activities that have been developed to manage change throughout the life cycle of computer software.
o Viewed as a software quality assurance activity that is applied throughout the software process.
· Software configuration management concept that helps control change without seriously impeding justifiable change.
· IEEE Std. No. 610.12-1990 defines a baseline as:
A specification or product that has been formally reviewed and agreed upon, that thereafter serves as the basis for further development, and that can be changed only through formal change control procedures.
· Software engineering context:
A baseline is a milestone in the development of software that is marked by the delivery of one or more software configuration items and the approval of these SCIs that is obtained through a formal technical review.
· After SCIs are reviewed and approved, they are placed in a project database (also called a project library or software repository).
· When a member of a software engineering team wants to make a modification to a baselined SCI:
o it is copied from the project database into the engineer's private work space.
o this extracted SCI can be modified only if SCM controls are followed
o A SCI is a document, an entire suite of test cases, or a named program component (e.g., a C++ function or Java class).
o SCIs are organized to form configuration objects
o cataloged in the project database with a single name.
o Has a name, attributes, and is "connected" to other objects by relationships.
o Configuration objects, Design Specification, data model, component N, source code and Test Specification are each defined separately.
o Each of object is related to others by the arrows.
§ Curved arrow indicates a compositional relation.
· data model and component N are part of the object Design Specification.
§ Double-headed straight arrow indicates an interrelationship.
SCM introduces a set of complex questions:
Must define five SCM tasks:
Two types of objects can be identified:
A "unit of text" that has been created by a software engineer during analysis, design, code, or test.
A collection of basic objects and other aggregate objects.
Each object has a set of distinct features that identify it uniquely:
a character string that identifies the object unambiguously
list of data items that identify
o the SCI type (e.g., document, program, data) represented by the object
o change and/or version information
entities that are provided, processed, referenced or otherwise required by the object
a pointer to the "unit of text" for a basic object and null for an aggregate object
· Identification scheme must recognize that objects evolve throughout the software process.
o Before an object is baselined, it may change many times
o After a baseline has been established, changes may be quite frequent.
o Possible to create an evolution graph.
o Describes the change history of an object
§ Configuration object 1.0 undergoes revision and becomes object 1.1.
§ Minor corrections and changes result in versions 1.1.1 and 1.1.2
§ major update that is object 1.2.
· Version control combines procedures and tools to manage different versions of configuration objects that are created during the software process.
· Change control combines human procedures and automated tools to provide a mechanism for the control of change.
· Change Control Process:
o A change request is submitted and evaluated to assess technical merit, potential side effects, overall impact on other configuration objects and system functions, and the projected cost of the change.
o The results of the evaluation are presented as a change report, which is used by a change control authority (CCA) — a person or group who makes a final decision on the status and priority of the change.
o An engineering change order (ECO) is generated for each approved change.
The ECO describes the change to be made, the constraints that must be respected, and the criteria for review and audit.
o The object to be changed is "checked out" of the project database, the change is made, and appropriate SQA activities are applied.
o The object is then "checked in" to the database and appropriate version control mechanisms are used to create the next version of the software.
· "check-in" and "check-out" process implements two elements of change control
o Access control - governs which software engineers have the authority to access and modify a particular configuration object.
o Synchronization control - helps to ensure that parallel changes, performed by two different people, don't overwrite one another
How to ensure that the change has been properly implemented:
The formal technical review focuses on the technical correctness of the configuration object that has been modified.
A software configuration audit complements the formal technical review by assessing a configuration object for characteristics that are generally not considered during review.
· The audit asks and answers the following questions:
Configuration status reporting is an SCM task that answers the following questions:
(1) What happened?
(2) Who did it?
(3) When did it happen?
(4) What else will be affected?
· Each time an SCI is assigned new or updated identification, a CSR entry is made.
· Each time a change is approved by the CCA (i.e., an ECO is issued), a CSR entry is made.
· Each time a configuration audit is conducted, the results are reported as part of the CSR task.
· Output from CSR may be placed in an on-line database so that software developers or maintainers can access change information by keyword category.
· CSR report is generated on a regular basis and is intended to keep management and practitioners appraised of important changes.
Number of software configuration management standards:
· ANSI/IEEE standards (Non-Military)
o ANSI/IEEE Stds. No. 828-1983, No. 1042-1987, Std. No. 1028-1988