CONTENTS

Design and Evaluation of Software *

Discipline of Human-Computer Interaction Design *

• Design Principles *
• Design Methodologies *
• Getting to know users and their tasks *
• Idea Generation *
• Envisionment and Prototyping *
• The Role of Metaphor *

Evaluating systems and their user interfaces *

Five approaches to system & interface evaluation *

Theory-based design *

Preserving Design Rationale *

Usability Design Process (Gould) *

• General Observations about system Design: *
• Steps in Designing a Good System *
• Principles of Usability Design *
• Principle 1: Early and Continual Focus on Users *
• Principle 2: Early - And Continual - User testing *
• Principle 3: Iterative Design *
• Principle 4: Integrated Design *

---

Design and Evaluation of Software

• Design
• Build
• Test

• Evaluate:
• Existing System
• Current Work Practices and Products
• Output:
• Rapid Prototypes and Mockups
• Alpha & Beta Versions

• Engineering
• Programming
• Cognitive Psychology
• Human Factors
• Anthropology
• Graphics Design
• More?

Discipline of Human-Computer Interaction Design

Answers: Found In:

Design Principles

1. Know the user
2. Optimizes operations
3. Minimize memorization
4. Engineer for errors

Heckel (1991) - 30 design elements

Shneiderman (1992) - Eight golden rules:

1. Strive for consistency
2. Enable frequent users to use shortcut
3. Offer informative feedback
4. Design dialogues to yield closure
5. Offer simple error handling
6. Permit easy reversal of actions
7. Support internal locus of control
8. Reduce short-term memory load

• Early focus on users
• Designers should have direct contact with intended or actual users
• Early - and continual - user testing
• Only feasible approach to successful design is empirical through observation measurement, enhancement
• Interactive design
• Design, implementation, testing, feedback, evaluation, change iterative.

 

Design Methodologies

Informal methodology - 4 Phases:

1. gearing up phase
• learn about related systems for good starting points
• learn about user interaction standards, guidelines, and development procedures
2. initial design phase
• preliminary specification of user interface
• collect information about users and work
• develop testable behavior goals
• organize work to be done
3. iterative development phase
• continuous evaluation and feedback based on user feedback
4. system installation phase
• techniques for installing, introducing, marketing , etc.

Nielsen (1993) Develops "usability engineering lifecycle model" - 11 stages:

1. Know the user
• each user's characterization
• user's current & desired tasks
• functional analyses
• evolution of user and job
2. Competitive analysis
3. Setting usability goal & Financial impact analysis
4. Parallel design
5. Participatory design
6. Coordinated design and total interface
7. Apply guidelines and heuristic analysis
8. Prototyping
9. Empirical testing
10. Iterative design to capture design rationale
11. Collect feedback from field use

• Competitive analysis can be cost effective way of studying and evaluating approaches to product functioning and interface design
• Set quantitative usability goals
• (e.g. number of user errors per hour
• level of satisfaction expressed by users)
• Parallel design useful in brainstorming
• Participatory design - users become designers
• Constancy of UI achieved through corporate standards and GUI guidelines

• figure out who's going to use the system to do what
• choose representation tasks for task-centered design
• plagiarize
• rough out design
• think about it
• create a mockup or prototype
• test it with users
• iterate
• build it
• tweak it
• change it

• Suggest that designers work within existing interface frameworks
• Incorporate existing applications within their solutions
• Copy interaction techniques from other systems

Lewis & Rieman -

• Close personal contact
• Profile user's:
• age,
• education
• computer task literacy and experience
• reading and typing ability
• attitude, motivation
• Choose representative tasks that provide functional coverage
• Profile jobs and tasks:
• structure and importance
• Level of training and support provided
• Nature of system use (e.g. regular, mandatory)
• Produce scenarios
• Profile behavior:
• analysis of social setting (usually requires questionnaires interviews)

Idea Generation

• observation
• listening
• brainstorming
• metaphor
• sketching
• scenario crations
• free association
• mediation
• juxtaposition
• searching for patterns
• "lateral thinking"

Envisionment and Prototyping

Envisionment of interfaces - the formulation and exploration of system and interface, ideas and metaphors at a very early stage of the design process.

Methods of envisionment - stories, scenarios, storyboards, flipbooks, animated drawings, cutout animation, animation of real objects, computer animation, computerscripted interactive prototypes.
 

The Role of Metaphor

Def: a figure of speech in which a word or phrase denoting one kind of object or action is used in place of another to suggest a likeness or analogy between them

Metaphors help users understand a target domain they don't understand in terms of a source domain they already understand
(e.g. typewriter - wordprocessors)

Source               Target.

Joseph McGrath (1994) - framework for understanding methods of evaluation:

• "generalizability" of evidence
• Over population of "actors"
• "precision" of measurement
• of the "behaviors" studied
• "realism" of the situation
• or "context" where evidence is gathered

• correlation vs. causation
• randomization, true experiment and statistical inference

1. Field strategies - studies systems in use on real talk, real work settings
1. Field studies - observing without interviewing
2. Field experiment - observe impact of changing aspect of environment of system (e.g. beta testing products)
2. Experimental Strategies - carried out laboratory
1. Experimental simulations create real system in lab for experimental purposes by real users
2. Laboratory experiments - controlled experiments used to study impact of particular parameter.
3. Respondent strategies
1. Judgment studies responses from small set of judges - designed to give information about stimulus (e.g. Delphi methods)
2. Sample surveys - responses from large set of respondents about respondents (e.g. questionnaires)
4. Theoretical strategies
1. Formal theory - gives qualitative insights (e.g. theory of vision)
2. Computer simulation - run on computer to derive predictions about computer performances.

1. Heuristic evaluation with usability guidelines
• Must make guidelines so they are testable.
• Sometimes difficult to apply guidelines to real design problems
• Difficult to deal with large set of guidelines


Nielsen (1994) - 10 Design Heuristics:
(Based on factor analysis of 249 usability problems on 11 projects)

1. Visibility of System State
2. Match between system and real world (e.g. speak user's language)
3. User Control and Freedom (e.g. easy exit, undo)
4. Consistency and standards (e.g. same words mean same thing in different contexts)
5. Error prevention
6. Recognition rather than recall (e.g. options and actions should be visible)
7. Flexibility and efficiency of use (e.g. accelerators, short cuts, customization)
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors (e.g. error messages in plain language)
10. Help and Documentation

2. Cognitive Walkthroughs
• Inspired by structured code walkthroughs
• Here - a set of representative tasks is selected and stepped through - keystroke-by-keystrokes, menu selection-by-menu selection.
• Developers fill out forms that require them to specify information about user's goals, tasks, subtasks, knowledge, visible state of the interface, and relations and changes analysis.
Questions to ask:
• Will users try to achieve the right effect?
• Will users notice that the correct action is available?
• Will users associate the correct action with th effect trying to be achieved?
• If the correct action is performed, will the user see that progress is being made toward solution of the task?
• Theory underlying cognitive walkthroughs based on psychology of inexperienced users.
• Approach is intended to evaluate a design for ease of learning
• Cognitive walkthroughs suggest specific psychological explanations for problems thereby pointing developers toward solutions.

2. Usability testing
• Cognitive walkthroughs rely on intuition of interface specialists
• Need to be augmented with experience
• Participants are recorded
• "Errors" identified
• Make interface changes
• Videotaping users trying to use system
• "thinking aloud technique"
3. Usability Engineering
4. User testing that is more formal in the sense that interface specialists set explicit quantitative performance goals known as metrics
5. e.g. new users must be able to create and save forms in first 10 mins.
6. Arguments for explicit quantitative goals:
7. Human factors engineers taken more seriously in engineering setting because they adopt quantitative, objective goals familiar to engineers
8. Progress can be charted and success recognized

5. Controlled Experiments
• Adv:
• Studying individual behavior without influence of groups
• Focus on studying low-level perceptual cognitive or motor activities
• Types of designs:
• Within-subjects: each subject experiences the different options being tested
• Between-subjects: each subject just tries one of the alternatives

• Heuristic evaluation with empirical usability testing
• Cognitive walkthrough can be helpful

Possible Uses of Evaluation Methods in a Development Process:

• Information Collection - e.g. interviews and questionnaires
• Concept Design - e.g. interviews, usability testing, controlled testing
• Functionality and Interface Design - usability testing, human information processing simulations
• Prototype Implementation - usability testing, heuristic evaluation
• Deliverable System Implementation - usability testing
• System Enhancement and Evolution - interaction analysis, interviews and questionnaires, field studies

Theory-based design

• Most methods ad hoc need solid theory
• Theories exist but are they fundamental?

Recording the design history of its rationale could serve several purposes:

• Lead better design by looking at the process of considering alternatives
• Useful when modifications considered.

General Observations about system Design:

• Nobody gets it right the first time.
• Development is full of surprises.
• Developing user-oriented systems requires living in a sea of change.
• Developers need good tools.
• Even with the best-designed system, users will make mistakes using it.

1. Define problem customer wants to be solved.
2. Identify tasks user wants to be performed.
3. Learn user capabilities.
4. Learn hardware/software constraints.
5. Set specific usability targets.
6. Sketch out user scenarios.
7. Design and build prototype.
8. Test prototype.
9. Iteratively identify, incorporate, and test changes until:
• Behavioral targets are met
• Critical deadlines are reached.
10. Install system
11. Measure customer reaction and acceptance.

Principle 1: Early and Continual Focus on Users

• Decide who users will be & what they will be doing with system
• Methods to carry this out:
1. Talk with users
2. Visit customer locations
• Find out problems, difficulties, worries
• Show them what you had in mind for design
• Assume the intended users are the experts
3. Observe users working
4. Videotape users working
5. Learn about the work organization
• (might have to match system design with corporate design)
6. Make potential user think out loud during actual work
• (realtime connect)
7. Try worker's job yourself
8. Make user part of design team
9. Perform task analysis (step-by-step)
10. Use surveys and questionnaires
11. Set testable behavior target goals

1. Create a query on the displayed query panel using the table SCHOOL.COURSES.
2. Delete all column names except COURSE and TITLE.
3. Save the current query with the name CTITLE.
4. Run the query once.
5. Get current query panel displayed.
6. Clear current query panel so it contains nothing.

Principle 2: Early - And Continual - User testing

• First Design is not perfect.
• Must make measurements.
• Methods to carry this out:
• Printed or video scenarios
• Exact details:
• exact layout and wording on screen
• What keys to press, system's response
• Early user manuals
• Physical mock-ups
• Simulations - paper and pencil or computer
• Early processing - get people to work with system
• Early demos
• Think aloud
• Make videos of workers using system
• Hallway-and-storefront methods - put in situ
• Online forums
• Formal prototype tests with formal behavioral methods
• Try to destroy contests
• Field studies
• Follow-up studies

Principle 3: Iterative Design

• Identify required changes
• Make changes
• Methods to carry this out:
Software Tools (e.g. UIMS)
• Need to make changes in UI possible and easy
• Brings UI within control of human factors people not programmers
• Makes work more productive
• Tools have inherent conceptual, semantic, and procedural ways of doing things
• Tools facilitate consistency

Principle 4: Integrated Design

• ALL Design work evolves in parallel