Touch, Gesture, and Marking

CS631p- Human-Computer Interaction

Spring 2001

Lecture 7

Touch, Gesture, and Marking

Haptic input (Greek for contact) - involves physical contact between computer and user.

Hands on mouse
Foot on pedal
Tongue with joystick

Relating Task and Technology

Designers must find match between application and input technology.
Most recognize the relevant dimensions along which an application's demands can be characterized
Must know how each technology performs along those dimensions

Basic set of generic tasks:

Pursuit tracking

Test: Fly moves over screen under computer control

Operator uses control device to track fly's motion
Test how many times fly can be swatted in a given time interval
Parametrics: speed target moves
Control: display ratio (C:D)

Ratio between distance the controller must be moved to cause tracker to move given distance on display.

e.g. C:D 2:1, 2 cm of controller results in 1 cm. on screen.

C:D ratio can be a variable that changes as controller moves

Button push is another parameter - good on mouse more, difficult on tablet

Target Acquisition

Test: User selects each of a number of rectangles displayed on screen

selected by positioning tracking symbol and signaling with button
statistics: how long it takes to select full set of targets
check out target size effect on speed

Fitts law: MT = a + b log2 (D/W + l)

Movement time (MT): time to move the hand to a target of width (W) which lies (D) distance away, where a is a constant and b=100 [70 - 120] msec/bit.

Variations on task:

Homingtime - moving between text entry device and pointing device

- time it takes (test by having user hit space bar after clicking on rectangle)

Number of dimensions - higher dimensional tasks

What is effective target within two dimensions or more?

What is effect of approach angle?

Dragging vs. rubber-band lines - object is dragged from square to square.

- Also Fitts law task -> Holding mouse down can effect target acquisition

Left hand vs. right hand - moving from dominant to non-dominant varies across devices.
Free-hand inking - attempt to write signature with different technologies.
Tracing and digitizing - CAD, cartography, graphic arts

- Relative devices useless

- Absolute devices vary widely

- Resolution important (eg cartography)

Constrained motion - must be able to move tracker rapidly over a straight-line path.

e.g. - scrollbar mechanism

- move mouse in x or y

- Thumbwheels work better than a mouse

- Task: tracking symbol is a ball that is dragged along a linear path without crossing parallel lines.

How is speed affected by input device?
How is speed affected by path width?
How about horizontal vs. vertical?
What about circular motion ?

3D Input and Interaction:

Need 3D equivalents of 2D tasks.
What devices support applications of higher dimensionality?
Can support 3D with lower devices (eg. virtual 3D trackball).

A Taxonomy of Input Devices

Categorization

Table uses hierarchy of criteria.

Generality and Extensibility: tradeoff between the generality of a computer workstation and groups of specialized tasks it is used for. THUS: No optional device.
Relative vs. absolute controls: affects dialogs.
What our taxonomy does not show: Only considers continuous devices.

Chunking and Phrasing

Human -Computer dialogs can benefit from appropriate phrasing such as found in music or speech.
Most human computer dialogs are composed like speech: selecting/positioning, positioning/scaling, navigating/scaling.
Structure that emerges from appropriate phrasing can accelerate the process.

e.g proof-reading marks

Novices "chunk" together concepts.

Modes and Mode Errors:

Mode error: misclassification of a situation resulting in actions that are inappropriate for the true situation.

Marking
Style of interaction (gesture driven, e.g. pen-based)

User's input is stream x,y coordinates called "digital ink", thus marking interfaces
Most marking systems are different to use and prone to mode errors.

Who Does the Recognition?

Recognition is heart of marking system: block characters, cursive script, etc.
Pattern recognition is hard and still unsolved.
Should focus on systems where marks are recognized by user.

What is Recognition?

High level marks, such as proofreaders, easier to learn.

Self-Revelation and Marking Menus:

Marking systems assume a form-filling style. May not have enough space, such as on PDA.
Alternatively marking-menus:

User presses down, waits until a pie menu appears and strikes through slice.
Alternatively, mark-ahead by making a mark without waiting for menu. System recognizes physical movement.

Working within the Marking Idiom:

Problems when designers do not adequately conceptualize the medium.
Free-form text sometimes faster than keyboard, 2² & 22 easier drawn.
But if text is entered on a line, using text with a graphical keyboard and stylus is faster and more accurate (eg. 0 vs. O, number vs. letter).
PDA too small to vie with graphical keyboard but could use some shortcuts.

e.g .Uni-strokes alphabet

e.g. With mnemonics

Strengths:

Can take about one hour to learn.
Only single stroke.
No segmentation problem - which stroke belongs to what.
Can recognize character when each one is written upon other.

Marking system leave an explicit audit trail of user's actions.
Figure/ground relationship on computer - know which marks made by computer and which by person

Situated Design

If electronic version of system is not faster than pen and paper then redesign system.

Gestures

Some systems "gesture" refers to marking interfaces.
Other system "gesture" itself is recognized.

e.g. Myron Krueger - Videoplace - non-invasive (video camera, image processing)

The "Videoplace" system perceives one or moreparticipants and responds to their movements in realtime using video cameras.
The "Videoplace" system identifies each participant's head, arms, legs, hands and fingers and determines their rate of movement.
Each participant's image may be moved, scaled or rotated anywbere on the screen.
"Videoplace" has two modes:

It interacts directly with an individual.
It mediates a dialog between two participants: one who understands and controls the system and a second naive participant. The controller can interact with the other person by using the image of his or her hands.

Gestures in Collaborative Work