Hypertext and Multimedia

• Xerox PARC -Notecards System - research prototype
• Blocks of text or graphics about the size of index card - called notecards - can be arranged and linked together
• Apply Hypercard (1987) - interface builder and scripting lanuguage - could be used to implement hypertext systems
• Other System - OWL's Guide (1986)
• 1993 - 11 hyperfictions published
• Contemporary Sample: Hegirascope2 by Stuart Moulthrop
• Hyperizons - Web Resource
• 1993 - Mosaic released

• Nodes And Connected Links To Form Networks Or Webs
• Nodes May Contain One Or More Media (Text, Graphics, Sound Animation Video)
• Links - uni- / bi- Directional
• Anchors Are points in a node where links are attached (e.g position marked by button)
• Browser - provides set of navigational aids such as:
• active table of contents
• back tracking and history lists
• bookmarks
• guided tours
• maps and web viewers
• timestamps
• Advantages of Hypertext:
• ease of tracing forward and backward through links
• ease of adding links
• information structuring = hierarchical and non-hierarchical
• global views
• customized documents
• modularity of information
• consistency of information regardless of how node is referenced
• collaboration
• Disadvantages of Hypertext
• disorientation
• cognitive overhead

• Scope - from small-local systems to world-spanning information systems
• Writing space vs. information tool:
• all writing becomes part of one huge hypertext - OR -
• Hypertext becomes information access tool to existing texts
• Unlimited vs. Fixed node size - should node size be limited?
• Literal vs. virtual links - i.e. hardwired vs. dynamically created links
• Browsing vs. authoring - some systems are "read-only" others authoring
• Structuring vs. navigation - is overall structure of hypertext important or local navigation?
• Target task domain - different tasks have different constraints

• Bellcore's SuperBook
• Structures existing documents that were written in one of several text formats
• Converts pre-structured text directly into Hypertext
• Allows several ways of browsing
• Rich indexing - finds relevant text related words
• Fisheye viewing - outline with greater on nearby topics and less detail on distant topics
• Word lookup - table of contents of document with number of occurrences of a given word in each section
• Experiments against paper documents
• For most tasks superbook was equal or better
• But when superbook did not contain the exact search work - text was better
• Lessons learned from SuperBook
• Does not cost much to convert from text to Hypertext
• Slightly faster on search
• Saves on paper for Big Big texts ( e.g. commercial airplane documents - too big to fit in plane) - saves time with hypertext)
• Problems with structuring hypertext
• Users benefit from structure
• But have difficulty in specifying structure explicitly
• Tools must help users structure environment

Standards and Commercial Market

• Prominent standard (SGML) standard generalized markup language
• e.g. authoring tool (Framemaker 5.5 and SGML)
• Operating System support:
• Microsoft's (OLE) object linking and embedding
• Apple's publish/subscribe
• Sun/HP distributed objects management facility
• Hypertext authoring tools - Macromedia Dreamweaver, MSWord, Netscape Composer, MS FrontPage

• Dexter Model:
• Developed at the Dexter Inn in Sunapee, NH in 1988 by a group of hypermedia designers.
• Three Layers:
• Run-time Layer: deals with user interface issues
• Storage Layer: models a hierarchical database interconnected by relational links - model's major focus
• Within-Component Layer: contains the content and structure within the nodes.
• Primarily a Hypertext model, although nodes in the storage layer may contain multimedia components
• Amsterdam Model:
• Developed by Lynda Hardman, Dick Bulterman and Guido van Rossum (CACM, Feb. 1994)
• Adds time and context to Dexter Model.
• Deals with synchronization. Asks Question: What would happen if a hypermedia document tried to link to the 10 o'clock news when it is only 9:00 pm?
• Hypermedia documents are composed of atomic elements called assest.
• Assets are various components such as text, still images, audio, and video, are called assets.
• Hyperdocuments are documents that include assets..
• Each asset has its own standard format, and trade-offs in size and quality.
• A composite element may consist of several (time-based) atomic elements

• Web search engines - Yahoo, Google, Altavista, Northernlights, etc. - Must formulate query

• On-line Textbook - Elements of Hypermedia Design by Peter Gloor
• A Framework for Generating Adaptable Hypermedia Documents by Lloyd Rutledge, Jacco van Ossenbruggen, Lynda Hardman , and Dick C.A. Bulterman
• The Dortmund Family of Hypermedia Models - Concepts and their Application by Klaus Tochtermann and Gisbert Dittrich

• 1970's - laser videodisks - holds about 54K still images per side
• 1978 - Aspen Movie Map Project ( MIT Architecture Machine Group)
• Used  videodiscs, controlled by computers
• Allowed user to "drive" down the streets of  Aspen, CO.
• Every street and turn was filmed in both directions, with photographs taken every three meters.
• Straight street segments on one videodisc and curves on another, created artificial, seamless driving
                   experience.
• Used two screens:
• vertical screen showed video
• horizontal screen showed an Aspen street map.
• Users pointed to a spot on the map and jump to that spot, enter buildings, see archival photographs, take guided tours and leave an electronic trail.
• 1980's - replaced by cd-roms - 600mb of data e.g. CD-DVD
• 1980's - prototypes of computer-based navigation
• Dataland (1984) - wall-size display of objects such as faces, maps, TV's, telephones, calculators, calendars, books, letters
• Users could wander over word, watch, listen, zoom in to hear more
• Movie Manual (1981) - included video and sound in electronic manual for auto transmission repair
• Today - MIT's StreetScape Project
• Allows users to click on the orthophoto in the upper left frame to display the image at the lower right.
• The image can be navigated by panning, zooming and clicking "hotspots" to move to different locations.


Encoding and Representation of Multimedia Documents

• 1986 - Phillips CD/I - Compact disk interactive for storing multimedia
• 1987 - RCA digital video interactive (DVI) for encoding one hour of full-motion video and FM-quality sound
• 1988 - ISO standard CD-ROM
• Apple Quicktime digital compression
• 1991 - JPEG still image compression
• 1991 - MPEG movie compression

• Requires hardware such as ISDN, ATM, etc.

• Easy to use tools for authoring:
• Apple's Hypercard
• Macromedia Director
• Rumpus Toys
• Ezone Games
• Venice Waterways
• Others

• Can create new content from old (sampling)
• Two issues:
• How do you get legal rights?
• How do you find, represent, and manipulate pre-existing multimedia objects
• Answer to latter question
• Need deep descriptions of video - in essence, access to original script
• e.g. Marc Davis - Media Streams - gives programmer hierarchical iconic language for indexing sequences of video
• Categories that describe actions, characters, objects, locations, etc.
• Three parts:

1. Media streams timeline window
- Contains iconic annotations of video.
- Annotation has an icon and an adjacent color-bar over where icon's description is valid
- Annotations are distributed in the Timeline window's 44  different streams
- Each stream shows a unique aspect of  video content such as settings, characters, objects, actions, camera motions.
2. Media Streams Icon Space
- Interface into hierarchically-indexed "dictionary" of iconic descriptors
- Top row of buttons -  access to top-level icon hierarchies:
 Time, Space, Characters, Objects, Character Actions, Object Actions,
 Cinematography, Screen Positions, Weather, Subjective Thoughts,
 Recording Media, Transitions, Relative Positions.
- White upper region is a cascading browser for the icon  hierarchies.
- Icon arrangement from left to right denotes increasing specialization
 read as: "space, geography, land, continent, North America, USA, Northwest States, Washington State".
- Vertical row arrangement denotes an orthogonality of descriptive domain;
•  first row contains descriptors for geographic space
•  second and third rows contain icons for functional and topological divisions of space.
 - Lower half of the Icon Space window is the icon palette
utility for constructing and retrieving iconic sentences.

3. Examples of Media Streams icons
•  Small selection from more than 6000 icons in Media Streams visual annotation lexicon
• Icons show a cross-section of the icon hierarchies, including:
• Historic Period, Calendar Time, Time of Day, Functional Building
• Space, Topological Relationships, Public Outdoor Space, Wilderness,
•  Vehicles, Character Body Types, Occupations, Tools, Food, Animals,
•  Weather, and a variety of other objects and cinematographic  relationships.

• Education and training
• Information resources, documentation, and research
• Communication, coordination, and collaboration
• Entertainment

• VISIPHONE (Sociable Media Group - MIT Media Lab)


 
• VisiPhone displays two parallel visualizations:
• (input audio) derived from the local sound reaching the device
• (output audio) derived from the sound emanating from it

	Speaker 1       Speaker 2
	Both Speakers

• musicBottles  (Tangible Media Group - MIT Media Lab)
• Tangible interface for interaction with musical composition
• Core concept - using bottles as graspable containers and controls for digital information.
• Primary mode of interaction with information - physical  manipulation of  bottles
• Installation :
• specially designed table on which three stoppered bottles are placed.
• Glass bottles represent three performers in a classical music or jazz trio.
• Electromagnetic tags embedded in  bottles allow each  to be wirelessly identified and sensed.

Movement and uncorking of the bottles controls the different sound tracks and colored light patterns rear-projected onto the tables translucent surface.