CS835
- Data and Document Representation & Processing
|
Lecture 12 - Hypermedia VI – Physical
Hypermedia |
1) Combines real and virtual
2) Interactive in real time
3) Registered in 3D
Real desk with virtual lamp and two virtual chairs
The user wears or carries a device, usually on the head or hands, to obtain information about physical objects.
Augment the physical object
· The physical object is changed by embedding input, output or computational devices on or within it.
e.g. Electronic bricks contain simple electronic
devices such as sensors (light, sound, touch, proximity), logic devices
(and-gates, flip-flops, timers) and action bricks (motors, lights).
·
Ubiquitous computing in which specially-created objects are detected by
sensors placed throughout the building.
Augment the environment
surrounding the user and the object
· Independent devices provide and collect information from the surrounding environment, displaying information onto objects and capturing information about the user's interactions with them.
e.g.
o
The Digital Desk uses a video camera to detect where a user is
pointing and a close-up camera to capture images of numbers, which are then
interpreted via optical character recognition.
o
A projector overhead projects changes made by the user back onto the
surface of the desk.
1) The user may have a column of numbers printed on a particular page.
2)
The user points to numbers, the digital desk reads and interprets them,
and then places them into an electronic spreadsheet, which is projected back
onto the desk.
3)
Using his fingers, the user can modify the numbers and perform
"whatif" calculations on the original data
A user points to a column of numbers printed on a paper
document and then uses an electronic calculator projected on the desktop.
·
eTag system - simple connection between physical object and an action
or a piece of information on the Web, such as the related Amazon page.
·
FindEntity system [ http://www.thax.de/english/frame.html
] - provides support for locating physical material inside buildings and
offices using Radio Frequency Identification (RFID).
Examples of augmented reality approaches, with relevant
technologies and applications
Peel back the MRI skin and see where the internal structures are located relative to the viewpoint of the camera
Superposition of MRI Scans on patients
Left Figure: Attached several Logitech 3D trackers (the small triangles in
the figure shown
above) to key components of the printer, allowing the system to monitor
their position and orientation
Right
Figure: shows a virtual world designed by KARMA, viewed ``live'' through a
see-through head-mounted display.
1. Direct placement of information
2. Can be tuned to intended job training
3. Technology can be applicable to any sequenced procedure
4. Applications include assembly, disassembly, maintenance and training
5. Utilizes either optical see-through or video see-through
6. Can reduce or potentially eliminate paper training material
Used to annotate objects and environments with public or private information.
The user points at the exhaust manifold on an engine model, and the label "exhaust manifold" appears.
Windows on
the World - 2D Windows for 3D Augmented Reality
Windows attached from a standard user interface
onto specific locations in the world
To accomplish the simultaneous capture and
display:
Telecubicle - 3D real time acquisition data combined with static 3D
background (latter is a laser scan of an office). Remote participant Amela
Sadagic in Armonk, NY, and a local participant Wei-Chao Chen in Chapel Hill,
NC.
Prototype campus information system.
The user wears a backpack and headworn display, and holds a handheld display
and its stylus
Implementation Framework
e.g. a FieldWorks laptop machine for the backpack computer, which offers
us three PCI and three EISA expansion slots (currently used among others for a
powerful 3D graphics adapter and a 6-serial port expansion card).
|
Physical
Hypermedia: Organizing Collections of Mixed Physical and Digital Material,
Kaj Grønbæk, Jannie F. Kristensen, Peter Ørbæk Mette Agger Eriksen
Physical Hypermedia – make objects from physical world into first class
objects in hypermedia systems.
Ubiquitous and pervasive
computing
·
Main
focus to develop infrastructures for dealing with display enabled devices in a
variety of different scales, from interactive walls to PDAs, cell phones, and
wrist watches etc.
·
Far
less focus on how to associate computing with familiar physical artifacts such
as paper, folders, binders, models, samples etc. as being used in many work
domains
Augmented Reality
Hypermedia and spatial
relationships
Empirical studies:
1) Landscape architects take different approaches for
organizing material depending on the degree of formalization of procedures in
terms of work context.
2) Physical space is not just used for organizing with
the purpose of re-finding material
3) It is also used as an exhibition of ongoing work, as
well as the creation of an inspirational and creative atmosphere.
·
Relationships
between physical and digital world:
1.
Physical-only: Only physical object, the digital world has no trace of the
object, at all.
2.
Physical-with-digital-id: The digital world posses an ID plus some meta-data
relating to the physical object but no digital representation; for example: a
stone or a brick with an RFID tag on it.
3.
Physical-with-low-resolution-digital representation: a pen tracked drawing, a
scanned document or a photo of an object.
4.
Physical-generated-from-digital: a printed map, drawing or report.
5. Digital-only: Content that cannot be
printed or otherwise made physical/tangible, e.g. a video, sound or source
files (they may be stored on removable media like CD/DVD, but the content cannot
be accessed in the non-digital physical world).
A document object in
Topos. A red corner (top left) indicates that a tagged item is linked to it but
not present at the tag-reader.
RFID reader: a hand-held
that only reads a single tag at a time.
A
tube with an associated collection tag is placed on the tag-reader.
· Representing physical material – snapscanning
Design is a circular behavior
o
a snapshot of a sketch
in Topos makes the use of tags more efficient and flexible.
The first working prototype
of the “Snapscanner” allowing taking a snapshot and linking a tag to the
physical material in one operation.
· A tooltag is an RFID-tag
which is coupled to a command in the Topos hypermedia system rather than to a
piece of information.
· Allows issuing of commands
in Topos by placing the physical tooltag on the tag-reader alone or together
with a piece of physical material.
Tooltags: specific RFID tags
have been associated with commands to be invoked via the tag-reader.
Invoking a command on an object by placing object and tooltag on the RFID reader.
Modeling Physical Hypermedia Applications,
Silvia
Gordillo Gustavo Rossi Fernando Lyardet
·
Example scenario –
o
Museum in which visitors are equipped with portable
computer devices, and there is some location sensing mechanism.
o
When the visitor stands in front of an artwork, he
can see its digital representation.
o
He is presented with a set of anchors that allows
him to navigate to other nodes (information items) related with the artwork.
o
When one of these nodes represents a physical
object, he is informed on how to reach that object (perhaps another artwork);
o
Can choose to traverse the physical space (“walk”
the link) towards this node or just continue his tour.
o
Not just augmenting the physical object (artwork)
with some digital information but also providing some kind of linking to other
digital or physical objects.
·
Approach to model PH applications extends the
Object-Oriented Hypermedia Design Method (OOHDM) by adding new abstractions and
re-defining the semantics of basic navigational behaviors.
The Design Approach
·
They define a PH application as a hypermedia
application (i.e. the access to information objects is done by navigation), in
which all or some of the objects of interest are real-world objects which are
visited by the user “physically”.
·
They assume that in a PH application there is some
underlying location-sensing technology that allows the application to be aware
of the actual user’s position.
·
Two different ways to implement hypermedia
navigation:
·
Must specify unambiguously the system’s intended
structure and behavioral semantics.
·
Must express which are the objects of interest and
their properties including:
o their
locations
o how
they are linked
o which
links should be implemented as conventional
o which
should be “walked” by the user
·
OOHDM partitions the development space into four
activities:
o conceptual
modeling
o navigation
design
o abstract
interface design
o implementation
·
During conceptual modeling we describe the
application classes and their relationships using UML.
o Focus
is placed on generic application’s behavior
o application
is modeled neutrally with respect to navigation issues
·
In OOHDM, a hypermedia or Web application is seen as
a navigational view over the conceptual model
o can
specify different views according to the user profile or role.
o can
define a different navigational structure, which will reflect objects and
relationships in the conceptual schema according to the tasks this kind of user
must perform.
o The
navigational structure of a Web application is defined by a schema, containing
navigational classes such as nodes, links, anchors and access structures.
o The
semantics of nodes, links and anchors are as usual in hypermedia applications.
o Access
structures, such as indexes, represent possible ways for starting navigation.
·
The abstract interface model defines which interface
objects the user will perceive (in particular how nodes will look like) and
which interface transformations will take place.
·
During implementation the whole set of models
is mapped into a run-time environment.
·
OOHDM does not prescribe a particular strategy for
implementing a hypermedia or Web application
·
The design style facilitates the use of
object-oriented languages and architectural styles such as the
Model-View-Controller metaphor.
Dealing with physical Objects
·
OOHDM meta-model extended by adding the concept of
Physical Object.
·
A physical object is an application object that can
be explored “physically”
o it will
have a physical presence in the system
o we can
sense if the user is near it
o e.g.
the museum example we may be interested
in modeling artworks and even rooms as physical objects.
o Approach
for modeling physical objects:
§
consider that not all objects in a class (e.g.
Artwork) must be tagged as physical
§
e.g. relate artworks that exist physically with
others that are
·
not in exhibition
·
are in another geographical place
·
simply do not exist anymore.
§
Representing physical objects as sub-classes of a
particular class (Artwork) introduces a specialization criteria that might
collide with others in the intended domain (paintings, sculptures, etc)
o Chosen
to model physical objects as roles that can be assumed by conceptual objects.
§
A role type (in this case “physical”) indicates
those properties and behaviors of an object when playing that role.
§
Roles can easily be mapped to implementation settings
using for example decorators.
There is a specific role type for each of them.
·
Physical objects are characterized by an attribute position whose semantics depends on the location
sensing technology
o Must be
refined for each application
o Different
role types (e.g. Museum and Boutique) might use different ways of location
sensing and representation
§
e.g. infrared position technology may require might be implemented using just an identifier,
§
outdoor applications that use GPS or other sensing
techniques, position must be implemented using more complex
location models.
·
Physical objects implement the inFrontOfMe (user) behavior that is triggered by the
underlying software when the user is sensed to be in the object’s vicinity.
o Standard
behavior is to open the corresponding node
o Also
implement the howToReachFrom (location) which
is used by walking links to indicate how the user can find the object.
·
Separating the conceptual from the physical aspect
of an object :
o allows
decoupling of design decisions
o allows
building of different browsing strategies according to the dimension considered
(e.g. physical or digital)
Specifying Navigation aspects
·
The navigation schema shows which nodes the user
will perceive and which links he can follow.
·
Nodes are built from conceptual objects and links
are derived from relationships in the conceptual model.
·
Cornerstones of OOHDM is that a different navigation
schema can be built for different user roles.
·
Museum application - can for example build a
different navigation schema for the regular visitor or for an expert (for
example a person working in the Museum).
o Some
artworks might be even (physically) inaccessible for a visitor, while the
museum worker should be able to access them for performing his work.
·
Differences between a conventional and a physical
hypermedia regarding the navigational schema:
o the
activation of nodes
o the
semantics of link traversal
·
Conventional hypermedia - a node is opened when we
navigate a link having that node as a target.
o Preserve
this behavior for “pure” digital nodes
o A node
that stands for a physical object should only be opened when the user is facing
the object
o introduced
changes in the physical objects (role) classes and in the link class behaviors.
·
Different navigation semantics - walking links
(WLinks) - those links whose target node is the digital counterpart of a
physical object.
·
Main difference between the operational semantics of
a navigational and a walking links:
o navigational
links close the current node and opens the target node
o walking
links indicate the user’s intention to reach the corresponding physical
object
·
the link invokes the howToReachFrom behavior in the physical object
corresponding to the target node, using as a parameter the current user
location.
·
Figure - the navigational schema for the visitor
user role that corresponds to the conceptual model.
·
WLinks with a <<W>> in the style of UML
stereotypes [10]; as said before we can have “non-walking” instances of a WLink
simply by specifying it at the instance level.