Brygg Ullmer
Hiroshi Ishii
Robert J. K. Jacob
Skip Abstract Section
Abstract
We identify and present a major interaction approach for tangible user interfaces based upon systems of tokens and constraints. In these interfaces, tokens are discrete physical objects which represent digital information. Constraints are confining regions that are mapped to digital operations. These are frequently embodied as structures that mechanically channel how tokens can be manipulated, often limiting their movement to a single degree of freedom. Placing and manipulating tokens within systems of constraints can be used to invoke and control a variety of computational interpretations.We discuss the properties of the token+constraint approach; consider strengths that distinguish them from other interface approaches; and illustrate the concept with eleven past and recent supporting systems. We present some of the conceptual background supporting these interfaces, and consider them in terms of Bellotti et al.'s [2002] five questions for sensing-based interaction. We believe this discussion supports token+constraint systems as a powerful and promising approach for sensing-based interaction.
- Ahlberg, C. and Shneiderman, B. 1994. Visual information seeking: Tight coupling of dynamic query filters with starfield displays. In Proceedings of Computer-Human Interaction 1994. 313--317.]] Google Scholar
- Aish, R. and Noakes, P. 1984. Architecture without numbers---CAAD based on a 3D modelling system. In Computer-Aided Design 16, 6 (Nov.) 321--328.]]Google Scholar
- Anagnostou, G., Dewey, D., and Patera, A. 1989. Geometry-defining processors for engineering design and analysis. In The Visual Computer. Chapter 5, 304--315.]]Google Scholar
- Anderson, D., Frankel, J. Marks, J., Agarwala, A., Beardsley, P., Hodgins, J., Leigh, D., Ryall, K., Solliva, E., and Yedidia, J. 2000. Tangible interaction+graphical interpretation: A new approach to 3D modelling. In Computer Graphics Proceedings (SIGGRAPH'00), 393--402.]] Google Scholar
- Ballagas, R., Ringel, M., Stone, M., and Borchers, J. 2003. iStuff: A physical user interface toolkit for ubiquitous computing environments. In Proceedings of Computer-Human Interaction 2003. 537--544.]] Google Scholar
- Bell, R. 1979. Board and Table Games from Many Civilizations. Dover Publications, New York, NY.]]Google Scholar
- Bellotti, V., Back, M., Edwards, Grinter, R., Henderson, A., and Lopes, C. 2002. Making sense of sensing systems: Five questions for designers and researchers. In Proceedings of Computer-Human Interaction 2002. 415--422.]] Google Scholar
- Calvillo-Gámez, E., Leland, N., Shaer, O., and Jacob, R. 2003. The TAC paradigm: Unified conceptual framework to represent Tangible User Interfaces. In Proceedings of Latin American Conference on Human-Computer Interaction. 9--15.]] Google Scholar
- Cohen, J., Withgott, M., and Piernot, P. 1999. Logjam: A Tangible Multi-Person Interface for Video Logging. In Proceedings of Computer-Human Interaction 1999. 128--135.]] Google Scholar
- Cutkosky, M. and Howe, R. 1990. Human grasp choice and robotic grasp analysis. In Dextrous Robot Hands. 5--31. Springer Verlag.]] Google Scholar
- Durham, J. 2002a. Abrasives, trust, and how the Abacus got its name. http://bart.cba.nau. edu/∼durham-j/newsite/id153.htm {Feb}.]]Google Scholar
- Durham, J. 2002b. Personal communications. Feb. 10, 2002.]]Google Scholar
- Fernandes, L. 2001. The abacus: The art of calculating with beads. http://www.ee.ryerson. ca:8080/∼elf/abacus/ {Feb. 2002}.]]Google Scholar
- Fitzmaurice, G., Ishii, H., and Buxton, W. 1995. Bricks: Laying the foundations for graspable user interfaces. In Proceedings of Computer-Human Interaction 1995. 442--449.]] Google Scholar
- Fjeld, M., Bichsel, M., and Rauterberg, M. 1998. BUILD-IT: An intuitive design tool based on direct object manipulation. In Gesture and Sign Language in Human-Computer Interaction, v. 1371, Wachsmut and Fröhlich, Eds. Springer-Verlag, Berlin, Germany. 297--308.]] Google Scholar
- Frazer, J. 1995. An Evolutionary Architecture. Architectural Association. London, UK.]]Google Scholar
- Frazer J. H., Frazer J. M., and Frazer P. A. 1989. Intelligent physical three-dimensional modelling systems. Computer Graphics' 80 Conference, Conference Proceedings, Online Publications. 359--370.]]Google Scholar
- Gellersen, H., Schmidt, A., and Beigl, M. 2002. Multi-sensor context-awareness in mobile devices and smart artifacts. In Mobile Netw. Applica. 1, 5, 341--351.]] Google Scholar
- Gibson, J. 1979. The Ecological Approach to Visual Perception. Erlbaum Associates, New York, NY.]]Google Scholar
- Guiard, Y. 1987. Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. J. Motor Behav. 19, 4, 486--517.]]Google Scholar
- Hinckley, K., Pausch, R., Proffitt, D., and Kassell, N. 1998. Two-handed virtual manipulation. ACM Trans. Comput.-Hum. Inter. 260--302.]] Google Scholar
- Holmquist, L., RedströM, J., and Ljungstrand, P. 1999. Token-based access to digital information. In Proceedings of Handheld and Ubiquitous Computing (HUC 99), 234--245.]] Google Scholar
- Hornecker, E. 2002. Understanding the benefits of graspable interfaces for cooperative use. In Proceedings of Cooperative Systems Design 2002. 71--87.]]Google Scholar
- Ifrah, G. 2001. The Universal History of Computing: From the Abacus to the Quantum Computer. John Wiley & Sons, New York, NY.]] Google Scholar
- Ishii, H., Underkoffler, J., Chak, D., Piper, B., Ben-JOSEPH, E., Yeung, L., and Kanji, Z. 2002. Augmented urban planning workbench: Overlaying drawings, physical models and digital simulation. In Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR '02). 203--214.]] Google Scholar
- Ishii, H. and Ullmer, B. 1997. Tangible bits: Towards seamless interfaces between people, bits, and atoms. In Proceedings of Computer-Human Interaction 1997. 234--241.]] Google Scholar
- Jacob, R., Ishii, H., Pangaro, G., and Patten, J. 2002. A tangible interface for organizing information using a grid. In Proceedings of Computer-Human Interaction 2002. 339--346.]] Google Scholar
- Jones, W. and Dumais, S. 1986. The spatial metaphor for user interfaces: Experimental tests of reference by location versus name. ACM Trans. Office Inf. Syst. 4, 1 (Jan.) 42--63.]] Google Scholar
- Kirsh, D. 1995. The intelligent use of space. Artif. Intel.]] Google Scholar
- Klemmer, S. 2003. Papier-Mâhé: Toolkit support for tangible interaction. In Proceedings of User Interface Software and Technology 1995.]]Google Scholar
- Larkin, J. and Simon, H. 1987. Why a diagram is (sometimes) worth ten thousand words. Cognit. Sci. 11, 65--99.]]Google Scholar
- Lütjens, J. 2002. Abacus online museum. http://www.joernluetjens.de/sammlungen/abakus/abakus-en.htm {(Feb.) 2002}.]]Google Scholar
- Maclean, K., Snibbe, S., and Levin, G. 2000. Tagged handles: Merging discrete and continuous manual control. In Proceedings of Computer-Human Interaction 2000. 225--232.]] Google Scholar
- Masters, J. 2002. The royal game of Ur and Tau. http://www.tradgames.org.uk/games/Royal-Game-Ur.htm {(Aug.) 2002}.]]Google Scholar
- Mazalek, A. and Jehan, T. 2000. Interacting with music in a social setting. In Extended Abstracts of Computer-Human Interaction 2000. 255--256.]] Google Scholar
- Mcnerney, T. 2000. Tangible programming bricks: An approach to making programming accessible to everyone. MS Thesis, MIT Media Laboratory.]]Google Scholar
- Nelson, L., Ichimura, S., Pederson, E., and Adams, L. 1999. Palette: A paper interface for giving presentations. In Proceedings of Computer-Human Interaction 1999. 354--361.]] Google Scholar
- Neurosmith. 1999. MusicBlocks product. http://www.neurosmith.com/.]]Google Scholar
- Norman, D. 1999. Affordances, conventions, and design. In Interact. 6, 3, 38--43.]] Google Scholar
- Norman, D. 1993. Things that Make Us Smart. Addison-Wesley, Reading, MA.]]Google Scholar
- Oxford English Dictionary (OED). 1989. OED Online Oxford University Press.]]Google Scholar
- Pangaro, G., Maynes-Aminzade, D., and Ishii, H. 2002. The actuated workbench: Computer-controlled actuation in tabletop tangible interfaces. In Proceedings of User Interface Software and Technology 2002. 181--190.]] Google Scholar
- Patten, J., Recht, B., and Ishii, H. 2002. AudioPad: A tag-based interface for musical performance. In Proceedings of the International Conference on New Interfaces For Musical Expression.]] Google Scholar
- Patten, J., Ishii, H., Hines, J., and Pangaro, G. 2001. Sensetable: A wireless object tracking platform for tangible user interfaces. In Proceedings of Computer-Human Interaction 2001. 253--260.]] Google Scholar
- Perlman, R. 1976. Using computer technology to provide a creative learning environment for preschool children. MIT Logo Memo #24.]]Google Scholar
- Petre, M. 1995. Why looking isn't always seeing: Readership skills and graphical programming. Comm. ACM, 38 (June), 33--44.]] Google Scholar
- Polynor, R. 1995. The hand that rocks the cradle. I.D. (May/June), 60--65.]]Google Scholar
- Preece, J., Rogers, Y., and Sharp, H. 2002. Interaction Design. John Wiley and Sons. New York, NY.]] Google Scholar
- Redström, J. 2001. Designing everyday computational things. Ph.D. thesis, Göteberg University.]]Google Scholar
- Rekimoto, J., Ullmer, B., and Oba, H. 2001. DataTiles: A modular platform for mixed physical and graphical interactions. In Proceedings of Computer-Human Interaction 2001. 269--276.]] Google Scholar
- Retz-Schmidt, G. 1988. Various views on spatial prepositions. AI Magazine, 9, 2. 95--105.]] Google Scholar
- Scaife, M. and Rogers, Y. 1996. External cognition: How do graphical representations work? Int. J. Hum.-Comput. Stud. 45, 2, 185--213.]] Google Scholar
- Schäfer, K., Brauer, V., and Bruns, W. 1997. A new approach to human-computer interaction---synchronous modelling in real and virtual spaces. In Proceedings of Designing Interactive Systems 1997. 335--344.]] Google Scholar
- Schieβl, S. 2002. Digital cubes. http://www.aec.at/festival2002/texte/schieβl_e.asp.]]Google Scholar
- Schmandt-Besserat, D. 1997. How Writing Came About. University of Texas Press, Austin, TX.]]Google Scholar
- Shneiderman, B. 1983. Direct manipulation: A step beyond programming languages. IEEE Comput. 16, 8, 57--69.]]Google Scholar
- Singer, A., Hindus, D., Stifelman, L., and White, S. 1999. Tangible progress: Less is more in somewire audio spaces. In Proceedings of Computer-Human Interaction 1999. 104--111.]] Google Scholar
- Smith, D. 1975. Pygmalion: A creative programming environment. Ph.D. thesis, Stanford University.]] Google Scholar
- Suzuki, H. and Kato, H. 1993. AlgoBlock: A tangible programming language, a tool for collaborative learning. In Proceedings of 4th European Logo Conference. 297--303.]]Google Scholar
- Ten Hagen, P. 1981. Interaction and syntax. Int. J. Man-Mach. Stud. 15.]]Google Scholar
- Tomoe Soroban Co., Ltd. 2002. Soroban museum: Roman Soroban. http://www.soroban. com/museum/∼5s_eng.html {Feb. 2002}.]]Google Scholar
- Ullmer, B., Ishii, H., and Jacob, R. 2003. Tangible query interfaces: Physically constrained tokens for manipulating database queries. To appear in Proceedings of International Conference on Computer-Human Interaction 2003.]]Google Scholar
- Ullmer, B. 2002. Tangible interfaces for manipulating aggregates of digital information. Ph.D. dissertation, MIT Media Laboratory.]] Google Scholar
- Ullmer, B., and Ishii, H. 2001. Emerging Frameworks for Tangible User Interfaces. In HCI in the New Millenium, John M. Carroll, Ed. 579--601.]]Google Scholar
- Ullmer, B. and Ishii, H. 1997. The metaDESK: Models and prototypes for tangible user interfaces. In Proceedings of User Interface Software and Technology 1997. 223--232.]] Google Scholar
- Ullmer, B., Ishii, H., and Glas, D. 1998. mediaBlocks: Physical containers, transports, and controls for online media. In Computer Graphics Proceedings (SIGGRAPH'98). 379--386.]] Google Scholar
- Underkoffler, J., Ullmer, B., and Ishii, H. 1999. Emancipated pixels: Real-world graphics in the luminous room. In Computer Graphics Proceedings (SIGGRAPH'99). 385--392.]] Google Scholar
- Want, R. and Russell, D. 2000. Ubiquitous electronic tagging. In IEEE Distrib. Syst. Online 1, 2 (Sept.).]] Google Scholar
- Yarin, P. and Ishii, H. 1999. TouchCounters: Designing interactive electronic labels for physical containers. In Proceedings of Computer-Human Interaction 1999. 362--368.]] Google Scholar
- Zhang, J. 1997. The nature of external representations in problem solving. Cogn. Sci. 21, 2, 179--217.]]Google Scholar
- Zhang, J. and Norman, D. 1994. Representations in distributed cognitive tasks. Cogn. Sci. 18, 87--122.]]Google Scholar
Index Terms
- Token+constraint systems for tangible interaction with digital information
Recommendations
Peripheral tangible interaction by analytic design
TEI '09: Proceedings of the 3rd International Conference on Tangible and Embedded InteractionTangible User Interfaces (TUIs) are commonly accepted as those in which the configuration of physical objects embodies digital system state, providing "graspable" digital media that can be manipulated in the focus of users' attention. In this paper we ...
Alternative Tools for Tangible Interaction: A Usability Evaluation
ISMAR '02: Proceedings of the 1st International Symposium on Mixed and Augmented RealityIn this work we compare an in-house designed Tangible User Interface (TUI) with three alternative single-user tools through an empirical investigation. These three alternative tools are a 3D physical, a 2D cardboard, and a mathematical tool. We expected ...
Reconfiguring and Fabricating Special-Purpose Tangible Controls
UIST '15 Adjunct: Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software & TechnologyUnlike regular interfaces on touch screens or desktop computers, tangible user interfaces allow for more physically rich interactions that better uses the capacity of our motor system. On the flipside, the physicality of tangibles comes with rigidity. ...
Comments