Towards a tangible web: using physical objects to access and manipulate the Internet of Things

Mankind has progressed to the point of manipulating objects at almost unlimited distances. Today this is possible because the communication abilities of the internet have been combined with the actuating and sensing functionality afforded by mechatronics. Recently, scientists and engineers have taken this remarkable achievement one step further by extending geographically independent remote manipulation to scenarios that have the potential to affect every living person, both now and in the future. This additional step has resulted in the phenomenon commonly referred to as the Internet of Things (IoT). In order to realise the full potential of the IoT, individuals need a mechanism to access and manipulate it. A potential mechanism for achieving this is the World Wide Web (web). However, accessing and manipulating web information using current mechanisms remains mostly a discriminatory activity due to the prerequisites the potential user has to meet. One prerequisite is good fine motor skills for manipulating web data by means of a traditional computer keyboard and mouse. This requirement discriminates against persons with muscular dystrophy. Many research and commercial projects have addressed this problem by devising input mechanisms as alternatives to the traditional computer keyboard and mouse. This paper presents an alternative input mechanism for accessing the IoT via the web. The presented input mechanism consists of four components: low-cost two-dimensional input surface with tangible objects that represent the web data to be manipulated, image processing software, command interpretation software, and actuation software that serves as an interface between the input surface and a web browser.  The design of the input mechanism can accommodate customised tokens crafted to suit the individual who will be using it. This mechanism differs from others in that the tokens can be made by an unskilled person using modelling clay, paper mache, or most other crafting material. By removing the need for professional intervention when the input mechanism has to be adjusted for an individual, the ownership cost is significantly reduced. It is now possible for a family member, or even the user herself, to customise the tokens. The paper describes in detail the four components that comprise the web-based IoT interface mechanism, and photographs of its implementation. Diagrams show how the data flow (consisting of visual images) from the input surface, to the image processing software where essential parameters are extracted, to the actuation software from where commands are sent to the web browser. Our current design has a number of limitations which we hope to eliminate in future iterations. In the paper we describe these limitations in greater detail and we offer recommendations on how they may be overcome.