The Augmented Workspace allows the robots to communicate with their human co-workers in the virtual space. The ``real" shared human-robot workspace is thus integrated with an Augmented Workspace where the humans and robots can communicate seamlessly by means of holograms. For example, a robot involved in an assembly task can project the objects it is manipulating into the human's point of view, and annotate them with holograms that correspond to intentions to use or pickup or already using. The human can, in turn, access or claim a particular object in virtual space and force the robot to re-plan, without there ever being any conflict of intentions in the real space. The humans in the loop can thus not only infer the robot's intent immediately from these holographic projections, but they can interact with them to communicate their own intentions directly and modify the robot's behavior online. Thus, the Augmented Workspace provides artificial agents a rich vocabulary to communicate effectively with their human co-workers in a shared mutually understood language, that can overcome the many limitations and ambiguities of communication in human-robot interactions via natural language.

The HoloLens communicates through the REST API server. The API server is implemented in python using the Flask web server framework. All external traffic to the server is handled by an Apache2 server that communicates with the python application through a WSGI middle layer. The Apache2 server ensures that the server can easily support a large number of concurrent requests. The REST service exposes both GET and a POST endpoints. The GET links provide the HoloLens application with the current state of the workspace. While the post link provides the HoloLens app control over the robot’s operation. The API server first tries to fetch the robot state from the memcached layer, and would only try a direct query from the MySQL database if the cache entry is older than a specified limit. Each query to the MySQL database also causes the corresponding database entry to be updated, to serve future future calls. The MySQL server itself is updated by a daemon that runs on Azure and keeps consuming state messages sent in the “Robot State” queue by the robot.