Multimodal Interfaces for Technical Manuals

When maintenance workers in the Navy need more information about equipment or a task, they must go to a large shelf with many thick manuals. Each bulky manual is printed in legal size. Maintenance workers often have another tech go through the manual while they are completing a task: to placehold information, to read aloud, or to find the appropriate diagram.

The Office of Naval Research approached Carnegie Mellon for a new tool to support maintenance workers in locating information. As part of a team of 6, we went through an end-to-end process from upfront research with key stakeholders to design concepts to final production of a prototype. I led research into understanding voice user interfaces and how to incorporate such novel interaction into the prototype. Additionally, I planned and conducted usability tests with paper prototypes and experience prototypes.

The final prototype was a tablet PC with a headset that allowed maintenance workers to interact via voice commands and direct pen input.

Role
User Researcher
Voice User Interface Developer

Documentation
Project Website
Demo Video
Final Prototype