Publication | ACM Symposium on User Interface Software & Technology 2012
Waken
Reverse Engineering Usage Information and Interface Structure from Software Videos
Abstract
Waken: Reverse Engineering Usage Information and Interface Structure from Software Videos
Nikola Banovic, Tovi Grossman, Justin Matejka, George Fitzmaurice
ACM Symposium on User Interface Software & Technology 2012
We explore the possibilities and opportunities related to reverse engineering usage information from screen-captured application video tutorials. We develop an application-independent system that recognizes UI components and activities, such as cursor movements and icon clicks, from an input set of video tutorials, We then present Waken, an enhanced video player, which showcases some of the design opportunities the are introduced by having this additional meta-data. In particular, users can directly interact with UI components in the video, such as icons and menus, to display associated information, or navigate to relevant moments in other videos. Initial results suggest that the system can successfully reconstruct many aspects of a UI without any prior application-dependent knowledge.
Download publicationAssociated Researchers
Nikola Banovic
Dept. of Computer Science University of Toronto, Ontario, Canada
Related Resources
2024
Interlayer Temperature Optimisation: An Approach to Maximising Strength in the Large-Scale Additive Manufacture of Fibre-Reinforced PETGInvestigating anisotropy in material extrusion additive manufacturing:…
2012
A Classification of Opening Posts in Commercial Software Help ForumsThe opening posts in software help forums reflect the users’…
2019
A First-Order Logic Formalization of the Industrial Ontologies Foundry Signature Using Basic Formal OntologyBasic Formal Ontology (BFO) is a top-level ontology used in hundreds…
2022
Reconstructing editable prismatic CAD from rounded voxel modelsReverse Engineering a CAD shape from other representations is an…
Get in touch
Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.
Contact us