Project Idea: Using a Game Controller as a Navigation Aid
Organization: Inclusive Design Institute | Fluid Community
Repository: Gamepad Navigator
Mentor: Tony Atkins (the-t-in-rtf)
Student: Divyanshu Mahajan (dmahajan980)
Fluid is an open, collaborative community consisting of an international team of partners, individuals, and institutions focused on designing inclusive, flexible, customizable, user-centered interfaces. The community is focused on the following areas of interest:
- Building web tools that provide an easy way to build JavaScript-based user interfaces that are highly flexible and reusable.
- Creating design resources, including tools and techniques that are easy to use, learn, and modify.
- Design of inclusive user interfaces, which combine both design and technology to create sharable user interface components.
- Teach inclusive design to others at conferences and meetings.
- Nurture an open and inclusive culture to support an agile approach to design and development in an open-source project.
The Google Summer of Code 2020 has been a fantastic and cheerful learning experience for me over the past few months. I worked on the Gamepad Navigator, a Chrome extension that allows the users to navigate web sites using a game controller. Traditionally, browsers are controlled using a mouse, keyboard, or touch input. Gamepads are popular devices, and a lot of work has been done to extend their usefulness to a wide variety of people, such as one-handed controllers, which are designed for people with limited mobility or amputees. So the basic idea was to benefit from the work done to make gamepads more accessible and make browser navigation more inclusive.
The project was completely new. As this had to be written from scratch, the primary requirement was to use Infusion. Apart from that, the following necessary conditions had to be fulfilled:
- Should accept and read gamepad inputs.
- Should be able to control a web browser.
- The project should be compatible across multiple platforms and devices.
- Allowing the users to reconfigure controls as they desire.
The following goals were framed according to the requirements and the scope of the project:
- Comparing multiple potential candidate technologies that can serve as the base of the project and evaluating them.
- Adding a set of features to navigate across the browser.
- Creating a configuration panel for users to re-map controls/actions.
- Adding documentation for the gamepad controls, the new Infusion components written for the project, and instructions for users.
- Publishing the extension on the Chrome Web Store (if Chrome extension is chosen as the potential candidate).
- Adding tests for each navigation feature.
The project does contain tests for scrolling, clicking, and tab navigation. However, the remaining navigation features lack tests and will be added after the program ends.
The following set of navigation features are supposed to be a part of the project:
- Page scrolling (left, right, up, and down)
- History Navigation (backward and forward)
- Click on page elements
- History Navigation (back and next)
- Open a new tab/window
- Close current tab/window
- Tab/window switching (forward and backward)
- Zoom in/out on the active web page
- Maximize/restore the browser window
- Re-open the last closed browser tab/window
- Refresh page
The proposed navigation features were a subset of the above list. After further exploration of the possibilities with the Chrome and the Gamepad API, it was decided to add the zoom in/out, maximize/restore windows, and re-open tabs/windows features. Also, there are some open tickets for feature addition and improvements based on the feedback received from community members. Some of those are long-term goals for the project and will be worked upon after the program completes.
The details of the discussions in the weekly check-ins and the work done throughout the coding period can be found in the work log.
All the code written during the Coding Period is available as release v0.1.0-dev.
The extension will be published shortly on the Chrome Web Store. It can also be manually installed through the project's repository by following the Installation instructions.
Configuration Panel (Normal State)
Configuration Panel (Expanded State)
I created a demonstration video demonstrating how the extension works (see below).
As this project was to be written from scratch, there was no scope of code contributions to the project before the Coding Period began. But candidates were encouraged to research a few topics in preparing their proposal. Those were mentioned on the project ideas page under the project description.
In addition, I wrote a prototype of the project using Infusion to get an idea about how things will work together in place. The demo is just an abstraction of the concept, but is different from the project in many contexts. However, this can be used for demonstration purposes.
In general, some issues and polish work need to be completed along with adding new features as we progress. Some of the essential tasks out of these are as follows.
- Completing remaining tests for the navigation features.
- Adding support for "long-press" and "chording".
- Adding vibrational feedback to some actions.
- Adding a feature to view the current gamepad configuration.
- Collecting and responding to bug reports and feature requests from users.