Introduction

The following outlines the areas of development needed to ensure that the GNOME Desktop not only remain accessible, but also continues to develop into a more compelling platform available to more people. It is admittedly an ambitious plan, but still one we think is achievable -- so long as the GNOME community and other stakeholders continue to allocate resources commensurate with the needs.

1. Scope

This road map is NOT everything we as a team hope and intend to achieve; it is merely those goals which are shared by us all. The individual modules that make up GNOME Accessibility each (should) have their own road maps which in many cases far exceed what is described here. You are encouraged to view these road maps as well:

• AT-SPI2

• Accerciser

• Caribou

• GNOME Shell Magnifier: Features in Development

• GnomeShell

• Mousetweaks

• OCRFeeder

• Orca

• Wiican

2. Definitions Used in This Document

• "Assistive Technology" (AT) is used here to mean those tools end users employ to access items which would otherwise be inaccessible to them. Examples: the Orca screen reader, the Caribou on-screen keyboard, the GNOME Voice speech recognition client, etc. Within this road map, we are NOT considering testing tools such as Accerciser to be an AT.
• "Testing tools," though arguably ATs, is used here to refer to tools whose purpose is to directly test accessibility (e.g. Accerciser) and/or use the accessibility libraries as a means through which to perform testing (e.g. LDTP).

October 2010 - April 2011

1. GNOME 3.0

Status: Mostly Completed (see notations below)

GNOME 3.0 is now scheduled for release in March 2011; 3.0.1 in April. There are a number of significant changes taking place which threaten the accessibility of the GNOME Desktop. The current areas of concern, along with the status of each area, is being tracked in detail. In summary:

• AT-SPI over D-Bus:
  • Finish the implementation
  • Finish the C-based refactor of the AT portion of at-spi and Python bindings to provide compatibility with the original pyatspi API.
  • Look at Orca regression tests with at-spi2, and fix other bugs that show up (Nagappan will likely find more bugs testing with LDTP, for instance).
  • Look at performance again and determine where most of the round-trip calls are being made now that at-spi2 hopefully caches much of the information that previously accounted for most of the requests. Make improvements if this can be done without breaking API for AT consumers.

• WebKitGtk (needed for Yelp 3):

  • Fix the bugs being tracked by WebKit metabug 25531

  • Finish the implementation of Orca's support for WebKitGtk

  • Enlist the Orca user community to perform testing
• Assistive technologies and testing tools must work with the new GNOME 3 environment, including GNOME Shell, Gtk 3, and GSettings.
• Screen magnification must continue to be available to Orca users and also work as a stand-alone solution for non-Orca users:

  • Finish specification of the common magnification framework.

    • Note: framework requires the use of GSettings to control some of a magnifier's functionality. See previous bullet about AT's working with GSettings.

  • Finish and test GNOME Mag's D-Bus implementation (needed for users who cannot run GNOME Shell).
  • Finish and test GNOME Shell's built in magnifier implementation.

  • Not done Ensure the available magnification tools (GNOME Shell Mag, GNOME Mag, eZoom) have cursor tracking and focus tracking.

  • Convert any tool which "drives" magnification (rather than work seamlessly in conjunction with it) to the new, common magnification framework.

• Users who require an on-screen keyboard must continue to have this option:
  • Finish and test Caribou
• The new Universal Access Preferences UI and the assistive technologies configurable through it must work well together.

2. Increased and Improved User and Developer Documentation

Status: Mostly Completed (see notations below)

Accessibility documentation for both end users and developers has historically been incomplete and not up-to-date. This makes GNOME accessibility less approachable to users, would-be contributors, new accessibility developers, and developers of other modules implementing accessibility support in their application or toolkit. We must:

• Review and update the documentation for assistive technology tools
• Review and update the documentation for developer testing tools

• Not done Review and update the documentation for the accessibility libraries

In addition, Yelp 3.0 and Mallard allows documentation to be tagged and organized according to context and use case. Each Accessibility application team should:

• Come up with high level documentation structure.
• Port existing documentation to new Mallard-based structure where appropriate.

3. Improved Regression Testing Tools for Applications and Toolkits

Status: Not completed

The Accessibility team spends a non-trivial amount of its time triaging and filing bugs introduced by changes in the applications and toolkits GNOME ATs provide access to. It would be much better if these regressions could be automatically detected when they are made so that the problematic changes are identified and not committed. A new Accessibility testing infrastructure, which will provide generic automated accessiblity tests as well as a core framework for commit-triggered running of tests within a test environment, is therefore being created.

• Low level test libraries
• Prototype tests
  • Initial desktop focus
  • Label Check
  • Keyboard navigation
  • Several applications (gnome-panel, gnome-calculator, gedit)
• Create sample "broken accessibility" application
• Expand tests to more applications and begin running them under gnome-shell
• Commit-triggered built of test environment
• Commit to result association and reporting
• Create more complex tests:
  • Event monitoring
  • Widget associations

4. Performance Evaluation and Improvement

Status: Not completed

Many users and developers complain frequently about performance with respect to GNOME accessibility, both the tools themselves (e.g. Orca) and the performance degradation seen in applications when accessibility support is enabled for the session -- even when no assistive technologies are being used. This latter issue is frequently cited as the cause for developers not enabling this support as well as for the community and distros being unwilling to enable this support by default.

• Conduct a full evaluation of Orca's performance problems and fix all the issues found in Orca
• Conduct a full evaluation of Gtk/Gail's performance problems and fix all the issues found therein
• In partnership with application and toolkit developers, evaluate where we are with Atk/AT-SPI to determine whether or not it is serving the community needs. If it is, conduct a full evaluation of its performance problems and fix all the issues found therein.

April 2011 - April 2012

Note: Additional planning will occur once the accessibility team has a better understanding about what manpower and financial resources will be available to perform the work described here. With that understanding, we will be able to provide a better, and more specific, estimate regarding what can be accomplished in the first six-month period and what will wait until the second six months.

1. Bug Fixing

Status: In progress, but not completed

Despite the best efforts of the teams working on GNOME 3.0, there will undoubtedly be bugs which are not caught in time to make the GNOME 3.0 release. We will not fully know what all is broken until a significant number of GNOME users have worked with GNOME 3.0 on a regular basis. In addition, there are already a non-trivial number of accessibility bugs logged in GNOME's bugzilla, many of which are not bugs in accessibility modules, many of which have yet to be triaged, and many of which have been open for years. If we want to provide a truly compelling desktop environment, we need to fix these bugs.

• Triage all of the existing accessibility bugs logged in GNOME's bugzilla to see if they are still issues
• Fix as many of the existing accessibility bugs logged in GNOME's bugzilla which have be confirmed as still present
• Fix bugs related to the new accessibility infrastructure (AT-SPI2, Atk2)
• Fix bugs related to migration to PyGObject
• Fix bugs related to accessing GNOME Shell
• Fix bugs related to accessing Gtk 3 widgets

2. Implement "Testing Days" for GNOME Accessibility

Status: Not completed

Accessibility testing in GNOME has historically been a "catch as catch can" effort, done primarily by members of the team as part of the work they are doing on their individual modules. As a result, we currently have gaps in our testing. It would be advantageous to organize and formalize our testing efforts and to involve the community.

• Look to what the Fedora community has done with their "testing days" (see, for example, their GNOME 3 Alpha testing day).

• Identify at least one distro which would be suitable for this task, with the aim being to use the chosen disto(s) for testing every cycle.
• Determine if we should collaborate with an existing testing effort or create our own.
• Identify and write the list of tests to be conducted along with instructions.
• Create and provide a tool for gathering the testing data.
• Schedule the tests.
• Engage the user community (with sufficient lead time) so that they will participate.

3. Improved and Increased Access to Applications, Toolkits, and Environments

Status: Not completed

The Accessibility team would like to provide more compelling access to currently-supported modules and implement support for modules which are currently not supported due to problems with their accessibility implementation. This requires collaboration between our team and the teams whose applications and toolkits we would like to provide access to. It also requires that the teams with whom we wish to collaborate have the time (or other resources) to work with us.

• Continue to improve the accessibility support of Evince/Poppler
  • PDF text should reflect the structure of the document (headings, paragraphs, etc.), rather than be a single text object
  • Formatted text should expose its text attributes to ATs
  • ATs should take advantage of the newly-implemented support to provide better access to users
• Continue to improve the accessibility support of WebKitGtk/Epiphany
  • Embedded Object support should be implemented
  • ARIA and HTML5 accessibility should be implemented
  • ATs should take advantage of the newly-implemented support to provide better access to users

• Port AT-SPI to gdbus if performance can be improved sufficiently (see bug 634471).

• Implement support for other applications, toolkits, and environments including but not limited to:
  • "Lightweight" Gtk-based desktop environments
  • Qt applications
  • Abiword
  • Gwibber

4. Augmented and Consistently-Implemented Atk

Status: Not completed

Applications and toolkits do not all implement Atk consistently. This has a negative impact on the assistive technologies' ability to provide a consistent cross-application user experience. Even in applications and toolkits in which the Atk implementation is complete, the information obtained from a single event and/or object is not always sufficient for an AT client to proceed immediately; instead it is often necessary to perform further queries and make decisions based on heuristics rather than concrete data. This has a negative impact on both performance and reliability.

• Examine IA2 for items which would improve the user experience if added to Atk.
  • Like STATE_MIXED for mixed state checkboxes

• Enhance Atk events to make them more informative (ie: AtkText::text-changed)

• Make AtkObject Attributes the standard way to expose additional, application- and toolkit-unique information (i.e. similar to what Gecko already does).

• Create more detailed documentation, including "best practices."
• Make any required, corresponding changes in AT-SPI.
• Accessibility issues, including the proper implementation of Atk, should be “GNOME Goals.”
• GObject introspection is the future and present, but:
  • Current Atk introspection annotations are incomplete
  • There are some method overlapping on the automatic generatic bindings

• Analyze and redesign AtkUtil (some problems detected in this mail

• Review the factory system and expecifically, AtkGObjectAccessible
  • Recent toolkit (St) developers asks to not use them
  • What happens when you expose non-gtyped objects (ie: Java, C++ hierarchies)
  • Still required? Should be move Gail to not use this system?
• Window related methods
  • at-spi expect window related methods (activate, deactivate, etc)

  • But the equivalent methods are not defined on ATK, but each implementation should emit them regardless that, in a hacky way (check GailUtil and CallyUtil)

  • Check if this could be solved defining a new AtkWindow interface

5. Alternative Input Devices: Needs Assessment/Study

Status: Not completed

GNOME has very few options for users who require alternative input device(s), including users with physical disabilities and users with learning disabilities. The Accessibility team lacks sufficient domain expertise in these areas to create a definitive list of user requirements from which our existing tools could be improved and "missing" tools created. Similarly, because we lack compelling solutions in these areas, we do not have an extensive user population providing us with feedback and requests. In order to ensure that the GNOME Desktop is an environment which is truly universally accessible, we need to provide solutions based on a detailed and accurate understanding of user needs in this area.

• Identify individuals and institutions with the required domain expertise in accessibility for users with physical disabilities (Must be done prior to April 2011)
• Identify individuals and institutions with the required domain expertise in accessibility for users with learning disabilities (Must be done prior to April 2011)
• Seek funding (if necessary) to support the work needed to conduct the needs assessments/studies (Must be done prior to April 2011)
• Collaborate with the individuals and institutions identified to conduct a thorough needs assessment/study in each area
• Create design documents based on the results of the assessments/studies
• Create prototype solutions based on the design documents
• Follow up with those having the needed domain expertise to evaluate the prototypes

6. Optical Character Recognition (OCRFeeder) as an Assistive Technology

Status: Not completed

Optical Character Recognition can be an extremely valuable tool for accessing print materials, both for users who are blind and for users who have a print-related learning disability. OCRFeeder is the module best-suited for addressing this need in the GNOME Desktop, though it was never designed with this purpose in mind. Through the Guadalinfo Accessible projects from Andalucía, Spain, a number of improvements were made to enhance OCRFeeder's accessibility. A number of additional improvements would be helpful to make OCRFeeder a truly compelling tool for users who are blind. In addition, there are likely changes and new features which would make OCRFeeder a more compelling tool for users with learning disabilities.

• Solicit the input of users who are blind for the purpose of determining what additional enhancements would be of value
• Solicit the input of users who have a print-related learning disability for the purpose of determining what additonal enhancements would be of value
• Add additional export options:
  • Plain text
  • A single-page option for HTML
• Add an option to exclude the graphics contents (so only text is considered when genertating a document)
• Add an option to use all OCR engines and automatically choose the best results so that users who are blind no longer have to "tweak" recognition settings without being able to see the document
• Make further improvements to the UI:
  • Use only one status bar instead of three to improve accessibility with screen readers
  • Review the UI text to check if beginner users realize easily what the different actions do
  • Review the focus events beign issued and eliminate the extraneous ones
• Evaluate the value of an assistant/wizard to simplify the recognition process for novice users and implement this tool if it is indeed needed

June 2011 - June 2012

1. Meego

Status: Not completed but no longer relevant

Meego is the joining of Maemo and Moblin, targeted for mobile devices, netbooks, tablet computers etc. Both extensively use GNOME technologies. Although on the handset flavour it is intended to use Qt by default, recently the Foundation opened a public bid in order to ensure that gtk works on Meego Handset (specific flavour for smartphones).

This project has the potential to be a real competitor to Apple IOs (but more open) and to Android (but with more GNOME technologies), with two really big companies pushing for it, Intel and Nokia. In addition, because Meego will provide an infrastructure shared by a variety of devices, having Meego fully accessible would mean accessible smartphones, accessible netbooks, accessible set-top boxes, etc.

Therefore, the Accessibility team is interested in seeing if the current GNOME accessibility libraries can be ported to this platform. This project would encompass:

• Study the GNOME technologies used on the plattform
• Study the accessibility support of these GNOME technologies
• Study the different use-cases for the different device-flavours of Meego
• Create prototypes, based on the results from the studies, of AT solutions for Meego

Please note that the projected dates for this work are especially tentative as they depend upon the porting of Gtk to Meego.

2. Create a Set of "Presentation" Libraries

Status: Not completed

Currently Orca is, for all intents and purposes, the only AT which presents on-screen information to the user in alternative formats. It is highly desirable for other applications to be able to perform similar functions independent of Orca. In addition, there are enhancements found in the ATs of other platforms which users find helpful and which GNOME lacks. The Accessibility team would therefore like to create a set of libraries which all applications (ATs and non-ATs alike) could use including:

• text-to-speech library
• text-to-braille library (liblouis should work, but needs to be made an external dependency)
• making the mouse pointer more easily visible
• making the caret more easily visible
• highlighting the current line of text
• providing filters for users who are color blind (libcolorblind exists and is used by Orca and gnome-mag but needs work, and also needs to be made an external dependency)
• others

October 2011 - October 2012

Note: Additional planning will occur once the accessibility team has a better understanding about what manpower and financial resources will be available to perform the work described here. With that understanding, we will be able to provide a better, and more specific, estimate regarding what can be accomplished in the first six-month period and what will wait until the second six months.

1. Speech Recognition as an Assistive Technology

Status: Not completed

There are a couple of solutions for speech recognition available to GNOME users: GNOME Voice Control and Vedics. Neither of these solutions seems to be under active development. In addition, each solution depends upon CSPI which currently has not been implemented as part of the D-Bus-based AT-SPI. If that issue is resolved, testing and additional development will be required to ensure it works with GNOME 3.0. If these issues are not resolved, and/or if the conclusion is reached that neither of these tools provide the needed support, a new solution will need to be created.

• Perform a functional assessment of each of these two tools to determine if they would meet our needs if made to work in GNOME 3.

Based on the findings from that assessment, either:

• Evaluate the feasibility of converting the preferred tool to python if CSPI continues to be deprecated
• Implement support for GNOME Shell within that tool

Or:

• Take what has been learned from examining each tool, along with the proposed needs assessment, and begin developing an alternative solution

2. Alternative Input Devices: Implementation

Status: Not completed

By April 2012 (and hopefully much sooner) we anticipate having completed a needs assessment/study to determine what alternative input devices, along with the corresponding software tools, will best address the needs of users with physical disabilities and users with learning disabilities. We also anticipate having had created, and received feedback on, prototype solutions. Armed with this information, we will formally implement the software-based solutions.

3. External Usability Reviews

Status: Not completed

The Accessibility team lacks expertise in the area of Human-computer interaction (HCI). The team would, therefore, like to engage the HCI community in the hopes of finding experts to conduct several (i.e. two or three) independent studies of the usability of our AT modules.

Given the number of planned changes in both the ATs and in the GNOME Desktop, some members of the Accessibility team feel it would be best to give the environment a chance to stabilize before these reviews are conducted. Alternatively, reviews can be conducted sooner as desired and as resources permit, but then follow-up reviews should still be conducted in FY2012 so that we can separate known, temporary "rough edges" from things we thought we finished but could/should have done differently.

4. Begin Implementation of Changes Identified from Usability Reviews

Status: Not completed

It is the hope of the Accessibility team to have at least some feedback from the aforementioned external usability reviews prior to October 2012. Based on this feedback, the appropriate module teams should work on implementing the recommended changes.