Ease of Access for GDM

This page tries to draft how pointer only and switch accessibility could work in a user friendly way during GDM. It also proposes a strategy for the automatic activation of accessibility tools in the GNOME session depending on the accessibility tools activated during GDM. See Jon McCann's blog posting and bug #526070 for more information.

GDM already provides a way to start the accessibility tools by keyboard shortcuts or mouse gestures. There is no indication about them on the GDM login screen; for them to be useful, users have to know about them beforehand. This page proposes a way of doing things so that users get informed and are able to activate the (or some) accessibility tools by using the gui of GDM. It is not a replacement for the keyboard shortcuts and mouse gestures; it is meant as an additional way to activate the accessibility tools.

(Under accessibility tools I understand things like onscreen keyboards, slow keys,... and not atspi-registryd, which is active by default.)

How to open the Accessibility Preferences of GDM

Let's suppose that the computer shows the default login screen that also displays an "Open Accessibility Preferences" button/item. As by default, no accessibility tools are running, the first question is: How does a user that needs accessibility tools get to the Accessibility Preferences dialog, where he can enable the accessibility tools that he needs? It should be possible in several ways; and since there are no accessibility tools running at this stage, it is GDM itself that has to offer these possibilities. Here is a partial list to be completed:

1. The "Open Accessibility Preferences" dialog opens automatically if the login screen detects a switch event. (This will probably not disturb a regular user, since he will not be using switches.)
2. The "Open Accessibility Preferences" opens automatically if the user places the pointer on the corresponding button/item; this will be useful for pointer only users. This corresponding item could for example be a button showing the disability-icon and a keyboard-icon; this way, disabled users and tabletpc users are addressed. The item would flash a bit when the pointer is over it and then the Accessibility Preferences dialog pops up. But the HIG team will probably be the best to decide what item it should be and how it should look like; what is important is that it can be operated by the pointer without using clicks and that the users gets clues about what it is for by simply looking at it. (The automatic behavior of the item might surprise a regular user the first time, but he will not be stuck in some cryptic input state, as he can simply close the Accessibility Preferences dialog. Moreover, afterwards he can avoid the automatic popup of the Accessibility Preferences dialog by simply not pointing with the pointer (usually a mouse for a regular user) on the corresponding item. )
3. The "Open Accessibility Preferences" opens by using the usual keyboard navigation and activation
4. Please fill in other methods

How to enable the various accessibility tools

The Accessibility Preferences dialog is now open; but the accessibility tools are not running yet, so GDM still has to do the job. The question is: How will the users with the alternative input devices enable the features that they need:

1. Pointer only users able to click: they can simply click on (or activate the checkbox of, or...) the accessibility tools that they need. These users will typically start an onscreen keyboard. I guess gok in osk mode until other more appropriate osks are available in GNOME.

2. Pointer only users not able to click: the Accessibility Preferences dialog should have an item named "Use software clicks" which will start mousetweaks in dwell mode with click-type-window. (On this page of the Cummunity Help of Ubuntu you can see the click type window, where the user indicates to mousetweaks what type of click it has to perform next.) The "Use software clicks" item might react similarly to the item to open the Accessibility Preferences dialog: when the pointer is on the "Use software clicks" item, the "Use software click" item flashes a bit before its status gets toggled. (An alternative is to toggle it immediately as soon as the pointer enters the item, or even some other way; the important is that the user is able to activate it by using the pointer without having to click.) Here is a sample command to start mousetweaks in dwell mode: mousetweaks --enable-dwell --dwell-mode window --show-ctw --ctw-x 50 --ctw-y 20. With software clicks enabled, these users become pointer only users able to click as in the category "1." here above.

3. Switch users: I guess they need an item (checkbox) to start gok in switch mode
4. Keyboard only users: I guess the usual keyboard navigation and activation methods
5. Other types of accessibility users? Please fill in...

Automatic activation of accessibility tools in GDM

• For single-user system, a "Remember the GDM AT settings for the next login" option in the Accessibility Preferences dialog would be convenient. In fact, it would save him from having to re-enable the accessibility tools that he needs each time he gets to the GDM login screen.
• On multi-user systems, an enabled accessibility tool might confuse a normal user inducing him to think that the system could even be broken. Consequently, there should not be a "Remember the GDM AT settings for the next login" option available; this way, it is sure that it cannot be accidently activated.

What about a boolean configuration option for GDM to make the "Remember the GDM AT settings for the next login" option appear in the Accessibility Preferences dialog? This option would have to be set to 'false' by default.

Passing of accessibility tools between the GDM session and the Desktop session

1. From GDM session to Desktop session

Imagine a pointer only user that is not able to click. GDM gives him the means to activate dwelling and an onscreen keyboard; he is then able to work with the computer during GDM. He arrives at his GNOME session where no accessibility tools are enabled and, since he is not able to click, he cannot work in the desktop session. (unless there is also a way for him to activate the accessibility tools in the GNOME session)

A "Save these settings also in your GNOME session" option in the Accessibility Preferences dialog would:

• if enabled: provide an easy way to setup the GNOME session to automatically start the same accessibility tools as in the GDM session.
• if disabled: startup the GNOME session the way it was set to start in the previous GNOME session. (for example useful if the user has set up his GNOME session to use accessibility tools not available in GDM)

The "Save these settings also in your GNOME session" does not remain active across GDM sessions. Moreover, this option cannot be a button, because GDM does not know for what user to save the settings until the user has authenticated himself.

2. From Desktop session to GDM session

• For multi-user system: the GDM session has to start with no accessibility tools enabled; so there is nothing to do.
• For single-user system: the job is done by the "Remember the GDM AT settings for the next login" option described in the section above; so there is nothing to do here either.

Conclusion: items required in the Accessibility Preferences dialog of GDM

• On the GDM login screen, provide the accessible ways to open the Accessibility Preferences dialog; you can see an unfinished version of the Accessibility Preferences dialog on this page.

• Add a "Remember the GDM AT settings for the next login" option to the Accessibility Preferences dialog; the option is only visible on single user systems and its visibility is controlled by a boolean configuration of GDM; default status of the boolean is off (not visible); default status of the option is off; the status of the option is kept across GDM session
• Add "Save these settings also in your GNOME session" option to the Accessibility Preferences dialog; default status is deactivated; if activated, the active status is not kept across GDM session (in other words, it will be inactive at each GDM session start)
• For pointer only users: either 1 and 2 or 3 and 4; default state of all items is inactive
  1. Add a "Use software clicks (dwell clicking)" item: This item would start mousetweaks in dwelling mode with click-type-window. Of course, it has to be possible to get to this item and activate it by only using pointer movements (without clicking).
  2. Rename the "Type without the keyboard (onscreen keyboard)" item to "Use a software keyboard (onscreen keyboard)": In fact the keyboard might not only be used for typing, but for example also to do multiple selections by activating the control modifier of the osk and clicking on several icons. (I don't know for gok, but onboard emulates modifier+click combinations.)
  3. Add a "Use only pointer movements with button clicks" item: starts onboard, or gok in direct selection mode
  4. Add a "Use only pointer movements without button clicks" item: starts onboard and mousetweaks in dwell mode with click-type-window, or gok in dwell selection mode. Remark: gok in direct selection mode + mousetweaks in dwell mode does not work because gok does not see the software clicks performed by mousetweaks.
• For switch users:
• For keyboard only users:
• ...

Login Screen Interaction Model Specification

WDW: work in progress. This draws from the work of Jon McCann and others and is meant to read more like a specification.

Goal: Provide a mechanism for the user to enable accessibility features for the login screen. As such, full blown customization of features may not be as important as it is on the regular desktop. The following represents the "out of the box" behavior for the login screen.

• Accessibility Icon. As shown in Jon McCann's prototype, there will be an Accessibility Icon in the lower left hand corner of the desktop display, with the icon guaranteed to be covering the left-most and bottom-most pixel of the display.

  • As with getting to the top and bottom panels of a session, the user can navigate to and activate this icon from the keyboard using Ctrl+Alt+Tab.

  • If the user clicks with the mouse on the icon, it will launch the Accessibility Dialog.

  • For users that can move the pointer but are not able to click: if the user moves the pointer over the Accessibility Icon and dwells on it for x.y seconds, the Accessibility Dialog will open. The image of the icon should somehow indicate that it reacts on dwelling and give feedback that something is going on as soon as the pointer is over it.

  • What about switch only users?

• Accessibility Dialog. As shown in Jon McCann's prototype, the Accessibility Dialog provides the user with a quick way to launch assistive technologies. The exact items in the dialog are check boxes for each of the following. Note that these are check boxes and not radio buttons, meaning more than one item may be enabled at a time. Potential conflict when enabling more than one thing is noted below along with solutions.

  • Use on screen keyboard. Launches GOK in login and direct selection mode, meaning that it shows only a keyboard and that mouse button 1 clicks are used to select items.

    • Command: /usr/bin/gok --login --access-method=directselection

    • UI Gesture: click with the mouse on the "Use on screen keyboard" item.

    • Pointer Gesture: TBRL I 10000 /usr/bin/gok --login --access-method=directselection

    • Button/Switch Gesture:

      • <Mouse1> 3 3000 10000 /usr/bin/gok --login --access-method=directselection

      • <Mouse3> 3 3000 10000 /usr/bin/gok --login --access-method=directselection

    • Keyboard Gesture:

  • Click on pointer dwell. This item has to react on dwell; it is intended for users that can only move the pointer. Launches GOK in login and dwell mode, meaning that it shows only a keyboard and that hovering/dwelling is used to select items.

    • Command: /usr/bin/gok --login --access-method=dwellselection

    • UI Gesture: move the pointer over the corresponding item and stay motionless for a little while (dwell). Also activated via dwelling on the Accessibility Icon.

    • Pointer Gesture: TBLR I 10000 /usr/bin/gok --login --access-method=dwellselection

    • Button/Switch Gesture:
    • Keyboard Gesture:

  • Use switch input. Launches GOK in login and scanning mode, meaning that it shows only a keyboard and that the user's switch device will be used to start row/column scanning of the keyboard and to select items on the keyboard.

    • Command: /usr/bin/gok --login --access-method=automaticscanning --scan-action=switch1 --select-action=switch1

    • UI Gesture: Is it possible to navigate the Accessibility Dialogue and select items with the switches?
    • Pointer Gesture:
    • Button/Switch Gesture:

      • <Switch1> 1 5000 0 /usr/bin/gok --login --access-method=automaticscanning --scan-action=switch1 --select-action=switch1

      • <Switch2> 3 50 3000 /usr/bin/gok --login --access-method=inversescanning --scan-action=switch1 --select-action=switch2

      • <Switch3> 3 1000 10000 /usr/bin/gok --login --access-method=automaticscanning --scan-action=switch3 --select-action=switch3

    • Keyboard Gesture:

  • Press keyboard shortcuts one key at a time (Sticky Keys). Enables the StickyKeys feature of XKB with the default settings.

    • Command:
    • Pointer Gesture:
    • Button/Switch Gesture:
    • Keyboard Gesture:

  • Ignore duplicate keypresses (Bounce Keys). Enables the BounceKeys feature of XKB with the default timings.

    • Command:
    • Pointer Gesture:
    • Button/Switch Gesture:
    • Keyboard Gesture:

  • Press and hold keys to accept them (Slow Keys). Enables the SlowKeys feature of XKB with the default timings.

    • Command:
    • Pointer Gesture:
    • Button/Switch Gesture:
    • Keyboard Gesture:

  • Use screen reader (Speech Output). Launches Orca with speech and/or braille if they are available. See also note on "Enable Screen Magnifier" below.

    • Command: orca --no-setup --disable main-window

    • Pointer Gesture:
    • Button/Switch Gesture:

    • Keyboard Gesture: <Control>s 1 1000 10000 /usr/bin/orca --no-setup --disable main-window

  • Use screen magnifier. Launches Orca with the magnifier enabled and speech and/or braille if they are available. Note that this is basically the same as "Use screen reader (Speech Output)", but that it also includes magnification. So, treating this entry and the screen reader entry as independent checkable items may not make sense.

    • Command: orca --no-setup --disable main-window --enable magnifier.

    • Pointer Gesture:
    • Button/Switch Gesture:

    • Keyboard Gesture: <Control>s 1 1000 10000 /usr/bin/orca --no-setup --disable main-window --enable magnifier

  • Use more contrast in colors (High Contrast). Enables the high contrast theme.

    • Command:
    • Pointer Gesture:
    • Button/Switch Gesture:
    • Keyboard Gesture:

  • Use a larger font size (Large Print). Enables the large print theme.

    • Command:
    • Pointer Gesture:
    • Button/Switch Gesture:
    • Keyboard Gesture:

  • Use visual cues for sounds (Visual Sounds). Flashes the entire screen on a system beep.

    • Command:
    • Pointer Gesture:
    • Button/Switch Gesture:
    • Keyboard Gesture:

Notes:

• The UI gesture is meant for the Accessibility Dialogue; the pointer gesture is meant for the login screen; the button/switch gesture and the keyboard gestures are meant for the login screen and the Accessibility Dialogue.
• The Accessibility Icon location is in the corner for easier mouse navigation. Need to take this into account should the distribution choose to move things around.
• The Accessibility Icon and the "Click on pointer dwell" both have to react on pointer dwell. If both have an image, the indication of their dwell feature should be the same on both images and items. (inflating, or compression of a spring, or morphing from one into the other during the dwell time, or...)
• GDM has to check for itself if the pointer is dwelling on the Accessibility Icon and on the "Click on pointer dwell" item in the Accessibility Dialog. (It is not possible to use mousetweaks for this as it would disturb regular users.)
• For every option of the Accessibility Icon and of the Accessibility Dialog, it should be possible to start more than one application; for example an osk and mousetweaks.
• The applications to be launched/killed need to be customizable by distributions and system administrators.
• The input gestures for launching applications are available at all times.
• The GUI components also work as normal with the default desktop keyboard and mouse techniques.
• Activating/Deactivating items in the Accessibility Dialog also launches/kills any associated programs.
• Launching/killing applications using input gestures results in setting the appropriate state of the associated checkbox in the Accessibility Dialog.
• For single user machines it should be possible to customize the default behavior of the login screen. As such, care needs to be taken in the implementation to allow for this.
• Maybe in a later stage, passing activated ATs to the gnome session. If so, care might b needed to be to be taken in the implementation to also allow for this.

Issues:

• Does invoking an input gesture a second time kill the associated application(s)?
• What is the dwell time for the Accessibility Icon?

• Can we do a dwell for the Click on pointer dwell check box?

• The pointer dwell and switch items below both launch the on screen keyboard. Should we consider making the dwell and switch items "sub check boxes" (e.g., indented) of the Use on screen keyboard item?