This page is to gather ideas, and possible to-do items with regards to using GNOME Shell, and GNOME 3 on touchscreen devices.

This can either be all-in-one computers (such as the ones marketed by HP, Lenovo, Asus and others), as well as slates, and touchscreens on laptops.

Existing problems

• Open Touch Bugs

• Mouse

  • the cursor should not be visible if no mouse are plugged in

  • maybe gnome-settings-daemon should set a different theme ("no cursors") if no mice are plugged in, and revert to the original configuration when one is

  • Busy cursor should use the spinner next to the AppMenu that is used for app startup notification (bug 652066)

  • Should not show Mouse and Touchpad System Settings unless there is one (bug 650699)

• Keyboard

  • Need a usable on-screen keyboard: On Screen Keyboard Design

  • Should only show Keyboard System Settings when a physical keyboard is attached (bug 650699)

  • On-screen keyboard should keep the input area visible (bug 681199)

• Workspaces

  • Bug 652714 - make the workspace sidebar always visible when there are multiple workspaces in use. (We could also make it always visible when in "touchscreen mode".)

  • Bug 646409 - allow dragging of workspaces to reorganize them (windows within the workspace thumbnail make impractical click targets on touch devices)

• Applications search
  • searching for applications is harder and much slower without a keyboard to input search terms
  • it should be easier to look for "end-user" applications in the overview, without finding utilities or preferences

    • this should be fixed once the search design has been implemented. -- AllanDay

• Hardware button shortcuts
  • having to look in "Keyboard" to assign actions to physical buttons is a bit of a stretch
  • it should be easier to list the physical buttons on the machine, and assign actions to them

  • If we need to allow hardware buttons to be remapped, it might be appropriate to add these options elsewhere in the control center (under Details, perhaps) -- AllanDay

  • I think hardware buttons configuration could be used as well for configuring accessible switches. This can make a difference for those users. http://en.wikipedia.org/wiki/Switch_access -- JuanjoMarin

• Maximization
  • Currently, most applications start in a windowed, non-maximized mode

  • The shell should make it possible for applications to show up maximized as soon as launched (bug 651075)

  • Some applications might have a fullscreen mode that is more appropriate than their maximized mode (eg. Cheese, or Totem)
• Drag'n'drop actions
  • Some of the actions only seem available via drag'n'drop, such as removing launchers from the dash
  • Launching applications in specific workspaces also requires drag'n'drop

    • AllanDay - what's the issue with drag and drop? We already comfortably accomodate it alongside long press for context menus.

      • Drag'n'drop across large surfaces is cumbersome and prone to errors. Launching an app in a workspace means dragging across the whole length of the screen. (-- BastienNocera 2012-08-07 09:50:57)

• System Status area

  • On a high-resolution slate (tested on a 11.6" 1366x768 slate), hitting some of the functionality like the icons in the top of the bar is very hard and fiddly

  • The volume bar only seems to interact when the slider itself or the trough is clicked

• Message tray

  • 677213 - don't rely on hover to access tray items

  • 691534 On touch screens, swipe up from the bottom of the screen to open the message tray

  • 657273 need a way to show menu on tray items on touch devices

• Lock, login and user switching

  • Instead of bringing up the unlock/login dialog when the user input moves we need to make it a more explicit action to avoid continuously trying to unlock when the locked tablet is being handled.

  • Login screen - don't show the user list if there's only one user.

    • I think this is being worked on... -- AllanDay

  • 735782 Offer a touch-friendly alternative to passwords for authentication, such as a PIN

    • This can be offered as an option during initial setup

  • Need to support hardware keys (and on-screen keyboard) when screen is locked

  • various status indicators and notifications should be available on the locked screen

• Keyring and secrets
  • A keyring is useful but should use authentication methods appropriate for a touchscreen interface
    • might have encryption based on the hardware UUID
    • or using the TPM chip as a store instead of on-disk
• Screen orientation
  • Configurable orientation-based automatic rotation using accelerometer/gyroscope/magnetometer
  • Both the shell and apps should be able to switch orientation based on sensors
  • Support for orientation input from many sources, including iio, i2c, and usb devices
  • Easy ability to temporarily disable and enable auto-rotation (rotation lock)
  • Rotation lock or other rotation commands should be easy to bind to hardware buttons
  • Some applications might want to lock the screen in a particular direction, for example, landscape for a game (or a video player in some cases)

  • 673075 improved orientation settings in "Displays" panel for touch devices

• Tooltips won't work on touch screen devices. 755241

• Activities Overview

  • Bug 664995 - can't close windows in overview on touchscreen.

Mutter

• Multi-touch support

Toolkit

• Touchscreen devices obviously do not have "right-click" functionalities

  • Hold as right click bug for GTK+

  • The shell already implements hold-as-right-click on its own in some places (eg, the dock)

• GTK+ scrolling bugs

  • Scroll widgets (scrollbars, etc.) should probably not appear in the interface when touchscreens are used

  • Kinetic scrolling support

    • Web (Epiphany) seems not to make use of it for touch scrolling/zooming as of 3.14.1.

  • 681286 - show overlaid toolbars when a pointing device isn't present

• Multi-touch support (X.org support needs to be sorted out before support is added in GTK+ and Clutter)

• Deprecate gtk-touchscreen-mode, This is rather dynamic now, as the source input device can change on the fly, so generally gdk_event_get_source_device() should be used to determine any changes in behavior.

  • It would actually be useful to have an easy way for applications and other system devices to know what environment they're in, eg. we'd handle a slate with no attached input devices differently from a desktop computer with no touch support, and no attached devices (starting an on-screen keyboard, vs. starting a Bluetooth pairing assistant for keyboard and mouse) (BastienNocera)

    • My point is that the environment can also be rather dynamic :), with the HP Touchsmart I can trigger any input mode in laptop mode (pen/touch/touchpad/keyboard) or get into touch/pen only in tablet mode, so we can only do the right thing based on the hardware input device behind an event.

      • A yes/no setting is obviously wrong, but gnome-settings-daemon can probably figure out whether you have access to 1) touchscreen only 2) touchscreen + keyboard + touchpad 3) no touchscreen, etc. using the list of devices attached to the computer, and hardware switches state (when you convert your laptop into touchscreen only), and export this to GTK+ apps through XSettings. GTK+ can then make it easy for apps to use using a GtkSettings property. I'm just talking about presentation though, not about behaviour (which should change depending on the source of the event)

• Onscreen keyboard support

  • CarlosGarnacho: Should be done as an IM module I think, integration with the shell as described in the link below sounds great, but we need to support the classic mode as well IMHO.

• Text selections

  • See GnomeOS/Design/Whiteboards/Selections#Tentative_Design

Stack/kernel problems

• Faster Wi-Fi scanning, automatic 3G fallback. See NM TODO

• Audio routing (headphones, internal speakers, Bluetooth, HDMI output) (PulseAudio)

Test hardware

• WeTab slate computer (FlorianMuellner)

• Lenovo A600 all-in-one computer (BastienNocera)

• HP Touchsmart TX2 (CarlosGarnacho)

• Lenovo X220t (CarlosGarnacho)

• ExoPC Slate (DanWinship, Jon McCann, AlexandreFranke, DavidKing)

• Mimo USB Touchscreen Monitors (Looks like a tablet and behaves like a tablet but is just an USB Display with integrated touchscreen which you plug into your PC and provides it with a second seat) (LennartPoettering)

• Lenovo ideaPad S10-3t convertible netbook (DavidKing , PhilippeCoval, AlexandreFranke)

• Lenovo X1 Carbon (BastienNocera)

• Dell S2340T touchscreen (BastienNocera)

• Aava Inari 8 tablet (MichaelShigorin)

• Lenovo ThinkPad Helix (ultraviolet)

• Bungbungame Photon 2 (ReneLinder)