Feedback for GNOME Shell Usability Test Plan Phase I

Please place your commentary on the GNOME Shell Usability Test Plan here!

Please sign your comments with the @SIG@ @DATE@ command. -- MairinDuffy 2010-02-11 02:20:03

Commenting specifically on rev 6:

WM 6 ("Users shifting from full-size windows..."). Needs to be clearer; I'm not sure what it means / what the antithesis would be. ("Users will spend more time in the overview than they will actually working"??)
App Launching 4 ("Locating an application..."). This assumes that the current panel menu does not allow the use of spatial memory, but that's not true; the categories always stay in the same order and the apps within each category stay in the same order, and so after a while you know where to click without explicitly thinking about the categorization. So I would expect a test based on this hypothesis to be inconclusive. I think what you want is something like "Locating a previously-unused application in a categorized menu is not faster than locating it in a flat menu" or something like that, to specifically attack the idea of categories.
App Launching 7 ("That the search box..."). Seems like two separate hypotheses ("not confusing" and "convenient") that could be right/wrong independently of each other.
Notification 3 ("Incoming mail notifications..."). Isn't that already ridiculously well-established in the literature?
Notification 5 ("...without having to switch context..."). Might want to define "switch context"/"context switches" more explicitly. (Do you mean "change the focused application" or merely "think about something other than what they're currently thinking about"? Or do you mean one of those the first time and the other the second time?)
User Focus 5 ("...than those that are."). Then those that are what? The sentence looks like it got rewritten at some point and now the start and end don't match up any more.
User Focus 6 ("...less clickable spots...") "fewer"
Documents 2 ("That the search box..."). As above, may be two separate hypotheses.
Task Switching 2 ("...allow users to keep their gaze directed on their document..."). Assuming this is true, why is it good? Especially given that this is in a section titled "Task Switching", why do they want to keep their eye on the task they're leaving?
Workspaces 1 ("It is easier to transfer applications..."). Seems a little Grandma's-house-y? It assumes users want to move applications between workspaces.

Possible other hypotheses:

User Focus: "We are able to distinguish the parts of the screen related to the task from those that are not"? (ie, particularly UF3 becomes irrelevant if this hypothesis is false)
Documents: "Searching is faster than navigating a hierarchy"? (we have "less frustrating", but that's not the same thing. But we might not actually be making this claim...)
Task Switching: we don't have anything here about the new App Switcher

-- DanWinship 2010-02-11 15:38:41 2010-02-11 15:38:41

"Displaying document launching and location launching tools alongside application launching tools will not cause users trying to launch an application to lose focus.”

-- MairinDuffy 2010-02-11 15:58:59 2010-02-11

From Menti:

"Notifications shown in a short messaging tray aligned along the length of the bottom of the screen will not be missed too often."
"A zoom animation of the whole screen to shift the user from full-size windows to an overview of open windows will not cause eye fatigue to an excessive number of users."

-- MairinDuffy 2010-02-11 20:47:13 2010-02-11

Application Launching

2 - "Category-based application lists are more distracting than usage-based application lists."

This seems like a dubious claim to me. I also happen to disagree with the change to the UI that this implies, so when looking for the justification, this was particularly baffling. "Distracting"? What is distracting about seeing a list of categories? Is the idea that if I just want to use "f-spot" and I see a list that says "Productivity/Internet/Games/etc." I might see "Games" and get confusing? How is this less distracting than a larger list.

On the point of whether categories should be done away with, I also think that the "Application Launching" ignores the use case of somebody discovering what functionality is available on a new install (i.e. every new linux user's first experience). That's a case where categories play a clear, important role and it's not clear to me how a "usage-based" application list would cover the case of a brand new user curious about what e.g. multimedia editing capabilities the computer has.

"3. If an application list that only shows a subset of the total applications available does not contain the type of target application the user seeks, the user will be able to locate it in an extended flat list containing all applications in a reasonable amount of time in comparison to locating the application in a full and category-based application menu listing."

It seems important to me to specify how much information the user has. It's one thing if the user knows they want to launch a program "cheese", it's another if they're looking for a way to use their computers built in camera but have no idea if the program to do so even exists. There are of course numerous options between those two extremes (user has seen the program in use but doesn't recall it's name; user thinks they remember the name, user knows the windows equivalent name, etc.). Where you are on that spectrum makes a big difference in the outcome (also, what application you're switching for makes a difference -- some applications have well-known names but aren't easy to categorize; others are easily categorized but don't have well known names, etc. etc.).

"7. That the search box... will be convenient to them."

What sort of performance are we assuming? The speed of the search box matters *a lot* and I've already noticed it can be quite slow on my not-all-that-obsolete hardware. I realize there are still kinks being worked out, but I think it's worth looking at the performance implications of e.g. searching absolutely everything vs. doing a targeted search since user processor speed, filesystem size, etc. etc. can vary quite a bit.

-- ThomasHinkle 2010-02-14 01:22:25 2010-02-14