Text Improvements

As a writer, it appears to me that GNOME does not provide the writing experience it should do. There is no GNOME application for writing, or it's WYSIWYG (a writer does not want to leave his keyboard), or it's for code (things that are good for code are not necessary good for writing). Worse: the writing experience is inconsistent through the environment. There is many things we can improve.

In this document, I will describe several tools specifically thought for writers and how they should work if GNOME was the idealistic typing machine it's deserve to be. But keep in mind that what is good for writers is actually good for everyone who write : almost all these writing companions are supposed to function everywhere in GNOME where text is involved.

Except for the document improvement and the Writer application, everything we'll talk about in this page is intended to operate throughout the GNOME environment. More precisely, some of them like the improved cursor, auto-completion and text selection are supposed to work anywhere, but others may only function where it's relevant: in applications like Abiword, Writer, Conversations, into text areas in Web, into strings in Gedit, &c.

Documents

Plain text files are documents. Same for those that use lightweight markup syntaxes, such as Markdown, Textile, reStructuredText, txt2tags, &c. Thus, files with .txt (or without any extension) or more specific extensions (for Markdown this would be .markdown, .md & .mdown) might appear into GNOME Documents. Selecting those documents might display the text like it is in GNOME Writer's preview, and propose the user to edit the document in GNOME Writer.

Comments

Writer Application

This section was too long so I've created a dedicated page for it.

Improved Cursor

When I don't remember where my mouse pointer is, GNOME let me press the Ctrl key. When I drink a tea and come back to my text, I like to see what I was writing previously, just in case… If I was just writing, that's OK, but if I was modifying something? My cursor is blinking. Blinking is obviously not enough: I loose my cursor way too much, forcing me to write a few random letters to see were things move. Worse: blinking is not enough, but it is enough to disturb me when I stop writing for a few seconds…

So we might have a fixed cursor but we need it to be more visible. What do we do? Although almost everywhere, including GNOME, the cursor is black, or has the same colour than the text typed, in iA Writer, the cursor is blue. And in UberWriter, which aims to be an Ubuntu equivalent to iA Writer, the cursor is orange. Since it is one of major colours in Adwaita, I propose blue.

Comments

Invisible Characters

When a user is writing something, nothing should be placed in his way. Thus, invisible characters might remain invisible, letting him write the way he's thinking. But sometimes, for specific reason, the user may want to display them for a few time. It may even for some reasons be relevant to display them by default, in some specific situations. In such situations, those characters must be shown with a certain emphasis. What I propose here is not to display them by default but, when it is relevant to show them, to show them well. I propose to do exactly like we did with our new cursor: to show them blue. This has a second advantage on the traditional black: since black is the colour of the text, if for some reason the user added a central dot or a return symbol in his text, he'll know which is the special character and which is the displayed invisible character.

Comments

Text Selection

In GNOME, as in many other environments, text selection start when you begin to click on a text and stop when you release the left button of your mouse. That works fine and is precise. But. But usually people do not care about precision. We think it's normal when we use a mouse, because it's a precise pointer, but this become really annoying when we use touch. Take a look at how exactly you select text; that's me with my touchpad (or a mouse):

• I target the first word of my selection,
• I double-click and hold,
• I target the last word of my selection,
• That's it.

But normal people don't know about the double-click thing. For normal people, it look more like me on a touchscreen:

• I target more or less the beginning of a word,
• I try to be precise and to target the good character,
• I begin to click and target the end of my selection,
• I try to be precise and to target the good character,
• I failed the second or fourth step and try again.

In order to correct that, I propose a less precise but more effective behaviour, that can work for both mouse and touch. It begin like the previous behaviour by targeting more or less a character, clicking and holding. But here is the nice part: when you reach the beginning of a new word, your first word is selected and every new word is selected as a whole. In the same way, when you reach a new sentence, you pass to sentence-selection. And when you quit the first paragraph, you pass to paragraph-selection. I don't think we need more than paragraph selection, but if paragraphs make sense together it might be a good idea to extend this behaviour to this new kind of block.

Then, if you didn't want to select that much, the environment might provide you sliders (like in iOS), in order to get back to precision.

Comments

Auto-Completion

Before we start with this, let me bring to your attention that we're not talking about code completion: there are specific tools for that and coders might be better placed than me to talk about it. No, I'm talking about natural language completion.

Now, most of you are imagining some possibility to auto-complete a word after typing the first letters. Well, this is part of auto-completion: when a language is selected, you type a few letters (let's say four) and, if the system matches something in a dictionary, something is proposed to the writer, who can complete his word by pressing Enter. That's a really nice detail in a desktop environment, but we can go further.

Substitutions

As a beginning, we might propose to the user some common substitutions. (tm) becomes ™. (c) becomes ©. Some other substitution may be common in a specific language. Here are some example of common substitutions in French (that's my mother tongue):

• dico → dictionnaire (dictionary),
• rdv → rendez-vous (meeting),
• a+ → À plus tard (See you soon),
• ab, tb → assez bien, très bien (well enough, very well)…

Continuing with language specific substitutions, Diagonal Prolexis (a correction tool for French for Mac OS X and Windows) proposes a function called “sandwich completion” that may be interesting for us. The idea is that many words are beginning with a same base, with different endings. Here are some examples given on their website:

• exce + mt → exceptionnellement (exceptionally),
• adm + tn → administration,
• adm + tr → administrateur (administrator),
• inst + tr → instituteur (teacher)…

We also might consider trivial corrections (see below) as language specific substitution.

Typographic completion

Now, let's go further again. In French, we have to put a space inside quotation marks and before two-parts punctuation signs. But it's not a normal space: it's a thin non-break space. This space doesn't exist on the keyboard. When I write for my blog I use the entity   but normal people have never been told that there's a thing called “entity” they can use for that, and for a good reason: normal people don't know HTML. So it'd be really nice if the system could add those automatically. Another example: quotation marks in French are like this: «» Let me say you that they're not on a standard keyboard either (but we have "). Yes, French people cannot write in correct French with a normal computer.

And I speak about French, but I'm sure there are plenty of example you can find in many language. Take English. Yeah, English, that language that does not have any diacritic nor exotic punctuation. Well even in English John Gruber has found enough glitches to create SmartyPants. That, is auto-completion for natural language.

But remember: sometimes the user may not want auto-completion. He always might be able to undo an auto-completion with Ctrl + Z.

In the same way, one might want to add automatically a closing mark when he types an opening. As an example, if I type « ( or {, GNOME may add » ) or } just after my cursor.

Snippets

Until there, we have spoken about auto-completion and substitution proposed by default with a language or even the system. But there are plenty of situations where someone may want auto-completion that the system cannot guess. A customized signature, by example. Or some expression we write really often but is specific to a very specialised domain. Or a script that may add the date and time in a specific format (files for my blog posts are named like that: YYYY-MMDD-Title.blog.markdown). Or… many other things that we don't know yet…

Concretely, what I propose here is the ability for the user to tell the system to substitute on the go an abbreviation of his choice by some text. The text may be generated by a script (in JavaScript, since this is the default language chosen for GNOME, or in the language the user may choose if a shebang is set at the beginning of the script).

Of course, these snippets may be used everywhere where text may be typed by the user: every text field, even for writing the name of a file, even when writing a new snippet (by example, if I write my snippet scripts in Python, the shebangs could be written through a snippet)!

TextExpander does this on Mac OS X and AutoKey also does on GNU/Linux, but why do we need to install a software in order to do such a simple thing as system-wide snippets? I'd really prefer something in my system settings (maybe with the shortcuts?).

Quote Completion

There is one last thing with auto-completion: quotes. Let me explain what I mean.

When you paste a text (a real text, not an URI) from outside the text you're writing, it might be marked as “quote”. In Conversation this means adjusting the quote level; in Writer it'd be placed into a quote block, depending on the syntax; in Abiword this would be applying a “quote” style to it; &c. Again, this might be undoable: the mark as quote follows the paste.

But why doing this? Well, it allow you to play with this quote. By example, if you delete some words inside the quote (but not at its beginning nor at its end, where this is not needed), the system should add this: […] And if you select some text and write something directly? Easy: this something should be surrounded by square brackets. Same for adding text instead of nothing. And if one does want the “mark as quote” thing but not the square brackets? Piece of cake: Ctrl + Z after the first character. What if I delete the text inside the brackets but not the brackets? If there is an history of changes and there was nothing before the text into brackets, the system might delete the brackets too, else they're not removed. In auto-completion, there's auto-deletion too

Bonus: there are smiling faces in Unicode; that deserves auto-modification too.

Comments

Correction Tool

Before we start this section, I'd like to draw your attention to two particular points.

First, although it may seem that such a thing is obvious, but correction may not be relevant everywhere in the system. For example, if spelling and typo corrections are relevant for naming documents, we do not want to take care of a syntax or about orthotypography. Same way, when one code, He does absolutely not want any correction for natural language, except in strings… An IDE should then ask for correction in the right place…

Second, correction implies manipulation of dictionaries and semantic engines. But this section is not about designing a user interface to these dictionaries. Such interface would be a really nice thing but needs its own design page. However, these dictionaries need to exist in some way for our correction tool to work. It will ask them informations directly for spell and typo correction, or through semantic engines for grammatical correction. In order to let Correction Tool work efficiently, these dictionaries may include in addition to definitions synonyms, antonyms, pronunciation (for homophones), thesauruses, and occurrences (partial list).

Now that we've seen those points, let's begin.

The interfaces

Correction tool has three interfaces: trivial correction, pop-up correction and in-line correction.

Trivial correction is used when the error is obvious and there is only one possible correction. For example, when a letter is missng, when one has been doubbled with no reason, or when two letters are invetred. It also work when we use a letter that is clise to the letter we wanted to use in our keyboard, or when a user uses two keymaps and obviously write with one insteqd of the other (here it's AZERTY instead of QWERTY). For these typo corrections, or for obvious misspellings, there is no need to disturb the user with correction: trivial correction is an interface that work transparently to the user and simply substitute one word by another, when space bar is pressed or a punctuation is inserted.

Trivial correction uses syntax engine when it's relevant, then spelling engine (engines are explained later). This interface is used everywhere where natural language is involved and may be undone, like other text substitutions, with Ctrl + Z.

Pop-up correction is the most visible interface for our correction tool, yet it is the more discreet. When the user is typing, it is out of sight, but may be displayed on demand. The problem is that traditionally the user has to think to use it, so he does not. The traditional solution to this is to underline with red the errors. Therefore, the correction is no longer discreet and the user is disturbed.

Our solution is different: the user needs to remain thinking of what he writes and proofread only afterthought, rather than being constantly annoyed by a tool. This is therefore the tool to adapt to the user and not vice versa: the application must provide the correction at the right time and not at another. In Writer, as it has already been see, this is when the user closes the window, goes back to the entries list, or when he is going to export the entry to another format. This would be quite similar in Abiword (but also when the user is going to print his document) or in Gedit. In Conversation, it'd be when sending a message. &c.

But we've not finished yet with pop-up correction. Indeed, we still have to talk about the interface itself… Well here is how I see things. The window takes the full screen and shows four columns. The first column lists all the errors detected by every engines. If an error (with the same possible corrections) has occurred several times, it will be listed once only. The first error in the list is selected by default, but the user can select another error. The second column is larger than the others and shows the selected error in context. If there are more than one occurrence of an error, this second column authorises the user to select an occurrence (else first is default). The penultimate column proposes a list of possible corrections, with a text field on its top (if used and there's an error in it, this new error will be added at the bottom of the first column), and the last column gives the user explanations about why this was an error and why this correction. At the bottom of the last column, six buttons in four rows are waiting for us to use them:

• Replace All / Replace Occurrence
• Ignore All / Ignore Occurrence
• Always Ignore in [name of document]
• Add to Dictionary

When there is only one occurrence of an error, the second button of the two first row are greyed. The last button is greyed unless the error has been detected by the spelling engine. If invisible characters are involved in an error (by example, two spaces instead of one), they are displayed in the context and correction (and explanation if needed).

In-line correction is the traditional coloured underline for erroneous text. If pop-up correction was possible everywhere, in-line correction should not exist. Thus, in-line correction should not be proposed where pop-up correction is possible. But in other situation such as the renaming of a file through Files, or the filling of an input field in Web, system may propose this stopgap. This should use every engines too, but since it is a stopgap, we don't need to pick a colour per engine.

The Engines

The GNOME Correction Tool uses five different engines for: spelling, grammar, orthotypography, syntax, and semantics.

First of all, a text passes through the syntax engine. This engine might use the syntax of a document to distinguish between the labour text, the titles, and the markup. By example, it should in an IDE be able to isolate strings or comments from the rest of the code, and even to understand concatenation or the use of a variable inside of the string. Another example: if I use markdown to emphasise only a part of a word like this: “Fish could have been written ‘gheoti’ like in enou*gh*, p*eo*ple, and na*ti*onal” then the syntax engine will isolate the words from the markup and send “enough”, “people” and “national” to the next engines (i.e. the spelling engine won't detect any error).

Of course, the syntax engine also aim to correct syntactical errors, such as a markup opened as an emphasis and closed as a strong emphasis… Then, the syntax engine will return two things: the errors for the interface, and the texts cleaned of their markups to the other engines.

The spelling engine simply uses a set of dictionaries and take a look at every words in the input to see if it matches. The user should be able to activate or deactivate every single dictionaries through the language panel in system settings, except for the personal dictionary, where words “added to dictionary” are listed. There can be more than one dictionary per language, and not only for variants like British English & American English, but also for general language & domain specific vocabulary. By example, there can exist a dictionary for vernacular names of plants in English, a dictionary for anatomy, &c.

Here I talk about words, but this engine may work for common expressions or co-occurrences, such as:

• an user → a user
• (in French) c'est à dire → c'est-à-dire (it means “i.e.”)

• &c.

When a word doesn't matches but is considered as trivial (see above the interface for trivial correction), the error is directly sent to the user interface. Else, the engine send its results to the semantic engine, which could veto some of them (in the example given for the syntax engine, “gheoti” is not to be listed in the errors).

The grammar engine mainly checks for gender and number accordance, conjugation, construction of neologisms, improper connections, confusions, or suspicious constructions. It verifies that the words passed through the spelling engine is in good shape.

The orthotypography engine is aimed to help us to improve our compositions. It may check the following of typographic rules for punctuation (like I already said, French is capricious with spaces, but there are also subtleties such as: “etc…” → “etc.” and “etc.,” → “etc,”), abbreviations (the French Lexique des règles typographiques en usage à l'Imprimerie nationale (Lexicon of typographical rules in use at the National Printing Office) has a long list of standardized abbreviations), formats (for date, time, numbers, acronyms, chapters…, and good use of superscript, subscript and small caps), and many other things. It also check that the user didn't doubled any spaces, nor forgot an unpaired parenthesis (or bracket, or quotation mark, or… whatever needed to be paired and wasn't).

Like for dictionaries, the user can choose the typographic convention of his choice (like for dictionaries, this is not only related to the language, but also to the country or even be specific to an institution we work for).

Finally, the semantic engine uses different patterns to make sense from the analysed text. Thereby, it might be able to correct pleonasms, misuse of a word that was orthographically and grammatically correct but was homophone or paronym of the word you meant, or even dates (“Monday, December 14^th, 1989” → “Thursday, December 14^th, 1989”).

Comments

Conclusion

At last, we reach the end of this document. Thank you for reading that much. I hope that it was not too boring and that it gave you plenty of ideas. If you have questions, or comments, well… the comments subsections are there for you! If something is not precise enough, simply ask.