Spoon

NOTE: This page has now been superseded by Rarian. It has been left here for historical value.

What is it

Spoon is an experiment first and foremost. It's an experiment to see if we can kill Scrollkeeper, without causing huge amounts of pain to everyone involved.

In other words, Spoon is a replacement for Scrollkeeper, that might someday, in some form, be used within GNOME to find help documentation.

What's wrong with scrollkeeper

Taken from the ProjectMallard wiki page Scrollkeeper is the current document registration system for GNOME and Yelp. It tells yelp where to look for your documents. It has a few problems.

• Yelp has to access it through a command-line interface, grabbing the output
  • This output isn't a list of all the document titles, locations and descriptions or anything crazy like that. It is a temp file (e.g. /tmp/scrollkeeper-$USER/content.0) which yelp must then open and parse
  • The temp file isn't a list of all the document titles, locations and descriptions or anything crazy like that. It is a list of categories with entries for doc titles and paths to omf files
  • These omf files must then be opened in turn, the contents parsed again, finally yeilding the doc title, location and description
• It doesn't understand languages that it isn't translated into. E.g. i a Spanish doc is registered through scrollkeeper, but scrollkeeper itself isn't translated into Spanish, it will silently fail
• The code is (seemingly) needlessly complex for what it does

• It only handles omf files if they are installed in <prefix>. Installing a new program in your home directory won't make it show up in Yelp, without some serious dark-magic, if scrollkeeper is installed in /usr (default).

• Registration process at install time is a) slow and b) a pain for doc writers, packagers, maintainers, tool maintainers, translators and virtually everyone else
• It is unmaintained. At all. Not even bug fixes
• We are maintaining 3 patches for various seg faults just now. We get lots of bugs about these
• (from the TODO file in scrollkeeper) Registering an updated document won't overwrite a previous version. It will add a new doc to the index, leaving 2 copies of the same doc (potentially) on the system.
• It is missing a few features it would be nice to have

Structure of Spoon

1. Libspoon

The basis of Spoon is the library, libspoon. This (for now) consists of 2 calls: a for_each and a shutdown

• for_each

  • This calls a specified function for each doc Spoon knows about
  • Extra info can also be passed in if desired

• shutdown

  • This clears the internal list and frees all memory used in the library

2. Utilities

The package also contains several utilities that are useful

• spoon-sk-migrate

  • A program designed to take in a directory containing omf files (and a basename for the omf files), read through them and produce a corresponding spoon scroll file, dumped to stdout

• spoon-sk-update

  • A script like scrollkeeper-update. When run, it looks for new / deleted / changed omf files and adds / removes / alters the corresponding scroll files

• spoon-example

  • A sample program used to test and show off libspoon. It iterates through each doc found by libspoon, printing its title

Comparison with Scrollkeeper

As mentioned above, scrollkeeper has a few difficulties. This section examines these difficulties and shows how Spoon handles them

• Command line access

  • Scrollkeeper provides access to its index through a command-line interface in yelp
  • Spoon avoids this by providing access through a library

• Silent fail on unknown languages

  • Spoon isn't translated at all. There are no translatable string in libspoon. If a relevant translation is in a spoon scroll, it'll be used, otherwise libspoon falls back to the C locale

• Needless Complexity

  • I'll let someone else judge that in Spoon

• omf's need to be in <prefix>

  • Any scrolls found in $XDG_DATA_DIRS/help or $XDG_DATA_HOME/help will be used. $XDG_DATA_HOME takes priority, then $XDG_DATA_DIRS, in descending order.

• Registration

  • Spoon doesn't need any registration process. Take a spoon scroll and drop it into one of the searched directories, and Spoon'll pick it up

• Others

  • Principally complaining that scrollkeeper is unmaintained
  • Spoon is being actively worked on by myself

Use Case

1. Developers

This section describes a typical use case for libspoon: how a document would be installed and how yelp (or another help browser) would access it. Here, the fictional package "Foo" and is used.

The developer of Foo (Mark) has previous documentation in docbook format that is viewed in yelp. He hasn't converted it to gnome-doc-utils yet (bad man. No cookie for you). Instead, in his "help" Makefile, he copies 4 omf's (foo-{C,es,de,ru}.omf to the directory /usr/share/omf/foo/ and runs scrollkeeper-update. This is the traditional scrollkeeper-y way of installing scrollkeeper files.

On the devel announce list, he notices that from now on, yelp will be using libspoon to access documentation metadata. In the short term, he doesn't need to do anything: Spoon at install time will take care of moving currently installed omf's to their new format. Any new versions will still support the old scrollkeeper-update method of installation (with the installed omf's being translated to new spoon files within the scrollkeeper-update program).

In the longer term, he decides to update his documentation and move to the new-fangled Spoon scrolls. He writes a spoon scroll file (foo.document) and removes the old omf files. The Makefile is then changed to only copy this new spoon scroll to the <prefix>/share/spoon directory, overwriting any previous foo.document file. That's all that's needed. Done.

Another package, Baz, is also maintained by Mark. This one has already been converted to gnome-doc-utils (okay, you can have you're cookie back). When Mark is updating Foo, he decides to move Baz over to spoon registration as well, seeing as how easy it is. This time, there is no mucking about with Makefiles. As gnome-doc-utils is already altered to cope with both scrollkeeper and spoon, he just removes the omf.in file and adds a baz.document file in its place. That's it.

2. Users

A user, Mary, uses Foo. Her distro has a Foo package installed in /usr. It has already been translated to use spoon for its help registration (Good distro. Have a cookie). She wants to install the latest and greatest Foo, complete with its new spoon-ised help, in her home directory. She does the ./configure --prefix=/home/mary && make && make install dance and the package is installed. In yelp now though, the manual for Foo has changed to show the updated help. Spoon has found a new version of foo.document in $XDG_DATA_HOME/help and is now using that. As other people on her system are still using the older Foo package, they won't see this new help, instead getting the older manual.

Current Status

Spoon has now reached 0.4. This has redone the library to mimic the Help Spec (proposed here). It also added the spoon-sk-update utility.

V0.3 allows installation, using a configure-time option. This defaults to OFF (i.e. --enable-installed MUST be specified in order to work properly). There are a load of instructions and such in the enclosed README file (though its pretty messy).

For those brave souls eager to try it out, the latest release can be found here. Instructions are included in the README. Please read them. I'm not taking responsibility if it deletes all your omf files, or anything else on you're computer .

More details can be found in the emails: Spoon 0.1 release announcement , Spoon 0.2 release announcement and Spoon 0.4 release announcement

Before progressing any further with this, I'd really like some feedback from people about the structure of the library, the scroll file format and anything else. Without any feedback, I get the feeling this'll turn into another scrollkeeper style arrangement, where the idea is good, but the implementation sucks. Since this is based on a draft help system spec, please comment on that. The future now lays in trying to get yelp to use spoon internally. Then, getting jhbuild to run fully with Spoon, getting all docs showing up as before. Then and only then, could Spoon be considered for addition to GNOME. I'm hoping to get these 2 ideas done before 2.20 and get Spoon into that.

0.1. Comments

MikkelKamstrup: Firstly, I totally support your efforts. Scrollkeeper has always struck me as stoneage technology. Now for the constrcutive critisism: In the Libspoon section you write that spoon goes through all registered documents during initialization, but in Comparison With Scrollkeeper you write that spoon needs no registration... Which one is it? I'm not sure I totally grok how libspoon works - perhaps you can outline a use case describing a concrete example using libspoon.

• DonScorgie: Thanks. That was a slight error on my part. Spoon scrolls don't need registered. At runtime, each directory that libspoon is told to check (through a config file) is scanned and each scroll file there is read and added to the list. I've also added a "use case" section above

  • MikkelKamstrup: Just to be 100% sure I grok this; in the use case the series entry of the .spoon is used to determine the newer version right?

  • DonScorgie: In the New World Order, the "DocSeries" is used to determine newer versions. This falls back to "DocHeritage" when that exists.

EmmanueleBassi: thanks for tackling this stuff: SK has become a pain for everyone to use. with regards to the libspoon API: why are the init/finalize calls even necessary? there has been an interesting thread on dbus-list about the initialization functions, and I kind of agree with the position expressed by Havoc here and here: if libspoon_init() is really needed before calling libspoon_foo(), then it should be implicitly called by any of the functions inside libspoon API and cache its data so that any subsequent call is a no-op. if you really care about releasing memory as soon as possible, a libspoon_shutdown() might be provided (this might be an issue with dynamically loaded modules using libspool, by the way).

• DonScorgie: The init has now been made implicit in the library (as of 0.4).

• ÉtienneBersac: So what about rarian

• DonScorgie: Spoon is now Rarian. I've added a note at the top of the page.