Shotwell Architecture Overview: Thread Model

Threading in Shotwell was introduced carefully. Our design philosophy has been to use them sparingly and only where absolutely necessary. When a task can be performed in the GTK main thread, even if it requires using the Idle or Timeout queue, we’ll use it. However, that does not mean all tasks currently in Idle / Timeout belong there; there are plans to move more tasks into background threads.

Today (0.8) there are seven background thread clients: PixbufCache, BatchImport, Exporter, LibraryMonitor, MetadataWriter, and the ThumbnailCaches.

To minimize locking and synchronization issues, the basic model for threading in Shotwell is one of queued background jobs, each with an associated completion callback that is invoked in the context of the GTK main thread (from the Idle queue). Thus, the background job should be prepared in the main thread so it has all the information and references it needs when it’s invoked in the background thread context. When the completion callback is invoked, the background job is holding its results and/or error codes.

This design is to focus work in the background thread on the IO- or CPU-intensive task at hand. Data structure manipulations should be done pre- or post-execution, in the GTK main thread.

1. BackgroundJob

The work queue holds a list of BackgroundJobs, which is an abstract class. The BackgroundJob contains a CompletionCallback delegate and a Cancellable object. (Both are optional, although without the CompletionCallback the caller will never be notified when the job is finished.)

The only abstract method the caller must supply is execute (). This method is called in the background thread’s context. The caller can also override the get_priority () method to alter the job’s ranking in the work queue. (Note: This priority has nothing to do with thread scheduling, only its ranking in the queue.)

If the Cancellable object is supplied, the caller can halt the job by calling its cancel () method. It’s possible to call cancel () in the middle of the user’s execute () method. How that is respected by the code there (or even if it is) is up to the subclass. However, even if the execute () method completes with no errors, if the Cancellable flag is set, the CompletionCallback is not invoked.

BackgroundJobs may also have a CancellationCallback. This is similar to a CompletionCallback, but is only called if the job’s Cancellable flag is set.

Introduced in 0.5 is the ability for BackgroundJobs to send NotificationObjects back to the main thread in a while executing. The BackgroundJob must be supplied with a NotificationCallback to receive the objects. (In the future, GLib.Value will be used to box up whatever values the thread wants to deliver to the callback, however, due to two bugs in Vala, they must today be subclasses of NotificationObject.) Because of threading issues and the way work is queued in the event loop, it’s possible for NotificationCallbacks to come in after the CompletionCallback; it is up to the client of Worker to deal with this. A state machine is a good way to deal with this situation, and indeed, may be implemented in Shotwell in the future. Both the completion / cancelled callback and the notification callbacks may be given a Priority for the GTK main queue. Their defaults are reasonable for their use purposes.

2. Semaphores

If the thread launching the BackgroundJob must wait for it to complete before continuing, the job (s) may be given a semaphore which is automatically signalled when the job completes. There are various kinds of semaphores, all deriving from AbstractSemaphore, which handles the work of allocation and maniuplating the appropriate GLib primitives. Three semaphores exist in Shotwell today: Semaphore (which goes from Waiting to Signalled on the first Notify, broadcasting the change to all listeners), CountdownSemaphore (which goes from Waiting to Signalled only after the specified number of Notifies have been received, broadcasting to all listeners), and EventSemaphore (which is like Semaphore but is resettable).

3. Workers

The Workers class creates the background threads and has them monitor a priority queue of BackgroundJobs. Each worker thread dequeues the first job and calls its execute () method. If the job is cancelled, the job is dropped on the floor (although the CancelledCallback will be called, if specified).

Workers uses GLib.ThreadPool. It can be configured to only use so many threads (or a multiple of threads per CPU). Threads may be allowed to float between other Workers / ThreadPools by configuring Workers to be non-exclusive.