Shotwell Architecture Overview: RAW Files and RAW Developments

As discussed in the Photo File Formats section, Shotwell cannot work with RAW image data directly. RAW files are not traditional raster images made up of rows of RGB pixel values at some level of precision (such as 16 bits or 24 bits per pixel). Instead, RAW files are a direct dump of the luminant intensities recorded by a digital camera's CCD sensor at the moment of exposure. Like traditional raster image files, RAW files are rows and columns of intensity numbers. Unlike traditional raster images, what these numbers mean is highly camera-specific. The best way to think of a RAW photo is as a digital negative. Just like film negatives in traditional analog photography, RAW digital negatives aren't usuable until they're "developed" into conventional, 8 bit-per-component RGB pixel data for display. In digital photography, this development involves the application of mathematical tone-mapping curves that scale and bias the RAW image data into traditional pixel values.

Shotwell does not support the parameterizable development of RAW files internally (although RAW processing is planned for a future Shotwell release). Shotwell does, however, allow the user to switch between different kinds of RAW developments. Switchable developments have important implications for the photo pipeline and for the import system. This section discusses these implications.

1. Where RAW Developments Come From

In digital photography parlance, a "development" of a RAW photo is a normal, 8 bit-per-component RGB image file that results from applying tone-mapping curves to the RAW CCD data. Developments are usually stored in JPEG format, and it is possible to produce a virtually unlimited number of JPEG developments from the same underlying RAW CCD data. Shotwell acknowledges two RAW developments:

• The development produced by a user's camera
• The development produced by Shotwell itself

As discussed above, Shotwell doesn't yet support parameterizable RAW development, so when Shotwell needs to produce a development of a RAW photo, Shotwell picks some parameters for the tone-mapping curves that work in about 2/3rds of cases and applies them to generate a development. Internally, Shotwell produces its developments with help from the LibRaw library. Since neither LibRaw nor Shotwell know anything about the inherent CCD sensitivity of the camera used, the lens that a photo was shot with, or the lighting conditions under which a photo was exposed, the parameters used by Shotwell to produce its development are likely to be less than optimal. RAW developments produced by Shotwell may appear too dark, too light, or have lens distortion.

The user's camera, however, does know about lenses, lighting conditions, and CCD sensitivities, and so (at least in theory) can produce a faithful JPEG development from the underlying RAW image data. Not all RAW images have corresponding developments produced by the user's camera, so in some cases Shotwell may be only RAW developer available. If JPEG developments are produced by the camera at exposure time, they can come in two forms:

• JPEG data embedded in the RAW file itself
• JPEG data stored in a separate JPEG file with the same filename as the RAW file but with its extension changed to .jpg

The first kind of camera developments are called "embedded" in Shotwell parlance, while the second kind are called "sibling," "paired," or "RAW+JPEG".

2. RAW Developments and the Photo Pipeline

As discussed above, Shotwell can't use RAW image data directly. So when Shotwell works with a RAW photo, its nondestructive photo transformation pipeline has to start from a JPEG development of the photo. Shotwell allows the user to choose whether the photo pipeline should be headed by the camera-generated JPEG development (if one is available) or the Shotwell-generated JPEG development. The user can switch between developers by selecting his or her desired developer from the "*Developer*" submenu of the "*Photos*" menu.

3. Representing and Switching RAW Developments

Different kinds of RAW developments are represented by different values of the *RawDeveloper* enum declared in src/photos/RawSupport.vala. Switching between RAW developers for a given Photo object or set of Photo objects should be done by asking the Shotwell *CommandManager* to execute a *SetRawDeveloperCommand*.

When new RAW photos are imported, Shotwell must pick a developer to use with new photos. For this reason, Shotwell has the notion of a _default developer_, which is the developer assigned to new RAW photos at import time. The user can set the default RAW developer through the Preferences dialog. You can query and set the default RAW developer programmatically via the *get_default_raw_developer( )* and *set_default_raw_developer( )* methods of the *ConfigurationFacade*.

Whenever the user switches between RAW developers, Shotwell discards all transformations performed on the photo, including crops, red-eye removals, color adjustments, and rotations. While this may seem unnecessarily intrusive to the user, there's a very good reason for it: different developments of a RAW photo can be dramatically different. Since different tone mapping curves are applied, the color and tone of one development may differ greatly from the color and tone of another development, so any color adjustments performed on the first development aren't applicable to the second. Furthermore, different developments may have different sizes and aspect ratios, making coordinate-sensitive transformations like crops and red-eye removals non-portable between them.

4. RAW+JPEG Pairing and the Import Process

As discussed above, the camera development of a RAW photo, if available, might be embedded within the RAW file itself, or it might be stored in a separate JPEG file in the same directory as the RAW photo. Even though they are stored in _two_ physical files, the RAW file and its paired JPEG development are only _one_ logical photo -- the associated JPEG just happens to be a particular development of the RAW photo. For this reason, Shotwell treats RAW files and their sibling camera development JPEGs as one photo file internally. This pairing is done during the *on_import_files_completed( )* method of the *BatchImport* class. Associations between a RAW photo and its development JPEGs are maintained through the *BackingPhotoRow* class and the *BackingPhotoTable* in the Shotwell database.