There are some advances in technology that are actually steps backwards: features that look nice on paper, but always get in the way when implemented in reality. One of my pet peeves in this category is the Exif Orientation Tag, a little flag present in JPEG files generated by digital cameras that causes all kinds of havoc. It’s one of the places where the old proverb »the road to hell is paved with good intentions« holds true, because the idea behind this flag is a good one, whereas the flag itself is a product of pure evil. But let’s start the story at the beginning …
The world before the Orientation Tag
The first digital cameras simply captured images in their default (i.e. landscape) orientation, period. If you held the camera in portrait orientation, you got a JPEG file with an image that was rotated by 90 degrees. Most image viewers can rotate images temporarily for display, but if you wanted to permanently fix the image to appear in its correct orientation, you had to modify it manually.
When done the naive way (using standard image editing software), this means that the JPEG file is decompressed, rotated and compressed again, which is a lossy process. However, due to the way the compression works, it is possible to rotate JPEG images losslessly (at least if both width and height are divisible by 16 pixels; if not, a few pixels may need to be cropped off at the edges). Such algorithms are implemented in most image viewers, and they already were back in the day.
So, to conclude, portrait photos from early digital cameras required manual conversion to rotate them into upright position. Since tools for that were already well established, this was not a serious problem, only a minor nuisance. All could be well if the story ended here.
Orientation Sensors and the Exif Orientation Tag
Some camera manufacturers wanted to solve the problem with portrait photos and thus integrated orientation sensors into their products. If the sensor detects that the camera is held in a portrait orientation, the image would automatically be rotated so that this doesn’t have to be done manually later on.
However, there was a small technical problem: The image signal processing chips (ISPs) in the cameras were simply not able to encode images in a rotated manner. If the ISPs were able to generate rotated JPEG files directly, the problem of portrait images would have been solved once and for all in the best possible manner. But we don’t live in that kind of perfect world.
Since doing it the right way was out of the question, the camera manufacturers chose what they thought was the next best thing: Writing a flag into the metadata of the JPEG file that records in which orientation the photo has been taken. The natural choice for this was the Orientation field in the Exif data. Exif is the industry standard for metadata in files generated by digital cameras. It is based on the old TIFF image format, but slightly modified so it can be embedded into JPEG files (among others). The Orientation field itself isn’t an Exif invention either: It was already part of the TIFF specification for years, it simply was not very commonly used. With Exif though, it finally came into the spotlight.
The result of this metadata trickery is that portrait photos are written as JPEG files where the image data itself is stored in the wrong orientation, but a field in a metadata block contains information how the image has to be rotated to restore the proper orientation. What could possibly go wrong…?
Rotation confusion
The main problem with this scheme is that is ambiguous: While applications with full Exif support will show the photos as intended, Applications that just care about the image data and ignore all optional metadata will show the photos in a rotated fashion.
Back when the first cameras with orientation sensors appeared, almost no existing software was able to interpret the Exif Orientation Tag. The situation has improved since, but there are still many programs in existence that don’t care about Exif data.
All kinds of confusion
Even with the existence of the Exif Orientation Tag, transforming the photos so that the image data is physically stored in the proper orientation is still required to resolve any ambiguities. But how?
Using an application that does Exif-based rotation itself, correcting an image is not possible at all, because, well, it’s already shown in the correct orientation, so all bets are off about what happens when doing additional transforms.
This is why many Exif-capable applications offer an option to ignore the Orientation Tag when displaying images. This causes them to behave like old pre-Exif image viewers and allows for manual rotation into the proper orientation – at least if the application also modifies the Orientation Tag appropriately, so that the consistency between the layout of the image data and the Tag is kept.
If an application fails to update the Orientation Tag when rotating an image and instead preserves the Exif information wholesale (including the old value of the Orientation Tag), another zombie-like abomination is generated: Such a file will be displayed properly by non-Exif applications, but it will be rotated wrongly in Exif-compliant applications.
(Apparent) solutions
The good news is that the vast majority of image editing and managemant applications you may come across on desktop systems today is capable of handling the Orientation Tag in a fully compliant and consistent manner.
Transformation of the image data so that the Orientation Tag doesn’t call for further rotation on display is still a desirable process, though – after all, you may still come across software that doesn’t interpret Exif information even today, be it web browsers or photo viewers integrated in home electronics (TVs, DVD/BD players, …).
Options to batch-process whole folders of images to rotate them appropriately are abundant: Some image management tools can do it, there are multiple Windows Explorer add-ons that integrate into folder context menus and finally, there’sexifautotran and jhead for Unix-like OSes.
So everything’s fine again, you think? Not completely. There’s still one thing that poses problems.
Thumbnails and bugs
All the operations mentioned so far only deal with the main image. However, an Exif JPEG file can (and often does) also contain a smaller image – a thumbnail. This is used by some (but not all) image management tools in their picture browsers.
Not all applications rotate the thumbnail along with the main image; some programs just leave the thumbnail as-is, thus making it inconsistent with the main image (exifautotran is an example for that). This results in thumbnails being displayed with the wrong orientation in picture browsers, even though the image is displayed just fine when viewed normally.
OK, you may now think that the thumbnail issue is a minor one, or even not an issue at all if you don’t care about the thumbnails. Again, you would be right if this was a perfect world. In reality though, there’s still room for screwing things up. Let me introduce my new friend, the faulty Exif parser (seen on my Samsung TV, for example).
The thumbnails are encoded within the Exif data, and to make things more complicated (or consistent, if you want to see it that way), the thumbnail also has an Orientation Tag. As a result, an application that parses all the Exif data it can get (regardless of which image it belongs to) will read two values for the Orientation Tag: one for the image itself, one for the thumbnail. If these two values don’t agree because rotation of the thumbnail isn’t consistent with the main image, you may just be unlucky enough that the thumbnail’s value »wins« and you end up with a erroneously rotated image even though you took care to prevent that exact situation from happening. In other words, welcome to the inner circles of hell.
Conclusion
To sum everything up: The Exif Orientation Tag is a mess and the world would be a better place without it. There are so many ways where subtle brokenness of one piece of software can screw up everything that it’s just not funny anymore.
Why can’t cameras not just generate properly rotated JPEG data in the first place? Why do they still rely on the fragile tag hackery? Current cameras have so sophisticated ISPs (with motion tracking, HDR, panorama generation and more funky stuff) that rotation is likely to be no problem any longer. But still I’ve yet to come across a camera doing this. This makes me sad, very sad.