Improvement of Nomacs Raw Loader

The RAW image data we get from digital cameras are the values taken from the cameras sensor. This image is mosaiced: every pixel has just one of the three colors: red, green, blue. The pixels form a pattern that is called color filter array. The most common of them is the Bayer filter which is composed of two green pixels on one diagonal and of one red and one blue pixel on the other diagonal. Bellow picture shows a small part of the Bayer filter.

 In order to create an image that represents what a human eye would see we need to process the RAW data. There are a few essential steps that we need to do:

1. Normalization of RAW data: the RAW image needs to be normalized depending on the black point of the used digital camera and its dynamic range.

2. Demosaicing: for each uni-colored pixel we need to find the other two colors of this pixels. This is done by using the interpolation that is based on the type of the color filter array.

3. White balance: The demosaiced image is very green. To remove this effect we multiply each image RGB channel with a constant. The three constants are based on the photographed scene illuminant and they differ for each digital camera.  

4. Color correction: The colors of the white balanced picture are still not right. We need to do another transformation that converts colors from the camera color space to a standardized color space such as sRGB or AdobeRGB. This is done by multiplying pixels RGB values with a so called color matrix.

5. Gamma correction: the human visual response to the light is nonlinear. On the other hand the digital camera response is linear. The process of changing the linear data to nonlinear data is called applying gamma correction.

The previous version of Nomacs RAW Loader had a little different sequence of this changes and it was missing one correction: the color correction. After changing this the improvements are clearly visible. Bottom image shows the difference between previous and modified RAW Loader.

So when we postprocess a RAW image we need to be careful that we make all needed corrections. Missing one completely changes  how the processed image looks.

Hue, exposure, gamma

As promised, we will now take a look at the other three image manipulation tools that lie inside the Nomacs image manipulation dialog: hue, exposure compensation and gamma correction. The range of changes that we can make are:

Hue from -180 to 180. Step size for the bottom picture is 36.

Exposure compensation from -6 to 6. Step size for the bottom picture is 1.2.

Gamma correction from 0.01 to 9.99. The bottom picture shows changes for gamma 0.1, 0.2, 0.4, ..., 1.0, 2.0, 4.0, ..., 8.0, 9.99.

Gamma and exposure changes in the top and bottom slider range are very extreme. If we are changing 24 bit images then we also have a great loss of data. So the values in the extreme zones are not recommended unless if the images are very under or over exposed.

New Image Mnaipulation Tools Dialog design

In the last two weeks the image manipulation tools was completely redesigned. Not only the GUI changed but also the source code changed a lot.

Let's start with the most visible thing. Sliders have a new look which makes the dialog look more professional and appealing. The difference between this and the previous look is noticeable.
To the list of manipulation tools was added the exposure control. I will present it in a separate post together with hue and gamma.

The work on the code in the past couple of weeks was mainly a research project. I was trying to find the best method to use for applying the sliders changes to the image in the main window. Because the changes made by different sliders stack on each other, the process to make them can be slow. Especially for big images.
At first the best way seemed to use 16 bit lookup tables (LUT). We create them very quickly and also we apply them very quickly on images. They are the quickest way of making changes on big images. In RGB space they can also be stacked together. We can then apply the LUT on the main image once. It sounds very promising but there is a problem. In our image manipulation dialog we can change the saturation and hue of an image. For this we need to convert the current picture from RGB to HSV space, change the Hue or Saturation value and convert back to the RGB space. Because the transformation is nonlinear the change can't be done using a lookup table in RGB space. With a simple 16 bit LUT in RGB we can't take into account all possible changes of saturation and hue in HSV space. For this we would need a 16^3 bit LUT which is too big.
The remaining solution was to use LUTs for changes on a local scale (within one slider) and to create a history of changes. When hitting the OK button we go through the history an apply all changes to the main image. This process can be slow because we need to go through image pixels for every change in history. But this is the only way to do it without losing the accuracy of the image data.

The positive side of having a history vector of the changes is that it enabled us to create the undo and redo buttons. We just need to go up or down through the history and recalculate the changes to that point.

Image Mnaipulation Tools Dialog

In the previous post I showed the first three image manipulation tools that were added to Nomacs. The tools were placed in a Qt Dock Widget and a change in tools values resulted in a change of the displayed image colors. For changing the colors of an image we need to go through each of its pixels and do a calculation depending on the tools values.  As we can see on the graph in the post Image histogram - speed upgrade the time needed for going through all the pixels exponentially increases with the image size. So the manipulation of larger pictures took more time and this resulted in a bad user experience.

The solution was to move the tools from the main window to a dialog with a small picture preview. This way all the changes that user makes are firstly displayed on a scaled image. This is done faster then on an unscaled image. This way the changes on the slider are smooth and the preview is redisplayed right away. Only after the user is satisfied with the preview, the changes are made to the image in the main window.

The current look of the dialog is displayed bellow:

Image manipulation tools

The main goal of my first part of the Google Summer of Code program is to create image manipulation tools. I started programming them after I finished the histogram. I've added the Qt Dock Widget, which will contain different image manipulation tool widgets.

The widgets I've added so far are three sliders for changing the brightness, contrast and saturation of the opened image. The picture bellow shows the current look of the added widgets.

Let's also take a look at the changes we get with different slider values of brightness, contrast and saturation.

Brightness from -100 to 100, step 20:

Contrast from -100 to 100, step 20:

Saturation from -255 to 255, step 25.5:

Image histogram - speed upgrade

When creating an image histogram we need to go through all the pixels in the image. In larger images there can be a lot of pixels so this process can be slow if not programmed correctly.

First time I used Qt and its QImage::pixel(x, y) function to access pixels in the image. It was very slow. It took more then one second to go through an image of size 4320 by 2868. One second waiting for a histogram to show is a long time and an user certainly wouldn't be pleased with it. The alternative way is to use OpenCV and its class Mat which needed a quarter of a second. This is a pretty big difference and a big improvement in user experience.

Bellow I've created a graph showing the time in milliseconds needed for calculating a histogram of a square image. The horizontal axes shows the number of pixels in one side of the image.

We can see that using Qt gives us larger calculation times. This is why I don't recommend it for image manipulations where we need to read or modify all the pixels in the image.

Image histogram

Every beginning on a new project that is already a fully functional program is hard. There is a lot of code already written that you need to get used to and in Nomacs there is also the Qt framework that I am using for the first time. After some days of looking at the code and getting a general feel on how things work and how the code is written I started programming.

As my first coding task I wanted to do something fairly simple that would introduce me to Nomacs and Qt. I decided to create a panel with a histogram of the displayed picture. The histogram will be useful also when coding tools for image manipulation because we can compare the histogram with other programs. To create the histogram we go through each image pixel and for each channel we count the number of pixels with the same color. From the normalized calculated data we draw the histogram in a DkHistogram widget that is based on DkMetadata and DkInfoLabel widgets as it was kindly suggested by my mentors.

The final result is shown in the image bellow.