This year I decided to apply for GSoC again. The main reason is the great experience I had last year. I really liked working with Nomacs and the Nomacs developers. They were really great mentors, always available to talk and help. This is also why I wanted to work with the Nomacs team again.
When April came I was glad to see that Nomacs team (part of the Computational Science and Engineering at TU Wien organisation) was again selected as a mentoring organisation. I looked at the project ideas they've prepared and one of them completely caught my eyes. It was the plug-in system idea. I decided that I want to do this project and I want anyone else take it from me.
Every year there are thousands of students applying to GSoC. So to get selected ones needs to distinguish himself from the others. I already had the advantage of knowing Nomacs and Qt from last years GSoC so I could really focus to prepare the project so it would best fit Nomacs. I read some books on the topic of Qt plug-ins and then implemented a simple test plug-in in Nomacs to show the feasibility. I also became a Nomacs translator. I've translated it into Slovene and Italian. So before the selection I already did a lot of work to show my commitment to the project.
My proposal with a full description of the project and timeline is accessible at: http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/tjerman/18001
In the weeks to come I'll be writing posts on the development of the plug-in system.
Friday, June 28, 2013
Saturday, August 25, 2012
The End
The Google Summer of Code 2012 came to an end. I would like to thank all Nomacs mentors for this great experience.
Thursday, August 9, 2012
Filtering image noise
Every image we shoot with a digital camera has some noise in it. Depending on the environment where we shoot the noise can be noticeable or unnoticeable. A very big component of the camera noise depends on the ISO speed. Higher the ISO value we select, more sensitive will the camera sensor be and a more noise picture we will get. The noise can be divided to color and luminance noise.
To get a prettier image we need to filter the noise out. In Nomacs RAW Loader we added median filtering of the image because it is a good method to filter noise without ruining the edges in the image. We replace every pixel in the image with the median value of the pixels within a square around the pixel we are replacing. The size of the square is called window size and it has to be an odd number. Because we only want to filter color noise we transform the RGB image to the YCrCb color space where we filter both chroma channels and then we transform back to the RGB space.
The only parameter in the median filter is the window size. Bigger the window size more time will the filtering take and also we can over filter an image. This is why we need to be careful which window size we select for each ISO value. To find them we did a small scientific research.
We put a Nikon D300 on a tripod and shoot a Colorchecker SG card 19 times. Each time we changed the ISO value that ranged from 100 to 6400. For determining the noise of a picture we calculated the peak signal to noise ratio (PSNR) . For the reference we used the image with the lower noise = ISO 100. Bottom graph shows the PSNR value to ISO value dependency. Higher is the PSNR value, lower is the noise.
We see that in general the PSNR value is decreasing with increasing the ISO value. Because the exposure is not the same in every image there is some zigzagging but the general trend is clearly visible. Lets now take a look what happens with PSNR values after median filtering with increasing the window size: 3,5,...,13:
We see that the PSNR value is increasing. But we also see that there is a limit to where the noise can be reduced. For higher ISO values we need higher window size to get there. The best way to decide the window size for each ISO is to take a look at PSNR as a function of window size. Bellow are the graphs for ISO 250 and 3200.
ISO = 250:
ISO = 3200:
We can see that the noise is decreasing with the increase of window size but if the number is too big the noise starts to increase again. For the window size we select the value where the noise stops to improve. The results for the window size are:
ISO > 6400: winSize = 13
ISO >= 3200: winSize = 11
ISO >= 2500: winSize = 9
ISO >= 400: winSize = 7
ISO < 400: winSize = 5
To get a prettier image we need to filter the noise out. In Nomacs RAW Loader we added median filtering of the image because it is a good method to filter noise without ruining the edges in the image. We replace every pixel in the image with the median value of the pixels within a square around the pixel we are replacing. The size of the square is called window size and it has to be an odd number. Because we only want to filter color noise we transform the RGB image to the YCrCb color space where we filter both chroma channels and then we transform back to the RGB space.
The only parameter in the median filter is the window size. Bigger the window size more time will the filtering take and also we can over filter an image. This is why we need to be careful which window size we select for each ISO value. To find them we did a small scientific research.
We put a Nikon D300 on a tripod and shoot a Colorchecker SG card 19 times. Each time we changed the ISO value that ranged from 100 to 6400. For determining the noise of a picture we calculated the peak signal to noise ratio (PSNR) . For the reference we used the image with the lower noise = ISO 100. Bottom graph shows the PSNR value to ISO value dependency. Higher is the PSNR value, lower is the noise.
We see that in general the PSNR value is decreasing with increasing the ISO value. Because the exposure is not the same in every image there is some zigzagging but the general trend is clearly visible. Lets now take a look what happens with PSNR values after median filtering with increasing the window size: 3,5,...,13:
We see that the PSNR value is increasing. But we also see that there is a limit to where the noise can be reduced. For higher ISO values we need higher window size to get there. The best way to decide the window size for each ISO is to take a look at PSNR as a function of window size. Bellow are the graphs for ISO 250 and 3200.
ISO = 250:
We can see that the noise is decreasing with the increase of window size but if the number is too big the noise starts to increase again. For the window size we select the value where the noise stops to improve. The results for the window size are:
ISO > 6400: winSize = 13
ISO >= 3200: winSize = 11
ISO >= 2500: winSize = 9
ISO >= 400: winSize = 7
ISO < 400: winSize = 5
Friday, July 20, 2012
Improvement of Nomacs Raw Loader
The RAW image data we get from digital cameras are the values taken from the cameras sensor. This image is mosaiced: every pixel has just one of the three colors: red, green, blue. The pixels form a pattern that is called color filter array. The most common of them is the Bayer filter which is composed of two green pixels on one diagonal and of one red and one blue pixel on the other diagonal. Bellow picture shows a small part of the Bayer filter.
In order to create an image that represents what a human eye would see we need to process the RAW data. There are a few essential steps that we need to do:
1. Normalization of RAW data: the RAW image needs to be normalized depending on the black point of the used digital camera and its dynamic range.
2. Demosaicing: for each uni-colored pixel we need to find the other two colors of this pixels. This is done by using the interpolation that is based on the type of the color filter array.
3. White balance: The demosaiced image is very green. To remove this effect we multiply each image RGB channel with a constant. The three constants are based on the photographed scene illuminant and they differ for each digital camera.
4. Color correction: The colors of the white balanced picture are still not right. We need to do another transformation that converts colors from the camera color space to a standardized color space such as sRGB or AdobeRGB. This is done by multiplying pixels RGB values with a so called color matrix.
5. Gamma correction: the human visual response to the light is nonlinear. On the other hand the digital camera response is linear. The process of changing the linear data to nonlinear data is called applying gamma correction.
The previous version of Nomacs RAW Loader had a little different sequence of this changes and it was missing one correction: the color correction. After changing this the improvements are clearly visible. Bottom image shows the difference between previous and modified RAW Loader.
So when we postprocess a RAW image we need to be careful that we make all needed corrections. Missing one completely changes how the processed image looks.
In order to create an image that represents what a human eye would see we need to process the RAW data. There are a few essential steps that we need to do:
1. Normalization of RAW data: the RAW image needs to be normalized depending on the black point of the used digital camera and its dynamic range.
2. Demosaicing: for each uni-colored pixel we need to find the other two colors of this pixels. This is done by using the interpolation that is based on the type of the color filter array.
3. White balance: The demosaiced image is very green. To remove this effect we multiply each image RGB channel with a constant. The three constants are based on the photographed scene illuminant and they differ for each digital camera.
4. Color correction: The colors of the white balanced picture are still not right. We need to do another transformation that converts colors from the camera color space to a standardized color space such as sRGB or AdobeRGB. This is done by multiplying pixels RGB values with a so called color matrix.
5. Gamma correction: the human visual response to the light is nonlinear. On the other hand the digital camera response is linear. The process of changing the linear data to nonlinear data is called applying gamma correction.
The previous version of Nomacs RAW Loader had a little different sequence of this changes and it was missing one correction: the color correction. After changing this the improvements are clearly visible. Bottom image shows the difference between previous and modified RAW Loader.
So when we postprocess a RAW image we need to be careful that we make all needed corrections. Missing one completely changes how the processed image looks.
Thursday, July 19, 2012
Hue, exposure, gamma
As promised, we will now take a look at the other three image manipulation tools that lie inside the Nomacs image manipulation dialog: hue, exposure compensation and gamma correction. The range of changes that we can make are:
Hue from -180 to 180. Step size for the bottom picture is 36.
Exposure compensation from -6 to 6. Step size for the bottom picture is 1.2.
Gamma correction from 0.01 to 9.99. The bottom picture shows changes for gamma 0.1, 0.2, 0.4, ..., 1.0, 2.0, 4.0, ..., 8.0, 9.99.
Gamma and exposure changes in the top and bottom slider range are very extreme. If we are changing 24 bit images then we also have a great loss of data. So the values in the extreme zones are not recommended unless if the images are very under or over exposed.
Hue from -180 to 180. Step size for the bottom picture is 36.
Exposure compensation from -6 to 6. Step size for the bottom picture is 1.2.
Gamma correction from 0.01 to 9.99. The bottom picture shows changes for gamma 0.1, 0.2, 0.4, ..., 1.0, 2.0, 4.0, ..., 8.0, 9.99.
Gamma and exposure changes in the top and bottom slider range are very extreme. If we are changing 24 bit images then we also have a great loss of data. So the values in the extreme zones are not recommended unless if the images are very under or over exposed.
Thursday, July 5, 2012
New Image Mnaipulation Tools Dialog design
In the last two weeks the image manipulation tools was completely redesigned. Not only the GUI changed but also the source code changed a lot.
Let's start with the most visible thing. Sliders have a new look which makes the dialog look more professional and appealing. The difference between this and the previous look is noticeable.
To the list of manipulation tools was added the exposure control. I will present it in a separate post together with hue and gamma.
The work on the code in the past couple of weeks was mainly a research project. I was trying to find the best method to use for applying the sliders changes to the image in the main window. Because the changes made by different sliders stack on each other, the process to make them can be slow. Especially for big images.
At first the best way seemed to use 16 bit lookup tables (LUT). We create them very quickly and also we apply them very quickly on images. They are the quickest way of making changes on big images. In RGB space they can also be stacked together. We can then apply the LUT on the main image once. It sounds very promising but there is a problem. In our image manipulation dialog we can change the saturation and hue of an image. For this we need to convert the current picture from RGB to HSV space, change the Hue or Saturation value and convert back to the RGB space. Because the transformation is nonlinear the change can't be done using a lookup table in RGB space. With a simple 16 bit LUT in RGB we can't take into account all possible changes of saturation and hue in HSV space. For this we would need a 16^3 bit LUT which is too big.
The remaining solution was to use LUTs for changes on a local scale (within one slider) and to create a history of changes. When hitting the OK button we go through the history an apply all changes to the main image. This process can be slow because we need to go through image pixels for every change in history. But this is the only way to do it without losing the accuracy of the image data.
The positive side of having a history vector of the changes is that it enabled us to create the undo and redo buttons. We just need to go up or down through the history and recalculate the changes to that point.
Let's start with the most visible thing. Sliders have a new look which makes the dialog look more professional and appealing. The difference between this and the previous look is noticeable.
To the list of manipulation tools was added the exposure control. I will present it in a separate post together with hue and gamma.
The work on the code in the past couple of weeks was mainly a research project. I was trying to find the best method to use for applying the sliders changes to the image in the main window. Because the changes made by different sliders stack on each other, the process to make them can be slow. Especially for big images.
At first the best way seemed to use 16 bit lookup tables (LUT). We create them very quickly and also we apply them very quickly on images. They are the quickest way of making changes on big images. In RGB space they can also be stacked together. We can then apply the LUT on the main image once. It sounds very promising but there is a problem. In our image manipulation dialog we can change the saturation and hue of an image. For this we need to convert the current picture from RGB to HSV space, change the Hue or Saturation value and convert back to the RGB space. Because the transformation is nonlinear the change can't be done using a lookup table in RGB space. With a simple 16 bit LUT in RGB we can't take into account all possible changes of saturation and hue in HSV space. For this we would need a 16^3 bit LUT which is too big.
The remaining solution was to use LUTs for changes on a local scale (within one slider) and to create a history of changes. When hitting the OK button we go through the history an apply all changes to the main image. This process can be slow because we need to go through image pixels for every change in history. But this is the only way to do it without losing the accuracy of the image data.
The positive side of having a history vector of the changes is that it enabled us to create the undo and redo buttons. We just need to go up or down through the history and recalculate the changes to that point.
Friday, June 22, 2012
Image Mnaipulation Tools Dialog
In the previous post I showed the first three image manipulation tools that were added to Nomacs. The tools were placed in a Qt Dock Widget and a change in tools values resulted in a change of the displayed image colors. For changing the colors of an image we need to go through each of its pixels and do a calculation depending on the tools values. As we can see on the graph in the post Image histogram - speed upgrade the time needed for going through all the pixels exponentially increases with the image size. So the manipulation of larger pictures took more time and this resulted in a bad user experience.
The solution was to move the tools from the main window to a dialog with a small picture preview. This way all the changes that user makes are firstly displayed on a scaled image. This is done faster then on an unscaled image. This way the changes on the slider are smooth and the preview is redisplayed right away. Only after the user is satisfied with the preview, the changes are made to the image in the main window.
The current look of the dialog is displayed bellow:
The solution was to move the tools from the main window to a dialog with a small picture preview. This way all the changes that user makes are firstly displayed on a scaled image. This is done faster then on an unscaled image. This way the changes on the slider are smooth and the preview is redisplayed right away. Only after the user is satisfied with the preview, the changes are made to the image in the main window.
The current look of the dialog is displayed bellow:
Subscribe to:
Posts (Atom)