Many years ago, those who had a webcam on their PC were privileged, but today they are embedded in spaces as small as the edge of the screen of a laptop or smartphone. This was possible because the optics were considerably reduced while the quality of the images captured was improved, and this thanks to a small processor called FAI that all of them incorporate.
What is the ISP (Image Signal Processor) of cameras
The ISP is a small processor built into the camera (in the case of smartphones it can be built into the SoC), and to understand how it works you need to know that pixels are sensitive to light between certain sets of lengths. wave; essentially, they are independent of color. The way to get a color image is to place a filter on top, usually a bay pattern color filter, and then interpolate the color of adjacent pixels. Thus, the CMOS sensor of the camera does not detect red, green and blue for each pixel but rather detects a different color for each of them and it is the FAI processor which guesses the color according to the one to side.
This is called a demonstration, and it is the main job of the ISP in front of the camera. In addition, the FAI takes care of all the general tasks of a camera such as autofocus, exposure or white balance, for example. Recently, the function of correcting lens imperfections such as vignetting or color shading conferred by imperfect lens system has been added, as well as items such as HDR recombination, noise reduction, filters, face detection and conversion between color spaces.
Obviously, this whole last part means that if the camera manufacturer wants, they can give the ISP all the image setup functions that we then use in the software, although they also have the option to make everything automatic. (Apple) or that the options are configured by the user to their liking. This is what allows, for example, to blur the background of your Skype videoconferences.
Why is this processor important?
While smartphones have become the main device for capturing images and videos in people’s daily lives (and recently webcams too, albeit those built into laptops), this series of functions we consider As Basics As autofocus has become more and more important, manufacturers have been forced to improve the ISP and increase the number of tasks it is able to perform.
At present, the user experience and the interface of the camera are changing rapidly from generation to generation and more and more serious functions are integrated in the sense that they require a greater use of resources (like the example blur of the Skype background we put before), so like the controller of an SSD, a camera’s ISP will increasingly play a more important role beyond what optics are capable or megapixels that have the camera.
For video, we also need to add an additional factor to this equation, the encoder. The way it works is very similar, although the FAI normally works with subsamples from the camera’s CMOS sensor depending on the capabilities of the sensor. The encoder takes these images and compresses them into a format and bit rate of the OEM or the user’s choice (this is basically H.264), and while not all encoders are the same, they should work in conjunction with the ISP to be able to record a video.
Likewise, video is recorded at a higher resolution and with effects added in real time, which creates a load on the camera’s processor that not everyone is capable of. In short, as the technology progresses more and more functions are added to cameras and their processor is overloaded, so it is important that their development continues to take place according to the evolution of the industry. , otherwise we will get stuck where we are.
