the camera can guide the blind or split the bill in a restaurant

The presentation of the new Snapdragon 8 Elite processor brings, beyond a power never seen before in a smart phonea series of tools and applications specially designed to make life easier for thousands of people in Europe and the rest of the world. They use artificial intelligence to perform everyday tasks that were previously very complex, or even impossible to do on a phone, in just a few seconds; like deleting an element in a video.

All of them take direct inspiration from the second generation of Oryon CPU processors, which the company first introduced just a year ago, and which now reaches the smart phones after having has seen some improvements in key aspects such as performance or battery. The Snapdragon 8 Elite also integrates significant improvements in image processing which are directly inspired by the cameras integrated into the devices.

Both parties, the visual captured by the cameras and the processing power of its new coresare combined with the Hexagon Neural Processing Unit (NPU) to run anything with its dose of artificial intelligence. One of the most important novelties in this regard is the compatibility with multimodal AI capable of recognizing more types of content – such as images – which expands the possibilities and considerably speeds up any type of task.

Do the math

The new multimodal artificial intelligence that Qualcomm is integrating into its new range of processors allows it to incorporate certain object recognition functions and implement them in a context. One of the practical examples that EL ESPAÑOL – El Androide Libre was able to refer to during the presentation event on the island of Maui (Hawaii, United States) was as protagonist, a restaurant ticket.

The built-in artificial intelligence is able to recognize that it is actually a purchase receipt listing a series of dishes and drinks. Just take a photo to start an automatic recognition process it takes no more than a second and opens the door to a very wide range of possibilities.

The user of smart phone You can order the AI ​​to add a certain percentage for the tip to the ticket total and divide the result by the number of guests. In a few seconds, the result is executed and gives the result directly on the screen. In addition, it details by default all the calculations carried out and how we arrive at the final figure.

This ability to recognize and contextualize – knowing that it is a list of dishes and other items and that it fits into a restaurant receipt – is only a first approximation for which will be the most imminent mobile phones. Some models from Samsung, Xiaomi, Honor or Realme have already announced that they will integrate this processor in their most immediate launches and it only remains to be seen how they will implement this series of capabilities.

Returning to the field of applications, this recognition AI could be particularly useful for people with visual difficulties. In another hands-on test conducted by Qualcomm, the device only needs a single photo to analyze the image taken.

Describe objective aspects such as the person’s appearance (the clothes they wear, details on accessories such as bracelets…) and contextualizes it according to the scene in which it is located. In this way, the AI ​​was able to recognize that the subject of the image was at an event or presentation given that an accreditation was hanging around their neck.

As Qualcomm explained during the presentation, the new AI engine integrated into the Hexagon NPU provides 45% more performance for each watt consumed compared to previous versions.

In previous processors, commands, commands and guest that the user gave out loud had to be converted to text so that the AI ​​could understand. A additional work that hindered the user experience delaying the response in time and, sometimes, losing information during transcription.

Now you can get data directly from other types of sources such as audio, sounds, photos, videos, images captured in real time by the phone camera or text. All this content is process directly on the device without the need for an Internet connection.

The best photo of your pet

Another area of ​​application of AI is to get the best snapshot of pets. As Qualcomm explains, the software helps the user choose between 40, 50 or 60 photos taken in each burst. This is achieved by analyzing one by one using artificial intelligence algorithms to identify which of all the animals is in the correct position and is correctly focused by the lens.

In the case shown during the presentation of this technology, a dog jumped to try to catch a ball in the air. Once all the photos have been reviewed independently and you have selected one from all of them, the next step is to improvement of pet hair.

In this step, the generative artificial intelligence incorporated in the processor itself can be used to highlight the details of the fur, which is complex because the snapshot was taken in motion and a significant part of the details can be lost.

Remove intruders

Deleting certain elements from a photograph is a functionality present on certain devices for several generations, mainly supported by the cloud like the tool that integrates Google Photos. With this tool, user can delete objects and even people who do not want them to appear in the final photograph in a simple and intuitive way.

However, the removing these unwanted elements in videos It had become arduous, complex and very demanding work. Basically, the user had to delete this element present in each of the captured images and wait until the time invested had really been effective.

Qualcomm, in its new Snapdragon 8 Elite processors, has integrated a function that does all this in just a few seconds (for short videos) and automatically. With a selection tool, the user only has to Choose the item you want to remove from your footage and press a button.

After some processing time, the result is displayed as a new video with the intruder gone and, in its place, a reconstruction of the background. Taking into account that it is a tool in development, the results obtained during testing can be considered correct.

The function is effective when the selected object does not have a very important role in the video, such as a street light on one side and as long as the background is uniform.

In the attached images, you can see how the AI ​​built into the device – and independent of any internet connection – removed the two torches on the right. The background, in this case, is a completely blue sky, which which plays in favor of technology which in other scenarios yielded poorer results.