Apple Intelligence brings interesting AI features - but there's a catch

After all the hype around GPT chat and AI, there were already plenty of theories about Apple’s reaction. Apple opted for a double solution:

Apple relies on both a partnership with OpenAI and internal development. The most important role will be played by its own AI, called “Apple Intelligence”. AI functions in the form of image and voice processing will be fully integrated into iOS 18, iPadOS and macOS Sequoia (not watchOS) and will significantly improve convenience and productivity.

Apple Intelligence took the lion’s share of the WWDC keynote, with other innovations such as iOS 18 and Vision Pro seeming almost marginalized.

The big but: for now, Apple’s AI only works in English and will only be available in beta starting this fall. Another downside: Apple Intelligence requires an A17 Pro (iPhone 15 Pro/Pro Max only) or an M1 as minimum hardware. This means that Apple Intelligence won’t work on the current iPad 10 either.

If you have a device that can run Apple Intelligence, here’s what it will be able to do.

Personal intelligence

What is special about Apple’s AI concept is the aim of combining AI with personal context: if you ask the new Siri a question, for example, Apple Intelligence can recognize and use the context.

A simple example: you receive an email and the system can create an appointment from the text content. Or you ask Siri the question “When is my mother’s plane landing?” » and Siri creates a response based on previous messages and the current online flight schedule. Or with the command “Play the podcast recommended by Jamie”, Siri will find the desired title.

Siri can now also help you use your Apple device: if you have any questions about operation, for example how to activate dark mode, Siri now provides step-by-step instructions.

Apple

Apple has also adapted the design of Siri, the assistance function now appears more as a background effect and a light effect at the edge of the screen signals the activation of Siri.

“Writing tools” also add many additional functions to applications such as Mail or Notes. You can rewrite text from Apple Intelligence or have it rewritten in a new language style. With a feature called “Proofreading,” you can request editing suggestions with explanations that you can accept or decline.

Find out why our Home Tech editor doesn’t like parts of Apple Intelligence so much

Apple Intelligence can summarize long text for you (“Summarize”) or transcribe a voice recording on demand. This transcription feature is also available for voice recordings in notes and for phone calls. Not only can you record an entire phone call and convert it to text, but you can even receive a summary if you want (participants are automatically notified of the recording).

Developers can also integrate these functions into their applications. However, automatic text recognition allows additional functions. For example, Apple Intelligence can automatically sort emails and messages based on their importance; an email containing an invitation is then displayed at the top of a preview. Instead of showing only the first few lines of a notification message, the system can automatically display the most important part of the text using AI.

IDG

Image recognition – Image Playgrounds

A previously leaked Messages feature is also AI-powered: “Genmoji” is the name of this new feature that adds custom emojis to chats. For example, if you’re sending a birthday message to a friend, the feature can create a suitable image from a portrait photo – in a comic book style if you like.

IDG

With the Image Playground feature, you can create your own images, choosing from a range of themes and also using images from your photo collection as a basis. This feature is also integrated into Pages, Freeform and Keynote.

The “Image Wand” feature is aimed more at Apple Pencil users: if you draw a sketch – like a building – Image Wand generates a detailed drawing of it.

Photo editing is also possible, in Photos you can even better remove objects from photos thanks to AI called “Clean Up” and you can search for a specific photo or video using natural language.

The new Memory Videos function also relies on this image analysis. The keynote showed how to create a complete video slideshow from old photos using a search term like “Fishing” – including background music.

Data protection

To ensure privacy when processing personal data, most of Apple Intelligence’s actions will take place on the device itself – which is why Apple has strict minimum system requirements. However, some requests require the use of a server.

Apple has therefore developed a concept called “Private Cloud Computer” to ensure data protection. The servers, which run on Apple chips, do not store any data and are regularly checked by independent experts.

Apple goes to great lengths to ensure privacy here. But this is probably also necessary, because a new function of Microsoft’s Copilot has caused a lot of problems here.

GPT chat

Rumors were circulating about a cooperation with Google, but Apple has indeed decided to partner with OpenAI.

You can use Chat GPT-4o for free in addition to Apple Intelligence. The AI will be accessible via Siri, but you will be asked for permission before using it – your IP addresses will also be made unrecognizable and OpenAI will not store any of your requests.

Apple

You can also use Chat GTP through the writing tools, for example to produce a bedtime story or invitation using AI. You can also use the “Compose” function to generate images – if you wish, matching what you have written.

Conclusion

The new features aren’t as dramatic as some might have hoped, but should significantly improve Apple’s iPhone in particular. Apple lags behind Google and Microsoft in this regard. However, the high hardware requirements are disappointing and versions in other languages will only be available next year – probably including German.

This article was originally published in our sister publication Macwelt and has been translated and adapted from German.