Apple Intelligence: every new AI feature coming to the iPhone and Mac

Apple announced “Apple Intelligence” at WWDC 2024, its name for a new suite of AI features for the iPhone, Mac, and more. Starting later this year, Apple is rolling out what it says is a more conversational Siri, custom, AI-generated “Genmoji,” and GPT-4o access that lets Siri turn to OpenAI’s chatbot when it can’t handle what you ask it for.

The company that once refused to use the term “artificial intelligence” when talking about its machine learning features is now, as best as we can tell, catching up with just about all of the table-stakes features that Google, Microsoft, and others have rolled out in the last year. The company says the AI features will be available only on the iPhone 15 Pro and 15 Pro Max and iPads or Macs with M1 or later chips, and only when their language is set to English. The features will start rolling out this fall in iOS 18, iPadOS 18, and macOS Sequoia.

Siri gets AI smarts

Apple said its AI features will carry out actions between apps, as well as manage notifications, automatically write things for you, and summarize text in mail and other apps. As with other built-in AI writing tools, you can change the tone of something you’re writing with a prompt. The company also says its AI features can do things like reference one app to carry out an action in another, such as asking it to play a podcast your partner sent you.

When you invoke Siri, Apple says it has “onscreen awareness,” meaning it can understand what you’re asking and “take action with users’ content in more apps over time.” The company says this will work in both Apple and third-party apps, as well. And you’ll be able to type to Siri, not just talk.

Of course, this is Apple, so there’s a privacy story. The company says that many features will work on-device, where that’s possible. But when you ask for something more complicated, your device — whether an iPhone, Mac, or iPad — will make a call to shunt the request to the cloud automatically.

Apple says it’s using what it calls “Private Cloud Compute.” The models that process more complicated requests are servers “powered by Apple Silicon,” and the company says it will never store or be able to access your data on its servers. Also, the company says that it will let independent experts inspect code running on its servers “to verify privacy.”

Apple image generation features

Apple says it’s rolling out “Genmoji,” a new feature that will generate emoji-like reactions on the fly so that when you can’t find the one you want, you can just have it created for you by AI. Similarly, a new image generation feature is coming to iOS 18, called Image Playground. Apple says the feature works in several apps, but it also has its own app by the same name. It will also let developers access an API to include Image Playground on their devices.

Additionally, Apple is bringing AI to the Photos app. You can already search for objects pictured in your library, but Apple says it’s improving how this works in the future, so you can search for things like a person who is doing cartwheels. It also added features similar to Google’s Magic Eraser, like the ability to erase a person from a photo.

Apple’s OpenAI deal is real

ChatGPT, powered by GPT-4o, will come to iOS, macOS, and iPadOS “later this year,” as a chatbot that Siri can turn to when it’s not able to do what you ask. When it does this, Apple says it will ask your permission before it sends questions, documents, or photos to ChatGPT and that, after it responds, Siri will present its answers to you. Apple said it was starting with the best chatbot but that it would support other AI models down the road.

The company also says ChatGPT will be available to use in its systemwide AI writing tools, can be used to generate images, and that it obscures IP addresses. OpenAI won’t store user requests, either, Apple writes. Finally, while you won’t need to create an account or pay for ChatGPT; Apple says subscribers can connect their accounts to use paid features within the built-in integrations. OpenAI published the same information on its website.

Related: