Apple has officially announced Apple Intelligence during its WWDC 2024 keynote. Apple Intelligence is the company’s umbrella term for new generative AI features baked across the board for iOS 18, iPadOS 18, and macOS Sequoia.
Apple Intelligence can generate and summarize text with a new system-wide feature called Writing Tools. Writing Tools is available everywhere users can write and edit text in iOS 18, iPadOS 18, and macOS Sequoia including built-in apps like Mail and Notes and third-party ones.
Writing Tools empower users to rewrite, proofread, and summarize pieces of text. Writing Tools features a tone adjustment feature that intelligently rephrases input text to sound more professional, relaxed, or concise. Proofread can check for and correct grammar, word choice, and sentence structure while explaining editing suggestions. Summaries of different kinds can be quickly generated and users can opt for a summary paragraph, bullet points, a table, or a list.
The Mail app gets a massive boost thanks to Apple Intelligence. Priority Messages is a new section that floats the most urgent emails to the top of the inbox. Email previews now show a quick summary recap of the mail’s content instead of the first few lines. Apple Intelligence can also summarize key points across long email threads for a concise recap of important information. To help users craft an appropriate response, Smart Reply intelligently generates a response with key details included.
Apple Intelligence can understand and infer notifications to ensure the most important ones are present front and center. Priority Notifications can scan users’ incoming notifications, automatically presenting the most urgent ones at the top of the stack. Summarized notifications scan key details of long notifications thread for a quick recap of the most important details in a group chat or other stacks. A new Focus mode, Reduce Interruptions, blocks all but the most urgent notifications that require immediate attention.
Audio transcription takes advantage of Apple Intelligence, now present in more apps like Notes and Phone. Users can directly record, transcribe, and summarize audio notes right from the Notes app. Phone calls can now be recorded from the Phone app, and all parties within a call are explicitly notified when a recording is initiated. Phone recordings can be transcribed and summarized into key points.
Image Playground is a new system-wide feature for generating images on the spot. Harnessing the power of Apple Intelligence, Image Playground can create unique images and illustrations in seconds based on a user’s prompt description. Image Playground is available as a standalone app and inside other apps like Messages, generating images entirely on-device in three styles: Animation, Illustration, or Sketch.
Image Playground supports limitless concepts, themes, and categories, and users can type a description to define an image the exact way they desire. In Messages, Image Playground can create new images based on users’ friends from the Photos app and the current conversation context. Image Playground can also create Genmojis, which are custom emojis based on users’ descriptions.
Image Playground also powers the Image Wand in the Notes app, a creative new tool for the Apple Pencil. Rough sketches can be transformed into new images, and Image Wand can create new illustrations in empty spaces.
Apple Intelligence powers a new natural language search feature in the Photos app, empowering users to search for specific images with detailed written descriptions. Search now extends to video, allowing users to search for exact video frames and relevant segments. The new Clean Up tool can remove unwanted objects in images with a single swipe all while keeping the subject untouched.
Siri is smarter and more capable than ever thanks to Apple Intelligence. Thanks to richer and deeper natural language understanding, Siri is more contextually aware and more personal, empowering users to perform specific tasks with just their voice. Siri can follow along and understand users even if they stutter and stumble during their voice requests, in addition to remembering and saving context from prior conversations. Type to Siri allows users to interact with the assistant via text, with a smooth transition between written and voice requests.
Siri now includes device support, allowing users to ask how to perform certain functions like how to schedule an email or toggle settings on their devices. Siri also has a new interface when activated with a glowing light effect surrounding the display’s edge. Siri’s contextual awareness is proactive, meaning that Siri can understand the written context on the display even before users ask Siri to take action.
Apple Intelligence is built with privacy in mind. While most of it features run on-device thanks to local models, more complex tasks are processed in the cloud in a secure manner thanks to a new Private Cloud Compute architecture. Private Cloud Compute runs on dedicated servers built on Apple silicon that run larger and more complex AI models privately and securely thanks to hardware components like the Secure Enclave. User data processed by Apple Intelligence and Private Cloud Compute is never stored or exposed to anyone.
In addition to Apple Intelligence, ChatGPT is baked right into iOS 18, iPadOS 18, and macOS Sequoia for additional flexibility. ChatGPT powers features like images and documents understanding right with the system. ChatGPT also works with Siri, where users can explicitly request their tasks be handled by ChatGPT instead of Siri if they desire. ChatGPT is closely integrated with Writing Tools for features like text and image generation through Compose.
ChatGPT integration will be powered by the GPT-4o model completely free of charge for Apple devices. OpenAI subscribers will be able to link their paid accounts to access extra features. On the privacy front, ChatGPT requests through Apple Intelligence are not stored by OpenAI, and users’ IP addresses are hidden.
Apple Intelligence will launch in beta later this year and is completely free for iOS 18, iPadOS 18, and macOS Sequoia users. Available for devices with language set to U.S. English, Apple Intelligence will run on the iPhone 15 Pro, iPhone 15 Pro Max, and iPad and Mac devices with the M1 chip and later.
