- photos/album search now includes video understanding, which imo seems very good from the first 2 examples they showed. includes scroll to exact time of the moment you describe.
- Mail and Notifications will show summaries instead of str[:x]
- Siri now knows iPhone, becomes the ultimate manual on how to use the increasingly complicated iOS 18. and can read your texts (!) to suggest actions with Personal Context Understanding (also it will try to advertise apple tv shows to you... i'm SURE it will be totally objective and aligned to your preferences amirite)
- new iphone 16 camera control button is PRIME real estate - notice how OpenAI/ChatGPT is now next to Google search, and both are secondary clicks to Apple's visual search, which comes first
- camera adds events to calendar!
- "all done on device" and on cloud (though craig doesnt say that haha)
overall i think insanely good ideas on ai + phone integrations.
With the months of nonstop and over-the-top hype of "Apple Intelligence" these all seem underwhelming.
Even in the idealized world of demoware there's no killer feature, not to mention that none of it is even available to real customers yet.
I was expecting something only Apple could do with their vast ecosystem of IP and partnerships, the must-have cool new thing that would stimulate a leap in demand for iPhone.
It's hard to see how any of this is enough to trigger a new upgrade cycle instead of just waiting until battery life goes to shit or the screen cracks so you have to upgrade.
True. All the whining was coming from lazy "industry observers" who never mustered up an example of what, exactly, they expected Apple to be doing with "AI."
Apple was notoriously quiet on AI until they recently revealed Apple Intelligence, it made sense to me since iPhones already have access to all of the user's data, and data is the core of all of the best use cases for AI.
The sources in those articles are user conversation and speculation from Reddit, Twitter, and Discord. There's very little information in that massive article that actually came from Apple.
I think you're seriously underestimating the impact of actually-useful and reliable AI features at your fingertips. This is what everyone imagined computers would be, since I don't know... the 80s?
Sure, but this isn’t one of those AI startups like Humane where you’re expected to drop $800 on a brand new thing. Most people aren’t buying the top end models and they probably aren’t paying full price due to promos, trade-ins, etc.
That means Apple has a much simpler problem: they don’t need to convince you that AI as a concept is worth hundreds of dollars, they just need to convince you that the hundreds of dollars you were already going to spend on a smartphone should go to them instead of Samsung or Xiaomi.
Are you sure ? People do not move in significant numbers from Android anymore, a lot of people just do not want to learn a new interface and workflows and happy with their ecosystem.
Anecdotally I tried to get my dad to a iPhone 14 and shift from Android, he basically kept using his Android and never switched,so my mom is using that device now(along with an Android) not too happily just cause no else one is using it and will switch back to Android fully next upgrade. There are plenty of people who simply prefer the Android interface or by now so used to it that don't want to change and vice-versa. I myself use and need both and cannot imagine exclusively committing to one ecosystem.
> photos/album search now includes video understanding, which imo seems very good from the first 2 examples they showed. includes scroll to exact time of the moment you describe.
This is a killer feature. Everyone can understand the usefulness of just describing the video moment you want to find. Digging through years worth of photos and videos looking for that one specific moment can be a huge chore when you want to show it to someone.
You know when you go to a restaurant and the menu is like 3 different cuisines, it's 12 pages, and each page has 20+ items?
The Intelligence section of the presentation felt a bit like that.
Generally, I was surprised how lightly Intelligence featured into the keynote: it wasn't a standalone talking point, nothing on the Apple site talks to it directly. It's conspicuous, given that they've dedicated hardware real estate to it ... and the hype cycle.
I go to a restaurant and look at the menu/see a commercial on TV, I can usually order the item now (other than films).
Apple Intelligence - certain feature(s) (can't recall now) are US-only in November, rolling out to other English-speaking countries in December. No idea when non English-speaking countries will get that feature.
Who knows what Apple Intelligence features are in the device on day 1?
I thought you were going to say the iPhone would be able to load the QR code for that menu, summarize it, sort it by ingredient or calories and give you a recommendation without you having to read it... Now I'm kinda bummed out.
Apple rarely ever innovates, so it's not surprising these new features seem underwhelming. What Apple does best is to perfect (others') existing ideas so that they work more much more reliably than the competition. The number one pain point of technology today is that hardly any of it consistently works correctly more than 95% of the time. But when Apple releases a feature, it works 99% of the time instead. That's their key differentiator.
The action button being prime real estate I think shouldn't be understated.
It's a button that'll be in the hands and pockets of >1 billion people soon.
Each press allows these billion people to connect with a supplier of Intelligence. Whether it's ChatGPT, Google's search, Apple AI or whatever supplier. Apple controls the hardware.
Just think of Apple's contract with Google that nets them $20 billion a year in profit. For comparison, Walmart, Amazon, NVIDIA, Volkswagen, VISA, all had about 20 billion in net income in 2023.
Or think of the appstore as an analogy. Apple controls the gateway between consumer and supplier, and gains from it. This action button will be no different.
1) i am as ready to shit on apple and siri as the next guy, but imo the HN cynicism is overbearing here. apple isnt building to impress internet neckbeards like us. they are building for the normies. features they understand intuitively and cannot fuck up. maybe wait til it gets in the normies hands to judge. ~none of us here know what its like to work on a hardware/software platform for >1 billion? people.
2) i actually do think dedicated visual intelligence button IS a killer feature. suddenly phone is ai's view into the world. yes probably v1 today will disappoint. but 5 years from now the kids will laugh when we say we had to take photos and upload them to an app back in our day. or manually enter in any information from the real world into our calendars and emails and texts.
As a "tech person" I feel like we've all been told for 20+ years that we can't have UX that makes any sense to us, or lets us do simple things we want to do, because we are power users and software has to cater to some imagined demographic of incredibly dumb people who can't understand basic computing metaphors. It can get frustrating.
I'm a Growth Engineer. What that actually means is "make the UI so incredibly simple people who are functionally illiterate somehow make their way to the place you want them to be so you can eventually monetize them".
I view users moving through interfaces in a mass abstracted form like electrical or hydrological currents. My job is to widen the pipes.
Well, cynically that fits with my own experience. :) But I don't know if I'd say it's simpler, just that you don't have any choices. For me Apple UX often feels complicated and I can't figure out how to do what I want, or figure out what is going to happen when I accept some dialog they want me to accept. But I guess I end up going through the pipe eventually, out of frustration.
My impression of my non-tech friends is that they aren't too dumb to follow instructions, they just don't enjoy operating a computer like I do, so they are more okay with their choices being restricted if they don't have to waste time doing something they find boring.
Shortcuts.app exists, you can even tie those scripts to a hardware button now or ask Siri to execute them or place them as app-like icons or as interactive widgets on your home screen or Lock Screen, or tie them to automation events (schedules, but even more fine grained like "when I get a message from this person, send them an email", or "when I get home, log the current weather to a new line in a table on a spreadsheet". A script gallery even exists where scripts can easily be shared and ran by anyone or shared across iMessages.
Not everything is available like on a rooted Android of course, but I can at least create a Shortcut script and give it to any iOS user regardless of their technical expertise and they can utilize it, which is leagues ahead of the situation amongst non-rooted Android and non-technical Android users.
Agreed, and Apple's differentiator has always been seamless integration. With the iPhone, they own the boundary layer between the physical and digital worlds. It's an obvious place to introduce AI for daily tasks without requiring any behavioral changes from the users.
> apple isnt building to impress internet neckbeards like us. they are building for the normies. features they understand intuitively and cannot fuck up. maybe wait til it gets in the normies hands to judge.
Apple has lost its way, just look at how the "normies" and the "neckbeards" responded to the Vision Pro. Apple doesn't know how to innovate anymore, just how to market, and mass produce things, which is honestly good enough to keep making money for now.
> Apple has lost its way, just look at how the "normies" and the "neckbeards" responded to the Vision Pro.
Such people are likely confused. The AVP makes sense as a high end POC and dev kit. It takes a minute for a new modality of computing to make sense to people, even for Apple's own devs to figure out how people can use it, to iterate the OS and default app suite for a few years.
you’re suggesting apple planned to sell a $3.5k headset to normies? That doesn’t make sense. Sure, many normies had an opinion on it. Most normies never tried it so i don’t care what they think. It’s like asking a normie 20 years ago about what driving an electric car is like.
Vision pro was an over-hyped dev kit that offers a taste of the future. That’s it.
They have classes full of "normies" trying them out every time I pass by my local store. Sure looks like its meant to be something other than a dev kit to me.
Asking when your moms flight gets in, to pull up the picture of you and your partner on vacation at the beach or to take a picture of a restaurant from afar at 5x zoom and get reviews is insanely nice.
What? How could you possibly say that? They can take a picture of something and then auto make a note, or a reminder, or an email, or find out what it is.
Normies struggle with the nuance of devices. They understand what they want, they don't understand steps, like start here then do this, then go here and copy this, then go there. This SKIPS all the steps. This is huge for them, but (probably) just mildly convenient for others.
It remains interesting to me how much we have been able to do, already, with technology. But discoverability, walled gardens, and <hand waving>marketing/positioning/hype</hand waving> have obscured awareness/adoption.
Semantic search was always going to be the killer feature of this AI boom. But it’s just a cherry on top of existing software and requires training users to expand what they think is possible.
As far as I can tell, AI is one of several features that have actively made search worse. There are a handful of novel things it does well enough, but I'd rather engage those in their own LLM sandbox than have the facilities formerly available in a searchbox replaced with a guessbox.
What kind of search has it made worse? I’m not talking about Google search. I’m talking about searching your personal collection of emails or photos or text messages.
Technically it's an iOS 18 feature rather than a phone feature, except the only previous phone model with the hardware for all of it is the iPhone 15 Pro, so it doesn't really feel like an iOS 18 feature
Like what? People have been whining "where's the innovation" for years about phones, totally ignoring the fact that this is a mature product category.
Where's the innovation in word processors?
At this point, we're often talking about regression. Apple removed the audio output from its best-selling music player. It removed TouchID. It removed the SIM slot.
You know what would be "innovative?" Making a goddamned phone that can withstand being used for its primary purpose. The failure of the iPhone's design has been confirmed by millions upon millions of people, who have concluded that they must bury their "thin, elegant" phones in bulky, tacky cases. You have to shake your head at Apple's breathless announcement of NEW COLORS!!!! when the vast majority of them will never be seen.
Then there's the fact that Apple ignores an entire market: men who don't wear cargo pants all day every day. Maybe some of us don't want to look like a schlub with a TV tray jammed into his pocket (again, made even worse with a dumb case).
I mean ‘could do’ is subjective. The ROI that apple receives on R&D is substantial, so on that basis, include a portable projector in phones such that I can replace my tv with my phone - not being able to use my phone while watching tv wouldn’t be nice, so add anti-shake gyroscopes such that as long as my phone is pointing in vaguely the correct direction, the projection stays stable.
Possible with current tech? Hell no. But within the realms of reality for handhelds to do this in the next 15 years? I think so. Probably less valuable over health and other potential device functions, I don’t think about these things and so I wouldn’t know.
Thus, we are not at a point where portable computers should be considered ‘mature’ from an innovation standpoint, even if current innovation is slow.
I'm on iOS 18.0 on my iPhone 15 Pro, rather that the 18.1 with the "AI". Recently had an eye doctor appt for one of my children and they scheduled a follow-up, giving me a little card with the details. I was in a hurry so took a picture of it intending to enter the calendar event later, and it immediately pulled up a fully-filled in, completely accurate new calendar event. Little affordance that is just so nice.
Indeed, in general the camera is getting...crazy. We have a pair of Maine Coon kittens and every picture is tagged with Maine Coon, despite the cats barely even coming into the breed's features yet. Constantly amazed at the stuff the built in camera is identifying.
I have a hilarious list of animals that my cat has been misclassified as by my iPhone. Everything from a beetle to a nematode. It doesn't give me a lot of confidence in these features!
As opposed to Adobe [R] Photoshop [R] Lightroom [TM]* Classic's person identification feature which has identified my dog as no fewer than 5 different members of my family, ranging from ~5 years old to > 90 years old.
on device text recognition, and on device text pattern recognition allows you to call phone numbers, select dates, copy text, copy qr codes, etc.
but few times, there are instances where it doesn't recognize the date and time. I guess this is now a supported feature, instead of accidental feature.
Yeah I took a photo of a weird moth in my house the other day, and accidentally scrolled up the next day to realise that the iPhone had categorised it as the exact species - a Poplar Hawk Moth. I was stunned. Didn’t even have to google for 5 minutes, the camera already knew.
They have their big generative AI push with 18.1, broadly called Apple Intelligence, but they've been building so many AI inference features for a few iterations now, with the camera and Photos getting loads of those additions. Object detection, subject identification, text extraction, etc, though this was the first time I've taken a picture of an appointment card and had it automate extraction like this.
I feel like loads of these features just quietly appear to be randomly discovered by users.
So it's AI, but not the big hype stuff that gets most of the attention.
I have this sinking suspicion that the AI segment today was so quick, so high level, and so future-tense ("Siri will be able to..."), identical to the similar segment at WWDC; I have to wonder how many of these features we'll actually see. I'm getting vibes like the old "Facetime will be an open protocol" promise; its like someone is making them do this, their soul isn't in it, and for that reason they still don't have one narrative for why I should care. Instead, they have a quickfire list of twenty features, the titles of twenty epics from their jira.
The iPhone 16's camera button will gain the ability to soft-press to lock focus "later this year". Their software teams are so incredibly far behind, the new phone has literally one new feature, a feature medically prescribed to induce drowsiness, yet they can't even get the software for it finished for release.
Apple will be one of the least impacted once the market corrects to this reality, because they can push off the cost of the inference silicon to customers for many many queries (especially given 3-4 years, as models become more efficient and Apple Silicon becomes more powerful).
EU ruled that Apple cannot link components. This suggests that Apple would need to provide API’s to allow competitors to be listed alongside the “Apple AI”, or any device to enable screen remote control (ala remote iPhone control).
If you’re a dev, you might appreciate why that is an absolute PIA from a software perspective. Since they want to bring these features to market ASAP, launching now and foregoing the EU is a sensible step.
Huxley sends his regards, really.
I think they say French (the language), but that might be for people outside the EU. German is notably missing from their list, though it’s a bigger market for Apple than French.
Privacy concerns? They're just trying to bully the EU. There's no legal or technical reason iOS 18's remote access features wouldn't work in EU, but alas, they do not.
> no legal or technical reason iOS 18's remote access features wouldn't work in EU
How do you know that? (How much of your net worth are you willing to bet on it?)
Between GDPR, the DMA and now the AI Act, there is a lot of unique regulatory cross section that Apple has in the EU. It would be surprising if there weren't additional legal checks it had to do, internally and on its suppliers, before launching a product or feature in the EU.
Because the same exact tech, remote desktop access on macOS, has been working fine for years. That's how I know this has nothing to do with DMA, GDPR and the AI act. Yes, there might be AI features that are gatekept due to DMA and GDPR, cool, that is kind of understandable. But they also threw a hissy fit and excluded the EU from having lots of unrelated functionality. Or, maybe, they're really doing something horribly wrong if they can't be in compliance with GDPR when sharing a screen of your phone on your macbook.
DMA allows EU regulators to designate certain platforms as gatekeepers. Platforms that are designated gatekeepers are subject to different requirements under EU law than those that are not designated gatekeepers. The EU designated iOS as a gatekeeper[1], but did not do the same for MacOS.
So while you are correct that there is no technical barrier, you are incorrect that they are the same under EU regulation. Involving EU regulators in product design for designated gatekeepers was the entire point of the law, so this is the desired effect. Regulators want the flexibility to e.g. work with the gatekeeper to design feature modifications, like the default browser selection screen[2], for other aspects of designated gatekeeper platforms.
One can choose to see this as good or bad, but this is clearly the intent of the law.
> the same exact tech, remote desktop access on macOS, has been working fine for years. That's how I know this has nothing to do with DMA
Apple just got dinged under the DMA [1][2]. The text of the law may not have changed, but the reality of its meaning has.
> they're really doing something horribly wrong if they can't be in compliance with GDPR
The point is verifying you are in compliance with a law has costs. Plenty of start-ups, for instance, are better off geoblocking jurisdictions they don't have the resources to comply with but don't want to accidentally do something illegal in. Not because they think they're violating anything. But because it isn't worth losing (a) nimbleness over or (b) future access to. (Common ones being the EU, China and India.) Apple isn't a start-up. But they probably don't want their design team to be half staffed with lawyers either.
The document you are referencing has no mention of remote access to a phone. The fact that apple chooses to gatekeep unrelated features and seemingly cites EU regulation to explain this behavior is just absurd. And the amount of people coming to their defense is equally absurd. Have you even understood my main gripe? If Apple feels it is unable to comply with EU regulations with apple intelligence, I can understand that given the amount of raw data that is required to deliver the current generation of AI products. But to then turn around and cite the same regulation when it comes to features that are completely unrelated to data gobbling is mindblowingly stupid, and to me, definitely feels like a tactic to gaslight the consumers in the EU, to try and turn them against this legislation.
I don't even think it's a question anymore, they opted out of new features on non-European handsets. Gemini is probably already on Nokia phones for all we know.
The part I am most interested in is how much of it is on device and how much of your data is trained only on your own model vs hovered up into a larger model.
> Mail and Notifications will show summaries instead of str[:x]
While it's true that the first x characters of an email don't always have the most important bits, they're never _wrong_. It's always exactly what's in the email.
I'm worried about cases like this [0], where a summary is inaccurate or misleading. It may not happen often, but it's possible, so I have to be a little skeptical of every summary. Though I can see where AI summaries are a plus, the move from 100% accurate to <100% feels like a big downgrade.
I predicted those features weeks ago [1], but I still feel they could do so much more. If anything, their implementation feels like a rushed afterthought.
In their demos, they use the action button to capture an ambient song for Shazam, the power button to capture a voice command for Siri, and the camera button for an image for Visual Intelligence. All 3 captures should be performed using the same button.
And screenshots still require pressing 2 buttons simultaneously. Unless you want to share your screen with Siri in which case it's the power button...
People are going to use this button as a voice recorder, and Apple will announce native support next year.
I quite like AppleTV's content choices, but I am not looking forward to getting shilled on it. Again. In between the other spam that they don't block. Ahhhhhg!
("Get an Android?" Yeah, I hear you, I switch ecosystems in anger with every phone, which is why I am so keenly aware of how badly Apple's anti-spam sucks -- but also why I am not over the moon at the prospect of my next switch: I know where the Android skeletons are buried, although hope springs eternal.)
i undersatnd i might come across as an apple shill/fanboi here but just pointing out that its fairly easy to punish any abusive app by killing their notifs and apple does sometimes make it easy to do that by letting u go straight to settings to kill the notif (i'm not conscious of when it does that but i think it does?)
No I can’t. Here’s an example: my security system app sends me push notifications when there is motion detected at times and places where there shouldn’t be motion. I need these critical (“time sensitive” in Apple-ese) notifications. It also helpfully sends me random “SUBSCRIBE TO SOME ADD-ON SERVICE, 50% OFF SALE THIS WEEK” at all hours of the day and night. Sure I could rip out my entire security system and replace it but that’s a lot of physical labor and wasted hardware.
Another example is my air filter. It’s nice to get a reminder when it’s time to change the filter since it’s so infrequent, but I can’t get that notification without also getting spammed with “news” about new machines I should buy. Apple actively enables this behavior while claiming that spam is against their terms of service.
They are clearly working for the spammers, not the user, because they profit off of their cut.
This can always be turned off using in-app settings. It's an App Store requirement. Note: in-app settings, not general settings for the app. Check. It's there.
> Make sure people can manage their notification settings within your app. In addition to requesting permission to send informational or marketing notifications, you must also provide an in-app settings screen that lets people change their choice. For guidance, see Settings.
The options are “On” or “Off”. Please read the examples I provided for why that isn’t good enough.
I’ll even leave you with a third example: I can’t get shipping notifications about an online order I made without also opting into spam about sales and discounts. Apple could take a pro-user stance here and help us push back against scummy behavior, but they choose not to be helpful. They also happen to profit off of it, I’m sure that’s unrelated.
By “choose not to be helpful”, do you mean that they don’t use AI to detect spam notifications and distinguish them from useful notifications? That seems like quite a difficult engineering problem, though I’m sure their new AI could help.
There are many tools at their disposal, of which AI may be only a small one.
1) They could implement a “promotional” notification category the same way that they do “time sensitive” and let the user customize that the same way, or at least offer an opt-out.
2) Add a “report” button next to the disable notifications button.
3) Stop delivering notifications on behalf of apps that don’t correctly categorize promotional notifications.
4) Skip all this complexity and just ban developers that send spam. This is what their monopoly on app distribution is supposedly justified by: protecting users from malicious app developers.
As someone whose mother tongue isn't English, and whose residence isn't the US, apple intelligence is basically useless to me. Like every other ai assistant gimmick out there.
Honest question: why though? If someone uses a language other than English on their phone, the feature is quite literally useless to them, isn't it? I don't get why they're "self centered" for saying so.
I think it's quite telling that one of the main new features of the new iPhones won't even be available until next year for most of the world.
because obviously its coming and its not a huge technical lift at all. transformers were born to do translation. its just impatient and a waste of space complaining about how we know the world works for already very good if unfair reasons.
The state of the camera app pisses me off. It could do so much but it does so little. Why can’t I snap a picture of a written WiFi password to connect to WiFi? QRs are so 2008. Why can’t it fully parse a menu and show me what each dish might look like? It’s mind-boggling to me that with thousands of engineers on staff they produce so relatively little. Thousands!
> Why can’t I snap a picture of a written WiFi password to connect to WiFi?
You can actually do that, as of last year's iOS release. It's sort of a pain, but it's doable.
You need to be at the point where it's asking you for the WiFi password, tap the text field, choose "AutoFill", choose "scan text", and use the camera interface to pick the text.
This is extremely undiscoverable, no argument, but it is at least a system-wide standard for how to scan in arbitrary text when you have a text input. (They've picked "AutoFill" as the brand-name for all their get-text-from-somewhere-else features, so it also contains standard password-filling from apps.)
Bafflingly underwhelming additions from the previous innovation leader who had a last mover advantage in the space.
Think we need to be holding Apple to a higher standard, what they did in spaces before with the OG iPhone, iPad, iPod was completely upend the industry and change the game through focusing a product on a few pain points of the existing market. Yet Apple Intelligence feels like a brief struggling to find a reason to be there.
notes:
- photos/album search now includes video understanding, which imo seems very good from the first 2 examples they showed. includes scroll to exact time of the moment you describe.
- Mail and Notifications will show summaries instead of str[:x]
- Siri now knows iPhone, becomes the ultimate manual on how to use the increasingly complicated iOS 18. and can read your texts (!) to suggest actions with Personal Context Understanding (also it will try to advertise apple tv shows to you... i'm SURE it will be totally objective and aligned to your preferences amirite)
- new iphone 16 camera control button is PRIME real estate - notice how OpenAI/ChatGPT is now next to Google search, and both are secondary clicks to Apple's visual search, which comes first
- camera adds events to calendar!
- "all done on device" and on cloud (though craig doesnt say that haha)
overall i think insanely good ideas on ai + phone integrations.