Apple's New Home AI Brings Video Descriptions and More for Home Security

Apple's New Home AI Brings Video Descriptions and More for Home Security

Siri AI and Apple Intelligence took center stage at Apple's WWDC 2206 event Monday, and I noticed some especially interesting features for home security. While Apple's rumored push into the smart home will have to wait until fall (fingers crossed), we got a look at major software changes, including video descriptions via Apple Intelligence in the Apple Home app. 

If you've never seen video descriptions before, they use an algorithm to analyze video footage and put it into text, so you get an alert about what a camera "sees" without needing to watch the video yourself. That saves a lot of time when it comes to home security clips. For example, Apple showed off Siri notifying a phone that "Robert arrived with a fruit bowl." You'll also be able to search for specific objects or terms in your history of video clips.

An Apple Home notification on a phone showing a description of video.

Apple's new video descriptions are no surprise, but it's approach is new.

Apple/Screenshot by CNET

I've seen this feature pop up across many brands, including Google Nest (note that Apple partnered with Google Gemini for some of Siri's AI prowess here), Arlo, Ring, Blink and others. What makes Apple's version different is that Siri's Visual Intelligence appears able to pull data from any camera supported by Apple Home instead of being bound to just one security brand. 

I like this option a lot, but the trick is finding cameras that work on Apple Home, which are currently few and far between, as the latest Matter standard updates have been slow to add camera support. That's starting to change, and this year could be a key turning point with Apple's new features. Apple wouldn't announce abilities like this if it didn't have some cams in mind -- its Gemini partnership could indicate that Nest cameras will be on the list.

Notifications from Apple Home on phone showing multiple events.

Apple Home notifications are getting smarter, but I'm waiting to see what devices they work with. 

Apple/Screenshot by CNET

Two additional important notes about these new Apple Home features. First, I don't know if these video descriptions and searches will cost extra or not, such as a higher tier of iCloud. Apple didn't mention any costs when it announced these options. Second, Apple Home can also combine related activity notifications and video descriptions, so you can get a single alert about people spotted, doors unlocked and so on, saving even more time. 

Finally, it's worth noting that these alerts are identifying people by name. That suggests Visual Intelligence is tapping into some kind of facial recognition data, which is a tricky subject, considering Ring was just sued for its similar Familiar Face technology

Apple indicated that these Siri tools will roll out this year and will be available for beta testing soon. When I get a chance to test them with compatible cameras, I'll let you know how they work and how they may affect our privacy. 

Sponsored
Sponsored
Upgrade to Pro
Choose the Plan That's Right for You
Sponsored
Sponsored
Ads
Read More
Download the Telestraw App!
Download on the App Store Get it on Google Play
×