wwdc

Apple’s Voice Control improves accessibility OS-wide on all its devices

Posted by | accessibility, Apple, Gadgets, hardware, iOS, macos, Mobile, Speech Recognition, TC, voice control, wwdc, WWDC 2019 | No Comments

Apple is known for fluid, intuitive user interfaces, but none of that matters if you can’t click, tap, or drag because you don’t have a finger to do so with. For users with disabilities the company is doubling down on voice-based accessibility with the powerful new Voice Control feature on Macs and iOS (and iPadOS) devices.

Many devices already support rich dictation, and of course Apple’s phones and computers have used voice-based commands for years (I remember talking to my Quadra). But this is a big step forward that makes voice controls close to universal — and it all works offline.

The basic idea of Voice Control is that the user has both set commands and context-specific ones. Set commands are things like “Open Garage Band” or “File menu” or “Tap send.” And of course some intelligence has gone into making sure you’re actually saying the command and not writing it, like in that last sentence.

But that doesn’t work when you have an interface that pops up with lots of different buttons, fields, and labels. And even if every button or menu item could be called by name, it might be difficult or time-consuming to speak everything out loud.

To fix this Apple simply attaches a number to every UI item in the foreground, which a user can show by saying “show numbers.” Then they can simply speak the number or modify it with another command, like “tap 22.” You can see a basic workflow below, though of course without the audio cues it loses a bit:

Remember that these numbers may be more easily referenced by someone with little or no vocal ability, and could in fact be selected from using a simpler input like a dial or blow tube. Gaze tracking is good but it has its limitations, and this is a good alternative.

For something like maps, where you could click anywhere, there’s a grid system for selecting where to zoom in or click. Just like Blade Runner! Other gestures like scrolling and dragging are likewise supported.

Dictation has been around for a bit but it’s been improved as well. You can select and replace entire phrases, like “Replace ‘be right back’ with ‘on my way.’ ” Other little improvements will be noted and appreciated by those who use the tool often.

All the voice processing is done offline, which makes it both quick and robust to things like signal problems or use in foreign countries where data might be hard to come by. And the intelligence built into Siri lets it recognize names and context-specific words that may not be part of the base vocabulary. Improved dictation means selecting emoji and adding dictionary items is a breeze.

Right now Voice Control is supported by all native apps, and third party apps that use Apple’s accessibility API should be able to take advantage of it easily. And even if they don’t do it specifically, numbers and grids should still work just fine, since all the OS needs to know are the locations of the UI items. These improvements should appear in accessibility options as soon as a device is updated to iOS 13 or Catalina.

Powered by WPeMatico

Photos on iPhone is about to look completely different

Posted by | Apple, Apple Photos, Gadgets, Mobile, photos, wwdc, WWDC 2019 | No Comments

When you need to find a photo on your iPhone, what do you do? There are tons of ways, but let’s be honest. You probably just go to the camera roll and scroll through at lightning speed, trusting your eyes to pick out the target shot. That may change with Apple’s new layout for Photos, which organizes shots into easy-to-browse days, months, and years.

Right now the Photos app is kind of a mess. There are so many ways your photos are organized that it feels like none is the “main” one. The “For You” tab has a best of the last (insert random duration here) and randomly selected photos from random periods; Search shows me “one year ago,” “spring,” “trips,” and “dining” — great, thanks! And even the chronologically listed “Photos” tab has such tiny images that it’s hard to pick out the ones you want. So we all just go to camera roll and scroll and scroll.

That may change with the Days, Months, Years theme Apple just announced at WWDC. Under the default photos tab in the Photos app, you’ll now see a new set of tabs — yeah, probably too many tabs — with the different durations on it.

The default mode looks a lot like camera roll. But when you hit “Days,” each day shows as a selection of highlighted images, large and mosaiced, and with live photos active. Each day is marked by a full-width shot, so you can easily slide down to the next day — more recent shots are towards the bottom.

Flip over to Months, and it compresses each calendar month of shots into a handful of events, such as events or locations the system has detected. Years does the same thing, except as the “cover” for each year’s album it shows an image from the same day or period — for example, if it’s someone’s birthday party, you’ll see shots from when you (hopefully) attended their party in years past.

Of course Photos already had a “on this day” type feature, but this makes much more sense to me. You can dive into a year and it breaks into months, and of course months break into days. It’s just a more efficient way of scrubbing through your camera roll — though I have no doubt we’ll still do that from time to time.

In a way this is a minor change to iOS, but because users interact with Photos so much, it could significantly change how you think about getting to the images you want.

Powered by WPeMatico

Apple introduces the AI phone

Posted by | AI, Apple, artificial intelligence, iOS, iOS 12, iOS at WWDC 2018, Mobile, siri, wwdc, WWDC 2018 | No Comments

At Apple’s WWDC 2018 — an event some said would be boring this year with its software-only focus and lack of new MacBooks and iPads — the company announced what may be its most important operating system update to date with the introduction of iOS 12. Through a series of Siri enhancements and features, Apple is turning its iPhone into a highly personalized device, powered by its Siri AI.

This “new AI iPhone” — which, to be clear, is your same ol’ iPhone running a new mobile OS — will understand where you are, what you’re doing and what you need to know right then and there.

The question now is will users embrace the usefulness of Siri’s forthcoming smarts, or will they find its sudden insights creepy and invasive?

Siri Suggestions

After the installation of iOS 12, Siri’s Suggestions will be everywhere.

In the same place on the iPhone Search screen where you today see those Siri suggested apps to launch, you’ll begin to see other things Siri thinks you may need to know, too.

For example, Siri may suggest that you:

  • Call your grandma for her birthday.
  • Tell someone you’re running late to the meeting via a text.
  • Start your workout playlist because you’re at the gym.
  • Turn your phone to Do Not Disturb at the movies.

And so on.

These will be useful in some cases, and perhaps annoying in others. (It would be great if you could swipe on the suggestions to further train the system to not show certain ones again. After all, not all your contacts deserve a birthday phone call.)

Siri Suggestions will also appear on the Lock Screen when it thinks it can help you perform an action of some kind. For example, placing your morning coffee order — something you regularly do around a particular time of day — or launching your preferred workout app, because you’ve arrived at the gym.

These suggestions even show up on Apple Watch’s Siri watch face screen.

Apple says the relevance of its suggestions will improve over time, based on how you engage.

If you don’t take an action by tapping on these items, they’ll move down on the watch face’s list of suggestions, for instance.

AI-powered workflows

These improvements to Siri would have been enough for iOS 12, but Apple went even further.

The company also showed off a new app called Siri Shortcuts.

The app is based on technology Apple acquired from Workflow, a clever — if somewhat advanced — task automation app that allows iOS users to combine actions into routines that can be launched with just a tap. Now, thanks to the Siri Shortcuts app, those routines can be launched by voice.

Onstage at the developer event, the app was demoed by Kim Beverett from the Siri Shortcuts team, who showed off a “heading home” shortcut she had built.

When she tells Siri she’s “heading home,” her iPhone simultaneously launched directions for her commute in Apple Maps, set her home thermostat to 70 degrees, turned on her fan, messaged an ETA to her roommate and launched her favorite NPR station.

That’s arguably very cool — and it got a big cheer from the technically minded developer crowd — but it’s most certainly a power user feature. Launching an app to build custom workflows is not something everyday iPhone users will do right off the bat — or in some cases, ever.

Developers to push users to Siri

But even if users hide away this new app in their Apple “junk” folder, or toggle off all the Siri Suggestions in Settings, they won’t be able to entirely escape Siri’s presence in iOS 12 and going forward.

That’s because Apple also launched new developer tools that will allow app creators to build directly into their own apps integrations with Siri.

Developers will update their apps’ code so that every time a user takes a particular action — for example, placing their coffee order, streaming a favorite podcast, starting their evening jog with a running app or anything else — the app will let Siri know. Over time, Siri will learn users’ routines — like, on many weekday mornings, around 8 to 8:30 AM, the user places a particular coffee order through a coffee shop app’s order ahead system.

These will inform those Siri Suggestions that appear all over your iPhone, but developers will also be able to just directly prod the user to add this routine to Siri right in their own apps.

In your favorite apps, you’ll start seeing an “Add to Siri” link or button in various places — like when you perform a particular action — such as looking for your keys in Tile’s app, viewing travel plans in Kayak, ordering groceries with Instacart and so on.

Many people will probably tap this button out of curiosity — after all, most don’t watch and rewatch the WWDC keynote like the tech crowd does.

The “Add to Siri” screen will then pop up, offering a suggestion of voice prompt that can be used as your personalized phase for talking to Siri about this task.

In the coffee ordering example, you might be prompted to try the phrase “coffee time.” In the Kayak example, it could be “travel plans.”

You record this phrase with the big, red record button at the bottom of the screen. When finished, you have a custom Siri shortcut.

You don’t have to use the suggested phrase the developer has written. The screen explains you can make up your own phrase instead.

In addition to being able to “use” apps via Siri voice commands, Siri can also talk back after the initial request.

It can confirm your request has been acted upon — for example, Siri may respond, “OK. Ordering. Your coffee will be ready in 5 minutes,” after you said “Coffee time” or whatever your trigger phrase was.

Or it can tell you if something didn’t work — maybe the restaurant is out of a food item on the order you placed — and help you figure out what to do next (like continue your order in the iOS app).

It can even introduce some personality as it responds. In the demo, Tile’s app jokes back that it hopes your missing keys aren’t “under a couch cushion.”

There are a number of things you could do beyond these limited examples — the App Store has more than 2 million apps whose developers can hook into Siri.

And you don’t have to ask Siri only on your phone — you can talk to Siri on your Apple Watch and HomePod, too.

Yes, this will all rely on developer adoption, but it seems Apple has figured out how to give developers a nudge.

Siri Suggestions are the new Notifications

You see, as Siri’s smart suggestions spin up, traditional notifications will wind down.

In iOS 12, Siri will take note of your behavior around notifications, and then push you to turn off those with which you don’t engage, or move them into a new silent mode Apple calls “Delivered Quietly.” This middle ground for notifications will allow apps to send their updates to the Notification Center, but not the Lock Screen. They also can’t buzz your phone or wrist.

At the same time, iOS 12’s new set of digital well-being features will hide notifications from users at particular times — like when you’ve enabled Do Not Disturb at Bedtime, for example. This mode will not allow notifications to display when you check your phone at night or first thing upon waking.

Combined, these changes will encourage more developers to adopt the Siri integrations, because they’ll be losing a touchpoint with their users as their ability to grab attention through notifications fades.

Machine learning in photos

AI will further infiltrate other parts of the iPhone, too, in iOS 12.

A new “For You” tab in the Photos app will prompt users to share photos taken with other people, thanks to facial recognition and machine learning.  And those people, upon receiving your photos, will then be prompted to share their own back with you.

The tab will also pull out your best photos and feature them, and prompt you to try different lighting and photo effects. A smart search feature will make suggestions and allow you to pull up photos from specific places or events.

Smart or creepy?

Overall, iOS 12’s AI-powered features will make Apple’s devices more personalized to you, but they could also rub some people the wrong way.

Maybe people won’t want their habits noticed by their iPhone, and will find Siri prompts annoying — or, at worst, creepy, because they don’t understand how Siri knows these things about them.

Apple is banking hard on the fact that it’s earned users’ trust through its stance on data privacy over the years.

And while not everyone knows that Siri is does a lot of its processing on your device, not in the cloud, many do seem to understand that Apple doesn’t sell user data to advertisers to make money.

That could help sell this new “AI phone” concept to consumers, and pave the way for more advancements later on.

But on the flip side, if Siri Suggestions become overbearing or get things wrong too often, it could lead users to just switch them off entirely through iOS Settings. And with that, Apple’s big chance to dominate in the AI-powered device market, too.

Powered by WPeMatico

Kayla Itsines’ Sweat app will rake in $77 million this year

Posted by | Apps, fitness, Mobile, sweat, wwdc | No Comments

In earlier years, fitness gurus would market their programs for getting in shape on VHS tapes and, later, DVDs. These days, it’s an app business. At Apple’s Worldwide Developer Conference this week in San Jose, the company brought in one of the fitness app industry’s superstars, Kayla Itsines, co-creator of the BBG (Bikini Body Guides) and the Sweat app — which will pull in $77 million USD this year — to lead a morning workout for around 200 conference attendees.

For Apple, Kayla’s brand represents not only a good App Store success story, but also spreads the message of how its own products, like iPhone and Apple Watch, enable access to better health through their platforms.

From e-books to apps

Kayla’s fitness company was started several years ago by two personal trainers — herself and business partner Tobi Pearce. Both were using social media, including Instagram, to drive leads for their own PT sessions and bootcamps.

But Itsines’ online profile really took off and soon, people from all over the world wanted to know how they could train like her and get the same results.

WWDC Workout with Kayla Itsines

So the trainers packaged up her program materials and sold it as an e-book online starting in early 2014. Over the next year or so, the business grew, as they distributed more e-books and a broader content series.

But Kayla and Tobi wanted to reach even more people, so they turned to the App Store.

“That’s where our customers are,” said Tobi, speaking of the company’s decision to launch a mobile app, in a conversation with TechCrunch backstage at the WWDC fitness event.

“We have mostly millennial consumers — 25 to 35 is our main market,” he continues. “Part of being a personal trainer is that you get to be there — personally — and train people in real time. Now, obviously, you can’t do that for every person in the world, and an e-book can’t do that. But Apple allowed us to do that,” he says. “The Apple ecosystem is kind of a no-brainer.”

WWDC Workout with Kayla Itsines

The team launched the Sweat app in November 2015, but it got a big refresh — almost a full relaunch — early last year, with three to four times the amount of content.

Today, the Sweat app is a one-stop shop for fitness programs for women, featuring not only Kayla’s own content, but other trainers’ programs as well, across areas like yoga, pregnancy and gym workouts, for example.

Subscribers pay $19.99 per month to use the Sweat app, which is cheaper than the gym, or they can opt for an annual membership to save 50 percent.

However, not all of Sweat’s users are turning to the app instead of the gym — can also can be a companion for those who want the assistance of a personal trainer in a gym environment, but don’t want to pay the hundreds of dollars they tend to charge.

And thanks to team Kayla’s social media savvy and the team’s marketing prowess, they’ve built a community that’s happy to pay, it seems.

These days “well over a million” people use the app on a monthly basis, out of 30 million total app downloads, Tobi tells TechCrunch. And though the company’s now 70-person team is largely based in Australia, the U.S. is Sweat’s largest market.

“Since re-launching [the Sweat app], we had a really big growth year — we grew about 86 percent last year, which is pretty huge for us. And this year, we’re on track to hit about $100 million in revenue this year — that’s AUD,” Tobi clarifies.

In U.S. dollars, that’s around $76.75 million — not bad for a fitness app that never took in outside capital.

“When we first started doing the e-books, I had a few bootcamp franchises of my own, and Kayla had a small studio that she ran…I put up most of my own money, initially,” Tobi explains. “It was sort of big turning point in both of our careers because we could — you know: the Australian dream, buy your own home — or we could invest a hundred thousand dollars and hope something comes out of it.”

WWDC Workout with Kayla Itsines

What’s next: Apple TV, AR and… funding?

Part of the Sweat app’s appeal — beyond its promised results, of course — is its use of new technologies to keep people engaged.

The current app leverages Apple Watch’s visual interface to give video cues, and it added audio cues to the iOS app so the trainers can talk to you as you work out — much like an in-person trainer would. (That feature is coming to the Watch soon, so more advanced users who don’t need the videos can just listen through their headphones or AirPods to hear what to do next.)

Sweat also includes its own curated music playlist streamed through Apple Music, and, in the future, the Sweat program is expanding to Apple TV.

Tobi says they have plans to do something with augmented reality as well, but couldn’t offer more details.

“I’m not too sure yet [what we’re doing with AR], I guess we’re kind of curious,” he admits. “It’s almost part of our responsibility and obligation. We’re a market leader for women’s fitness, and if we want to continue to be that, we want to have the best technologies,” he says.

While Sweat isn’t in need of outside investment, the team isn’t ruling out the idea entirely.

“I don’t necessarily think it would be a bad idea. I think, obviously, for all businesses at any stage — whether it be really early on with venture capital, or whether it be a different type of funding later in the life cycle of the business — I think it always serves a purpose, honestly,” says Tobi.

“Now we’re in the cycle of trying to optimize the experience to get the best results for the user — whether it be content or features or whatever. Having a funding partner — not so much necessarily just for the capital, but also for the resources and the network — would be really handy,” he says. 

Powered by WPeMatico

Apple introduces watchOS 5

Posted by | Apple, Apple Watch, Apps, Gadgets, Mobile, watchOS, watchOS at WWDC 2018, wwdc, WWDC 2018 | No Comments

Kevin Lynch from the watchOS team introduced the next version of watchOS at Apple’s WWDC keynote. It’s been a slow and steady rise for the Apple Watch. It’s by far the most popular smart watch, and it’s becoming slightly more useful every year.

This year is no different. There’s a new workout type for yoga, another one for hiking. You can now challenge your friends for a 7-day competition.

But I’m even more excited about automatic workout detection. If you grab your bike and your heart start beating more rapidly, your Apple Watch will track your workout automatically. You’ll also get notifications to end a workout.

As rumored, Apple is introducing a new Walkie-Talkie app for Apple Watch users. You press to record a message, release to send it. Your friend will receive a notification. That could open up interesting professional use cases. Cellular Apple Watches make this feature more useful too.

The Siri watch face is getting more integrations thanks to Siri shortcuts. You can receive a Citymapper suggestion for instance.

When it comes to the actual voice assistant, you won’t need to say “Hey Siri” anymore. You can just raise your wrist and start talking.

Apple has ported WebKit to watchOS, which opens up a lot of possibilities. You can view web content from your watch. Apple is adding native podcast support and background audio on the Apple Watch too.

Overall, Apple tackled a lot of low hanging fruits. But it’s a compelling pitch and makes the Apple Watch more essential than ever.

Powered by WPeMatico

iOS 11’s new ‘Password Autofill for Apps’ won’t work with (or replace) your favorite password manager

Posted by | Apple, Apps, developers, icloud keychain, iOS, ios 11, Mobile, password autofill, password autofill for apps, Passwords, Security, TC, wwdc | No Comments

 The next version of Apple’s iOS operating system, iOS 11, is going to make it easier for you to log into your apps. The new mobile operating system’s software will introduce a new feature called “Password AutoFill for Apps,” which will offer easy access to your passwords right from the iOS keyboard when you’re on an app’s login screen. Read More

Powered by WPeMatico

iOS 11 will help you free up storage on your iPhone through personalized suggestions

Posted by | Apple, ios 11, iPhone, iphone storage, Mobile, Storage, wwdc | No Comments

 Apple is introducing a new feature in iOS 11 that will help users free up space on their iPhone by offering personalized recommendations about actions they can take to increase the available storage. The feature will go a long way towards helping all iPhone owners get better control over their phone’s storage, but will be especially useful for those who own one of Apple’s… Read More

Powered by WPeMatico

Everything Apple announced at its WWDC keynote

Posted by | Apple, Apps, Developer, Gadgets, Mobile, TC, wwdc, wwdc 2017 | No Comments

 WWDC is usually a key event for Apple. This is the company’s developer conference. While Apple has mostly focused on software news in recent years, this one was a bit different. We got a bit of everything — major software updates, new devices as well as a sneak peek at Apple’s roadmap for the coming months.
In case you missed it, here’s everything Apple announced today. Read More

Powered by WPeMatico

Get ready for a new iPad and a mysterious Siri speaker at WWDC

Posted by | Apple, Gadgets, iPad, rumor, Siri speaker, TC, wwdc | No Comments

 According to reliable analyst KGI’s Ming-Chi Kuo and 9to5mac, Apple is about to launch not one but two new hardware devices at its WWDC conference in a few weeks. A new iPad with thinner bezels and a bigger screen is likely. This iPad has been rumored for months. There could also be an Amazon Echo competitor from Apple. As the name suggests, WWDC is a developer conference. For the past… Read More

Powered by WPeMatico

Apple’s WWDC is moving to San Jose

Posted by | Apple, Developer, Gadgets, TC, wwdc | No Comments

wwdc Apple just announced that its Worldwide Developers Conference is going to move back to San Jose after a long run at the Moscone Center in San Francisco. It’s going to happen on June 5-9, 2017 and registrations will open on March 27.
Why is Apple leaving San Francisco for the McEnery Convention Center in San Jose? San Francisco and Apple have changed quite a lot for the past decade. Read More

Powered by WPeMatico