The voice control challenge

Published on 02/20/2023, 606 words, 3 minutes to read

An image of 1girl, pink hair, blue eyes, electric guitar, holding guitar, holding plectrum, guitar strap, hoodie, microphone, singing, hatsune miku

1girl, pink hair, blue eyes, electric guitar, holding guitar, holding plectrum, guitar strap, hoodie, microphone, singing, hatsune miku - Counterfeit

A lot of software these days is difficult to use with accessibility tools. A lot of this problem is not really exposed to the people that create the software. A lot of software seems to be made with the assumption that people have working hands, they can use a mouse, and that the text is legible. This is simply not the case for many people.

I would like to propose a challenge for people. Let's call it the voice control challenge. For this challenge, you need to use voice control on your mac, iPad, or iPhone and see how many of your day-to-day tasks you can do without having to reach for a keyboard or type on the phone manually.

Here are some important commands with voice control that will be useful:

"Show numbers" – this command highlights every UI element that is interactable with a number. Saying the number will interact with that element. You can also use this to create commands like "drag 5 to 6".
"Show names" – this command will give names to every single UI element on the screen. You can use these names as arguments to commands like tap or click or drag. These names will also let you see inner details of how applications work.
"Go to sleep" and "Wake up" – the first command disable voice control. This is useful for things like calls where you were going to say a lot of things that could possibly be interpreted as voice control commands. I also use this when I am talking with my husband. When I am done with the call or talking with my husband, I say "wake up". You can also use device classes as arguments to this command. For example, "go to sleep iPhone".
"Press <key> key" – this will insert the key code for an arbitrary key. I use this a lot when using voice control for messaging apps. "Press enter key" is one of my most used commands.
"New line" or "New paragraph" – this will create either a new line or a new paragraph in the document that you are editing. Note that this will not send messages in most applications.
There are also a variety of text manipulation motions, and overall it feels a lot like Vim. It has the notion of paragraphs, so you can say things like "delete paragraph" and it will delete the entire paragraph of text. You can also say things like "go to end of sentence" or "go to end of line".

Many more commands are documented on iMore.com.

You can enable voice control in the accessibility menus in the settings app. I think it is well worth peoples time to learn how to use this. This will give a new light into how accessible programs really are, and hopefully it will give people the motivation to change things for the better.

It is frustrating that people aren't going to figure out how difficult it is to use a computer when you violate fundamental design assumptions unless they do so. Also, make sure to stay hydrated. Holy crap when you speak a lot more you end up drying out your throat a lot more easily.

You win the challenge if you can do most of what you can normally do without using an input device. You will also quickly find out that electron apps are the bane of your existence. There is a reason that most of my voice control is done with an iPad. It's not just because I'm an apple fan.

Facts and circumstances may have changed since publication. Please contact me before jumping to conclusions if something seems wrong or unclear.

Tags: dictated, voiceControl