Cursorless is alien magic from the future
Published on , 1396 words, 6 minutes to read
Just in time for me to start a new job at a new place, my RSI has decided to flare up.
For the record, I'm fine, I've known this has been coming for a while. The flare-up is on its exit anyways. I've gotten lucky and I'm going to be fine. But it's still a bit of a bummer.
The last time this happened, I was able to get by by doing mostly writing about technology things, but I think I'm going to need to be able to program again. I know I'm a bit of an emacs user, but for this I've been using visual studio code because of one extension in particular: Cursorless.
Cursorless is a plugin that integrates with voice control software to let you do AST level code editing with your voice. This is crazy alien magic from the future.
I've talked about cursorless before on my blog, but I have decided to really get deep into it this time around. The last time I used it, I didn't actually use it for much more than moving around the screen, but this time I'm going to try to use it for everything.
I wish I had this as an input method for slack and discord messages.
The most magic parts about this are the ideas of destinations and targets when it comes to cursorless inputs. Targets are individual anchors in a document and destinations are places relative to individual targets. Every single token in a document is given a hat over a letter with a color. These hats act as anchors that let you give commands based off of locations, destinations, and paths between them. Here's a simple example. Consider this code:
function fetchBlog() {
fetch("https://xeiaso.net/blog.json")
.then((response) => {
if (!response.ok) {
throw new Error("Network response was not ok");
}
return response.json();
})
.then((data) => console.log(data))
.catch((error) => console.error("Error:", error));
}
This is a fairly standard looking JavaScript function. But, cursorless puts a bunch of hats over all of the code so it may look something like this:
So that's why your editor is full of random video artifacts?
They're not artifacts, they're targets!
Take a good look at that picture again:
The hats are color coded above individual letters. The position tells you the name and the color tells you how to disambiguate it. For example, That word function would be referred to as green urge because the hat is green over the letter u. If I wanted to delete that word for some reason or if I wanted to move it somewhere else, I could use green urge as the target for that action.
By itself, this gives you some pretty powerful actions and effectively lets you do spoken vim motions. But, that is only thinking in terms of simple actions that you can do with your editor. The real power of cursorless comes in from not only the idea of paths (such as green urge past green bat to select the function fetchBlog in that screenshot), but the fact that cursorless knows what the AST of the language is doing. This means that you can do things across the entire function, like deleting it or moving it somewhere else. As an example, here are the lambdas of this function visualized separately (with the "visualize lambdas" command):
These AST units are also targets. This means that I can do things like select the body of a definition and then work off of that. So if I want to refactor this into an asynchronous function, the refactoring becomes trivial:
That's pretty cool, but I don't think I'd be able to remember all of those commands.
After a while, they just become second nature like Vim commands do. I have been forcing myself to use this over and over again for the past few days and it's starting to become second nature. I'm introducing individual commands at one of the time and building up into bigger and better things.
The real magic comes when you start writing your own commands with the full power of Cursorless and Talon. In that example I just showed you, I have a action for inserting "async " before the function definition. Here is the code for that:
[state] async <user.cursorless_destination>:
user.cursorless_insert(cursorless_destination, "async")
You can break talon commands into two basic parts: patterns and captures. Patterns are the spoken words that you say and captures are the things that you want to extract out of what you say. In this case, the pattern is just the word async and the capture is the destination that you want to insert the word async before. The <user.cursorless_destination> capture is a special capture lets you specify if you want something before or after a target. Of course, this is just a very simple example and it can get way more intricate than this.
Here is the most complicated Talon rule I've written so far:
(method|meth) <user.letter> [<user.go_pointer>] [<user.go_visibility>] <user.text> [over] [<user.go_visibility>] named <user.text> [over]:
user.go_method(go_pointer or "", letter, go_visibility_1 or "public", text_1, go_visibility_2 or "public", text_2)
What the heck is going on there?
This looks like a lot, but it is actually really simple. This lets you declare a method in Go. In Go, a method looks like this:
func (reciever *Type) MethodName() {
// function body here or something
}
Pedantically, Go doesn't have methods in the traditional sense, it just has functions that take structs as the receiver (read: a hidden first argument but in a way that is namespaced to that struct in particular). Without something to automate writing this for you, you would have to say something like this:
state funk args word reciever space star hammer type over go right space hammer method name args go right brack enter
That's a lot of words to say. But, with this talon rule, you can just say:
meth r raised type named method name over
That's still a lot of words.
Well, yes, you're not going to be able to get over that. But this is at least more efficient and it makes more sense. I'm not going to be able to get rid of all of the words, but I can at least make it so that it's close to how I conceptualize it in my head.
I guess that makes sense, but what do you mean by raised? I don't think go has raised types I know it has pointers but not "raising". What is raising?
I'm glad you asked! This is something that I'm experimenting with to try to find a different way to explain the concept of pointers in Go. I think that one of the oversights in the Go language is that pointers use C-style syntax. This specifically has you use an * to lower a value from a pointer value to a normal value and & to raise the value from a normal value into a pointer value.
Since I'm taking the opportunity to radically redesign the Talon bindings for Go, I want to try unifying the syntax of pointer values into the idea of raising and lowering to see how it makes it easier to understand Go programs. I don't know if this is a good idea, but you have to fuck around in order to find out.
Maybe some parts of our industry are actually good. I really hope that I get led into the GitHub copilot voice beta soon, I want to compare how Talon does voice coding versus how copilot voice does it.
Facts and circumstances may have changed since publication. Please contact me before jumping to conclusions if something seems wrong or unclear.
Tags: cursorless, vscode