For the past 5 ½ years or so, I live in the South of Spain. At first, I called Almería my home but in 2024 I moved to the outskirts of Granada. That move not only brought some green nature into my life (Almería holds the only desert in Europe – check out Tabernas if you are interested) – it also helped with a recurring issue I faced every winter. The coldness.
Yes, Granada has a ski resort but the houses are equipped with something houses in Almería usually don’t have: calefacción, a heating system. And while you can go swimming in December in Almería, using your keyboard inside the house can become quite painful. Outside, you have 17° Celsius (62.6° Fahrenheit), inside you barely reach 9°C (48.2°F). And maybe that’s warm enough for other folks – my hands and feet were frozen.
Winter 2022 was extremely bad, and stopping to work was obviously not an option, so I looked into ways to improve my work setup and found Talon Voice – a hands-free input replacement for the keyboard and mouse. I started playing around with it and tried several exercises – with more or less luck. But apart from updating the alphabet so it would recognise my commands better, I didn’t make much progress. I spent the winter that followed tucked in in my van in Ireland (it has a heating system) and since last winter, I enjoy the heating system in Granada. So cold hands became a thing of the past.
May AI help with hands-free coding?
The interest in hands-free coding didn’t vanish and every now and then I talk about it with my colleagues. It was during one of these conversations that Jose mentioned Serenade.ai – an open-source voice assistant using Artificial Intelligence. My brain immediately reacted: I have to try it – maybe it will make things easier.
I also remembered GitHub Voice (“Hey, GitHub”) and thought about trying that one as well. In the end, I’ve tested 3 tools/toolsets: Copilot Voice (that’s what was called “Hey, GitHub” before), Serenade.ai and Talon Voice in combination with Cursorless.
This is what I tried to achieve: Opening a file in VS Code, finding the word “Rob”, adding the line :authors_location: United Kingdom, copying that line, finding the word “Fritz”, pasting the line and changing United Kingdom to South Africa. Here’s how it went:
Copilot Voice
What was previously known as “Hey, GitHub” is now a Visual Studio extension called “Copilot Voice”. The fact that it’s a VS code extension is not a problem for me since I’m using the editor. What I do find irritating is that despite calling it “Voice”, there’s not much you actually can do without your keyboard. By default, you have to use a keyboard command every time you want to speak to your machine. This can get updated in the settings though:
You have different options on where the “chat” shall appear when you say “Hey Code”: chatInView, quickChat, inlineChat or chatInContext.
inlineChat will display the chat in the file and line you are currently working in. You will see in the short video below how that worked out for me. chatInContext and quickChat weren’t much better.
chatInView kind of works – and quite nicely, too. But here’s the thing: it’s a chat, not a tool that lets you use your voice to write code. You can have a chat with Copilot. Not more, not less. For some this might be enough. For me it didn’t fit the purpose.
Here’s the attempt with Copilot Voice:
Serenade.ai
As the name implies, Serenade uses AI to transform voice commands into code or keyboard commands. I like that it’s open-source and it’s nice that it’s not restricted to the editor: you could also use it in the browser. For the purpose of this blog post, I focus on the editor part.
After installing Serenade, I started with a short Ruby exercise which worked quite well – although I had the feeling that it just works so great because it was an exercise. As soon as I entered the world outside of this exercise, I had a bit of a hard time with my pronunciation: to be fair, I’m German, so it might be easier for you. Also, please don’t ask why I somehow didn’t realise that I used the German “U” pronunciation instead of the English one. But since this is a safe space, I didn’t want to withhold this fail from you. English is hard.
After some more attempts, I reached this point:
I have to say, it was quite enjoyable to use Serenade. I also tested it in combination with GitHub Copilot which worked okay but in the end, it didn’t convince me, mainly because of these two reasons:
- the open-source project is no longer actively maintained. To this day, the last commit is from 3 years ago.
- I don’t really see a reason that justifies the usage of AI for this.
I know that the second point is a very subjective one but at least for now, I prefer to not use a decision-making tool when the differences aren’t superb.
Talon and Cursorless
First of all, Talon works absolutely fine without Cursorless, but since I found it, I don’t want to miss it anymore. I will tell you more about it after you watched this little video of me using Talon and Cursorless:
Some things are very different from the Serenade.ai video. The little symbols above some characters for examples are elements from Cursorless – they act as anchors and can vary in form and colour. You also might have heard that I don’t really speak English, like in whole sentences. With Talon, you have a special alphabet you can use when you want to write something. This alphabet has some huge advantages for me: if the engine doesn’t understand my English – for example because I use the very German pronunciation of an “U” – I can use it and the input will be correct. For my little Serenade “South” disaster, I could have just said: “shift (to capitalise the next letter) sun – ost - urge - trap - ham”. Easy-peasy, isn’t it?
It’s also nice that I can adapt the alphabet – I use “ham” instead of “harp” because Talon’s voice engine does not know what I want when I say “harp” but it does understand it when I say “ham”. Something else I like about Talon is that it’s not restricted to the editor – I can use it for everything where I used my mouse or keyboard before. When I’m in the editor, I use the support from Cursorless and when I search the web Rango is helping me. There’s also a fabulous cheatsheets that I can recommend. And all that without AI.
Final words
By no means shall this blog post be understood as AI bashing – I do think there’s value in using Artificial Intelligence. If you use it wisely. But I do want to highlight that you don’t really need it to improve a situation. If you have trouble using your hands when working on your laptop, don’t jump instantly on the AI train – check out and support projects like Talon. In order to not get overwhelmed the way I did when I tried Talon the first time, do small steps: start with some short daily exercises and increase the intensity over time. And despite saying that it wasn’t helpful at all when I talked to ‘the machine’ – it did have a positive aspect: I think more before I write something; be it code or text which leads to fewer bugs and saves a lot of energy at the end of the day.