What it does
I wanted to be able to work during a walk or a drive without looking at a screen. When I call, a front desk answers and I ask for the session I want. The line can take over a session that is open on my laptop, continue it by voice and return it afterwards with the conversation from the call included.
If I ask for something that takes a short time, the line stays connected while the work is done. For longer jobs it offers to call me back when the job has finished.
Problems that appeared on real calls
Three problems appeared only once I was using the line on real calls.
The first was that the model would tell me it was switching me to another session and then fail to make the switch. I had the call program changed so that it recognizes those phrases and performs the switch itself, without depending on the model to do it.
The second was that anything I said while the assistant was still speaking was delivered to it afterwards, and it treated my words as the answer to a question it had asked in the meantime. Those words are now marked as having been spoken earlier.
The third involved the speech recognizer. I had given it a list of terms I expected to use, and it began to report those terms when it heard background noise. I removed the list, and requests that depend on those terms are now read back to me before they are acted on.
Controls
The line verifies the caller before it does anything, and verification codes are always processed on my own hardware. Requests that have consequences are read back to me before anything is sent, and requests with financial consequences are approval-gated.
When I add a guest to a call, a voiceprint can label who is speaking, and the guest's words do not operate the controls of the call. I do not treat a voice as proof of identity for any purpose. Requests that reach household devices go through the same rules, enforced in code, as requests typed at a keyboard.
Speech processing
Calls use a premium voice service, and the house voice on my own hardware is the backup. Other people have published similar phone projects. What is particular to this one is that it connects to working sessions that are already governed by Nero's rules.
"Then uprose smooth-tongued Nestor, the facile speaker of the Pylians, and the words fell from his lips sweeter than honey."
