All Things AI · Season 3 · What You Can Build · Episode 10
The call that comes after dinner can get a real conversation and a booked appointment instead of voicemail. Voice models now hold a natural phone conversation, answer questions, and put something on your calendar. This is available today and almost nobody in the industry is running it.
The call you already lost
Somebody sees something at 9:40 on a Tuesday night. They are interested enough to call right then, which is the most interested they are ever going to be.
It rings out. They leave a message or they do not. You call back at nine the next morning, which is eleven and a half hours later, and by then they have called two other people and one of those picked up.
Everyone in this industry knows speed matters and everyone has read the studies. Almost nobody has solved it, because the actual solution has always been either answering your phone at 9:40 at night forever, or paying a service to answer it badly.
What changed
Voice models crossed a threshold recently that is hard to appreciate without hearing it. They interrupt naturally, they handle someone changing their mind mid-sentence, they pause where a person would pause, and they do not have the flat rhythm that made older systems obviously mechanical within three words.
More importantly they can hold a real conversation rather than walking a script. Someone can ask a question that was not anticipated and get a sensible answer, then go back to what they were saying, and the whole thing holds together.
It can answer actual questions
Given the right material, it can answer what somebody asks rather than deflecting to a callback. This is what separates it from every phone system anyone has previously experienced, and it is the reason people stay on the line.
It can book
Connected to your calendar, it can offer real times and put something in. The caller hangs up with an appointment rather than a promise that someone will get back to them.
It hands off
When something is beyond it, or when the caller wants a person, it takes the details properly and gets them to you immediately. A good handoff is a feature rather than a failure, and designing it well is most of the work.
What it should and should not do
This is the part that needs deciding before anything is switched on, and it is a judgment call rather than a technical one.
- It should answer, listen, and understand what the person is calling about. That alone beats voicemail comprehensively.
- It should answer straightforward factual questions from material you have given it, and say it does not know rather than guessing.
- It should offer times and book, because that is the outcome the caller wanted.
- It should never negotiate, advise, or say anything that commits you to something.
- It should say clearly what it is. A caller who works out mid-conversation that they were misled is worse off than one who was told at the start, and being upfront costs almost nothing in practice.
Where the line sits
The honest version of this decision is that there is a spectrum and you have to pick a point on it.
At one end it is a very good message-taker: answers, understands, captures the details, tells you immediately. Nobody has any concerns about that and it already beats what most agents have.
At the other end it is having a full conversation and closing an appointment. That is more valuable and it is where the judgment is required, because the caller is now having a substantive exchange with something that is not you, about your business.
Most people land somewhere in the middle: it can answer, it can book, it does not do anything that requires judgment, and anything with a hint of complexity goes to a person fast. Where exactly you draw it is yours to decide, and it is worth deciding deliberately rather than discovering by accident.
How you set one up
Less involved than it sounds, and the technical part is the easy half.
- A number that forwards to it, usually only outside your hours, which limits the exposure while you learn what it does.
- A brief covering what it is, what it may do, what it must never do, and how it introduces itself.
- A short set of facts it can answer from. Keep it small and accurate, because everything you give it is something it can say.
- Calendar access for booking, and a clear rule about which slots it may offer.
- A handoff path with a real threshold: any of these situations, get them to a person now.
Listen to the first fifty
This is the whole quality process and there is no substitute for it.
Recordings of the first fifty calls, listened to properly, will tell you more than any amount of configuration. You will hear the question it keeps fumbling, the point where people get frustrated, the thing it says that sounds wrong in your voice, and the moment it should have handed off and did not.
Fix those five things and it is genuinely good. Skip this step and you have an unsupervised system talking to your prospects, which is a materially worse position than the voicemail you started with.
Where this goes wrong
Four failures, and the first one is the one that causes actual harm.
Letting it answer questions it should not. Anything requiring judgment, anything that commits you, and anything where a wrong answer creates a real problem must be out of scope explicitly, because a voice model with no boundary will answer confidently rather than decline.
Giving it too many facts. Everything you load in is something it can say, including saying it slightly wrong. A small accurate set beats a large one.
Never listening to the recordings. It is running unsupervised on your most valuable moment. Fifty calls of attention buys you years of it working properly.
Pretending it is a person. It will come out eventually, and the caller who feels tricked is worse than the call you never took. Say what it is in the first sentence.
What it grows
The leads you are currently losing, which is a larger number than most agents want to look at.
Every enquiry outside your hours currently resolves to voicemail and a next-day callback, and the industry has simply accepted that as the cost of being one person. It is not a small leak. It is most of the evenings and all of the weekend.
It also changes what your marketing is worth. Every campaign that generates calls has been operating at a fraction of its value because a portion of those calls arrive when nobody can answer them, and that portion is not small.
What it costs
This is a metered one. It is charged by the minute of conversation, at a rate that is small per call and worth watching if volume grows, plus a number to route through. Against a single additional appointment a month it is not a close comparison, and the honest framing is that it is cheap relative to what it recovers rather than free.
Questions agents actually type
Can AI answer my phone and book appointments?
Yes. Voice models hold a natural conversation, answer questions from material you have given them, and book real times into a connected calendar. The caller hangs up with an appointment rather than a promise of a callback.
Will callers know they are talking to AI?
They should be told in the first sentence. Modern voice models are good enough that many people would not notice, which is exactly why saying so matters. A caller who works it out mid-conversation is worse off than one who was told.
What should an AI voice agent never do?
Anything requiring judgment, anything that commits you, and anything where a wrong answer creates a real problem. Those have to be excluded explicitly, because a voice model with no boundary will answer confidently rather than decline.
How do I know if my AI phone agent is any good?
Listen to the first fifty calls. That single step tells you more than any configuration, and it surfaces the question it fumbles, the point where callers get frustrated, and the moment it should have handed off to you and did not.
Season 1 is Claude Code for Real Estate Agents, filmed and on its own hub. All three seasons: AI for real estate agents.