All Things AI · Season 2 · The Models · Episode 9
A local model is any model running on hardware you own. Free permanently, works with no internet, and nothing you type goes anywhere. It is a category rather than a product, and for an agent it is the single most under-adopted option available.
What the category is
Not a specific tool. A way of running one. The model is published as a file, you download it, and an application on your computer loads it and lets you use it.
That single architectural difference produces every property that matters here. No account, because there is nothing to sign into. No monthly cost, because nothing is being provided to you. No request leaving the building, because nothing is being sent.
The four properties worth caring about
Each of these is a genuine difference rather than a marketing distinction, and together they are why this deserves its own episode.
It is free, permanently
Not a trial and not a limited tier. There is no meter, so running something two thousand times costs the same as running it once, which changes what work is worth doing at all.
Nothing leaves your machine
For anyone handling other people’s documents this is not a preference, it is the correct default. The question of where the file went does not arise.
It cannot change underneath you
A file on your disk behaves identically in three years. Anyone who has built a process on a hosted service has watched it behave differently one morning with no notice, and that is a real cost once you rely on something.
It works with no connection
Which matters more in this job than in most, given how much of it happens in cars, in buildings, and in places with poor signal.
What it is genuinely good enough for
Being honest about the boundary is what stops people from either dismissing this or over-relying on it.
- Reading long documents and extracting specific things. Excellent, and this is the highest-value use.
- Structured extraction from unstructured text, repeatedly and identically. Excellent.
- Long-form drafting from your own outline and material. Good, particularly with a model suited to it.
- Rewriting, reformatting, comparing, and summarising. Good.
- Hard multi-step reasoning where the answer is not in the material. This is where the hosted services still win.
The hardware question, answered plainly
Memory is the constraint. Not processor speed, not graphics, not age within reason.
Sixteen gigabytes runs a small model comfortably and is enough to do real work. Thirty-two runs a genuinely capable one and is where most people stop noticing the difference from a paid service on everyday tasks.
Apple machines have a structural advantage because memory is shared with the graphics side, which is exactly what a model needs, so a Mac with adequate memory performs better here than the specification sheet suggests. If you are replacing a laptop anyway, memory is the number to spend on.
What to try in the first week
Three exercises, in order, and the whole thing is one evening.
Install the application and download the smallest capable model on the list. Not the largest. The most common way this fails is someone downloading the biggest thing available, finding it unusable on their machine, and concluding the category does not work.
Then turn off your internet and use it for twenty minutes on real work. Everyone reads that it runs offline and nobody believes it until they watch it. This is the moment the argument stops being abstract.
Then give it a long document and ask it to extract every date and deadline with the page for each. Tedious by hand, immediately checkable, and it tells you within minutes how much to trust it.
Where this goes wrong
Four failures, and the first accounts for most abandonment.
Starting too big. A model too large for the memory will run and be miserable, and the conclusion drawn is about the category rather than about the choice. Start small, confirm it is pleasant, size up only when you need to.
Expecting current information. It has none and cannot get any, and it will not reliably tell you so.
Assuming private means correct. Running locally answers where the data went and says nothing about accuracy.
Treating it as an all-or-nothing decision. This is not a replacement for hosted services, it is the right tool for a large category of work, and the sensible setup uses both.
Why almost no agent has done this
It is free, it is private, it takes an evening, and adoption in this industry is close to zero. Worth asking why.
Partly because nobody markets it, since there is no product to sell. Every other option has a company behind it explaining why you need it, and this one has nobody.
Partly because it sounds technical. Downloading and running a model reads as an engineering task and it is closer to installing an application.
And partly because the industry’s understanding of AI was formed by one hosted chat window, so the mental model is that this is a service you subscribe to. The idea that the capable version might be a free file on your own disk does not fit that frame, which is exactly why it remains available as an advantage.
What the application actually does
Worth demystifying, because the phrase running a model locally is what makes this sound harder than it is.
The model is a large file of numbers. On its own it does nothing, the same way a document does nothing without something to open it. The application is what loads it into memory and turns your typing into something the model can respond to.
There are several of these and they all do the same job. The good ones show you a list of available models with a size next to each and a download button, and once something is downloaded you type into a box exactly as you would anywhere else. There is no configuration to understand and nothing to connect.
That is genuinely the whole thing. The gap between how this sounds and what it involves is the reason so few people have tried it, and twenty minutes closes it permanently.
Running two, and why it is worth it
Since there is no cost per use, running more than one model stops being a decision and becomes a habit worth forming.
Different models are tuned for different things. One follows instructions rigidly and is the right choice for extraction where the format has to be identical every time. Another writes at length and holds a piece together, which is what you want for drafting.
Having both installed means picking the right one takes no thought and costs nothing, and it also means you can run the same request through both when something matters. Where they agree you can be more confident. Where they differ, the difference almost always marks a place where your request was ambiguous rather than where one model failed.
On paid services nobody does this, because running everything twice doubles a bill. Here it doubles nothing, and it is one of the few genuine advantages of the free option over the paid one.
How to split work between local and paid
The rule that works is by material rather than by task.
Anything with a client, an address, a document, or a name goes local. That is not a judgment about any provider, it is that the question stops existing, and for a category of work this sensitive the question not existing is worth a lot.
Anything needing current information, or the hardest available reasoning, goes to a paid service with the sensitive parts removed first. Most agents find that is a smaller share of the work than they expected once they actually sort it.
What it costs
Nothing, permanently. Disk space, an evening, and eventually a decision about memory when you next replace a machine. There is no tier, no meter, and nothing that can be repriced.
Questions agents actually type
Can I run AI on my own laptop for free?
Yes, permanently. Open models are published as files you download and run with a free application. There is no account, no subscription, and no per-use charge, and everything works with the internet turned off.
What computer do I need for local AI?
Memory is the constraint rather than processor speed. Sixteen gigabytes does real work and thirty-two runs a genuinely capable model. Apple machines perform better than their specifications suggest because memory is shared with the graphics side.
Is local AI good enough compared to ChatGPT?
For reading documents, extraction, drafting from your own material, and reformatting, yes. For hard multi-step reasoning where the answer is not in the material, hosted services still have an edge. Most agent work falls in the first category.
Why do so few real estate agents run AI locally?
Largely because nobody markets it, since there is no product to sell, and because it sounds like an engineering task when it is closer to installing an application. That gap is why it remains an available advantage rather than a standard practice.
Season 1 is Claude Code for Real Estate Agents, filmed and on its own hub. All three seasons: AI for real estate agents.