Don't Marry Your LLM Provider — A Love Story About Infrastructure
Let me tell you about the most boring, most important architectural decision I've made on this entire project, a decision that produced zero visible improvement to how my insurance agent talks to customers and yet might be the thing that saves the whole business in two years.
I rebuilt the entire LLM integration layer to be provider-agnostic. Meaning my agent doesn't know or care whether it's talking to Claude, or GPT, or Gemini, or whatever comes next month. It just talks to an abstraction layer that handles the translation.
I know, I know. You're thinking "that's just good engineering, why are you writing a blog post about it?" And you'd be right, it is just good engineering, but here's the thing — most people building AI products right now are NOT doing this, and the reason they're not doing it is the same reason most businesses fail: they optimize for today and get killed by tomorrow.
The seduction of vendor lock-in
When you start building with any LLM, there's this incredibly seductive path where you use their specific SDK, their specific prompt format, their specific function calling syntax, their specific streaming protocol. Everything just works. It's like the first months of a new relationship, everything is amazing, you finish each other's sentences, you can't imagine being with anyone else.
And then prices change. Or a better model comes out from a competitor. Or your provider has an outage on the day your biggest client is running a campaign. Or, and this is the one that keeps me up at night, your provider decides to change their terms of service in a way that doesn't align with your business anymore.
I come from fintech, I come from credit and banking and regulated environments, and if there's one thing that world taught me it's this: never let a single dependency have existential power over your business. Never. Not your bank, not your investor, not your technology provider. That's not paranoia, that's capital structure 101, and it applies to AI infrastructure just as much as it applies to funding rounds.
What I actually built
The implementation is almost disappointingly simple, which is usually a sign that it's correct. There's an adapter layer that translates between my agent's internal format and whatever provider I'm pointing at. The agent constructs its thoughts, its tool calls, its response strategy in a provider-neutral format. The adapter handles the rest.
Switching from one provider to another is changing one configuration variable. Not one file, not one module — one variable.
Now, the truth is that prompt behavior varies across models. A prompt that works perfectly with Claude might produce slightly different results with GPT-4. So provider-agnostic doesn't mean provider-ignorant, I still tune and test for specific models. But the infrastructure doesn't care. The infrastructure is the road, the model is the car. I can drive any car on this road.
The organizational parallel
This is what I keep coming back to, because my whole project is about finding the patterns between AI systems and organizations, and this one is so obvious it hurts.
How many companies have I seen — including my own, let me be honest — that built their entire operation around one key partner, one key technology, one key employee? And when that dependency shifted, the whole thing wobbled. The financial structuring equivalent is a single-lender capital structure. Looks great on the spreadsheet, absolute disaster when terms change.
The companies that endure, and I think about this constantly, they build with substitutability in mind. Not because they plan to substitute, but because the ability to substitute is itself a form of power. It changes how you negotiate. It changes how you plan. It changes what threatens you and what doesn't.
My LLM adapter is a tiny piece of code. But conceptually, it's a governance decision. It's me saying: no single provider gets to own my business logic.
The cost of doing this
I'm not going to pretend this was free. It added maybe a week of work, it introduced an abstraction layer that adds a tiny bit of latency, and it means I can't use some of the fancier provider-specific features without wrapping them. Every time Claude releases some beautiful new capability, I have a moment where I think "should I just use this directly?" and then I don't, because the discipline matters more than the convenience.
It's like, you know how in accounting we have this concept of provisions? You set aside resources for risks that haven't materialized yet. Most people think that's conservative and inefficient. Until the risk materializes. And then suddenly the conservative one is the only one still standing.
That's what provider-agnostic architecture is. It's a provision against a risk you can't predict but know is coming, because in this industry, the only thing that's certain is that everything will change.
What this means for Organization as Code
If I zoom out to the Organization as Code framework — this idea that you can map an entire business into explicit, testable, improvable components — provider independence is a core principle. Not just for LLMs. For everything.
Your processes shouldn't depend on one specific person's knowledge. Your sales strategy shouldn't collapse when one channel changes its algorithm. Your capital structure shouldn't crumble when one investor has a bad quarter.
At the deepest level, what I'm building is not just an insurance agent. I'm building an organization that's designed for substitutability at every layer. The human (me) can be substituted eventually too, if the architecture holds. That's not depressing, that's the whole point — the ultimate test of good architecture is that it survives the absence of its architect.
But let's not get too philosophical. For now, my agent can talk to any LLM on the market without changing a line of business logic, and that's a win I'll take.
Next up: What my agent forgets every night — and why the four types of organizational memory are the key to everything.
— Vasile Tămaș, building from Cluj-Napoca, Romania