Build your own AI platform when the workflow is the product: when the way your team does the work is the reason clients hire you, and no vendor sells that. Buy when the job is commodity and a subscription covers most of it. I build the internal AI platform at Tandem Interactive, a Fort Lauderdale agency, and everything below comes out of drawing that line in production.
Should an agency build or buy its AI tools?
My honest starting point is to default to buy, which is strange advice coming from the guy who built the in-house platform, but it’s where any build versus buy decision should start. Vendor tools are maintained by teams; your build is maintained by whoever wrote it, in whatever time is left after client work. The first working version is always the cheap part, because the real invoice arrives in year two, when models get deprecated, APIs shift under you, priorities pile up, and the tool everyone now depends on needs the same care as a product, because that’s what it became.
So the bar for building has to be specific, and mine has two parts. The first is that the workflow you’re automating has to be the thing clients actually pay your agency for. If your process is your edge, renting a generic version of it means sanding that edge down to fit someone else’s product decisions. The second is that the vendor demo dies at the exact step that matters, because every tool looks brilliant for the first 90% of a job and agencies live in the last 10%: the weird client constraints, the judgment calls nobody documented, the exceptions to the exceptions, and the sanity check a senior person runs before a number leaves the building. When that last stretch is where your value sits and no product handles it, build.
There’s a third trigger nobody puts on a slide: custody. Client data flowing through a rotating cast of third-party AI tools is a conversation you’ll eventually have with somebody’s legal team. Owning the pipe is how you end up owning the answer.
I spent ten years on the marketing side before I was the engineer, so when I built the platform I was building for a user I used to be. That closeness is the fourth trigger, and it’s the one that makes the other three answerable, because you can’t tell from the outside which part of a workflow actually hurts - you know that from having had the job. If nobody on the build side has ever done the job the tool serves, that’s an argument for buying, and not a small one.
An internal AI platform is a promise you make to your own team: the boring parts of the job get faster and the trustworthy parts don’t get looser. The moment your tool ships a wrong number quicker than a human would have, you’ve automated the loss of your own credibility. Speed is the feature everyone asks for, but trust is the one that decides whether anyone opens the thing a second time.
What does fail-closed AI design mean?
Fail-closed means the system degrades to silence instead of a guess: when it can’t verify something, it says so or says nothing. Fail-open fills every gap with the most plausible-sounding thing available. Language models fail open by nature - they’re confidence machines - so the platform’s job is to supply the humility the model doesn’t have.
This matters at an agency because the blast radius of a hallucination is a client deliverable. A made-up stat doesn’t die in a chat window. It graduates into a deck, gets presented with a straight face, survives the meeting, and becomes your agency’s claim. The outputs that worry me aren’t the absurd ones but the plausible ones, because a plausible number rarely gets a second look at the end of a long week.
The rule I build to: an AI feature earns the right to make a claim by grounding it in data the system actually retrieved, and when the data isn’t there, the gap stays visible. Nothing gets smoothed over and no estimate gets presented as a finding. A blank cell is information, while a fabricated cell is a liability with nice formatting.
Fair warning that fail-closed loses the demo every time, because “I couldn’t verify this” is a worse sentence than a confident wrong answer right up until the wrong answer reaches a client. Build it that way anyway, since what you’re optimizing for is month six, when people have stopped double-checking the tool because it earned that.
Why AI numbers need to carry uncertainty
Ask a language model the same question twice and you’ll get two answers. That’s not a bug you can patch; it’s what the technology is. Which means any metric built on a stochastic system is a sample, and one sample proves nothing no matter how good the dashboard in front of it looks.
The fix comes from old statistics rather than new AI. Sample repeatedly and report rates instead of one-off observations, then put a range around every number before anyone compares it to anything. Alerts should fire when ranges genuinely separate, not when two points differ. A number without its range is not yet a measurement.
The payoff I didn’t expect is that when reports carry their uncertainty honestly, people trust them more rather than less. Sometimes the honest read is “this hasn’t moved enough to say anything yet”, and a system willing to admit that is the one the team believes when it finally says something did move. The first time a reported “drop” turns out to be sampling noise, you’ve spent trust you don’t buy back at par.
How should AI agents be permissioned and audited?
The moment an AI feature stops saying things and starts doing things, you’ve hired a coworker with keys, and keys mean policy. The useful principles are old security ideas, and they hold up:
Least privilege, always. An agent gets the narrowest access that still does the job, gated by the role of the human driving it. Nobody hands an intern admin rights on day one, and software that occasionally hallucinates has earned the same treatment.
Deny by default. New capabilities ship locked and get opened deliberately, one person at a time, with a name attached to the call. “On for everyone because it’s launch day” is how incidents get scheduled.
Everything leaves a receipt. Every action an agent takes should land in a record a human can read later. The question is never whether you’ll need to reconstruct what happened, only when, and for whom. Sometimes that’s a client asking why something changed, and sometimes it’s you at 11pm trying to reconstruct your own system.
A human signs the irreversible steps. An agent can’t be the responsible party; responsibility doesn’t compile. Anything you can’t undo gets a person’s explicit yes attached to it.
The first time an automation does something technically correct but surprising, this stops being compliance homework. You want the audit trail before the surprise; afterward is a lousy time to start logging.
How do you get a team to actually use internal tools?
No engineering blog warned me about this part: shipping the platform is the easy half. Internal tools don’t die in code review, they die on a random Tuesday when someone is under deadline and quietly goes back to the old spreadsheet, because the old spreadsheet never surprises them.
Nobody owes your tool their attention, and it competes with habit, which is free and already installed. These are the things that actually moved adoption for me:
Start with one person’s worst task. This doesn’t come from a survey or a committee, it comes from finding the job one real teammate dreads and making it dramatically better end to end, then let them do the marketing for you, because one unprompted “you have to try this” in a team channel beats every launch announcement I’ve ever written.
Meet people where they already work. Every extra step between a teammate and your tool is a toll, and tolls compound. The closer the tool sits to the workflow they already run, the less adoption you have to beg for.
Treat teammates like paying customers. Onboarding, release notes, fast answers when something breaks, and a real apology when it’s your fault. Treating it as “just internal” is how these tools lose the team’s trust.
Guard trust like revenue. One wrong number costs you ten right ones, and the person it burned tells everyone at lunch. Fail-closed design and honest ranges aren’t academic hygiene; they’re adoption strategy. People come back to tools that have never lied to them.
Rules I’d give anyone doing this
- Buy the commodity and build the differentiator. Re-ask every quarter; the vendor market keeps moving.
- The maintenance budget is the real budget. Price year two before you write week one.
- Fail closed. A visible gap beats a confident guess, every single time.
- No naked numbers. Every metric ships with its range, and alerts require genuine separation.
- Agents get the least access that does the job, and every action leaves a receipt.
- A human signs every irreversible step. Software doesn’t get to be the responsible party.
- Ship to one enthusiastic user before you ship to everyone.
- Run it like a product, because to your team, that’s what it is.
FAQ
Should a marketing agency build its own AI platform?
Usually the answer is no, so buy for commodity jobs and spend the savings on your actual service. Build when the workflow is your edge and vendor tools keep dying at the exact step that matters, or when client-data custody makes a parade of third parties a hard conversation. If neither applies, the honest answer is a subscription.
Can one developer really build and run an internal AI platform?
Yes, with ruthless scope and a preference for boring choices, since the constraint was never writing the thing but carrying it while models churn and priorities shift. I’d been building my own tools for years before this one, from trained models to desktop apps, and the habit that mattered most was saying no to features I would have enjoyed writing, which is roughly the spread covered on my capabilities page.
What does fail-closed mean in AI product design?
When the system can’t verify a claim, it declines to make one. Missing data stays visibly missing instead of getting papered over with a plausible guess. It’s the difference between a tool that’s occasionally silent and a tool that’s occasionally lying.
How do you keep AI agents safe in client work?
Give them the minimum access the task requires, gate capabilities by role, keep a human approval on anything irreversible, and log every action so you can reconstruct it later. If a feature can act on the world instead of describing it, treat it like a new hire with production access.
How do you know an internal tool is actually working?
Watch retention rather than compliments, because praise is free and the real signal is whether people come back next week without being reminded. The clearest version of that test is what the team reaches for when a deadline hits. If it’s your tool, you built the right thing.