Part IV of IV

Ship and Track

You have a feature that works on your machine, running on a model you never chose. This part takes it to a live product: pick the model on evidence, wire it in, pre-flight it, ship, and learn from real use.

The outcome

By the end you can take an AI feature from a working demo to a live product strangers use, then decide from watched use whether to iterate, hold, or kill.

Who this is for

For builders with an AI feature working locally who want to ship it to real users. No engineering background needed.

What you’ll walk out with

You arrive withYou leave with

A default you never choseChoose your model on evidence
Accepting code you never readReview AI-written code without coding
Keys and bills exposedKeep keys and spend contained
No policy, or claims you can't backShip the legal minimum safely
A build only you can reachMove your build to a public address
Deciding from memory, not dataDecide from watched use, not memory

Start with chapter one The Build Plan · Six fields, one page, no code.

A real incident

A frontier model topped the charts while its database sat open to anyone who looked. The six chapters next door are how you ship without leaving that gap.

01 / 09

A real incident

DeepSeek's chatbot topped the App Store, then researchers found its database open on the public internet, no password required, exposing a million log lines of chat history and secret keys.

DeepSeek closed the hole within 30 minutes of the warning. The model that put it at No. 1 was never the broken part; the failure was an unsecured data store left running beside it.

Wiz Research, 2025, exposed ClickHouse database at oauth2callback.deepseek.com

02 / 09

What happened

A stranger needed no credentials to query the live database from a browser.

OPEN TO THE INTERNET

oauth2callback.deepseek.com:9000, no password. The log_stream table held over 1,000,000 lines of chat history, API keys, and backend secrets, all in plaintext.

A routine scan found it in minutes.

Authentication was switched off on a non-standard port that faced the public internet. Anyone scanning for open databases could run queries and read every logged message and key.

03 / 09

What actually failed

The model was frontier-class, and the choices wrapped around it were not.

Secret keys sat in plaintext logs, so one open port handed out the backend credentials.

The database faced the public internet with authentication turned off entirely.

Nobody had checked what a stranger could reach before millions of users arrived.

A stronger model would not have closed that port, because the gap was never in the model.

04 / 09

The pattern

Exposed data stores are the ordinary way AI products leak, not exotic attacks.

minutes

for researchers to find the open database

1M+

log lines: chat history, API keys, secrets

cost to read it, no login needed

Logs reached back weeks before anyone outside the company checked. The window between shipping and a stranger finding the gap is short.

Wiz Research and TechCrunch, 2025

05 / 09

The one idea

A feature that runs on your machine is not a live product; the distance between them is a set of ship decisions, not a better model.

You pick the model on evidence, wire it in safely, cap what it can spend and expose, then watch real use. Skip those and the demo ships its gaps straight to strangers.

06 / 09

What this part builds

Five moves take a working demo to a product strangers can trust.

Pick on evidence

Run five real inputs through two model tiers and ship the cheapest that clears your bar.

Wire it safe

Server-side key, a fallback, and a spend cap before the first real request lands.

Pre-flight the risks

Close the gaps in secrets, money, data, and the law before outsiders arrive.

Learn from real use

Send the link to ten strangers, read their sessions, then iterate, hold, or kill.

07 / 09

The objection worth answering

"That was a big company's mistake. My small build is not a target."

DeepSeek's database was found within minutes by a routine scan of internet-facing ports, not by anyone who cared who owned it. Scanners do not check your headcount, they check whether the door opens.

The gaps that leak are the boring ones: a key in the wrong place, a store left open, a default nobody chose. A pre-flight catches them in an afternoon, which is far cheaper than catching them in the news.

08 / 09

What actually fixes it

Each gap in this story is a chapter that closes it.

A model you never choseChoose a model

→ Run five real inputs through two tiers and ship the cheapest that clears your bar.

A key in the openFirst hour

→ Keep the key server-side, add a fallback, and set a spend cap before the first request.

An unread buildReading the build

→ Read the diff for where every value comes from and how each one fails.

An unguarded storeGuardrails

→ Close secrets, money, data, and law before an outsider can reach them.

The pre-flight takes an afternoon, far less than the incident took back.

09 / 09

From story to part

Every gap in this story is one of the six chapters ahead.

The six chapters take one feature from a default you never chose to a live product strangers use, and end with a one-page Build Plan that records the model, the wiring, the caps, and the call you made from real use.

Start with chapter one →