This guide is for business owners and staff who have built an app, internal tool or automation by describing it to an AI tool, and who now want customers or colleagues to rely on it. It covers the checks you can run yourself, without being a developer, and how to tell when you need one.
Who This Guide Is For
You built something with ChatGPT, Claude, Cursor, Lovable, Bolt, Replit or a similar tool. That way of working is usually called vibe coding. Now it is about to take real bookings, store real customer details or run without you watching. You want to know what could go wrong before that happens, and you would like to fix the obvious problems yourself.
Before You Start
Write down, in a short paragraph, what the app is meant to do, who will use it and what information it handles. This becomes the yardstick for every step below, because you can’t check whether something is right without saying what right is. If you later ask a developer to review the app, it is the first thing they will want.
You will also need access to the code (the project in your AI tool, or a repository), the hosting account, and any outside services the app uses, such as a payment provider or an AI model’s API.
Step 1: Find the Secrets
Most apps talk to outside services using an API key: a long string that works like a password. AI tools often put these keys straight into the code, and sometimes into the part of the code that runs in the visitor’s browser, where anyone can read them.
To check, open your live app in a browser, right-click, choose “View page source”, and search for words like key, secret, token and the name of each service you use. Many AI-built apps load their code from separate script files, so also open the browser’s developer tools (F12), go to the Sources tab and search there. Then search the project code for the same words. A key visible in the browser is a key anyone can copy and spend on your account. Keys should live in the server’s environment settings, never in the code itself.
If you find one exposed, move it, then go to that service’s dashboard and generate a new key, because the old one should be treated as already copied.
Step 2: Try to See Someone Else’s Data
Create two ordinary user accounts. Log in as the first, open one of its records, and note the web address. Then log in as the second and paste that address in. If the second user can see the first user’s record, the app checks that someone is logged in but not whose data they are asking for. AI-built software is prone to this gap, because the tool builds the screen you asked for and rarely the rule you didn’t mention.
While you are there, log out completely and try opening the admin pages directly by their address. They should send you to a login screen. Our guide on how to implement role-based access control explains what a proper permission setup looks like if you need to describe it to your AI tool.
Step 3: List the Data It Holds
Make a list of every piece of personal or business information the app stores: names, emails, phone numbers, addresses, payment details, notes about customers. Next to each one, write where it is stored (which database, which service), who can see it, and whether you need it at all.
Under UK GDPR you need a reason to hold personal data, you need to keep it secure, and people can ask you to delete it. If you can’t say where a customer’s details live, you can’t delete them when asked. Anything you don’t need is worth removing now, because data you never collected can’t leak.
Step 4: Set Up Backups and Restore One
Check whether the app’s database is backed up, how often, and where the backup goes. On some hosting platforms, backups are a setting you have to switch on, or are only included on a paid plan.
Then do the part most people skip: restore a backup into a test copy and check the data is there. Until you have restored one, you don’t know the backups work.
Step 5: Break It on Purpose
Real users will do things you never tried. Spend half an hour being one of them:
- Submit forms empty, with very long text, with an emoji in the name field, and with a date in the past.
- Double-click the submit button on anything that takes payment or sends an email.
- If the app calls an AI service, disconnect it (or use an invalid key in a test copy) and see what the user gets.
- Check whether errors are written down anywhere. If the app fails at 2am, would you find out from a log, or from a customer?
At minimum, set up uptime monitoring so you hear when the app is down before your users tell you. Our guide on how to set up uptime monitoring covers the options.
Step 6: Cap the Running Costs
Apps that call AI models are charged by usage, and a bug, a loop or a misuse of your form can run up a bill overnight. Log in to each service that charges by usage and set a monthly spending limit and an alert at a lower figure. Check the hosting plan too: some charge per request or per minute of server time, and a busy week can cost more than you expect.
Look out for anything that calls an AI service on every page load or keystroke; it can usually be changed to call once and save the result.
Step 7: Decide What Automations May Do Alone
If the app includes an automation or an AI agent, write down every action it can take without a person approving it: sending emails, issuing refunds, changing records, posting messages. For each one, ask what happens if it does the wrong thing a hundred times before anyone notices.
Anything that spends money, contacts customers in bulk or can’t be undone should need a person’s approval, at least to begin with. It should also be obvious how the automation hands a case to a human when it is unsure.
Step 8: Answer “Who Can Fix It?”
Picture the app breaking on a Friday evening while you are away, then check:
- Is the code in version control, such as a GitHub repository, so changes are recorded and can be undone?
- Are the hosting, domain and service accounts in the business’s name, with logins someone else can reach?
- Could another developer open the code and understand it? Ask your AI tool to write a plain README explaining how the app is set up and how to run it, then read it yourself.
If the answer is that nobody but you, and the AI tool, could fix it, that is a risk worth knowing about even if you decide to accept it.
Common Mistakes
- Prompting fixes straight into the live app. Each new prompt can change code you didn’t ask it to touch. Keep a test copy and change that first.
- Trusting the AI’s own summary of its work. Asking the tool whether its code is secure gets you a confident answer, not a check.
- Testing only as the admin. Most permission problems only show up when you log in as an ordinary user.
- Leaving the old exposed key active after moving it out of the code.
- Keeping test mode switched on for payments or email after going live, or the reverse: testing against live payments with real cards.
When to Get It Signed Off
These checks catch the obvious problems. They don’t catch the ones that need someone to read the code: input that isn’t cleaned before it reaches the database, packages with known security flaws, logic that only goes wrong in a particular order of steps. An automated AI review tool can catch some of those, and our comparison of AI code review tools vs developer sign-off sets out where each one stops.
If the app is internal, handles nothing sensitive and a day of downtime would be an inconvenience, working through this guide may be enough. If it takes payments, stores customer details, acts on its own or would hurt the business if it failed publicly, it is worth having a developer review it before you rely on it. Our article on who is responsible for software built with AI explains why that decision sits with you either way.
What Good Looks Like
Every key lives on the server and has been rotated if it was ever exposed. An ordinary user can see only their own data, and admin pages ask for a login. You have a list of the personal data the app holds and a backup you have restored. Errors are logged, downtime alerts reach your phone, and every paid service has a spending cap. Automations ask before doing anything that costs money or can’t be undone. The code is in version control, the accounts are in the business’s name, and a written brief says what the app is for.
Next Steps
If you are planning a wider launch, how to launch a software product safely covers the go-live itself. If you would like a developer to read the code, fix what fails and sign off the version you are about to rely on, that is our AI Sign-Off service, and the brief you wrote at the start of this guide is where it begins. If your checks turned up problems, what it costs to fix a vibe-coded app helps you work out what kind of fix you need before you ask for a quote. Other practical walkthroughs are in our guides.
Written by
Alex
CEO
I’m a software developer and CEO of Digital Royalty, helping growing teams scale their SaaS platforms without losing quality, visibility, or control. I focus on building structured, maintainable systems with clear processes, reporting, and accountability. With over a decade of experience across agency and in-house roles, I specialise in delivering long-term, scalable solutions that support complex, evolving products.