How to test an AI-built app before your team relies on it

An AI-built app can go from idea to working screen in an afternoon. That speed is the point. It also creates a new risk: the app looks finished long before anyone has checked whether it is right. The schedule screen is clean, the buttons work, the totals add up for the three records the builder tried, and the team starts relying on it the next morning.

Testing is how you close that gap. You do not need a developer or a testing department. You need a clear plan, a few hours, the people who know the work, and the discipline to write down what passing looks like before you start. This guide gives you that plan, with numbered checklists you can copy, a launch table, and a simple sign-off.

Why AI-built apps need deliberate testing

AI is very good at producing something that looks right. Developers, who use AI more than anyone, are clear about where it falls short. In the 2025 Stack Overflow Developer Survey, 66 percent named "AI solutions that are almost right, but not quite" as their top frustration, and 46 percent said they actively distrust the accuracy of AI tools. In the 2026 survey, 48 percent said they trust AI output when they can easily verify it. Microsoft's 2026 Work Trend Index found that 86 percent of AI users treat AI output as a starting point.

The lesson for an owner is simple: verify before you rely. An app that is almost right can mean a commission calculated wrong every month, a job assigned to someone who left, or customer records visible to the wrong people. That last one has happened at scale. The Register reported in February 2026 that one app hosted on Lovable had 16 vulnerabilities, 6 of them critical, exposing 18,697 user records. An August 2026 scan by reeve.page found that 57 percent of the 3,553 Lovable app databases it could check had tables readable without logging in.

Before you start: set up the test

1. Name a test lead. Usually the business owner of the tool, the person whose team will use it every day.
2. Write down what passing means. One sentence per thing the tool must do. "A dispatcher can assign a job to a technician in under 30 seconds." "The monthly total matches the spreadsheet to the cent."
3. Make test records. Create 10 to 30 records that look like your real ones but belong to nobody. Include a few awkward ones: a customer with a very long name, a job with no address, an order with zero items.
4. Make a test account for every role. At minimum one for each role the tool uses, such as manager, office staff and field worker.
5. Keep real data out. Do not load real customer, employee or financial records until the access and security checks pass.
6. Keep a simple log. For each problem: what you did, what you expected, what happened, and a screenshot.

The seven areas to test

1. Does it do the job?

Walk through the real work, start to finish, the way your team does it on a busy day.

1. Create a record the way a new request actually arrives.
2. Move it through every status it can have, in order.
3. Move one backwards, such as reopening a closed job.
4. Hand it from one person to another the way your team does.
5. Finish it and confirm it lands where finished work should.
6. Check that every field your team needs exists, and that none are missing from your paper form or spreadsheet.

2. Are the numbers right?

Every total, count, rate and date calculation should be checked by hand at least once.

1. Pick five records and calculate the expected result yourself.
2. Compare to what the app shows. Check to the cent or the minute.
3. Try a record that crosses a month or year boundary.
4. Try zero, a negative number if one is possible, and a very large number.
5. If the tool replaces a spreadsheet, run last month's data through both and compare the results line by line.

3. Does each person see only what they should?

This is the test most often skipped, and the one that matters most for data safety. Builders test as themselves and see everything, so they never notice what others can see.

1. Sign in with each role's test account in turn.
2. Confirm each role sees exactly the records and screens it should, and nothing more.
3. Confirm a role that should only view cannot edit or delete.
4. Copy the link to one record, sign in as a role that should not see it, and paste the link.
5. Open the app in a private browser window without signing in. You should see a sign-in screen and nothing else.
6. Remove a test user and confirm they can no longer get in.

Our guide to role-based access for employee-built apps explains how roles should be set up in the first place.

4. Basic security checks

You are not running a full security audit. You are catching the common, documented mistakes. Our guide to how AI-built apps expose customer records explains each in detail.

1. Try to create an account without an invite. It should not be possible.
2. Change a number at the end of a record's web address while signed in as a limited user. You should not see someone else's record.
3. Search for the app's name and address in a search engine. An internal tool should not appear.
4. Ask the builder or vendor what is checked for security before changes are saved, and get the answer in writing.
5. For apps holding health, financial or other sensitive records, have a qualified security professional review the app before launch.

5. Does it work where your team works?

1. Open it on the actual phones and tablets your team carries, not just the office computer.
2. Try it with gloves on if your crew wears them, and outside in daylight if they work outdoors.
3. Try it on a weak connection, such as a basement or the far end of a warehouse.
4. Take and attach a photo if the tool uses photos.
5. Count the taps for the most common task. If it takes more than a handful, people will skip it.

6. Odd and missing inputs

Real users do unexpected things. Find out what happens before they do.

1. Leave required fields empty and try to save.
2. Paste a very long note into a short field.
3. Enter a date in the wrong format, or a date in the past.
4. Save the same record twice quickly.
5. Have two people edit the same record at the same time and see whose change wins.
6. Upload a very large photo or an unusual file type.

7. Volume and recovery

1. Load a realistic number of test records, such as a year's worth, and check that screens still open quickly.
2. Confirm you can export your data in a usable form.
3. Ask how a deleted record is recovered and how far back recovery goes. Our guide to backups, rollback and recovery lists the questions to ask.
4. Open the activity record and confirm you can see who changed what, and when.

Run a pilot

Testing finds what you thought to check. A pilot finds everything else.

1. Pick three to five real users, including at least one skeptic and one person who is not comfortable with technology.
2. Run the old and new way side by side for one or two weeks if the work is critical, so nothing is lost if the tool has a gap.
3. Check in twice a week. Ask what was slow, what was missing and what they worked around.
4. Fix and retest each problem before the wider rollout.
5. Measure one thing you can compare to the old way, such as time to assign a job or number of calls to ask for status.

An example: testing a dispatch board

Here is how the plan looks for one tool, as an example rather than a real customer. A 20-person plumbing company has a new dispatch board.

• The job: the dispatcher enters five calls from last Monday, assigns them, reassigns one when a technician calls in sick, and closes them out.
• The numbers: the end-of-day count of completed jobs per technician matches what the dispatcher counts by hand.
• Access: a technician's test account shows only that technician's jobs, and cannot see customer billing notes. A signed-out browser sees only the sign-in screen.
• Devices: two technicians try it on their own phones in a van and in a basement.
• Odd inputs: a job with no address cannot be assigned; a duplicate call is flagged.
• Pilot: one dispatcher and three technicians use it for a week alongside the whiteboard, then the whiteboard comes down.

Total time before the pilot: about half a day.

The launch table

Before launch, every must-pass item should be marked pass. Copy this table and fill it in.

Sign-off

Keep it short. A one-paragraph sign-off from the business owner records that the tool was tested, by whom, against what, and that the must-pass items passed. Save it with the app register. If something goes wrong later, you will know what was checked and when.

After launch

Testing does not end at launch.

• Retest after every meaningful change, at least the areas the change touched plus the access checks.
• Re-run the access tests quarterly, and whenever a new role is added.
• Keep the pilot users as your early warning group. They will notice problems first.
• Retire tools nobody uses. An unused tool full of data is risk with no benefit.

How FORGE handles it

FORGE builds testing into how every tool is made, so your team starts from a tool that has already been checked, not a first draft.

• Planned before it is built. A panel of top AI models plans each tool before any building starts.
• Checked at every step. A dedicated AI crew builds the tool, every piece is checked for quality and security before it is saved, and the tool is checked at every step of the build.
• Roles already enforced. Owners, admins, builders and app users each see and do exactly what their role allows, so the access tests above confirm what the platform already enforces.
• Every app inside your company. Anyone outside gets nothing back, and people join only by invite.
• A complete record. Every step of every build is recorded with who or what did it.
• Changes are easy to make after the pilot. A change is a small fraction of a build and only counts when something actually changes, so fixing what your pilot group finds is quick and cheap. A build FORGE fails is free.

Your team still runs the business test and a short pilot, because only your people know whether a tool fits the way they really work. Most teams have their first real tool the same day, and our onboarding workshop walks your team through testing it on your real work. Talk to us.

Questions people ask

Sources

1. Stack Overflow Developer Survey 2025: AI
2. Stack Overflow Developer Survey 2026: AI data
3. Microsoft Work Trend Index 2026
4. The Register: AI-built app on Lovable exposed 18K users (February 2026)
5. reeve.page: vibe-coded app security 2026