How to Test an AI Analyst: 6 Prompts for Bankers
How do you test an AI analyst before it touches board work? Give it the tradeoffs your analysts face every day, not the questions from the vendor's demo. Six prompts, run over one week, tell you more than a month of pilots.
To show how, follow one person through that week. She is a composite, not a client: the head of risk at a mid-sized Southeast Asian bank, trialling an AI analyst before she lets her team rely on it. Every test is one you can run yourself, on any tool.
Monday: short, but complete
She forwards a 40-page regulator circular and asks for five bullets. Easy. Then she adds the second constraint: "Five bullets, but don't drop anything that changes our capital plan."
This is the test most tools fail. They keep the length and lose the substance, or keep the substance and ignore the length. A good answer finds the one clause that matters and fits it in. A great one says, in bullet five, that a second clause might matter and why.
**Following one instruction is easy. Judgment starts at two.**
Tuesday: the latest data, but only from official sources
Next she asks for the current deposit rates of three peers, "latest figures, official sources only". The trap is that the freshest number often sits in a news story or a blog, while the official figure is a week older.
She is watching for one thing: does the tool choose the official source and say how old it is, or does it grab the newest number from wherever it found it? The right answer here is slightly older and fully defensible.
**For a bank, "recent" without "official" is just gossip with a timestamp.**
Wednesday: one number, with its page and date
Midweek she picks a single figure from the tool's answer, a peer's cost-to-income ratio, and asks where it came from. She clicks through. Does the link open the right document? Is it the right page? Does the date match the quarter the answer claimed?
This takes two minutes and catches most of the problems that would embarrass her later. MAS's AI risk management guidelines, issued on 7 October 2026, keep institutions accountable for third-party AI. This test is what that accountability looks like at a desk.
**Trust is not a feeling about a tool. It is a link that opens the right page.**
Thursday: the question it cannot answer
On Thursday she asks something with no public answer: a peer's internal funding target for next year. She is not testing knowledge. She is testing honesty.
A tool that invents a number fails the whole trial, however good the rest of the week was. A tool that says "this isn't published; the closest public figure is X, from their annual report" passes. In finance, a clear "I don't know" is worth more than a confident guess.
**The most valuable sentence an AI analyst can say is "I can't verify that."**
Friday: answer what was asked
Her Friday test is a blunt question about a borrower's exposure, phrased the way she would ask a colleague. She is checking for something bankers dislike more than errors: being lectured.
Does the reply answer the question, or open with three lines of caveats and a reminder to consult compliance? A real risk deserves a flag. Boilerplate does not. A senior user wants the answer first and the warning only when it is earned.
**Respect for a senior user looks like a direct answer.**
Next Monday: did it remember?
The last test happens a week later. On the first Monday, she had told the tool to keep summaries to five bullets and flag anything touching the Jakarta book. Now she sends a new circular without repeating either instruction.
If the answer comes back in five bullets with the Jakarta flag, the tool learned how she works. If not, she will be re-briefing it every morning, and her team will stop using it within a month.
**An assistant you have to re-brief every week isn't an analyst. It's a search box with good manners.**
Run the week before you sign
Six prompts, one week, no vendor in the room. Score each one pass or fail, and treat Wednesday and Thursday as must-pass. We run Rapi™ against exactly these kinds of tradeoffs, because they are what directors actually ask. Why the stakes are so high is in AI hallucinations in banking.
The point of a trial is not to see the tool at its best. It is to see whether you can trust it on an ordinary day.
Sources
- Monetary Authority of Singapore: MAS Sets Out Supervisory Expectations on Responsible AI Adoption by Financial Institutions (7 October 2026)
- Otoritas Jasa Keuangan: Peluncuran Tata Kelola Kecerdasan Artifisial Perbankan Indonesia (SP 67/GKPB/OJK/IV/2025)
Common questions
- How do you test an AI analyst for a bank?
- Run real tradeoff prompts over a week instead of relying on a vendor demo. Test whether it keeps two constraints at once, prefers official sources, links every figure to its page and date, admits when something cannot be verified, answers without unrequested lectures, and remembers your instructions a week later.
- What is the most important test for an AI analyst?
- The source test and the honesty test. Pick one figure and click through to its source, page and date. Then ask a question with no public answer and see whether the tool invents a number. A tool that fails either one should not touch board work, however strong it looks elsewhere.
- How long should an AI analyst trial take?
- About one week of normal work, with roughly 20 minutes a day on deliberate tests, plus one check a week later to see whether it remembers your instructions. That is usually enough to see how a tool behaves on ordinary days, which matters more than how it performs in a prepared demo.
