Should you trust AI
with your money?

An AI is not a registered fiduciary

When a real fee-only fiduciary advises you, federal law binds them. Under the Investment Advisers Act of 1940, the SEC's 2019 Commission Interpretation reaffirms that an investment adviser owes clients a fiduciary duty made up of a duty of care and a duty of loyalty, the obligation to act in your best interest and to eliminate or fully disclose every conflict of interest.^[1] That duty cannot be waived away by fine print, regardless of how sophisticated the client is.^[1] Brokers face a parallel bar: Regulation Best Interest, effective June 30, 2020, requires them to act in a retail customer's best interest when recommending securities, and to disclose material facts and conflicts.^[2]

A general-purpose chatbot by itself is not your retained investment adviser. A regulated firm using automation or AI, however, does not cease to have obligations just because software helps provide the service. Research the responsible firm, its registration, disclosures and your actual service agreement, not the model name alone.^[1][2]

Hallucination gets worse on the niche stuff

The dangerous failure mode isn't that an AI says "I don't know." It's that it fabricates a confident, specific, wrong answer, a "hallucination." OpenAI's own 2025 research frames this bluntly: language models are "optimized to be good test-takers, and guessing when uncertain improves test performance," so training and evaluation "reward guessing over acknowledging uncertainty," which produces "plausible yet incorrect statements instead of admitting uncertainty."^[3] verify×DON'T TRUST, VERIFYClaim: Language models are optimized to be good test-takers and rewarded for guessing over admitting uncertainty, producing plausible-but-incorrect statements.Verify at: arXiv:2509.04664 ↗ Source 3 in the list belowThis is OpenAI's own research. Read the abstract, the phrasing is verbatim from the paper. The model guesses for the same reason a student bubbles in a random answer rather than leaving it blank, and it has been rewarded for doing so.

Now apply that to personal finance. General questions ("what is an index fund?") are everywhere in the training data, so the model is usually fine. But the moment you ask about a niche or edge-case rule, an obscure tax provision, a state-specific quirk, a recently changed contribution limit, the answer gets shakier, because the correct fact appears rarely, if at all, in the training corpus. Peer-reviewed work shows a strong positive correlation between how often a fact appears in training data and how accurately a model reproduces it; performance gains from bigger models concentrate on high-frequency facts while rare, long-tail facts stay unreliable.^[4]

The cruel part: the prompts where you most need a reliable answer, the unusual situation no blog has covered, are exactly the prompts where hallucination risk is highest. The model is most confident and least correct precisely where you have the least ability to catch it.

Output quality is capped by input quality

An LLM has no independent knowledge of your situation. It only has the words you give it. Leave out your filing status, your state, your existing accounts, or your time horizon, and it will quietly assume defaults and answer a question you didn't ask. Garbage in, garbage out, except the garbage comes back beautifully formatted and reads like expertise.

Worse, the answer is sensitive to phrasing. Ask "is a Roth better than a Traditional IRAIndividual Retirement Account (IRA)A retirement account with tax advantages and eligibility rules. Traditional IRA contributions may be deductible; taxable withdrawals generally enter income. Roth contributions use after-tax money, and qualified Roth withdrawals are tax-free.Full definition?" and ask "why is a Traditional IRA better than a Roth?" and you can get two confident, contradictory replies, not because the facts changed, but because the model is steered by the framing of your prompt. A human fiduciary pushes back on a bad assumption baked into your question. A model often runs with it.

Some AI products have hidden conflicts

An advisory firm’s disclosures and business model matter whether advice arrives from a person or through software. A standalone general-purpose chatbot and a registered automated adviser are not the same service. Read the responsible firm’s disclosures, fees and limitations; investigate sponsored placements and referral incentives instead of treating fluent answers as disinterested advice.^[1]

Ask how the service is paid, whether recommendations involve affiliates or sponsorship, what information about your circumstances it uses, and who is accountable for the advice. Verify material tax rules, limits and calculations independently. Compensation labels and polished interfaces are not substitutes for that work.

Not for stock picks, and watch the herd

A model cannot predict prices, it has no edge, no live market data unless explicitly connected, and a training cutoff that leaves it blind to anything recent. But there's a subtler, structural risk: if millions of people ask the same model the same question and act on the same answer, you get herding, correlated behavior that crowds into the same trades and inflates the same names. The "insight" stops being an edge the moment everyone shares it, and concentrated, correlated bets are how people get hurt.

This site's position doesn't change here: a low-cost index fund beats stock-picking for almost everyone, and a Bitcoin allocation is a conviction decision you make deliberately, not a tip you take from a chatbot. If anything, AI makes the case for boring, mechanical, rules-based investing stronger, because it removes one more story you could talk yourself into.

It models language, not judgment

This is the whole thing in one line. A large language model is a map of how words tend to follow other words. It does not understand your retirement, weigh your risk tolerance, or feel the weight of being wrong with your life savings. It predicts plausible text. When it sounds wise, that's because wise-sounding text is well represented in its training data, not because there's judgment or intent behind it.

Real financial advice is judgment under uncertainty, applied to one specific human, by someone who is accountable for the outcome. The model has no accountability and no skin in the game. If it steers you wrong, there is no one to complain to, no regulator to file with, no Form ADV to amend. The confidence is real; the responsibility is not.

Use it as an accelerator, not an oracle

None of this means avoid AI. It means use it for what it's good at and verify the rest. A practical posture:

The discipline is the same one this whole site runs on: don't trust, verify. Make the AI cite a source, then go read the source. If it can't produce one, or the source doesn't say what it claimed, you've caught a hallucination. For anything material, an AI answer is a starting hypothesis you confirm against the IRS, the SEC, or a fee-only fiduciary, never the conclusion. See Verify Everything for the full posture, and Influencer Red Flags for the human version of the same problem.

How this site uses AI (the honest note)

It would be hypocritical to write all this and hide my own hand, so here it is plainly.

The standard I hold AI to in this article is the standard I try to hold myself to: show the source, let you check the work, and never ask you to trust a confident voice over a verifiable fact.