AI Vendor Red Flags to Watch For
The biggest red flags are no evaluation story (how they measure correctness), no data-ownership terms, a tool-first pitch that skips your process, and ROI claims with no baseline. In a market where 80–95% of projects fail, vendor selection is a primary risk control.
The five red flags
Watch for five signals: no way to measure output quality, no data-ownership or exit terms, a pitch centered on a tool rather than your workflow, ROI promises without a baseline, and reluctance to document or hand over the system.
- No evaluation story, they cannot explain how they will know if the AI is wrong. Reliability is the whole job; skipping it guarantees silent failures.
- No data-ownership terms, they will not put in writing that your data and configuration stay yours. This is how lock-in starts.
- Tool-first pitch, they lead with a product to install instead of understanding the workflow you want to fix. Process comes before tool.
- ROI with no baseline, they promise a return but never ask what the task costs you today. 49% of organizations cannot measure AI value because they skipped this (AI ROI research, 2025).
- No documentation or handover, the system lives only in their heads, so you are dependent forever.
Spotting “AI-washing”
AI-washing is branding ordinary automation or a thin wrapper as advanced AI. The defense is specificity: ask exactly what the system does, on what data, how quality is measured, and what happens when it fails. Vague answers signal marketing over engineering.
A credible vendor answers “what, on what data, measured how, failing how” without hedging. If every answer is a buzzword, “proprietary AI,” “next-gen models”, with no mechanics, you are looking at a wrapper, not a system.
Frequently asked questions.
What is the biggest AI vendor red flag?
No evaluation story, the vendor cannot tell you how they will measure whether the AI’s output is correct, or what happens when it is wrong. Reliability is the core of AI engineering, so skipping it all but guarantees silent, costly failures.
What is AI-washing and how do I avoid it?
AI-washing is marketing ordinary automation or a thin model wrapper as advanced AI. Avoid it with the specificity test: ask what the system does, on what data, how quality is measured, and how it fails. Buzzwords without mechanics are the warning sign.