Request evidence for an AI vendor's promises
For procurement leads, in-house counsel and security reviewers.
What evidence would let a buyer rely on an AI vendor's statement about its product?
Identify the exact feature, plan, environment and proposed use. Compare the written terms with the settings that apply to that account and an example showing how the feature behaves. A conflict between them belongs in the decision record.
Turn a broad promise into a testable question
A phrase such as private by design leaves several questions open. Ask where user prompts, uploaded files, outputs and diagnostic logs go. Find out who may access them, whether any are used for model improvement and which subcontractors handle them. Request the applicable terms and configuration instructions for the specific product tier.
Ask what happens when a user deletes an item, an administrator terminates the account or the supplier changes model providers. Separate removal from the active interface, retention in backups and retention required by other obligations. Ask the supplier to identify exceptions and the evidence available after the process runs.
Compare commitments, settings and a demonstration
Keep a row for each important promise, with the vendor's wording, dated source, exact plan, evidence reviewed, gap, owner and disposition. Check whether a marketing statement appears in the governing contract and what terms take precedence if documents conflict.
Use invented data for a demonstration. Ask an administrator to show the relevant setting and walk through a test file's life cycle. A missing item in a user interface demonstrates a visible result only. Obtain separate evidence for a claim about backup deletion or model training.
Decide which gaps affect the purchase
Distinguish a missing document from a confirmed product limitation. Ask who may approve an exception and which use it covers. Record whether the buyer needs a contract change, a configuration change or a narrower pilot. NIST's supplier due diligence guide and its Generative AI Profile provide background for evaluating supplier and AI risks.
Worked example
Illustrative example. A vendor says that customer files are never used for training. The proposed plan's terms make model improvement depend on an administrator setting. The buyer requests the exact terms and setting, verifies who can change it, and records which plan the evidence covers before deciding whether confidential use is acceptable.
Primary sources
Related work at Agentic Attorney
This vendor evidence worksheet is part of the shared Agentic Attorney resource collection. The hub describes AI consulting and implementation for legal teams. Explore Agentic Attorney’s approach to AI implementation.