Amazon Revises AI Safety Tests After AVERI Review

Amazon detailed in a company blog post an independent review of its internal AI safety evaluation datasets by AVERI. The nonprofit examined a random sample of prompts without input from Amazon's evaluation team, assessing their diversity, difficulty, and methodology.

AVERI found broad coverage across safety categories, varied testing approaches, and a low risk that test content appeared in model training data. It also recommended wider multilingual coverage, more multistep adversarial prompts and clearer grading criteria.

Amazon responded by removing redundant prompts, adding harder adversarial scenarios, and introducing tiered difficulty levels. It also established quarterly reviews and expanded multilingual prompts across all risk categories.