AI Output Evaluation Skill
Information SystemsReliability: alpha 0.883 sample items
About this scale
A multi-item measure of an individual's skill in critically evaluating AI-generated outputs — verifying accuracy, identifying hallucinations, and detecting biased framing.
Sample item(s)
“I check AI outputs for factual accuracy”
“I can usually tell when an AI has made up information”
“I am skilled at recognizing biased framing in AI outputs”
Reliability
Reported internal consistency: alpha 0.88. As a rule of thumb, Cronbach’s α ≥ .70 is considered acceptable, ≥ .80 good, and ≥ .90 excellent for established measures.
Notes
Increasingly important as critical-thinking skill in AI-mediated work.
Source & citation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Madotto, A., & Fung, P. (2023): Survey of hallucination in natural language generation. ACM Computing Surveys, Vol. 55, No. 12, Article 248.
Ji et al. (2023): Hallucination in NLG. ACM CSUR, 55(12), 248.
Source. Scale details are summarised from the publication cited above. Always consult the original source for the full item set, scoring, and permission to use.
Use it in your research
Need to check the reliability or psychometrics of this scale in your own data? Use the free DrAIStudio research tools:
Cronbach’s alpha calculator
All statistical calculators
Search the full Scale Atlas
Academic writing humanizer