Question Taxonomy

 

Graesser et al., 2000
Lessler & Forsyth, 1996
Willis, 1996
Beatty, 1996
Willis & Lessler, 1999
Unfamiliar Technical Term
Comprehension:
(questions):
technical terms-
present, undefines, ambiguous, or vague
 Comprehension/Communication:
"subject didn't know or misunderstands a particular word or pharse"
Problems with Terms:
not the mose appropriate or accurate term, unknown, unfamiliar, or overly technical language
-and-
Conceptual Problems:
cental concept is mis-specified
 Clarity:
technical terms
Vague or Ambiguos Noun Pharse
     Comprehension: 
(questions):
technical terms-
ambiguous or vague
-and-
general reference set-
vague reference set
 Comprehension/Communicaiton:
"subject asks what is to be included/excluded," "subject includes/excludes inapproaitely," "questions is vague/open to mutiple interpretation," "unclear what is to included/excluded," "wording problems that impede communication"
-and-
Logical Problems:
"question has no general anwser (it depends)"
 Conceptual Problems:
ambiguous concepts with multiple interpretations
Clarity: 
vague, references period
-and-
Response Categories:
overlapping
Vague or Imprecise Relative Term
Comprehension:
(questions):
technical terms-
ambiguous or vague
-and-
general reference set-
vague reference set
-and-
reference period problems-
unachored boundary, nonfixed boundary, ill-defined boundary, embedded period, carry over reference period
Comprehension/Communicaiton:
"subject asks what is to be included/excluded," "subject includes/excludes inapproaitely," "questions is vague/open to mutiple interpretation," "unclear what is to included/excluded," "wording problems that impede communication"
-and-
Logical Problems:
"question has no general anwser (it depends)"
 Conceptual Problems:
ambiguous concepts with multiple interpretations
Clarity: 
vague, references period
-and-
Response Categories:
overlapping
Complex Syntax
 Comprehension:
(problems with instructions):
complex syntax
-and-
(question structure):
complex syntax
 Comprehension/Communicaiton:
"subject misunderstands a particular word or pharse", question is difficult to read", "wording problems that impede communication" 
 Complex Structure:
unnecessary grammatical complicaitons
Clarity:
wording
-and-
Knowledge/Memory:
communicaiton
Working Memory Overload
 Comprehension:
(questions):
question structure-
several questions
 Comprehension/Communication:
"question is too long"
 Complex Structure:
too many ideas to keep straight within one question/too much detail, important ideas "buried"/easily missed within thequestion
-and-
Conceptual Problems:
too many ideas to keep straight within one quesiton/too much detail
Knowledge/Memory: 
computation
Misleading or Incorrect Presupposition
 Comprehension:
(questions):
question struture-
implicit assumption
 Logical:
"faulty assumptions about resopondant's or reality", "inappropriate question for some class of respondent's", " double barrelled question"
 Unintentional Influences on Respondent Judgement:
quesiton comes with an unintended context or order effect
-and-
Conceptual Problems:
"double barrelled" questions-multiple frames of reference combined
Assumptions: 
inappariate assumptions, assumes constant behavior, double barelled
Unclear Question Category
 X
 X
 X
 X
Amalgamtion of More Than One Questions Category
 Comprehension:
(questions):
question structure-
several questions
 X
 X
Assumptions:
 double barrelled
Unclear Question Purpose
 Comprehension:
(questions):
question structure-
unclear goal
 Comprehension/Communicaiton:
"subject asks for clarification concerning question intent/meaning"
 X
X
Mismatch Between Question Category and Answer Option
 Comprehension:
(question):
question structure-
Q/A mismatch
 Logical:
"response categories don't match those implied by the question:, "response categories are not the sane as those in the quesiton"
 X
Response Categories:
 mismatch, missing
Difficult to Access Specific to Generic Knowledge
 Comprehension:
(reference period problems):
unanchored boundary, ill defined boundary, undefined boundary, embedded period, carry over reference period
 Recall/Computational:
"subject is unsure of answer", "information will be difficult to remember"
 Problems with Recall, Knowledge, or Salience:
asking something respondent is unlikely to know, low salience (especially for attitudes)
Knowledge/Memory: 
recall, computation
 Respondant Unlikely to Know Anwser (no information source)
 Judgement:
(information/response congruence):
incongruent
 Recall/Computational:
"subject never knew the information", "answer will not be known"
 Problems with Recall, Knowledge, or Salience:
asking something respondent is unlikely to know
Knowledge/Memory: 
knowledge
 X
Logical Problems:
"skip pattern errors", "poor flow", "doesn't ask for useful information analytically"
 X
 Reading:
what to read, missing information, how to read
 X
 Comprehension:
(problems with instructions):
conflicting instructions, inaccurate instructions, hidden instructions, unclear examples, unclear layout
(questions)
question structure-
hidden question, violates con coven
 Comprehension/Communicaiton:
"subject can't understand question", "subject interrupts question administration", "interviewer repeats question/answer category", "interviewer notes that (s)he had diffulty reading question"
 Response Options Inadequate:
incomplete response options, non-mutually exclusive response options, inappropriate response options, inappropriate units
 Instructions:
conflicting or inaccurate instructions
 X
 X
 X
 X
 Knowledge/Memory:
attitude
 X
 Judgement:
(information evaluation):
accuracy evaluation, sensitive behavior, sensitive attitude, sensitive (general), socially undesirable
Consequence evaluation-
safety consequences, legal consequences, social consequences, behavioral consequences
 Bias:
"interviewer observes evidence of misestimation", "social desirability effects", "sensive question"
 Unintentional Influences on Respondent Judgement:
loaded or biased quesiton, quesitons comes with an unintended context or order effect, sensitivity of topic/social desurability
 Sensitive/Bias:
sensitive content, sensitive wording, socially acceptable
 X
 X
 Recall/Computational:
"subject had difficulty computing answer", "information is difficult to compute"
 Problems with Recall, Knowledge, or Salience:
fails to provide adequate memory cues (including repsonse cards, reading response categories, etc), unintentionally broad estimaiton/use of heuristics
 X
 X
 Response Generation:
(instructions):
hidden instructions, hidden definitions
 Response Category:
"subject doesn't understand what response categories to use", "subject answer doesn't fit response categories", response categories don't match subject's thinking", "response categories may not match respondent answer", "response categories not phrased in the manner respondents usually think"
 Flow/Consistency Problems:
illogical juxtaposition of topics, technical errors such as skip logic, inconsistency of format across similar or related questions
 Response Categories:
illogical order, open-ended question
 X
Retrieval:
high detail, low detail, unexpected detail, shift in psychological reference period
 X
 X
 X
 X
 X
 X
 Scope of the Questionnaire:
apparent omissions ("Why didn't you ask about..."),
 X
compiled by Frances Daniel

Back to Research Information

Email Webmaster