Preface |
|
xv | |
About the Author |
|
xix | |
|
Chapter 1 Conducting Surveys: Everyone Is Doing It |
|
|
1 | (28) |
|
|
1 | (1) |
|
|
2 | (1) |
|
Examples: Surveys to Meet Policy or Program Needs |
|
|
2 | (1) |
|
Examples: Surveys in Evaluations of Programs |
|
|
2 | (1) |
|
Examples: Surveys for Research |
|
|
3 | (1) |
|
|
3 | (2) |
|
Examples: From an Overly Ambitious Self-Administered Questionnaire |
|
|
3 | (2) |
|
Examples: Surveys Combined with Other Information Sources |
|
|
5 | (1) |
|
Self-Administered Questionnaires and Interviews: The Heart of the Matter |
|
|
5 | (5) |
|
|
5 | (1) |
|
Example: Forced-Choice Question |
|
|
5 | (1) |
|
Example: Open-Ended Question |
|
|
5 | (1) |
|
|
6 | (1) |
|
Planning for Data Analysis |
|
|
7 | (1) |
|
|
7 | (1) |
|
|
8 | (1) |
|
|
8 | (1) |
|
Example 1 The Look of Survey Results |
|
|
8 | (1) |
|
Example 2 The Look of Survey Results |
|
|
9 | (1) |
|
Example 3 The Look of Survey Results |
|
|
9 | (1) |
|
|
10 | (6) |
|
|
10 | (1) |
|
Usefulness or Credibility of Results |
|
|
10 | (1) |
|
|
10 | (1) |
|
The Special Case of Online Surveys |
|
|
11 | (1) |
|
Checklist for Deciding the Best Uses for Online Surveys |
|
|
11 | (2) |
|
Guidelines for Surveyors Who Work with Commercial Survey Companies |
|
|
13 | (1) |
|
The Special Case of Cell or Mobile Phones |
|
|
13 | (1) |
|
|
14 | (2) |
|
A Survey Continuum: From Specific to General Use |
|
|
16 | (1) |
|
Example: Survey with a Specific Use |
|
|
16 | (1) |
|
Example: Survey with a General Use |
|
|
16 | (1) |
|
Ethics, Privacy, and Confidentiality |
|
|
17 | (4) |
|
|
18 | (1) |
|
Contents of an Informed-Consent Form |
|
|
18 | (1) |
|
The Internet and Ethical Surveys |
|
|
18 | (3) |
|
Example Questionnaire: Maintaining an Ethically Sound Online Survey |
|
|
21 | (3) |
|
Example: Informed-Consent Form for an Online Survey |
|
|
22 | (2) |
|
|
24 | (1) |
|
|
25 | (2) |
|
|
27 | (2) |
|
Chapter 2 The Survey Form: Questions, Scales, and Appearance |
|
|
29 | (28) |
|
|
29 | (1) |
|
The Content Is the Message |
|
|
29 | (1) |
|
|
30 | (1) |
|
Select Your Information Needs or Hypotheses |
|
|
30 | (1) |
|
Make Sure You Can Get the Information |
|
|
31 | (1) |
|
Do Not Ask for Information Unless You Can Act on It |
|
|
31 | (1) |
|
Example: Plan for Survey of Satisfaction with the Youth Center |
|
|
31 | (1) |
|
|
32 | (2) |
|
Open-Ended and Closed Questions |
|
|
32 | (1) |
|
Example: Open-Ended Question |
|
|
32 | (1) |
|
|
32 | (1) |
|
Example: Open-Ended Question for Elementary School Teaching Program |
|
|
32 | (1) |
|
Example: Closed Question for Elementary School Teaching Program |
|
|
33 | (1) |
|
Making the Decision: Open-Ended Versus Closed Questions |
|
|
34 | (1) |
|
Organizing Responses to Open-Ended Survey Items: Do You Get Any Satisfaction? |
|
|
34 | (1) |
|
Step 1 Asking Respondents' Opinions |
|
|
34 | (1) |
|
|
34 | (1) |
|
Example LB/LL: Response Categories |
|
|
35 | (1) |
|
Example LB/LL: Participant Responses |
|
|
35 | (1) |
|
|
35 | (3) |
|
Example LB/LL: Number of Responses for Each Code |
|
|
35 | (1) |
|
Example LB/LL: Participants' Response Pattern |
|
|
36 | (1) |
|
Example LB/LL: Summary of Responses |
|
|
37 | (1) |
|
Rules for Writing Closed Survey Questions |
|
|
38 | (3) |
|
Example: Item-Writing Skills---Length, Clarity, Abbreviations, and Jargon |
|
|
38 | (1) |
|
Example: Item-Writing Skills---Concrete Questions |
|
|
39 | (1) |
|
Example: Item-Writing Skills---Specificity of Questions |
|
|
39 | (1) |
|
Example: Item-Writing Skills---Hidden Biases |
|
|
40 | (1) |
|
Example: Item-Writing Skills---Hidden Biases |
|
|
40 | (1) |
|
Example: Item-Writing Skills---Very Personal Questions |
|
|
40 | (1) |
|
Example: Question-Writing Skills-One Thought per Question |
|
|
41 | (1) |
|
Responses for Closed Questions |
|
|
41 | (1) |
|
|
41 | (1) |
|
Example: Yes-and-No Responses |
|
|
41 | (1) |
|
|
42 | (5) |
|
Categorical or Continuous? What about Ordinal? |
|
|
42 | (1) |
|
Example: Categorical Rating Scale |
|
|
42 | (1) |
|
Example: Ordinal Rating Scale |
|
|
42 | (1) |
|
Example: Graphic Rating Scale for Assessing a City Council's Effectiveness |
|
|
43 | (1) |
|
Example: Poor Formatting of Graphic Scale |
|
|
43 | (1) |
|
Example: Interpreting Graphic Scales |
|
|
43 | (1) |
|
|
44 | (1) |
|
|
44 | (1) |
|
Example: Selecting the Number of Categories |
|
|
44 | (2) |
|
Example: Rank Order Scale |
|
|
46 | (1) |
|
Example: Comparative Rating Scale |
|
|
46 | (1) |
|
|
46 | (1) |
|
Example: Checklist Responses in which Respondent Must Choose One from a List of Several |
|
|
46 | (1) |
|
Example: Checklist Responses that Respondents Answer Yes, No, or Don't Know for Each Item in a List |
|
|
46 | (1) |
|
|
47 | (3) |
|
Example: Survey Progress Bar |
|
|
48 | (2) |
|
Plain and Simple Survey Questions and Responses |
|
|
50 | (1) |
|
|
50 | (4) |
|
|
50 | (1) |
|
Example: A Survey with an Additive Scale |
|
|
51 | (1) |
|
Example: A Survey of Foreign Language Skills |
|
|
51 | (1) |
|
|
52 | (1) |
|
Example: Scoring a Differential Scale |
|
|
52 | (1) |
|
|
52 | (1) |
|
Example: Creating a Summated Scale for a Self-Esteem Survey |
|
|
53 | (1) |
|
Example: Scoring a Summated Scale |
|
|
53 | (1) |
|
|
54 | (1) |
|
|
55 | (2) |
|
Chapter 3 Getting It Together: Some Practical Concerns |
|
|
57 | (22) |
|
|
57 | (1) |
|
|
58 | (1) |
|
Example: How a Survey's Circumstances Can Influence Its Length |
|
|
58 | (1) |
|
Getting the Survey in Order |
|
|
58 | (2) |
|
Example: An Introduction to a Telephone Survey and Its First Question |
|
|
58 | (1) |
|
Example: Ordering Survey Questions |
|
|
59 | (1) |
|
Example: Ordering Survey Questions |
|
|
59 | (1) |
|
Example: Providing Transitions |
|
|
60 | (1) |
|
Checklist to Guide Question Order |
|
|
60 | (1) |
|
Questionnaire Format: Aesthetics and Other Concerns |
|
|
60 | (1) |
|
|
60 | (1) |
|
Example: Response Formats |
|
|
60 | (1) |
|
Branching Questions, or the Infamous "Skip" Pattern |
|
|
61 | (1) |
|
Example: Skip Patterns or Branching Questions |
|
|
61 | (1) |
|
Administration: Who Gives What to Whom? |
|
|
61 | (1) |
|
Self-Administered Questionnaires |
|
|
61 | (1) |
|
Checklist for Using Self-Administered Questionnaires |
|
|
62 | (3) |
|
|
63 | (2) |
|
The Survey Is Put on Trial |
|
|
65 | (1) |
|
Reliability and Validity: The Quality of Your Survey |
|
|
65 | (3) |
|
|
66 | (1) |
|
Example: Internal Consistency Counts |
|
|
66 | (1) |
|
Example: Internal Consistency Does Not Count |
|
|
67 | (1) |
|
|
67 | (1) |
|
Selecting and Adapting Surveys |
|
|
68 | (5) |
|
Example: Excerpt from a Real Online Service Agreement for Free Survey Items (Names Changed) |
|
|
68 | (1) |
|
Guidelines for Finding Useable and Useful Surveys in the Research Literature |
|
|
68 | (2) |
|
Example: Search for School Dropout Surveys: Sample Findings from the Web of Science |
|
|
70 | (1) |
|
Example: Sample Reports of Reliability and Validity in the Research Literature |
|
|
71 | (1) |
|
Finding Surveys on the Web |
|
|
72 | (1) |
|
Example: Search for Education Surveys: A Sample Page from the National Center for Education Statistics |
|
|
73 | (1) |
|
Guidelines for Pilot Testing |
|
|
73 | (2) |
|
A Far-Reaching World: Surveys, Language, and Culture |
|
|
75 | (1) |
|
Guidelines for Translating Surveys |
|
|
75 | (1) |
|
Example: Question about Ethnicity |
|
|
75 | (1) |
|
|
76 | (1) |
|
|
77 | (1) |
|
|
78 | (1) |
|
|
79 | (20) |
|
|
79 | (1) |
|
Sample Size and Response Rate: Who and How Many? |
|
|
79 | (2) |
|
Example: Random and Convenience Sampling |
|
|
80 | (1) |
|
|
81 | (1) |
|
Example: Simple Random Sampling |
|
|
81 | (1) |
|
Example: Not Random Sampling |
|
|
81 | (1) |
|
Example: Simple Random Sampling |
|
|
82 | (1) |
|
|
82 | (1) |
|
Stratified Random Sampling |
|
|
82 | (1) |
|
Example: Stratified Random Sampling |
|
|
83 | (1) |
|
|
83 | (1) |
|
Simple Random Cluster Sampling |
|
|
83 | (1) |
|
Example: Simple Random Cluster Sampling |
|
|
84 | (1) |
|
|
84 | (1) |
|
|
84 | (1) |
|
|
85 | (1) |
|
|
85 | (1) |
|
|
86 | (1) |
|
Other Convenience Sampling Methods |
|
|
86 | (1) |
|
Example: Other Convenience Samples |
|
|
86 | (1) |
|
Finding the Sample: Who Is In? Who Is Out? |
|
|
87 | (1) |
|
How Large Should Your Sample Be? |
|
|
88 | (1) |
|
|
88 | (1) |
|
Statistical Methods: Sampling for Two Groups and an Intervention |
|
|
89 | (6) |
|
Example: Sample Size Calculations for Sampling Two Groups and an Intervention |
|
|
89 | (1) |
|
Subgroups, Measures, Resources, and Schedule |
|
|
90 | (1) |
|
Five Questions to Ask When Determining Sample Size |
|
|
91 | (1) |
|
Example: Calculating Sample Size in a Survey of Employees in an Experimental and Control Group |
|
|
92 | (1) |
|
Example: Power to Detect Differences |
|
|
93 | (1) |
|
Example: 80% Power and Effect |
|
|
93 | (1) |
|
Example: Sample Size, Effect, and Power |
|
|
94 | (1) |
|
|
95 | (1) |
|
Tips for Improving Response Rate |
|
|
95 | (1) |
|
|
95 | (2) |
|
|
97 | (1) |
|
|
98 | (1) |
|
|
98 | (1) |
|
Chapter 5 Survey Design: Environmental Control |
|
|
99 | (16) |
|
|
99 | (1) |
|
Which Designs Are Available? |
|
|
100 | (1) |
|
Example: Surveys with Differing Designs |
|
|
100 | (1) |
|
Cross-Sectional Survey Designs |
|
|
101 | (2) |
|
Example: Cross-Sectional Design |
|
|
101 | (1) |
|
|
102 | (1) |
|
Longitudinal Surveys or Cohorts |
|
|
103 | (1) |
|
|
103 | (1) |
|
|
103 | (1) |
|
Comparison Group Survey Designs: Quasi- and True Experiments |
|
|
103 | (4) |
|
Example: A Quasi-Experimental Design |
|
|
104 | (1) |
|
Comparison Group and a Longitudinal Design |
|
|
105 | (1) |
|
Example: A True Experimental Comparison Group Design |
|
|
105 | (1) |
|
Example: A True Experimental Comparison Group Design and a Longitudinal Design |
|
|
105 | (1) |
|
Factorial Designs: Special Cases |
|
|
106 | (1) |
|
Other Survey Designs: Normative and Case Control |
|
|
107 | (1) |
|
|
107 | (1) |
|
|
107 | (1) |
|
Example 1 Normative Design |
|
|
107 | (1) |
|
Example 2 Normative Design---Comparison to a Model |
|
|
107 | (1) |
|
|
107 | (1) |
|
|
108 | (1) |
|
|
108 | (1) |
|
Example: Case Control Design |
|
|
108 | (1) |
|
|
108 | (2) |
|
|
109 | (1) |
|
|
110 | (1) |
|
Surveys, Research Design, and Internal and External Validity |
|
|
110 | (2) |
|
Example: Cross-Sectional Survey and Threats to Validity |
|
|
111 | (1) |
|
Example: Cohort Design and Threats to Validity |
|
|
111 | (1) |
|
Example: Comparison Group Design (True Experiment) and Validity |
|
|
111 | (1) |
|
|
112 | (1) |
|
|
113 | (1) |
|
|
113 | (2) |
|
Chapter 6 Analyzing and Organizing Data from Surveys |
|
|
115 | (30) |
|
|
115 | (1) |
|
What Is Typical Anyway? Some Commonly Used Methods for Analyzing Survey Data |
|
|
116 | (11) |
|
|
116 | (1) |
|
Example: Preschool Purposes Questionnaire |
|
|
117 | (1) |
|
Example: Frequency of Questionnaire Responses |
|
|
117 | (1) |
|
Example: Grouped Ratings of Preschool Purposes by 50 Directors |
|
|
117 | (1) |
|
Averages: Means, Medians, and Modes |
|
|
118 | (1) |
|
Example: Computing the Median for an Even Number of Scores |
|
|
119 | (1) |
|
Example: Computing the Median for an Odd Number of Scores |
|
|
119 | (1) |
|
Variation: Range, Variance, and Standard Deviation |
|
|
120 | (1) |
|
Correlation and Regression |
|
|
121 | (1) |
|
Example: Rank Order Correlation |
|
|
121 | (1) |
|
Differences between Groups |
|
|
122 | (2) |
|
|
124 | (1) |
|
|
125 | (2) |
|
Surveying Differences: Usual Methods |
|
|
127 | (3) |
|
|
127 | (1) |
|
|
128 | (1) |
|
|
128 | (1) |
|
|
128 | (1) |
|
|
128 | (1) |
|
Example: Mann-Whitney U Test |
|
|
129 | (1) |
|
|
129 | (1) |
|
|
129 | (1) |
|
Example: Odds Ratio and Relative Risk |
|
|
129 | (1) |
|
To Be or Not to Be: Statistician or Qualitative Analyst? |
|
|
130 | (1) |
|
Content Analysis, Open-Ended Responses, and Comments |
|
|
131 | (1) |
|
Hypothetical Content Analysis: Teasing Boys and Girls |
|
|
131 | (1) |
|
Putting the Horse in Front of the Cart: Selecting Analysis Methods |
|
|
131 | (1) |
|
|
131 | (11) |
|
Creating a Code Book or Operational Manual |
|
|
135 | (1) |
|
Example: Excerpt from the CARPS, a Survey to Detect Binge Drinking in College Students |
|
|
135 | (1) |
|
Example: Excerpt from the Code Book for the CARPS, a Survey to Detect Binge Drinking |
|
|
136 | (1) |
|
Establishing Reliable Coding |
|
|
137 | (1) |
|
Measuring Agreement: The Kappa |
|
|
137 | (1) |
|
Measuring Agreement between Two Coders: The Kappa Statistic |
|
|
138 | (1) |
|
Reviewing Surveys for Missing Data |
|
|
138 | (1) |
|
|
139 | (1) |
|
Example: Survey Responses from Six People |
|
|
140 | (1) |
|
|
141 | (1) |
|
|
141 | (1) |
|
|
142 | (1) |
|
|
143 | (2) |
|
Chapter 7 Presenting the Survey Results |
|
|
145 | (24) |
|
|
145 | (1) |
|
Reproducing the Questionnaire |
|
|
145 | (2) |
|
Example: Reporting Results with the Questionnaire |
|
|
145 | (1) |
|
Example: Online Survey Statistics in Real Time |
|
|
146 | (1) |
|
|
147 | (2) |
|
Example: Shell Table Describing Children in Two Schools |
|
|
147 | (1) |
|
Example: Shell Table for Comparing Children in Two Schools |
|
|
147 | (1) |
|
Some Table Preparation Rules |
|
|
148 | (1) |
|
|
149 | (1) |
|
|
150 | (1) |
|
|
151 | (4) |
|
Example: Downloaded Results of a Customer Satisfaction Survey |
|
|
153 | (1) |
|
Example: Transferring the Results of One Question into Bar and Line Graphs |
|
|
154 | (1) |
|
Drawing Diagrams or Pictures |
|
|
155 | (1) |
|
Example: Words and Diagrams in Survey Reports |
|
|
155 | (1) |
|
Writing the Results of a Survey |
|
|
156 | (6) |
|
|
156 | (1) |
|
Example: Structured Abstract of a Survey Report |
|
|
156 | (2) |
|
|
158 | (2) |
|
|
160 | (1) |
|
|
160 | (2) |
|
Oral versus Written Reports: A Difference in Conversation |
|
|
162 | (4) |
|
Example: Table Used in a Written and an Oral Report |
|
|
162 | (1) |
|
Written Interpretation of the Table |
|
|
162 | (1) |
|
Oral Interpretation of the Table |
|
|
162 | (1) |
|
|
163 | (3) |
|
|
166 | (1) |
|
|
167 | (2) |
Index |
|
169 | |