Appendix: Research Methodology
This appendix describes the methods used to collect the survey and interview data analyzed in this book.
Survey Methodology and Sample Statistics
For the Wave 4 National Study of Youth and Religion (NSYR) survey, every attempt was made to reinterview all Wave 1 youth respondents. At the time of the fourth survey, respondents were between the ages of 23 and 28. The survey was fielded between February 2013 and December 2013 and was administered through the Center for the Study of Religion and Society at the University of Notre Dame, using Qualtrics online survey software. The Wave 4 survey instrument replicated many of the questions asked in Waves 1, 2, and 3, with some changes made to questions to take into account the respondents’ lives as they grew older, such as getting a full educational or relationship history; other questions were added to get greater specificity on old questions. Additional questions were added that were of interest to researchers involved with the project that had not been included in the survey previously.
The Notre Dame Center for the Study of Religion and Society conducted a pretest to ensure that the survey instrument was working properly and that the survey length was appropriate. Fifty respondents were recruited with the assistance of Survey Sampling International (SSI), a research firm that specializes in digital research. The survey link was sent to SSI, and they distributed the link by email to respondents in February 2013. Responses from the 50 respondents were not included in the Wave 4 dataset.
Between Waves 3 and 4, NSYR project staff maintained all in-between survey tracking efforts. This included maintaining contact each year through US Postal Service–mailed birthday cards and then recording change of address information for returned cards. All program mailings included a link to the respondent website that allowed respondents to submit changes to contact information and other questions or comments. Approximately six months before the survey was fielded, all Wave 1 respondents for whom we had a mailing address were sent a postcard and a $2 bill asking them to contact us with their current contact information and informing them that the Wave 4 survey would soon be fielded.
Of the total 3,328 original respondents, the 2,144 complete responses in Wave 4 represented a 65 percent retention rate from Wave 1, which was fielded in 2002. Of the 2,144 who participated in the Wave 4 survey, 81 percent had participated in all three previous waves, while another 14 percent had participated in two of the three previous waves. In the first three waves, 100 percent of surveys were conducted by telephone. In Wave 4, only 15 percent of surveys were conducted on the telephone; the other 85 percent were completed online. All but two phone surveys and all of the online surveys were fielded in English.
On key demographic variables, the Wave 4 respondents were very similar to the Wave 1 respondents. shows a column for the sample distribution from each wave of the survey. The demographic characteristics were collected at Wave 1. In the Wave 4 survey, there were only minor fluctuations in the region, age, and income demographics of respondents who remained in the survey compared with the original Wave 1 sample. The percentages of female respondents increased slightly with each wave of the study, suggesting that retention was marginally higher for females than males. Similarly, white respondents made up 72 percent of the Wave 4 sample compared with 65 percent of the Wave 1 sample and 69 percent of the sample from Waves 2 and 3. The percentages of black and Hispanic respondents declined by 4 and 1 percent, respectively. The other noticeable change is in the percentage of the sample whose parents were married at the time of the Wave 1 survey. Those respondents made up 57 percent of the sample in the Wave 4 survey compared with just 51 percent at Wave 1. These are not large differences, such that the final sample still is demographically very similar to the original sample. The changes that are present, however, reflect the fact that retention was slightly lower among people of color and respondents from nonintact families.
Table A.1 Comparison of Waves 1, 2, 3, and 4 Respondents on Key Demographic Characteristics
Wave 1
W1 Respondents
W2 Respondents
W3 Respondents
W4 Respondents
Census Region
Northeast
15
15
15
15
Midwest
23
24
25
26
South
42
41
41
38
West
20
20
19
21
Gender
Male
51
50
49
47
Female
49
50
51
53
Age
13
19
19
19
18
14
20
20
20
20
15
21
22
22
22
16
20
20
20
21
17
20
19
19
19
Race/Ethnicity
White
65
69
69
72
Black
17
16
16
12
Hispanic
12
10
10
9
Asian/Pacific Islander/American Indian/mixed/other
5
5
5
5
Missing
1
1
1
1
Family Structure
†
Lives with two biological/adoptive parents
51
56
55
57
Parent Income
Less than $10K
3
3
3
2
$10K–20K
7
6
6
6
$20K–30K
12
11
11
10
$30K–40K
13
12
12
11
$40K–50K
13
14
14
14
$50K–60K
11
11
11
12
$60K–70K
7
8
8
8
$70K–80K
6
7
7
8
$80K–90K
5
6
6
5
$90K–100K
4
4
4
5
More than $100K
11
12
13
14
Missing
6
6
6
6
N
‡
3,259
2,530
2,458
2,071
Numbers represent unweighted percentages. Percentages may not add up to 100 because of rounding.
Family structure contains some missing data.
N does not include the Jewish oversample cases (N = 80 for Wave 1; N = 74 for Wave 2; N = 78 for Wave 3; N = 73 for Wave 4).
Source: National Study of Youth and Religion surveys 2003, 2005, 2008, 2013.
Methodological Design and Procedures for the NSYR Wave 4 Personal Interviews
The fourth wave of in-person interviews for the NSYR involved in-depth personal interviews with 303 of the Wave 4 survey respondents, resulting in 292 useable interview transcripts. The purpose of the interviews was to provide extended follow-up discussion about the religious, spiritual, family, and social lives of emerging adults. The questionnaire included much of what was used in Waves 1, 2, and 3, with the addition of several new topics that were relevant to their lives as 23- to 28-year-olds. The length of the interviews ranged from slightly less than one hour to approximately three hours, with most lasting about two hours. Respondents were offered a cash incentive ranging from $110 to $200 to participate in the study.
The in-person interviews were conducted between May 6, 2013 and December 19, 2013, with 92 percent of the interviews conducted between May and August 2013. To be interviewed, respondents were required to have completed the survey, and all interviews were completed between three and six months following respondents’ completion of the survey. All respondents who were interviewed in Wave 1 were recontacted for Wave 4 interviews. Of the 303 individuals who were interviewed, 205 had been interviewed as teens in Wave 1, and 41 were added to the interview pool to refresh the sample and provide additional diversity.
Our sampling strategy for Wave 1 interviews represented a range of demographic and religious characteristics and took into account region, residential location (urban/suburban/rural), age, sex, household income, religion, and school type. The new in-person interview respondents were chosen by combinations of geographic region, religious type, race/ethnicity, and sex to balance out the original sample. The researchers clustered the respondents with whom they were still in contact into geographic areas for ease of travel in interviewing. provides demographic information about the 292 interview respondents.
Table A.2 Wave 4 Personal Interview Demographics
N
N
Gender
Region
Female
150
Midwest
67
Male
142
Northeast
51
South
101
Age
West
73
23
33
24
58
Religious Tradition
25
60
Conservative Protestant
46
26
57
Mainline Protestant
15
27
60
Black Protestant
17
28
24
Catholic
32
Jewish
20
Latter-day Saints
15
Race
Not Religious
111
White
200
Indeterminate Christian
24
Black
35
Indeterminate Other
3
Hispanic
30
Asian
8
Native American
6
Other
13
N = 292.
Source: National Study of Youth and Religion.
Fourteen interviewers conducted interviews with respondents living in 44 states, Washington DC, and several other countries. While the majority of the interviews were conducted in person and near where the respondent lived, some of the interviews took place by phone or Skype because of costs and other logistics related to traveling to where the respondents were located. Each interviewer conducted between nine and 47 interviews. Eight of the interviewers had conducted personal interviews for the NSYR in either Wave 1, 2, or 3 (or all waves), and respondents were matched with their previous interviewers as much as possible. Before the fielding of the in-person interviews, all new interviewers participated in a day-long training meeting held by the NSYR project manager and the principle investigator. The training covered the logistics of the interview process, procedural requirements, Institutional Review Board (i.e., Human Subjects) concerns, protection of human participants, safety and liability concerns, and keys to NSYR interview success. In addition to that training, all interviewers were required to obtain Human Participants Training Certification.
The 14 project interviewers were assigned to sets of specific geographic locations around the United States. Each interviewer was provided with groups of contact sheets for respondents in their assigned geographic areas. The contact sheets included respondent name, respondent nickname (if known), parent name, address, phone number, respondent birth date, respondent gender, respondent race, household income, religious affiliation, and religious denomination or tradition. The contact sheet also included space to note any changes to the contact information provided (e.g., new phone number, additional email address) and provided a call record. Interviewers recorded each household contact, noting the date, time, person with whom they spoke, and content of each contact. In addition to the original sample, as the project progressed through the interviews, the priorities shifted somewhat according to what types of respondent interviews were still needed (e.g., location, religion, gender). In this way, the researchers attempted to match the demographics of the original Wave 1 in-person interview sample as much as possible.
In addition to the demographic information on the contact sheet, before their interviews, NSYR interviewers were asked to review the transcripts of the previous in-person interviews. If an interviewer had not conducted the previous personal interviews with his or her respondents, the transcripts provided background information about that respondent to help facilitate a smooth interview. Using a standard call script provided by the NSYR, interviewers made contact with potential interviewee households. Interviewers identified themselves as researchers with the “National Youth Study.” The full name of the project was not used in order to avoid introducing any response bias by identifying religion as a key focus of the study. It was often helpful for interviewers to explain their connection with the project, for example, graduate student, coinvestigator, and so on. However, since much of the interview was about religion and the researchers did not want to bias the respondents’ answers, interviewers were instructed to avoid divulging revealing information about their own personal beliefs and commitments (of which there was considerable diversity on the interviewing team).
Interviewers were required to obtain initial verbal consent from the respondent during the phone call to conduct the interview. If respondents seemed hesitant about participating, an additional script provided more information about the project and offered the phone number of the principle investigator, whom they could call with questions or concerns. In addition, interviewers offered to mail written information about the project to hesitant respondents and then call back in a few days. When a respondent seemed reluctant to participate or was hesitant about the time commitment, an additional incentive was offered. However, in cases in which respondents refused to participate even after being offered additional information, the interviewers made no further attempts to convert those who refused. On receiving verbal consent from the respondents and scheduling an interview time and location, interviewers mailed packets of information to households. The packet contained a cover letter from the principal investigator, multiple copies of respondent written consent forms, and an appointment letter, including a portrait photo of the interviewer. Interviewees were required to bring the written signed consent with them to their interviews. Interviewers also called the respondents at least one week before the interview and again the day before the interview to confirm that the respondents were planning to participate.
Interviews were conducted in public settings that still provided confidentiality for the respondent. The ideal location for these interviews was in study rooms at local libraries. However, when these were not available, interviews were also conducted in restaurants, coffee shops, mall food courts, and outdoor settings. Interviewers were given guidelines for how to present themselves during the interviews as well as appropriate attire to be worn in order to ensure consistency in the presentation of interviewers across the interviews. Specifically, interviewers did not attempt to “relate” to respondents by dressing down or dressing in a more “trendy” fashion. Interviewers presented themselves as professional researchers with a sincere interest in young people’s lives. At the close of the interview, respondents were given the cash incentive for their participation and in appreciation of their time and effort.
