Chapter 13 of 16

Appendix: Research Methodology

Appendix: Research Methodology

This appendix describes the methods used to collect the survey and interview data analyzed in this book.

Survey Methodology and Sample Statistics

For the Wave 4 National Study of Youth and Religion (NSYR) survey, every attempt was made to reinterview all Wave 1 youth respondents. At the time of the fourth survey, respondents were between the ages of 23 and 28. The survey was fielded between February 2013 and December 2013 and was administered through the Center for the Study of Religion and Society at the University of Notre Dame, using Qualtrics online survey software. The Wave 4 survey instrument replicated many of the questions asked in Waves 1, 2, and 3, with some changes made to questions to take into account the respondents’ lives as they grew older, such as getting a full educational or relationship history; other questions were added to get greater specificity on old questions. Additional questions were added that were of interest to researchers involved with the project that had not been included in the survey previously.

The Notre Dame Center for the Study of Religion and Society conducted a pretest to ensure that the survey instrument was working properly and that the survey length was appropriate. Fifty respondents were recruited with the assistance of Survey Sampling International (SSI), a research firm that specializes in digital research. The survey link was sent to SSI, and they distributed the link by email to respondents in February 2013. Responses from the 50 respondents were not included in the Wave 4 dataset.

Between Waves 3 and 4, NSYR project staff maintained all in-between survey tracking efforts. This included maintaining contact each year through US Postal Service–mailed birthday cards and then recording change of address information for returned cards. All program mailings included a link to the respondent website that allowed respondents to submit changes to contact information and other questions or comments. Approximately six months before the survey was fielded, all Wave 1 respondents for whom we had a mailing address were sent a postcard and a $2 bill asking them to contact us with their current contact information and informing them that the Wave 4 survey would soon be fielded.

Of the total 3,328 original respondents, the 2,144 complete responses in Wave 4 represented a 65 percent retention rate from Wave 1, which was fielded in 2002. Of the 2,144 who participated in the Wave 4 survey, 81 percent had participated in all three previous waves, while another 14 percent had participated in two of the three previous waves. In the first three waves, 100 percent of surveys were conducted by telephone. In Wave 4, only 15 percent of surveys were conducted on the telephone; the other 85 percent were completed online. All but two phone surveys and all of the online surveys were fielded in English.

On key demographic variables, the Wave 4 respondents were very similar to the Wave 1 respondents. shows a column for the sample distribution from each wave of the survey. The demographic characteristics were collected at Wave 1. In the Wave 4 survey, there were only minor fluctuations in the region, age, and income demographics of respondents who remained in the survey compared with the original Wave 1 sample. The percentages of female respondents increased slightly with each wave of the study, suggesting that retention was marginally higher for females than males. Similarly, white respondents made up 72 percent of the Wave 4 sample compared with 65 percent of the Wave 1 sample and 69 percent of the sample from Waves 2 and 3. The percentages of black and Hispanic respondents declined by 4 and 1 percent, respectively. The other noticeable change is in the percentage of the sample whose parents were married at the time of the Wave 1 survey. Those respondents made up 57 percent of the sample in the Wave 4 survey compared with just 51 percent at Wave 1. These are not large differences, such that the final sample still is demographically very similar to the original sample. The changes that are present, however, reflect the fact that retention was slightly lower among people of color and respondents from nonintact families.

Table A.1 Comparison of Waves 1, 2, 3, and 4 Respondents on Key Demographic Characteristics

Wave 1

W1 Respondents

W2 Respondents

W3 Respondents

W4 Respondents

Census Region

Northeast

15

15

15

15

Midwest

23

24

25

26

South

42

41

41

38

West

20

20

19

21

Gender

Male

51

50

49

47

Female

49

50

51

53

Age

13

19

19

19

18

14

20

20

20

20

15

21

22

22

22

16

20

20

20

21

17

20

19

19

19

Race/Ethnicity

White

65

69

69

72

Black

17

16

16

12

Hispanic

12

10

10

9

Asian/Pacific Islander/American Indian/mixed/other

5

5

5

5

Missing

1

1

1

1

Family Structure

†

Lives with two biological/adoptive parents

51

56

55

57

Parent Income

Less than $10K

3

3

3

2

$10K–20K

7

6

6

6

$20K–30K

12

11

11

10

$30K–40K

13

12

12

11

$40K–50K

13

14

14

14

$50K–60K

11

11

11

12

$60K–70K

7

8

8

8

$70K–80K

6

7

7

8

$80K–90K

5

6

6

5

$90K–100K

4

4

4

5

More than $100K

11

12

13

14

Missing

6

6

6

6

N

‡

3,259

2,530

2,458

2,071

Numbers represent unweighted percentages. Percentages may not add up to 100 because of rounding.

Family structure contains some missing data.

N does not include the Jewish oversample cases (N = 80 for Wave 1; N = 74 for Wave 2; N = 78 for Wave 3; N = 73 for Wave 4).

Source: National Study of Youth and Religion surveys 2003, 2005, 2008, 2013.

Methodological Design and Procedures for the NSYR Wave 4 Personal Interviews

The fourth wave of in-person interviews for the NSYR involved in-depth personal interviews with 303 of the Wave 4 survey respondents, resulting in 292 useable interview transcripts. The purpose of the interviews was to provide extended follow-up discussion about the religious, spiritual, family, and social lives of emerging adults. The questionnaire included much of what was used in Waves 1, 2, and 3, with the addition of several new topics that were relevant to their lives as 23- to 28-year-olds. The length of the interviews ranged from slightly less than one hour to approximately three hours, with most lasting about two hours. Respondents were offered a cash incentive ranging from $110 to $200 to participate in the study.

The in-person interviews were conducted between May 6, 2013 and December 19, 2013, with 92 percent of the interviews conducted between May and August 2013. To be interviewed, respondents were required to have completed the survey, and all interviews were completed between three and six months following respondents’ completion of the survey. All respondents who were interviewed in Wave 1 were recontacted for Wave 4 interviews. Of the 303 individuals who were interviewed, 205 had been interviewed as teens in Wave 1, and 41 were added to the interview pool to refresh the sample and provide additional diversity.

Our sampling strategy for Wave 1 interviews represented a range of demographic and religious characteristics and took into account region, residential location (urban/suburban/rural), age, sex, household income, religion, and school type. The new in-person interview respondents were chosen by combinations of geographic region, religious type, race/ethnicity, and sex to balance out the original sample. The researchers clustered the respondents with whom they were still in contact into geographic areas for ease of travel in interviewing. provides demographic information about the 292 interview respondents.

Table A.2 Wave 4 Personal Interview Demographics

N

N

Gender

Region

Female

150

Midwest

67

Male

142

Northeast

51

South

101

Age

West

73

23

33

24

58

Religious Tradition

25

60

Conservative Protestant

46

26

57

Mainline Protestant

15

27

60

Black Protestant

17

28

24

Catholic

32

Jewish

20

Latter-day Saints

15

Race

Not Religious

111

White

200

Indeterminate Christian

24

Black

35

Indeterminate Other

3

Hispanic

30

Asian

8

Native American

6

Other

13

N = 292.

Source: National Study of Youth and Religion.

Fourteen interviewers conducted interviews with respondents living in 44 states, Washington DC, and several other countries. While the majority of the interviews were conducted in person and near where the respondent lived, some of the interviews took place by phone or Skype because of costs and other logistics related to traveling to where the respondents were located. Each interviewer conducted between nine and 47 interviews. Eight of the interviewers had conducted personal interviews for the NSYR in either Wave 1, 2, or 3 (or all waves), and respondents were matched with their previous interviewers as much as possible. Before the fielding of the in-person interviews, all new interviewers participated in a day-long training meeting held by the NSYR project manager and the principle investigator. The training covered the logistics of the interview process, procedural requirements, Institutional Review Board (i.e., Human Subjects) concerns, protection of human participants, safety and liability concerns, and keys to NSYR interview success. In addition to that training, all interviewers were required to obtain Human Participants Training Certification.

The 14 project interviewers were assigned to sets of specific geographic locations around the United States. Each interviewer was provided with groups of contact sheets for respondents in their assigned geographic areas. The contact sheets included respondent name, respondent nickname (if known), parent name, address, phone number, respondent birth date, respondent gender, respondent race, household income, religious affiliation, and religious denomination or tradition. The contact sheet also included space to note any changes to the contact information provided (e.g., new phone number, additional email address) and provided a call record. Interviewers recorded each household contact, noting the date, time, person with whom they spoke, and content of each contact. In addition to the original sample, as the project progressed through the interviews, the priorities shifted somewhat according to what types of respondent interviews were still needed (e.g., location, religion, gender). In this way, the researchers attempted to match the demographics of the original Wave 1 in-person interview sample as much as possible.

In addition to the demographic information on the contact sheet, before their interviews, NSYR interviewers were asked to review the transcripts of the previous in-person interviews. If an interviewer had not conducted the previous personal interviews with his or her respondents, the transcripts provided background information about that respondent to help facilitate a smooth interview. Using a standard call script provided by the NSYR, interviewers made contact with potential interviewee households. Interviewers identified themselves as researchers with the “National Youth Study.” The full name of the project was not used in order to avoid introducing any response bias by identifying religion as a key focus of the study. It was often helpful for interviewers to explain their connection with the project, for example, graduate student, coinvestigator, and so on. However, since much of the interview was about religion and the researchers did not want to bias the respondents’ answers, interviewers were instructed to avoid divulging revealing information about their own personal beliefs and commitments (of which there was considerable diversity on the interviewing team).

Interviewers were required to obtain initial verbal consent from the respondent during the phone call to conduct the interview. If respondents seemed hesitant about participating, an additional script provided more information about the project and offered the phone number of the principle investigator, whom they could call with questions or concerns. In addition, interviewers offered to mail written information about the project to hesitant respondents and then call back in a few days. When a respondent seemed reluctant to participate or was hesitant about the time commitment, an additional incentive was offered. However, in cases in which respondents refused to participate even after being offered additional information, the interviewers made no further attempts to convert those who refused. On receiving verbal consent from the respondents and scheduling an interview time and location, interviewers mailed packets of information to households. The packet contained a cover letter from the principal investigator, multiple copies of respondent written consent forms, and an appointment letter, including a portrait photo of the interviewer. Interviewees were required to bring the written signed consent with them to their interviews. Interviewers also called the respondents at least one week before the interview and again the day before the interview to confirm that the respondents were planning to participate.

Interviews were conducted in public settings that still provided confidentiality for the respondent. The ideal location for these interviews was in study rooms at local libraries. However, when these were not available, interviews were also conducted in restaurants, coffee shops, mall food courts, and outdoor settings. Interviewers were given guidelines for how to present themselves during the interviews as well as appropriate attire to be worn in order to ensure consistency in the presentation of interviewers across the interviews. Specifically, interviewers did not attempt to “relate” to respondents by dressing down or dressing in a more “trendy” fashion. Interviewers presented themselves as professional researchers with a sincere interest in young people’s lives. At the close of the interview, respondents were given the cash incentive for their participation and in appreciation of their time and effort.

↑ Back to top

↑