Appendix
Appendix
Research Method and Procedures
The findings presented in this book emerged from the Churches Engaging Young People (CEYP, pronounced “keep”) Project, conducted from 2012 to 2015 by the Fuller Youth Institute (FYI) at Fuller Theological Seminary. This appendix provides an abbreviated description of the research method and procedures. For a more in-depth treatment of the research method, including the survey instruments and scales, interview protocols, and other more detailed research documents, please visit Churches GrowingYoung.org.
Based on extensive preliminary work and literature review, the research team identified one primary question and two secondary questions for study.
• Primary research question: What congregational practices lead to effective engagement of young people?
• Secondary research questions: How does engaging young people contribute to a thriving church? What are next-step processes for congregations that want to enact changes toward more effective ministry with young people?
Early in the project, a working definition was formed to clarify “effectively engaging young people.” An effective church was defined as one that is involving and retaining young people in the congregational community, as well as helping them develop a vibrant faith in Jesus Christ. While involving and retaining were much easier to operationalize, the team employed a variety of methods to understand and measure a vibrant faith in Jesus Christ.1
Building on a review of the existing literature, the research team also developed an initial working list (prior to beginning Stage 1 research) of characteristics that seemed likely to be present in churches that are effectively engaging young people. Both quantitative and qualitative items were developed to explore, assess, and revise these characteristics across the three stages of the research.
Nomination Process
In order to understand how and why churches are effectively engaging young people, it was critically important to first identify churches that fit this description.2 Over 35 nominators (described on page 30) were invited to submit names of up to 15 noteworthy churches via an online questionnaire from June through November 2013.3 Requirements included that churches be located within the US and that the pastors be able to complete a survey and potential subsequent surveys or interviews in English. The nominations were based upon the following criteria:
• Churches that are effectively engaging a growing number of young people4 (ages 15 to 29) or are effectively engaging a large number of young people in relation to the size of the congregation. (Up to 10 churches per nominator.)
• Churches that seem to have something exciting or missional going on with young people but their numbers aren’t large or growing. (Up to 5 churches per nominator.)
Nominators provided basic demographic information about each congregation, as well as a brief explanation of why the congregation was being nominated.5
Based on this nomination process, names of 363 noteworthy congregations were submitted for study.
Research Method and Procedures
Stage 1 Research
OVERVIEW AND INSTRUMENTS
A pastor and youth/young adult leader (either paid or volunteer) from each church was contacted by the research team via email and/or phone beginning in November 2013 and invited to participate in the study. Participating leaders completed a quantitative and qualitative survey that was delivered through an online questionnaire. Participants were assured of the anonymity of their responses and were offered a $50 gift card for their involvement.
The Stage 1 surveys included questions about the church’s size, attendance, growth patterns and presence of young people, socioeconomics, ethnic diversity, and the leader’s definition of spiritual vibrancy (and evidence of vibrancy among young people). In addition, a scale was developed to rate the presence of the eight congregational characteristics predicted to be associated with effective engagement of young people,6 as well as a scale gauging the faith maturity of young people in the congregation.7 Open-ended questions invited leaders to describe characteristics they believe account for their church’s success with engaging young people, challenges they face when it comes to ministering to young people, and ways they have seen young people contribute to the health or growth of their congregation. Slightly different versions of the survey were administered to the pastors and youth/young adult leaders, in particular the exclusion of the faith maturity measure from the senior pastor survey, assuming the pastor may have less awareness of young people’s spiritual maturity.8
PARTICIPANTS
Stage 1 data collection concluded in February 2014. A total of 373 church leaders from 259 nominated noteworthy congregations participated. See chapter 1 for detailed demographic information of participating congregations. In addition, as for socioeconomic diversity of congregations (as described by reporting churches), 7 percent indicated they were primarily upper class, 41 percent primarily upper-middle class, 41 percent primarily middle class, 8 percent primarily lower-middle class, and 3 percent primarily lower class.
Please see ChurchesGrowingYoung.org for a detailed description of data analysis methods for each stage of the research.
Stage 2 Research
OVERVIEW, SELECTION PROCESS, AND INSTRUMENTS
Based upon the results of the analyses in Stage 1, the research team identified several questions for further exploration in Stage 2. This research began in April 2014, consisting of one-hour interviews with 535 young people, parents of teenagers and emerging adults, church staff, and youth/young adult ministry volunteers across 41 exemplary churches from the 259 congregations that participated in Stage 1 (up to 20 interviews were conducted per church). The 41 exemplary congregations that participated in Stage 2 were selected through the following process.
Group 1: Fifteen churches were selected that had the highest combined score based on the percentage of young people participating (total number of 15- to 29-year-olds divided by total number of people participating in the church) and faith maturity ratings of young people. A formula was created that gave equal weight to each of these two criteria, and then the top 15 churches were selected for inclusion in Stage 2.
Group 2: The next 15 churches were selected based on those with the highest combined scores related to average ratings across all of the eight church characteristic questions.9
Group 3: Next, 14 remaining churches were selected that scored highest on each of the individual eight characteristic questions. Two churches were included for each characteristic, excluding the characteristic related to “communicating the gospel.”10
Group 4: Finally, 14 churches were chosen by expert qualitative analysis based on the team’s examination of the narratives of the Stage 1 surveys and unique or exemplary features of these churches.
The team invited 58 exemplary churches to participate in Stage 2 of the research, and 41 churches agreed. A point person from the congregation was selected by each church and was responsible for working with the congregation’s leaders to identify congregants to participate in an interview. Using an online nomination form, this point person identified young people ages 18 to 29,11 parents of teenagers and emerging adults, church staff, and youth/young adult ministry volunteers, up to a total of 40 people per church. Invitations were sent to nominated congregants until up to 20 agreed to participate in an interview. FYI’s research team, including the primary researchers and 14 graduate students from Fuller Seminary’s Schools of Theology, Psychology, and Intercultural Studies, conducted the approximately 60-minute structured phone interviews. Interviewers received extensive training by the project’s senior research advisors. All interviews were recorded, transcribed, and coded.
Stage 2 interviews were conducted individually and included several questions regarding demographic information of the participant, areas of involvement in the church, a description of the church, the participant’s perspective on why the church is effective with young people, how young people contribute to the church, Likert scale ratings and open-ended questions based on the church characteristics identified in the literature review,12 Likert scale ratings and open-ended questions about the participant’s beliefs and faith practices,13 and an opportunity to share any other information the participant deemed important. Participants were assured of their anonymity and the removal of personally identifiable information from the transcripts. One interview template was created for use with young people, while a slightly different version was used with parents, volunteers, and church staff.
PARTICIPANTS
Over 14 denominations were represented in Stage 2, in addition to 7 congregations claiming no denominational affiliation. The largest 5 specific traditions represented were Baptist (7 churches), United Methodist (4), Roman Catholic (4), Nazarene (3), and Evangelical Covenant (3). Churches ranged in size of active congregational participants, including 100 or fewer participants (2 percent of churches), 101–250 (20 percent), 251–500 (17 percent), 501–1000 (24 percent), 1001–3000 (20 percent), over 3001 (17 percent). In terms of census regions of the US, they were distributed among the Midwest (29 percent), West (29 percent), South (27 percent), and Northeast (15 percent). In terms of ethnic diversity (based on reporting churches), 48 percent of churches identified as “mostly White,” 34 percent were “multiracial” (meaning the congregation included 20 percent or more from two different ethnic/racial groups), 8 percent were “mostly African American,” 5 percent were “mostly Asian,” and 5 percent were “mostly Hispanic/Latino.” Regarding socioeconomic status (based on reporting churches), 37 percent reported primarily upper-middle class, 49 percent primarily middle class, and 14 percent primarily lower-middle class.
The following churches participated in Stage 2 and gave permission for their church to be named in association with this project (congregations also participating in subsequent Stage 3 research are marked with an asterisk [*]):
1. Centenary United Methodist Church, Winston-Salem, NC
2. Chapelgate Presbyterian Church, Marriottsville, MD*
3. The Church at Brook Hills, Birmingham, AL
4. College Church of the Nazarene, Nampa, ID
5. Covenant Congregational Church, Boston, MA
6. The District Church, Columbia Heights parish, Washington, DC*
7. Epic Church, Fullerton, CA
8. Faith Lutheran Church, Lacey, WA
9. Fellowship Memphis, Memphis, TN*
10. Fellowship Monrovia, Monrovia, CA
11. The First Baptist Church of Chicago, Chicago, IL*
12. First Baptist Church of South Gate, South Gate, CA*
13. First Covenant Church Minneapolis, Minneapolis, MN
14. First Presbyterian Church of Berkeley, Berkeley, CA
15. First United Methodist Church, Tulsa, OK*
16. Flood, Kearny Campus, San Diego, CA*
17. Frontline Community Church, Grand Rapids, MI
18. Ginghamsburg Church, Tipp City Campus, Tipp City, OH
19. The Highway Community Mountain View, Mountain View, CA
20. The Hills Church, North Richland Hills Campus, North Richland Hills, TX
21. Holy Apostles Episcopal Church, St. Paul, MN
22. Holy Trinity Orthodox Church, San Francisco, CA
23. Immanuel Church of the Nazarene, Lansdale, PA*
24. Lifeway Church, Indianapolis, IN
25. Mars Hill Bible Church, Grandville, MI
26. Metro Community Church, Englewood, NJ*
27. Millennium Revival Center, Raleigh, NC
28. Mountainside Communion, Monrovia, CA
29. Our Lady of Mount Carmel Church, Ridgewood, NJ
30. Our Lady of the Immaculate Conception, Dayton, OH
31. Redeemer’s Church, Reedley, CA
32. Saints Constantine and Helen Greek Orthodox Cathedral, Richmond, VA
33. St. Andrew United Methodist Church, Highlands Ranch, CO
34. St. Francis of Assisi Parish, Fulton, MD
35. St. John the Evangelist Catholic Church, Indianapolis, IN*
36. St. Paul’s Episcopal Church, Fayetteville, AR
37. Sugar Creek Baptist Church, Sugar Land Campus, Sugar Land, TX
38. Trinity Church, Miami Gardens Campus, Miami, FL*
39. Trinity Grace Church, East Village parish, New York, NY
40. Upper Room, Edina, MN
41. Volga Christian Reformed Church, Volga, SD*
Of the 535 individual interview participants in Stage 2, 60 percent were male and 40 percent were female. Reported age or role in the church (based on the reason they were nominated for the study) was distributed as 18 years old (6 percent), 19–23 years old (17 percent), 24–29 years old (21 percent), youth/young adult ministry volunteer (19 percent), parent of a teenager or young adult (14 percent), and church leader (23 percent). In terms of racial/ethnic diversity, interview participants identified as Asian/Asian American (10 percent), Black/African American (6 percent), Hispanic/Latino (6 percent), Causasian/White (73 percent), Pacific Islander (0.4 percent), and from multiple races (4.6 percent). Regarding highest education level completed, participants reported less than high school (1 percent), high school (9 percent), some college (18 percent), college degree (41 percent), master’s degree (27 percent), and doctoral degree (4 percent). In terms of household income,14 2 percent reported “far below average,” 16 percent “below average,” 38 percent “average,” 38 percent “above average,” and 6 percent “far above average.”
Stage 3 Research
OVERVIEW, SELECTION PROCESS, AND PROTOCOLS
In spring 2015, the research team selected 12 churches (from the 41 in Stage 2) to serve as illustrative case studies for deeper analysis in Stage 3. This analysis consisted of participant observation, document analysis, interviews, and focus groups. In order to select these 12 churches, the team began by eliminating 10 churches that had the lowest number of interview participants in Stage 2 or that were not well positioned to participate in this final stage of research.15 Next, the research team eliminated two churches in which less than 14 percent of their overall church population consisted of young people. A formula was then created that assigned a numeric value to each church by combining the percentage of the overall church that was made up of young people and a composite score based on three questions from the Stage 2 interviews that indicated the church included a high percentage of people with a vibrant faith.16 These weighted values were used to rank the remaining 29 churches.
This process resulted in a pool of potential churches displaying both high vibrancy and high participation of young people. In order to select churches that might serve as a diverse sample for the sake of the illustrative case studies, the research team created multiple categories deemed important for representation. This included a variety of denominations, ethnicities, socioeconomic statuses, and church sizes. For each representative category, the church with the highest vibrancy/participation weighted rating was selected. A total of 14 churches were invited in order to identify the final 12 that participated. Churches agreed to forego anonymity in this stage of the study, but individuals interviewed were assured their identities would not be used without further permission.
Teams of two or three researchers conducted the site visit to each of the 12 churches. While each visit was customized to the congregation’s unique context, a typical site visit took place over one weekend. Before the visit, each team reviewed the church’s online survey responses, interview transcripts, church website, printed documents sent by the church’s point person, several sermons, and any other information that was publicly available. The church point person set up interviews and focus groups with church staff, high school students, young people ages 19–23, young people ages 24–29, parents of teenagers and emerging adults, youth/young adult ministry volunteers, congregants who could speak to the church’s history, and others as deemed necessary. The research team also attended all possible church activities and programs, including services, youth/young adult gatherings, Sunday school, and any other weekend programming. During all visits, interviews and focus groups were recorded and extensive field notes were kept.
PARTICIPANTS
Over eight traditions were represented in Stage 3 research, in addition to two churches indicating no denominational affiliation. The makeup of this stage included three Baptist churches and one church each that identified as Assemblies of God, Evangelical Covenant, Christian Reformed, Nazarene, Presbyterian, Roman Catholic, and United Methodist. Churches ranged in size of active congregational participants, including 100 or fewer participants (1 church), 101–250 (2), 251–500 (1), 501–1000 (4), 1001–3000 (3), and over 3001 (1). In terms of census regions of the US, they were distributed among the Midwest (3 churches), West (2), South (4), and Northeast (3). Ethnic diversity included four churches that identified as “mostly White,” four “multiracial” churches, two “mostly African American” churches, one “mostly Asian” church, and one “mostly Hispanic/Latino” church. Regarding socioeconomic status, five churches were primarily upper-middle class, five churches primarily middle class, and two churches primarily lower-middle class. Participating Stage 3 churches are marked by an asterisk (*) in the list of Stage 2 churches in the prior section.
DATA ANALYSIS
The Stage 3 site visits concluded during the summer of 2015. An average of 60 pages of field notes were generated from each visit, for a total of nearly 720 pages across all Stage 3 churches. After each visit, the two or three research team members met to combine their notes into a comprehensive report. The core FYI research team read and reflected on these reports, both while the visits were being conducted and again after they were complete.
This on-the-ground research from Stage 3 was used alongside the quantitative and qualitative analyses from Stage 1 and Stage 2 and became a lens through which the data from earlier stages was interpreted. The core research team was also in regular dialogue with the advisors from the project regarding potential interpretation of the findings. The eight church characteristics were narrowed down to a list of five that stood out as most common and important across the churches in the study. Upon further reflection on the site visits and ongoing data analysis, the research team added a sixth and final characteristic of “empathizing with today’s young people.” Also at this time, the research team chose to describe these as the six “core commitments” of churches that effectively engage young people.
During the summer and fall of 2015, the core research team sent detailed reports of the project’s findings to each member of the senior research advisors, Expert Advisory Council, and Pastor Advisory Council. These 30 advisors provided feedback through one-on-one and group phone calls or meetings.
In addition, the core research team presented select findings to a group of scholars and practitioners at the Association of Youth Ministry Educators conferences in fall 2014 and fall 2015, as well as to a group of over 50 church leaders at a fall 2015 Engaging Young People Summit at Fuller.
Assumptions and Limitations
While the research team made every effort to ensure that the CEYP Project was as academically rigorous, practically accessible, and generally thorough as possible, no research (ours included) is without assumptions or limitations. Following are just a few of the assumptions made and associated limitations of the study.
Our context as researchers. The research team understands our context as researchers to be both a strength and a weakness. The Fuller Youth Institute is located within Fuller Theological Seminary, one of the largest seminaries in the world. Given Fuller’s multidenominational makeup and overall diversity, it has served as a wonderful and unique home in which to conduct this research. However, we recognize our location in Southern California, our own ethnic backgrounds, our middle-class socioeconomic status, and our level of education have undoubtedly shaped the way in which the team approached this research. Significant effort was made to include other voices from a variety of contexts and backgrounds to provide the widest perspective possible.
The importance of church participation. In forming the foundations for the CEYP Project, the team assumed that church participation is important for all Christians. As a result, the research team chose to study church participation directly rather than starting with the question, “What helps a young person grow in their faith?” The CEYP Project did not ask questions about the detailed spiritual experiences of young people but rather focused on their experiences of church.
Reliance on perceived effectiveness. Though the core commitments were found to be common across a variety of churches, the CEYP Project was largely reliant on what various leaders and congregants perceived to be effective. There are undoubtedly some variables that may not have been examined, such as those that did not come to mind during interviews or those that were unnoticed by participants.
A study of those who are involved. Since a key aspect of the study was to focus on churches that are actually effective in their ministry to young people, this project did not study the thousands of young people who have disengaged from and are no longer part of church. While their voices are important, less research has been conducted on the reasons why some stay involved in the church.
Generalizability of results. The CEYP Project’s findings will not apply to all people, at all times, in any setting. Rather, this study focuses on a particular group of exemplary churches in the US. Undoubtedly, some churches will find the research difficult to apply to their unique context.
Correlation versus causation. The research team does not claim to have demonstrated a causal relationship between these core commitments and a church that is effective with young people. However, we’re confident that the CEYP Project has identified six core commitments demonstrably present (through a variety of methods of data collection and analysis) within churches that grow young. The particular expression of each core commitment will vary based on the church’s context. In the end, our desire is that this research will initiate helpful and life-giving conversations among church leaders. We’re also hopeful that other scholars and researchers will review, test, and build on this work.
Our theological commitments. Last, but certainly not least, the Fuller Youth Institute (as part of Fuller Theological Seminary) holds particular beliefs and theological commitments that have grounded and guided our research. Undoubtedly, these commitments have influenced the research in ways that those from other traditions may find unhelpful.17