The Journal of Extension - www.joe.org

February 2018 // Volume 56 // Number 1 // Feature // 1FEA6

Feasibility of Implementing a School Nutrition Intervention That Addresses Policies, Systems, and Environment

Abstract
We conducted a process evaluation of the Shaping Healthy Choices Program, a multicomponent school-based nutrition program, when implemented in partnership with University of California (UC) CalFresh and UC Cooperative Extension (UCCE). There were positive impacts on participating students, but results varied across counties, possibly due to variation in fidelity to the curriculum and implementation of program components. Our evaluation identified the strength of UCCE in delivering nutrition education and a need for additional support and training for building capacity to effect change in school policies, systems, and environment. Because educators throughout Extension are working to integrate programs addressing policies, systems, and environment, our results may have applicability in other Extension programs.


Jacqueline J. Bergman
Postdoctoral Scholar
Department of Nutrition
University of California, Davis
Davis, California
jjbergman@ucdavis.edu

Jessica D. Linnell
Postdoctoral Scholar
Department of Nutrition
University of California, Davis
Davis, California
jessica.linnell@oregonstate.edu

Rachel E. Scherr
Assistant Research Scientist
Department of Nutrition
Center for Nutrition in Schools
University of California, Davis
Davis, California
rescherr@ucdavis.edu

David C. Ginsburg
Director
University of California CalFresh Nutrition Education Program
University of California, Davis
Davis, California
dcginsburg@ucdavis.edu

Kelley M. Brian
Advisor, Youth, Families, and Communities
Cooperative Extension, Placer and Nevada Counties
University of California Agriculture and Natural Resources
Davis, California
kmbrian@ucanr.edu

Rosemary Carter
Community Education Specialist
Cooperative Extension, Placer and Nevada Counties
University of California Agriculture and Natural Resources
Davis, California
carter@ucanr.edu

Susan Donohue
Advisor Emeritus
Nutrition, Family, and Consumer Sciences
Cooperative Extension, Butte County
University of California Agriculture and Natural Resources
Davis, California
ssdonohue@ucanr.edu

Shannon Klisch
Community Education Supervisor
Cooperative Extension, Santa Barbara and San Luis Obispo Counties
University of California Agriculture and Natural Resources
Davis, California
sklisch@ucanr.edu

Suzanne Lawry-Hall
Nutrition Program Supervisor
Cooperative Extension, Butte County
University of California Agriculture and Natural Resources
Davis, California
sllawry@ucanr.edu

Jona Pressman
Nutrition Education Program Manager
Cooperative Extension, Butte County
University of California Agriculture and Natural Resources
Davis, California
jmpressman@ucanr.edu

Katherine Soule
Advisor, Youth, Families, and Communities
Cooperative Extension, Santa Barbara and San Luis Obispo Counties
University of California Agriculture and Natural Resources
Davis, California
kesoule@ucanr.edu

Sheri Zidenberg-Cherr
Cooperative Extension Specialist, Department of Nutrition
Co-Director, Center for Nutrition in Schools
University of California, Davis
Davis, California
sazidenbergcherr@ucdavis.edu

Background

Research has continued to suggest that children in the United States are not consuming recommended amounts of fruits, vegetables, and other nutrient-dense foods (Keast, Fulgoni, Nicklas, & O'Neil, 2013; Storey & Anderson, 2014). Supplemental Nutrition Assistance Program (SNAP) Education, or SNAP-Ed, is a federally funded initiative that promotes healthful eating patterns among those eligible for SNAP benefits, emphasizing nutrient-rich foods such as fruits and vegetables. In California, multiple agencies administer SNAP-Ed, including the University of California (UC) CalFresh Nutrition Education Program, which is implemented through UC Cooperative Extension (UCCE).

In recent years, SNAP-Ed programs have begun placing greater emphasis on nutrition education interventions that integrate improvements to policies, systems, and environment to support children's health. In response to the need for such interventions, our research team worked with UC CalFresh and UCCE more broadly to conduct a pilot implementation of the Shaping Healthy Choices Program (SHCP), which had been shown to improve nutrition-related outcomes among fourth-grade children in two consecutive studies (Linnell et al., 2016; Scherr et al., 2017). The SHCP combines nutrition education with (a) family and community partnerships, (b) inclusion of regional produce on the school campus, and (c) enhancement of school wellness policies. Our objective was to understand the feasibility and efficacy of implementing the SHCP with students participating in SNAP-Ed–eligible schools. Because SNAP-Ed educators throughout Extension are working to integrate programs that address policies, systems, and environment in accordance with new SNAP-Ed requirements, our results may have applicability in other Extension programs.

Program Overview

The SHCP is a school-based nutrition intervention with the following objectives:

  1. Increase nutrition knowledge and use of science process skills.
  2. Promote availability, consumption, and enjoyment of fruits and vegetables.
  3. Improve dietary patterns, and encourage physical activity.
  4. Foster positive changes in the school environment.
  5. Facilitate development of an infrastructure to sustain the program.

To achieve these objectives, program implementers are to conduct various activities encompassed by the four program components—nutrition education and promotion, family and community partnerships, inclusion of regional produce on the school campus, and enhancement of school wellness policies—all integrated to bring about and sustain positive student health outcomes. Program activities involve classroom nutrition education, cooking demonstrations, family newsletters, facilitation of a salad bar, local procurement of produce, lunchroom enhancements, an instructional school garden, a community health fair, and formation of a school-site wellness committee. The theoretical foundation of the SHCP includes the social ecological model (U.S. Department of Agriculture and U.S. Department of Health and Human Services, 2010) and social cognitive theory (Bandura, 1986). A detailed explanation of the SHCP can be found elsewhere (see Scherr et al., 2014) and is included in SNAP-Ed Strategies and Interventions: An Obesity Prevention Toolkit for States (U.S. Department of Agriculture, 2016).

Pilot Implementation

Participants

In 2014, UCCE area advisors in nutrition, family, and consumer sciences and youth, families, and communities in three counties were recruited to pilot test the SHCP through their SNAP-Ed programs. The advisors worked with their county teams to recruit at least one SNAP-Ed–eligible school to participate in the pilot implementation. Nine upper elementary classes (fourth, fifth, and sixth grades) in four schools participated in the 4-month pilot test. Participating students (n = 242) were enrolled in the study through informed consent. All procedures were approved by the UC Davis Institutional Review Board.

Comprehensive Professional Development

Throughout the process of preparing county teams to implement the SHCP, we applied the community-of-practice model of professional development. Because communities of practice have been identified as key components of effective professional development efforts (Baughman et al., 2010; Chang & Jacobs, 2012) and program implementation (Duerden & Witt, 2012), we considered development of a community of practice to be integral to the success of the pilot implementation. Before program implementation, the participating UCCE county teams attended two multiday professional development workshops. In the first workshop, we presented an overview of the program and assisted the county teams in developing implementation plans for all four components of the SHCP. During the workshop, teams brainstormed the optimal methods of program implementation within the contexts of their communities. In the second workshop, participants learned more about the program's nutrition education and promotion component. We modeled several nutrition education lessons, emphasizing learner-centered, experiential learning and inquiry-based facilitation techniques. Participants further developed their lesson facilitation skills via an experiential professional development approach by taking turns facilitating lessons (Linnell et al., 2016; Smith, 2008). County teams were encouraged to implement all components of the program to achieve high fidelity to the research-tested program.

Throughout the implementation of the nutrition education and promotion component of the program, nutrition educators attended three check-in meetings via web-based conference calls. The meetings involved use of the community-of-practice approach, whereby teams discussed their experiences and learned from one another to continue developing knowledge and skills needed to facilitate the program curricula Discovering Healthy Choices (Linnell, Smith, & Zidenberg-Cherr, 2015) and Cooking Up Healthy Choices (Brian, Linnell, & Zidenberg-Cherr, 2015).

Ongoing Technical Assistance

Support for the educators continued as they implemented the program. Representatives of the UC Davis Center for Nutrition in Schools and UC CalFresh provided technical assistance for several program activities, including implementation of salad bars, development of instructional school gardens, local procurement of school foods, lunchroom enhancements, community health fairs, formation of school-site wellness committees, and program evaluation. All UCCE county teams participated in a local procurement webinar in which a member of the UC Davis Agricultural Sustainability Institute discussed best practices for procuring foods for school meal programs. In-person assistance with conducting program evaluation was provided as needed. Members of the UC Davis Center for Nutrition in Schools provided additional support to UCCE county teams via email and telephone. The forms and frequencies of the technical assistance provided varied, depending on the unique needs of each county.

Program Evaluation

Student Outcomes

Before and after UCCE county teams implemented the SHCP, they performed various assessments.

  • Members of the county team measured students' nutrition knowledge by distributing a previously validated, 35-item questionnaire (Morris, Briggs, & Zidenberg-Cherr, 2002) to students during the school day. This questionnaire contains questions relevant to concepts of the program's nutrition education component. Students received a paper copy of the questionnaire, which also was projected and read allowed by the educator. This method of administration was used to address various levels of reading comprehension among the students.
  • Members of the county team verbally administered a food identification questionnaire to measure each student's ability to identify 10 vegetables the students likely would learn about through the SHCP nutrition education curriculum and/or grow in their school garden. The rationale behind the use of this assessment tool lies in the theory that children who are able to identify and willing to taste produce items are more likely to consume them (Ashida, Wilkinson, & Koely, 2012; Somerset & Markwell, 2009; Toral & Slater, 2009). The individual administering the questionnaire showed each of the 10 vegetables to the class, one by one. The students were asked to silently record the name of the vegetable shown or select "I don't know" on the questionnaire. The vegetables shown were purchased from a local grocery store and were carrot, yellow squash, kohlrabi, sugar snap pea, chard, broccoli, beet, radish, spinach, and green onion.

Because the participating schools were all UC CalFresh–eligible, income data were not collected. School-wide demographics were similar among all participating schools.

To measure the impact of each county's delivery of the SHCP on student outcomes, we conducted a paired-samples t-test to compare students' knowledge levels and identification abilities before and after the intervention.

Program Implementation

Following implementation, county teams used an online questionnaire to report on how they implemented the various SHCP activities, including which nutrition education lessons they completed, what type of garden they implemented, how many family newsletters they distributed, what type of community health event they hosted, and what type of salad bar they implemented.

We used their responses to generate a score of program completion for each of the program activities.

Fidelity to Curriculum Procedures

UC CalFresh supervisors and UCCE advisors conducted classroom observations. We used these observations to evaluate fidelity to the Discovering Healthy Choices curriculum (i.e., the extent to which nutrition educators completed all procedures of the curriculum). Observations were structured according to the four phases of curriculum activities: opening questions; procedures; sharing, processing, and generalizing; and concept/term introduction. We used a three-response scale to rate the level of curriculum fidelity: did not do = 0, partially delivered according to curriculum = 1, fully delivered according to curriculum = 2. Researchers have used this scoring method in similar studies (Wind et al., 2008).

We calculated percentage of fidelity by dividing the sum of scores of all phases of the activity by the total possible (n = 8). In some cases, the nutrition educators were unable to complete all phases of an activity due to time limitations. In these cases, we calculated percentage of fidelity using the total possible score for only those activity phases directly observed.

Feedback Forum

UCCE county team members' perceptions of the feasibility and acceptability of the program were identified at a feedback forum for participating counties at the end of the intervention. Perceived challenges and successes related to program implementation were also discussed. We used a traditional focus group technique to facilitate discussion (Morgan, 1997). A moderator asked a variety of questions and wrote participant responses on flip-chart paper. A scribe also took verbatim notes throughout the session.

We analyzed qualitative data from 15 participants by organizing data into themes and patterns (Marshall & Rossman, 2011), using NVivo qualitative analysis software to facilitate the organization of data into themes.

Results

Student Outcomes

A significant improvement in participants' nutrition knowledge from before the program to after the program was demonstrated when all counties' results were combined (p = .001) (Table 1). An analysis by county showed that participants in Counties A and C had significant increases between pretest and posttest (p = .003 and p = .009, respectively) (Table 1). With regard to food identification, the percentage of students able to identify the target vegetables increased significantly for five of the 10 vegetables for students in County A, three of the 10 vegetables for students in County B, and six of the 10 vegetables for students in County C (Table 1).

Table 1.
Shaping Healthy Choices Program Student Outcomes

Combined County A County B County C
Outcome variable Pre Post p Pre Post p Pre Post p Pre Post p
Total score of nutrition knowledgea 20.7 (4.2) 22.6 (4.2) .001* 21.5 (3.7) 23.3 (4.3) .003* 20.6 (4.1) 21.6 (3.4) .051 19.4 (4.9) 22.0 (4.3) .009*
Total number of correctly identified vegetablesb 4.9 (1.6) 6.5 (1.9) .001* 5.3 (1.6) 7.0 (1.7) .001* 4.7 (1.2) 5.8 (1.6) .110 4.3 (1.7) 6.2 (1.9) .001*
% of correctly identified vegetables
Carrot 98.9 98.3 .570 98.9 97.8 .312 93.6 93.6 1.000 98.2 98.1 1.000
Yellow squash 54.9 53.1 .670 68.5 75 .276 41.9 41.9 1.000 37.0 20.4 .019*
Kohlrabi 0.0 16.6 .001* 0.0 4.4 .044* 0.0 0.0 1.000 0.0 46.3 .001*
Snap pea 55.4 84.6 .001* 58.7 92.4 .001* 74.2 87.1 .043* 37.0 66.7 .001*
Swiss chard 0.6 49.1 .001* 1.1 82.6 .001* 0.0 12.9 .043* 0.0 11.1 .013*
Broccoli 94.3 97.7 .034* 95.7 97.3 .341 90.3 93.6 .573 90.7 96.3 .182
Beet 22.7 45.7 .521 35.9 40.2 .483 0.0 12.9 .043* 22.2 72.2 .001*
Radish 63.4 78.9 .000* 63.0 72.8 .072 74.2 83.9 .264 55.6 83.3 .001*
Spinach 58.9 73.7 .001* 59.8 72.8 .009* 54.8 61.3 .602 57.4 79.6 .001*
aTotal score (standard deviation). bTotal possible score out of 10 (standard deviation).
*p < .05.

Program Completion and Fidelity

All counties completed the Discovering Healthy Choices curriculum in its entirety. Counties A and C completed the Cooking Up Healthy Choices curriculum; however, due to school-imposed restrictions, County B was unable to complete these lessons. All counties successfully implemented the instructional garden activity and sent home all family newsletters. Data relating to completion of these and other aspects of the program are shown in Table 2.

Table 2.
Shaping Healthy Choices Program (SHCP) Reported Program Implementation

Activity Combined County A—
School 1
County A—
School 2
County B County C
Nutrition education and promotion
% completed Discovering Healthy Choices 100
(SD = 0)
100 100 100 100
% completed Cooking Up Health Choices 66.7
(SD = 58)
100 100 0 100
Description of instructional garden Raised beds; expansion of existing garden space, community build Raised beds; expansion of existing garden space, community build, garden club Raised beds; used well-established preexisting garden, students helped prepare one bed for the SHCP Raised beds; recently enhanced, community-built garden for the SHCP
Family and community partnerships
% Team Up for Families newsletters distributed 100 100 100 100 100
Description of community event(s) Three school-based events with farm stands County-based event School-based health fair hosted with community partners Student-run physical activity fair
Regional foods available on the school campus
Description of salad bar Preexisting salad bar No salad bar Preexisting salad bar Preexisting salad bar
School wellness policies
Presence/description of site-specific wellness policy committee N/A N/A Student-run wellness group N/A

Data relating to fidelity to the SHCP Discovering Healthy Choices curriculum are shown in Table 3. When all county results were combined, total fidelity was 85.6%±18.3, with variation among the counties (County A = 80.2%±21.4, County B = 75.8%±21.7, and County C = 95.5%±6.8).

Table 3.
Average Fidelity to Shaping Healthy Choices Program Curriculum Procedures

Procedural phase Combined
% (SD)
County A
% (SD)
County B
% (SD)
County C
% (SD)
Totala 85.5 (18.3) 80.2 (21.4) 78.5 (21.7) 95.6 (6.8)
Opening questionsb 96.2 (13.6) 87.5(25.0) 95.0 (15.8) 100.0 (0.0)
Proceduresb 80.8 (24.8) 87.5 (25.0) 65.0 (24.2) 91.7 (19.5)
Sharing, processing, generalizingb 80.8 (28.6) 75.0 (28.9) 70.0 (35.0) 91.7 (19.5)
Concept/term introductionb 87.0 (22.5) 66.7 (28.9) 77.8 (26.4) 100.0 (0.0)
aTotal represents the percentage of fidelity of all phases of the activity that were observed. bPercentage of fidelity was calculated for each phase separately.

Relationship Between Student Outcomes and Lesson Fidelity

Variation in student outcomes across the counties appeared to mirror variation in the levels of fidelity of implementation of Discovering Healthy Choices. Significant improvements in student outcomes were observed in Counties A and C and not in County B. Improvement in student outcomes corresponded to the levels of lesson fidelity observed across the counties (Figure 1).

Figure 1.
Shaping Healthy Choices Program Curriculum Fidelity and Student Outcomes


*Indicates statistical significance of p < .05.

Perceived Challenges of SHCP Implementation

Implementing counties reported several challenges, including lack of school readiness to implement the program, limited personnel time, inability to build capacity, and challenges related to establishing site-specific wellness policy committees. County teams identified establishing site-specific wellness policy committees as the most challenging aspect of implementing the SHCP. They cited difficulty setting meetings and communicating to district-level wellness policy stakeholders as reasons. They also reported uncertainty with regard to what the expectations were for the wellness policy component of the SHCP.

Perceived Successes of SHCP Implementation

County A used community-based approaches to building instructional gardens, thereby allowing community members to get involved and develop ownership of the gardens. The garden projects were successfully completed by volunteers using materials that were donated from local businesses. The contributions of members across the school communities also sparked interest among teachers regarding using the instructional gardens for other lessons. To support and maintain its garden, one of the two participating schools in the county established a garden club.

County B relied on community partnerships for success, including by working with community members to host a school-based health fair. The event included participation by the Bel Air Grocery Market, Health Education Council, Walgreens, teachers, UCCE farm advisors, and the UCCE 4-H Youth Development Program. Students and families who attended the health fair expressed appreciation that these community stakeholders showed interest in supporting their health and well-being by contributing their time, funding, and materials.

County C successfully formed an active Student Nutrition Advisory Council (SNAC). Members of SNAC participated in an assessment of the school environment, using the Shaping Healthy Choices Self Health Check (SHC2), a whole-school evaluation tool developed by the UC Center for Nutrition in Schools for the SHCP. They used results of the SHC2 to set goals for making improvements to the school environment. For example, they created signage for the lunchroom to promote healthful lunch menu items. In addition, they helped the school district's nutrition services director organize "Try-it Tuesdays," an effort to encourage students to try new foods.

All three counties found the communities of practice to be a useful model for learning how to implement the SHCP. The monthly check-in meetings and technical assistance throughout the implementation allowed teams to discuss solutions to challenges they experienced.

Feasibility

On the basis of the student outcome data and results from the feedback forum, we determined that it is feasible for county teams to implement all aspects of the SHCP, including direct nutrition education, environmental improvements in the school lunchroom, and instructional gardens, while engaging multiple stakeholders within the school community to sustain the positive changes. At the feedback forum, there was extensive discussion about overcoming barriers to implementation, allowing for the development of problem-solving strategies. Although county teams encountered challenges, the overall perception of the SHCP was positive, and all counties opted to continue, and even expand, program implementation.

Conclusions and Implications for Extension Personnel

Here, we share conclusions and implications that may be useful to Extension and SNAP-Ed program developers considering implementing the SHCP and similar programs that combine nutrition education with changes to policies, systems, and environment.

Build Strong Community Relationships

Schools observed to have greater improvements in student outcomes tended to be those with school staff who were open to comprehensive nutrition programming. We recommend establishing partnerships from the beginning so that participating school community members can support, and eventually take ownership over, the implementation of program activities. The more engaged school stakeholders become, the more likely they are to participate in improving the social norms and cultural values that support a healthful school environment.

Emphasize Fidelity to Curriculum Procedures and Program Activities

Variation in student outcomes across counties appears to be tied to variation in levels of fidelity to implementation of the nutrition curriculum (i.e., the extent to which nutrition educators completed all procedures of the curriculum). Although there is limited research suggesting a relationship between curriculum fidelity and student outcomes (Stirman et al., 2012), our findings suggest that greater fidelity to the curriculum procedures may be associated with the success of the SHCP. Therefore, we recommend that training emphasize the importance of adherence to the procedures of all four phases of all modules of the curriculum (opening questions; procedures; sharing, processing, and generalizing; and concept/term introduction) for greatest impact.

Variation in student outcomes across counties also seemed to be consistent with variations in levels of other aspects of program implementation. For example, when cooking demonstrations were not implemented as part of the nutrition education, students' ability to identify vegetables did not improve significantly. Therefore, the importance of delivering Cooking Up Healthy Choices should be emphasized during the professional development.

Additionally, it is important that the implementation not only focus on direct nutrition education but also emphasize establishing family and community partnerships, enhancing foods available on the school campus, and empowering school stakeholders to enhance the school district's wellness policies through the formation of school-site wellness policy committees. Although this claim and the underlying data relate specifically to SHCP implementation, the broader concept can be applied in relation to implementation of any multicomponent program that integrates policies, systems, and environment. Training and ongoing support for multilevel interventions such as the SHCP should underscore the idea that when implemented together, all program activities form an infrastructure that promotes healthful diet and lifestyle choices while fostering a healthful school environment.

Although it was difficult to interpret the specific components of the SHCP that drove student outcomes in our study due to the complexity of the multilevel intervention, we recommend that additional research be conducted to help elucidate the critical components of multilevel interventions such as the SHCP that are required for the greatest improvements in children's health outcomes.

Balance Program Fidelity and Adaptability

Historically, Extension professionals have been encouraged to work closely with their communities to support the unique needs of each community, an endeavor that requires flexibility. The results of our study indicate the importance of finding a balance between program fidelity and cognizance of the unique culture of a community. Olson, Welsh, and Perkins (2015) suggested that the most important step Extension professionals can take when seeking to modify an evidence-based program is to identify the program's core components. Because the core components are directly related to a program's theory of change, these become essential (Blase & Fixsen, 2013). Consequently, eliminating core components should be avoided because doing so means eliminating a key ingredient for programmatic success. Therefore, it is important for program managers to work closely with Extension professionals and other educators to ensure that they are confident in identifying and implementing core components of the SHCP and similar programs. Ultimately, success is tied to the willingness of the school partner to identify priorities and follow through on them with the Extension professionals. As a result of this circumstance, a readiness assessment was developed for subsequent SHCP dissemination that includes questions pertaining to a school's readiness for instructional gardens, goals for student health and wellness, and identification of existing and potential community partnerships.

Develop a Community of Practice

County teams attributed the support from their peers as a source of confidence they drew from to overcome seemingly insurmountable challenges. Through the community of practice established during program implementation, they learned to become comfortable with the uncomfortable and developed a sense of resiliency and self-sufficiency when things did not go as planned. Therefore, it is recommended that program implementers create a community of practice by providing a platform for peer discussion and support throughout the implementation process.

Future Directions

As SNAP-Ed program implementers seek to integrate policies, systems, and environment interventions into their programming, they can use models such as the SHCP that allow for a comprehensive, integrated approach. One hallmark of the SHCP is that it builds off existing school–community partnerships (such as Harvest of the Month and partnerships with 4-H youth development, master gardener, or master food preserver programs) to develop a truly comprehensive and cohesive program that includes direct education related to policies, systems, and environment.

Acknowledgments

We would like to acknowledge the hard work of nutrition educators on each county team, the graduate and undergraduate student assistants, and the UC CalFresh State Office staff. This project was funded by California State Grant: SNAP-Ed, and the UCANR Competitive Grant 11-1018.

Author Notes

Some affiliations changed after this research was completed. Jacqueline Bergman is now a lecturer in the Department of Nutrition at University of California, Davis. Jessica Linnell is now an assistant professor (practice) in family and community health at Oregon State University Extension Service. Susan Donohue has retired from University of California Agriculture and Natural Resources.

References

Ashida, S., Wilkinson, A. V., & Koely, L. M. (2012) Social influence and motivation to change health behaviors among Mexican-origin adults: Implications for diet and physical activity. American Journal of Health Promotion, 26, 176–179.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

Baughman, S., Arnold, M., Boyd, H. H., Franz, N. K., Mead, J. P., Rowe, E., & Silliman, B. (2010). Evaluating for impact: Professional development educational content delivery through learning communities. Journal of Extension, 48(3), Article 3TOT3. Available at: http://www.joe.org/joe/2010june/tt3.php

Blase, K., & Fixsen, D. (2013). Core intervention components: Identifying and operationalizing what makes programs work. ASPE Research Brief. Washington DC: U.S. Department of Health and Human Services.

Brian, K., Linnell, J. D., & Zidenberg-Cherr, S. (2015). Cooking Up Healthy Choices. Davis, CA: University of California, Davis.

Chang, J., & Jacobs, R. (2012). Determinants and outcomes of employee participation in a strategic community of practice. Human Resource Development Quarterly, 23(3), 341–362.

Duerden, M. D., & Witt, P. A. (2012) Assessing program implementation: What it is, why it's important, and how to do it? Journal of Extension, 50(1), Article 1FEA4. Available at: http://www.joe.org/joe/2012february/a4.php

Keast, D. R., Fulgoni, V. L., Nicklas, T. A., & O'Neil, C. E. (2013). Food sources of energy and nutrients among children in the United States: National Health and Nutrition Examination Survey 2003–2006. Nutrients, 5(1), 283–301.

Linnell, J., Smith, M. H., & Zidenberg-Cherr, S. (2015). Discovering Healthy Choices. Davis, CA: University of California, Davis.

Linnell, J. D., Smith, M. H., Briggs, M., Brian, K. M., Scherr, R. E., Dharmar, M., & Zidenberg-Cherr, S. (2016). Evaluating the relationships among teacher characteristics, implementation factors, and student outcomes of children participating in an experiential school-based nutrition program. Pedagogy in Health Promotion, 2(4), 256–265. https://doi.org/10.1177/2373379916649603

Marshall, C., & Rossman, G. B. (2011). Designing qualitative research (5th ed.). Thousand Oaks, CA: Sage Publications.

Morgan, D. (1997). Focus groups as qualitative research (2nd ed.). Thousand Oaks, CA: Sage Publications.

Morris, J. L., Briggs, M., & Zidenberg-Cherr, S. (2002). Development and evaluation of a garden-enhanced nutrition education curriculum for elementary schoolchildren. Journal of Child Nutrition and Management, 26(1).

Olson, J. R., Welsh, J. A., & Perkins, D. F. (2015). Evidence-based programming within Cooperative Extension: How can we maintain program fidelity while adapting to meet local needs? Journal of Extension, 53(3), Article 3FEA. Available at: http://www.joe.org/joe/2015june/a3.php

Scherr, R. E., Linnell, J. D., Dharmar, M., Beccarelli, L. M., Bergman, J. J., Briggs, M., . . . Ontai, L. L. ( 2017). A multicomponent, school-based intervention, the Shaping Healthy Choices Program, improves nutrition-related outcomes. Journal of Nutrition Education and Behavior, 49(5), 368–379.

Scherr, R. E., Linnell, J. D., Smith, M. H., Briggs, M., Bergman, J., Brian, K. M., . . . Nguyen, L. M., (2014). The Shaping Healthy Choices Program: Design and implementation methodologies for a multicomponent, school-based nutrition education intervention. Journal of Nutrition Education and Behavior, 46(6), e13–e21. https://dx.doi.org/10.1016/j.jneb.2014.08.010

Smith, M. H. (2008). Volunteer development in 4-H: Constructivist considerations to improve youth science literacy in urban areas. Journal of Extension, 46(4), Article 4IAW2. Available at: https://www.joe.org/joe/2008august/iw2.php

Somerset, S., & Markwell, K. (2009). Impact of a school-based food garden on attitudes and identification skills regarding vegetables and fruit: A 12-month intervention trial. Public Health Nutrition, 12(2), 214–221. doi:10.1017/S1368980008003327.

Stirman, S. W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7(1), 17. https://dx.doi.org/10.1186/1748-5908-7-17

Storey, M., & Anderson, P. (2014). Income and race/ethnicity influence dietary fiber intake and vegetable consumption. Nutrition Research, 34(10), 844–850.

Toral, N., & Slater, B. (2009). Perception of eating practices and stages of change among Brazillian adolescents. Preventative Medicine, 48, 279–283.

U.S. Department of Agriculture. (2016). SNAP-Ed strategies and interventions: An obesity prevention toolkit for states (2nd ed.). Washington, DC: Government Printing Office.

U.S. Department of Agriculture and U.S. Department of Health and Human Services. (2010). Dietary guidelines for Americans (7th ed.). Washington, DC: Government Printing Office.

Wind, M., Bjelland, M., Perez-Rodrigo, C., Te Velde, S. J., Hildonen, C., Bere, E., Klepp, K. I., & Brug, J. (2008). Appreciation and implementation of a school-based intervention are associated with changes in fruit and vegetable intake in 10- to 13-year old schoolchildren—The Pro Children study. Health Education Research, 23(6), 997–1007. https://doi.org/10.1093/her/cym078