August 2005 // Volume 43 // Number 4 // Feature Articles // 4FEA4

Previous Article Issue Contents Previous Article

Idaho 4-H Impact Study

Abstract
A study measured the impact that the 4-H experience has on the quality of life of young people. Data were collected from 5th, 7th, and 9th grade students in Idaho. Students were selected from four randomly selected schools in each of the 16 randomly selected counties across the state. There were 3,601 surveys returned from 53 schools. Youngsters who have participated in 4-H for 2 years or more are less likely to engage in "at-risk" behaviors such as drinking alcohol, shoplifting, drug use, damaging property, or smoking cigarettes than their non- 4‑H classmates.


Jeff Goodwin
Director, 4-H Youth Development Programs
Colorado State University
jeff.goodwin@colostate.edu

Cyndi Barnett
Graduate Student
Department of Agricultural Education & Studies
Iowa State University
cyndib@iastate.edu

Michele Pike
County Extension Educator
University of Idaho
Nez Perce County
mpike@uidaho.edu

Joey Peutz
County Extension Educator
University of Idaho
Canyon County
joeyp@uidaho.edu

Rhea Lanting
County Extension Educator
University of Idaho
Twin Falls County
rhlantin@uidaho.edu

April Ward
County Extension Educator
University of Idaho
Bannock County
aprilw@uidaho.edu


Background

In 2002, many states passed the 100-year mark for the existence of the 4-H program. Will there be a 200-year mark observed for the 4-H program in your state in 2102? The answer to that question lies in the ability of the 4-H program to demonstrate to citizens and elected officials the continued relevance and worth of this 100-plus-year old institution. In the past, the value of 4-H has been assessed and conveyed by anecdotal success stories and the "gut-feeling" of those familiar with the program. Those methods are no longer adequate to ensure its continued impact and existence. This study described here was designed to provide sound, research-based information to the public about the effect of 4-H membership on the lives of young people.

Does the 4-H program and the associated activities these young people become involved in just give them something to do in order to keep them out of trouble--or are life skills being developed to help them become successful, contributing, and competent adults? This question seems to be answered in the study conducted by Astroth and Haynes (2001). Astroth and Haynes reported that their research showed that 4-H participants were more likely than other youth to:

  • Succeed in school, getting more A's than other kids
  • Be involved as leaders in their school and community
  • Be looked to as role models by other youth
  • Help others in the their community (p. 9)

Furthermore, Astroth and Haynes say that the 4-H kids surveyed told them that they were less likely than other kids to:

  • Shoplift or steal (3 times less likely)
  • Use illegal drugs of any kind to get high (2 times less likely)
  • Smoke cigarettes
  • Damage property for the fun of it (2 times less likely)
  • Skip school or cut classes without permission (p. 10)

Methodology and Data Analysis

The Idaho 4-H Impact Study is actually a replication of the Astroth and Haynes study. The study was conducted in a nearly identical format, with only minor adaptations made for the situation in Idaho. The strengths of the Astroth and Haynes study were that it provided a simple method to acquire a random sample of 4-H and non-4-H members in a population of youth that represent the state.

One perceived weakness in the Astroth and Haynes study was that there were no ethnic demographic data collected. This was because, given the low ethnic minority populations in Montana, the researchers felt that there would not be a large enough number of surveys completed by this audience to conduct an accurate statistical analysis of the data. A valid reason for not asking the ethnic demographic question is that anonymity of the subjects could be comprised if there were only one or two ethnic minorities present in a classroom where data were collected. For these reasons, no ethnic demographic question was present in the Idaho study, either.

Organization of methods in this study were intended to determine the awareness and attitudes of students in 5th, 7th, and 9th grades within the state of Idaho concerning the depth of the perception of their own development and the impact 4-H may have had on them. These grades were chosen because they are a broad representation of the audience served by the 4-H program.

The populations used in this study consisted of a stratified random sample of 18 selected counties of the 44 in the state of Idaho. A stratified random sample was used to ensure an equal distribution of counties in all four Extension Districts in the state.

In each county, four schools were randomly selected. The list of schools was retrieved from the 2001-2002 Idaho Educational Directory. Using the complete list of schools that had grades 5th, 7th, and 9th within their populations; a computer random generator was utilized to determine the random selections. This group composed a total of 6,334 possible students in all three grades.

The questionnaire used to assess the ways young people spent their time out-of-school and how that use of time may affect their academic, emotional, social, and cognitive development was patterned after the Astroth and Haynes (2001) model, which was used by the Montana State 4-H Office for a similar study. A request was made to The Search Institute to use some of their questions, and that approval was granted. The questionnaire included basic demographic and family-oriented questions as well as questions categorized into seven subscales:

  • Positive identity--personal power, self esteem
  • Social competency-®leadership, planning and decision making, resistance skills
  • Relationships with adults
  • Self confidence
  • Empowerment
  • Kindheartedness
  • Skills

The student questionnaire was divided into seven sections and consisted of 73 questions. In addition to an English version of the survey, a Spanish version was available for those pupils who had a hard time reading or comprehending English. The format of the questions consisted of fill-in-the-blank, yes/no, essay, circle those that apply, and multiple-choice. The reliability of the Likert-type items in this survey was assessed by Cronbach's standardized coefficient alpha, a measure of the extent to which a summated rating scale can be formed meaningfully from a set of correlated items. Cronbach's (1951) standardized alpha has a maximum value of 1 and a minimum that can be less than one (when items have opposite valence, that is, some are worded negatively and others positively).

In these results, with one exception, the alpha values are strong. For items q4 through q22 (measuring positive identity and social competency), alpha = .8591; for items q32 through q44 (measuring self-confidence, character, and empowerment), alpha = .8257; for items q48 through q51 (measuring skills learned), alpha = .5729; and for items q63 through q69 (measuring the impact of 4-H), alpha = .8658. Items on the questionnaire addressed several areas. The survey included basic demographic and family-oriented questions as well as questions categorized into seven subscales: 1) Positive identity--personal power, self esteem, 2) Social competency--leadership, planning and decision making, resistance skills, 3) Relationships with adults, 4) Self confidence, 5) Empowerment, 6) Kindheartedness, and 7) Skills.

The Idaho questionnaire was pilot tested in the Filer School District in April 2002. The pilot test was done to ensure that the survey was both readable and understandable and to determine time constraints, if any. This was also when the Spanish version of the survey was discovered as necessary. A total of 185 surveys were gathered at this time.

Questionnaires were not coded in advance. However, when each class finished within the participating grade, the instruments were wrapped with an accompanying tracking sheet completed by the facilitator with pertinent information.

Collection of the Data

The questionnaires were given to the corresponding county Extension educators within the randomly selected counties. Packets of instructions on how to administer the instrument were sent to all county educators within these randomly selected counties. Initial contact was made by the educators to the school superintendents and/or principal to gain permission to conduct the survey in their schools. Actual execution of the questionnaires was conducted in the fall of 2002 and early spring of 2003. The parental "opt out" letters were sent previous to the actual administration of the study. On April 3, 2003, the survey portion of the study was declared complete. Response rates are reported in Table 1.

Table 1.
Response to the Survey Instrument

 

Surveys Returned

Respondent Groups

Frequency

Counties

16

Schools

53

Students

3601

The following table shows the actual numbers of responding districts and the corresponding schools:

Table 2.
Responding Districts and Schools

Counties

Schools

Districts

Planned

Actual

Planned

Actual

1

4

2

72

53

2

4

4

   

3

5

5

   

4

5

5

   
         

Logistic regression using the chi-square (χ2) test was performed for analysis. Frequencies and percentages were run on the questions regarding extracurricular participation, negative behaviors, positive identity, social competency, contact with adults, self confidence, empowerment, caring about others, skills learned, skipping school, age, grade, gender, household size, family types, and impact of 4-H for members.

Findings and Discussion

Respondents were asked to indicate a number of background characteristics to help with statistical analysis. Of the 3,601 respondents who participated in the study, 50.2 % were male, and 49.8 % were female. Of the 3,601 respondents to the questionnaire, 935 reported they had been involved in the 4-H program for at least one year. Of the 935 respondents who had been in 4-H, 555 of them had been a member for 2 years or more. (Table 3) Data analysis of 4-H membership in these findings is confined to 4-H membership of 2 or more years.

Table 3.
Frequencies and Percentages According to 4-H Participation (2 Years or More)

4-H Participation

n=

Percent

4-H Member

555

16

Non-4-H Member

3046

84

Total

3601

100

Students were asked to specify which grade they were in at the time of the survey: 5th, 7th, or 9th. Of the participants, 43.3 % indicated the 5th grade, 25.3 % indicated the 7th grade, and 28.6 % indicated the 9th grade. (Table 4)

Table 4.
Frequencies and Percentages According to Grade

Grade

n=

Percent

5th

1558

43.26

7th

911

25.29

9th

1,029

28.57

Missing

103

2.88

Total

3,601

100

All respondents were asked to designate where they lived at the time of the study. Of the 3,601 students, 15.2 % lived on a farm, 33.8 % lived in the country but not on a farm, 47.4 % lived in town, and 128 did not indicate where they were living.

To determine the involvement in the identified at-risk behaviors, respondents answered yes or no to the wording on the survey instrument as follows:

During the past year did you . . .

  • Cheat on a test?
  • Drink any alcohol without parental permission?
  • Shoplift?
  • Use any drugs like marijuana, methamphetamines, or cocaine; or sniffed glue or other fumes to get high?
  • Drive a car when you've been drinking?
  • Carry a gun to school?
  • Smoke cigarettes?
  • Use smokeless tobacco (such as Copenhagen/Skoal)?

As 4-H membership versus non-4-H membership was compared with at-risk behaviors, the following differences are observed. When all grades (5th, 7th, and 9th) are in the population, 4-H members were less likely to engage in shoplifting, drug use, damaging property just for the fun of it, and smoking (Table 5).

Table 5.
Percentage of 4-H and Non-4-H Students Engaged in At-risk Behaviors (4-H membership = 2 or more years)

Variable

4-H Students

Non-4-H Students

Cheated on a test

25.7 (571)

25.4 (2,976)

Drank alcohol

10.4 (568)

12.3 (2,975)

Shoplifted

4.6* (568)

6.9* (2,961)

Used drugs

4.7* (569)

7.3* (2,972)

Drove after drinking

1.4 (568)

2.4 (2,973)

Damaged property

7.9* (568)

11.1* (2,972)

Carried a gun

0.4 (569)

0.8 (2,975)

Smoked

6.2* (567)

8.6* (2,977)

Used smokeless tobacco

2.8 (568)

2.6 (2,972)

P < 0.05; * = significant difference; (#) = (Total Observations)

The argument could be made that many of the 5th and 7th grade respondents have not yet been exposed to many of the at-risk behaviors asked about in the study. For example many 5th and 7th graders have not had the opportunity to drive, much less drink and drive. Perhaps a more realistic picture emerges when only 9th grade responses are examined for these at-risk behaviors (Table 6). 4-H Students in the 9th grade were less likely to drink alcohol, shoplift, use drugs, damage property, or smoke cigarettes than their non-4-H classmates.

Table 6.
Percentage of 4-H and Non-4-H 9th Grade Students Engaged in At-risk Behaviors (4-H membership = 2 or more years)

Variable

4-H 9th Students

Non-4-H 9th Students

Cheated on a test

40.2 (199)

44.6 (2,976)

Drank alcohol

16.6*(199)

30.3* (2,975)

Shoplifted

5.6* (198)

11.6* (2,961)

Used drugs

7.0* (199)

15.5* (2,972)

Drove after drinking

2.0 (199)

4.5 (2,973)

Damaged property

10.1* (199)

18.8* (2,972)

Carried a gun

0.5 (199)

0.7 (2,975)

Smoked

11.6* (198)

18.1* (2,977)

Used smokeless tobacco

3.5 (199)

5.3 (2,972)

P < 0.05; * = significant difference; (#) = (Total Observations)

As shown in Table 7, 4-H members were more likely to report that they received mostly A's than non-4-H students.

Table 7.
School Performance for 4-H and Non-4-H Youth

Performance

4-H

Others

Mostly A's

35.9*

23.9*

A's and B's

37.3*

38.4*

Mostly B's

5.6*

7.8*

B's and C's

14.3*

17.1*

Mostly C's

1.6*

3.6*

C's and D's

3.7*

6.0*

Mostly D's

0.7*

1.2*

Less than D's

1.0*

2.1*

Total Observations

574

2,870

P < 0.05; * = significant difference

This research revealed that 4-H members are more likely to succeed in school (Table 7), help others within their communities (Table 8), and be involved as leaders in their school (Table 9). By being active in leadership roles, these 4-H members are preparing themselves to become productive citizens within their community and world.

Table 8.
Percentage of Youth Who Have Helped Others (4-H and Non-4-H)

Variable

4-H

Non-4-H

Helped others

95.0* (575)

87.7* (2,917)

Involved in help project

68.0* (575)

46.6* (2951)

Given time or money to Charity

61.8* (576)

52.7* (2,946)

Helped sick, poor, others

53.8* (573)

45.7* (2940)

P < 0.05; (#) = total observations; * = significant difference

Table 9.

Percentage of Youth Holding Leadership Positions

Variable

4-H

Non-4-H

Elected leadership

21.0* (572)

13.7* (2951)

Held leadership position

21.5* (573)

14.2* (2955)

Committee chair

5.4* (575)

5.2* (2,958)

Committee member

18.3* (575)

10.2* (2,943)

P < 0.05; * = significant difference; (#) = total observations

 

Data shown in Table 10 below indicate Idaho 4-H youth have better relationships with adults than those students who are not active in 4-H. Youth were asked if there was an adult with whom they felt comfortable going to for help when they had an important question about their life. Seventy-six percent of active 4-H members said they did have someone they could go to other than a parent. Additionally, 79.2 % of Idaho 4-H youth said they have had a good conversation in the past month with one of their parents that had lasted longer than 10 minutes. Comparatively, when asked if they had a good conversation that had lasted longer than 10 minutes with someone besides their parents, 63.8 % of Idaho 4-H members said they had.

Table 10.
Conversations with Adults

Variable

4-H

Non-4-H

If you had an important question about your life, is there an adult (other than your parents) to whom you felt comfortable going to for help?

76.7* (572)

72.1* (2,938)

In the last month, did you have a good conversation with one of your parents that lasted 10 minutes or more?

79.2* (577)

71.0* (2935)

In the last month, did you have a good conversation with an adult (other than your parents) that lasted 10 minutes or more?

63.8* (572)

57.3* (2,935)

P < 0.05; * = significant difference; (#) = Total Observations

When asked if they could go to their parents to discuss issues related to drugs, alcohol, sex, or any other serious issue, 4-H members were more likely to do so than were non-4-H members (Table 11).

Table 11.
Percentage of Youth Who Would Talk to Parents

Variable

4-H

Non-4-H

About drugs

81.2* (579)

72.6* (2,936)

About alcohol

79.8* (578)

72.0* (2,922)

About sex

69.1* (573)

63.0* (2,920)

About other issues

82.2* (574)

73.7* (2,923)

P < 0.05; (#) = total observations; * = significant difference

There were 33 questions on the survey that related to the respondents attitude or perceptions about the world around them. The authors have chosen to use the term "world view" to describe this series of questions as a whole. This series of questions are further broken down into five groups of:

  • Positive Identity--personal power, self-esteem, sense of purpose, and positive view of the future
  • Social Competency--planning, interpersonal competency, and resistance skills
  • Social Competency--leadership
  • Self-confidence, Character, and Empowerment
  • Skills Learned

Responses to these world-view questions were on a Likert-type scale: strongly agree, agree, neutral, disagree, and strongly disagree. As the data is reviewed, please note that five of the questions are reported in the opposite direction of the rest of the questions and are reported as disagree and strongly disagree. Four of these questions are noted in Table 14 with a superscript "a," while the fifth is noted in Table 16. The first group of Positive Identity Statements shows that there was significant positive difference in the way 4-H members responded to the following statements (Table 12).

Table 12.
Percentage of 4-H and Non-4-H Students Who Agree or Strongly Agree With the Following Identity Statements (%)

Statements

4-H Students

Non-4-H Students

When things don't go well for me, I am good at finding a way to make things better.

67.0* (570)

62.2* (2,963)

I have little control over the things that will happen in my life.a

52.9* (569)

47.7* (2,949)

On the whole, I like myself.

81.8* (566)

75.9* (2,943)

At times, I think I am no good at all.a

39.8 (566)

38.6 (2957)

All in all, I am glad I am me.

85.1 (572)

82.6 (2,971)

I feel I do not have much to be proud of.a

77.1* (564)

61.5* (2,944)

Sometimes I feel like my life has no purpose.a

66.9* (568)

59.6* (2.952)

When I am an adult, I'm sure I will have a good life.

77.3 (572)

74.1 (2,972)

P < 0.05; * = significant difference; (#) = (Total Observations)
a = Disagree and strongly disagree responses;

 

The next group of social competency questions also revealed that 4-H members were more likely to respond in a positive manner to the statements of:

  • I'm good at planning ahead.
  • I feel really sad when one of my friends is unhappy.
  • I am good at making and keeping friends.
  • I know how to say "no" when someone wants me to do things I know are wrong or dangerous.
Table 13.
Percentage of 4-H and non 4-H Students That Agree or Strongly Agree with the Following
Social Competency Statements (%)

Statements

4-H Students

Non-4-H Students

I am good at planning ahead.

57.7* (572)

50.6* (2,961)

I think through all of the good and bad results of different decisions before making a decision.

54.8 (573)

51.5 (2,951)

I care about other people's feelings.

86.2 (574)

83.4 (2,974)

I feel really sad when one of my friends is unhappy.

77.4* (571)

73.0* (2,951)

I am good at making and keeping friends.

81.0* (574)

76.9* (2,971)

I know how to say "no" when someone wants me to do things I know are wrong or dangerous.

88.0* (576)

83.8* (2,961)

I stay away from people who might get me in trouble.

69.1 (573)

67.4 (2,951)

P < 0.05: * = significant difference; (#) = (Total Observations)

The data in Table 14 shows that 4-H members were more likely to respond in a positive manner to 10 out of 13 of the Self Confidence, Character, and Empowerment statements than non-4-H members.

Table 14.
Percentage of 4-H and Non-4-H Students that Agree Or Strongly Agree With the Following Self Confidence, Character, and Empowerment Statements (%)

Statements

4-H Students

Non-4-H Students

I can make my own decisions.

79.4 (578)

76.0 (2,949)

I can do things on my own.

79.1* (575)

72.6* (2,940)

I set goals.

76.3* (578)

69.6* (2,948)

I like to try new things.

84.4 (578)

81.3 (2,953)

Ten years from now, I think I will be very happy.

80.2* (577)

76.3* (2,959)

I am responsible for my actions.

89.1 (579)

88.0 (2,951)

Adults in my town or city make me feel important.

60.4* (576)

53.6* (2,938)

Adults in my town or city listen to what I have to say.

52.5* (573)

44.3* (2,937)

Adults in my town or city don't care about people my age.a

65.4* (572)

58.8* (2,925)

In my town or city, I feel like I matter to people.

58.7* (576)

51.5* (2,922)

In my family, I feel useful and important.

78.3* (577)

72.5* (2,953)

I'm given lots of chances to help make my town or city a better place to live.

53.0* (576)

41.7* (2,937)

Students help decide what goes on at my school.

65.6* (572)

58.3* (2,930)

P < 0.05; * = significant difference; (#) = (Total Observations)
a = Disagree and strongly disagree responses

The data reported in Table 15 show that 4-H members were more likely to volunteer in class to lead activities and that they feel other kids look up to them and follow their example (Table 16).

Table 15.
Percentage of 4-H and Non-4-H Students that Agree or Strongly Agree With the Following Social Competency--Leadership Statements (%)

Statements

4-H Students

Non-4-H Students

I volunteer in class to lead activities.

52.0* (569)

41.7* (2,943)

I can meet and greet new people easily.

61.7 (574)

59.2 (2,950)

I am comfortable in new situations.

47.5 (570)

45.2 (2,938)

I feel other kids look up to me and follow my example.

53.4* (569)

47.8* (2, 950)

P < 0.05; * = significant difference; (#) = (Total Observations)

The data in Table 16 suggest that 4-H members are more comfortable at public speaking, organizing, and at managing money.

Table 16.
Percentage of 4-H and Non-4-H Students that Agree or Strongly Agree with the Following Skills Learned Statements (%)

Statements

4-H Students

Non-4-H Students

I have good written record keeping skills (such as keeping a journal or diary).

38.3 (561)

36.9 (2,884)

I am comfortable giving a speech or demonstration in front of people.

48.5* (575)

36.8* (2,890)

I am a good organizer.

58.8* (577)

50.7* (2,900)

I am a good money manager.

60.0* (573)

54.4* (2,915)

P < 0.05; * = significant difference; (#) = (Total Observations)

Conclusions and Recommendations

The data collected in the study reported here support the conclusions made by Astroth and Haynes (2001). Authenticating the significance of successful youth development programs like 4-H should be a priority in the minds of facilitators, legislature, and educators. 4-H activities don't simply teach youth skills in agriculture and home economics, but include nonformal, experiential educational programs that teach youth valuable life skills (Boyd, Herring, & Briers, 1992).

The study supports the conclusions made by Boyd et al. (1992). It is hoped that the study will also be useful in the future support of the 4-H program in Idaho as well as other states. Federal, state, and county government budgets to deliver mandated programs have been stretched to their limits in many areas of the country. The future existence of non-mandated programs such as Cooperative Extension and 4-H educational efforts in the future will depend on informing elected officials, potential 4-H families, and potential donors about the value of this program. The 4-H program must demonstrate the positive impact that it has on youth and families if it is to remain a relevant educational program in this country. Data such as that obtained in the research study reported will help to fill this need.

One of the major recommendations of the principle investigators of the study is that other states replicate it. As a larger group of states duplicate the original Montana Out of School Time/4-H Impact Study, those data could be compiled into a larger and much stronger research project to benefit 4-H nationwide.

Another recommendation is to communicate these data in a systematic and effective manner to the elected officials, opinion leaders, school officials, and citizens of the state of Idaho. Such an effort could have the following results:

  • Increase awareness and support for the 4-H program at the local and state level.
  • Enhance 4-H volunteer leader recruitment and retention.
  • Increase parent interest in having their kids involved in the 4-H program.
  • Enhance 4-H support from University of Idaho faculty who are not directly involved in the program.

"I have developed presentation skills, work skills, social skills, organization skills, and participation skills."
9th grade 4-H member, Bingham County, ID

The bottom-line observation by the investigators is that 4-H does make a difference in the lives of young people. Now it is time to share this information with others.

"I have learned better organizing and money management skills. I have learned how to work hard and be dedicated to my work."
9th grade 4-H member, Oneida County, ID

References

Astroth, K. A., & Haynes, G. W. (2001). Final report of the Montana public school students' out-of-school time study. Bozeman, MT: Montana State University.

Astroth, K. A., & Haynes, G. W. (2002). More than cows & cooking: Newest research shows the impact of 4-H. Journal of Extension [On-line], 40(4). Available at: http://www.joe.org/joe/2002august/a6.shtml

Boyd, B. L., Herring, D. R., & Briers, G. E. (1992, Winter). Developing Life Skills in Youth. Journal of Extension [On-line], 30(2) Available at: http://www.joe.org/joe/1992winter/a4.html

Idaho Educational Directory (2001-2002). Compiled and produced by Idaho's State Department of Education. Available On-line at: http://www.sde.state.id.us/admin/eddirectory/