Measures
of Success:
Evaluation Results Show Scope of State's Asset Initiative
Assets for Colorado
Youth (ACY) is the first statewide initiative to have completed a comprehensive
evaluation of the impact of community asset-building efforts in communities.
Recently released findings of a two-year evaluation of the ACY Initiative,
conducted by OMNI Institute and funded by The Colorado Trust, shed light
on how the developmental asset framework has taken hold in Colorado and
the role ACY has played as a statewide leader in this effort. The findings
show that asset-building efforts help strengthen relationships, improve
organizational capacity and expand partnerships to improve the lives of
youth.
The OMNI evaluation
set out to capture the range of outcomes promoted by asset building in
organizations across the state. Through interviews, participant observation,
document reviews and surveys, the OMNI evaluation team determined the
catalyzing forces that led to the spread of assets in Colorado:
ACY's leadership in
innovation and engagement of diverse sectors and communities
- Asset champions
who served as local messengers, engaged communities in asset building
and advocated for youth
- Relationship building
between individuals and organizations on behalf of youth
- Organizational
and cultural innovations in applying the asset framework
- Youth engagement
in asset-building organizations
- Funding to support
training, local asset builders and the development of asset-related
products
Evaluation results
confirm that the asset framework has taken hold in Colorado. OMNI identified
more than 600 organizations in Colorado actively basing their work on
the 40 developmental assets. Groups integrating the assets report experiencing
greater organizational effectiveness in terms of youth engagement, staff
morale, articulation of goals and community networks.
In 1997, The Colorado
Trust created the ACY Initiative to help adults become more involved in
strengthening young people's lives. "This evaluation shows that ACY
has made great strides toward this goal," said Carol Breslau, senior
program officer at The Colorado Trust. "It has been wonderful watching
assets grow from an idea to a powerful influence in the lives of so many
young people and adults across the state."
Existing evaluation
research in the field of youth development is limited; evaluation tools
tend to focus solely on the measurement of risk and problem behaviors
in youth, and rarely include the results of promoting positive youth development.
OMNI's findings may help researchers and practitioners better understand
the asset-building process and measures of success.
The ACY evaluation
indicates what is needed to support positive youth development initiatives
at the community, statewide and national levels:
A need for strong
leadership to:
- Generate widespread
recognition and support of the asset framework
- Support the growing
base of asset builders
- Contribute to new
innovations, including cultural adaptations of the framework
- Work with policymakers
in strategically targeted arenas
Making positive youth
development intentional by:
- Tying the asset
message to everyday, culturally familiar experiences
- Developing partnerships
between organizations and youth
- Unifying organizational
policies and programming around asset building
- Utilizing the asset
framework to build new community partnerships on behalf of youth
- Heightening individual's
awareness of teachable moments
- Celebrating achievements
and rewarding asset builders
- Witnessing and
sharing the transformations of youth
"With these components
in place, the asset framework can become the common language and common
practice in organizational and community settings," said Suzanne
Kennedy-Leahy, senior researcher at OMNI.
For more information on the ACY evaluation, visit www.buildassets.org.
Not Everything that Counts
Can Be Counted:
The Value of Qualitative Data
By Dr. Nick Cutforth
Hanging on the wall
of Albert Einstein's office at Princeton was a sign saying, "Not
everything that can be counted counts, and not everything that counts
can be counted."
Surveys and pre- and
post-tests cannot fully capture the fits and starts of youth development
and asset-building programs, their forward and backward progress, or their
unique evolutionary and transformational dynamics. Qualitative data, on
the other hand, can illuminate our understanding of what people do, know,
think and feel as they participate in youth development and asset-building
initiatives.
Qualitative data are
increasingly being incorporated into studies of the cognitive, psychological
and social aspects of school-based and community-based youth development
programs. These data typically come in three forms:
- In-depth, open-ended
interviews
- Direct observations
- Analyses of documents
Interviews yield quotations
from people about their experiences, opinions, feelings and knowledge.
The data from observations provide detailed descriptions of program activities
and organizational or community processes. Documents can reveal historic
trends, recurring issues and common-but not conscious-realities that can
influence program delivery and success.
Qualitative data collection
methods can be emergent, artistic and interpretive. However, the qualitative
research process is rigorous: the researcher must make sense of multiple
sources of evidence by developing categories and themes that are mutually
reinforcing and that result in a faithful rendering of participants' experiences.
Results can provide rich and accurate pictures of the project under study.
For example, during
2002, DU researchers studied the impact of the Expect Success Project,
a parent engagement initiative led by Assets for Colorado Youth in five
Denver Public Schools. Through in-depth observations and interviews, we
were able to:
- Describe the implementation
of Expect Success
- Explain how teachers
and parents embraced the assets during workshops and meetings
- Provide poignant
testimonies illustrating the connections being forged between teachers
and parents
- Identify characteristics
of schools that influenced the receptivity of the school community to
Expect Success events and activities
Here are some of the
voices of Expect Success participants:
"My role is
to support the parents in helping their child, to support them in what
they're doing. Tell them, 'You're doing a great job.' Give them suggestions,
encourage them, and reinforce whatever is going on." -DPS Teacher
"I've learned how to help my family and how to improve our lives."
-DPS Parent
"My mom is more involved now. She came to the parent training and
follows the assets at home more. We have the 40 assets list on the fridge."
-DPS Student
Such insights cannot
be quantified. Qualitative data can preserve the richness and vitality
of experience. At the same time, such data can provide valuable insights
to youth development practitioners and policymakers that can result -often
in conjunction with quantitative data-in program improvements that bolster
program success and enhance the learning and developmental experiences
that ultimately improve the lives of youth.
Dr. Nick Cutforth is an associate professor at the University of Denver's
College of Education. He also coordinates DU's Community Based Research
Project (ncutforth@du.edu).
Questions
Answered:
Evaluating Positive Youth Development Efforts
Assets for Colorado
Youth staff field numerous calls each month from youth development advocates
across the country inquiring how to measure the effectiveness of their
work. Because the asset approach is a framework for social change, and
not a model for intervention, measuring the presence of outcomes from
asset-building efforts necessitates a new approach. Assessment tools designed
to measure reduction of risk may not serve to capture the accumulation
of support and engagement of youth. Youth advocates will find that the
ACY evaluation has implications for positive youth development initiatives
wanting to quantify their success.
Following are representative questions ACY staff often hear and our responses
based on what we have learned:
Question: Should
I gauge my program's effectiveness by measuring increases in the number
of youths' assets?
Answer: ACY
has found that measuring increases in the number of assets in youth involves
complex determinants that are not easily measured. The ACY evaluation
focused on defining and quantifying effective youth engagement in asset-building
organizations as a gauge of how deeply the asset framework was integrated.
This focus helped distinguish between "asset building" and "asset
integration."
Evaluation results
demonstrated that positive changes witnessed in youth were inextricably
linked to the process of asset integration and transformation within organizations
and communities.
Asset integration
in organizations can be described as an upward moving spiral upon which
groups move from pre-engagement to full engagement in incorporating a
strength-based approach to working with youth. Using a case-study approach,
OMNI evaluators determined the organizational benefits - and the benefits
to youth - as organizations moved up the Asset-Building Spiral. The Asset-Building
Spiral is a tool that youth-serving organizations may find valuable in
evaluating the effectiveness of their asset-integration efforts. (Go to
www.buildassets.org and click on Evaluating Impact to find out more about
the Asset-Building Spiral.)
Question: How can we best communicate these findings to funders?
Answer: The findings from the ACY evaluation demonstrate positive impacts
on both adults and youth in organizations integrating the asset framework.
Not only do organizations experience better alignment between individual
staff efforts and broader organizational goals and the ability to more
effectively form community collaborations on behalf of youth, they also
witness the transformations of youth participants who, because they are
given meaningful roles and opportunities in the organization, show positive
impacts. Youth demonstrated increases in self-confidence, self-esteem
and giving back to the community.
The findings
can help demonstrate positive outcomes associated with asset integration.
Impacts
on Youth
Findings Show Youth's
Lives Strengthened
The OMNI evaluation
of Colorado's asset initiative explored the impacts of asset building
on youth. Focus groups with youth from selected asset-building organizations
across Colorado examined the quality of youth engagement in programs designed
to intentionally increase developmental strengths in youth. The findings
uncovered many areas in which youth's lives were strengthened, including
a greater sense of belonging as a result of increased positive adult recognition
of youth.
Said one youth interviewee:
"In the community, they (adults) recognize you and know you're there,
and they remember you and that feels good. That really feels good."
OMNI's findings show
that transformations in youth-serving organizations (YSOs) are creating
positive shifts in support of youth:
Traditional youth
involvement:
- Youth are present,
but have limited engagement in positive opportunities or supports
- Youth do not play
a role in program planning and decision-making
- Youth activities
are not tied to the asset framework
- Measures of youth
involvement focus solely on their participation level
Asset integration
in youth-serving organizations:
- Adults share some
degree of power with youth
- Youth have significant
levels of responsibility within the organization
- Youth are more
aware of opportunities and feel empowered to act on them
- Adults and youth
work side by side
- Strong, enduring
relationships exist between youth and adults
- Youth and adult
asset champions are present
Youth outcomes:
- A sense of belonging
- Stronger and more
effective relationships
- Self-confidence
- Skill building
- A positive view
of the future
"Collectively,
the efforts of ACY and other asset builders have begun to transform the
social institutions-including government, schools and community-based
organizations-that impact the lives of youth in Colorado, as well as the
quality of adult and youth interactions, " said Suzanne Kennedy Leahy
of OMNI.
OMNI's findings advance
the knowledge of how to improve and sustain youth engagement in experiences
that are likely to enhance their development and well being.
One young focus group
participant stated, "With assets in my life, I now show more empathy,
and it's easy to make others feel good about themselves."
Download
a summary of OMNI's evaluation findings.
The full evaluation
report will be released in March 2003.
|
