A Reach, Effectiveness, Adoption, 
Implementation, Maintenance Evaluation of 
Weekend Backpack Food Assistance 
Programs
Authors: Carmen Byker Shanks & Samantha Harden
This is a postprint of an article that originally appeared in American Journal of Health Promotion 
on September 2016. https://dx.doi.org/10.4278/ajhp.140116-QUAL-28
Shanks, Carmen Byker, and Samantha Harden. "A Reach, Effectiveness, Adoption, 
Implementation, Maintenance Evaluation of Weekend Backpack Food Assistance Programs." 
American Journal of Health Promotion 30, no. 7 (September 2016): 511-520. DOI: 10.4278/
ajhp.140116-QUAL-28.
Made available through Montana State University’s ScholarWorks 
scholarworks.montana.edu 
Nutrition; Underserved Populations
A Reach, Effectiveness, Adoption, Implementation,
Maintenance Evaluation of Weekend Backpack
Food Assistance Programs
Carmen Byker Shanks, PhD; Samantha Harden, PhD
Abstract
Purpose. The purpose of this study was to evaluate an ongoing statewide weekend backpack
program through the Reach, Effectiveness, Adoption, Implementation, and Maintenance
(RE-AIM) framework.
Design. Mixed-methods inquiry was used to explore the reach, effectiveness, adoption,
implementation, and maintenance of backpack programs within Montana.
Setting. Study participants completed audio-recorded one-on-one phone interviews.
Participants. Key informants (e.g., managers at food banks, staff at participating schools,
policy makers) were purposively sampled (N¼ 20).
Method. Semistructured interviews were conducted to gather data to describe each RE-AIM
dimension. Interviews were transcribed verbatim and deductively (i.e., using RE-AIM as
themes) coded for meaning units, placed into higher-order categories, and summarized in
narrative. Supporting quantitative data (e.g., the proportion of eligible students that joined the
program, rate of school-level adoption) were calculated using descriptive statistics.
Results. Backpack programs with a broad reach and evidenced effect may be appealing to
adopt. Weekend food bags cost an average $3.87 (SD 6 .94) and there were some positive (i.e.,
ease, protecting participants’ privacy) and very few negative (logistical) components of
implementation. Collaborators and community partners are necessary for long-term
sustainability.
Conclusion. Backpack programs are widespread and have potential to relieve weekend
hunger; however, more efforts need to be made to end childhood hunger. (Am J Health Promot
0000;00[0]:000–000.)
Key Words: Food Security, RE-AIM, Program, Children, Prevention Research.
Manuscript format: research; Research purpose: program evaluation; Study design:
qualitative; Outcome measure: productivity; Setting: local community/state; Health
focus: nutrition; Strategy: policy; Target population age: youth; Target population
circumstances: income level
PURPOSE
Twenty percent of households in the
United States with children were iden-
tified as food insecure in 2012 through
the U.S. Department of Agriculture
Economic Research Service annual
food security survey.1
Among these households, 10% of
children were also reported as food
insecure. When children are hungry,
their physical,2 mental,3 and behavior-
al and emotional health4 is compro-
mised.
There are policy measures in place
to help protect households with chil-
dren from food insecurity in the
United States. The recent reauthoriza-
tion of the Healthy, Hunger-Free Kids
Act of 2010 included legislation for the
National School Lunch and Breakfast
Programs, the Supplemental Nutrition
Assistance Program, the Special Sup-
plemental Nutrition Program for
Women, Infants, and Children, the
Child and Adult Care Food Program,
and the Summer Food Service Pro-
gram.5 These programs provide sup-
plemental assistance to purchase food
and provide nutrition education to
recipients.
Beyond public assistance, communi-
ties also strategize several private ways
to provide immediate food assistance
though emergency food programs
such as food banks, food pantries, and
community feeding programs.6 For
example, to solve the problem of
children repeatedly arriving at school
on Monday hungry as a result of their
nutrient needs not being satisfied over
the weekend, communities across the
nation have implemented programs
within schools where easy-to-prepare
Carmen Byker Shanks, PhD, is with the Food and Health Lab in the Department of Health and
Human Development, Montana State University, Bozeman, Montana. Samantha Harden,
PhD, is with the Human Nutrition, Foods and Exercise Department, Virginia Tech, Blacksburg,
Virginia.
Send reprint requests to Carmen Byker Shanks, PhD, Department of Health and Human
Development, Montana State University, 222 Romney Gym, Bozeman, MT 59717; carmen.byker@
montana.edu.
This manuscript was submitted January 16, 2014; revisions were requested March 15, May 4, August 9, and November 11,
2014; the manuscript was accepted for publication January 25, 2015.
Copyright  0000 by American Journal of Health Promotion, Inc.
0890-1171/00/$5.00 þ 0
DOI: 10.4278/ajhp.140116-QUAL-28
foods are distributed to students on
Fridays to help reduce weekend hun-
ger.7,8 Feeding America runs the larg-
est program in the nation, the
BackPack Program, serving 230,000
children each year and working
through food banks in states across the
United States.9
In Montana, 14.1% of households
were identified as food insecure in
2012.1 Several efforts to increase food
security for children exist in the state,
including free or reduced-price (F/R)
breakfast and lunch programs, sum-
mer lunch programs, and the Montana
Partnership to End Childhood Hunger
(http://mt.nokidhungry.org/
partnership-end-childhood-hunger).
Among these programs, the Monta-
na Food Bank Network (MFBN) initi-
ated the first Feeding America
BackPack Program in 2008 at one
school and the program has since
grown statewide.10 To be considered a
Montana Feeding America BackPack
Program, the school must follow three
standards: (1) approved agency in
good standing with MFBN that com-
plies with all applicable federal and
local statutes, ordinances, and regula-
tions; (2) healthy, nutritious food must
be distributed to children free of
charge; and (3) backpacks must be
provided a minimum of once a month
during the school year.10
Other food banks in Montana have
also adopted the original model pro-
posed by Feeding America, but operate
independently of Feeding America and
MFBN. During the 2011–2012 school
year, the MFBN Feeding America
BackPack Program served 45 K-12
schools through 17 food banks or
pantries, and four other food banks
served 25 other K-12 schools through a
similar model (i.e., Kid Packs, Back
Pack Program, Backpack for Kids, and
Kids Pack Program) in Montana.
BackPack and similar programs are
referred to as backpack programs
throughout the remainder of this
paper.
The collective scope of the success
and sustainability of the five backpack
programs in Montana is unknown.
Stakeholders in Montana identified
that evaluation would assist in program
development and a collective approach
was taken to researching the combined
function of backpack programs across
the state. The Reach, Effectiveness,
Adoption, Implementation, and Main-
tenance (RE-AIM) framework provides
a guiding metric to better understand
the ability of these programs to reach
those who need food relief, the effec-
tiveness of relieving weekend hunger,
fidelity to the programmatic model,
the rate of adoption, and sustainability
of the weekend backpack food assis-
tance program.11 The multifaceted
dimensions of the RE-AIM framework
capture internal and external validity
factors in order to increase the gener-
alizability of the program, policy, or
practice. Although RE-AIM reviews on
food security programs do not pres-
ently exist, we hypothesize that part-
nerships would influence the
generalizability, program tailoring, and
sustainability of a community food
assistance program.
The purpose of this study was to (1)
evaluate backpack programs across
Montana to understand the collective
public health impact of the emergency
food program and (2) provide current
and potential adopters with informa-
tion to guide future practice. The
results identify adaptations to the food
security program to ensure setting-level
program sustainability and individual
access.
DESIGN
Overall, the methodology of this
study aligns with a RE-AIM public
health evaluation. As such, this pri-
marily qualitative study utilized Glas-
gow and colleagues’ 21-item measure12
to ensure the reporting on internal
and external validity factors to enhance
generalizability of research findings.
Quantitative measures were utilized to
support qualitative data. Guided by this
21-item measure, represented in Table
1, quantitative or qualitative measures
were developed to obtain information
wherever applicable. For instance, the
backpack programs provide anonymity
for participants and collecting individ-
ual dietary data is not within protocol.
As such, the entire dimension of
effectiveness is based on intent-to-treat
measures (e.g., Healthy Eating Index
[HEI]–2010 is used to understand the
nutrient quality and adequacy being
delivered).13 Effectiveness of the back-
pack program to deliver sufficient
nutrients in other areas of the United
States has been measured through HEI
previously.14 HEI-2010 measures com-
pliance with the 2010 Dietary Guide-
lines for Americans.13 HEI-2010 uses
standards that are expressed as either a
percentage of calories or per 1000
calories, separating diet quality from
quantity. The sum of the scores for the
food components is the total HEI-2010
score, which ranges from 0 to 100, with
a higher score indicative of a more
healthful diet. In order to determine
the nutrient value of a typical back-
pack, the investigators randomly se-
lected one food bag (i.e., of those that
would be delivered to the students)
from each participating food bank.
Food bag nutritional content was ana-
lyzed by entering food codes into
Nutritionist Pro Diet Analysis Module
(version 2.5; Axxya Systems, Stafford,
Texas) and using HEI-2010 SAS code
(version 9.2; SAS Institute Inc., Cary,
North Carolina), provided by the
USDA Center for Policy and Promo-
tion.15 In addition to the HEI-2010,
other items that determined the effect
of the backpack program on relieving
weekend hunger and reaching those
most in need as well as information on
program attrition were measured
through interviews (reported in Table
1). Data was also collected through
secondary sources, such as food bank
program Web sites and manuals and
Montana Office of Public Instruction
public F/R meal eligibility data and
school enrollment data.16–21 This study
was conducted from July 2012 to
August 2013, with key informant inter-
views and data collection occurring
during the first 5 months and qualita-
tive and quantitative analysis occurring
during the remaining 7 months.
SETTING
The lead author interviewed key
informants (e.g., school and food bank
staff at all levels) during one-on-one
audio-recorded telephone interviews
to provide qualitative data based on the
RE-AIM dimensions. The lead author
took notes about points that were
important to follow up about at a later
time in the interview and key concepts
that the participant discussed.
PARTICIPANTS
In order to gain a collective and
statewide perspective of the program,
the researchers utilized two purposive
sampling strategies identified by Pat-
ton.22 These included maximum vari-
ation sampling, where participants
were invited based on an array of
positions, and snowball sampling,
where additional participants were
identified by study participants and
then interviewed by the lead author to
better understand phenomena within
this program. The researchers sampled
for participants until at least three
individuals from an individual pro-
gram (total programs across Montana
[N ¼ 5]) consented to contribute data
to the statewide perspective. In addi-
tion, researchers recruited individuals
with a state policy perspective about
child hunger and backpack programs
only and individuals with a state policy
perspective and local involvement in
backpack programs combined. This
approach ensured triangulation of
sources from people with different
positions in the system and therefore
was able to provide varied viewpoints.22
METHOD
The semistructured interview script
was developed to align with Glasgow
and colleagues’ 21-item measure.12
Interview questions are included in the
‘‘Measure’’ portion of Table 1. Inter-
views were deidentified by replacing
each participant’s name with a code
number for anonymity and transcribed
verbatim using Microsoft Word and
Excel (v. 2008, Microsoft Corporation,
Redmond, Washington). The deiden-
tification process gave authors the
ability to interpret meaning from a
broad perspective of stakeholders
across the state and gave participants
the confidence to report personal
perspectives without potential identifi-
cation. In addition, although some of
the participants responded using ex-
amples from their individual program
perspective, using the described meth-
odology below to analyze the Montana
backpack program’s collective scope of
Table 1
Items, Measures, and Sources for RE-AIM Dimensions in Montana-Based Weekend Backpack Food Assistance Program
Evaluation During 2011–2012 School Year*
Dimension Items Measure Data Source
Reach Method to identify target
population
How are children identified to participate in the program? Interviews; program Web sites
Inclusion criteria Process for programs to include children to participate? Interviews; program Web sites
Exclusion criteria Why would a child be excluded from program participation? Interviews; program Web sites
Participation rate No. food bags distributed/No. eligible children Interviews; program Web sites
Representativeness Does the program identify chronically hungry children? Interviews; F/R eligibility data
Efficacy/effectiveness Measures for at least one
follow-up
Nutrients provided in food bags compared to HEI-2010† Food bag contents
Intent to treat Are programs adequate in addressing childhood hunger? Interviews
Quality of life Results reported from stakeholders after implementation Interviews
Percentage attrition Do participants drop out of program? If so, why? Interviews
Adoption Description of intervention
location
% schools .50% F/R eligible; location in Montana;
travel distance
F/R eligibility; interviews
Inclusion of setting How are students chosen to participate? Interviews; program Web sites
Exclusion of setting Why would a school be excluded from program participation? Interviews; program Web sites
Description of staff Characteristics of staff who implement the program Interviews
Level of expertise Who implements the program? Interviews
Adoption rate No. schools enrolled/No. schools .50% F/R eligible Enrollment; F/R eligibility;
interviews
Implementation Type, frequency, and
intensity
Frequency of program delivery Interviews
Extent protocol delivered
as intended
Protocol for program distribution Interviews
Measures of cost Cost of program delivery by year by participation schools Interviews
Maintenance
(setting-level only)
Assess outcomes 6
months
Individual long-term effects of the program Interviews
Sustainability of program Opportunities and challenges for program sustainability? Interviews
Measure of cost of
maintenance
What is the cost to maintain a program? Interviews
* RE-AIM indicates Reach, Effectiveness, Adoption, Implementation, and Maintenance; F/R, free or reduced-price school lunch program; and HEI,
Healthy Eating Index.
† HEI-201014 measures compliance with the 2010 Dietary Guidelines for Americans. HEI-2010 uses standards that are expressed as either a
percentage of calories or per 1000 calories, separating diet quality from quantity. The sum of the scores for the food components is the total HEI-2010
score, which ranges from 0 to 100, with a higher score indicative of a more healthful diet.
success and sustainability increased the
ability to interpret meaning from
multiple perspectives across the state.
Authors were sure to include the
diversity of all emerging themes from
individual and statewide perspectives
in the interpretation of data. Respons-
es were not analyzed by position or
relationship to individual backpack
programs to ensure anonymity and
garner a comprehensive perspective at
the stakeholder level (i.e., across vari-
ous positions rather than by various
positions). Before conducting the
study, the lead author discussed pro-
tocol with select stakeholders, and it
was apparent that anonymity was para-
mount to collecting authentic per-
spectives about Montana backpack
programs in this study.
Meaning units (MUs), or distinct
data that constitute a single idea, were
separated into distinct fragments. The
two authors independently coded the
transcripts based on deductive ap-
proach using the predetermined RE-
AIM themes. The authors, both female
doctorates with extensive training and
experience in qualitative research
methods, then independently coded
for appropriate categories within each
RE-AIM dimension.
To establish interrater reliability,
authors met to confirm meaning or
resolve discrepancies as needed. For
example, the authors divided catego-
ries (e.g., separating ‘‘policy’’ from
‘‘proposed innovation’’) or collapsed
categories (e.g., ‘‘program reach,’’
‘‘program expansion,’’ and ‘‘program
growth’’ included in ‘‘expansion of
program’’) where appropriate. When
the codes were divergent in meaning
(,15% of the time), the authors
revisited the MU within the transcript
to collaboratively decide the primary
significance of the content.
The number of interviewees that
reported MUs within a given category
was also calculated to demonstrate data
saturation within a theme and RE-AIM
dimension (see ‘‘Category Name,’’
‘‘Interviewees,’’ and ‘‘Meaning Units’’
columns in Tables 2 through 4). This
provided data on the saturation of
each category across the interviewees.
Sample quotes are also offered in
Tables 2 through 4 to demonstrate data
authenticity. The lead author contact-
ed participants to review their own
transcript and results in order to
member check the study interpreta-
tion. Approval was obtained from
Montana State University Institutional
Review Board for all components of
this study with informed consent ob-
tained from all participants.
RESULTS
The lead author conducted one-on-
one, semistructured interviews with 20
key informants. The interviews were
conducted on the phone, with only the
interviewee and interviewer on the line
and in the room. The interviews lasted
approximately 24 minutes (63.2 min-
utes). Interviewees were from state
agencies (n¼ 2), food bank directors/
staff (n ¼ 5), policy makers (n ¼ 2),
from school food service (n¼ 2),
teachers (n ¼ 2), parents (n ¼ 1),
registered dietitians (n¼ 1), from
nonprofits invested in child well-being
(n¼ 2), from public health research (n
¼ 1), national level program infor-
mants (n¼ 2), principals (n ¼ 2), and
superintendents (n ¼ 3). Five individ-
uals declined to participate; their
professional roles included superin-
tendent (n ¼ 2), policy maker, state
agency director, and principal. Each
individual declined because they did
not see their knowledge about back-
pack programs as informative and then
provided contact information for a
more knowledgeable key informant.
Of participants, 11 (55%) were directly
involved with the design and/or im-
plementation of a local backpack pro-
gram only (program 1 [n ¼ 2],
program 2 [n¼ 3], program 3 [n¼ 2],
program 4 [n¼ 2], and program 5 [n¼
2]), 4 (20%) participants were involved
with backpack programs at the policy
level in the state only, and 5 (25%)
participants were involved with a back-
pack program at the local level and at
the policy level in the state (program 1
[n¼ 1], program 3 [n¼ 1], program 4
[n ¼ 2], and program 5 [n ¼ 1]).
Participants represented 100% of the
backpack programs in Montana. Par-
ticipants had various levels of knowl-
edge about local, state, and larger
policy level issues regarding backpack
programs, food security, and child
nutrition. In total, 387 MUs were
identified across the 20 interviews.
Reach
Reach was estimated from a quanti-
tative perspective. Because of program
anonymity, reach was calculated based
on the number of children participat-
ing in Montana backpack programs (n
¼ 2888) divided by the number of
children eligible for F/R lunch across
participating schools (n ¼ 12,465).
Each Montana backpack program re-
ported the number of children partic-
ipating on a weekly basis. F/R lunch
participation was calculated using
public data provided by the Montana
Office of Public Instruction by school.
This resulted in a participation rate of
23%.
Six categories emerged from the
qualitative interviews (see Table 2)
around the dimension of reach: eligi-
bility criteria (n¼ 17 MUs), expansion
(n¼ 15 MUs), high reach (n¼ 4 MUs),
sharing food (n¼ 4 MUs), stigma (n¼
11 MUs), and supplemental (n¼ 10
MUs). The first emergent category
within reach was the eligibility criteria
related to recruiting children into the
backpack programs. Some programs
required parents to submit paperwork
for their child to join the program:
‘‘The first year we did it, it was an opt-
in program. We sent information to
parents about the program. We told
them that if their children need
supplemental food, call the office, and
sign the bottom of the paper.’’ Other
programs used an opt-out option: ‘‘We
start the year with all of the children
who are getting free and reduced
meals. We put information in the
backpack that if their child doesn’t
need the program to call the main
office and they would be taken off of
the list.’’ There were mixed opinions
about whether the program reached a
representative proportion of children
in need or if the program was still
missing some of those who ‘‘need it
most’’ because of eligibility criteria
such as filing paperwork. The expan-
sion category referred to the key
informants’ suggestions to expand the
reach as well as current barriers to
expansion, such as, ‘‘It’s pretty much a
private program, which can have its
benefits, but it’s not a universally
available program.’’ Four informants
provided feedback that the program
had a high reach because it assisted
several children in schools across
Montana. One barrier to reach was the
stigma associated with participation in
a hunger relief program: ‘‘You do run
into some issues with parents around
pride. They don’t want their children
to be seen in need.’’ Finally, the
supplemental category was operation-
alized as the informants’ perception
that backpack programs were intended
as supplemental to other hunger relief
efforts across the state.
Effectiveness
Food bags scored an average HEI-
2010 score of 58.7 6 15.6 (0 [mini-
mum score] to 100 [maximum score];
range 41.9–75.1).13
Qualitative information about attri-
tion was measured by seeking data
about program dropout rates and
reasons for dropout. Causes for pro-
gram attrition included moving
schools or the parent’s no longer
needing additional food resources. Key
informants provided 100 MUs about
the effectiveness of backpack programs
across five categories; see Table 2. The
categories were comprehensive solu-
tions (n¼ 10 MUs), need to satiate
hunger (n¼ 35 MUs), pack content (n
¼ 11 MUs), positive perceptions (n¼
36 MUs), and unknown effectiveness
(n ¼ 8 MUs). Unknown effectiveness
related to informants’ opinion that
they either only have ‘‘hypothetical
answers’’ or that ‘‘the numbers [they]
get don’t tell the whole story’’ about
the program’s ability to relieve child
hunger over the weekend. On the
other hand, 14 informants provided 36
MUs about positive perceptions of the
program, from ‘‘students learn and
behave better after receiving the
packs’’ to ‘‘the kids love and consume
it all.’’ Informants also provided mixed
opinions about the pack content’s
nutritional value and ease of use. Yet
there was consistent reporting (i.e., 35
MUs across 15 informants) that there is
still a need to satiate hunger on a
larger scale. The consensus was that
backpack programs ‘‘are not filling the
true nutrition gap, but it really is
helping the recipients.’’ Informants
also spoke about needing to combine
efforts with other entities for more
comprehensive solutions.
Adoption
There were 270 schools in Montana
with .50% children eligible for the F/
R lunch program in 2011–2012. This is
the denominator for potential schools
to deliver backpack programs. In total,
five food banks facilitated backpack
program delivery at 70 schools during
2011–2012. Therefore, the adoption
rate is 26%. The Figure provides a map
of Montana indicating where programs
are being implemented and proximity
to food banks. In general, most back-
pack programs were implemented in
the western half of Montana. Partici-
pating schools were an average of 120
Table 2
Qualitative and Quantitative Results for Reach and Effectiveness Dimensions in Montana Backpack Program Evaluation During
2011–2012 School Year*
Dimension Category Name
Interviewees,
No.
Meaning
Units, No. Example Meaning Unit Quantitative
Reach Eligibility criteria 9 17 ‘‘I look through my F/R list and sent the letter home to
families that I knew were homeless or in F/R lunch
program.’’
23.2% of children
participating who
are eligible to
participate†
across 72
participating
schools
Expansion 9 15 ‘‘Efficient and well-designed public relations campaigns
would actually make a huge difference.’’
High reach 4 4 ‘‘We fed 520 very low-income, Title 1 students that year.’’
Sharing food 3 4 ‘‘We hear about some kids sharing food. It’s not the intent
of the program, but we can’t tell the kid not to,
obviously.’’
Stigma 5 11 ‘‘The social stigma of being a backpack kid. The kid who
doesn’t have enough food at home.’’
Supplemental 6 10 ‘‘We send outreach materials for families to apply for
SNAP, WIC, and Summer Meals.’’
Effectiveness Comprehensive
solutions
7 10 ‘‘Backpacks alone don’t meet the need but there are other
resources that hopefully the family participates in.’’
.50% of students
eligible for free or
reduced lunch or
special
consideration
based on need
Need to satiate
hunger
15 35 ‘‘The backpack only puts a Band-Aid on a gaping wound.
It’s not really addressing the full hunger need and the
larger issue.’’
Pack content 7 11 ‘‘They need to be able to prepare and eat the foods by
themselves.’’
Positive perceptions 14 36 ‘‘The kids feel better on Mondays and aren’t afraid to go
home on Friday.’’
Unknown
effectiveness
7 8 ‘‘We have not done any studies on [how far the food
stretches or meets the child’s needs].’’
* F/R indicates free or reduced-price school lunch program; SNAP, Supplemental Nutrition Assistance Program; and WIC, Special Supplemental
Nutrition Assistance Program for Women, Infants, and Children.
† Calculated based on number of children participating in backpack programs weekly divided by number of children who are F/R eligible.
miles (range ¼ 1–568 miles) from the
food bank facilitating the backpack
program.
Qualitative adoption questions
asked about the representativeness of
settings and intervention agents who
are willing to initiate the program.
Setting-level results are seen in Table 3.
Overall, seven categories emerged
from the qualitative interviews that
represent opportunities for and barri-
ers to adoption: community involve-
ment (n¼ 4 MUs), evaluation (n ¼ 4
MUs), expansion of program (n¼ 21
MUs), financial constraints (n ¼ 7
MUs), food needs (n ¼ 10 MUs), role
clarity and acceptance (n ¼ 15 MUs),
and school eligibility (n ¼ 17 MUs).
School eligibility occurred when fund-
ing and resources (e.g., implementing
agency, food, staff) were identified and
based on need (.50% students eligi-
ble for F/R lunch or special consider-
ation). If funding and resources were
not available or no need was identified,
a school was excluded from participat-
ing. Staff who implemented the pro-
grams were invested in working to end
childhood hunger. For role clarity and
acceptance, interviewees noted that
one salaried individual at the food
bank and several volunteers typically
operated the program and at least one
school staff contact was necessary for
helping to facilitate logistics and gar-
nering school support. The level of
expertise required for the program was
based on navigating appropriate oper-
ating procedures for each site, which
include garnering community involve-
ment, overcoming financial con-
straints, expanding the program
sustainably, appropriately satisfying
food needs, and utilizing evaluation
tools.
Implementation
Backpack programs (n¼ 5) deliver
bags to students (N ¼ 2889) in each
school (n¼ 70) every Thursday or
Friday during the school year (Sep-
tember/August–May/June). Bags are
distributed confidentially and into
children’s backpacks or lockers. The
individual who delivers the bag is aware
of this protocol and ensures appropri-
ate bag delivery. However, if a student
is not present on a particular Thursday
or Friday, a bag is not distributed to
that particular student. To measure
implementation costs, we collected
data and calculated the average cost
per bag. Across the 70 schools, bags
averaged $3.87 (SD 6 .94). Schools
whose bags were less costly found ways
to purchase food items in bulk and
increase food donations.
Qualitative questions about imple-
mentation inquired about protocol—
specifically the consistency of delivery,
time, and cost of backpack programs.
Seven categories emerged: adaptations
(n¼ 8 MUs), bag contents (n¼ 7 MUs),
challenges (n¼ 7 MUs), costs (n¼ 7
MUs), delivery systems (n¼ 9 MUs),
fidelity (n¼ 7 MUs), and privacy (n¼ 5
MUs). Each key informant described a
similar delivery system with protocol
variations to fit the needs of each site,
including where the bags are placed
(e.g., in the child’s backpack or locker),
strategies for maintaining confidential-
ity, the day the foods are delivered (e.g.,
Thursday or Friday), who identifies
children to participate, and what type of
volunteers pack and distribute bags.
The delivery systems are set up individ-
ually because ‘‘the program needs to be
essentially seamless for the school.’’
Interviewees indicated that several ad-
aptations were made to similar back-
pack programs from the original
protocol of the national BackPack
Program. Adaptations included trying
new and different bag contents (e.g.,
the addition of fresh produce), distrib-
uting bags during the summertime,
sourcing foods at wholesale cost, and
making special considerations for
schools with ,50% of students eligible
for F/R lunch. Privacy to maintain the
child’s confidentially was key to pro-
gram implementation. Challenges to
implementation were based on logis-
tics and included a flexible protocol,
providing a variety of foods each
week, transportation of foods from
the food bank to the school, consis-
tent funding and resources (e.g.,
volunteers and time), finding storage
space for bags, and increasing school
enrollment. The bag contents varied
within each protocol. Key informants
reported that including nutritious
foods was important, but sometimes
the need for kid-friendly, shelf-stable,
and easy-to-prepare foods limited the
health of foods. ‘‘We make nutritious
Figure
Map of Montana Backpack Programs and Food Bank Locations
Darkened color denotes at least one backpack program within county. Asterisk denotes
food bank location.
food choices each week . . . we get low
sodium, whole grain, and low fat
whenever we can. But there aren’t a
whole lot of choices in the single
servings.’’ Agencies received mone-
tary and resource support to supple-
ment the costs of implementing
backpack programs, including from
community organizations, food
drives, and the state food bank
network. This support to cover costs
helped some agencies fulfill food
needs for bags 100% of the time.
Several strategies have been em-
ployed to encourage fidelity to back-
pack program protocol, including
documenting program protocol (i.e.,
how to find funding and resources,
how to purchase foods) as well as
reporting why some sites succeed
more than others.
Maintenance
At the time of the study (summer
2012), backpack programs reported
being in operation for an average of
2.8 years (range ¼ 2–4 years).
Questions related to program main-
tenance were designed to assess out-
comes, program sustainability, and cost
of long-term program delivery. Overall,
10 categories emerged from the qual-
itative interviews around program
maintenance: barriers (n ¼ 4 MUs),
collaboration, (n ¼ 4 MUs), communi-
ty involvement (n ¼ 17 MUs), evalua-
tion (n ¼ 2 MUs), funding (n¼ 37
MUs), innovation (n ¼ 6 MUs), long-
term maintenance (n ¼ 8 MUs),
partners (n¼ 7 MUs), policy (n¼ 6
MUs), proposed innovations (n¼ 6
MUs), and sustainable expansion (n¼
7 MUs). Results for the maintenance
dimension are reported in Table 4.
The single barrier reported to program
maintenance was the lack of funding
for resources (e.g., staff and time).
Ideas to overcome this barrier included
partners and community involvement.
Interviewees echoed that a mix of
public and private support for the
program would provide for program
maintenance, but that community
support (especially through funding)
was especially important. Similarly, key
informants recognized the need for
community involvement through
fundraising, volunteer time, food do-
nations or drives, and overall buy-in.
Key informants noted that informal
program evaluations were completed
to help identify and decrease barriers
to maintenance. Key informants con-
tributed differing thoughts about
funding for program maintenance.
Some key informants conveyed that
Table 3
Qualitative Results for Adoption and Implementation Dimensions in Montana Backpack Program Evaluation During 2011–2012
School Year*
Dimension Category Name
Interviewees,
No.
Meaning
Units, No. Example Meaning Unit
Adoption Community
involvement
2 4 ‘‘The local food banks work with a lot of the programs or hospitals.’’
Evaluation 3 4 ‘‘. . .evaluations, challenges, and what works.’’
Expansion of program 10 21 ‘‘There is certainly room to add to the number of schools.’’
Financial constraint 5 7 ‘‘It’s a difficult balance because there are more schools that want the
program than don’t. The money is an issue with that.’’
Food needs 7 10 ‘‘We are very stretched for food resources already in eastern Montana
with the energy boom.’’
Role clarity and
acceptance
10 15 ‘‘Definitely the workload. Is it added onto someone’s position? . . .
Sometimes it takes a while to actually figure out who will do it.’’
School eligibility 12 17 ‘‘We do try to go to the schools that are in the highest need first or the
most eligible for F/R.’’
Implementation Adaptation 13 8 ‘‘The model looks different at every food bank. We encourage that
because every community is different.’’
Bag contents 15 7 ‘‘Backpack meals come in single servings. That is really important so
the food is ready to eat and the kids don’t have to prepare.’’
Challenges 16 7 ‘‘Other challenges are mostly logistical—distributing food, especially
perishable, transportation costs, constant funding and resources.’’
Cost 16 7 ‘‘We average about $3 per bag . . . times 230 bags, that is $690. . . by
40 school weeks and your budget starts going up fast.’’
Delivery systems 20 9 ‘‘That same person lives near the food bank, picks it up, brings it to
school, and distributes to the person’s locker on Fridays.’’
Fidelity 10 7 ‘‘We’ve created materials based off of this about how to start a
backpack program, the best way for the teachers to identify high-risk
kids, and all of the resources, research, and materials from Feeding
America, and 150 other food banks’ experiences around the nation.’’
Privacy 15 5 ‘‘Our school staff does a great job in discreetly getting these bags out
to kids. I applaud them for their efforts.’’
* F/R indicates free or reduced-price school lunch program.
partial state or federal funds would
assist in program maintenance, where-
as others held that funding should be
generated from the community. Fund-
ing was noted as an essential part to
sustainability: ‘‘The most obvious
[challenge to sustainability] is finding
a steady source of income for the
program.’’ Key informants noted that
policy support for community-based
backpack programs was challenging
because of the cumbersome legislative
process, lack of awareness about hun-
ger in the public, and ideologies about
supplemental programs. For long-term
maintenance, interviewees recom-
mended building self-sustaining pro-
grams. Several innovations have
occurred to build self-sustaining pro-
grams: food pantries in schools, school
gardens, and expanding the program
to family members. Key informants
also proposed innovations that includ-
ed increased feedback from parents,
expanding fundraising events, adding
new foods, and starting a public
relations campaign to build support.
Lastly, sustainable expansion was re-
garded as important to program
maintenance: ‘‘We have to ensure all of
our programs can survive in the future
and expand.’’
CONCLUSION
This is the first investigation, to our
knowledge, that has used a mixed-
methods approach to evaluate an
ongoing community-based backpack
program through the RE-AIM frame-
work.23 Our preliminary results indi-
cate that backpack programs aimed at
providing weekend food assistance (1)
are successful at reaching a proportion
of representative children (reach), (2)
are effective at providing food bags of
modest dietary quality based on HEI-
2010 (effectiveness), (3) have large
community adoption and support, with
a pragmatic level of expertise required
to deliver the program (adoption), (4)
have protocol fidelity that is relatively
easy to follow (implementation), and
(5) have had adaptations (e.g. food
donations), that increase the potential
program sustainability across Montana
(maintenance).
The overall intent of the study was to
gather information that aligned with
each RE-AIM dimension in order to
enhance the inferences drawn related
to the backpack program. Identifying
factors that may enhance each dimen-
sions of RE-AIM may generate larger-
scale adoption and promotion (e.g.,
statewide backpack programs). Many
systematic reviews of health promotion
programs (using the RE-AIM frame-
work) have found that issues associated
with adoption and maintenance are
drastically underrepresented in the
literature.24,25 That is, the indicators
associated with the adoption and
maintenance dimensions (see Table 1,
‘‘Items’’) are the least reported across
the five dimensions. This study found
resources and funding as important
barriers to adoption and maintenance.
Notably, the full support of adopting
agents directly influences implementa-
tion fidelity and program sustainabili-
ty.26 Our results indicate that program
adopters explore several program ad-
aptations (e.g., fundraising, collabo-
rating with partners) in order to
promote sustainability. Most impor-
tant, to promote long-term sustain-
ability of backpack programs,
communities must explore solutions to
sustainable funding.
More nuanced findings suggest that
those who are involved in individual
and school identification, food pro-
curement, and delivery procedures
support the mission and values of
backpack programs. Key informants
also felt the program was effective in
providing supplemental food relief
and also assisted in decreasing the
Table 4
Qualitative Results for Maintenance Dimension in Montana Backpack Program Evaluation During 2011–2012 School Year
Dimension Category Name
Interviewees,
No.
Meaning
Units, No. Example Meaning Unit
Maintenance Barrier for
maintenance
3 4 ‘‘I think that one of the challenges is that a lot of the local communities don’t
necessarily have the extra resources—especially ones with the most need.’’
Collaboration 4 4 ‘‘A pairing of reimbursable component and community support would make the
program sustainable in the long term.’’
Community
involvement
9 17 ‘‘I really see it as a new aim for our community to see what teachers are dealing
with every day.’’
Evaluation 2 2 ‘‘It gives you fresh ideas for building and obstacles. Also a comparison for how
quickly our program is growing.’’
Funding 14 37 ‘‘We want to generate enough funds to expand the program to communities that
need it.’’
Innovation 3 6 ‘‘We have schools planting school gardens. ‘‘
Long-term
maintenance
5 8 ‘‘We also need permanent solutions.’’
Policy 7 7 ‘‘The legislature really needs to support efforts to decrease childhood hunger.’’
Proposed
innovation
4 6 ‘‘If we could add nutrition components and education that would be a good
thing.’’
Sustainable
expansion
3 7 ‘‘I am not willing to expand the program unless it can be sustainable.’’
negative effects of hunger (e.g., emo-
tional, behavioral). Some salient con-
cerns were around continued support
(i.e., financially) from entities inside
and outside of the community. Key
informants were consistent with feed-
back about the ease of delivery and the
importance of discretion to avoid
stigma. Concerns were also raised
about the nutritional value of the
backpack contents, and the HEI-2010
analysis validated these concerns. The
average HEI-2010 score of 58.7 is
within the range of the average dietary
quality score for the American child
(54.7 and 59.6).27 Finding foods that
were high in nutrient quality, afford-
able, shelf stable, and easy to prepare
provided a challenge to increased
nutrient quality of bag contents.
A key strength of this study was the
use of the RE-AIM framework as a
guiding metric for the evaluation of
emergency weekend backpack food
assistance programs. RE-AIM has pre-
viously been used to evaluate numer-
ous health promotion programs, a
peer leadership lifestyle intervention
for adolescent girls,26 and weight loss
interventions,25 yet the data collected
often lack information on external
validity factors.24 This study directly
addresses the paucity in the literature
related to the perspectives of adopters
and implementers around sustainabil-
ity and logistics of community-based
program delivery. The evaluation
methodology presented here can also
be used to evaluate other health
promotion programs, especially inter-
ventions that do not have direct mea-
sures of effectiveness. The authors
were able to apply their extensive RE-
AIM expertise in this study and re-
flected that the development of a RE-
AIM–based interview guide may be
helpful in other evaluation studies.
The deductive approach (i.e., using
each dimension of RE-AIM as a theme)
allowed the authors to capture barriers
and facilitators related to the success of
the backpack program.
Some limitations of this study exist
around the inability to gather primary
data from program participants be-
cause of anonymity. Divergent per-
spectives from different participants’
positions were also not able to be
distinguished. From a RE-AIM per-
spective, we were unable to identify
individually based effectiveness and
behavior maintenance. However, this
limitation is moderated because the
current study evaluated backpack pro-
grams from a systematic and not an
individual standpoint. Individual-level
data (i.e., reach, effectiveness, mainte-
nance) were collected and aggregated,
when possible, to provide support for
the system-level variables. Using a
mixed-methods approach through an
esteemed evaluation framework pro-
vides strength to this analysis. Specifi-
cally, Kessler et al.28 reemphasize the
need to include qualitative measures to
capture rich data regarding external
validity factors. This study was designed
with the specific intention of collecting
data from key informants to assist in
the translation of our findings; 20
interviewees contributed to the devel-
opment of categories via saturated data
points. Further, the use of RE-AIM
assists in the ability to make national
recommendations for emergency
weekend backpack food assistance
programs.
These mixed-methods results inform
practice by acknowledging that the
backpack programs in Montana are a
valued part of the food assistance
system for children and that there is
room for improvement in the foods
offered and expansion of the program
to those in need. Facilitating venues to
share information about best practices
in food sourcing and distribution
between backpack programs will in-
crease knowledge about how to im-
prove the program statewide. The data
from key stakeholders represent an
underlying notion that encouraging
communities to take ownership (and
to acknowledge that childhood hunger
exists in their community) will help to
promote growth and sustainability of
the backpack program. The backpack
programs are just one way for com-
munities to provide families with sup-
port that includes alternative
supplemental food resources. Other
community-supported social services
also exist to help families find solutions
to food and resource shortages and
ultimately decrease demand on the
backpack program.
Findings from this study support that
backpack programs are widespread and
growing in Montana and are effective
in relieving weekend hunger. Further-
more, program findings were shared
with stakeholders to validate responses
and also serve as a platform for
coordination and discussion in com-
munities. This program is also run
nationally, and it is important to con-
sider its larger-scale implications for the
8.3 million children who struggle with
hunger nationwide.1 Further research
and evaluation efforts investigating
how to best support backpack program
adoption and maintenance from state
to state are warranted (e.g., differences
in rural versus urban settings). Ulti-
mately, community-based hunger relief
programs with a broad reach, evi-
denced effect, inviting adoption, and
feasible implementation may perpetu-
ate a system that will simultaneously
alleviate immediate family food needs
and promote long-term solutions to
end hunger alongside public assistance
programs.
SO WHAT? Implications for
Health Promotion Practitioners
and Researchers
What is already known on this topic?
Weekend backpack food assistance
programs are implemented nation-
ally and serve more than 8.3 million
children struggling with hunger.
What does this article add?
Backpack programs are run na-
tionally and it is important to con-
sider their larger-scale implications.
Our preliminary results indicate that
backpack programs aimed at provid-
ing weekend food assistance (1) are
successful at reaching a proportion
of representative children (reach);
(2) are effective at providing food
bags of modest dietary quality based
on HEI-2010 (effectiveness); (3) have
large community adoption and sup-
port, with a pragmatic level of ex-
pertise required to deliver the
program (adoption); (4) have pro-
tocol fidelity that is relatively easy to
follow (implementation); and (5)
have had adaptations (e.g. food
donations) that increase the poten-
tial program sustainability across
Montana (maintenance).
What are the implications for health
promotion practice or research?
This is the first investigation, to our
knowledge, that used a mixed-meth-
ods approach to evaluate an ongoing
community-based backpack program
through the RE-AIM framework.
References
1. US Dept of Agriculture Economic
Research Service. Food security in the US.
Available at: http://www.ers.usda.gov/
topics/food-nutrition-assistance/food-
security-in-the-us/key-statistics-graphics.
Accessed September 20, 2013.
2. Kilpatrick SI, McIntyre L, Potestio ML.
Child hunger and long-term adverse
consequences for health. Arch Pediatr
Adolesc Med. 2010;164:754–762.
3. Melchior M, Caspi A, Howard L, et al.
Mental health context of food insecurity: a
representative cohort of families with
young children. Pediatrics. 2009;124:e564–
e572.
4. Kleinman RE, Murphy JM, Little M, et al.
Hunger in children in the United States:
potential behavioral and emotional
correlates. Pediatrics. 1998;101:E3.
5. US Dept of Agriculture, Food and
Nutrition Service. Healthy Hunger-Free
Kids Act of 2010. Available at: http://www.
fns.usda.gov/cnd/governance/
legislation/cnr_2010.htm. Accessed
September 20, 2013.
6. Guo B. Beyond the public safety net: the
role of nonprofits in addressing material
hardship of low-income households.
Nonprofit Volunt Sect Q. 2010;39:784–801.
7. Cotunga N, Forbes S. A BackPack Program
provides help for weekend child hunger. J
Hunger Environ Nutr. 2008;2:39–35.
8. Rogers Y, Milewska M. Food assistance
through the school system: evaluation of
the Food for Kids Backpack Program.
Journal of Children and Poverty. 2007;13:75–
95.
9. Feeding America. BackPack Program.
Available at: http://feedingamerica.org/
how-we-fight-hunger/programs-and-
services/child-hunger/backpack-program.
aspx. Accessed September 20, 2013.
10. Montana Food Bank Network. BackPack
Program. Available at: http://mfbn.org/
learn/programs/backpack-program.
Accessed September 20, 2013.
11. Glasgow RE, Vogt TM, Boles SM.
Evaluating the public health impact of
health promotion interventions: the RE-
AIM framework. Am J Public Health. 1999;
189:1322–1327.
12. Glasgow RE, Klesges LM, Dzewaltowski
DA, et al. The future of health behavior
change research: what is needed to
improve translation of research into
health promotion practice? Ann Behav
Med. 2004;27:3–13.
13. Guenther PM, Casavale KO, Reedy J, et al.
Update of the Healthy Eating Index—
2010. J Acad Nutr Diet. 2013;113:569–580.
14. Harnack L, Hearst M, Harrison M. Report
to Feeding America: evaluation of the
nutrition quality of BackPack Program
menus; 2012. Available at:
fromhungertohealth.files.wordpress.com/
2012/05/evaluation-of-the-nutritional-
quality-of-backpack-program-menus-final-
report.pdf. Accessed September 20, 2013.
15. Byker C, Smith T. Food assistance
programs for children afford mixed
dietary quality based on HEI-2010. Nutr
Res. 2015;35:35–40.
16. Montana Office of Public Instruction.
Free/reduced eligibility data. Available at:
https://apps.opi.mt.gov/
opireportingcenter/frmPublicReports.
aspx?ProcName=procPublicReporting
FreeReducedSelect&ScreenTitle=Free/
Reduced%20Eligibility%
20Data&SelectStateFy=1. Accessed March
28, 2015.
17. Montana Food Bank Network. Homepage.
Available at: http://mfbn.org. Accessed
September 20, 2013.
18. Gallatin Valley Food Bank. Homepage.
Available at: http://www.
gallatinvalleyfoodbank.org. Accessed
September 20, 2013.
19. Helena Food Share. Homepage. Available
at: http://www.helenafoodshare.org/
janda/index.php. Accessed September 20,
2013.
20. Livingston Food Pantry. Homepage.
Available at: http://livingstonfoodpantry.
org. Accessed September 20, 2013.
21. Flat Head Food Bank. Homepage.
Available at: http://www.
flatheadfoodbank.com. Accessed
September 20, 2013.
22. Patton M. Qualitative Evaluation and
Research Methods. Thousand Oaks, Calif:
Sage; 1990:169–183.
23. RE-AIM. Homepage. Available at: http://
www.re-aim.org. Accessed January 22,
2014.
24. Gaglio B, Shoup J, Glasgow RE. The RE-
AIM framework: a systematic review of use
over time. Am J Public Health. 2013;103:
e38–e46.
25. Akers J, Estabrooks PA, Davy BM.
Translational research: bridging the gap
between long-term weight loss
maintenance research and practice. J Am
Diet Assoc. 2010;110:1511–1522.
26. Jenkinson KA, Naughton G, Benson AC.
The GLAMA (Girls! Lead! Achieve!
Mentor! Activate!) physical activity and
peer leadership intervention pilot project:
a process evaluation using the RE-AIM
framework. BMC Public Health. 2012;12:55.
27. Fungwe T, Guenther P, Juan W, et al. The
quality of children’s diets in 2003–04 as
measured by the Healthy Eating Index
2005. Alexandria, Va: Center for Nutrition
Policy and Promotion, US Dept of
Agriculture; 2009. Available at: www.cnpp.
usda.gov/Publications/NutritionInsights/
Insight43.pdf. Accessed September 20,
2013.
28. Kessler RS, Purcell EP, Glasgow RE, et al.
What does it mean to ‘‘employ’’ the RE-
AIM model? Eval Health Prof. 2013;36:44–
66.