A Policy Research Commentary
July 2016
In 2010, the U.S. Department of Housing and Urban Development launched its ambitious Family Options Study with the promise of developing a rigorous, evidence-based national strategy for ending family homelessness.
Following hundreds of families in 12 communities over the course of three years and measuring progress on dozens of outcomes, it has been called “the largest experimental study ever conducted to test different interventions designed to address family homelessness.”1
Many hoped a study this large would shed light on—and possibly settle—a crucial policy debate:
Should anti-homelessness efforts focus primarily on immediately placing families in homes and subsidizing their rent, the approach known as “rapid rehousing”?
Or should those efforts also address the factors that cause many to become homeless in the first place— such as a lack of education or job skills, mental health issues, and substance abuse—through service-rich “transitional housing”?
Are the results blurred?
The study did not prohibit families from accessing any service available locally. As such, many families did not use the service intervention to which they had been randomly assigned. In addition, many of the families in the “control group”—those not offered a specific intervention—used the very service whose effectiveness was being tested, thus blurring the lines between groups and making the results less reliable.
Would rapid rehousing really save money?
At first glance, rapid rehousing programs were the least expensive of the three interventions being compared. The savings were largely erased, however, once all services families used during the interim period were factored in.
What constitutes a success?
For many families, the short-term change in their housing situation did not lead to long-term improvements in stability or well-being. Despite a shorter initial stay in shelter, rapidly rehoused families were not less likely to return to homelessness, nor were most able to address deeper issues like mental health or substance abuse.
But when an interim report, entitled Family Options Study—Short-Term Impacts of Housing and Services Interventions for Homeless Families, was published in July 2015, definitive answers were nowhere to be found.
The study’s complex methodology made the preliminary results difficult to interpret. For one thing, the authors compared the various interventions based on what services families were offered, not necessarily what they accepted or ultimately received.
Beyond that, the results of the direct comparisons between programs were frequently mixed, insignificant, or contradictory, raising doubt about the study’s ability to justify definitive policy prescriptions.
In spite of these issues, some observers—including high-ranking HUD officials—sought to portray the still-unfinished study as providing a clear road map for policymakers. “The short-term findings are compelling enough to allow us to state that, based on the evidence, we know a pathway to ending family homelessness,” Katherine O’Regan, HUD’s assistant secretary for policy development and research, said when the interim report was published.2 In a close reading of the study, however, that pathway is hard to discern.
The study highlights one approach—open-ended rent subsidies—as producing the most positive outcomes. Presumptive in this approach, however, is the oversimplified notion that one size can and should fit all. While the study is serious in its intent, it does not grapple with the core challenges faced by service providers and government agencies across the country, namely how to balance the provision of housing and services to foster long-term family stability.
Nevertheless, supporters of rapid rehousing have seized on an interim finding that their preferred approach was less expensive than the alternatives on a per-month basis. But the study found that rapid rehousing had no real impact on the chance of families slipping back into homelessness—and that any savings largely disappeared as temporary rent subsidies run out and families once again seek shelter and other services.
Meanwhile, the interim findings failed to capture the benefits of transitional housing, in part because 20 months is not enough time for long-term interventions to work properly, and because there was wide variation in the characteristics of the programs being labeled as transitional housing for the purpose of the study.
At the same time, certain issues were simply beyond the study’s scope—such as whether the availability of rent subsidies incentivizes families to enter the shelter system when they otherwise would not, or whether certain programs are available or feasible in all communities.
The Family Options Study
The purpose of the Family Options Study is “to gather evidence about which types of housing and services interventions work best for homeless families.”
The study examines three approaches:
- Permanent housing subsidy—an open-ended voucher used to pay rent, with assistance in finding housing but no other support services;
- Rapid rehousing—temporary rental assistance for up to 18 months, with limited, housing-focused services; and
- Transitional housing—up to 24 months in agency-controlled apartments, with intensive support services.
From 2010 to 2012, study participants were recruited at homeless shelters in the 12 communities shown in Figure 1: Alameda County, California; Atlanta, Georgia; Baltimore, Maryland; Boston, Massachusetts; Bridgeport/New Haven, Connecticut; Denver, Colorado; Honolulu, Hawaii; Kansas City, Missouri; Louisville, Kentucky; Minneapolis, Minnesota; Phoenix, Arizona; and Salt Lake City, Utah.
Researchers enrolled 2,282 families, all of which included at least one child of 15 or younger and had resided in shelter for at least seven days.
After a baseline survey, families were randomly offered one of the three interventions—permanent housing subsidy, rapid rehousing, or transitional housing—or were assigned to that intervention’s control group and would receive “usual care” (referring to the mix of services generally available to homeless families in a given community).
It should be emphasized that these offers were not mandatory, nor were they guaranteed. Families had the option of not participating in the program selected by researchers, and the program sponsors were not obliged to accept the families directed to them by researchers.
Researchers then set out to track each family for 36 months, periodically measuring their status on multiple benchmarks in five outcome categories: housing stability, family preservation, adult well-being, child well-being, and self-sufficiency. The cost of services provided to each family was also tallied.
The July 2015 publication was an interim report based on detailed surveys conducted after an average of 20 months, in which 1,857 families, or 81.4% of the original group, participated.
Why the Interim Findings Are Inconclusive
After 20 months, families assigned to permanent housing subsidy “appear[ed] to be doing better” than those assigned to the other interventions or usual care. 4
Compared with those assigned to [usual care], the families randomly assigned to [permanent housing subsidy] on average have had fewer negative experiences (homelessness, child separations, and intimate partner violence). [Permanent housing subsidy] families are also somewhat more likely to live in their own place. Moreover, children in [permanent housing subsidy] families move among schools less, and families experience greater food security and less economic stress. On the negative side, heads of these families exert less work effort.5
Rapid rehousing appeared less expensive than other interventions, but the report explained that temporary rent subsidies “do not appear sufficient to improve housing stability during the period studied and hence have little effect on outcomes presumed to emanate from achieving housing stability.6 In other words, there is no evidence that rapid rehousing helps families remain housed.
As for transitional housing, which was costlier than other interventions, the authors wrote:
The study provides less support for the theoretical model underlying [transitional housing]. … [Transitional housing] led to modest reductions in homelessness when compared with [usual care], but it did not produce effects in other aspects of family well-being.7
Because this was an interim report, there was no discussion of homeless families’ long-term stability. Unlike HUD, ICPH does not believe that the preliminary findings indicate a “pathway to ending family homelessness”; any conclusions drawn thus far are premature and potentially misleading.
Beyond Housing Needs
On one subject there is little doubt: the Family Options Study reveals a grim picture of the reality faced by homeless families.
Even after 20 months in the system, half of the families not offered a specific intervention had returned to being homeless or doubling up in the six months preceding the follow-up survey, or had lived in shelter in the previous 12 months (Figure 2).8
In the previous six months, 12% of parents had experienced intimate-partner violence. Thirty-seven percent were separated from a partner and 15% from at least one child.9
One-quarter of parents had experienced symptoms of post-traumatic stress disorder in the previous 30 days, and 15% were dependent on alcohol or abusing drugs at the time of the survey.10
Thirty-nine percent of adults had not worked since the study began, and more than one-third of families were food insecure at the time of the follow-up survey.11
Children had attended an average of 2 schools during the 20-month period, and ten percent had been held back a grade. A fifth of students aged 13 to 17 had been suspended or expelled.12
Given the magnitude of the hardships documented by the Family Options Study thus far, there is a need for a clear direction about how to effectively address family homelessness beyond housing needs. Furthermore, prioritizing HUD funding for a specific intervention, and thus limiting the options municipal and county leaders have at their disposal, is shortsighted.
Measuring a Moving Target
A major complicating factor in interpreting the results of the Family Options Study is the manner in which families were assigned to different groups for comparison purposes.
Each participating family was randomly offered one of the three interventions—permanent housing subsidy, rapid rehousing, or transitional housing—or assigned to usual care as a control group, one for each intervention.
But families were not required to accept the intervention that was offered to them, nor were they guaranteed admission by the local agency providing that intervention. Not all interventions were necessarily available in every community, and families were free to change programs as months went by. The result is that many families experienced more than one type of intervention during the study period.
This study design was meant to reflect the day-to-day reality of how homeless services are delivered, and how they are consumed. Homeless families are, after all, free to make their own decisions, which may not necessarily line up with what authorities want.
As the authors wrote: “… [F]amilies who are homeless do not always pursue the programs offered to them, which suggests that some programs deliver assistance that some families perceive as less valuable to them than other assistance available in their communities.”13
The result of this design is illustrated in Figure 3. In the head-to-head comparison of families receiving permanent subsidies versus those with “usual care”, only 84% of families in the experimental group actually received the subsidy, and 12% of the control group managed to receive a subsidy even without priority access.
For the rapid rehousing and transitional housing comparisons, the lines are even more blurred. Less than two-thirds of the families offered access to these interventions actually used them, while between 20–30% of the families in the control group also participated in them.
In short, this study’s methodology tends to muddy the interpretation of the results. Given that the study permanently categorizes families according to the intervention originally offered to them (regardless of what they actually used) for better or worse, outcomes attributed to one intervention actually reflect results partially or completely achieved by other interventions.
For example, take the head-to-head comparison of rapid rehousing and transitional housing: Of the “rapid rehousing” group, only 51% had participated in rapid rehousing while 24% had used transitional housing (including some families who used both). Yet all of that group’s outcomes, both positive and negative, were credited to rapid rehousing. And vice-versa for the “transitional housing” group, just 55% of which actually experienced transitional housing, and 13% of whom participated in rapid rehousing.14
Given this intermixing, drawing definitive conclusions about the effectiveness of any one intervention is difficult at best.
Unscalable Results on Permanent Housing Subsidies
Unsurprisingly, the intervention with the least amount of intermixing, the permanent housing subsidy, yielded the most statistically-significant results. Priority access to this program, which was typically a federal Section 8 Housing Choice Voucher, reduced subsequent episodes of homelessness by over half compared to usual care. It also led to improved housing stability, family preservation, adult and child well-being, and self-sufficiency.15
However, permanent subsidies were also associated with a lower percentage of families who worked for money in the week and month prior to the survey and who were employed during the 20-month period. At the same time, more of these families were receiving cash assistance and food stamps to supplement their income.16
While permanent subsidies cost, on average, only two percent more than usual care over the course of 20 months,17 the extent to which families continue using vouchers at 36 months and beyond may make them more expensive in the long term.
The authors cite a study indicating that non-elderly, non-disabled families with children who enter such voucher programs stay in them an average of 3.8 years. Families with greater needs might use vouchers for longer periods—or, their length of stay could be shorter if additional challenges make it difficult for them to maintain program requirements.18
Among study participants who participated in permanent housing subsidy, the average length of stay at the time of the follow-up survey was 16 months, and three-quarters were still in the program.
Although the effects of permanent housing subsidy in preventing the recurrence of homelessness and improving family well-being are encouraging, it has been, and will likely continue to be, difficult to win legislative support for expanding the availability of such vouchers.
A 2004 study found close to a million families were on the waiting list for housing vouchers in the nation’s largest cities.19 Forty percent of public housing agencies have closed their lists.20 Fewer than 12,000 vouchers have been added annually since 2002, and the 67,000 Housing Choice Vouchers eliminated by sequestration in 2013 are not likely to be restored.21
In order for Section 8 to be a viable solution to end homelessness, the number of vouchers available in each state would need to expand by a significant margin. As shown in the map in Figure 4, in over half of all states the number of homeless households is equal to 15% or more of the total Section 8 caseload in that state; all but six states would require at least a ten percent increase. In reality, rather than growing Section 8 programs significantly shrunk between 2012 and 2014. Almost half of all states lost over 1,000 vouchers at the same time that the Family Options Study was wrapping up its interim results. Funding levels have increased since the end of sequestration, but only far enough to continue renewing current vouchers.22
In an effort to mitigate the impact of demand far outpacing supply, HUD issued guidance in 2013 as to how local public housing authorities can prioritize serving families without homes.23 In the proposed budget for fiscal year 2016, the Obama administration requested that 30,000 of the vouchers to be restored be set aside for homeless households. Nevertheless, past and present congressional action indicates that—despite indications of permanent housing subsidies’ positive effects—the most viable solution to address family homelessness is to strengthen the existing service system.
Dubious Results on Rapid Rehousing
Long before the Family Options Study was conducted, federal funding streams had already undergone a slow but significant shift, with application guidelines and spending restrictions favoring programs that offered “rapid rehousing” over existing models that combined temporary housing with long-term services. More recently, this shift has resulted in decreased funding for service providers across the country, leading many homeless shelters to close or reduce the number of families they are able to assist.24
Supporters contend that the central cause of homelessness is a lack of affordable housing, and that quickly placing families in apartments—with the help of time- limited rental subsidies—will not only resolve their housing crisis, but provide the stability necessary to address other issues in their lives.
The Family Options Study hypothesized that rapid re- housing, when compared to usual care, “will reduce shelter use and may improve housing stability, employment and earnings, family preservation, adult well-being, and child well-being. It will reduce the length of the shelter stay at the time of study entry and may be less costly.”25
As of the 20-month surveys, however, most of these expectations went unmet. In comparison to the usual care cohort, the rapid rehousing group saw no statistically significant benefit in all key measures of housing stability, family preservation, and adult well-being, as well as most measures of child well-being and economic self-sufficiency.
In the six months preceding the follow-up survey, 48% of the rapid rehousing families had spent time in shelter, lived in substandard housing not meant for human habitation, or doubled up with friends or family for an average of 51 days. Excluding those who had lived doubled up, almost one-quarter of families had been homeless for an average of 23 days in the previous six months.26
“The reason for the lack of effects in this most strongly hypothesized area of potential impact is unclear,” the authors wrote.27
Families offered rapid rehousing moved more quickly out of emergency shelter than those assigned to usual care. But the total amount of time spent in shelter during the course of the 20 months was not significantly affected.28
Compared to usual care, the rapid rehousing group saw a 13% increase in total family income, including cash benefits. But there was no effect on earned income or the number of people working for pay in the week before the survey.29
Results of a separate, smaller study based on interviews with homeless families suggest that families participating in rapid rehousing feel pressure to quickly find work to be eligible for subsidies, but find that the jobs they secure are often inadequate to cover rent after the subsidy ends.30
Other positive effects of rapid rehousing identified by the Family Options Study were a reduction in absences from school and childcare31 and increased food security.32
The study further raises doubt about the cost-effectiveness of rapid rehousing.
The authors found, as expected, that rapid rehousing was significantly less expensive than either transitional housing or emergency shelter on a per-month basis and over the course of a family’s typical stay.
Rapid rehousing cost an average of $878 per month, compared to $2,706 and $4,819 for transitional housing and emergency shelter, respectively.33 The cost of an average stay in rapid rehousing, at $6,578 for seven months, was also far less than the corresponding averages for transitional housing ($32,557, 13 months) or emergency shelter ($16,829, four months).34
Given these figures—and the fact that rapid rehousing reduced the amount of time families spent in emergency shelter—it might have been expected to cost dramatically less than the alternatives, including transitional housing, over the full 20 months.
When all program utilization is factored in, the cost difference between intervention groups narrows sharply, as shown in Figure 5. Although a rapid-rehousing intervention itself was relatively inexpensive, this cost savings was more or less erased by the cost of other services these families used during the study period. Overall, families in the rapid rehousing cohort cost just ten percent less, on average, than the usual care group, and families in the transitional housing cohort cost nine percent more.35
Estimating the cost savings of particular programs can also be difficult because they are based on the current demand for services, and may not factor in increased participation once a program is in place. As the availability of housing subsidies increases and more families experiencing housing instability become aware of these programs, there is the possibility that homelessness numbers will increase. Although this represents more families receiving assistance, it is still an unknown factor that can complicate cost-cutting efforts.
While measuring this effect falls outside the scope of the Family Options Study, it must not be ignored by policymakers.
Puzzling Results on Transitional Housing
From the point of view of those who see service-rich transitional housing as the best option for many homeless families, the preliminary results of the Family Options Study are both disappointing and puzzling.
The study indicates that families offered transitional housing used emergency shelter less than those assigned to usual care. However, it found no significant effect on families’ housing stability, employment, earnings, education, or on the well-being of parents.36 Given the transitional housing programs’ emphasis on providing supportive services to address all of these areas, these results are unexpected.
One explanation suggested by the authors is that, in practice, the services provided by the transitional housing programs examined in the study were not vastly different from those provided by emergency shelters.37 In fact, among providers included in the study, the emergency shelters spent more overall than the transitional housing programs ($4,819 per month vs. $2,706) and devoted a greater share of their funding to supportive services (62% vs. 42%). Both of those comparisons are the opposite of what might have been expected—raising a question as to whether these programs accurately represented their respective categories.
An example of the variation seen across transitional-housing providers included in the study is shown in Table 1. Two providers, located in the same city, can have very different program requirements and expectations. Most required families to pay a fee based on either their income or household size, and about half required families to save money while in the program. Many transitional housing providers included in the study design had programs lasting less than one year, while others allowed families to stay up to 24 months. The frequency of case management also varied across providers in the study, with some families only meeting with case-management staff on a monthly basis.
The authors noted that transitional housing entails families moving out when they complete the program, while rapid rehousing, by design, intends for families to stay
in the same apartments after subsidies end—which has implications for comparing housing stability results.38
Also, about one-fifth of the families in the transitional housing group were still enrolled in the program at the time of the follow-up survey, suggesting the interim study period was not long enough to study the intervention’s full impact.39
As the authors wrote, the full results after 36 months “will examine whether the focus of [transitional housing] on addressing psycho-social challenges and enhancing skills leads to benefits over the longer term that were not evident at this point.”40
What To Expect from the Family Options Study Final Report
As a broad cross section of stakeholders await the final report of the Family Options Study in late 2017, here are key issues for policymakers, funders, advocates, service providers, and taxpayers to keep in mind.
What are we measuring?
Untangling the groupings
With so many homeless families in the study utilizing multiple forms of assistance, are the results stark enough to draw definitive conclusions? Given both the muddling effect of the research design and the difference in the political feasibility of certain solutions, what are the limits on the policy implications for this study?
How are we measuring?
The dangers of over-simplification
With regard to both transitional housing and the shelters included as “usual care”,
how is the wide range in quality and services being incorporated into the discussion of results? Given the capability of services to be tailored to a family’s unique needs, are the outcomes for transitional housing being lost among the randomly assigned groups?
Why are we measuring?
Choosing the right outcomes
How should policymakers be defining “success” when it comes to evaluating the effectiveness of the different types of program? When measuring the well-being of a homeless family as well as the cost-effectiveness of an intervention, what are the trade-offs between the short- and long-term effects of a program?
Given the high stakes for homeless families, asking the correct questions—and getting clear and correct answers—is critical for all concerned.
