Comments
iAUDIT! - Last week, the City of Los Angeles created a website showing homelessness program performance and payment data. I strongly suggest you keep the website open while reading this column so you can follow along with the document references. The website’s creation was ordered by federal judge David Carter, who told the City taxpayers should be able to easily see where their money is going. At the moment, the site is pretty rough and not at all intuitive. You really have to hunt for data—you have to know what you’re looking for and what you’re looking at once you find it. In all fairness, Judge Carter gave the City a very short deadline—two weeks from the date of his order, so we shouldn’t expect a seamless experience. One hopes the site’s utility will improve as the City works on its evolution.
The website has three main sections of interest:
- Alliance Settlement Program Quarterly Reports, where you will find information the Court required the City to submit under the settlement agreement with the L.A. Alliance for Human Rights.
- The Freeway Agreement Roadmap Program, containing reports and financial information required by the settlement agreement on encampment and housing programs.
- Inside Safe Program Reports. As the name implies, these are reports and documents associated with Mayor Bass’ Inside Safe initiative.
Each of the main sections are divided into numerous subsections where you can find program reports, invoices, and provider best practices. There is too much data for me to cover in one column, and if we assume the site’s user interface improves over time, there is no point in going into great detail for something that may change over the next few weeks. However, we can glean some important –and shocking--information from the present data. I describe a few examples in the following paragraphs.
Under the Alliance Settlement Agreement section, the LAHSA metrics data should be evaluated in relation to the subsection on LAHSA cash requests since both focus on the Highland Gardens Interim Housing Center. Notice how few clients transitioned to permanent housing from July through December 2023; only three in six months. For just two of those six months, the City paid LAHSA $1,373,728 to operate the shelter. If you project that cost over the six months in the performance reports, it’s $4,121,184 to house three people, or $1.37 million per person. Within those same six months, 40 people fell back into homelessness.
The percentage of exits to homelessness don’t make sense. In the first quarter, the number of exits were 15 and 65.2%, but 15 isn’t 65.2% of the 120 clients entering the system. In Quarter Two, the number of exits were 25 and 75.8% but 25 isn’t 75.8% of the 132 clients entering the system. The percentages don’t relate to any other number in the report. Of course, losing 40 people to homelessness--13 times the number of clients than were housed--isn’t a great outcome.
Within the Freeway Agreement section, the LAHSA Metrics Reports subsection appears to be the location of metric reports for street outreach and Interim Housing facilities listed in the Scope of Required Services Section. Currently, there is one report in this section covering the first two quarters of fiscal year 2023-24.
There is a lot to unpack in this report from a performance viewpoint, beginning on page 2:
Page 2 shows the calculation used to determine service targets for Street Outreach. Using the 1,127 unduplicated clients served shown on Page 3 as a base:
80% (902) should receive referrals to permanent or interim housing. 20% (225) will be placed into interim housing.
5% (56) will be placed into permanent housing.
One would expect the performance metrics on pages 3 and 4 to match the targets on page 2. However, the metrics are unrelated to the targets. Rather, they show the number of engagements needed before a referral can be made, and the number of clients entered into the HMIS and CES systems, neither of which are effectiveness measures. According to the footnotes, the measure that should show the number of clients referred to higher levels of care has been omitted at the request of the program lead.
In addition, the list of top three services provided are unrelated to housing. They are 1) provided basic hygiene items; 2) provided food and drink; 3) distributing service information or brochures. None of these are performance measure—they barely qualify as workload indicators. Nor do they show how they helped get people housed. Distributing service information is not the same as a housing intake action.
Page 6 shows the outcome goals for each program component. For example, Project Homekey goals are a 95% occupancy rate and 20% placed into permanent housing. The occupancy rate is a workload indicator, not a performance measure, because it’s simply a function of availability and utilization with no qualitative component.
Page 7 shows the performance measures for Project Homekey and the other Roadmap programs. In the first quarter, there were 686 unique Homekey clients. Of those, 20 (3.2%) were connected to Time Limited Subsidy (Rapid Rehousing) housing. Of the 20, 8 (47.3% of 20) have a move-in date. Nine were referred to permanent housing. The percentage shown with the 9 is 30.37%, but 9 is not 30.37% of the 149 connected to housing navigation; the percentage does not correspond to any other number in the table. The remaining columns show average time to connect to housing navigation and exit from Interim Housing, plus the number of other exits, such as death, return to homelessness, or incarceration.
None of the measures on pages 7 and 8 are outcome measures. They are process measures, since they simply count the number of clients at a given stage in the system. One may define housing placement as an outcome, but the columns (vi and vii) measuring housing actions are only move-in dates, not completed housing actions. In fact, the columns showing exits to permanent or temporary housing are zero in both quarters. Stated plainly, none of the actions in the six months shown on the tables on pages 7 and 8 resulted in confirmed housing placements.
The Inside Safe Program page consists of three sections: a Scope of Required Services for LAHSA Inside Safe service providers, quarterly metrics reports from the CAO to the Council, and LAHSA cash requests and invoices submitted to the City.
Looking at the most recent Inside Safe report, the 13th Status Report March 28, 2024:
The expenditure breakdown (page 10 of 24 in the PDF) is good but buried in the middle of the report. The same can be said for the program activity table on page 15—good information but buried. Also, 20% of program entrants fell back into homelessness, with almost 14% exiting with no further tracking. Where did they go?
Currently, per the report, $255,355,490 has been spent on 2,482 participants for a cost of $98,450 per person, but 20% of those 2,482 people left, so the actual cost per participant is higher. It’s important to note the report says data is based on HMIS entries, and we already know LAHSA doesn’t enforce data integrity procedures like telling the City when someone vacates an Inside Safe room.
The numbers in Attachment 3 (page 17 of the PDF) don’t match the numbers on pages 1 of 4 and 2 of 4, (pages 15 and 16 pf the PDF).
Inside Safe Payment Package # 4: These are just financial metrics with no mention of actual services provided. What are the service milestones or metrics used to determine payment? The website should include links to contracts and key performance indicators, so we know what the money is spent on.
There are many other subsections, reports, and charts on the website, but the previous descriptions should give the reader an idea of the data currently available. So far, what we see tends to confirm what we’ve long suspected: the City pays LAHSA and service providers despite little or no proof of performance. Few, if any, performance measures show program outcomes, other than far more people falling back into homelessness than are housed. The Scopes of Required Services may be helpful for knowing what providers should be doing, but there is no data to show how well a provider is meeting those requirements, or even if those requirements have any effect on homelessness.
As I’ve written in previous columns, LA’s homelessness programs are processed-based rather than outcome-focused. As long as providers go through the motions and submit the right paperwork, they get paid. Those payments can be huge: more than $255 million for Inside Safe, more than $11 million per year to manage the Highland Gardens interim housing facility; an average of almost $81,000 per month to run the Safe Parking lot on La Cienega. A casual review of some of the invoices show a staggering number of subcontractors and a myriad of contracts, yet there is no reference to actual performance metrics. What we get for the money spent cannot be answered by the information on the website.
There are really two things we should be asking for on the website. The first is to make it more intuitive and easier to use. Simple pie charts showing where funding comes from (like Measure H or the General Fund), and where the money goes (Inside Safe, housing construction, etc.) would be helpful. Second, and far more important, we need empirical proof the billions of dollars we spend are making a difference. Judging by what we see so far, the answer is an emphatic No.
In my next column, I’ll review some of the information on the City’s website in relation to the April 9 State Auditor’s report on how ineffective $24 billion worth of State homelessness funding has been.
(Tim Campbell is a resident of Westchester who spent a career in the public service and managed a municipal performance audit program. He focuses on outcomes instead of process.)
