Most LMI solar programs lack strong evaluation metrics, report says

A growing number of state and local governments, utilities, and non-profit organizations are promoting solar power to reduce energy bills for low- and moderate-income (LMI) households.

Now, a new study from Berkeley Lab looks at how those programs are being evaluated. It finds LMI solar programs are mostly young, operate on tight budgets, and vary considerably in their design and stated goals. As a result, the report said that well-designed program evaluation “is critical to better understand what is working, what could be improved, and how to maximize program impacts under budget constraints.”

The report said that at least 41 active programs in 21 states have made funding commitments totaling well over $1 billion. The programs provide favorable financing to eligible households, give upfront or production-based incentives, or create LMI set-asides in broader distributed solar programs. A growing number are community solar projects that credit the value of the generation to the bills of eligible customers.

The report, An Assessment of Evaluation Practices of Low- and Moderate-Income Solar Programs, said that “only a few of the programs” have undertaken rigorous third-party evaluations, looking at both processes and impacts, and analyzing internal and external data sources. While these are best practices in evaluating utility energy efficiency programs, they are not common in LMI solar programs.

Under reporting?

The report said that around one-third of the programs have made evaluation reports public. Instead, most programs issue progress reports that summarize basic metrics such as program expenditures and installed solar capacity, often to show compliance with laws and regulations. A smaller number tally the number of jobs created, bill reductions, and power generation.

The quality of the evaluations often hinges on whether the programs follow a number of best practices. For example, clear, specific, and measurable program goals help define what and how to evaluate performance. Building consistent data collection and analysis into program operations enables the program to perform a continuous evaluation that feeds back into improved operations. Given the importance of social outcomes in LMI solar programs, evaluations can also focus on non-energy benefits such as improved wealth, health, and safety.

The report said that collecting multi-dimensional data that directly aligns with program goals and can measure progress allows for a more detailed look at correlations and causations. The study expands on this point by exploring statistical methods that can tap publicly-available data as a low-cost way to increase the rigor of evaluations.