• Featuring articles from past issues of Contributions
How Many Stars Does Your Organization Rate?
Outside groups are beginning to rate nonprofits and donors are watching
by Gayle Gifford
There’s a snowball gathering mass on its way down the hill and your board needs to pay attention to it now.
That snowball is the growing movement by independent intermediaries to develop systems to rate nonprofit performance and social impact. The goal of these intermediaries is to provide accessible, online rating systems to steer philanthropic dollars to the “best performing” nonprofits.
Who will be doing the rating?
The staff of GiveWell as listed on its website are former hedge fund employees and recent college graduates who appear to have little-to-no experience working within nonprofits.
GiveWell is described on the Hewlett website as “the sometimes controversial ... independent, nonprofit evaluator of nonprofit organizations. A self appointed watchdog, it performs in-depth research to help people accomplish as much good as possible with their donations.”
GiveWell boasts that it has already evaluated 500 nonprofits and only found four worthy of its top ranking!
||ALSO BY THIS AUTHOR: Gayle Gifford is also the author of How Are We Doing?, published by Emerson & Church. For more information about this book, please click here.
Great Nonprofits invites donors, volunteers or anyone with firsthand knowledge of a nonprofit to post a review on its website. Out of curiosity, I searched on a number of organizations I give to and not one of them had any reviews at the time of this article.
Philanthropedia has taken a mutual fund approach. They compile a team of “experts” who select a small group of worthy nonprofits to include in a category fund like Global Climate Change or Education, and then apportion any donations that come in through their site (hmm, reminds me of both the old and new United Way).
Perhaps one of the more thoughtful approaches is coming from the more widely known Charity Navigator. Ken Berger, its President and CEO, has heard the criticisms of its four-star ratings which assess only financial indicators, not program performance. He is determined “to transform our evaluation system of charities to include two additional dimensions – accountability (including transparency) and outcomes.”
Charity Navigator has appointed an advisory group to help it design its system. You can read more about who is on this committee in Mr. Berger’s President’s Letter.
Does your organization have any outcomes measurement?
We all know that measuring societal outcomes is one of the hardest tasks that most nonprofits and social benefit organizations face. It’s also one of the most controversial.
Just look at the maelstrom stirred up by the government’s No Child Left Behind Act which tests students to assess public school performance. Critics of the testing process point out many shortcomings including: school cheating, lack of comparable standards across states, and the failure to test the same students over time, a too narrow focus on test scores that excludes other factors that contribute to student success (like social-emotional intelligence.
While I for one am highly dubious of simplistic rating systems, I agree that we have a responsibility to our investors and donors to use their money responsibly. And we have an even bigger obligation to the people whose lives we touch, or the world we live in, to produce something very good with those dollars.
Unless we attempt to measure our performance, how will we ever know that we’re moving toward our desired vision of societal change?
Unfortunately, far too many organizations use the excuse “It’s too hard” to avoid any outcome measurement at all. Under these public rating systems, no one can use that excuse anymore.
Your organization will be especially penalized if it lacks any attempts at all to assess its performance and social impact.
What measures will be used?
In a conversation I had this summer with Ken Berger, he indicated that Charity Navigator’s goal is to include all nonprofits with budgets over $1 million, though they intend to start with the very largest nonprofits – those that consume most of the charitable dollars and have the most capacity for measurement.
One of the instruments Charity Navigator is reviewing as a prototype for its ratings was developed by Hunter Consulting LLC for social and human service organizations. Its name is revealing of its intent: “Protocol for assessing Investment Risk – with regard to the likelihood that an organization is producing Social Value.” You can take a look at that assessment instrument here .
The esteemed Urban Institute has teamed up with Social Solutions, a producer of performance management software for human services, to create the Outcomes and Effective Practices Portal, the first phase of which is promised in this spring.
According to a December 1, 2009 press release, “the portal will feature detailed guides to help agencies identify and use proven and promising practices to serve their clients effectively, and a library of tools to assess whether their lives have improved as intended. Portal content will be developed in conjunction with well-known organizations and consultants. “
The Center for What Works is developing a common framework for outcomes measurement and has already completed this for 14 issue areas such as advocacy, youth mentoring, or affordable housing.
I have many reservations about the efficacy of this approach
Though I am absolutely a cheerleader for every nonprofit developing a vision of the societal change they are trying to create, along with measures and assessment of their progress, I have deep reservations about intermediary-developed rating systems and their attempt to influence donor dollars.
I worry, for example, about the smallest nonprofits like neighborhood associations, small watershed organizations, food pantries or community advocacy groups, which won’t even be on the radar of these rating systems, yet perform invaluable services at the micro-level.
I worry that organizations that contribute very individualized outcomes like human dignity, artistic joy, personal knowledge or leading a fulfilled life will drop off the radar screen entirely.
I worry about a rating approach that focuses solely on individual organizations and ignores the critical networks of organizations that only collectively create the desired change they seek.
I worry about funding for advocacy organizations that take up the most difficult of causes, like ending the death penalty in the US or protecting human rights around the world, and how a national push to rate effectiveness may result in the abandonment of life-saving activities that don’t meet funder expectations for large numbers of “successes.”
I wonder what timeframe will be used, as many of us know that social change takes decades, or that the consequences of actions are not often revealed for years.
I worry about the loss of funding for experimental or emerging organizations.
I worry about the arts, culture and humanities which may be forced in greater numbers to justify their work only by non-artistic measures like economic impact because those measures are the easiest to assess.
I also wonder how these intermediaries and their supporters will develop and confidently proselytize their measurement formulas in mere months while some of the best minds in academia and government and the nonprofit sector have been struggling for decades to isolate agency actions and measure social outcomes.
But regardless of my reservations or yours, change is on the way. So dust off those logic models, start your indicators, and bring on the evaluators.
How-to Library Index