April 11, 2010

Why Whanau Ora should be evaluated

If the key idea behind Whanau Ora—that social service agencies should work more closely together—was so good, it would have worked the first time. And therein lies the problem. The idea that social service agencies should work more closely together is not an innovation in social services, but a recurring staple of reform.

Of course every generation of policy makers and social service workers has to discover this for themselves, and dress up the resulting discovery in a local context. Hence the name Whanau Ora. Every decade in the twentieth century some government or social service agency in the western world was "discovering" that health, education, criminal justice, and housing problems and solutions were intertwined, and that a solution was for institutions and professionals to talk to each other, and work a little better together. Things were maybe more integrated in the nineteenth century when "charity visitors" were untrained, well-meaning middle class men and women visiting the homes of the poor to see what they could do to help. Someone could usefully write an article pointing out the long recurring history of this new idea, and make it compulsory reading for social policy analysts.

Social services became less integrated with professional training and specialization in the late nineteenth and first half of the twentieth century. The 1920s and 1930s saw lots of calls for integrated services as the first large generation of social work professionals examined their field. It was this impulse that gave New Zealand milk in schools and health camps for sick children. Same idea, different time period. And certainly not unique to New Zealand.

While you can't really question the sincerity of the ideals and ideas behind the policy the fact that we've had this laudable impulse to integrate social service before and it didn't solve all our social problems then means we need to be skeptical of the idea that Whanau Ora will work that much better than the services already in place. That's not to say that Whanua Ora will fail, just that people who are not trying to sell you a political policy should assume any government initiative is no better than what it's replacing until proven otherwise.

Whanau Ora will very likely look great in the first year or so. Positive stories of families helped and crises averted and multi-cultural harmony unfolding will be easy to uncover. Were one to do an evaluation of the old and the shiny new services, the shiny new services would look good. Social service workers, like everyone else, often throw themselves into the new thing enthusiastically. The real question is how do things compare in year 3, when this is no longer the exciting new programme that the government is supporting? I am pretty sure we'll never find out the answer to that question. There is also a near certainty that something bad will happen in at least one family receiving Whanau Ora support, and this will look bad for Whanau Ora in the Sunday newpapers. That's unfortunate too, because no social services eliminate all the nasty things that happen in families. The important question is what minimizes nasty stuff at an acceptable cost.

Calling for integrated services and a holistic focus on families is easy. Getting it to work in practice and show that it's worth the upheaval in what we currently do is more difficult. The flipside of enthusiasm for new projects and new ways of doing things is that some people doing a totally OK job of working with families in need don't like having their routines disrupted. There are lots of possible models of integrating services. The idea has recurred so often in the past hundred years that the evidence for integration is, of course, mixed. Sometimes it works OK, sometimes not so much. The specifics of the policy and practice matter. This iteration of an old idea will be different again in how it works. So it's entirely possible that moving to new service models is bad in the short-term. The point is, we don't know. We won't know unless we try and find out.

What we do know is that how the families respond is critical. One thing that's hard to do in social services is blind evaluations. People know if they are getting the new service. It's not like drugs or surgery where you can trick people into thinking they're getting treated. Since the response of families to social interventions makes a huge difference to the effectiveness of those interventions, perhaps some good will come of this. In that sense pretending to do something new in social services every few years is not a waste of time or a forgetting of knowledge, but a necessary part of making the people being served feel someone cares. But you still should find out that the reorganization didn't make things worse, or cost a whole lot more.

Good ideas and the thrill of the new are not the same as good outcomes. If the good idea of integrated social service delivery was that good we wouldn't have a century long international record of people calling it a new idea. It is a good idea, but it's an old idea whose time has come, and will come again, and again. Print this out and come back to me in twenty years, and we'll probably be onto the next integrated social service initiative.

If the New Zealand government is really committed to redesigning the way it delivers it billions of dollars worth of social services it should implement Whanau Ora in a limited way and see that it works. But that is not often the New Zealand way. There is an assumption that new programmes just work, and that it is inequitable to deny people the benefits of the unproven new thing. It makes for good politics in the short term because it looks like the government is doing something innovative to improve people's lives. But it makes for bad policy in the long-term because we don't know what really worked.

Posted by eroberts at April 11, 2010 11:04 PM