Evidenced-based policy seems like the healthy option. Be gone with ideological witchcraft, wayward whims and fancy. This supposedly improved diet has shifted the focus towards facts and arguments about alternative analysis.
Where’s the evidence?
Take three recent reports about entirely different subjects. Three different audiences. Three different research teams. Three sets of recommendations for different policy developers. The reports all deliver what it says on their packaging. Scratch the surface and all three reports reveal a single common theme.
In the space of just 10 days, these three reports highlighted the challenge. Put simply, the rear-view mirror does not show the way ahead. With hindsight, past mistakes might be revealed but foresight requires a different sort of vision. The data (or its analysis) may be deficient and the future is a different country. We are destined to live with ambiguity.
Upon a riverbank serene, a fisher sat where all was green – and looked it.
One day before the UK Chancellor’s budget, Stephanie Flanders made this her opening line – the lack of meaningful data to illuminate local economic priorities. And Stephanie should know. Having toiled for a year chairing the RSA’s Inclusive Growth Commission the former BBC Economics correspondent is also Chief Strategist at the JP Morgan school of economic endeavor. Facing the reality of macro v micro and knowing the systemic reliance on ‘evidence’, her views could be interpreted as a critique of the discipline. How can one argue for doing things differently if the data shows only past indifference?
He saw, when light was growing dim, the fish – or else the fish saw him – and hooked it.
But even when the recent past is closely examined, the pace of change in many arenas will render that analysis as increasingly irrelevant. That was the obvious lesson from the EU report on the state of National Broadband Plans. Moreover, much of the analysis must be qualitative and thus risks being rubbished by anyone with a less than holistic perspective. Bigger pictures inevitably arrive with bigger generalisations.
He took, with high erected comb, the fish – or else the story – home and cooked it.
Our 10-day trilogy was completed by the report on Rural Health and Wellbeing that gained media attention for a few moments last Saturday. ‘One of the main findings is that the general run of statistical ‘evidence’ is woefully lacking in granularity and that very real issues are lost in data averages that are ‘merely average’. But that data deficiency is only part of the story. All these reports are not really about the past. The authors surely want to look forward to a better future – to be forward looking.
Recording angels by his bed, weighed all that he had done, or said, and booked it.
In business as in politics, forward vision is a matter of trust. Innovation is often described as ‘disruptive’ and inherently risky. A shortfall in creativity reflects a lack of trust – personal uncertainties or insufficient collective ‘buy in’. Celebrated innovators often marvel, later, how they managed ‘to get away with it’. They are great storytellers but, even when there is a groundswell of generosity or tolerance, the naysayers will be out in force defending their vested interests. For forward thinkers, determination is at a premium.
Looking Forward
The conventional approach to garnering fresh ideas for policy interventions is to either publish a Call for Input or create a proposed course of action that is open for consultation – a ‘ConDoc’ or a Green Paper. Despite high response rates and great effort invested in preparing those inputs, few have faith in the assumption that these efforts will modify governmental or regulatory actions. And while most respondents are pleased to see their protestations published by the Department or Regulator concerned, there can be grave suspicion about inputs that are not fully disclosed. Some respondents will argue ‘commercial confidentiality’ but doubts linger over whether such claims are rigourously examined. Transparency, it seems, has several shades of obscurity.
Given that consultation is, by law, requisite the Department or Regulator will often try to ease their analytical burden. Typically they can suggest a set of questions so that the responses can be more-readily categorised. Some respondents, of course, choose to ignore the questions and then worry that their views might be ignored. Others object to the questions – concerned that the compilers have not fully understood the issues.
It is inevitable that qualitative analysis must be blended with enumeration of simple metrics – but deeper research is expensive and time-consuming. Respondents with strong vested interests in resisting change will always welcome delay. Balls inevitably get lost in the long grass.
How then can we make a better job of looking forward?
One route (exemplified by the LGA/PHE Rural Health & Wellbeing report) is to give prominence to small innovative projects. Highlighting the micro needs consistent and determined effort to shift those embedded macro assumptions. These projects may only be ‘straws in the wind’ but the story telling is powerful – and, as in books for young children, repetition is essential.
Another route is to invest in better qualitative research. Why depend on respondents’ literary skills when small discussion groups can yield deeper insights? Businesses must constantly battle against simplistic quantitative research. If we had believed only in polling results, then (back in the 1980’s) there was no case for eMail or mobile phones. The workshops currently being run by the National Infrastructure Commission provide a useful clue. The NIC’s existence was born of a cross-party push to remove partisan politics from long-term planning but there’s still a need to bang industry heads together and capture the sparks.
Carefully designed ‘open’ research – nowadays more easily managed – can enable an even wider public to inform the issues. We do not need to depend on the mobile operators’ own data to garner a more accurate sense of system performance and coverage. Ofcom recently consulted on Passive Infrastructure Access – their concern being that the shift towards ‘full fibre’ might be accelerated if the owners of holes and poles facilitated better use of those assets by new investors. No doubt Ofcom has been assured of the current quality and adequacy of the underlying infrastructure – but would those assurances stack up against open-sourced photos contributed by the general public? A new photo competition perhaps?
No doubt, in some administrative corridors, the implications of the three reports will sink in. Certainly some reports arrive with considerable forward lobbying. The words ‘Inclusive Growth’ featured in the budget. But, whatever the industry, the cost of compiling consultation responses is not insignificant and is most easily afforded by incumbents with most at stake.
The notion of policy and regulation becoming ‘evidence based’ was a welcome alternative to the myth-takes of deep-set ideologies. If, however, we are to look forward, then we should adopt new tools and techniques to find the data, probe the data, learn from the mistakes and then encourage fresh thinking.
________
Readers may also be interested in the 2013 report by the UK Public Administration Select Committee ‘Public Engagement in Policy-making.
Groupe Intellex 