Policy influence or evidence-informed policy: what is the difference?

By Catherine Fisher

“We all want a culture of evidence informed policy making, don’t we?” asked Dr Ruth Nyokabi Musila from African Institute for Development Policy (AFIDEP) at the opening of her presentation at the International Conference on Evidence Informed Policy.

It was a commitment to this ideal that had united over 50 researchers from 4 continents, brought togther in Ile Ife, Nigeria, earlier this year. I was attending under the auspices of the IDS Mobilising Knowledge for Development Programme (MK4D) and had been invited to present and chair a session.

Policy influence is not the same as evidence informed policy
 

Throughout the conference I was struck by a blurring between the (admittedly closely related) concepts of research having policy influence and evidence informed policy. The difference seems pretty obvious to me but I sometimes struggle to explain it.  

Let me try this…  

• Effective research communication (which aims to influence policy) is indicated by change in policy/process/discourse based on the research findings you are communicating.
  
  
• Effective evidence informed policy is demonstrated by a culture (systems, processes, attitudes and behaviours) that mean that people in decision making processes regularly engage with research from a wide range of sources when formulating, implementing, reviewing policy.

And to illustrate this difference, here are two examples from the conference:

Image: http://profile.ak.fbcdn.net/

Firstly, Kakaire Ayub Kirunda, shares his learning on how to influence policy. He observed that  “while members of parliament might be an ultimate target, they hardly have time and it is their clerks and assistants who do the lion's share of their research..."

He adds that, in a conversation with Ugandan MP, Honurable Obua Denis Hamson, who also chairs the Science and Technology Committee of Parliament, about how he would want researchers to approach him with evidence, the MP suggests “Probably the easiest way is to first give me a brief summary of your research findings. We can start from there.”


Ah yes, the ubiquitous policy brief. IDS' Impact and Learning Team recently conducted some research around the effectiveness of these as a communication tool, but that is for another blog.

By contrast, an example of supporting evidence informed policy was brilliantly illustrated by Jorge Barreto. He described the creation of an “Evidence Centre” in PiriPiri, a town in a poor region of Brazil. The Centre promoted the use of health evidence locally to improve municipal decision-making process.

Over a beer the night before, Jorge had told me that infant mortality rates in Piripiri were far lower than in other similar towns, his colleague added “20 babies survive a year because of these local policies”.  

Jorge’s presentation concluded that “current efforts to improve local government’s capacity to use research evidence to define problems, find tested interventions, assessing the quality of global and local evidence and translating evidence to key stakeholders are worth continuing. This is our little contribution towards addressing the knowledge to action gap.” Not so little for those children who survive and their families.


I feel it is worth maintaining the distinction between policy influence and evidence informed policy as the activities you undertake to influence policy with research will be different to those you might undertake if you wish to bring about a culture of evidence informed policy.

Such as...Research communication versus knowledge brokering


Two areas of activity which seek to either influence policy and/or support evidence informed policy are research communication (sometimes referred to as research uptake) and knowledge brokering (sometimes referred to as knowledge mobilisation). These distinct activities also often get confused (see my earlier post Buzzing about brokers).

Working closely with IDS Knowledge Services, engaged in knowledge brokering activities, and the IDS Communications Team, focused on supporting IDS research, this is something we decided to explore in more depth at an Impact and Learning team ‘learning lab’, a reflective practice tool we’ve been using to create a space for shared learning.

Here are some notes from the lab, which focused on "desired outcomes": 

"Research Communication and Knowledge Brokering get confused because while they start from different places (one piece of evidence versus many pieces of evidence) they use similar methods and communication tools (e.g. policy briefs). However, they can be untangled again when you look at the outcomes they are trying to achieve:

• Desired outcomes of ‘Research Communicators’ relate to a change in a specific/thematic policy or practice i.e. you know RC activities have succeeded if a specific policy decision is made
• Desired outcome of ‘Knowledge Brokers’ relate to a change in the information-seeking and decision making behaviour of policy/practice actors i.e. you know KB activities have succeeded if decision makers consider a diverse range of evidence to inform their decisions

Importantly, power matters: in Research Communication, the relationship between the researcher (or research institution) and decision maker makes a difference to whether the decision maker gets to hear about a specific piece of evidence (e.g. informal encounters, ‘Beer Buddies’) whereas knowledge brokers, such as the IDS Knowledge Services, can work to equalise that power imbalance for less powerful researchers (or research institutions). For example, the British Library of Development Studies' work around improving access to research published in the global South”

I will explore how ‘politics’ comes to play on these two strands of research uptake activity in my next blog. Meanwhile, you can follow me on Twitter @CatherineF_IDS; I'm currently at the K* Conference in Hamilton, Canada.

Digital information on the move: the rise of the Tablet

By Simon Batchelor

Here in the UK the ‘new’ devices of smartphones, tablet pcs and ipads are very evident. Just take a train, and you will see at least half the people on it are staring at a screen.  While we may think they are working on their emails, some of them are just playing games or watching films – nevertheless digital information on the move is becoming even easier.

So is this change in device use becoming common in the countries where we work?


Our recent study (yet to be officially launched and published), suggests that for the policy environment these devices are changing access to information.  When asking 368 policy actors in six countries (Bangladesh, Ethiopia, Ghana, India, Kenya, Nepal) what devices they have access to we get the responses as shown in the figure below.

Emerging findings from Impact and Learning Team (IDS) research, full report will be available on www.ids.ac.uk

The figure illustrates that 90% of respondents have a computer desktop either at home or the office, 88% have a laptop for use in either the office or home.  However what becomes interesting is their growing use of Tablet computers.  Tablet use among policy makers in the South is at 12%. 

So how does this compare with the UK scene?  While we don’t have the UK figures (and if anyone has please add as a comment), the Pew internet survey for the USA (June 2011) suggests that use of Tablets across the USA has risen to 8% over the last 12 months.  This means average use among policy actors in the South is slightly higher at 12%.  Perhaps interestingly but unsurprisingly, desktop and laptop use among policy actors is considerably higher than the USA general public average which stands at 58% and 52% respectively.  

Indeed in September 2011, India's Economic Times carried a story announcing that the computer allowance for MPs had been raised from Rs 150,000 to Rs 200,000.  The extra Rs50,000 was specifically to obtain a tablet device such as an ipad or Samsung Galaxy powered by Android. “Owning a tablet is mandatory for all MPs, officials said.”  The article states that “over 125 members from the total 245 have already bought the tablets”.

And how about Smartphones?  The graph shows that 40% of respondents had smartphones. Of these 8% had iphones, 12% Blackberries and 31% were ‘other’ smartphones – where smartphone meant they got their email over the phone and could surf the internet.

What does this increased use of mobile technology mean for Knowledge Intermediaries?  

The information ecosystem is changing.  Policy actors do indeed have access to the latest technology, and the proportion of early adopters among the policy actor subset is approximately the same as the averages of the general public in the USA.  While much intermediary work is digital, the debate continues as to whether it is the best pathway for getting research in front of the key people.

Our forthcoming report explores the behaviour of policy actors, but in terms of potential digital access, the data confirms that, increasingly, policy actors have access to this medium, and we should not miss the opportunity to develop "apps" which engages with these early adopters. A real-time example of the potential in this is the IDS Knowledge Services Open API. It allows developers to create apps for Android-driven Tablets and smartphones which could tap into the BRIDGE and Eldis research databases containing over 32, 000 summaries and documents.

What should the post-2015 MDG (on water and sanitation) look like, and how would we measure it?

By Elise Wach

IDS Bulletin 43.2

On World Water Day, I had the opportunity to attend the IDS STEPS Centre launch of the IDS Bulletin on Politics and Pathways in Water and Sanitation. The discussion focused strongly around the MDGs: it was recently announced that the target for water had already been met, but there are a lot of questions about what that really means and how it was determined that we have ‘halved the proportion of the population without sustainable access to safe drinking water’.

The panellists all lambasted the fact that the goal says nothing about equity.  And while the word ‘sustainability’ is included the target, there are serious doubts about how that is actually measured.  Katharina Welle’s research, for example, revealed that neither the neither the method used by the Ministry of Water and Energy (based on infrastructure completed) nor the method used by the JMP (based on household use of water) accurately captures the access issues that people in rural areas are grappling with.

This resonates quite strongly with the work I’m doing with the International Water and Sanitation Centre (IRC) to synthesize the learning streams for the Triple-S Initiative, which is attempting to completely transform the rural water sector.  And I have been asking myself, if we wanted another round of MDGs, and if we kept the same sector-based approach (both are big ‘if’s’ and there are more), what would we want the water sector goal to look like, and how would we measure progress towards it?

I essentially worked backwards through a simplified theory of change, starting with the end goal. Based on the principles of Triple-S, I went ahead and defined the end goal to be:

Everyone has sustainable access to safe, adequate, and reliable water.

Essentially, there are five core components here (the five words in bold).  While they are all intrinsically linked to one another, let’s attempt to look at just one aspect of this: sustainability.

How do you measure sustainability?  You could go back and see if water services are still there in ten years’ time.  That’s useful, and I think it should be done (and so does Water for People) but how do we know now if water services will continue sustainably in ten years time?   

Perhaps we’d want to think about what is needed to ensure sustainability, and try to measure that. According to Triple-S, one prerequisite for ensuring that water services last is to ensure that there is the capacity, financing, and planning for major replacements in the future (i.e. not just maintenance). And there are a variety of other direct and indirect requirements for sustainability: ownership, inclusion, accountability, transparency, government capacity, and coordination to name a few.

So a big challenge would be to agree on the requirements for sustainability.  Assuming we can overcome this (daunting) hurdle, we’d then need to assess whether these are in place.  But measuring these won’t be as straightforward as measuring the number of people who live within a certain radius of a borehole.  And there are other issues as well, such as the issue of Multiple-Use Systems, as Stef Smits discusses.

It will be, in a word, complex.   

But if the issues we’re trying to address are complex (and they are - read more about "complexity" in this ODI working paper (PDF)), then it isn’t surprising that measuring progress and achievements is complex as well. 

While the simplicity of the MDGs may have helped mobilise support for development, that simplicity comes at a cost.  As Lindsey Nelson discussed in her STEPS presentation on multi-modal discourse last week, there are consequences to basing your strategy on a bumper sticker slogan.  Something to think about as we discuss the post-2015 development agenda.

(Still) Seeking a cure for Portal Proliferation Syndrome

By Susie Page

As co-editor to a forthcoming issue of the IDS Bulletin (autumn 2012) which will be focusing on ‘research communications’ (all facets of it!), I am currently reading about some very exciting work around this.

During our Call for Submissions, Geoff Barnard, former Head of the Information Department here at IDS, and now Head of Knowledge Management at the Climate and Development Knowledge Network (CDKN) sent me a link his blog on Seeking a Cure for Portal Proliferation Syndrome.

Geoff aptly captures the dilemma that anyone working in research communication and knowledge brokering will be familiar with – the temptation to solve some of the challenges around research communication and uptake in development policymaking and practice by gathering all the relevant research into a super, sophisticated website. The underlying assumption being – if only people could access the research (at the click of a button), then the rest will follow.

He obviously hit a nerve, as there was a stream of responses to his blog, including one from Catherine Fisher who also contributes to this blog, highlighting her work “Ten Portal Pitfalls” – I would urge you to read Geoff's blog and contribute to the debate. 

Can we 'scientifically' test for what works when it comes to research uptake?

Image from: http://72.167.62.13/

Coming from a medical background, the words “cure” and “syndrome” had immediate resonance for me: in the medical world, we are acutely aware of persistent diseases and syndromes with millions of pounds spent on testing for cures (although take it from me the health sector struggles equally with the business of research uptake! See Lomas on this, for example).

But does similar testing occur in this sector – the one that wants to get good research results out of the lab and into development policy and practice? Is it even feasible to conceive of a scientific test for something as amorphous as “knowledge” and “evidence”?


Going back to 'the cure', we should perhaps be asking whether portals are a syndrome or a symptom. If they are symptom, the problem could be that we think research is not being used in policymaking and practice because people don’t have access to it. Yet, surely the very proliferation of portals in itself highlights that this isn’t the problem – after all, how will one more portal succeed where others have failed? What do we know about the success and failures of portals? What actually do we know about the relationship between portals and research uptake?

With people still wracking their brains over measuring impact of research, there is room for some robust ‘scientific’ testing what is and isn’t effective for supporting research uptake and the place (or otherwise) of portals within this. We recently teamed up with 3ie to carry out an experiment on the effectiveness of the ubiquitous ‘policy brief’ (even more ubiquitous than portals, I would argue). The results are just beginning to come through. Watch this space for more on this – we will of course be sharing our findings with you!

Convening research excellence and beating the budget squeeze: 15 top-tips on managing expert e-discussions

Guest post: Adrian Bannister, Eldis Communities Co-ordinator, draws on experience from IDS  Knowledge Services to argue that online discussion events can successfully connect global thinkers without costing the earth

We all know that a demanding funding environment where delivering more for less can often conflict with personal and institutional commitments to the environment and to diversity agendas.

Digital technologies have provided development institutions with many opportunities for being more effective and efficient (in a broad sense). But when it comes to the efforts to substantively engage others in co-critiquing, re-constructing and advocating for research excellence, it seems that we still rely on face-to-face gatherings.

Why aren't we making more use of the Internet for debate and discussion?

As an approach for supporting policy influence to instigate real change online discussions seem to offer benefits that real-world events can't.

For starters, they enable us:

• to instantly connect disparate individuals from around the globe
• to enable participants to engage around their existing commitments
• to provide a safe(r) and empowering space for private discussions
• to instantaneously document the event for retrospective viewing
• to avoid the substantial time, cost and environmental overheads of bringing individuals together

Unfortunately, generating online discussion is much harder than it might seem. Despite the plethora of new tools for commenting and contributing to the web, stimulating user generated content remains a real challenge.

It is worth remembering that while the 'like button' sets the standard for easy / ubiquitous interaction across the web, in October 2011 Facebook chose to quietly remove its discussion functions completely.

A quick Google search shows that genuinely great e-discussions are rare – instead the web is littered with numerous poorly received examples with few if any comments.


What factors help make for 'great' online debates? What can we learn from each other?

I’d like to share some lessons that IDS Knowledge Services and partners have learned in our recent experience convening experts' e-discussions with prominent researchers / actors / practitioners as participants.

During 2011 and into 2012 we have used the Eldis Communities platform to host several such events on a range of topics, including: climate change, food security, philanthropy, social movements and gender mainstreaming. The events have been commissioned by the likes of Irish Aid, Oxfam and The Rockefeller Foundation.

While each is unique, they all share several commonalities. They are co-produced with partners, are funded by / contribute to wider programmes of work and are held behind closed doors.

One event focusing on Gender and Food Security* really stands out in particular in teaching us how to be highly effective at stimulating participants. In that situation, which lasted just 48 hours, a group of around 30 individuals collectively generated nearly 100 substantive contributions across 3 threads.

Top-tips for successful experts' discussions:

Here are some 'top-tips' that we learnt from doing this and other discussions (PDF). They highlight the wide range of things an e-discussion project team can / should do to maximise the chances of success.

If you don't have time to read all 15 points, here are my personal top three:

1. Plan it with the same attention to detail that you'd give real-world events – e-discussions, like all social events, can buzz with energy or descend into an awkward silence. The time-pressure during them is intense and attempts to revive a flagging situation can look clumsy. So set things up well in advance and you'll be less likely to need to 'get someone talking' during the event itself.
   
2. Scrimping on the budget is a false economy. While a successful online discussion will cost a fraction of a real-world event it pays to think carefully about the particular roles required and recruit project team as appropriate. Fund them generously (in staff time / direct costs), give them plenty of lead-in time and reserve a slot after the event for reflection.
   
3. Ensure your VIPs (Very Important Participants) feel special. As they are almost certainly super busy, persuade them to give up some of their precious time by making your invitations personal (not just the greetings!), mention those who recommended them, and highlight other eminent participants who will be involved.

Lastly, do what any good host does: keep calm and carry on...


** Thanks to Susanne Turrall (convenor) and Carl Jackson (facilitator) who played critical roles in this event and helped to pull together our learning from it.**