Buzzing about brokers: knowledge brokers reach across silos

By Catherine Fisher

Early in December I found myself in the unusual situation of being in a room full of people talking about knowledge brokering at a conference entitled "Bridging the gap between research, policy and practice: the importance of intermediaries (knowledge brokers) in producing research impact" * organised by the ESRC Policy and Research Genomics forum .

The event brought together people from UK universities, NGOs, public bodies ranging from health to education and a sprinkling of upbeat Canadians. The development sector was well represented, with DFID the best represented of UK government departments, perhaps reflecting the emphasis placed on evidence-based policy and research impact by DFID itself and within development sector more broadly.

It was the first time I had attended a conference of this kind in the UK so I was unsure what to expect. We know that knowledge about knowledge brokering seems to be silo-ed, not crossing between sectors. There are also differences in terms used to describe this kind of work. So as a presenter I was nervous I would be stating the obvious to a crowd who knew far more than I did. As conversation and coffee flowed, my fears were allayed: I had a lot to learn but, as I reflect below, the debates in the development sector I have been involved in are not miles away from debates elsewhere and in fact have something to add.

I presented as part of a panel exploring Knowledge Brokering in Development Contexts, alongside Kirsty Neman from INASP, Ajoy Datta from ODI and Matthew Harvey from DFID ( All presentations are available on the conference webpage, our session was 3E).

Here I share 5 of my reflections from the event:

The term "knowledge brokering" encompasses a wide range of action
I was not the only person to reflect that the term "knowledge brokering" was being used differently by different people.  Many people were using "knowledge brokering" to describe what I understand to be “research communication” that is, trying to ensure a piece of research is effectively communicated so that it has impact. This is in contrast to how I understand knowledge brokering, which I see as about helping to ensure that people are able to access research when they need it and that decision-making processes are informed by a wide range of research.  Put simply,  it's the difference between seeking to change a policy or practice to reflect the findings of a piece of reserach (research impact)  as opposed to seeking to change the behaviours of those in policy processes so that they draw on a wide range of research (evidence informed policy). There are of course grey areas between these extremes, for example, knowledge brokers within universities who seek to ensure that the knowledge of that university is mobilised for the community in which they are located: the Knowledge Mobilisation Unit at York University in Canada is a great example of this kind of practice that effectively sits between the extremes I have described.

Why we need labels (even if we hate talking about them)
Which brings me to my next point! People resent the term "knowledge brokering" as much as they resent talking about labels: for an interesting debate about the value of a label see KMBeing blog. Personally, I feel that without a term to describe this kind of work we would be unable to come together to discuss it (what would you call the conference/network?!). Conversely if we use the same term to discuss totally different things we risk confusing rather than clarifying our work.  The summary of the Knowledge Brokers Forum discusssion about terms and concepts is a good attempt to clarify and understand terms.  I still feel that language is the main tool we have to communicate our ideas and that it matters!


Consideration of power and politics: development sector has something to add
I was a little nervous that the debate about knowledge brokering would be very advanced, and the insights I shared in my presentation would be stating the obvious. Yet this did not seem to be the case, many of the issues raised during plenary and earlier sessions were familiar (e.g. pros and cons of the policy brief as a communications tool, how to motivate researchers to communicate their work, etc). The presentations from development sector raised two areas in particular that did not appear in other presentations I attended. Firstly, an attempt to understand politics with big and small “p”: looking at the contexts and motivations around decision-making. Secondly, a consideration of power and equity within knowledge brokering and asking “whose knowledge counts?”

What is a good knowledge broker? A fleet-footed, cheerleading, creative therapist! 

Image credit: Mick Duncan

A highlight for me was the presentation by David Phipps (York Uni) and Sarah Morton (Centre for Research on Family and Relationships) exploring the qualities of a good knowledge broker (pdf). From their experience it is someone who is fleet-footed, a cheerleader, creative, and a therapist. That is they have soft skills or competencies rather than specific technical capacities (although they will need these too!) plus a passion for the area, tact, negotiation and commitment. Like David and Sarah, I think the soft skills of knowledge brokers are key;  a paper I wrote last year entitled Five Characteristics of Effective Intermediary Organisations (PDF) explored how these soft skills can be supported and enabled at an organisational level.


Why don’t knowledge brokers practice what they preach?
As part of a devastating critique of the ESRC “Pathways to Impact” toolkit, Dr Simon Pardoe pointed out how little reference it made to evidence from social science that is relevant to the art and science of effective knowledge brokering. This observation that knowledge brokering somehow has no need to be evidence-based itself has emerged a number of times, for example, in the summary of the Knowledge Brokers Forum discussion which recognised the need for “greater linking of theory and practice”. I wonder whether the hybrid nature of the role means there are so many potential bodies of knowledge to draw on that people don’t draw on any! Sarah Morten and David Phipps talked of their practical ways of addressing this through “practice what you preach” Community of Practice and “learning days” respectively. They have a forthcoming paper to watch out for.

Any of these areas could be a blog posting, a paper or indeed a PhD themselves – I have just skimmed the surface of a great day. I hope the enthusiasm generated and connections formed will build towards greater understanding of the theory and practice of knowledge brokering.

Links:
Archive of  tweets posted from the conference : contains some interesting thoughts and links to resources.

• The long titles of these events reflect the difficulty of describing them and the lack of shared language – check out the conference I organised in collaboration with HSRC in 2008 which laboured under the title “Locating the Power of In-between : how research brokers and intermediaries support evidence-based pro-poor policy and practice"

How can we make research communications stickier?

Guest Post: 
James Georgalakis, Communications Manager at the Institute of Development Studies (IDS)

James looks back on 2011 and reflects on the most-viewed news stories about IDS research, featured on the IDS website. Here are his thoughts on what made them popular:

Image from: http://serenelyfull.blogspot.com

Which were the "stickiest" stories on the Institute of Development Studies’ (IDS) website in 2011? Is the use of SEO (search engine optimisation) techniques cheating? And do we improve research impact by dropping in the names of royals and celebrities? Just some of the questions I try and answer in this end of year blog.

News or blogs on development research are highly unlikely to compete with viral YouTube sensations involving celebrities or pets that reach millions in hours. But a quick scan through the top ten most viewed news stories from the IDS web site from 2011 still tells us a lot about what it is that makes some research communication super sticky.

In fact, some IDS news stories were so sticky they came top of the 2011 list even though they were not even posted in 2011. For example, a story about Ian Scoones’ book on Zimbabwe’s land reform, posted in November 2010, came in at number one followed closely by Andy Sumner’s New Bottom Billion, also a 2010 story.What did these stories have that others did not? Well of course exciting, original research helps!

Both of these examples tick that box. Scoones’ revelation that Zimbabwe’s land reforms were not so bad after all quickly ignited a lively media debate that has just run and run. It was still running a year on with this BBC Radio 4 documentary looking closely at his claims. Sumner’s startling findings on the changing nature of poverty also ignited debate in the media and academia and stoked up the blogosphere. This is why web traffic just keeps on finding its way back to the original web stories.

However, not all of our top ten scorers from this year were about original research with counterintuitive findings.

Consider our official number one story from 2011 (the most page views of all the stories actually posted in that year). It is news of a special conference celebrating the work and ideas of IDS Research Associate, Robert Chambers, with links to all the related materials.

This is not hard news and it is not particularly surprising or controversial. Instead, what we have here is the awesome stickiness of a big, well stellar, name in the development research community. How often is Chambers’ name googled? How big are the networks of people who have shared the link with one another and will have flocked to access this content? How quickly did news of his conference spread across the blogosphere with links back to the original content?

Robert Chambers was not the only big name to make the IDS top ten. At number two we have Kate Middleton and Prince William who, according to the IDS headline from last April, ‘got engaged in Africa’s land grab hotspot’.

Yes that’s right! We took the convergence of a royal wedding, the happy couple’s obscure connection with land grabs and a recent IDS hosted academic conference on land grabbing to produce something really sticky. Just think how many royal wedding fans inadvertently became informed on the land grab issue. Such are the rewards of working in research communications.

To be fair to the Future Agricultures Consortium, who organised the International Conference on Global Land Grabbing, this is a pretty sticky topic even without royal endorsement. It features in no less than three of our top ten spots from 2011.

The tactic of using highly topical and sticky words or names in your headline is known as search engine optimisation (SEO) or cheating, and it works. When used well, SEO means that big search engines like Google, Yahoo! and Bing will list the relevant link(s) to your website in their first page, increasing the likelihood of people clicking on your links.

Of course topicality is itself one of the greatest assets of all. Just take the Arab uprisings or the Horn of Africa crisis which both, perhaps not surprisingly, made our top ten.

However, as any newspaper sub-editor will tell you: if all else fails you just need a great headline.

What else can explain a story about a podcast from an IDS Sussex Development Lecture coming in at number eight? Don’t get me wrong it was a really great (and packed out) lecture but a story about a podcast!

It was titled: ‘Decline of the NGO Empire – where next for international development organisations?’ Pretty good huh? Yes, it was one of mine. Also very nice, (and not one of mine) and slipping in at number ten is: ‘Taking the scare out of scarcity - how faulty economic models keep the poor poor’. See what we did there?

Now I know the whole subject of web traffic drivers and SEO is way more complicated than this.

Our study of the IDS top ten has severe limitations. Clearly content published early in the year has an advantage over the stories that came after and some of our high scorers were boosted by extended periods on the IDS home page. Plus it is about push as well as pull and for that we have to start thinking about the role of social media, where content is positioned on the site and a multitude of other factors.

This is another blog for another day, but my point is this: if you want lots of people to find the story about your research irresistible consider this 2011 IDS top ten. Methodologically sound and original research is great, but a crowd pulling name, timeliness, a big surprise and a great title all help a lot too.

If you want to know more about this stickiness business I suggest you read Made to Stick: Why Some Ideas Survive and Others Die, by Chip and Dan Heath which should be a compulsory text for all those working in research communications.

Happy holidays!
James Georgalakis, Communications Manager, IDS

And here is that 2011 IDS website news stories top 10 in full (listed by order of unique page views to date):

1. Revolutions in development reflecting forwards from the work of Robert Chambers
2. Prince William and Kate Middleton engaged in Africa’s land grab hotspot
3. Bellagio Initiative starts an IDS led global debate exploring the future of philanthropy and international development
4. Debating the global land grab
5. How a citizen led approach can transform aid to governance
6. The East African food crisis beyond drought and food aid
7. Experts warn of new scramble for Africa at an international conference on land grabbing
8. Decline of the ngo empire – where next for international development organisations?
9. The people revolt why we got it wrong for the Arab world
10. Taking the scare out of scarcity - how faulty economic models keep the poor poor

Think before you jump (into the social media ocean)

By Emilie Wilson

In the last few months, I’ve been reading and following debates about the use and impact of social media, especially blogging, so where better to share my findings and reflections than in this blog...

First, a spate of recent research and surveys on use and impact of social media in the development sector

The Global Development Network (GDNet) has recently published a review of the use of social media (or not) for research collaboration amongst southern academics. As a network comprising of more than 8,000 researchers worldwide, GDNet should be commended for not jumping onto the social media bandwagon without doing some homework around relevance and appropriateness for its members first.

Findings seem to show that, while there are regional and gender differences, the levels of up-take amongst academics is generally low. Barriers for adoption include poor infrastructure or equipment (still), usability, time and perceived value or credibility of the tools as well as lack of institutional incentives. Sound familiar?




Accessed from: http://pedagogy.cwrl.utexas.edu/

 In the global North, the development sector (at least when it comes to aid agencies, NGOs and think tanks) is well and truly on the social media bandwagon. In the last couple of months, Devex published Top 10 Development Groups on Social Media, Vodafone’s World of Difference charity recommended Top 10 Development blogs, and the Guardian highlighted 20 blogs in its Global Development Blogosphere.

Those engaged in social media seem to range from individuals (activists, aid workers or academics) to small groups with shared interests to institutions who have a clearly articulated ‘social media strategy’.

This has prompted some thinking around sustainability and impact.

Following the closing down of established and popular blogs, such as AidWatch, Duncan Green, who writes the popular From Poverty to Power blog, recently speculated on whether the blogging bubble was about to burst. However, when considering whether to wind down his own blog, Duncan came up with some good reasons to keep it going. Some were personal: “blogging forces you to read stuff more carefully and come to a view”, others aspirational “blogging has turbocharged a part of the development discussion best described as the ‘ideas space’”.

The blogosphere as an epistemic community

Duncan’s personal reflections are corroborated in a recent paper by David McKenzie and Ben Őzler on The Impact of Economic Blogs (PDF). This is a substantive attempt at collecting evidence around the following questions:
1. Do blogs improve dissemination of working papers or journal articles?
2. Do they raise the profile of their creators?
3. Do they cause changes in attitudes among their readers or lead to increased knowledge?

It seems that their evidence shows positive results:

• Blogging has a significant impact on abstract views and paper downloads
• Regular blogging is strongly and significantly associated with being more likely to be viewed as a favourite economist
• A majority of readers have read a new economics paper as a result of a blog posting, and more policy-oriented respondents say that blogs are having an influence on how people feel about the effectiveness of particular policies

This latter finding chimes well with a recent survey conducted by SmartAid, which asked respondents, amongst other questions, why they read [development] blogs.

This was their encouraging response.


Graph accessed from: http://findwhatworks.wordpress.com/2011/09/22/blog-survey-findings-5-why-the-audience-reads-blogs/


A full break-down can be found on Dave Algoso’s blog, Find What Works, and makes for some interesting perusal.

Phew! It’s all a good reason to keep blogging then!

However, in case we bloggers start to take ourselves too seriously, I wanted to share a view on blogging from the McKenzie and Őzler paper which I found amusing, but I hope no one who reads this thinks it applies to Impact and Learning...

“a largely harmless outlet for extrovert cranks and cheap entertainment for procrastinating office workers” (Bell, 2006)

Post-Script: do you write a blog? What do you do to measure influence and impact of your blog?

Are evaluators too fixated on objectives? The case of conflict-affected states

By Chris Barnett

It is a while since I was co-facilitating a workshop outside Brussels on evaluations in conflict-affected situations. We were exploring alternative approaches to evaluation, building on recent experiences of multi-donor evaluations in South Sudan and the DRC. Previously I’ve raised the issue about whether theories of change have their limitations (see my earlier post on evaluation approaches). In the end, we spent much more time exploring the use and limitations of objectives in evaluation practice. Here is a brief summary of those discussions.

Objectives form the reference point for most mainstream (development) evaluations. Objectives underpin the OECD/DAC evaluation criteria. For instance:

• Relevance: Are the programme objectives aligned to the situation?
• Effectiveness: Were the programme objectives met?

In development evaluation,[1] we sometimes come across programmes that are radically re-designed during the mid-term. When this happens, the evaluator is faced with a dilemma: do you measure success against the original objectives, but risk undervaluing a genuinely flexible and responsive programme? Or, do you measure success against the re-designed objectives, but risk excusing an initially poor design?  The former often seems too harsh, the latter too lenient.

In conflict-affected situations, flexibility is a must and re-design occurs more often. Many things cannot be planned in advance, and managers respond to constraints and opportunities as they arise. Yet, being responsive to a dynamic context can undermine plans made three or five years previous. There are also other reasons why clearly planned objectives may be undermined in such fragile contexts: 

• Terms such as peace and conflict prevention may not be well defined. Indeed, not all stakeholders may share the same understanding within a single programme (e.g. differing civilian and military perspectives). 
• Peace may not be an explicitly-stated objective. The Utstein palette (PDF) is helpful in this regard, as it unpacks the full range of activities that may contribute to conflict prevention and peace-building. Yet for the evaluator, this all-encompassing view of peace-building activities presents a challenge; with interventions contributing to peace even when this is not stated.
• Objectives may be politically-sensitive, and may be hidden or masked. For the ex post evaluator, a retrospective interpretation of such objectives opens up the real possibility of contested findings. This is especially so where the ‘evaluation space’ is highly contested, as was the case in the CPPB evaluation in SriLanka (PDF).

So, what provides an alternative point of reference?

Well, the perspectives of local people: their needs, and their understanding of what drives the conflict. The draft OECD Guidance on Evaluating Conflict Prevention and Peacebuilding Activities (currently under revision), provides an entry point for developing such an approach. It layers Conflict Analysis onto the traditional DAC evaluation approach, although it doesn’t fully integrate the two approaches. Instead, a more radical view would be to use the conflict analysis (especially an analysis by local people about what drives the conflict) as a reference point for evaluating the intervention. For instance:

• Relevance: To what extent does the programme reflect a local understanding of the conflict?
• Effectiveness: To what extent does the programme address the drivers of the conflict?
• Impact: To what extent is the programme leading to reduced conflict, by building peace?

Some will argue that this shifts accountability away from programme managers. After all, managers should be accountable for their design, and therefore measured against the objectives they stated. But, this suggested approach is more about making managers accountable to their constituents (their target audience), and focuses on whether they are addressing local needs and factors behind the conflict.

---

[1] By development evaluation, I am simply referring to evaluations conducted in relatively stable developing contexts, in contrast to those undertaken in fragile or conflict-affected states.

Exploring the black box together: evaluating the impact of knowledge brokers

Cartoon by Sidney Harris (2007)

By Catherine Fisher

I love this cartoon! 

It seems to capture the idea of the "black box" that lies between the activities knowledge brokers and intermediaries undertake and the outcomes and impacts they seek to achieve. That’s not to say that they don’t achieve outcomes in the real world, rather that the pathways by which their work brings about change are difficult to unpack and evaluate.

The Knowledge Broker’s Forum (KBF) has started exploring this "black box" of how to evaluate the impact of knowledge brokers and intermediaries in an e-discussion running from 31 October until 9 November. I am (lightly) facilitating this discussion, along with Yaso Kunaratnam from IDS Knowledge Services.

If you would like to participate, you can sign up on the forum's website, it's open to anyone with an interest in this area.

Challenges in evaluating impact

We know there are a lot of challenges to evaluating impact of knowledge brokering. Some challenges stem from the processes (psychological, social and political) in which knowledge and information bring about change, the contested nature of the relationship between research and better development results, and the challenges of identifying contribution to any changes in real world contexts. This is particularly challenging for actors that seek to convene, facilitate and connect rather than persuade or influence.

As well as these quite high level challenges, there are the very practical issues around lack of time and resources to dedicate to effectively understanding impact. These challenges are explored in a background paper (PDF) I prepared as food for thought for those taking part in the e-discussion.

Being an e-discussion amongst 400+ knowledge brokers from all over the world, I am not sure yet where discussions will go, but I am hoping that it will shed some light on the following areas:

Breadth and depth of impact and outcomes  

How far do people go to identify ultimate outcomes of knowledge brokering work? I feel we can certainly go beyond immediate impact (e.g. personal learning) to push towards what that resulted in, however I wonder if it is meaningful to start looking at human development and wellbeing indicators. It will be interesting to see how far others are going.

Understanding behaviour change

If knowledge brokering is about behaviour changes that ensure greater engagement with research evidence, how are people defining those behaviour changes and are how are they measuring them? Are we too easily impressed with stories of information use when these could in fact hide some very poor decision-making behaviours?

Opportunities for standardisation of approaches and data collection

If people have come up with ways of doing this, is there any appetite for standardising approaches to enable greater comparison of data between different knowledge brokering initiatives? This would help us build a greater understanding of the contribution of knowledge brokers beyond the scope of any one broker’s evaluation.

I’ll also be interested to explore and challenge some of my assumptions – in particular that building some kind of theory or map of change is an important starting point for defining and then seeking to evaluate impact. This has been discussed previously on this blog and is a hot topic at the moment.

Our discussion will face challenges – not least the huge variety of types of knowledge brokering and contexts in which it is undertaken may mean there is not enough common interest. But I am sure that there is a lot of experience in the group that can be brought to bear on these questions and, in 10 days time, we will have a better idea of what is known, who is keen to explore this further and and hopefully how we could move forward to develop our understanding in this area.