"The future is already here – it’s just not very evenly distributed"

By Emilie Wilson

I overheard this quote the other day, as I was out buying my morning coffee. It’s part of the joy of working on a university campus to be overhearing conversations such as these. The quote is attributed to William Gibson, “the "noir prophet" of the cyberpunk subgenre of science fiction”, according to WikiQuote.

Setting aside the fact that I know nothing about cyberpunk science fiction, the quote sparked a train of thought, which had begun that morning when I was looking at this excellent graphic published the Guardian website entitled “How Africa Tweets”. 

The map looks at the top 20 countries based on a 3-month analysis of geo-located Twitter traffic in Africa – no one will be surprised to see the biggest tweeters being South Africa, Kenya, Nigeria, Egypt, and Morocco. What surprised me were the presence in the ‘top twenty tweeters’ of countries like Sudan, Gabon or Angola, and absence of others such as Zimbabwe, Uganda, Botswana or Senegal (countries I associate, perhaps wrongly, as has having a decent technological infrastructure and a vibrant civil society).

With the West Africa Cable System, a 14, 000 kilometre long fibre optic submarine cable with a capacity of 5.12 terabits per second (Tbps), due to be operational in March this year, these figures are no doubt set to increase.

But what does this mean for development, and more specifically, for sharing, communicating and assimilating research which supports development?

ICT4Development – still the new kid on the block?

The development sector has a reputation for getting excited about technological innovation and hoping it will yield a quick fix to some of the most intractable problems relating to poverty and social injustice. At IDS, there is a whole Research Team devoted to analysing the interface between human development and technological progress. And ICTs have joined the fray alongside agriculture, biology, engineering and medicine – the 3rd International Conference on Mobile Communication Technology for Development is being held in Delhi next week, closely followed by the International Conference on Information and Communication Technologies and Development.
 

Does E.M. Rogers analysis on adopting innovation apply in this context?

IDS Knowledge Services has a small team dedicated to exploiting the opportunities afforded by innovative technology, and especially people in the South’s use of this, to improve access and demand for research. For example, the Open Application Programming Interface (API) project is designed to allow technical developers to access the datasets that sit behind renowned development research and information services, Eldis and BRIDGE. This access means they can pick and choose what data they want and need, and repurpose it by developing their own applications.

And it’s not about geeks playing with geeks - in a concrete example of how this can be used: BRIDGE has been working with Uruguayan NGO, ciedur, developing online resources that both bring together Spanish resources from BRIDGE alongside other relevant materials, in a way that is relevant to the Latin American policy context.

How is this being done? The Latin American resources are collected and shared online using an open source content management system called Drupal. A Drupal plug-in (developed by One World South Asia) allows those managing the system to automatically pull in data they want from the Eldis/BRIDGE dataset and repurpose it for their own website and online services.

And this is good because....?

• It challenges the ‘top-down’, ‘North-South’ direction of technological innovation and editorial decision-making – IDS Knowledge Services partners develop their own applications and select the data that is relevant and useful for their contexts
• It is relatively cheap: no need for high-tech R&D laboratories or factories, just creative minds and internet access. And these are active in abundance, not just in the Africa, but in Asia (East, South and West!) and Latin America
• “Open” also avoids duplication and the needless funding of multiple portals and websites which all do the same thing: collect and disseminate research in the hope that access alone is what we need to get research into policy and practice.

Handy technological short-cuts enable us to focus on emerging areas of research communications and knowledge brokering such as stimulating demand for research, or supporting non-academics to be ‘evidence-literate’ (see work of Kirsty Newman at INASP). 

Some challenges for this ICT4Development model

Having been in this sector for quite a while (BRIDGE is 21 years old, Eldis is 16 years old), we are quite aware of some of the challenges to this paradigm:

• Quality, trust and credibility: traditionally, “closed access” models for sharing research, such as peer-reviewed journals, ‘paid-for’ materials, such as books, or resources only available via a reputable institution, are supposed to guarantee all three. What happens when the credible is aggregated with the less credible? Can we maintain trust in the resources divorced from the branding and reputation of their original source? How is quality maintained once access is ‘opened’?
  
• Ownership, power and access: fundamental issues around funding and publishing research are not really addressed by the ICT4D model (e.g. the incentive systems where researchers achieve greater points for promotion by publishing in prestigious closed access journals), there is still a digital-divide, even with increasing internet access (e.g. urban v rural access);  many free internet services are privately owned and developed (e.g. Google or Twitter), particularly by companies based in the global North: should we be worried about our dependence on them? See ITforChange’s excellent thinking and research around this area
  
• Cost and financial sustainability: the open source model is one in which developers innovate ‘for free’, but then are more likely to be employed on the basis of their innovative contribution – but is this sustainable in the development research context? Who will pay for it?

We are still grappling with these challenges – are you?

Monitoring and evaluation in partnerships: why learning comes first

Guest post: Andre Ling, Research Officer/Technical Assistance with the Agricultural Learning and Impacts Network (ALINe) at the Institute of Development Studies (IDS)

The Impact and Learning Team and the ALINe project members share insights and good practice during bi-monthly Learning Labs, an afternoon of learning and reflection framed by our project work and esearch questions. Andre shares his reflections in this blog post.

Last week's Learning Lab with the Impact and Learning Team (ILT) touched on two topics that are probably relevant to just about all actors involved in development processes: partnerships and sustainability. Both are buzzwords, frequently used and misused, open to a range of interpretations and often obscuring the hard realities that confront development practice.

This post looks specifically at 'partnership' and what approaches to monitoring and evaluation (M&E) may be most appropriate in a partnership context.


The term 'partnership' frequently glosses over the complexity of inter-organisational relationships:

• The many reasons for which organisations find themselves in a partnership to begin with
• The power asymmetries inherent in the common model of grantor-grantees that defines many partnerships
• The different interests, values or ideological positions of partners
• The specific organisational development needs of different partners

Image from: http://www.ramsar.org

The prevalence of such factors can lead one to the conclusion that most partnerships are marriages of convenience as those joining in such ventures do so largely to serve their own interests. But, at the same time, it barely deserves mentioning that an individual actor can do little to address the complex problems of our times.

So how do we go on together honestly? How do we make partnerships work?

And what kind of M&E makes sense in partnership contexts?



To begin with, taking stock of power asymmetries within the partnership and mapping out the divergent interests, values, worldviews, spheres of concern and needs of partner organisations may be a good place to start. Often the leading agency in a partnership (usually a grant-maker of sorts) will have goals of achieving some kind of systemic change. The other partners (frequently grantees) may be more concerned with implementing specific activities to contribute to more localised changes and be less comfortable with confronting systemic change. Such a divergence creates a critical disjuncture, resulting in the leading agency wanting to make changes in the other partners; to make them see the same way. Failure to engage with this rift in a sensitive manner can lead to a variety of problems, for example low trust and limited ownership.

One response is, perhaps, to establish a joint M&E framework; a system of indicators and reporting requirements that can be deployed to ensure that all organisations are 'on the same page' and are held accountable to achieving desired outcomes in a standardised way. The danger here is that compliance and standardisation – both power mechanisms – ride roughshod over what might be considered the more crucial goal: learning how to work together effectively to achieve mutually desired change.

This is not to dismiss the contribution of joint M&E frameworks but rather to put them in their rightful place as servants of learning (individual, organisational or institutional) rather than formalities implemented for their own sake. To clarify, the point is that learning encompasses a far wider set of practices and activities than what usually goes by the name of M&E and, furthermore, that regular M&E is not a sufficient pre-condition for learning to take place. Just think of all the evaluation reports that have been shelved and forgotten.

An emphasis on learning prior to M&E opens the door to a potentially more diverse set of tools, techniques and practices that can be used to (a) build relationships of mutual trust; (b) reveal and question entrenched assumptions; (c) share and cultivate more systemic ways of thinking about the nature of the problems that the partnership is seeking to tackle.

This can prepare the ground for co-creating an M&E system that is both oriented toward learning and situated within a partnership culture that is supportive of learning. A significant consideration here is that this means taking the partnership itself as a unit of analysis to be monitored, evaluated and learned from and through over time.

Change is hard but not impossible with a little help from ELFs (part 2)

Guest post: We welcome back Elise Wach, Evaluation Consultant at the Institute of Development Studies (IDS)

To follow on from my post last week on how change is hard to achieve, even when you know an approach isn't working, here's an update on the Learning Retreat that the Impact and Learning Team facilitated for the IRC (International Water and Sanitation Centre) Triple-S Initiative. 

We kicked off the 2-day retreat by hearing reports back on the variety of learning streams that had taken place so far and assessing progress.  This led into discussions about the approach of the initiative as well as the approach of the evaluation and learning.  Were course corrections needed in order to achieve the desired outcomes of the initiative?  Were adjustments to the evaluation and learning methodologies needed in order to better capture initiative progress?

In terms of the approach of the initiative, one of the primary concerns was related to external communication - getting the right information out to the right people in a timely manner.  IRC’s key staff immediately made commitments to improve this:

1. Set up an organisational blog to get discussions started and put information out on the web (see Emilie Wilson’s entry about the merits of blogging).  
2. Post more information and resources on the website, even if not yet polished.  A key phrase throughout the retreat was to ‘not let the perfect be the enemy of the perfectly good’.  Resources and information may take the form of videos, slide decks, and less formal reports.
3. Improve the layout of the website to make it more user-friendly, and the search engine tags to make it easier to find.

Other commitments were also made to improve the approach of the initiative, including a re-visit to the Theory of Change.  This will surely be discussed in detail at the next Learning Retreat which will go into more depth into the data and its synthesis and also include the advisory group for the initiative (scheduled for the end of April).

See more at http://www.waterservicesthatlast.org  

In terms of the evaluation and learning approaches, a few changes will be made to the timing and scope of data collection and analysis.  For example, given the disconnect between policy and practice discussed in last week’s entry, it was decided that in addition to analysing the policy documents of key agencies and organisations in the water sector, IRC and ILT will also analyse documents that might indicate that shifts in policies are being reflected in practice, such as calls for proposals and project reports.  That information will of course be posted on the Triple-S website in an effort to give these agencies a little extra nudge towards sustainable services that last.  



It was also determined that impact weighting for different outcomes and milestones might prove useful, along the lines of DFID’s revised approach to logframes (PDF) (though Triple-S is looking to move away from a linear/tabular format and towards more mind mapping and video).  

What was interesting to me about the retreat was the fact that none of the data discussed revealed anything incredibly new or surprising to the Triple-S team.  But for some reason, getting a group of people in a room together and setting aside time to specifically discuss progress and obstacles can be extremely effective for getting decisions made…especially if External Learning Facilitators (endearingly referred to as ELFs) are there to help the process along.

IRC has committed to starting their blog in the next week and will soon be posting more resources on their website, including the full report from the Learning Retreat (which will cover much more than I’m able to include here). As for making it easier to find the Triple-S website, you can try to Google it for yourself, but I think they’re still working on this one (unfortunately there’s a clothing company that goes by the same name!), so just in case, the "Water Services that Last" website is here.

Change is hard

By Elise Wach

Elise Wach is currently working with the Impact and Learning team to support the IRC International Water and Sanitataion Centre through a learning process around its Triple-S Initiative 

Penelope Beyon’s blog about failure and learning brings up some interesting and very valid points about the recent attention to failure, evaluation and learning in development. While it is essential (and quite difficult) for the development community to know when their programming is unsuccessful, and admit this, it doesn’t do any good if we don’t then learn from our failures and change our approaches so as not to repeat them.  

But how does change happen?

The IRC International Water and Sanitation Centre is in the middle of a six-year initiative which is attempting to shift the rural water sector away from a one-off infrastructure-based approach towards a Service Delivery Approach; what they term, Sustainable Service Delivery at Scale (Triple-S). How to enact change is exactly what they’re trying to figure out.

For decades, most development organisations and agencies in the rural water sector (like most sectors) did not know about, or did not want to know about, their failures. They were oblivious to the fact that the majority of their boreholes fell into disrepair within five to seven years, or that many were never even used at all.  Without this knowledge, it is easy to see why development organisations and agencies charged ahead with the same unsuccessful approaches.  

However, in a recent round of interviews I conducted with key stakeholders in the sector (as part of the Impact and Learning Team's support to IRC on this initiative),  it was overwhelmingly apparent that now, everyone in the water sector knows that the standard approach to rural water supply has been ineffective and unsustainable. It is common knowledge that the sector has been failing.  

So the rural water sector has overcome that essential, but difficult first hurdle of finding out about and acknowledging failure. 

Source unknown, but widely available

And in the most recent round of interviews, key stakeholders in the sector generally agreed that the discourse at the top – the policy-level – is starting to reflect these revelations.  But funding practices and implementation on the ground seem to have continued relatively unchanged: the same infrastructure-focused, unsuccessful approaches continue to dominate. Why?

Because change is hard.

One interviewee explained, ‘We’ve been engineered to do small-scale piecemeal interventions…so of course shifting to more of a sustainable approach at scale (vis-à-vis financial flows, regulations, norms, and standards) is going to take time.  There will be resistance to change.’

Changing approaches to realising change

To date, IRC ‘s Triple-S initiative has been attempting to accelerate changes within the sector through three main approaches:

• Relationship-led (i.e. using champions to mobilise change)
• Value-led (i.e. leveraging peer pressure and creating coalitions for change)
• Evidence-led (i.e. providing proof that the current approaches don’t work and proof that other ones do)

The initiative has also been exploring the relationships between policy, funding and practice.

This week, IRC and the Impact and Learning team are holding a learning retreat to go over the findings from the most recent round of stakeholder interviews and other evaluation data.  Based on this, IRC may refine its Theory of Change and tweak its approach to help maximise the efficacy of the initiative moving forward. 

We’ll report back on the outcomes of that learning retreat next week.

In the meantime, a final thought. If more development actors followed a similar approach to IRC (i.e. if they thought through Theories of Change for their approaches and periodically revised them based on real-time evaluations and analysis), it’s not unrealistic to think that the way we work in ‘development’ would be quite different. That is, the phenomenon that Penelope termed as ‘reinventing broken wheels’ might not be as common. Change is hard, but not impossible, and it is certainly needed.

Are we reinventing broken wheels? Let’s talk about the ‘F’ word

By Penelope Beynon

A common saying goes "The only real failure in life is the failure to try."  I disagree.

I think the worst failure in life (and in knowledge brokering) is the repetition of an established mistake. That is to say, the worst failure is the failure to learn.

In recent months, I have come across an increasing number of websites, discussions and articles that almost celebrate failure, in an effort to foster a culture of sharing and learning from others’ mistakes. The Engineers Without Borders (EWB) website Admitting Failures is a good example. In their own words:

"By hiding our failures, we are condemning ourselves to repeat them and we are stifling innovation. In doing so, we are condemning ourselves to continue under-performance in the development sector.

Conversely, by admitting our failures – publicly sharing them not as shameful acts, but as important lessons – we contribute to a culture in development where failure is recognized as essential to success."

While I agree with the premise, often times it is not fully realised.

Image from: http://st-anley.blogspot.com

Ironically, perhaps, several of the ‘failures’ admitted on the EWB website are, in fact, examples of people’s failure to learn from past mistakes – their own and those of others. That is, they are reinventing broken wheels, sometimes under the guise of 'innovation'.



Innovation is important for progress, and with innovation comes a certain level of risk. But I think these risks need to be calculated and one of the key considerations should be a thorough investigation of whether this particular experiment is truly an innovation or whether it has already been tested elsewhere. That is, an honest commitment to learning before doing as well as learning after doing. I hear the echo of Catherine’s recent blog where challenges knowledge brokers to practice what they preach .

Lessons identified or lessons learnt? 

Learning is a big theme for the Impact and Learning Team at IDS  and we have recently been thinking a lot about the difference between a lesson identified and a lesson learned.

In our view, a lesson is only really 'learned' when the implications of the lesson are acted upon. Far too often we see After Action Reviews and evaluation documents that recite from their own experience ‘lessons’ that are insights long established internally and already documented in the experience of others (e.g. developing partnerships takes time, communication matters, etc.). Very seldom does anyone pick up that the worst failure here was not the failure to communicate but the failure to identify ahead of time that communication matters and to learn from others’ experiences about how to do it well.

One outstanding example of a lesson that was learned (albeit the hard way) is retold by Lieven Claessen, a researcher from the International Potato Centre (CIP),s  in two short videos produced the Consultative Group on International Agricultural Research (CGIAR)'s ICT-KM programme.

In the first video, Claessens identifies the lessons by bravely telling a rather sobering story about his failure to communicate research findings in a way that people likely to be affected could understand and use for decision making. Had the findings of his 2007 research been acted on, the devastating effects of the 2010 mudslides in Eastern Uganda could have been mitigated, potentially saving the lives of hundreds of people and the livelihoods of hundreds more.  In his second video, Claessens evidences his learning by telling how he has changed his approach and commitment to communicating research to ensure he does not repeat this same mistake.

I find Claessens' story deeply moving for two reasons.

Firstly, I take my hat off to anyone who owns up to their part in a failure with such devastating consequences. Especially where that failure could as easily have been passed off to someone else.

Secondly, I find the story unique in its clarity about the link between research communication and wellbeing outcomes. Or, in this case failure to communicate research and negative outcomes. Often that link is much less clear for knowledge brokering. In fact, just as it is difficult (if not impossible) to evidence attribution of development outcomes to knowledge brokering work, it is equally difficult (if not impossible) to evidence negative development outcomes to failure in the same area. Perhaps this provides something of a safety net that allows us to distance ourselves from consequences, or maybe it is one of the reasons that it is apparently so hard to talk about failure in the knowledge brokering arena.