From “Human in the loop” to “AI in the loop”

Reflections from the NetHope Global Summit on who owns AI outcomes, how trust is built, and why oversight should depend on risk.

ai in the loophuman oversightai governanceaccountabilitynethope global summit

Conference room at the NetHope Global Summit with attendees seated at round tables and a presentation on screen

A week after the NetHope Global Summit in Amsterdam, one of the many striking ideas from the opening plenary panel discussion is still on my mind: moving from “human in the loop” to “AI in the loop”. The panel participants - Michelle Parker, Director of AI at NetHope, and Charlotte Lindsay Curtet, an independent expert on digital transformation in the humanitarian sector, set the tone for the whole Summit, and much of how I'm thinking about CURA's growth stems from their discussion.

Who owns the outcome?

The panel was clear that the organisations using AI are ultimately accountable to the communities they serve, and that when something goes wrong, nobody formally has to take responsibility. This seems to be a growing concern overall. Within organisations, there is generally no clear consensus on who owns which AI outcomes. As AI policies and frameworks are introduced, a foundational accountability structure introduced at a department level is essential with agent owners, team owners, and ultimately leadership, who set the final AI guardrails.

NetHope Global Summit welcome slide introducing speakers Jean-Louis Ecochard and Stephanie Siy of NetHope

Trust is about transparency and timing

Everyone agreed that trust with communities comes from transparency of AI use. People have a right to know when they’re chatting with a chatbot for example, but perhaps not the technical details of how an AI model is deployed and at every second. The panel also asked whether people could opt out when AI affects their rights, and what the appeal process looks like. According to them, many organizations can’t answer this yet, partly because they don’t fully understand the tools themselves with everyone learning and adapting as they go.

I agree that clear, timely communication of AI use is essential. But what I keep coming back to from the discussion is how much information is too much, who should receive it, and at what point in the process.

Oversight depends largely on risk

Michelle’s recommended “AI in the loop” language puts people at the center of how the humanitarian and international development sector adapts to new ways of working. AI is one part of the process, with people staying in charge of it. And that’s where oversight came into the discussion. When it comes to using AI agents, it depends on how risky the task is: the riskier the task, the more human and the less agentic it should be. I agree, and I think a risk-tiered approach is very important across nonprofit use cases, whether that means more human reviews within a given agentic workflow or fewer.

Charlotte named three things that make that oversight meaningful:

  • Knowing the context, and how and where AI implementation makes sense
  • Understanding the impact and implications of using AI towards communities
  • Being able to override a decision which was AI-made

Local context gets lost

The panel also brought in the interesting perspective of localisation. A lot of contextual knowledge gets lost when AI models are more international in scope, so her recommendation was to build in contextual local knowledge from the start. I think this will matter even more as we think about agentic AI for field research teams and MEL functions, where the work depends so much on local context.

Get your hands dirty with AI

Michelle encouraged organisations to get hands-on with AI, safely and responsibly, and to share what worked and what didn’t. That also makes the procurement of these tools an intentional decision for functions across the organisation, and training people to use them just as important. I’d go further. It is absolutely crucial to get your hands dirty with any product or platform, because that’s the only way to really know what it can do for your organisation, or perhaps what it can’t.

Back at the CURA booth

These conversations set the tone for our booth interactions, where nonprofit and humanitarian stakeholders asked us how secure their data would be and where we envision their staff being involved in automating their work processes. Every organisation has a different risk appetite, and I believe it’s about finding a happy medium where everyone is comfortable: starting with AI training and implementing AI together with the organisation, not just for them.

A NetHope Global Summit attendee looks at a laptop screen held by Sunayana Sriram, who wears an Impact Mission AI badge, while another attendee looks on
Anand, Sam and Sunayana from CURA stand together smiling at the NetHope Global Summit, wearing CURA polo shirts and conference badges, with the CURA booth display in the background
Two people at the CURA booth at the NetHope Global Summit, smiling and gesturing toward a laptop, with a screen behind them showing the CURA interface and a banner for CURA’s ‘Deploy AI Responsibly’ messaging
Previous When More Help Isn’t Better Help