Introducing asago: Open source AI safety and governance orchestration

Today, Red Hat announced asago, a collaborative, open source AI safety project in partnership with  Alquimia AI, Brave Software, EvalEval coalition, IBM Research, Interdisciplinary Transformation University Austria, Microsoft, MIT Lincoln Laboratory, North Carolina State University, NVIDIA, and The Alan Turing Institute. In this blog post, I’d like to spend some more time elaborating on why we felt a new community was necessary, what we aim to achieve, and what the state of play is at present.

The open source AI safety ecosystem is a rich one. There are plenty of excellent and mature projects across areas such as guardrails, evals, red teaming, and agentic security, and these are complemented by comprehensive risk frameworks, ontologies, and mappings. These are important projects that need continued development. At Red Hat, our AI Safety team collaborates with, and contributes to, many of these upstream communities, and we'll continue to do so.

Additional enterprise challenges in AI safety

However, when you look at a typical enterprise organization, they have additional challenges beyond these tools. When you examine the process of onboarding a new, potentially custom built, AI agent there are wider stakeholders, assets, and processes that aren't touched by existing tools.

Onboarding a new agent typically follows this path:

• Policy development: Teams produce written policy documents encoding the organization's AI dos and don'ts, often referencing external regulation like the EU AI Act, and sometimes supplemented by use-case-specific constraints (for example, stricter rules for customer-facing agents).
• Risk extraction: Someone has to read those documents and identify which theoretical risks they contain.
• Risk triaging: Of those theoretical risks, someone has to decide which apply to this specific agent, and which are technical risks that can actually be tested.
• Scenario generation and red teaming: Technical teams must then build test scenarios for those risks and run them through existing red-teaming frameworks.
• Iteration: Results are then interpreted, guardrails or other fixes are applied, and the agent is retested until it's ready to deploy. If the process is manual, then this adds major overhead each time.

This process leads to slow approvals and a disconnected audit trail, making it hard to demonstrate to internal or external auditors what was actually done.

asago aims to solve these problems, collaboratively

The asago project aims to alleviate this pain. We want to help organizations get from their AI policies to production without needing to be experts in AI safety. Importantly, our aim isn't to replace existing AI safety tooling, but rather to act as an orchestration layer to join ecosystem components together. We'll integrate with the best tools where they exist, and fill in the gaps when they don’t.

Of course, this must be a collaborative process. We don’t believe that a single organization can or should do this alone. With the wide range of policy documents, model and agent types, deployment contexts, regulatory contexts, and more, a project like this needs a wide range of view points—diverse expertise is non-negotiable.

This is why we have excellent initial partners that represent technology companies, research institutions, community coalitions, and government organizations. Together we won't just write the code, we'll also make sure we’re asking and answering the right questions. And as a global community, we need to make sure we have sufficient linguistic and geographical representation. We’re launching with partners across US, UK, and Europe, and we strongly encourage more collaborators to join us to expand this coverage via the links below.

An evolving technical architecture

The technical architecture will evolve with the project, but here is our initial view. The process is split into 2 sections:

Figure 1: Blue nodes represent data. Yellow nodes represent functional components.

Policy document to scenarios

In the first section, risks are extracted from policy documents and mapped to the IBM Risk Atlas.These risks, indexed in a risk card, represent the whole range of theoretical risks that the policy touches on. Some of these will be technical risks that can be automated and addressed with asago. The main component for this is the policy mapper.

The next challenge is creating the right scenario to drive red teaming. This isn’t library-specific yet, but the work of the scenario generator is to assess the specifics of the agent in question, with respect to the identified risks, and to consider which type of tests and supporting context are needed. These form the scenario. At this point in the process we have a set of scenarios that can be evaluated for and defended against for the specific agent in the context of the organization's policies.

Figure 2: Blue nodes represent data. Yellow nodes represent functional components.

Iterative scenario to recommendations, via red teaming

In the second section, which is an iterative loop, we take those scenarios and generate run artifacts for the relevant red teaming and/or eval frameworks. This is where the data-driven approach starts. Popular eval and red teaming frameworks will be triggered (on EvalHub) with synthetically generated datasets and environments that address the scenarios.

The results of these are passed to a recommender process to provide candidate fixes for red teaming failures. We envision a flexible range of recommendations, starting with guardrails but likely evolving to further components across the stack. Importantly these components will be deployable (such as with Kubernetes custom resources (CRs) or config maps) and retestable.

It's important to note that this is just the initial architecture. As the community evolves, this architecture will evolve with it.

What can you do today?

While what we’ve announced today highlights the roadmap and pathway for the community, development is active and you can already start experimenting with our work. On the asago GitHub you can already check out the policy mapping work here, with examples here. You can also check out midojo, an open source framework for security testing AI agents against indirect prompt injection.  

Finally,  you and your organization can join us as collaborators. We need input from many organizations, particularly with representation across different languages, cultures, and geographies. Learn more and get involved here.

The best outcome for AI safety is one that no single company controls. asago is our contribution to that goal, with infrastructure built in the open by a community with genuinely diverse perspectives. The project is in its early stages, and there's a lot to build. And that's exactly why now is the best time to get involved.