Curated Data, Not a Better Model: What Building CMMC AI Taught Us About AI-Native SAFe

Any company can call the same large language model we use. If two teams build similar AI assistants on the same model, what makes one of them better?

While working through the AI-Native SAFe® material, two ideas stood out because we had lived them while building our product, CMMC AI: curated data is the real advantage, and outcomes matter more than output. This article shares how those ideas, and a few others, shaped what we built, with real decisions from the project and an honest account of what we did not do.

What we built

CMMC AI helps small defense contractors complete their annual CMMC Level 1 self-assessment. It walks a team through the 17 Level 1 practices, drafts security policies from their answers, organizes the evidence an assessor will ask for, and answers questions with the practices it relied on cited underneath. It runs on AWS and is sold through AWS Marketplace.

The customers are small businesses, often without a compliance team, that need to get an annual obligation right. That shaped every decision below.

Lesson 1: Curated data is the advantage, not the model

We could have tried to make CMMC AI stand out by choosing a cleverer model or writing longer prompts. Instead, most of our effort went into the knowledge the model works from.

• A structured practice registry. Each of the 17 Level 1 practices is captured with its requirement, its assessment objectives and what an assessor looks for, maintained as reviewed, versioned content rather than scraped text.
• An evidence map. We defined 25 evidence items and linked each one to the practices it supports, so the product can tell a customer exactly which evidence closes which gap.
• The customer's own context. Answers draw on the customer's recorded assessment as well as the requirements, and each customer's data is kept separate from every other customer's.
• Retrieval with citations. The assistant retrieves the relevant practices before answering and lists the practices it relied on, so a reader can check the source instead of trusting the answer blindly.

The model itself is replaceable. We call it through a standard interface and can move it to another provider without changing the product. The curated registry, the evidence map and the customer context are what another team would have to rebuild, and that is where the value sits.

Lesson 2: Outcomes over output

We defined the outcome as a small contractor completes its annual Level 1 self-assessment with confidence. Measured against that, several decisions became easy, and some meant doing less:

• Finish one journey before adding features. The evidence binder had to work end to end, from upload to the practice it supports, before we built anything new. A half-working feature is output; a customer finishing a task is an outcome.
• Remove what does not serve the outcome. We had provisioned a dedicated search service for the assistant. When we found a simpler approach inside our existing database served customers just as well, we removed it, saving roughly $175–350 a month for a small business product.
• Let customers actually buy. Listing on AWS Marketplace meant buyers could subscribe through their existing AWS account. Getting that path working and approved mattered more than another screen.
• Do not claim what customers cannot yet use. We left CMMC Level 2 off the product until there is demand and a secure environment for it, even though it would have looked bigger on a pricing page.

Lesson 3: Build quality and security in, and keep learning

For a compliance product, security is part of the value. A tool that helps companies meet security requirements has to meet them itself.

• Security from the start: multi-factor sign-in required for every user, encryption, and each customer's data isolated from the others.
• One source of truth: every change goes through version control and an automated pipeline that previews infrastructure changes before they are applied. Early on, quick fixes made directly in the cloud console kept being overwritten by later deployments. Moving everything into the pipeline ended that.
• No stored keys: the pipeline signs in to AWS with short-lived credentials instead of long-lived access keys.
• A learning log: we recorded every significant problem with its root cause and fix. More than thirty entries later, it is the most useful document we have, because the same class of mistake rarely happens twice.

Lesson 4: People stay accountable

AI speeds the work up; it does not take responsibility for it. Compliance decisions carry legal weight, so we designed the product so that people stay in charge:

• Every AI-drafted policy is labeled a draft for review, not an approved policy and not evidence of compliance.
• The assistant offers guidance, not a compliance determination, and shows the practices behind each answer so the reader can verify it.
• The final judgment, and the annual affirmation, remain with the company's own leadership, where the regulation places them.

What we did not do

We are a small team, and we did not adopt AI-Native SAFe as a full operating model. We did not run formal program-level planning events or set up large-scale portfolio practices. What we applied were its ideas at the scale of one product: invest in curated data, define the outcome first, build quality in, and keep people accountable. We share this as a practical example, not as a complete implementation.

Takeaways for leaders

• Your data is your differentiator. Before choosing a model, decide what knowledge you will curate, how it is reviewed and versioned, and how answers will cite it.
• Write the outcome down first. Then use it to decide what to stop building as much as what to start.
• Treat security and quality as features, especially in regulated work.
• Keep people accountable for decisions that carry legal or business weight.

If you are applying AI in a regulated environment and want to compare notes, we would like to hear from you.