Celebrating 25 years of DDD's Excellence and Social Impact. โœ–

Artificial Intelligence

Data Flywheel Strategy: How Production Feedback Loops Compound Model Quality Over Time

Data Flywheel Strategy: How Production Feedback Loops Compound Model Quality Over Time

Kevin Sahotsky

A data flywheel is a self-reinforcing loop: production usage generates data, that data improves the model, the improved model attracts more usage, and more usage generates more data. The idea has been around under different names for years, and in AI right now it is the difference between a model that was good on the day it shipped and a system that keeps getting better without anyone rewriting the architecture. Most teams describe wanting one. Few have actually built the mechanism that makes one turn.

MIT’s research on enterprise GenAI adoption found much the same failure pattern from the other direction: 95 percent of organizations in the study reported no measurable return from their GenAI pilots, and the core barrier the authors identify is learning, since most GenAI systems do not retain feedback, adapt to context, or improve over time.ย 

Meanwhile, Gartner’s tracking of the same problem found that at least half of generative AI projects were abandoned after proof of concept by the end of 2025. The gap between a flywheel that spins and one that does not is rarely the model. It is whether anyone built the loop.

This blog covers what a data flywheel actually requires: the four stages that make the loop close instead of leak, the data engineering underneath each stage, the failure modes that quietly stop a flywheel from turning even when the dashboards look fine, and how to tell a real flywheel from a demo that merely used the word.

Key Takeaways

  • A flywheel is a pipeline, not a metaphor. Production usage compounds model quality only when interactions are captured, triaged, labeled, and fed back into training and evaluation as an operating system, not as an occasional cleanup project.
  • The loop has four stages, and most programs build one or two. Capture, curation, retraining, and evaluation each have their own failure modes, and skipping any one of them stops the compounding effect even if the other three look healthy.
  • Not all feedback is signal. Production interactions include noise, edge cases nobody wants the model to learn from, and outright errors, and a flywheel without a curation stage learns the noise as confidently as the signal.
  • Evaluation is what proves the loop is actually compounding. Without a held-out, refreshed evaluation set, a team cannot tell a genuinely improving model from one that has simply drifted toward its own recent outputs.
  • A flywheel is only as fast as the labeling behind it. Every stage of the loop, from capture and curation to retraining feedback and evaluation, ultimately runs on human-labeled data, so the speed and quality of that annotation layer set the pace for the whole system.

Why Most GenAI Programs Never Get a Flywheel Spinning

The demo version of a flywheel is easy to describe and easy to fake: point to a chatbot that logs conversations and call the logging a feedback loop. What actually distinguishes a compounding system from a static one is much narrower. It is whether production data that reveals a gap (a wrong answer, a missed edge case, a user correction) gets systematically captured, judged, and turned into a labeled example that changes what the model does next. Most GenAI programs capture the data. Far fewer do anything structured with it. An unused log is not a flywheel. It is an archive.

This is also why the failure mode is so easy to miss internally. A program can have real production traffic, a real logging pipeline, and a real retraining schedule, and still not have a flywheel. None of those things guarantee that the specific interactions worth learning from are being found, labeled, and routed back in. The absence is invisible from the dashboard. It shows up as a model that has been in production for a year and performs almost exactly like it did on day one.

The Four Stages of a Working Flywheel

Stage 1: Capture

Every production interaction that could plausibly reveal a model gap needs to be captured with enough context to judge later: the input, the output, any user correction or explicit feedback, and the surrounding session state. The common failure here is capturing outcomes without context (a thumbs-down with no record of what was asked or why the answer was wrong). That produces a signal nobody can act on months later, once volume makes manual recall impossible.

Stage 2: Curation

Not every captured interaction belongs in the next training run. Curation is the triage step: separating genuine model failures from user error, deduplicating near-identical cases so the loop does not overweight whatever happened to be common that week, and routing ambiguous cases to human review rather than auto-including them. A flywheel without curation does not fail gracefully. It learns its own noise as confidently as it learns real signal. The model quietly gets worse in ways that look like improvement in aggregate metrics.

Stage 3: Retraining and Fine-Tuning

Curated examples feed back into training, whether that is full retraining, targeted fine-tuning, or updates to a retrieval corpus for a RAG system. The operational requirement most programs underbuild here is versioning: which corpus version produced which model version, so that if a retraining cycle makes something worse, the regression can be traced to a specific batch of feedback data rather than triggering a guess-and-rollback cycle across the whole pipeline.

Stage 4: Evaluation

This is the stage that proves the loop is compounding rather than just churning. A held-out evaluation set, refreshed on a cadence and kept separate from anything that entered training, is what lets a team measure whether the retrained model actually improved on the cases that mattered, rather than assuming improvement because the team did work. Without this stage, a flywheel can spin for a year while quietly drifting toward the model’s own recent output patterns rather than toward genuine quality gains, and nobody notices until a metric that was never being tracked finally breaks.

The Curation Problem Is the One Everyone Underestimates

Ask most teams running a flywheel what determines their retraining data, and the honest answer is often volume and recency: whatever came in since the last cycle, roughly deduplicated. That policy sounds plausible and quietly degrades a model over time, because production traffic is not a representative sample of what the model needs to learn. It overrepresents common, easy cases and underrepresents the rare, hard ones that actually move quality. A flywheel that trains proportionally to traffic volume learns to be very good at the easy majority and stays exactly as bad at the consequential minority.

The fix is a curation policy that deliberately overweights informative failures relative to their raw frequency: cases with explicit user corrections, cases flagged by automated quality checks, and a sampled slice of edge cases even when nothing flagged them. That sampled slice matters. It keeps the loop testing itself against the traffic it has not yet mastered, rather than only the traffic it already has.

How Digital Divide Data Can Help

Whether a program builds this loop internally or with a partner, the same four stages decide whether it compounds: disciplined capture, curation that separates signal from noise, versioned feedback into retraining, and evaluation that proves the improvement is real. Producing those is the work we do.

Capture and curation at production scale. Data collection and curation triages production interactions against defined criteria, so retraining data reflects what the model needs to learn rather than just what happened to come through that week.

The labeling layer every stage depends on. AI data preparation turns flagged interactions into structured, versioned training examples, with the provenance record that lets a regression get traced back to a specific batch.

Evaluation that proves the loop is working. Model evaluation services build and refresh the held-out sets that separate genuine compounding quality gains from drift toward the model’s own recent outputs.

If your next question is which of the four stages, capture, curation, retraining feedback, or evaluation, your current loop is actually missing, that’s the assessment we run. Talk to an expert.

Conclusion

A data flywheel is not a property a system acquires by being in production long enough. It is a pipeline someone has to build, with four stages that each have a specific job and a specific failure mode, and the compounding effect only shows up when all four are actually working together. Capture without curation teaches the model noise. Curation without evaluation cannot prove anything improved. Retraining without versioning cannot diagnose a regression when one happens.

The test for whether a program has a real flywheel or a well-instrumented static system: pull the last three retraining cycles and ask what specifically changed in the model’s behavior as a result of each one, measured against a held-out set that predates all three. If that answer is fast and specific, the loop is turning. If it takes weeks to reconstruct, or the honest answer is that nobody measured it, the flywheel is a metaphor the team has been using for a system that is not actually compounding.

References

Agent-in-the-Loop: A Data Flywheel for Continuous Improvement in LLM-Based Customer Support. (2025). arXiv. https://arxiv.org/abs/2510.06674

Challapally, A., Pease, C., Raskar, R., & Chari, P. (2025). The GenAI divide: State of AI in business 2025 (preliminary findings). MIT Project NANDA. https://nanda.media.mit.edu/ai_report_2025.pdf

Gartner. (2026). Why half of GenAI projects fail: Avoid these 5 common mistakes. https://www.gartner.com/en/articles/genai-project-failure

Frequently Asked Questions

Q1. We retrain our model monthly on new production data. Isn’t that already a flywheel?

A retraining schedule is one stage of a flywheel, not the whole thing, and the missing stages are usually curation and evaluation. Retraining monthly on undifferentiated new data trains proportionally to whatever traffic happened to arrive, which overrepresents easy, common cases and underrepresents the rare, hard ones that actually move quality. And without a held-out evaluation set that predates the new data, there’s no way to confirm the monthly retrain is actually improving anything rather than just moving the model. A useful test: can you show, for your last retrain, exactly what got better and what got worse, measured against a fixed evaluation set? If not, you have a retraining schedule, not a flywheel.

Q2. How much production data do we need before a flywheel starts producing real improvements?

Less than volume alone suggests, because the flywheel’s effectiveness depends on the informativeness of what gets curated in, not the raw count of interactions. A narrow, well-instrumented workflow with disciplined capture and curation can show measurable improvement within a few retraining cycles on a modest volume of genuinely informative examples. A high-volume, uncurated flywheel can run for a year and show nothing, because volume without curation just means more noise at the same signal-to-noise ratio. Start narrow: pick one workflow, instrument it fully, and prove the loop compounds there before scaling the same discipline to more of the product.

Q3. Should the same team that builds the model also curate the flywheel’s feedback data?

They should be closely connected but not solely responsible for the judgment calls, because curation at scale is a distinct discipline from model development, closer to annotation program management than to machine learning engineering. The model team should define what counts as a genuine failure worth learning from and review the curation guidelines, but the actual triage, separating real model gaps from user error, deduplicating, routing ambiguous cases, benefits from the same calibration discipline any annotation program needs: written guidelines, measured agreement, and a defined escalation path. Model teams that try to do this triage themselves alongside their core work tend to under-invest in it precisely because it competes for the same hours as model development.

Q4. What’s the biggest sign that a flywheel has stopped compounding even though it’s still technically running?

Flat or declining performance on a fixed, held-out evaluation set across multiple retraining cycles, while aggregate production metrics look stable or even improve. That divergence is the signature of a flywheel that has started optimizing for its own recent output distribution rather than for genuine quality: production metrics can look fine because the model has gotten good at handling the traffic pattern it already sees a lot of, while the evaluation set, if it is genuinely held out and periodically refreshed with new hard cases, keeps testing the model against what it has not yet learned. A flywheel that only improves on data resembling its own recent history has stopped compounding and started echoing.

Q5. Is a data flywheel worth building for a low-traffic, high-stakes application, or is the volume too low?

Volume matters less than informativeness for high-stakes applications, and these are often exactly the cases where a flywheel earns its cost fastest, because each individual failure carries more consequence and more signal. The adaptation is on the capture and curation side: with lower volume, capture should default to broader inclusion, since there is less traffic to be selective within, and curation should lean more heavily on domain-expert review rather than automated triage, since the cost of learning from a wrongly curated example is proportionally higher. The four-stage structure holds regardless of volume; what changes is how much automation versus expert judgment each stage uses.

Data Flywheel Strategy: How Production Feedback Loops Compound Model Quality Over Time Read Post ยป

AI Training Data

How to Build AI Training Data That Survives Regulatory Scrutiny

Documenting the origin of training data is moving from a good practice to a formal expectation, and in the EU it is now written into law. Since 2 August 2025, providers of general-purpose AI models placed on the EU market have been required to publish a public summary of their training content using a template published by the European Commission, and models already on the market before that date have until 2 August 2027.ย 

This blog shares what we see teams doing in practice around consent and provenance: the difference between a lawful basis for personal data and a rights basis for copyrighted or licensed content, the kind of documentation many teams keep per source before a model ships, and how organizations approach building a training corpus that holds up when a regulator, an enterprise buyer, or their own legal team starts asking questions.

Key Takeaways

  • Documentation is now an enforceable requirement, not a best practice. The EU AI Act’s training-data summary obligation for general-purpose models is live law with an active enforcement date, and it sits alongside a parallel data-governance requirement for high-risk systems.
  • Consent and licensing are not the same question. Personal data needs a lawful basis to process, typically consent or a defined legitimate interest. Copyrighted or licensed works need a rights basis: a license, a purchase, or a public domain or open-license status. Treating the two as one checkbox misses half the exposure.
  • A provenance record is a per-source obligation, not a corpus-level summary. Acquisition method, license or consent basis, date, and chain of custody need to exist for each source category, because that is the granularity any regulatory filing, procurement review, or internal audit actually asks for.
  • Documentation built after the fact is worth less than documentation built during sourcing. Recreating provenance retroactively from a corpus that already exists is expensive, incomplete, and unconvincing to a skeptical reviewer in a way that contemporaneous records are not.
  • Clear, provable sourcing is becoming a practical baseline. As documentation expectations spread from regulation into ordinary procurement, sourcing data with a well-documented basis is increasingly the simpler path.

Why Documentation Became a Live Requirement

The EU AI Act’s Article 53 obligation is worth understanding in some detail, because its structure offers a useful preview of where AI documentation expectations are heading. Providers of general-purpose AI models are required to publish a summary of their training content, using a standardized template the European Commission published. The template covers data modalities, source categories such as public datasets, licensed private data, and web-scraped content, and how the provider addresses copyright and data protection obligations.ย 

What makes this structurally important, beyond any one requirement, is what it establishes as normal. A summary of training content, organized by source category, with a rights and consent basis attached, is now a standard artifact that a regulator can request and a template that already defines what belongs in it. High-risk systems face a parallel, more detailed obligation under Article 10: documented governance of training, validation, and testing data, including examination for possible biases. Two different obligations, aimed at two different categories of AI systems, converging on the same underlying ask: show your work on where the data came from and what gives you the right to use it.

Enterprise buyers have started asking the same question independent of any regulatory requirement, because their own exposure depends on their vendors’ answers. A procurement review that used to stop at a security questionnaire increasingly includes a data provenance questionnaire, and a vendor without a good answer is a vendor whose risk is now the buyer’s risk too. The trend is the same whether the pressure comes from a regulator, a customer, or your own board: documentation moved from a differentiator to a baseline expectation.

How Teams Approach Consent for Training Data

Personal Data Needs a Lawful Basis, Not Just a License

When training or fine-tuning data includes personal information (names, behavior, images of identifiable people, customer interactions), licensing content does not resolve the exposure on its own. Frameworks such as the GDPR (the EU’s General Data Protection Regulation) require a lawful basis for processing personal data, and consent is only one of several available bases, each with different conditions.ย 

Consent for AI training generally needs to be informed about that use. A person who agreed to a privacy policy written before generative AI existed may not have consented to having their data used to train a model, and privacy regulators have increasingly raised questions about whether older consents extend to AI training. In practice, the teams we work with aim for a documented basis, tied to the specific use, for every source that contains personal data.

Copyrighted and Licensed Works Need Provenance You Can Prove

For creative, published, or proprietary content, the operative question is rights, not privacy. A work is usable when it is owned, licensed on terms that cover AI training, or genuinely in the public domain or under a license that permits the use. The rights question and the privacy question are independent: a dataset can clear one and fail the other, which is why a provenance program needs both checks running for every source, not a single approval step that assumes one covers the other.

The Provenance Record That Survives Scrutiny

A provenance record that would hold up under scrutiny answers the same handful of questions for every source category in a training corpus, not for the corpus as a whole. What is the source, and how was it acquired: purchased, licensed, scraped under what terms, collected directly with what consent flow, or received from a vendor. What is the legal basis: a specific license with its terms, a documented consent basis and its scope, a public domain or open license designation with its citation, or a legitimate-interest assessment for personal data. When was it acquired, and under what version of the source’s terms, since terms change and the version in effect at acquisition is what governs. Who is accountable for the record, so that when a question arrives it has an owner rather than a search. And what is the chain of custody from acquisition through any transformation to its use in a specific model version, since a model trained on a corpus that later gets partially remediated needs to know which version it actually learned from.

Building a Training Corpus That Survives Scrutiny

The organizations getting this right treat provenance as a sourcing-time discipline rather than a filing-time reconstruction project. Sourcing categories get defined and approved in advance: owned content, licensed content with AI-training terms confirmed, public domain and open-licensed content with citation, and directly collected content with a consent flow built for the actual use. Every acquisition decision then maps to a pre-approved category instead of a judgment call made under deadline pressure.ย 

Anything outside those categories escalates before it enters the corpus, not after. Sampling audits check that the documentation for a random slice of the corpus actually exists and actually supports the claimed category, on a cadence, rather than assuming the intake process worked. And version-level custody tracks which corpus version trained which model, so that a provenance question discovered later can be scoped to what it actually affects instead of calling every model into question at once.

How Digital Divide Data Can Help

Whether a program builds this discipline internally or with a partner, the same three things decide whether the corpus survives scrutiny: sourcing that maps to pre-approved categories, documentation built at acquisition time rather than reconstructed later, and a consent or rights basis that’s specific to AI training use. Producing those is the work we do.

Sourcing you can document. Data collection and curation runs against defined sourcing categories with the license, consent, or ownership basis captured at the point of acquisition, not assembled afterward under a deadline.

Provenance that holds up per source, not per corpus. AI data preparation maintains the acquisition method, basis, date, and chain of custody at the granularity a regulatory filing or procurement review actually asks for.

Delivery that tracks corpus versions to model versions. Data engineering for AI keeps the version-level custody chain intact, so a provenance question about one dataset doesn’t become a question about every model that was ever near it.

If your next question is which sources in your current training data you could document on 24 hours’ notice, that’s the assessment we run. Talk to an expert.

Conclusion

The trend we see across AI regulation and across the procurement conversations enterprise buyers are starting to have is consistent: the question is no longer just whether a model performs well, but whether the organization that built it can show, source by source, where the data came from and what gave them the right to use it.ย 

That is a workable standard to build a data program around: documented sourcing, a clear consent or rights basis per source, and records built while the data was acquired rather than reconstructed once someone asks. The test is simple to state and hard to fake.ย 

Can you produce, for any source in your training corpus, what it is, how it was acquired, and what gave you the right to use it? If that answer takes weeks to assemble, the gap is usually not the paperwork. It is that the discipline was never built into sourcing in the first place.

References

European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Articles 10 and 53. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689

European Commission. (2025). Explanatory notice and template for the public summary of training content for general-purpose AI models. https://digital-strategy.ec.europa.eu/en/library/explanatory-notice-and-template-public-summary-training-content-general-purpose-ai-models

European Commission. (2025, July 24). Commission presents template for general-purpose AI model providers to summarise the data used to train their model [Press release]. https://digital-strategy.ec.europa.eu/en/news/commission-presents-template-general-purpose-ai-model-providers-summarise-data-used-train-their

Frequently Asked Questions

Q1. We only operate in the US. Does the EU’s training-data summary requirement affect us?

Possibly, and the determination depends on where your models are placed on the market or put into service, which is a question for counsel rather than a blog post. What’s worth knowing going into that conversation: the obligation attaches to making a general-purpose model available in the EU, not to where the company is headquartered, so a US-based provider serving EU customers or users can fall in scope regardless of domestic law. Even for organizations genuinely outside its reach, the template the Commission published is becoming a reference format that enterprise buyers and other regulators borrow from, which means the documentation habit it requires is worth building regardless of where the legal obligation formally applies.

Q2. How is this different from the data privacy compliance work we already do?

Overlapping but not identical, and the gap is often where AI training exposure tends to hide. Existing privacy compliance typically covers processing personal data for the purposes stated when it was collected, such as customer service, transactions, and account management. AI training is often a new purpose that was not contemplated in the original consent or legitimate-interest basis, which means an existing compliance program may be sound for its original purpose and still leave the training use uncovered. A practical step many teams take is a purpose-specific review with their legal team: for any personal data entering a training set, confirm the basis covers this use, not just some use.

Q3. What about content we scraped from public websites years ago, before any of this regulation existed?

Age does not resolve the exposure, and in some ways it can compound it, since older scraping was often done with less documentation of what was taken and which site terms applied at the time. A practical path many teams follow is retrospective triage rather than a blanket assumption either way: inventory what was scraped and when, check with counsel whether the source’s terms at that time restricted the use, and separate sources with a clear basis from sources without one. Content that cannot clear that review is usually prioritized for remediation or removal, with the models trained on it tracked so the exposure is scoped rather than diffuse.

Q4. Our training data includes third-party vendor datasets. Whose responsibility is the provenance record?

Contractually the vendor’s, practically yours as well, because your model’s exposure doesn’t stop at the contract boundary. The workable structure is flow-down: contractual warranties from the vendor about how their data was acquired and what rights they’re conveying, backed by an actual right to audit rather than a warranty alone. A vendor’s promise is only as good as your ability to verify it. In our experience, vendor data that arrives without documentation, or with documentation the vendor will not let you inspect, is safer to treat as undocumented for your own risk purposes, regardless of what the contract says, because in a review you are likely to be the one asked to produce the record.

Q5. How far back do we need to go to document data we’ve already trained models on?

Many teams prioritize by exposure rather than chronology. They start with sources feeding models currently in production or customer-facing, since that is where a provenance gap has the most immediate consequence, then work backward through models still being actively developed. Deprecated models trained on undocumented data usually carry lower urgency, since the practical risk is tied to continued use. For any source where documentation cannot be reconstructed, common options include remediation, replacing that source with a documented alternative, or an explicit, recorded risk acceptance agreed with counsel, rather than leaving the gap silent. A gap that has been identified and assessed is a very different position from a gap nobody has looked for yet.

Disclaimer

This article is provided for general informational purposes only and does not constitute legal advice. Regulatory requirements, timelines, and their application to your organization depend on your specific circumstances and may change over time.ย 

How to Build AI Training Data That Survives Regulatory Scrutiny Read Post ยป

Scroll to Top