The biggest problem with synthetic data isn't that it's synthetic. It's that no one seems to mean the same thing when they use the term. That may sound like a minor issue, but in research, terminology matters. We spend an extraordinary amount of time defining populations, methodologies, confidence levels, weighting procedures, and analytical approaches because precision in language leads to precision in understanding.
When we become imprecise with our terminology, we create opportunities for misunderstanding, unrealistic expectations, and, ultimately, poor decision-making.
Nor is it an argument that synthetic data has no place in research. There are legitimate and growing applications for many of these technologies, and I suspect they will continue to improve over time. My concern is much simpler than that. We have adopted a single phrase to describe a growing collection of approaches that are fundamentally different from one another, making it more difficult for researchers, technology providers, and clients to understand whether they're actually talking about the same thing. Perhaps the easiest way to explain my concern is through an analogy. Consider the word organic. Whether you're shopping for produce, dairy products, or packaged foods, the term has a generally accepted meaning. In the United States, products marketed as organic must meet established standards before they can carry that label. Consumers may not know every requirement, but they understand that the word represents a defined set of practices and expectations. The label communicates something meaningful.
Ask five researchers, software vendors, or AI companies to define the term, and you may receive five different answers. One organization may use it to describe statistically generated datasets designed to preserve the characteristics of sensitive data while protecting respondent privacy. Another may use it to describe AI-generated survey respondents that answer questionnaires based on patterns learned from existing data. Others use the term to describe agent-based simulations, simulated populations, or other AI- and model-driven approaches. In some conversations, you'll may even hear the term extended to discussions involving digital twins and similar modeling techniques. Whether every one of those examples technically falls under the definition of synthetic data isn't really my point. My point is that the same phrase is increasingly being used to describe all of them. As a result, the term itself has become so broad that, on its own, it communicates very little.
In some cases we're generating entirely new datasets. In others we're generating respondents. In others we're simulating behaviors, populations, or future scenarios. They rely on different methodologies, require different validation approaches, and are designed to solve different problems. While computational modeling plays a role in each of them, they differ substantially in purpose, methodology, validation, and intended application. Methodology has always mattered in our profession because it determines what conclusions can reasonably be drawn from the results. We don't simply tell a client that we "conducted research." We explain how participants were recruited, how data was collected, how results were analyzed, what assumptions were made, and what limitations should be considered when interpreting the findings. Those distinctions provide the context necessary to evaluate the quality, applicability, and limitations of the research.
Imagine you're evaluating proposals from three research suppliers. You ask each whether they use synthetic data, and each answers yes.
• The first generates privacy-preserving datasets that preserve the statistical characteristics of an original dataset so analysts can work with realistic information without exposing personally identifiable information.
• The second uses a large language model to generate responses from simulated procurement professionals during early-stage concept testing.
• The third builds simulations to explore how changing market conditions might influence customer behavior over time.
In practice, however, they are offering very different capabilities. So my concern isn't with synthetic data itself. It's with the way we've begun using the term. Synthetic data has become a convenient shorthand for a growing collection of AI- and model-driven approaches that are related in some ways but fundamentally different in others. The more broadly we apply the label, the less useful it becomes.
Those aren't minor distinctions. They fundamentally change what the resulting information represents, how it should be interpreted, and the types of decisions it is appropriate to inform. The phrase synthetic data answers none of those questions. At best, it tells us that some form of computational modeling was involved somewhere in the process. The consequence is that conversations about synthetic data often become conversations about entirely different approaches without anyone realizing they're talking about different things. One person may be discussing privacy-preserving synthetic datasets while another is thinking about AI-generated respondents. Someone else may be referring to simulations or population modeling. Each of those approaches may have value, but it's difficult to have meaningful conversations about their strengths, limitations, or appropriate use cases when we're not even talking about the same methodology.
Artificial intelligence is already changing research in meaningful ways, and I believe many of these technologies will become increasingly valuable components of the researcher's toolkit. Privacy-preserving synthetic datasets can allow organizations to collaborate and analyze information while protecting sensitive data. Simulation models can help evaluate scenarios that would be difficult or, in some cases, impossible to test in the real world. AI-generated respondents may prove useful in certain exploratory contexts, provided their strengths, limitations, and intended applications are well understood and clearly communicated. Each of these approaches deserves to be evaluated on its own merits rather than grouped together under a single label. Research has always depended on precision, and that precision extends beyond the way we design studies or analyze results. It extends to the language we use to describe our work. Our terminology should help people understand what was done, not force them to guess. If synthetic data can refer to a privacy-preserving dataset, an AI-generated respondent, a simulated population, an agent-based model, or any number of other approaches, then the term has become too broad to be meaningful on its own.
The problem is that we've started using one phrase to describe a growing collection of fundamentally different approaches. Those approaches may share certain characteristics, but they are not interchangeable, nor should they be evaluated as though they are. As AI continues to introduce new tools and techniques into research, our vocabulary should evolve alongside it. Rather than relying on a catchall phrase, we should describe these approaches for what they actually are and what they are designed to do. Otherwise, we risk having conversations about synthetic data without realizing we're talking about entirely different things.
Contact: Ariane Claire, Research Director, myCLEARopinion Insights Hub
A1: Because research conversations involve multiple parties, and a term only works if everyone interprets it the same way.
The risk isn't that any one person is wrong. It's that two people can agree they're discussing "synthetic data" while picturing fundamentally different things.
A2: Useful shorthand still has to communicate something. This one increasingly doesn't.
The problem isn't shorthand itself. It's shorthand that has stretched so far it no longer narrows down what was actually done.
A3: They are genuinely different, not variations of a single methodology.
Computational modeling plays a role in all of them, but they diverge substantially in purpose, validation, and intended application. They are not interchangeable.
A4: Because methodology determines what conclusions can reasonably be drawn from the results.
Whether a dataset was recreated for privacy, generated by a language model, or simulated under market conditions changes what the information represents and which decisions it can responsibly inform.
A5: No — the concern is about terminology, not the technology.
The technology isn't the problem. The problem is using a single phrase to describe a growing collection of fundamentally different approaches, then evaluating them as if they were the same thing.