There’s a version of “good research design” that looks really solid when you’re reviewing it on paper. The audience is cleanly defined, the quotas are balanced, every subgroup is accounted for, and everything feels intentional and controlled in a way that signals rigor.
The audience is cleanly defined, the quotas are balanced, every subgroup is accounted for, and everything feels intentional and controlled in a way that signals rigor. It gives the impression that the structure itself is what’s going to ensure the quality of the data.
And the moment it doesn’t, it stops improving the data and starts distorting it. This shows up most clearly in industries where the workforce distribution doesn’t match what someone expects or wants it to look like. The skilled trades are probably the most obvious example. You’ll see age splits that assume a much younger workforce than is realistically there, or caps placed on more experienced professionals as if that segment needs to be controlled rather than understood. In other cases, diversity targets are applied in ways that don’t align with how certain roles or segments are currently structured, especially in more specialized or regional parts of the market.
The intent is almost always to create a dataset that feels more balanced, more inclusive, and more representative overall. But intent doesn’t change the fact that you can only be representative of what actually exists. Once quotas move beyond that, they’re no longer correcting imbalance, they’re introducing a different kind of it. Where this becomes an issue is in how those quotas get executed. The project still needs to be completed, the targets still need to be hit, and the timeline doesn’t shift just because the structure is misaligned with the audience. So the pressure moves into how the sample is actually sourced and qualified. Definitions start to stretch slightly to make someone fit. Targeting broadens just enough to keep things moving. In some cases, a very small subset of respondents ends up carrying a disproportionate amount of the sample simply because they meet all the criteria.
The quotas are filled, the dataset is complete, and nothing in the output immediately signals that anything is off. But what you’ve actually built is something that reflects the structure you imposed, not the market you were trying to understand. That distinction matters more than it seems, because the insights that come out of that data are going to follow the same pattern. Decisions end up being made based on an audience that has been shaped to fit a framework, rather than one that reflects how that audience actually exists in the real world.
When they’re grounded in reality, they can be useful for preventing overrepresentation and ensuring that certain groups aren’t disproportionately driving the results. But they’re not a mechanism for fixing a mismatch between expectation and actual workforce composition. And when they’re used that way, they don’t solve the problem. They just make it harder to see.
Contact: Ariane Claire, Research Director, myCLEARopinion Insights Hub
A1: No — stricter quotas can't fix a structural mismatch between your design and the market you're trying to reach.
Quotas are a tool for managing overrepresentation, not for correcting a framework that was built on faulty assumptions about how the market exists.
A2: Usually through friction — the sample starts requiring workarounds that wouldn't be necessary if the design were grounded in how the market actually looks.
None of these flags show up in the final dataset. By the time the data is in the table, the adjustments have already been made and the quotas are filled. That's what makes it easy to miss.
A3: The data looks fine — that's the problem. The distortion is already baked in before anyone sees the results.
The numbers will be clean. The crosstabs will balance. Nothing will look obviously wrong. But the foundation shifted early, and every insight built on top of it carries that same drift.