Building Synthetic Populations You Can Actually Trust

How we turn national survey data into thousands of distinct, statistically grounded synthetic people.

By Chretien Li

The fastest way to build a synthetic population is to ask a language model to improvise a diverse group of people. It's also the least trustworthy way — a shortcut that fails silently, producing confident-sounding output that quietly reflects whatever biases and blind spots the model happened to absorb in training, with no way to know how far off it is until it's already informed a real decision.

Doing it properly means solving two problems that have almost nothing to do with each other, and getting both right at once.

This article walks through how we approach both problems at a high level: the intuitions, the main strategies, and a few of the underlying mathematical ideas illustrated with simple examples. We're deliberately staying above the implementation details — the goal is to explain why the pipeline is built this way, not to hand over a blueprint.

Stage one of the pipeline — building a representative population — breaks down into four moves: gather real-world data, wire it into a variable registry of distributions and dependency edges, sample from that registry, and calibrate the result back to target. The rest of this section walks through each in turn.

Start from real numbers, not vibes

The foundation of every synthetic population we build is a set of marginal distributions — the share of a real population that falls into each category of a given variable — pulled from credible, citable sources. We draw heavily on the workhorse surveys that social scientists and government agencies have run for decades: the Census Bureau's American Community Survey (ACS), the General Social Survey (GSS), Pew Research and Gallup for attitudes and media habits, the CDC's NHANES/BRFSS for health, the American National Election Studies (ANES) for political behavior, the Federal Reserve's Survey of Consumer Finances (SCF), and comparable trade and industry data for consumer behavior — among many others. Age brackets, income bands, education levels, political affiliation, health conditions — each variable is grounded in a cited, dated source rather than an assumption.

This matters more than it sounds. It's easy to generate a population that “feels” realistic to a human reviewer while being systematically wrong in ways that only show up statistically — for instance, overrepresenting college graduates because they're overrepresented in the training data an LLM has seen. Anchoring every variable to a real, sourced distribution is what prevents that kind of quiet drift.

One thing worth calling out: this doesn't lock us into a fixed set of public surveys. The underlying pipeline is fully data-driven — a population is defined by its probability specification, not by hard-coded logic — so we can just as readily build a model around a client's proprietary research, a niche subpopulation public surveys don't cover well, or a non-US market, and it flows through the exact same sampling, calibration, and testing machinery described below.

This is what populates the variable registry — the pipeline's term for the full set of sourced marginals waiting to be sampled from.

The harder problem: variables don't move independently

Getting each variable's marginal right isn't enough. In the real world, variables are entangled: education correlates with income, income correlates with health outcomes, political identity correlates with dozens of downstream attitudes. If you sample every variable independently, you get a population where the marginals look correct in isolation but the combinations are nonsensical — for example, a synthetic population might end up with roughly the right number of people who are both highly educated and struggling financially, but pair them up randomly rather than in a real-world-correlated way.

So on top of the marginal distributions, we encode a network of dependency relationships — evidence-backed statements like “having attribute A roughly doubles the odds of also having attribute B,” sourced from the same kind of literature (regression coefficients, odds ratios from published studies, conditional breakdowns from survey crosstabs). Each one only gets included if there's real evidence behind it — we don't invent correlations to make the population feel more textured.

These dependency relationships get wired into the same variable registry alongside the marginals — which is why sampling from it later automatically respects both at once.

At this point, what we have isn't a population yet — it's a fully specified probability model: thousands of variables, each with a real-world marginal, wired together by hundreds of evidence-backed dependency relationships. The next step is turning that model into actual people.

With the variable registry built, stage one's remaining two moves are sampling and calibration — turning that registry into an actual population, then reconciling it back to target.

Sampling: turning a probability network into individuals

The probability model itself doesn't know or care how many people we're about to generate — it's the same network of numbers whether we're about to produce a pilot batch of 500 profiles or a full-scale study of half a million. Generating a population means walking that network once per synthetic person.

For each individual, variables are drawn in a deliberate order — demographic traits before the attitudes and behaviors they tend to shape — so that by the time we reach, say, a political or health-related variable, that person's relevant demographic picture is already decided and the right dependency relationships can be applied to it. This ordering (a topological sort, for anyone familiar with the term) is really just a formal way of encoding “which things plausibly cause which other things” so generation respects that direction rather than working backwards.

At each step, drawing a value is a simple, well-established technique. Line up a variable's possible categories along a 0-to-1 number line, sized proportionally to their (by-now parent-adjusted) probabilities; generate a random number; see which segment it falls into. It's the same logic as a weighted spinner — except every synthetic person effectively gets their own custom-weighted spinner, reshaped moment to moment by everything already decided about them. Because each person's spinner is built from a distribution that was itself derived from the real marginal and nudged by real dependency relationships, the population respects both the individual marginals and the correlation structure between variables, by construction — not as an afterthought.

Run that once per variable, in order, and you get one profile. Run it N times — off a seeded random number generator, so a given run is exactly reproducible — and you have a population of whatever size the use case calls for, generated from the identical underlying probability model each time.

Reconciling the small gaps — calibration

Here's a subtlety that trips people up: once you start layering dependency effects onto a population, the individual variable marginals can drift slightly away from their targets. If enough dependency edges push in the same direction, a category that should represent 25% of the population might land at 23% or 27% purely from interaction effects compounding across thousands of variables.

We fix this with a calibration step — an iterative reweighting process (in the statistics literature this general family of technique is often called “raking” or iterative proportional fitting) that nudges the population's weights until every marginal lines up with its target again, while disturbing the correlational structure we just built in as little as possible. It's a bit like adjusting a recipe by nudging one ingredient at a time until it matches the target flavor profile, without a single large correction that undoes everything else.

How do we know it worked?

Every population we generate gets scored against its own targets using a few complementary metrics:

None of these numbers are ends in themselves — they're diagnostics that tell us where a population needs another pass before it's usable for anything downstream.

That completes stage one. What comes out the other end is a synthetic population — a statistically sound table of rows — but, as the diagram's second stage makes clear, a row is not yet a person.

With Problem One solved, we turn to Problem Two — stage two of the pipeline, where rows become people. A statistically faithful table of demographics is necessary, but it's not a person. This part of the pipeline is about giving each row enough interior texture — beliefs, history, temperament, everyday habits — that a language model can inhabit it convincingly, and then making sure that inhabited persona behaves the way an actual person would when questioned.

Layering psychological texture onto demographics

Demographic and attitudinal variables answer what someone is. They don't answer who someone is. So each profile goes through what the diagram calls persona enrichment — additional layers drawn from a structured, curated bank of material rather than left to free-form generation:

Because these elements are selected from a fixed, categorized set rather than generated freely, every trait a synthetic person “has” can be traced back to where it came from — which matters both for quality control and for being able to explain a persona's behavior after the fact.

Catching contradictions before they matter

Randomly assembling traits, beliefs, and memories from independent pools occasionally produces combinations that are internally inconsistent — not merely unusual, but logically or temporally impossible (imagine a memory that presupposes a job the person's profile says they never had). Before a profile is finalized, it goes through a cohesion pass: a review step that looks specifically for hard contradictions and strips them out.

The important design choice here is what this step is not allowed to do: flag a combination just because it's statistically unusual or doesn't match a stereotype. A profile that pairs an unexpected trait with an unexpected background isn't a bug — real people are full of combinations a demographic model wouldn't predict. The cohesion pass is scoped narrowly to genuine impossibilities, specifically to avoid quietly sanding every persona down into a demographic cliché.

Getting the model to actually answer as the person

The final stage-two step — what the diagram calls LLM embodiment — is prompting the underlying language model to respond as the assembled persona rather than as a generic assistant reflecting on it from the outside. This is arguably the hardest part of the whole system, and it's worth explaining why, even though we won't detail exactly how we solve it.

Language models are heavily trained to be balanced, to hedge appropriately, and to give socially agreeable answers. Those are good qualities for an assistant. They're the wrong qualities for a stand-in for a real, opinionated, occasionally inconsistent human being. Left to its own devices, a model asked to “respond as this person” tends to drift back into something recognizably assistant-shaped: it summarizes both sides, softens strong opinions, and answers in a tone that reflects an AI trying to sound like someone — rather than sounding like that person's actual first reaction.

What we're pushing for instead is a response with a specific set of properties: unhedged, in the moment, occasionally unflattering, shaped by the persona's temperament rather than by a model's instinct to be helpful and fair to every viewpoint. Getting there consistently — across thousands of distinct personas, on topics that range from mundane to contentious — is enough of a craft problem that we treat prompt design here as one of our core areas of work, and we don't detail the mechanics further in this piece.

What we can say is that this step doesn't get to grade its own homework. Whether it's actually working is answered downstream, empirically, in Part 4: the consistency tests check whether a persona sounds like the same person call after call, and the tone-and-voice review checks whether their manner of answering — terse or rambling, guarded or forthcoming — actually tracks the temperament they were assigned, not just the topic-level opinion.

The diagram's bottom band isn't decorative — “results loop back to refine both the population model and the persona layer” is a literal description of how this section works. Testing isn't a one-time gate at the end; findings from these tests feed back into the variable registry, the enrichment banks, and the cohesion rules that produced them.

Building a plausible-sounding persona is the easy 80%. The harder, more important work is testing whether these synthetic people behave the way real people in that demographic and psychographic slice actually would — and we test for that along several distinct dimensions.

Every test in Part 4 checks a persona against something we already wrote into their profile. That's necessary, but it's a low bar — a model can pass recall testing by doing little more than echoing the prompt back. There's a harder, more meaningful bar sitting underneath it.

Psychometrics has a name for that harder bar: a trait's nomological network — whether a measured trait has lawful, predictable relationships to things you never directly measured, not just consistency on the items used to define it in the first place. A personality trait that's real should show up in places you didn't explicitly encode it. One that's just decoration won't.

Internally, we call the version of this we test for out-of-scope testing. Starting from a persona's explicit attributes, we ask about topics that are related but never stated, then keep moving outward to progressively more distant ones — mapping how far a coherent, textured understanding of that person actually extends.

Take a small-business-owner persona. Nothing in their profile mentions cash flow or fuel prices, but a persona with real texture should show some implicit sensitivity to both when asked. Push further out — toward a general point of view on small-business policy — and a well-built persona still holds a plausible, non-generic stance, rather than flattening into a generic answer the moment the question steps outside its authored script.

That's the shape of the test. The harder part is scoring it fairly. A modern language model already carries plenty of generic world knowledge on its own — so a persona sounding informed about fuel costs doesn't automatically mean the pipeline is doing anything. To isolate that, every out-of-scope result is scored against the same question asked of a stripped-down, demographics-only version of the same person. What we credit to the pipeline is the lift over that baseline, not the base model's own general competence — a dimension of testing we don't see many others in this space investing in.

We opened by saying why we don't take the shortcut. Here's what doing it properly buys you instead: every variable grounded in cited, dated survey data; every correlation backed by real evidence rather than invented for texture; every synthetic person tested against the same real-world benchmarks used to build the population in the first place. That's what closes the gap a shortcut leaves wide open.

None of this is a claim that synthetic populations are a perfect substitute for talking to real people. They aren't, and we don't market them that way. It's a claim that if you're going to use synthetic populations at all, they should be built and tested with the same rigor you'd expect from any statistical model that's going to inform a real decision. That's the standard we hold this pipeline to at Heura Lab.