Burke Introduces FAR Framework to Evaluate Synthetic Data Quality

Burke, Inc. has released new research on the reliability of synthetic data for business decision-making and introduced the FAR Framework, a method for evaluating synthetic data quality and decision reliability, announced in a press release.

The study compared multiple synthetic data approaches, including large language model generated panels and generative data models. It found that LLM based synthetic panels can assist in early exploration but are unreliable for quantitative decision making, producing false conclusions in about 60 percent of tested business scenarios at an 80 percent accuracy level. Generative data models based on validated human data performed better in preserving research conclusions.

Burke's FAR Framework measures synthetic data quality across three criteria: Fidelity, which assesses alignment with the source of truth; Authenticity, which examines whether synthetic responses show realistic variation; and Resolution, which checks whether variable relationships and conclusions are maintained.

The research, conducted by Burke Labs, also identified a threshold for decision reliability where synthetic approaches are more likely to maintain accurate outcomes. The company stated that the findings aim to help organizations understand the strengths, limits, and best uses of different synthetic data methods.