The Full Potential Gap

“Can we get this data?”

It is one of the most familiar requests an analytics team will hear, and one of the hardest to answer. The instinct is to treat it as a technical question. Which system holds the data? How fast can it be queried? But what really determines whether data changes a decision is usually settled before anyone opens a query tool. Who is asking? What do they need to know? Can the system reliably produce the answer? Can the requester accurately interpret it?

These are not soft considerations around the “real” work of building dashboards; they are the work. A technically sound dashboard built on the wrong question, delivered to an unready audience, or measuring something undefined, will underperform even when every calculation is correct. None of this reflects poorly on the people asking or building; it reflects how young most organizations’ data practices still are.

Know Your Stakeholders 

A request for data almost always arrives compressed. An early-learning program director says they need “enrollment numbers” for their school, when what they really need is enough information to decide whether to open a new site, adjust outreach, or report progress to a funder. Each underlying need implies a different metric, time frame, and granularity, but the analytics team often receives only the compressed version.

Getting to the real need means treating stakeholder identification as its own step, not a formality. Who is asking? What decision will this inform? What would they do differently depending on the answer? A dashboard for a compliance report has different requirements than one built for weekly outreach decisions, even if both show “enrollment.” Skipping this step is how teams build technically impressive dashboards that are opened once and often never used again because they answer a question that was not fully relevant.

The other half of this work happens on the data side. Is the underlying system stable enough to support the request? Are the data extractable? Many organizations are still moving from spreadsheets toward centralized systems, and that transition is often uneven. A field may exist but go unpopulated, or a metric may depend on a pipeline that has not yet been built. 

Identifying these constraints early is what keeps a simple-sounding request from becoming a broken promise. An unstable pipeline is one of the most common reasons a dashboard gets used once at launch and then abandoned. The numbers shift as the system changes, and trust erodes before anyone identifies the pipeline as the real problem.

Match the Request to the Requester’s True Need

Even a well-built dashboard fails if it assumes a level of data literacy that an audience does not have. Two organizations can ask an almost identical question and need entirely different deliverables. For example, one may have managers who read trend lines daily, and the other may have staff who have never used more than a static spreadsheet. Delivering the same dashboard to both is a mismatch waiting to happen.

Assessing a need honestly, before deciding how sophisticated a first deliverable should be, is part of the job. A dashboard ahead of an organization’s technical capacity to act on is not a success. The American Institutes for Research’s guidance on launching early warning systems for struggling students begins with a team that has defined roles and responsibilities before it sets up the data tool and builds in steps for interpreting the data and assigning interventions.¹

This is also where urgency and dependability must be reconciled. Not every request needs the same turnaround, and not every turnaround is compatible with a defensible answer. A short conversation about how a number will be used and how much precision that requires protects trust more than any amount of dashboard polish.

The gap between a requester’s need and the deliverable produced to address that need is not unique to any one organization, and part of the explanation may lie in how little data education most students receive. A December 2024 analysis by Data Science 4 Everyone found that only 3% of U.S. K-12 students were exposed to data science education in 2022–2023.² Low readiness to interpret a dashboard is rarely personal; it reflects how little structured exposure most people have had to data before they are expected to act on it.

Get the Context and Definitions Right

Data without shared definitions invite misreading. For example, two people can look at the same “enrollment” figure for a school and take away two different impressions if definitions are not clearly communicated. One person may count a family as enrolled in a school at intake, while another person may count that family only after services begin. Presenting a number without specifically indicating what it represents risks people walking away with incorrect information.
 
This matters most when the reasons behind a pattern matter just as much as the pattern itself. A decline in a metric can reflect a genuine drop in need, a changed counting method, or a data gap caused by a new system going live. Presenting a number without acknowledging the potential for different interpretations risks creating a dashboard that gets used once, leads to the wrong conclusion, and is quietly abandoned for whatever informal method people already trust.

Organizations that do data analysis well invest early in documented governance, including written definitions, a record of when and why a measure changed, and a named owner for each data source. The National Center for Education Statistics’ Forum Guide to Data Governance is a useful reference here. Its central point, that governance is mainly about people, processes, and leadership support, rather than technology alone, generalizes well beyond schools.3

Early-Childhood and Human Services Data

In the context of early-childhood and human services, the problem of misunderstood data shows up most clearly when the word “impact” is used. A program can serve more families this year than last, which are data worth tracking, but a different question is whether those families are benefiting from the program. Enrollment growth measures reach; it does not measure whether a child’s development is being supported to its full potential once a family is enrolled.

Getting closer to that question requires measures working together. This involves determining whether a child has been assessed to identify which services they qualify for, whether those services actually reach the child, and whether the child’s progress is tracked across the developmental domains that matter for them, so gaps are identified rather than assumed away by an aggregate enrollment count. Reporting enrollment and utilization numbers side-by-side, and being explicit about which is which, gives leaders a far more accurate picture than either number alone. 

The same distinction holds for applications outside of early education; throughput and utilization are rarely the same measure. Child Trends’ STEP Forward with Data framework, funded by the Gates Foundation, treats enrollment as only one of six distinct system steps, separate from learning experiences that occur further along. This keeps reach and developmental impact as separate, tracked functions.⁴ A 2020 Brookings analysis by Daphna Bassok and Anna Markowitz found that early-childhood data have remained siloed across sectors, and that the most available measures, largely licensing and safety compliance, are poor proxies for the quality of programs that can predict child development.⁵ 

When a state requires rather than invites systemwide quality measurement, the difference is substantial. After Louisiana mandated statewide classroom-quality ratings in 2012, the share of programs rated proficient rose from 62% to 85%, and the child-care-to-school-based quality gap was cut in half.6 Voluntary, patchwork measurement rarely closes a gap, but comparable, well-defined measures can.

Recommendations

A few practices consistently separate analytics that get used from those that get admired and ignored: 

• Start every request by naming the specific decision it will inform and the stakeholder who owns it – not just the metric requested.
• Put metric definitions in writing before building anything.
• Honestly assess how prepared your audience is to interpret what you are about to hand them, and calibrate the first deliverable to that level rather than to what is technically possible.
• Confirm the underlying data are stable and extractable before committing to a timeline.
• When reporting growth, explicitly distinguish between how many people were served and what actually changed for them.

None of this argues against ambition. Education and human services organizations have real reasons to want faster answers and dashboards that keep pace with their decisions. But the sequence matters. Getting the question right and confirming that the data can support your answer enable more sophisticated work to eventually succeed. 

The organizations getting this right are increasingly asking a harder question — not just whether their systems can produce a number, but whether that number reflects real change for the people the program serves. Getting there starts with understanding what comes before the model.

References

1. Marken, A., Scala, J., Husby-Slater, M., Davis, G., 2020, Early Warning Intervention and Monitoring System Implementation Guide, American Institutes for Research, https://pathwaystoadultsuccess.org/wp-content/uploads/2021/04/EWIMS_Implementation-Guide_FINAL_July_2020.pdf. 
2. Data Science for Everyone, 2024, “State of the Field: Data Science and Data Literacy Education in K–12,” Center for RISC, University of Chicago, https://nces.ed.gov/use-work/resource-library/report/forum-guide/forum-guide-data-governance. 
3. National Center for Education Statistics, 2020, Forum Guide to Data Governance (NFES 2020-083), U.S. Department of Education, https://nces.ed.gov/forum/pub_2020083.asp.
4. Van-Kim Bui, L., Esposito Hackett, S., Epstein Richards, D., King, C., Bredeson, M., 2024, System Transformation for Equitable Preschools (STEP Forward with Data) Framework, Child Trends, https://www.childtrends.org/publications/system-transformation-equitable-preschools-step-forward-data-framework.
5. Bassok D., Markowitz, A. J., 2020, “The value of systemwide, high-quality data in early childhood education,” Brown Center Chalkboard, Brookings Institution, https://www.brookings.edu/articles/the-value-of-systemwide-high-quality-data-in-early-childhood-education.
6. Bassok and Markowitz, “The value of systemwide, high-quality data in early childhood education.” The authors report that the share of Louisiana programs scoring above the “Proficient” threshold on the Classroom Assessment Scoring System (CLASS) rose from 62% to 85%, and that the quality gap between childcare and school-based settings was cut in half.