Long-running surveys increasingly need to do more, such as provide estimates for small geographic areas, answer emerging policy questions, or adapt to changing response patterns, all while maintaining data quality. Strategic sample design makes these adaptations possible without compromising rigor. NORC has developed three distinct sample design solutions that help flagship surveys meet evolving demands while balancing cost, precision, geographic detail, and operational feasibility.

Using sampling theory, data collection expertise, empirical evidence, and simulation-based evaluation, we have helped decision-makers navigate complex tradeoffs to produce designs that meet each study’s goals.

NORC has tailored innovations to address distinct challenges facing three large, long-running surveys:

1. We redesigned the Medicare Current Beneficiary Survey to support state-level analysis while maintaining national performance.
2. Using statistical modeling of cost and precision to identify an efficient strategy for in-person nonresponse follow-up, NORC revamped how we conduct our longest-running survey, the General Social Survey (GSS).
3. For the National Survey of Early Care and Education (NSECE), NORC developed an innovative design that links households, providers, and workforce samples to provide nationally representative estimates and explore local-area supply and demand. 

In all these projects, NORC has delivered adaptive, rigorous, and customized survey solutions for stakeholders facing increasingly complex information needs. 

The MCBS has provided policymakers and researchers with vital information on the health care experiences of Medicare beneficiaries since 1991. As policy needs evolved, however, the survey’s longstanding design did not fully support emerging demands for state-level estimates and subgroup representation. NORC’s work on the MCBS illustrates a core institutional capability: helping longstanding, high-profile surveys adapt to new requirements and changing field conditions without compromising data quality. The redesign was initiated to solve two problems:

1. The original design did not statistically support state-level estimates, and some states were excluded from the sampling frame entirely.
2. Sustaining representation for key subpopulations became harder due to declining response rates.

NORC partnered with the Centers for Medicare & Medicaid Services (CMS) to develop a plan for robust state-level estimates for all 50 states and the District of Columbia while continuing to support national estimation and adequate representation of key age groups.

Meeting CMS’s goals required trade-offs:

• Balancing state-level representativeness against national-level precision
• Accepting slightly lower response rates through telephone data collection while strategically employing in-person interviewing where possible
• Adapting the data collection system to remain feasible within budget constraints

NORC addressed these challenges through a structured, multi-phase research program using our expertise in sample design and data collection to balance feasibility, representativeness, and precision. We tested multiple stratification strategies—ways of partitioning the population and drawing independent samples to achieve robust representation within subpopulations—creating a national sample drawn from the full Medicare Enrollment Database.

We found that reducing age stratification categories from seven to three improved state-level precision without sacrificing analytic value. The change also reduced variability while preserving representation of the youngest and oldest beneficiaries.

NORC quantitatively assessed several trade-offs in the design. State-level stratification modestly reduces national precision due to unequal sampling probabilities across states, but analyses show that national estimates and subgroup analyses remain statistically robust and well within accepted thresholds. By quantifying these impacts, NORC ensured that CMS could make informed, evidence-based decisions aligned with the MCBS’s goals.

The redesign eliminates geographic clustering and adopts state-by-age stratification to provide representativeness at the state level and for three age groups. Adjustments to weighting and variance estimation procedures ensure analytic continuity and preserve the utility of the MCBS for users by supporting continued use of standard estimation methods.

With the MCBS redesign, NORC modernized a flagship survey for new policy and research demands. By rigorously evaluating alternatives, quantifying tradeoffs, and aligning design choices with CMS priorities, NORC delivered a state-stratified framework that expands the survey’s utility, preserves national-level analysis opportunities, and remains feasible given the challenges of data collection.

Since 1972, NORC’s General Social Survey has monitored trends in opinions, attitudes, and behaviors in contemporary American society. The survey has continually evolved its data collection methods to maintain high-quality, nationally representative estimates while adapting to changing respondent preferences and field realities.

Historically, the GSS relied on in-person interviewing. Recent cycles have adopted sequential, mixed-mode designs, including enabling web response in addition to in-person to improve efficiency. In 2022 and 2024, the design included two experimental data collection approaches:

1. Web-first: starting with web recruitment, with in-person follow-up eight weeks later for a subsampling of nonrespondents
2. Face-to-face-first: starting with in-person contact, with web recruitment follow-up for all nonrespondents (Davern, 2024; Davern, 2025; Wells et al., 2024)

Based on growing evidence of the cost-effectiveness of web-based recruitment, the 2026 GSS design used only the web-first approach.

That design choice raises a methodological question about how large the in-person follow-up subsample should be. In-person follow-up is expensive but plays a critical role in ensuring representative coverage, particularly for individuals who are less likely to respond online.

There is a meaningful trade-off between choosing a higher or lower subsampling rate. Higher subsampling rates bring more nonrespondents into in-person follow-up, which improves the precision of survey estimates by reducing the weight variation created by subsampling adjustments. However, higher subsampling rates also increase cost. Lower subsampling rates reduce field costs but increase weight variation and create larger margins of error. The challenge is to identify a subsampling rate that achieves an efficient balance—protecting the quality of key estimates while controlling costs.

To inform this decision, the GSS team conducted research that combined various cost models and levels of analyzing precision. We first estimated costs under multiple data collection scenarios. These ranged from “all face-to-face” and “all web” to several web-first designs with different in-person subsampling rates, including 20 percent, 40 percent, and 60 percent, plus the 2024 mixed design.

We then built a linear model that shows how total costs change depending on the mix of interviews completed through web-only recruitment vs. field recruitment. We used that model to project costs across a wide range of subsampling rates.

For each subsampling rate, we paired predicted costs with corresponding measures of precision, calculating the unequal weighting effect (Kish, 1965) attributable to subsampling and the resulting changes in estimates’ variance and margins of error. We calculated the product of variance and cost to understand this tradeoff between precision and cost.

When we look at the relationship between precision and cost across subsampling rates in Figure 1, the curve shows a broad, relatively flat region. Precision improves as we subsample more respondents for in-person follow-up, but after a certain point, gains are small compared to the added cost.

For a web-first design aiming for 2,800 completed surveys, the best balance of precision and cost occurred at a higher subsampling rate. However, the curve stays fairly flat even at much lower subsampling rates, meaning lower rates can still provide good value. Importantly, the team found that these patterns held up even when they tested different assumptions, including different targets for completed surveys, eligibility, and web and field response rates.

This work shows how NORC guides sample design decisions in light of real-world constraints, using empirical evidence to deploy higher-cost methods precisely where they add meaningful value. For the 2026 GSS, the analysis supported selecting an average 30 percent subsampling rate for in-person nonrespondent follow-up, balancing representativeness and precision while sustaining the efficiency of web-first recruitment.

The National Survey of Early Care and Education collects data on families’ needs, preferences, and choices for non-parental care for children under age 13. It also gathers data on early care and education (ECE) providers’ offerings and services and the characteristics and practices of the ECE workforce.

The challenge is how to represent multiple distinct populations nationally while supporting policy-relevant local data analysis. To solve the problem, NORC designed coordinated samples across populations to enable examinations of how families, care providers, and workers are connected in local childcare markets.

NSECE’s sample design has to achieve two goals simultaneously:

• Produce nationally representative samples of households, home-based providers, center-based providers, and the workforce
• Coordinate those samples geographically to study the interrelationships at the local level where family demand, local workforce, and childcare facilities meet.

Coordinating four independently sampled, nationally representative surveys while preserving statistical validity and controlling data collection costs is a major methodological and operational challenge. A design that is efficient for one population may not work with the others, and a design that supports local linkage must be careful not to compromise national representativeness.

NORC addressed this through a design that uses provider clusters to create coordinated sampling geography across all four surveys. The 2024 NSECE first selected counties or county clusters as primary sampling units for the 50 states and D.C. NORC built secondary sampling units from either individual census tracts or clusters of adjacent census tracts from primary sampling units.

We built the household sampling frame from the U.S. Postal Service Computerized Delivery Sequence File supplemented by remote listing (English and Fiorio, 2025). We then used machine learning models to prioritize addresses likely to be eligible for the survey (Dutwin et al., 2024). 

NORC formed provider clusters that anchor local sampling for households and unlisted providers while defining broader catchment areas for listed providers (Wolter et al., 2010). Importantly, the provider cluster approach reflects how families actually search for child care. Figures 2A and 2B show two hypothetical NSECE provider clusters, first in a metropolitan area and then in a non-metropolitan area. The areas shown in the maps are only for illustration purposes and do not suggest whether the area is included in the sample.

In metropolitan settings (Figure 2A), a secondary sampling unit in yellow defines the core from which households and unlisted home-based providers are sampled, while providers are sampled from a surrounding two-mile radius catchment area circled in blue.

In non-metropolitan settings (Figure 2B), where travel distances are typically larger, the catchment area expands to incorporate tracts in green within a five-mile radius depicted by the green line. This wider catchment area ensures the provider sample aligns with realistic service areas. The result is four coordinated yet independently drawn samples that can be linked at the cluster level to study local dynamics.

Provider cluster sampling enables analyses connecting families’ needs and choices with provider availability, characteristics, and staffing. It supports cross-cutting questions such as whether local supply meets demand, how workforce conditions shape provider capacity, and how geographic accessibility differs across communities. The expanded catchment areas increased data collected from low-provider areas, facilitating enhanced analyses in these communities.

In addition, NSECE’s repeated cross-sectional and longitudinal components spanning 2012, 2019, COVID-era follow-ups, and 2024 provide multiple ways to examine change over time, including comparisons across waves and longitudinal analyses within selected cohorts.

With this NSECE sample design, NORC delivered innovative, integrated survey solutions that connect consumers, providers, and workers within a single framework. By coordinating multiple nationally representative samples, NORC enabled policy-relevant analysis of how demand, supply, and workforce conditions interact.

NORC’s innovations for these three projects are supporting data users’ evolving information needs with sample designs that are both methodologically rigorous and practical to implement. Our systematic approach to problem-solving allows us to tackle challenges ranging from expanding a survey to support state-level estimates to cost-effectively optimizing nonresponse follow-up to linking multiple populations to study a complex system. We started by defining the analytic objectives, identifying real-world constraints, and developing a fit-for-purpose design that delivers high-quality data.

Each project uses design features to target resources where they yield the greatest return in representativeness and statistical performance. Despite differences in target populations and operational contexts, the common thread is intentional design to adapt long-running studies to changing conditions.

We also combined statistical rigor, addressed operational realities, and innovated to provide policy-relevant insights. In the MCBS, that meant evolving a longstanding federal survey to support state-level inference. In the GSS, it meant using modeling to guide efficient deployment of in-person follow-up. In the NSECE, it meant coordinating samples across households, providers, and workforce populations to generate policy-relevant insight into how local early care systems function. NORC is designing adaptive, integrated, and high-impact data collection strategies for increasingly complex research environments.