Methods and Data Sources

The conceptual framework provides a model to link Archstone Foundation activities and other work on systems integration across California with longer-term health and well-being outcomes among older adults and caregivers. This framework was developed from several activities:

• Environmental Scan. Conducted in late 2022, the scan leveraged literature and internet searches, along with reference mining. It focused on published reports, white papers, publicly available briefings and presentations, master/multisector plans on aging, and peer-reviewed literature. The RAND team focused on sources from the past decade, but the majority of selected resources were from the past three to five years, reflecting growing attention to the need to address social determinants of health and related social needs in the context of health care, as well as developments in technology (e.g., interoperability standards) that can facilitate integration. Using these sources, the team identified existing frameworks, evidence syntheses, best practices, and calls to action that addressed the integration of health care and social services.
• Interviews. To gain greater insight regarding (1) Archstone Foundation’s strategy and theory of change, (2) potential data and metrics to include in the Action Framework, and (3) how Archstone grants have addressed the integration of health care and social services, the RAND team conducted semi-structured interviews with Foundation staff and leadership and a subset of recent Foundation grantees.
• Expert Advisory Board. The RAND team convened experts who met multiple times to provide feedback and guide the development of the conceptual framework and the Action Framework. Advisory board members included advocates with lived experience, researchers, and leaders of coalitions and state and national organizations focused on older individuals or caregivers. We thank members of the Advisory Board who contributed their expertise:
  • Alexander Fajardo, MCP, CFC; Executive Director at El Sol Neighborhood Educational Center
  • Lisa Gibbs, MD; Interim Chair, Department of Family Medicine; Chief, Division of Geriatric Medicine and Gerontology; Medical Director, UCI Health Population Health; Chair, Ronald Reagan Endowed Chair of Geriatrics
  • Sei Lee, MD MAS; Professor of Medicine, UCSF Geriatrics Director/Senior Physician Scholar, VA Quality Scholars Fellowship
  • David Lown, MD
  • Didier Trinh, Director of Policy and Advocacy, Diverse Elders Coalition
  • Neil Wenger, MD, MPH; Professor in the Division of General Internal Medicine and Health Services Research at the University of California, Los Angeles; Adjunct Senior Scientist, RAND
  • Phyllis Willis, DSW: Director Aging Programs South Division at Watts Labor Community Action Committee
• Board of Directors. The Foundation's Board of Directors provided feedback throughout the development of the Action Framework via individual and group sessions. The RAND team also provided semi-annual presentations for discussion and feedback.
• Expert Consultation. The RAND team conducted early feedback sessions with colleagues at RAND who have expertise in health care quality, systems integration, older adults, caregiver well-being, and long-term care. The team also engaged leadership and staff from the California Dashboard on Aging to review and provide feedback on the Action Framework.

With the framework finalized, the team identified potential data sources and measures seeking those that broadly captured relevant constructs and avoiding those that were too specific for the purpose of the Action Framework

• Identification of potential data sources. The team first compiled a list of potential data sources from those that were identified during the environmental scan or were suggested in interviews and expert consultation. Keeping the vision of the Integration and Impact Roadmap in mind, the team set the following requirements:
  • Data sources were publicly available or readily available with appropriate data-use agreements. Data sources that required a purchase of a license were excluded.
  • Data sources were California-specific or enabled California-specific data to be abstracted. (e.g., national-level datasets without state identifiers were excluded).
  • The source provided the ability to disaggregate by characteristics, where possible (e.g., race, ethnicity, gender)
  • The source offered longitudinal data (e.g., availability of prior year data) and anticipated the continued collection of relevant data into the future to facilitate tracking of trends.
  • The source had measures that were relevant to conceptual framework constructs, older adults, or caregivers.
• Identification of measures. Team members scanned the data sources for potential measures of inclusion. The final list of measures is in the table below. Information about each measure was abstracted, including the definition, source, level of geographic disaggregation possible; whether data could be stratified by race, ethnicity, gender, urban/rural, or other characteristics; years of data available; frequency of data collection or fielding; and alignment to conceptual framework constructs. Two team members independently reviewed and rated measures for potential inclusion and measures were discussed by the full team. Those rated highly by both raters were included with team consensus. When possible, the team used data collected from 2013 to 2023.

In some cases, the sample sizes of subgroups were very small, precluding accurate estimates. If the denominator of a given subgroup was 20 or less, a note was included under the corresponding figure to indicate that there were insufficient data to display on the graph. The team also calculated the margin of error (MOE) for each data point using the following equation:

The MOE provides an assessment of the estimate’s precision: Higher MOEs mean that there is less confidence in the estimate (e.g., due to low sample size). For easier interpretation, MOEs are color coded on corresponding graphs:

• MOEs less than or equal to five percentage points are green, indicating the acceptable range and higher confidence in the estimate.
• MOEs greater than five and less than or equal to ten percentage points are yellow, indicating a borderline range and some confidence in the estimate.
• MOEs greater than ten and less than or equal to 15 percentage points are orange, indicating a borderline range that some consider outside the acceptable range and less confidence in the estimate.
• MOEs greater than 15 percentage points are red, indicating a generally unacceptable range with less confidence in the estimate. These were retained however to facilitate a general comparison across subgroups.

Where applicable, the number of older adults endorsing a given outcome was calculated by multiplying the weighted percentage by 9 million, which is the estimated number of older adults in California in 2022, according to CHIS.

To forecast trends and quantify potential for improvement, the team plotted available annual data for each measure. The team assumed a linear relationship over time and extrapolated values following that slope through 2030 via simple regression analysis. For measures with only two data points, the team did not fit a regression line but instead predicted that the value for 2030 would be an average of the two available years.

The team then set an accelerated action value for 2030, based on the predicted value in 2030. Based on discussions with Archstone, in most cases, the team set this as a 20 percent improvement in the predicted value of each measure in 2030. However, for predicted values above 83.4 percent, a 20 percent increase would yield an accelerated action value of more than 100 percent. Because of this, and to account for the potential of a ceiling effect, the team set the accelerated action to 10 percent higher than the predicted value when the predicted value in 2030 was between 60 percent and 80 percent, and 5 percent higher when the predicted value in 2030 was between 80 percent and 95 percent. There were no indicators with predicted values over 95 percent. Of note: a 5 percent increase might result in an increase of less than 5 percentage points depending on the value that is being increased. For example, a 5 percent increase of 80 percent will result in a percentage point increase of 4 (i.e., 80 * .05 = 4).

To translate accelerated action into an estimated number of people who would experience an improved outcome should the improvement be achieved, we multiplied the projected and accelerated action percentages in 2030 by 10,301,464—the projected population of adults age 60 and older in 2030, as estimated in the Master Plan for Aging Dashboard as of April 2024 (Index – Master Plan for Aging (ca.gov)) and calculated the difference.

The team calculated changes in disparities for every potential pairwise comparison between demographic groups, using the earliest date included in the Action Framework for that measure and the most recent date. Disparities were calculated at these two points in time by taking the difference between two subgroups and assessing the extent to which that difference increased or decreased over time. Reductions in disparities are not always good news, however. For example, if two populations (A and B) had a 10 percent disparity five years ago, a reduction in disparity would be observed if population B experienced an improvement in health to more closely align with the better health of population A (good news), but a reduction in disparity could also be observed if population A experienced a worsening in health to more closely align with the poorer health of population B (bad news). Therefore, the visuals included in this Action Framework, both the trends over time and disparities, were designed to display not only the change in disparity over time, but the trends of the two groups being compared to facilitate a more robust understanding of this change.

The search for relevant data sources and measures exposed several challenges related to the availability of data to optimally track progress on the integration of health and social services and the resulting impact on the health and well-being of older adults and caregivers. The key challenges were:

• Limited publicly available data. Other data exist but are proprietary or exist behind paywalls, precluding use or reuse for public good.
• Small sample sizes that preclude statistical precision. This is of particular concern for certain demographic categories among older adults or the ability to examine intersectionality (e.g., the experience of persons who are Black or African American + female).
• Challenges with continuity of data sources and measures over time. In some cases, potential measures of interest were only captured at one point in time or were no longer being collected by the survey, precluding an ability to track changes over time.
• Lack of information on care team composition and workflow. While we can capture policies and payment mechanisms that support expanded care teams, there is limited visibility into whether and how providers are leveraging these practices.
• Fragmentation of care settings and payors and associated data. This precludes a robust picture of whether and how health and social needs have been fully met among older adults in California and their caregivers.
• Inability to limit the scope of data to California and older adults and caregivers. Even when national data were available, in many cases it was not feasible to isolate data on older adults and their caregivers in California.

We define terms used in data displays below.

Race: adapted from Census 2000 Definition (American Indian or Alaskan Native, Asian, Black or African American, Other Single Race, White, More than One Race)

Ethnicity: OMB/Current California Department of Finance Race-Ethnicity Definition (Hispanic; Non-Hispanic White; Non-Hispanic; Black or African American; Not Hispanic; American Indian or Alaskan Native; Non-Hispanic; Asian; Non-Hispanic; Other Single Race or More than One Race)

Language: Language spoken at home (English only, Language(s) other than English)

Sex: Self-Reported Gender (Male, Female)

Income: Household income divided by the relevant Federal Poverty Level (FPL) based on household size and multiplied by 100 (0-99% FPL, 100-199% FPL, 200-299% FPL, 300% FPL and above)

Federal Poverty Level: Income level determined by the U.S. Department of Health and Human Services’ poverty guidelines, updated annually, and based on the size of the household

Urban/rural: 2-level Claritas definition, based on zip code¹.

Urban: Areas with a population density of 1,000 or more people per square mile (includes urban and suburban categories in the six-level Claritas system)

Rural: Areas with a population density of less than 1,000 people per square mile (includes Second City, Town and Rural, Farm, and Ranch categories in the six level Claritas system)

1. Claritas LLC. Assessing the role of urbanicity. 2018. Available at https://nhts.ornl.gov/assets/Assessing_the_Role_of_Urbanicity.pdf Accessed February 26, 2024.

Please email RAND project leaders: Jason Etchegaray, Ph.D., or Laurie Martin, Sc.D., MPH, at actionframework@rand.org.