Validity is the strength of the conclusions, inferences or propositions from a study. It is concerned with the study’s success at measuring what the researchers are aiming to measure. There are several types of validity. They are:
- Construct validity;
- Content validity;
- Criterion-related validity;
- External validity; and
- Internal validity
Validity of instrument
- Construct validity is the degree to which an instrument measures a theoretical concept (or construct) under investigation. Content validity refers to the degree to which the content of the items reflects the content domain of interest.
- Criterion-related validity, also referred to as instrument validity, is the degree to which scores on an instrument are correlated with some relevant criterion that has demonstrated validity. There are two types of criterion related validity:
- Concurrent validity is the degree to which an instrument can distinguish individuals who differ on some other criterion measured or observed at the same time.
- Predictive validity is the degree to which an instrument can predict some criterion observed at a future time.
Validity of research design
- External validity is the degree to which the results can be generalized to settings or samples other than the ones studied.
- For example, Health Plan A studied and reported on outcomes related to the frequency of foot exams for diabetics. The members recruited for study volunteered for participation in the plan’s diabetes management program and are representative of only 3% of Health Plan A’s entire diabetic population; therefore, the conclusions cannot be generalized to the whole.
- Internal validity is the degree to which it can be inferred that the manipulated or controlled activity (independent variable) rather than uncontrolled, extraneous factors is responsible for the observed change (dependent variable).
- For example, internal validity may be threatened by confounding factors (e.g., change in benefit design) or regression to the mean in Health Plan A’s reported results that hospital admissions declined significantly for its members enrolled in the congestive heart failure program. Upon analysis of the research design, members included in the study were enrolled based on high preintervention hospital admission rates and not the plan’s total congestive heart failure member population.
Cook, T.D. and Campbell, D.T. (1979). Quasi-Experimentation: Design and Analysis for Field Settings. Rand McNally, Chicago, Illinois.
Beck, C., Hungler, B., and Polit, D. Essentials of Nursing Research, 5th ed. Baltimore: Lippincott Williams & Wilkins Publishers (2003).
Carmines, E., and Zeller, R. Reliability and Validity Assessment. Newbury Park: Sage Publications (1991).
In population health management, variable selection can have two meanings:
- For evaluation, the selection of independent or dependent variables, depending on the research question.
- For predictive modeling, the choice of independent or causal variables.
Variable selection is a process that a modeler can use to determine the best model that fits (or predicts) the functional form between the dependent variable and the independent variables. This process measures a user-defined criterion such as R2, adjusted R2, root-mean-square error and compares models in an iterative fashion, ultimately coming up with the optimized model.
There are several methods that can be used in model selection for modeling: stepwise, backward selection and/or forward selection.
See predictive models and study design.
Variable transformation is the process of changing either the dependent or the independent variables using a functional form to improve the fit of the model.
In linear regression, the dependent variable may be transformed to make its functional form more consistent with the independent variables, providing a closer fit to the underlying normal distribution. A common transformation in this case is computing the natural logarithm of costs and using this as the dependent variable.
Value-Based Benefit Design (VBBD)
Value-based benefit design is the explicit use of plan incentives to encourage enrollee adoption of one or more of the following:
- appropriate use of high value services, including certain prescription drugs and preventive services;
- adoption of healthy lifestyles, such as smoking cessation or increased physical activity; and
- use of high performance providers who adhere to evidence-based treatment guidelines.
Enrollee incentives can include rewards, reduced premium share, adjustments to deductible and co-pay levels, and contributions to fund-based plans, such as Health Savings Accounts. Examples include increasing medication compliance rates, encouraging use of preventative services, encouraging use of health management programs, and promoting high performing providers.
DIFFERENTIATING VBBD FROM CDHD AND VALUE-BASED PURCHASING
VBBD differs from what is commonly referred to as a consumer-driven health plan (CDHP) in a fundamental way. In a CDHP, perhaps more appropriately called a high deductible plan, the enrollee is responsible for the cost of services subject to the deductible. With heightened cost awareness and incentives not to spend wastefully, it is the enrollee’s responsibility to determine what is of value. In a VBBD, the value proposition is integrated into the incentive structure. A VBBD plan does not have the potential risks associated with a CDHD plan – the risk of deferring needed services to either avoid paying the full cost of the services or to build the balance in a tax deferred account.
National Business Coalition on Health. Value-Based Benefit Design: A Purchaser Guide. Washington, DC: NBCH, 2009. http://www.sph.umich.edu/vbidcenter/publications/pdfs/VBBDPurchaserGuide%5B1%5D.pdf