SDOH AND SOCIAL CHANGE

Term

Definition*

Appraisal Considerations

AMSTAR II

A critical appraisal tool for systematic reviews (Shea, 2017)

The use of this instrument by authors is an indication they used a formal, well-established approach to their review

Affiliation

A formal link between an author and one or more organizations or groups that often provide support or recognition.

Affiliations may help the EBP team to determine if an author or team member has relevant training and professional standing. If not explicitly listed in a report, the team can do an internet search of a person’s name for more information.

Analysis

The systematic processes to describe, summarize, or evaluate data to create greater meaning through description and evaluation.

Authors should provide very clear and explicit information on the process they used to interpret their data, including what software was used. For quantitative analysis, this should also include statistical calculations. For qualitative analysis, this should include the process to code narrative data and generate themes, including how many people performed each step.

Attrition

The loss of participants during the course of a study, which can affect the validity and reliability of study outcomes.

Some loss to follow-up in a study is normal, but if those dropping out aren’t comparable to those remaining in, this can generate results that may not represent the truth of the subject of study. It is important to report attrition, as well as how this may have affected study results.

Bias

An influence that produces a distortion or error and results in the systematic alteration from the truth (McDonagh et al., 2013).

Biases can cause the findings from studies or reviews to not accurately reflect the truth. There are many types of biases, and it is the responsibility of study teams and reviewers to make efforts to mitigate them and include these efforts in their report. Of note, the terms “quality assessment” and “bias assessment” are often used interchangeably but do not mean the same thing. Quality assessment looks at the inclusion of safeguards to minimize bias and bias assessment evaluates the effectiveness of those safeguards (Furuya-Kanamori et al., 2021; Banzi et al., 2018)

Case-control study design

A type of epidemiological study design that compares two groups, people with an outcome of interest (cases) and a similar group without the outcome (controls) and looks back (retrospectively) into their lives to examine is the cases are more likely than the control to have been exposed to a risk factor (Polit & Beck, 2021)

This is a common type of observational study when a disease or condition is rare, or it would be unethical to expose a group to a risk factor (e.g. cigarette smoking). In these studies, it is important that both groups are similar other than the outcome of interest and there are measures taken to minimize recall bias since they are looking into people’s historic behaviors and data.

Causation

A relationship where one event is the result of the other's occurrence; more than correlation, causation indicates a direct effect.

EBP teams should ensure statements about causation are fully supported and authors are not implying causation when correlation (two things are related, but one doesn’t necessarily cause the other) is more appropriate. Causation is usually established with randomized control trials, and sometimes quasi-experimental studies.

Term

Definition*

Appraisal Considerations

Certainty/ confidence (level of)

A rating or assessment of how assured reviewers are in the body of evidence or their specific recommendations. This is usually based on data analysis and a quality or bias evaluation.

Different reviews use different approaches to establishing levels of certainty or confidence. The authors should explicitly state which approach they used and the level of certainty or confidence in each recommendation or outcome. They are sometimes expressed as “high to low” or with letters “A, B, or C.”

Clinical Practice Guidelines (CPGs)

Reports that generate recommendations on a specific healthcare topic based on rigorous collection of data, analyses, and processes to achieve consensus by a group of experts.

All CPGs are not created equal. EBP teams should look carefully at the methodology of a CPG (either provided in the document itself or on the organization’s website) to ensure it meets all necessary standards.

Conflict of interest

A situation in which a person or affiliation might compromise professional judgment or integrity due to a potential for personal gain.

All conflicts of interest should be disclosed by authors and considered when assessing information from a report or study. For example, if an author is employed by a company that produces the product a study is endorsing, the team should keep this in mind when reading and interpreting the findings.

Confounding

A situation in a scientific study where the effect or association between an independent and dependent variable is distorted by another factor.

EBP teams should look for study teams’ efforts to reduce confounding. This can include matching among groups, randomization and using statistics to control for different factors.

Congruency

The alignment of each of the parts of a study (aims, methods, results, discussion, and conclusions).

EBP teams should ensure that study teams have used and reported methods that adequately address their aims, all data introduced in the methods is reported in the results, all results have associated methods, and conclusions are based on those results. This helps establish the study was well done and all data is accounted for.

Control

The standard to which comparisons are made in a study. Often refers to a group of subjects that does not receive the intervention or treatment being tested.

Control groups should be similar to the group receiving an intervention. Exact similarities will depend on the nature of the intervention (e.g. sex, age, medical history). Keep in mind, control groups do not necessarily receive no intervention, they may the standard of care or a placebo intervention. This helps control for things like time spent with a member of the study team (e.g. an orientation to the hospital vs the intervention of disease process education) or the expectation of a positive result (e.g. a sugar pill vs the intervention of an antidepressant).

Correlation

Relationship(s) between variables that indicate an association, but not that one is the result of the other

Studies that investigate correlational relationships observe things that are happening naturally and use statistical calculations to describe negative and positive relationships between two or more variables. They are useful in situations where conducting an experiment is not possible (e.g. the area where a person grows up and their highest education level achieved). Epidemiologic studies such as case-control and cohort studies are examples of correlational studies.

Term

Definition*

Appraisal Considerations

Credibility

A component of trustworthiness. The confidence that findings and conclusions of a qualitative study represent the truth.

Study teams can increase credibility in both how they conduct the study and demonstrate it in their report by keeping details records, accounting for personal biases, data triangulation, including rich descriptions, transparency in data processing and interpretation, and respondent validation. This term speaks to the same idea as “internal validity” in quantitative studies (Noble & Smith, 2015).

Cross-sectional study design

A type of observational study that analyzes data from a population at a specific point in time.

Cross-sectional study designs typically collect data with surveys, observations, and sometimes secondary data analysis. It is often used to assess the prevalence of phenomena or current conditions within a particular population. It does not introduce an intervention but rather describes a phenomenon that is occurring naturally.

Current

Recent, occurring, or existing in the present time (Merriam-Webster)

The concept of “current” is subjective and the EBP team should determine what is a reasonable timeline for their topic at hand. Additionally, the inclusion of older literature on a topic should not necessarily be seen as a sign that a literature summary is not current, but rather it may be referring to foundational information on a subject (see seminal literature).

Data collection

The formal process for gathering information for analysis

Data collection should be explicitly and clearly described. This includes details of the tool(s) used, how the data was recorded (e.g. electronically, paper survey), and where that data was collated for future analysis. Data collection tool descriptions should include the number and types of questions and specific metrics gathered (e.g. blood pressure, Likert-scale feedback, open-ended questions).

Data pooling

The process of combining information from multiple studies or sources to allow for new statistical calculations that can increase the power and generalizability of results

This is a common technique when combining information from multiple studies in a systematic review with meta-analysis. To pool data, studies need to have similar populations, designs and analyses, and metrics (i.e. homogeneity).

Descriptive studies

A type of observational study designed primarily to describe the nature or status of the situation as it occurs naturally

Descriptive studies describe characteristics of a population or phenomenon using observational methods such as surveys, prevalence, and incidence data. It does not involve relationships between variables; instead, it aims to create a picture of a variable, condition, or situation of interest.

Delphi technique

A research approach to generate consensus among subject-matter experts on a topic that lacks robust, science-based data, to set priorities, or to create a stance where one has not existed before (McPherson, 2018)

Descriptive studies describe characteristics of a population or phenomenon using observational methods such as surveys, prevalence, and incidence data. It does not involve relationships between variables; instead, it aims to create a picture of a variable, condition, or situation of interest.

Eligibility criteria

The pre-determined list of criteria that outline the characteristics of who will and will not be included in a study.

Eligibility criteria should be clearly listed and should define the exact characteristics of who can and cannot be included in a study. It can be based on what is feasible and ethical, as well as who or what the team is truly trying to study.

Term

Definition*

Appraisal Considerations

Ethical Review

The process by which an institutional review board (IRB) assesses research proposals to ensure they are ethically acceptable.

In general, all research studies should undergo ethical review (there may be some exceptions based on the country in which a study is conducted and the amount of interaction with participants). Citing the ethical review process is an essential part of the report of a research study. Review boards may deem studies “approved” or “exempt.”

Other non-research activities, such as quality improvement (QI) can also undergo ethical review. If this occurs, the study team should provide the process and confirm the IRB deemed their project to be acknowledged as QI and outside of the IRB’s scope.

Evidence Summary

A peer-reviewed synthesis of scientific literature written by organizations following pre-determined methods to select and evaluate evidence. Information is presented in a succinct and actionable way for a broad audience with the intent to support point-of-care decision-making (Petkovic, 2016; Jordan, 2019).

The EBP team should ensure an evidence summary was completed using robust methods for selecting and appraising evidence. It may be helpful to reference organizations that are well-known for producing high-quality evidence summaries (e.g. UpToDate and JBI). Because of the goal of making the report easy to read, many times the methodology is not included in the document itself, and the team will need to look for further details on an organization’s website.

Experiment

In true experiments, a study team manipulates an independent variable and randomly assigns it to an intervention or control group.

Experimental studies use highly structured designs to establish cause-and-effect relationships. See Randomized Control Trials for further information.

Expertise

Special skills or authoritative knowledge of a topic (Merriam-Webster)

Expertise is not always readily apparent from looking at the listed authors in a publication. Further information can be found in their listed affiliations and by performing an internet search. Items to look for are their professional affiliations, publications on the topic at hand (see H-index), and credentials.

Findings

The results of systematic inquiry usually in the form of data or narrative information

Authors should provide both the data they are analyzing and the results of that analysis. Often this is displayed in tables or figures. The findings should be presented without commentary and reflect the information exactly as it was gathered and analyzed. The findings should help inform the study's aim and the process to generate them should be explained in the methods.

Forest plot

A graphical display designed to illustrate the relative strength of the effects of an intervention from multiple quantitative studies addressing the same question

These are a hallmark of systematic reviews with meta-analysis. EBP teams should ensure they are easy to read and match the results and discussion sections.

Term

Definition*

Appraisal Considerations

Funding

Money provided to aid in conducting and reporting studies or other reports. It can come from government grants, private foundations, corporations, or academic institutions

Studies can be commissioned by various organizations with various interests or priorities. Investigations have shown that commercially sponsored studies (e.g. from technology or drug companies) are more likely to have findings that favor a sponsor’s product than independently funded studies. Publications should include a statement addressing any funding received, if it poses a conflict of interest, and if so, how it was addressed.

Generalizability

The extent to which the findings from a study can be applied or extended to other settings, populations, or time periods. High generalizability means the conclusions are likely relevant beyond the study's specific conditions. Sometimes also called “external validity”

Study teams should make an effort to ensure their participants truly reflect the larger population, such as random sampling or subgroup analysis, and clearly report these measures. Authors should also provide detailed information about where the study took place and the included participants. They should do this in a way that allows the reader to determine if the findings can be applied not only to the larger population but also to their specific setting and population.

Of note, quality improvement projects do not have a main goal to be generalized, and these efforts may be minimal in this type of report.

Grading

A systematic way to assess and assign a rating to the quality or bias of evidence.

Reviewers can use a variety of tools/models to assess or “grade” their evidence. They should explicitly state the model used and list the grade or rating assigned for all the provided evidence or recommendations.

Gray literature

Scholarly output that is not formally published in peer-reviewed journals. This can include theses, dissertations, government reports, conference papers, and internal documents from organizations.

EBP teams should assess the source of their gray literature and ensure it is reputable. The report itself should provide sufficient information to conduct a formal assessment. Occasionally, this literature does not meet the requirements to be included in the evidence synthesis, but it may provide helpful background information.

H-index

A calculation to measure the amount and impact of scientific publications by an individual. The number is related to the number of published papers by the author and how many times each has been cited (Schreiber, 2019).

This can be a helpful metric to determine someone’s expertise on, and scientific contributions to, a topic. It can be found using search engines such as Scopus or Google Scholar. There is no required value, but for context, in the medical field, assistant professors tend to have h-indexes between 2 and 5, associate professors between 6 and 10, and full professors between 12 and 24 (Schreiber, 2019).

Incidence

A measure of the occurrence of new cases of a disease or condition in a specified population within a certain timeframe. It provides information about the risk of contracting the disease or condition.

This metric is often used to report on the outcome of interest. It is usually expressed as a rate, meaning a count over a certain time frame. When possible, authors should provide incidence rates in a well-recognized format (e.g. number of falls per 1,000 patient bed days).

Term

Definition*

Appraisal Considerations

Inclusion/ Exclusion criteria

The set of rules, markers, or guidelines used to determine who or what is eligible to be included in a study or evidence review.

In the context of literature reviews, the inclusion/exclusion criteria are the list of characteristics a study must HAVE or NOT HAVE to be included in the data analysis. In literature reviews with a systematic approach, they should be directly recorded in the report itself or supplemental content. The EBP team should ensure they are present and fit the question the reviewers are trying to answer. The team should also assess the given criteria for biases (e.g. excluding evidence from one region of the world without reasonable justification).

Institutional Review Board (IRB)

A group, usually associated with an academic organization, that reviews study proposals to evaluate their ethical implications. See “ethical review” for more information.

This term is primarily used in the United States. Authors should list their specific IRB and the designation assigned to a study. Other terms include Ethics Review Committee, Ethics Review Board, Research Ethics Board, and Independent Ethics Committee.

Intervention

An action or item purposefully introduced into a study to test its effects on outcomes of interest.

Interventions can be used in any type of experimental or quasi-experimental study and are often used to assess effectiveness of treatments, drugs, or techniques. An intervention should be deliberate and described in enough detail so the reader could replicate it.

Likert scale

A scale for measuring attitudes or opinions that uses a fixed number range with associated descriptions for each of the values in that range.

Likert scales typically ask people for their level of agreement, likelihood, or other opinions using a number range (usually between 3 and 7 options) with each side of the scale representing the extremes of each option. Although they are assessing subjective information (e.g. attitudes), Likert-scales are a type of quantitative measurement because they assign a numeric value to the measurement.

Limitations

The recognized flaws, constraints, or weaknesses within a study that may affect the results or implications of the findings.

All studies have limitations. If they are not provided, this is a limitation in and of itself. Ideally, authors provide limitations as well as explanations of how they were mitigated.

Literature Reviews with a Systematic Approach (LRSAs)

LRSAs use explicit methods to search the scientific evidence, analyze the information, extract data, and summarize the included studies.

These reviews go by different names (e.g. systematic, integrative, rapid, umbrella). To determine if a review uses a systematic approach the EBP team should look for the following:

· An explicit pre-planned method or protocol

· A clear question

· Clear and explicit inclusion and exclusion criteria

· A documented search strategy, including sources and terms

· Use of tables to provide pertinent characteristics of the studies included

· An explicit approach to assess the quality (risk of bias) of included evidence

· Exploration of the data to consistencies and gaps

· Use of tables or figures to support interpretation

*Some of this information may be provided in appendices or supplemental files (Booth, 2021)

Term

Definition*

Appraisal Considerations

Longitudinal

A study design that involves repeated observations or measurements of the same variables, among the same individuals, over time. This can span years or even decades.

Longitudinal studies involve multiple data collection points and are useful in understanding long-term efforts or changes. It is common in developmental psychology, sociology, and medicine.

Manipulation

The study team’s control over the independent variable (intervention) to observe its effect on the dependent variable.

Manipulation of a variable essentially means a study team “did something.” They intervened or changed a situation in some way to measure how that change affected other metrics (variables) of interests. This can range anywhere from introducing a program to giving a patient a medication or treatment.

Meta-analysis

A statistical technique that combines the results of multiple scientific studies addressing the same question to integrate findings and measure an overall effect size. This method enhances the overall understanding of the variable of interest by increasing the sample size and statistical power.

Meta-analysis is usually conducted after reviewers have completed a systematic search and selection of literature on their topic and outcome of interest. Essentially, in a rigorous and replicable way, reviewers attempt to gather all studies that answer their review question and meet their inclusion/exclusion criteria (see corresponding section), to pool data that measures the same variable in the same way. They can then combine those numbers to create a larger, more convincing statistical calculation.

Meta-synthesis

A method used in qualitative research to integrate, evaluate, and interpret findings from multiple qualitative studies. The goal of meta-synthesis is to build a greater narrative or comprehensive understanding about a phenomenon.

Meta-synthesis is the qualitative counterpart to meta-analysis. The analysis process begins after reviewers have systematically gathered and selected evidence that addresses their topic of interest. It uses systematic methods to not just pull information together but to create new interpretations and deeper insights that go beyond the findings of individual studies. This approach attempts to make the whole greater than the sum of its parts.

Mixed methods methodology

An approach that combines elements of both qualitative and quantitative methods to provide a more comprehensive analysis of the topic of interest than either method could offer alone.

Authors should provide their reasoning for selecting a mixed methods approach and how they used one type of data to inform the other. Both the quantitative and qualitative portions should be equally explained and analyzed with true integration of data.

Observational

Study design

A type of study in which the investigators observe the natural course of events with minimal or no intervention in the study subjects.

Observational design includes both descriptive and analytical studies (e.g. cohort, case-control, or cross-sectional studies). It is used to describe topics or outcomes of interest as they occur naturally and can simply describe a phenomenon or can suggest relationships between different variables.

Term

Definition*

Appraisal Considerations

Outcome

The result or effect of an intervention or exposure, which is measured to determine the impact of the independent variable in a study.

EBP teams should ensure all outcomes of interest are clearly listed. Authors should explain how they gathered and analyzed data to assess each one.

Participant

A person taking part in a study.

Authors should include information about how they selected and recruited participants, including the percentage of how many agreed to participate. They should also provide details about the participants that help the reader understand who the findings could be applied to.

Peer review

The process by which scholarly work (such as papers, reports, or proposals) is checked by a group of experts in the same field to ensure it meets the necessary standards before it is published or funded.

The purpose of peer review is to ensure that scientific and scholarly work is based on sound methods and that the findings are trustworthy. Peer review adds an additional level of scrutiny to published work and is an important part of the generation of clinical practice guidelines (CPGs) and evidence summaries, as well as work published in scholarly journals. While it is assumed for most journal work, the peer-review process should be explicitly explained in the methods for evidence summaries and CPGs

Phenomenon

A fact, situation, or concept

In qualitative studies, this is the concept the study team is exploring. Authors should explicitly state the phenomena of interest, and their methods should clearly match what they are attempting to explore. This can be considered the counterpart to “variable” in quantitative studies.

Prevalence

The proportion of a population who have a specific characteristic in a given time period. In epidemiology, it often refers to the proportion of people with a particular disease or condition.

This metric is often used to report the number of people who have a disease or condition among those at risk. It is usually expressed as a percentage or the number of cases per set number of people (e.g. 2.5 cases per 1,000 people).

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram

A flow chart that depicts the different phases of a literature review with a systematic approach (LRSA) and illustrates the flow of studies screened, included, and excluded from the search and appraisal.

PRISMA diagrams, or similar flow charts, should be included with all LRSAs. They help the reader understand the scope of the literature search and ensure the process was systematic and comprehensive. Keep in mind, sometimes these diagrams are included as supplementary material and are not available in the article or report itself. The diagram is a portion of a larger reporting checklist (see https://www.prisma-statement.org/).

Prospective

A design that gathers data from the beginning of the study period and forwards in time. Data collection can occur once or several times.

Prospective studies do not look back at any historical or previously collected data. They only collect and analyze data for the study period. This allows the study team to ensure they are gathering complete information and adjust their design as needed.

Term

Definition*

Appraisal Considerations

Qualitative methodology

Qualitative studies collect and analyze narrative data to gain an in-depth understanding of a phenomenon or experience, including opinions, meanings, and motivations. They provide insights into the problem or help to develop ideas or hypotheses for potential quantitative inquiry.

Considerations for qualitative designs are outlined in the Qualitative Appraisal Checklist in Appendix E2. See Chapter 6 for more details. Some words to look for that are associated with qualitative designs and may help the EBP team determine if they are looking at this type of study are: narrative, thematic, coding, phenomenology, ethnography, grounded theory, critical theory, or data saturation.

Quantitative methodology

Quantitative studies involve the collection and analysis of number-based data to quantify a problem by generating numerical information that can be transformed into usable statistics.

Considerations for quantitative designs are outlined in the Quantitative Appraisal Checklist in Appendix E2. See Chapter 6 for more details. Some words to look for that are associated with quantitative designs and may help the EBP team determine if they are looking at this type of study are: randomized control trial, experimental, quasi-experimental, statistics, calculations, power, significance, Likert, incidence, prevalence, case-control, or cohort.

Quasi-Experimental Studies

Quasi-experimental studies have an intervention but lack randomization and sometimes lack a control group. They can help to establish causal relationships, but because they are limited in their ability to control for confounding factors, are not as compelling as true experiments (Randomized Control Trials; RCTs).

Quasi-experimental designs are used when it is not ethical or feasible to randomly assign people to an intervention. Words commonly associated with this approach are pre/post, nonrandomized, nonequivalent, natural experiment, or opt-in.

Randomization

The process of assigning participants into different groups in a study to ensure each participant has an equal chance of being assigned to any group.

Randomization reduces bias by increasing the likelihood that groups are comparable at the beginning of a study. EBP teams should ensure participant assignments are truly random (e.g. random number generator, coin flip) and not haphazard (e.g. dividing a list in half) or introduce bias in another way (e.g. grouping patients by time of day they present to a clinic).

Randomized Control Trial (RCT)

RCTs are considered “true experiments” and are considered the gold standard for establishing causal relationships. They have three core components, randomization, control, and manipulation of a variable.

EBP teams should assess if RCTs truly used random methods that ensured each participant had the same likelihood of being in the intervention or control group, the control group was otherwise similar to the intervention group and the intervention is clear and well-described. RCTs typically follow very robust methods and use advanced statistical calculations that are approved by an institutional review board. To increase confidence in the study findings, the EBP team can look to see if the trial protocol was registered or published.

Term

Definition*

Appraisal Considerations

Reflexivity

The study team members’ awareness of their own influence on the study process and outcomes.

Study team members should reflect and provide information on their own biases, values, and decisions and how this might have affected the conduct of their study. This helps ensure transparency and objectivity.

Reliability

Reliability refers to the consistency of a measure or instrument. A reliable tool will yield the same results under consistent conditions across different times and settings.

Authors should provide specific information about the reliability of their data collection tools. This can sometimes be expressed with a statistic called Cronbach’s alpha (>.7 is usually considered adequate) or intra-class correlation coefficients (ICC). Other types of reliability relate to having consistent measurements regardless of who is collecting/analyzing the data (inter-rater reliability), and consistent measurements from multiple tests describing unchanged conditions (test-retest reliability).

Research

Research is a systematic investigation into, and study of, materials and sources to establish facts and reach new conclusions. It is an organized way to learn and understand more about a specific question or problem.

Research should be rigorous and replicable with the intention of creating new knowledge.

Response rate

The proportion of individuals who respond to or participate in a survey or study out of all those invited or selected to participate.

Response rates should be provided because they are an important indicator of the representativeness of the data collected. Low response rate may introduce bias, especially if those who did respond are fundamentally different than those who did not. Authors should provide the exact number of people they attempted to recruit for all data collection points and the number of those people who responded (usually expressed as a percentage). There is not one “gold standard” for acceptable response rates. For context, one systematic review found the average response rate in patients is 70% and 53% for doctors (across all modalities; Meyer et al., 2022).

Retrospective

A retrospective study design involves looking back at events that have already occurred.

Retrospective studies do not collect data generated during the study period but rather look back at previously recorded information (e.g. retrospective chart review) or through recollections of participants. This can make the conduct of a study more feasible or ethical, but also can lead to incomplete data because study teams cannot fill in missing information or participants’ memories might be limited. It is often contrasted with prospective studies, which follow participants into the future.

Review or research question

A clear and focused question that outlines the topic the study or review seeks to answer.

In the context of a review, the question should be explicitly listed in order for a reader to understand who, what, and where the review applies to. It defines the scope of the investigation, often expressed as a PICO question (Population, Interventions, Comparisons, and Outcomes of interest). It guides the literature search and inclusion/exclusion criteria for studies.

Term

Definition*

Appraisal Considerations

Sample

The subset of individuals, cases, or data points selected from a larger population for the purpose of conducting a study. The goal of using a sample is to obtain conclusions that can be generalized to the entire population while being cost-effective and more manageable in terms of size and practicality.

A sample should ideally represent the characteristics of the larger population from which it is drawn. This allows for the generalization of results back to the population. Authors should provide relevant details about their sample (e.g. demographics, past medical history, diagnoses) clearly and explicitly to help the reader understand the groups the findings apply to.

Sample size

The number of participants or data points included in a study.

Study teams should provide the number of people they intended to recruit, and how they arrived at that number (this can be based on a statistical calculation, power, or other methods such as comparison to similar studies that have been previously published). Authors should also provide the number of participants they successfully recruited at each data collection point in their report in a way that is easy to find and interpret. Larger samples generally provide more reliable estimates but are costlier and more time-consuming to manage.

Sampling

The process of selecting the participants for a study.

Authors should explicitly provide their methods for selecting potential participants for their study. This helps the reader determine if the eventual participants truly represent the larger group they were pulled from. Various methods include random sampling, stratified sampling (breaking the larger population into sub-groups that share similar characteristics and recruiting from each), convenience sampling (selecting participants who are easily and readily available), systematic sampling (selecting individuals at a pre-determined interval, e.g. every 5th person), cluster sampling (selecting entire groups) and snowball sampling (using participants to identify other participants). Snowball sampling can be used when populations are difficult to access, or a disease or condition is rare.

Saturation

In qualitative studies, the point at which data collection is not revealing any new information and themes or patterns are redundant. Saturation indicates that the data collection process can be concluded.

In qualitative studies, saturation is an indication the study team has collected enough data, and the sample size was adequate. They should explicitly explain how they determined saturation had been reached.

Search Strategy

A formal process used to retrieve evidence by identifying databases and creating search strings that include key concepts and synonyms with database-specific syntax (Booth, 2021; Bramer, 2018).

For literature reviews with a systematic approach (LRSAs), search strategies should be provided. This might not appear in the report itself but in online supplemental materials or technical development reports.

Term

Definition*

Appraisal Considerations

Seminal paper

Works of central importance to a topic or area of study. They often report a major breakthrough, insight, or a new theory. This kind of paper may describe a study that changes our understanding of a topic or describes and illustrates a new and highly useful scientific method. Also called pivotal, classic, or landmark studies.

When EBP teams are assessing the reference list of an article or report to ensure citations are recent, they may come across much older entries. This does not necessarily mean it is out-of-date, but they include foundational information in the form of a seminal paper (e.g. Benner’s Novice to Expert paper). There is no specific label to identify these works, rather the team may need to do further investigation to determine their status—citation analysis is one method.

Study Design

An approach or set of methods and procedures used to collect and analyze information (Ranganathan, 2018).

Study designs should be explicit and formal. A report is considered to have a formal study design if it meets most of the following criteria:

· Was pre-planned (prior to investigators initiating intervention or data collection)

· Received ethical review (by the institutional review board) ​

· Has formal and systematic data collection and data analysis ​

· Uses specific qualitative and/or quantitative information gathered for the purposes of the investigation ​

· The study team are not subjects of the intervention​

· Has a clear aim, reproducible methods, results, and discussion

· Do not only recount the authors’ personal, organizational, or literature-based experience.

Study setting

The physical location where data collection for a study takes place

Authors should include details about the environment in which a study takes place. This can include the type of facility (e.g. hospital inpatient, nursing home, school), the geographic location (e.g. region and country), and other information about the location that will help a team determine if it applies to their setting (e.g. academic hospital, rural hospital). It is common for authors to not use the name of the organization but general descriptors.

Triangulation

The use of multiple methods, data sources, investigators, or theoretical perspectives to cross-validate and corroborate findings.

Authors should explicitly address their efforts to enhance credibility and confirm their findings through triangulation techniques such as having multiple researchers analyze data, collecting data through different approaches or from more than one source, or approaching analysis with different interpretive frameworks.

Term

Definition*

Appraisal Considerations

Validity

Validity refers to the extent to which a research instrument or study measures what it is intended to measure.

Authors should describe if the tools they are using are valid, meaning they have undergone a process to ensure they are measuring what they intend to measure. This can be done through a variety of processes (from consultation with subject matter experts to statistical analyses) which establish different types of validity. Types of validity include:

· Content Validity: The extent to which a measure represents all facets of a given construct.

· Criterion Validity: The extent to which a measure is related to an outcome.

· Construct Validity: The appropriateness of inferences made based on observations or measurements (often using a test) of a particular construct.

· Face validity: The general perceiving appropriateness of a tool.

Variable

A variable is any characteristic, number, or quantity that can be measured or quantified. Variables can be considered dependent, independent, or confounding.

Authors should list all variables they intend to measure and how they will measure them. The variables they are collecting should link directly to the aim(s) of the study.

Statistics Terms and Definitions

Term

Definition*

Central tendency

A type of descriptive statistic to describe a “typical” value in a set of numbers that uses different calculations to quantify the center of the range of values. It includes mean (average), median (the middle value when data are put in order), and mode (the most frequently occurring value).

Confidence interval (CI)

Expressed as two numbers with an accompanying percentage, CIs are a range of values within which a metric is estimated to fall, at a specified probability (e.g. 95%). The specified probability tells you how confident the person performing the calculation is that the metric does in fact fall within the range. For example, an average of 10 with a 95% CI of 8-12 tells the reader they can be 95% sure the true average is between 8 and 12.

Effect size

The strength of the relationship between variables. Unlike significance tests that provide a yes-or-no answer to whether an effect exists, the effect size tells how substantial the effect is. Common measures include Cohen’s d (standardized difference between two means), correlation coefficient (strength of association between two variables), and odds ratio (ratio of the odds of an event occurring in one group to the odds of it occurring in another group).

Odds ratio (OR)

Expressed as percentage or integer, OR is a measure of the likelihood (odds) of an event occurring to a member of a group compared to another (a ratio of event to non-events). A negative OR means the odds of an event occurring in a member of an exposed group is lower than that of a non-exposed group. ORs of 1 indicate there is no difference between group members. Positive ORs mean there are higher odds of an event occurring in a member of the exposed group compared to the non-exposed group. For example, an OR of -.5 comparing the odds of increased body mass index for a member of a group who attended exercise sessions vs the odds of increased BMI for a member of a group person who did not attend the session means a person who went to the exercise sessions were 50% less likely to have an increase in their BMI. ORs explain the odds of something occurring to an individual whereas relative risk explains the probability of something occurring at the population level.

Power analysis

It is a statistical method used to determine the number of participants or observations (sample size) required to detect an effect of a given size with a certain degree of certainty.

Statistical significance

Is a determination made based on the probability that the observed results of a study could have occurred by chance alone. This probability is expressed as a p-value; a p-value less than a chosen significance level (commonly 0.05) indicates that there is a 95% likelihood the observed effects are true and not based on change alone. In some cases, lack of statistical significance is a good indication (e.g. when comparing baseline characteristics between an intervention and control group).

Relative risk (RR)

Expressed as percentage or integer, RR, also known as the risk ratio, is a measure of the probability of an event occurring in the exposed group versus a non-exposed group. For instance, if the relative risk of developing a disease for smokers compared to non-smokers is 2.0, it means that smokers are twice as likely to develop the disease as non-smokers. Relative risk helps in understanding the strength of the association between an exposure and an outcome at the population level.