Section 1
Preview this deck
What is the point of case studies/series? Can they demonstrate cause & effect?
Front
| 5 | 0 | ||
| 4 | 0 | ||
| 3 | 0 | ||
| 2 | 0 | ||
| 1 | 0 |
Active users
0
All-time users
0
Favorites
0
Last updated
6 years ago
Date created
Mar 1, 2020
Cards (394)
Section 1
(50 cards)
What is the point of case studies/series? Can they demonstrate cause & effect?
To explore extreme cases and to identify potentially surprising outcomes (e.g. penicillin) - Show relationship of exposure and outcome - Alert to possible adverse effects - CanNOT show cause & effect
What is a quasi-experiment?
An experiment that lacks randomization, a comparison group, or both - Threats to validity cannot be ruled out - Causality cannot be proved
What four steps are within identifying the question for research?
- Identify research question - Review literature - Identify variables - State hypothesis
What are some examples of observational studies that have led to important discoveries?
- Penicillin: mold zone of inhibition - Chlorpromazine: 1st antipsychotic - Fewer peptic ulcers with doxepin --> development of H2 antagonists
Do clinical trials need placebo controls, and why or why not?
No, sometimes placebo is not ethical (e.g. to see if parachutes work, you wouldn't give some people no parachute)
What are cons to RCTs?
- Time consuming - Costly - Randomization (assignment) is unethical - Variable of interest is an attribute
What is a primary outcome?
The main objective of the study
What are the two main types of study designs?
Observational and interventional
What is the weakness of factorial studies?
Design efficiency is lost if interventions interact
Parallel Trials - Strengths - Weaknesses
Strength - Relatively simple Weakness - Large number of people needed
What does a priori mean?
From the beginning, specifically what endpoints were defined at the beginning of the trial (before study is done)
What is external validity?
When the results of a study can be generalized to the population being studied - Whether the same thing happens in other settings
What is factorial design?
2 Interventions in 1 trial Studying two drugs at the same time by randomizing a sample and then randomizing each of those samples to another therapy 2 x 2 Factorial Design: - Drugs A & B - Drug A, B Placebo - Drug B, A Placebo - A Placebo, B Placebo
What are composite endpoints?
Putting multiple endpoints together to make analysis more straightforward and to make effect seem larger - Scared if we use a single outcome we won't find a difference - More opportunity to show a difference - Greater chance for deception; sounds more impressive
How did the Physician's Health Study randomize treatment with aspirin & beta-carotene?
20,000 randomized to aspirin or aspirin placebo Aspirin group then split to also get beta-carotene or beta-carotene placebo Aspirin placebo group then split to also get beta-carotene or beta-carotene placebo 4 Groups: - ASA + BC - ASA + BC Placebo - ASA Placebo + BC - ASA Placebo + BC Placebo
What is often found of the composite endpoint?
One of the individual parts of the composite (a 2˚ outcome) usually drives the entire composite
What type of endpoint are the individual parts of a composite endpoint?
Secondary
What is a parallel trial?
Sample is randomized to treatment and control and endpoint is analyzed
What are the five steps in research?
1. Identify question 2. Design study 3. Implement study 4. Analyze data 5. Communicate result
How is validity ensured in study design?
- Inclusion/exclusion - Randomization - Concealed assignment (people who assign groups are blinded) - Blinding
Case studies often lead to what kind of study?
Quasi-experiments
What are secondary and/or tertiary endpoints?
Other effects that may occur as a result of the trial but aren't the main goal - Secondary: interest, can provide some data - Tertiary: exploratory
What did the WHI (Women's Health Initiative) illustrate in regards to surrogate endpoints?
WHI looked to determine if hormone replacement therapy decreased coronary heart disease and osteoporosis factors but found instead that estrogen increases risk of cancers. Surrogate endpoints might demonstrate different results than we expect.
What three things make a good research question?
- Issue is important - Question must be answerable - Research must be feasible
The CAST (Cardiac Arrhythmia Suppression Trial) looked at the effect of anti-arrhythmics in decreasing the occurrence of arrhythmias. CAST compared morbidity and mortality in anti-arrhythmics vs. placebo. What was the surrogate endpoint and what does this show about surrogate endpoints?
Whether antiarrhythmics decrease arrhythmias. This is a substitute for morbidity/mortality. The study found that the anti-arrhythmics did decrease arrhythmias, but that mortality was actually greater when treated. This illustrates how surrogate endpoints might not be a good measure of drugs - we can see different results than we expect.
What can be done when it is unethical to use a placebo? What is the problem with this?
Can use an active control But does not provide the same level of information
What are the three goals of research?
- Is there a relationship between two variables - Does one variable cause another, and - Can these results be generalized to a population
What are the goals in designing a controlled clinical trial?
- Minimize bias - Minimize confounders - Ensure validity
What are the strengths & weaknesses of crossover trials?
Strengths - Increase power since patients serve as own controls - Small sample Weakness - Caution if have carry-over effects (must have adequate washout period) - Can't use for short term or curable diseases - Long participation time
What is internal validity?
The design of the study has as few flaws as possible - Whether the research is done right - Whether it avoids confounding variables (more than 1 independent variable/cause acting at the same time)
What is a surrogate endpoint?
An endpoint that points at a result that is difficult to measure - Substitute endpoint for a direct outcome
What are observational studies? Why are they important? Do they prove cause and effect?
- No intervention - Generate a hypothesis - Exploratory (not confirmatory) - "Simple" observations lead to important discoveries - Cannot prove cause and effect since not making an intervention
What is the strength of a factorial study?
Answer two questions at once - Cost efficient
What are the five types of observational studies?
- Case study/series - Case-control - Quasi-experimental - Cross-sectional - Prospective cohort
What are the benefits of randomized studies?
- Minimize confounders - Minimize bias - Increase generalizability
What is a crossover trial?
A sample is randomized to treatment and control. Both samples are washed out, and then they switch and endpoint at each step is measured and analyzed - Each treatment is given
What is the order of study designs in order or least to most reliable?
- Case report - Case series - Case-control study - Cohort study - Randomized controlled clinical trial
Is a control group ethical?
Yes, it may not benefit the participant but will benefit people in the future
What are the two types of validity?
Internal External
Example: You compare bone mineral density with use of Fosamax and Drug X. What is a surrogate endpoint?
Bone mineral density is a surrogate endpoint for fracture rate - BMD is not a direct endpoint (not an event/important aspect of someone's life), but it is a substitute for fracture rate (which would be a direct endpoint/event)
What is an example of a surrogate endpoint?
Blood pressure - Not a direct outcome - Tells is if there is an increased risk for those outcomes
Why do we need a control group?
Allows us to see if there is a difference between the two groups - allows us to say the intervention caused the effect to happen
What is a great reason to do an RCT?
Can never rule out threats of validity in the same manner as an RCT
What are the benefits of observational studies?
- Increase generalizability (depending on sample size) - Speed (short) - Cheap - Address broad range of questions
What is a direct endpoint?
Measures a significant aspect/event in someone's life - Probably the most significant/meaningful
What are examples of direct endpoints?
Hospitalization, MI, stroke
What is a controlled clinical trial (generally)?
An experiment where the investigator manipulates group assignment to different interventions - Interventional - ie. Treatment vs. Placebo
What type of endpoint is the entirety of a composite endpoint?
Primary
What does the hypothesis serve as the basis for?
- Sample size calculations - What hypothesis test to use - What type of inferential statistics to use
What is the most common type of clinical trial?
Parallel
Section 2
(50 cards)
What is observation basis?
The phenomenon where patient behavior is not reflective of how they would normally act since they know they are being observed
What is the strength of stratified randomization?
Prevents unequal distribution of certain characteristics
What is a confounder?
A variable that is not studied that affects the outcome
What is triple blinding?
Everyone involved doesn't who is in what group (3rd - statistician)
What is a benefit of unequal allocation over equal allocation?
Gives us more information on adverse events since more people are exposed to active drug
What does equal allocation refer to?
Equal chance of having placebo vs. drug (1:1) Each group gets an equal number of patients
What is the strength of equal allocation?
Maximizes power
What is single blinding?
Investigator or patient does not know
Surveillance Bias
When patients in 1 exposure group have a higher probability of having the study outcome detected due to increased surveillance, screening or testing of the outcome itself, or an associated symptom ie. drooling with anti-psych med
What is the goal of inferential statistics?
Disprove the null hypothesis
What do we do once we have identified the question and designed and implemented the study?
Analyze the data (step 4)
What is the weakness of equal allocation?
Imbalances in baseline by chance
What is blinding?
Inability for subject to know what they're getting (for entire duration of the study) Subject could behave differently Introduces bias/systematic error if don't or fails
What is SNOSE?
Sequentially numbered opaque sealed envelope - Method of blinding
What is an alternate hypothesis?
Describes the relation you are hypothesizing to exist (Ha) Ha: µ1 ≠ µ2
How is a hypothesis tested?
- Form a premise (1 group ≠ another) - Assume the opposite (null hypothesis) - Perform experiment - Accept or reject null hypothesis
What is the weakness of stratified randomization?
Not feasible for more than 2-3 characteristics, characteristics must be known
How is the null hypothesis written?
Ho: µ1 = µ2 µ = outcome States the mean of group 1 = mean of group 2 Ex. Beta blocker = placebo Beta blocker should have an effect on HR (want to reject null)
What is the Hawthorne effect?
The alteration of behavior by the subjects of a study due to their awareness of being observed
What is misclassification bias?
When patients are classified by a error-prone method - Classified into a category other than that to which it should be assigned
What is exclusion criteria? Is this usually long/short?
Who is NOT eligible to participate - Long; usually many more than inclusion criteria
How can validity be determined when reading a study?
Read the methods - Look for bias and confounders
What is selection bias?
Pick people that aren't relevant to a study - Failure to achieve proper randomization - Sample is not representative of the population intended to be analyzed
What is recall bias?
Differences in the accuracy or completeness of the recollections retrieved ("recalled") by study participants regarding events or experiences from the past
In studying an outcome of rate if MI's, what are potential confounders? How do we eliminate confounders?
Age Smoking Use randomization: make equal chance for patients getting into any group Ensure the exposure you are studying independently causes the outcome ie. Alcohol use, oral contraceptives
What is bias?
Systematic error in research
What is the purpose of blinding?
To reduce systematic error and bias in interpretation
What is type I error?
False positive - incorrectly rejecting the null hypothesis ie. Saying an ineffective drug works Drug does NOT work
What is the strength of blocked randomization?
Assures equal numbers in each group
What is double blinding?
Investigator and patient don't know
What are the different types of bias?
- Selection - Observational/Hawthorne - Misclassification - Surveillance - Recall - Interviewer
What is block randomization? What purpose does it serve?
Repetitive randomization in a set pattern ie. ABBA, ABBA, ABBA Ensures group & control will be the same size
Why is concealed assignment important?
Treatment benefit can be exaggerated/overestimated by up to 40% if HCP knows who is getting it
What is concealed assignment? Why is it important?
Person randomizing the patient does not know what treatment they are being allocated to If they were aware they would look for adverse events more
What is randomization?
A way to remove bias and confounders in research Make sure there are no differences in populations in groups at baseline (if they are different we can't be certain what caused the results)
What is inclusion criteria? Is this usually long/short?
Who is eligible to participate in a study - Short
What are the weaknesses of unequal allocation?
- Less power than studies with equal group numbers - Greatest loss at >3:1 If people think they didn't get the active drug they drop out
The effect of a drug used in pregnancy was assessed in a retrospective study. Which of the following is most likely to occur? - Selection bias - Observer bias - Interviewer bias - Recall bias
Recall bias
How do we perform randomization in research? How successful is it?
Statistical software Does NOT always work
What is interviewer bias?
When an interview asks slanted questions - Interviewer is partial toward a preconceived response through the structure, phrasing, or tenor in which they ask a question - Influence respondent and distorts the outcome
What are the strengths of unequal allocation?
- Helps with recruitment - Provides more info about side effects
How is the main objective of the study assessed?
Via the primary objective - Develop a null hypothesis around the primary endpoint - Then choose the proper statistical test to analyze results
What is a permuted block?
Design several combinations of blocks (ie. block size of 4 would consist of 2 participants in group A treatment, and 2 form group B placebo) Then assign a number to each combination Then generate a list randomizing these numbers (ie. 1-6 for 6 combinations/permuted blocks) The numbers represent the order in which you give the treatments Ensures the investigators will not figure out who received what
What is a null hypothesis?
(Ho) Describes what is being tested in inferential stats, usually is that there is no change or difference between two variables, want to reject this - Just states that the 2 groups are EQUAL - Opposite of what you are trying to prove (want to reject it)
What is a potential issue with block randomization?
May lead investigators to figure out who received active drug vs. placebo Can instill bias - bad if they figure it out
What is unequal allocation? Why might this be used?
When there is a greater chance of receiving the active drug rather than placebo More patients in active drug group vs. control group (2:1, 3:1, active:control) Encourage patients to participate
What is the weakness of block randomization?
Easy to predict the pattern
What are 4 strategies we can use to conceal assignment?
- SNOSE - Keep it pharmacy controlled: perform randomization and then move right onto pharmacist so investigators are left out of knowing (pharmacies often assign) - Use numbered/coded containers - Central randomization
What is stratified randomization?
Strategies that help ensure randomization doesn't fail Randomization based on baseline characteristics (2-3 variables at most) ie. Stratify on duration of disease Duration should be about the same in both the control group and active drug group
What is double dummy blinding?
When every person gets each type of treatment Ex. If looking at active drug in tablet vs. injection dosage form Group getting active tablet also gets placebo IV Group getting active IV also gets placebo tablet Ex. If dosing differs: QD vs BID Give QD group active + placebo Give BID group active + active
Section 3
(50 cards)
What type of error is a false positive?
Type I
For a given effect size, an increase in sample size will do what to power?
Increase
Which tests in sample size calculations are held constant?
- Statistical test - Alpha-level
When should sample size be calculated?
A priori - before the study begins
For a given sample size, a decrease in effect size will do what to power?
Decrease
What is biocreep?
The acceptance of inferior treatments due to the acceptableness of lower noninferiority margins over time
What is the formula for power?
1 - beta
What is type II error?
False negative - failing to reject the null hypothesis when a difference truly exists ie. failing to realize it is a life-saving drug Drug DOES work
What is power?
The ability to show whether a real effect when it really exists
What does it mean in a NI trial if the line crosses through the 0 line?
Cannot say it is superior ie. p = 0.2 For it to be superior, it cannot be less than 0 p < 0.05 when above 0 (treatment difference)
What does the vertical line at 0 represent?
The minimally clinically relevant margin Treatment difference
What is the challenge in trying to raise power?
It can be incredibly expensive Trade off in what it costs to increase power is usually not worth it ie. Increasing sample size by 200 participants only increases power by 2% but costs $1 million
What is an equivalence trial?
Look to see if 2 interventions have similar effects - Don't see often
How is the outcome determined in a non-inferiority trial?
Using the non-inferiority margin
What does a p-value < 0.05 represent?
There is a less than 5% chance that the investigators made a type 1 error
Drug X lowered DBP by 2 mmHg. The result was significant at p < 0.05. Is the result clinically significant?
No - there is not a difference in the practical implications/not a real-life difference Reducing DBP by 2 mmHg will not prevent stroke/death
What would be a type I error in a non-inferiority trial?
The null hypothesis is true but rejected Ho: Drug A is inferior to B Ha: Drug A is not inferior to B Type I error: Drug A is inferior, but claim that A is not inferior
When is it determined that the treatment is non-inferior to its control?
-𝛿 and above
What does it mean in an NI trial if the line crosses the non-inferiority line (-𝛿) through the 0 and past/above 0?
Since it crosses below the -𝛿, you cannot say it is not non-inferior For it to be non-inferior, it must be -𝛿 or above Cannot be less than -𝛿
If a study is reporting a ratio between two variables, what should the confidence interval NOT contain in order for the finding to be statistically significant?
1 - This means there is a likelihood the variables are the same and the results are statistically insignificant
What do p-values NOT report?
The strength of a observed finding
How can power be maximized in a study?
- Increase sample size - Demonstrate larger effect size Try to maximize power but often limited by $
Confidence intervals report what about findings?
The line of no effect - When the finding is NOT statistically significant
What type(s) of significance are reported by the p-value?
Statistical significant NOT clinical significance Even if the study is found to be statistically significant, it might not be clinically significant
What provides more information than just the p-value?
Detecting clinical indifference ie. 2 mmHg change in BP would not lead to a change in Tx
For a given power, as a the effect size decreases, what happens to the sample size?
Increases - It is easy to see big differences - But in order to see small differences, the sample size must be bigger/increased
What is the alternate hypothesis in non-inferiority studies?
Control - treatment < noninferiority margin
What would be a type II error in a non-inferiority trial?
The null hypothesis is accepted but should have been rejected Ho: Drug A is inferior to B Ha: Drug A is not inferior to B Type II error: Drug A is not worse than B, but claim is is inferior to B
Why is a range reporting mean difference not significant if it includes zero, but a ratio is not significant if it contains 1?
Difference: = substracting; there is no difference if there is 0 difference Ratio: = dividing; there is not difference if the ratio is 1/1
What does it mean if the line falls within the -𝛿 and 𝛿? What is needed for to prove this?
The treatments are equivalent Need a large sample size Not seen often
What is a p-value?
Equal or related to alpha, the probability of reporting a false positive (type I error) Includes the probability that the data are caused by chance
What is power used for?
Used to calculate an appropriate sample size
What increases power?
Increase in sample size
The NI margin can be defined as -𝛿 or +𝛿. Describe what the result must be if: - 𝛿 is negative - 𝛿 is positive What happens if you cross 𝛿 in either scenario?
NI margin is +𝛿: - Does not go beyond upper boundary of the 95% CI NI margin is -𝛿: - Does not cross the lower boundary of the 95% CI If you cross 𝛿, result is inferior. Result cannot cross 𝛿 for
What is effect size?
A way of quantifying the size of the difference between 2 groups Ex. Data on male & female height show that, on average, men are taller than women Difference between male and female height is known as the effect size The greater the effect size, the greater the height difference will be
Describe hypothesis testing in non-inferiority trials compared to superiority trials.
Hypothesis testing is reversed compared to superiority trials C = control T = treatment 𝛿 = NI margin Ho: C - T ≥ 𝛿 Ha: C - T < 𝛿 Ho: C - T is worse than some delta (set value) - it would cross the 𝛿 = would be inferior Ha: C - T is better - result would be less than the NI margin (would not cross the NI margin) If the result crosses the NI margin (𝛿), we conclude it is non-inferior Null for NI states there is a difference between the groups
When is it determined that the treatment is equivalent to the other group?
-𝛿 to +𝛿
What is a superiority trial?
Look to reject null hypothesis to show a difference between interventions
What are the five main components of power/sample size calculations?
- Stats test being used (to test 1˚ hypothesis) - Alpha (<0.05) - Beta - Sample size - Hypothesized effect size (ie. What we expect to see in the Tx; decrease in cardiac risk is small, curing cancer is large) BASSH
What measures type I error?
Alpha (p-value)
Alpha is a measure of the likelihood of what kind of error?
Type I
What is a confidence interval?
The range within which the true finding is likely to fall - Most CI's are 95% chance - 95% CI tells us there is a 95% chance the CI contains the true value
What type of error is a false negative?
Type II
If a study is reporting a difference between two variables (a measurement difference), what should the confidence interval NOT contain in order to be statistically significant?
0 - This means there is a likelihood there is NO difference and the results are statistically insignificant Ex. -2 to 21 0 falls within this range --> finding is NOT significant
When is it determined that the treatment is superior to the other group/control?
𝛿 = clinically relevant difference = NI margin
What is a noninferiority trial?
Look to show that an intervention is not worse than control
What is the null hypothesis in noninferiority studies?
Control - treatment ≥ noninferiority margin
Beta is a measure of what type of error?
Type II
What value of power is generally acceptable?
80%
What measures type II error?
Beta
Section 4
(50 cards)
What is nonparametric data?
Data that does NOT resemble a known statistical distribution Have to use alternate approach - use different statistical tests
What are the five key descriptors used in descriptive statistics?
Mean, median, mode, range, standard deviation
What is a student t-test? What do you assume in a t-test?
Compares means of independent samples Have to assume: - Normal distribution - Random selection & group assignment - Variance between groups is relatively equal
What is our goal in regards to statistics?
Figure out how close the statistics are to representing the truth
What is descriptive statistics used for?
Describing data within a population - Basis for inferential statistics
T/F: Statistical significance is the truth
F Statistical significance is NOT the truth - It is an approximation of the truth - Users of statistics don't have to be statisticians
What is the formula for standard error of the mean?
SEM = SD / sqr(n)
What is the difference between interval and ratio?
Interval: does not have a true zero Ratio: does have an absolute zero Interval: ˚F, years/dates Ratio: age, heart rate, weight, BP, Kelvin
What information do you need to determine the appropriate statistical test to use?
- Type of data - Independent & dependent variables - Types of samples - Number of groups
What is Poisson binomial distribution?
A type of binomial distribution where the sums of independent Bernoulli trials (binomial distribution math) is added up - Special case of binomial distribution used when there are rare outcomes
What is a paired student t-test?
Compares means of dependent samples
When should the mean not be used? What should be used instead?
If there are outliers - use the median
What is dichotomous data?
Data with two options (e.g. gender)
What is a chi-squared distribution used for?
Determining p-values and CIs for nominal independent data
Can standard deviation be used in all situations?
No - it requires a normal distribution to use SD
What is discrete data? What are the three types of discrete data?
Cannot take the mean - Nominal (categorical) - Ordinal (rank) - Dichotomous
What is the problem with the SEM?
Theoretical value It's misleading: SEM is always smaller than the SD Makes it seem like there is less variation - like more of the population is closer to the mean
What is parametric data?
Data that resembles a hypothetical statistical distribution - Follows normal distribution - Must be continuous
What are the two types of continuous data?
Interval and ratio
What is the formula for calculating alpha for multiple stats tests?
[1-(1-alpha)^n], n=number of tests done
What happens to alpha when multiple tests are performed on the same data?
Alpha = Type I error Risk of type I error increases because there is a risk of error associated with each test Error is increased with more tests
What is repeated testing? When is it necessary?
Conduct statistical test multiple times in same study because looking at multiple groups
Can % still be used to describe continuous data?
Yes ie. Hemoglobin A1c % or hematocrit %
What are the four types of data in inferential stats?
- Continuous (RI) - Discrete (NOD) - Parametric -Nonparametric
How does the SEM compare to the SD?
It would be smaller than the SD You would get closer to the real value each time
How can the exponential increase in alpha be avoided when comparing multiple means?
Use ANOVA (Analysis of variance)
Which standard deviation is reported for exam scores? Why?
1st standard deviation - This is the range within with most people fall - It excludes outliers
How many standard deviations are there typically in an average data set? What % of people fall in each?
3 - 68% in 1st SD - 95% in 2nd - 98% in 3rd
Why is it important to determine whether a study reports SD or SEM?
SEM will be narrower and may make the data look better
What is a dependent sample?
Groups/events are related
What are 4 types of dependent samples?
- Crossover: groups serve as own control, switch to other treatment - Pre-test/Post-test: in same group of people - Matched subjects: people enrolled based on similar characteristics - pick someone who matches a person's data - Twins
What is ordinal data?
Data with a specific order e.g. "I agree with this - 1) not at all 2) neutral 3) completely" Inherent rank orders such as: - Heart disease (1-4) - Implied severity (pregnancy category)
What are the four common types of distributions?
- Normal (bell or Gaussian) - Binomial - Poisson - Chi-squared
What are the two types of samples?
Independent and dependent
What are the two types of biostatistics?
Descriptive and inferential
What is nominal data?
Data with a name - Can describe using % (frequency/proportions) - i.e. gender, ethnicity
What is the dependent variable?
Observed result ie. Number of ventricular tachycardia episodes
What are three common tests for continuous data?
- Parametric tests - Student t-tests - Paired t-tests
When can the SD be replaced with the SEM?
Never Always inappropriate
A study compares means of drug effect in 3 groups: - Placebo - Drug A - Drug B They use t-test How many t-tests would they need to perform?
3 t-tests Placebo vs. Drug A Placebo vs. Drug B Drug A vs. Drug B Need to compare each group against one another
Which of the following is most appropriate to use a student t-test? - Average weight - Ethnicity - Gender
Average weight - Average is needed
What is needed for a t-test?
Means (for continuous data - interval, ratio)
What is continuous data?
Data that is on a sliding number scale - Can easily take the mean
What is an independent variable? Give 3 examples.
Variable being manipulated - Exposures - Treatments - Interventions
What is standard deviation?
- Measure of variability within your sample - Measure of the average distance from or dispersion around the mean
What is an independent sample?
Treatment groups are not related - Most studies ie. Coin flip
What is binomial distribution?
A distribution option when there are only two options (e.g. yes or no) that is measured by the number of responses times the number of successes
What is inferential statistics used for?
Drawing conclusions between two groups
What is the standard error of the mean (SEM)?
The standard deviation of all possible means drawn from a population - Measure of sampling variability - Multiple sampling results in a number of different means - Describes the spread of the distribution of those possible means - Reported as a +/- of some value Theoretical value ie. Summary of all SD's if took exam 1 multiple times
What are the two types of variables?
Independent and dependent
Section 5
(50 cards)
What can be determined from the methods?
Validity Generalizability (to your population)
What is the non-parametric equivalent to the t-test? Why?
Mann-Whitney U/Wilcoxon Rank Sum T-test = independent continuous data Mann Whit U = independent ordinal data
How common is if for information in the abstract to be inconsistent with the manuscript? - Specifically in large medical journals such as NEJM, JAMA< CMAJ, Ann Intern Med, Lancet, and BMJ?
18-68%
When can you NOT use a t-test?
When data is not normally distributed
How do study dropouts affect studies?
Decrease power
What is per protocol analysis?
Analysis of subjects who followed protocol, went to every follow up, took every dose - Exclude patients who did not finish the trial or did not follow protocol exactly May overestimate effect since not everyone does this
What is the propensity score method of imputing data?
Plug in average data
When is a chi squared test used?
With nominal data and when either no. of cells is >5 or total number of observations is >20
Which type of ITT is not appropriate for patients with Alzheimer's? - Dropouts count as failures - LOCF - Impute data - All are appropriate
LOCF Disease is progressive in its deterioration PT would decline and using LOCF would use data implying the patient had not declined as much as they actually have
Why do subjects dropout or are noncompliant?
- Adverse events - Die - If they think they didn't get active drug - Think drug isn't working
When is Wilcoxon Rank Sum used to analyze data?
Independent ordinal data (another name for Mann Whitney U) --> Don't need sum mann !
What should be present in the introduction?
- Background information (why this study) - Study objective should be clearly & concisely defined Objective should be measurable
Why does the following title demonstrate bias? The effect of a single dose of intravenous ketamine on suicidal ideation: a systematic review and individual participant data meta-analysis
Meta analysis conveys there is a high level of evidence Data from only 1 participant - Really not a meta analysis - It's pooled results of inconsistent scales
What is journal club? Who does it?
Reviewing articles and study results to ensure validity Healthcare professionals
Does ordinal data follow normal distribution?
No
What does a structured abstract look like?
- Objective - Research design - Clinical setting - Participants - Interventions - Primary outcome - Results - Conclusions
What two tests are commonly used in analyzing ordinal data?
Mann-Whitney U and Wilcoxon signed rank
Can interval/ratio data be analyzed using non-parametric tests?
Yes, if they are not normally distributed Interval/ratio (continuous data) are usually normally distributed and are analyzed using parametric tests though
When is Fisher's exact used?
Independent nominal data (non-parametric) When at least 1 cell has an expected value of ≤ 5 or total is ≤ 20 Use in situations similar to chi-square
A study analyzed the effect of drug vs. placebo on systolic BP in 2100 patients with no confounders. The primary outcome looked at response vs. no response. What type of stat test would be most appropriate? - Fisher's Exact Test - McNemar's Test - Chi-square - Mantel-Haenszel Test
Nominal, independent, non-parametric Fisher --> ind, 20 or less (too large sample) Chi --> ind, over 20 Mantel --> ind with confounder (no confounders here) McNemar --> dependent (not crossover, pre/post, matching, or twins Chi-square since over 20
What is a chi squared test?
A stats test used to analyze nominal data comparing expected count vs. observed
When can ANOVA be used?
When comparing the means of 2+ groups (usually 3+)
What does intention to treat refer to?
All patients who were enrolled and randomly allocated to treatment are included in the analysis and are analyzed in the groups to which they were randomized - Includes study failures
What do we do in preparation for noncompliance/dropouts?
Plan for dropouts when calculate the sample size - Anticipate 20% dropout higher for mental health trials
What are 2 ways to impute data?
- Propensity score - Sensitivity analysis
Drug A caused liver toxicity in 3 (0.3%) of patients, while placebo caused it in 1 (0.1%) of patients. What stat test is appropriate to use?
Not Mann Whit U --> non-parametric, equivalent of t-test use when comparing means not comparing means here Fisher exact --> nominal, small sample, comparing ADRs
What are problems with imputing data for study failures?
Mimics real life, but may instill bias since data is made up
When is Wilcoxon Signed Rank used to analyze data?
Dependent ordinal data (paired sets) - repeated measures on the same subjects
How are ordinal data sets analyzed?
Analyze the ranks
T/F: Mann Whitney U & Wilcoxon Signed Rank are used to analyze parametric data
F They analyze non-parametric data Ordinal data = ranked = not normally distributed
What type of test would be used where pre and post measurements of orthopnea (SOB) after treatment are compared?
Dependent since same group of patients before and after Ranking orthopnea Dependent, ordinal data that is non-paremetric = Wilcoxon Signed Rank
What is the sensitivity analysis method of imputing data?
Do multiple ways to see if get similar results
What is the purpose of an abstract?
Allows the authors to tell you what they want you to know
What is chi-square used for?
- Nominal data (categorical/count data) - Independent sample - Non-parametric (all discrete is non-parametric) - For rates, proportions, percentages (since nominal data can only be looked at this way - cannot take mean of a category) ie. gender
What is the most common use of the Fisher's exact?
Comparing ADRs
What type of test would be used when comparing feelings regarding implementing DASH diet before and after counseling?
Dependent (same group) before/after Ranking feelings about it = ordinal Dependent, ordinal data (so is non-parametric) = Wilcoxon Signed Rank
What kind of tests are conducted for ordinal/rank data?
Non-parametic - Data is not normally distributed - Based on assigned ranks: smallest score is assigned 1, etc.
When is McNemar used?
Analyzing dependent (paired) nominal data (e.g. attitude before and after a study)
What is LOCF?
Last observation carry forward, a strategy for dealing with dropouts Educated fudge factor
What is Mantel-Haneszel?
A variation of chi^2 used to associate 2 variables that are confounded by 1 variable Independent nominal data (non-parametric)
What type of test would be used to analyze an arbitrary (random) scale where a change between 2 units in the scale is not constant (such as NYHA HF classes or Likert scales)?
Non-parametric test (does not have a normal distribution)
What would be the null hypothesis in comparing stroke risk in normal cholesterol vs. elevated cholesterol? What would you use/do to analyze this?
Ho: Elevated cholesterol risk = Normal cholesterol risk for stroke - Nominal/categorical - yes/no - Independent samples (groups not dependent on one another) Use chi-square Use 2 x 2 contingency table: calculate expected cell values - Calculate % total strokes that occurred - Trying to reject null that elevated cholesterol and normal cholesterol have same stroke risk - Calc. expected stroke rate for elev. cholesterol and normal cholesterol using actual stroke rate (assumes same risk across both groups - null hypoth) - Stroke and no stroke would equal each other
What test is used to analyze dependent nominal data?
McNemar
What two tests can be used to analyze independent nominal data?
Chi-squared or fisher exact
How do noninferiority trials deal with dropouts?
Can bias toward the experimental group being non-inferior Not as much of a problem: presenting both per protocol and intent to treat should look relatively similar
When should chi-square be used in terms of amount of data? (number of cells, table size)
Expected cell values > 5 Total table size > 20
What is another name of the Mann-Whitney U test?
Wilcoxon Rank Sum
What is the format of a manuscript?
- Title - Authors - Abstract - Introduction - Methods - Results - Discussion/Conclusion - References - Acknowledgements - Conflicts of Interest
What does a non-structured abstract format look like?
- Purpose - Research design - Methodology - Results - Conclusion - Recommendation Typically paragraph form Not wrong, just makes it harder to read
When is Mann Whitney U used to analyze data?
Independent ordinal data
Section 6
(50 cards)
What is type I error? When is it set? What is the value and what is it referred to as?
Alpha Set prior to experiment Critical value: p < 0.05
What might it mean if the secondary outcomes are presented before the primary outcomes?
Primary outcome results were probably not so impressive - they are really reaching to demonstrate a difference
Why might RRR be used?
Presents the reduction as larger than it really is Makes results seem larger
What is included in the methods?
- Defines the type of study - Includes criteria to assess the investigator's ability to control for bias: blinding, randomization (blocking, stratification, permuted, stratification)
What are the 3 general types of clinical research?
- Observational - Quasi-Experimental - Randomized Clinical Trial
What is a randomized clinical trial?
Use random assignment to help control for confounders - even ones we don't yet know to exist
What are the 3 comparisons when comparing a drug vs. placebo? What are these referred to as?
1. Drug vs. placebo (A term) 2. Both treatments over time (B term) 3. Whether 1 drug works better than another over time (AB)
What is relative risk vs. absolute risk?
Risk of the event relative to the exposure vs. Risk between group differences in event rate
What is presented in the results? How should it be presented in relation to the methods section?
- Baseline characteristics: differences between groups at baseline Should be representative of the general population Should also mirror layout of methods section
Calculate RRR for a new drug that reduces risk from 0.4% (CER) to 0.3% (EER).
RRR = (CER - EER) / CER = (0.4 - 0.3) / 0.4 = 0.1 / 0.4 = 25% Only 0.1% change but RRR reports it as a 25% reduction
Dependent variables must be ___________________ in order to use ANOVA, a parametric statistical test.
Interval level Normally distributed
How might bias be present in the results?
Might scale the graphs Scale to show greater degree of between group differences
What is normally distributed data vs. not?
Normally distributed data = symmetric bell curve Not normally distributed: - Skewed to left (tail on left) - Skewed to right (tail on right)
What is this an example of? Give insulin to both placebo and intervention group on top of the drug you're testing - A1c should be similar
Stratification method --> ensure groups are equal in all realms besides intervention
Where in the methods can you detect bias?
- Appropriate intervention/not - Who is blinded: patients, data collection, data analyses, etc. - Are stratification methods appropriate: used to create equal groups
What does it suggest if the methods section is short?
Probably leaving out information
What should be done if your data is not normally distributed?
Transform the data or use non-parametric tests
How do you perform a Bonferroni procedure?
Each test alpha level = overall alpha level / # of tests Ex. 0.05/5 tests = 0.01 criteria for each test
Which is more important, type I or II error
Both equally important but type II is often downplayed
What is MANOVA? When is it used?
Multivariate analysis of variance Especially useful when dependent variables are highly correlated
What is an independent variable?
Groups we are interested in comparing Ex. drug group
What is the problem with comparing a new drug to a lesser used agent?
Inappropriate intervention --> doing this so their data cannot be extrapolated to 1st line/commonly used agent
What would be the 3 comparisons analyzed in a study looking at a drug to increase body weight vs. placebo? Which do we care about?
1. Drug vs. placebo over all time (if they are different collectively throughout the study) A term/Tx 2. Drug groups combined (placebo and drug collectively) at different times (whether all groups over time get heavier - which we are not trying to prove) B term/Time 3. Group time interaction (is there a difference between groups over time) AB/Group x Time interaction We care about whether the group over time interaction is significant (AB/Group x Time)
What is included in the discussion?
Author's conclusions & interpretation of results - Written persuasively - Statistical vs. clinical significance Limitations should be addressed but may not be all inclusive - won't point out issues Should answer the research question - may not have a direct answer
What do we look for in the one independent one repeated measure ANOVA results?
Make sure the p values for drug, time, and drug x time are significant (p < 0.05) Time: everyone got bigger over time Drug x Time: there is a difference in groups over time --> drug groups are different (1 or more groups better) over tome - what we want (the drug works)
What is a Quasi-Experimental Study?
Similar to true experiment but without random assignment Look at epidemiological data - not assigning people to groups Forced to do a quasi-experiment because can't force people to be male vs. female or smoker vs. non-smoker
What is a covariate?
A variable that is possibly predictive of the outcome of a study
What is the difference between when a t-test vs ANOVA should be used?
Comparing 2 groups: t-test More than 2 groups: ANOVA
What is the accepted standard for alpha? Beta? How much greater is the chance of one over the other?
Alpha = 0.05 (95% CI) Beta = 0.20 (80% power) 4 x greater chance of having a type II error
When is ANOVA used?
To compare differences in 2 or more groups with data that is - Normal - Interval level (continuous, does not contain an absolute zero) ie. year/time, ˚F
What are important guidance questions to consider for every article?
- Is this clinically relevant - Does this research add to the medical literature - Research objective relevant to practice - Study design appropriate to address research question - Use appropriate experimental or observational design - Study is an improvement over previous designs - Recruitment & selection of study sample are explained clearly - How sample was recruited/selected - Sample relevant to research objective - Study methods appropriately explained - Methods of randomization or observation explained - Any methods used to reduce bias - Endpoints relevant for patient care - Clinical/direct vs. surrogate vs. composite - Statistical analyses appropriate for the study - Appropriate for the design - Sample size adjusted for research questions - What are ket research findings - Is it stat significant - How large is the effect - Clinical vs. statistical sig - Can use NNT/NNH to evaluate - Are findings consistent with clinical rationale & previous literature - Any study limitations (in sample, design, analysis) - Can findings be generalized to other populations - Should findings be incorporated into practice
How is power affected if p is made more rigid: p < 0.001
Power is decreased Saying you're more confident there is not a type I error so decreased power
What is a covariate?
Factors to adjust for A baseline covariate is used when groups are different at baseline - we use a covariate (when we adjust the 2 groups statistically) so they score similarly Ex. 1 group starts off sicker than another --> adjust the 2 groups
What is type II error? What type of analysis does it warrant?
Beta Power analysis (ability to find a real effect if there is one: 1-beta)
How do we protect the overall experiment-wide alpha level when we have multiple dependent variables? (protect form type I error by chance)
Bonferroni procedure
What should you look for to identify bias in the inclusion/exclusion criteria?
- Overzealous exclusions - Vague criteria (renal disease) - Missing criteria
How can you determine generalizability?
Look at what patients make up the population of the study Inclusion & exclusion criteria
Which should a study use to present data - relative or absolute risk?
Absolute risk Relative risk is a limitation - its should NOT be used to present data
What kind of ANOVA is the most common in clinical trials?
ANOVA where a between group factor and where a repeated measures (within group) factor are present
If significant, what happens in ANOVA over: Time? Drug between group? Drug x Time?
Time: all groups changed (hopefully improve) over time (regardless of the treatment) Drug b/w Group: groups are different throughout the whole study - not an improvement for 1 drug over another - Error term for drug group Drug x Time: Drug groups are different (1 or more groups better) over the time of the study (what you want = drug works) - Error term for Drug * Time
How would observational studies progress in clinical research?
Start off with case study/series Start off with observational studies, if see a patient seems to do better with a drug (case study), then do it with a group of patients (case series), then a clinical trial comparing effect in 1 group vs. another
What are other considerations that should be made?
- Who published - Who wrote manuscript - Where conducted - Who funded
How can power be increased? (3)
- Increased sample size - Increased effect - Decreased error
What is a dependent variable?
Variable of interest What changes depending on the factors/treatments Can be any data type
What is included in the methods?
- Population - Study design - Interventions - Endpoints - Statistical analysis Who What Where When Why How
What is considered to be a true experiment?
Randomized clinical trial
What should be done in your initial assessment?
- First read article for a general impression - Make notes in margin - Put down, come back, focus on each section
What is a run-in period?
Method of excluding patients before the trial commences Try to pick off people that don't improve the results - limit to perfect population ie. Exclude patients who aren't adherent
What is ANOVA?
Analysis of variance
What is the equation for relative risk reduction (RRR)?
(EER - CER) / CER EER = experimental event rate CER = controlled event rate RRR = relative decrease in the risk of an adverse event in the exposed group compared to an unexposed group
Section 7
(50 cards)
How do you calculate the number of events avoided per 100 persons exposed?
ARR x 100
In a study comparing chicken soup and orange juice for the common cold over 2 weeks, what ANOVA term would be significant? A. Group B. Time C. AB Group x Time D. All but group x time E. None
B. Time Group x Time is not significant 1 group doesn't get better than another The 2 treatments are not different at all - so interaction would not be significant
What is poor adherence considered to be in a effectiveness trial: - Design flaw or - Outcome measure
Outcome measure
What is great internal validity?
Traditional RCTs that yield convincing, replicated results that at times do not seem to relate to the patients we treat & the outcomes we get
What is MMRM?
Mixed-effects models repeated measures A type of ANOVA that predicts what the missing values would be in a clinical trial Often used in lieu of LSCF/LOCF
What must be defined when starting a cohort study?
Exposed, unexposed, outcome, and hypothesis
What is placebo wash in?
A method of giving everyone placebo and removing people who have a therapeutic response Method to get rid of placebo-responders - don't include in trial
What is a moving average?
Take average of previous value and current value and import the average in between for the missing data point
What is an enriched trial design?
Criticized as a way to guarantee a positive result in a trial Way to preselect a study population that is more likely to respond to the proposed therapy Patients enter an open phase; responders enter the randomized discontinuation trial where time to failure is compared (by an event related analysis usually) Some stabilize on one drug, some stabilize patients on both agents (better, but still problematic) Only using people who responded to the drug and see differences
What is an effectiveness study?
A RCT to see if a drug works in daily clinical use Addresses the disconnect between clinical trial results & results in daily practice
What is NNH?
Number needed to harm Number of patients who would have to be treated with experimental Tx to incur 1 additional adverse event than if the control Tx had been used
What is the problem with interpreting non-inferiority trials?
Issue of interpreting no difference between active drugs in poorly designed clinical trials as a win Goal: drugs are the same
What is a washout?
Period in which a patient does not receive an active drug Don't always work: - Benzodiazepines suppress the withdrawal period
What is LSCF/LOCF?
Analyze data in a manner which includes the last score obtained as if data was collected
What is a good way to evaluate the reputableness of a journal?
Is there more text than ads? Is there an editorial process?
When is it questionable when industry sponsors a study?
When they are involved in data analysis
What is an efficacy study?
A study to see if the treatment works in a study population, used to get approval and is not always perfectly attributable to real life
What are 3 approaches to accounting for dropouts?
- Last score carried forward (LSCF) or Last observation carried forward (LOCF) - Mixed-effects Models Repeated Measures (MMRM) - Moving Average
What type of study can be used to explore a new hypothesis?
Cross-sectional, case-control
What is peer-review?
People in specialty fields pre-screen articles, make comments, send back to author, author responds, and article is sent back to reviewers
What is a typical setting? Why should this be used in an effectiveness trial?
More reflective of real world settings in which we use most medications (office settings, primary care/mental health clinics, nursing homes) Reflects initial care facilities available to a diverse population with the condition of interest Differs from efficacy studies which are usually done in premier institutions in large tertiary care referral settings - often not reflective of real world
What type of study can be used to confirm an association?
Experiment
What type of study is described below: 10 week RCT is performed to see if Wunderdrug is effective for schizophrenia. Assessed weekly. Inclusions: subjects 18-65 who meet RDC criteria for schizophrenia. Exclusions: schizoaffective disorder, other psychiatric disorders, history of substance or alcohol abuse, non-adherence, suicidal ideation, serious medical disorders, treatment resistance, use of other medications, etc. Many will come, few will be included. Fewer will make it through the beginning of the study.
EFFICACY study - Many exclusions - Shorter period of time - Less realistic population
What are the two types of cohort study?
Retrospective and prospective
How do observed health outcomes differ in efficacy trials vs. effectiveness trials?
Efficacy: - Evaluate symptom scores, lab data, time to disease recurrence - Health outcomes are NOT the primary outcome measure (functional capacity, quality of life, mortality, stopping of Tx) Effectiveness: - Health outcomes of the condition of interest is the principal dependent variable & we look at this for a LONGER time
What is the efficacy-effectiveness gap?
Discrepancy between what is found in efficacy RCTs and the results observed in common settings and in routine clinical practice
Do you want a large or small NNH?
Large Have to treat more patients to harm 1
What does "a trend toward significance" mean?
A persuasive tactic when discussing results that if more patients were enrolled, significance would have been reached, which cannot be assumed
What type of study can be used to explore newly claimed associations?
Case-control (replication), cohort (evidence towards causation)
Why do multicenter trials often have individual ANOVAs done?
To see if there is a difference between sites
How is NNT calculated?
1/ARR
What is the difference between efficacy and effectiveness trials?
Efficacy trials have a lot more exclusion criteria and are shorter and may present with a gap to clinical care
What is a randomized controlled trial (RCT)?
Carefully designed, prospective, longitudinal study where patients are randomly assigned to treatment groups - Make the groups the same except for the variable of interest - Randomization does NOT always work - Use blinding to eliminate bias
What type of study is described below: A 2 year RCT is performed in patients with schizophrenia or schizoaffective disorder. Inclusions: subjects 18-65 meeting DSM-IV criteria for schizophrenia Exclusions: few to none
EFFECTIVENESS study - Few to no exclusions: don't get to choose perfect patient Longer
Which method of accounting for dropouts is the most commonly used nowadays?
MMRM (Mixed-effects Models Repeated Measures)
What is the dogma of a cohort study?
Health -> Exposure occurs -> Exposed and unexposed -> disease occurs -> diseased and non-diseased
What are the criteria of effectiveness trials?
- Population in primary care or typical setting - Less stringent eligibility criteria - Health outcomes used in principal analysis - Long study duration & clinically relevant treatment modalities - Uses objective scales to assess predefined adverse effects - Adequate sample size to assess a minimally important difference from patient perspective - Intention to treat analysis used and they are still RCT
What is number needed to treat (NNT)?
How many more patients you need to treat with experimental drug instead of standard or control to see one or more patient benefit
At what point in time does a cross-sectional study analyze data?
Present
What does time continuing on treatment as an index of treatment effectiveness refer to? What is the outcome?
Does the drug work more than the side effect burden? Does drug effective and tolerable? Time to patient discontinuation is the outcome Includes clinician's and patient's perception with time on treatment
Lifetime tardive dykinesia risk is ~1 in every 5 treated with typical antipsychotics while it is ~1 in 100 for atypical. What is the NNH for typical antipsychotics as compared to atypical antipsychotics for TD?
Typical: 1/5 Atypical: 1/100 1/5 = X/100 X = 20 Typical: 20% Atypical: 1% 20% - 1% = 19% 1/0.19 = 5.26 is lifetime NNH for TD typical compared to atypical
Which groups are often under-represented in RCTs?
Minority groups Children Women Patients with comorbid conditions
What is a multicenter parallel study?
A study where the trial is happening at multiple locations
What type of study can be used to explore a new idea?
Ecologic or cross-sectional study
What types of participants are usually included in RCTs?
Highly selective - Often include patients not representative of typical patient population Some trials only enroll 1 of every 68 patients screened
How is absolute risk reduction (ARR) calculated?
ARR = CER - EER CER = control event rate EER = experimental event rate
What does a smaller NNT indicate?
Bigger benefit of the experimental treatment
What is great external validity?
Case reports & series Our clinical experience Open clinical trials BUT they have little internal validity so results are often unreliable
What is the definition of a cohort study?
A study in which two or more groups of people that are free of outcome and that differ according to the extend of exposure are compared with respect to outcome incidence
Describe efficacy trials: - Population - Where conducted - Price - How ensure efficacy is legit - ITT/Per protocol? - What is the outcome - How analyzed (what test) - Goal
- Large populations with intended disease - Multi-center (with genetic, geographic variability) - Costly - Randomized, blinded for efficacy - Intention to treat concept (include dropouts, anyone treated) - Outcome is often measure like a scale total, BP - Analyzed via ANOVA - Goal: statistical & clinical superiority Demonstrates more effective than placebo or gold standard therapy Less toxic/more safe than gold standard therapy
Section 8
(44 cards)