power analysis online

  • Calculators

Flattr this

  • Sign In Email Password Send

Power and Sample Size .com

Free, Online, Easy-to-Use Power and Sample Size Calculators

no java applets, plugins, registration, or downloads ... just free

Go Straight to the Calculators »

Power? What Power?

Statistical power is a fundamental consideration when designing research experiments. It goes hand-in-hand with sample size. The formulas that our calculators use come from clinical trials, epidemiology, pharmacology, earth sciences, psychology, survey sampling ... basically every scientific discipline.

Learn More »

We take the time to compare our calculators' output to published results. Moreover, our computation code is open-source, mathematical formulas are given for each calculator, and we even provide R code for the adventurous. The validation examples are cited at the bottom of each calculator's page.

See Validations »

By Nerds, For Nerds

We are a group of analysts and researchers who design experiments, studies, and surveys on a regular basis. This site grew out of our own needs. We have benefited from the wealth of knowledge and tools available online. This is our own small way of giving back to the analytics community.

About This Site »

Power & Sample Size Calculator

Use this advanced sample size calculator to calculate the sample size required for a one-sample statistic, or for differences between two proportions or means (two independent samples). More than two groups supported for binomial data. Calculate power given sample size, alpha, and the minimum detectable effect (MDE, minimum effect of interest).

Experimental design

Data parameters

Related calculators

  • Using the power & sample size calculator

Parameters for sample size and power calculations

Calculator output.

  • Why is sample size determination important?
  • What is statistical power?

Post-hoc power (Observed power)

  • Sample size formula
  • Types of null and alternative hypotheses in significance tests
  • Absolute versus relative difference and why it matters for sample size determination

    Using the power & sample size calculator

This calculator allows the evaluation of different statistical designs when planning an experiment (trial, test) which utilizes a Null-Hypothesis Statistical Test to make inferences. It can be used both as a sample size calculator and as a statistical power calculator . Usually one would determine the sample size required given a particular power requirement, but in cases where there is a predetermined sample size one can instead calculate the power for a given effect size of interest.

1. Number of test groups. The sample size calculator supports experiments in which one is gathering data on a single sample in order to compare it to a general population or known reference value (one-sample), as well as ones where a control group is compared to one or more treatment groups ( two-sample, k-sample ) in order to detect differences between them. For comparing more than one treatment group to a control group the sample size adjustments based on the Dunnett's correction are applied. These are only approximately accurate and subject to the assumption of about equal effect size in all k groups, and can only support equal sample sizes in all groups and the control. Power calculations are not currently supported for more than one treatment group due to their complexity.

2. Type of outcome . The outcome of interest can be the absolute difference of two proportions (binomial data, e.g. conversion rate or event rate), the absolute difference of two means (continuous data, e.g. height, weight, speed, time, revenue, etc.), or the relative difference between two proportions or two means (percent difference, percent change, etc.). See Absolute versus relative difference for additional information. One can also calculate power and sample size for the mean of just a single group. The sample size and power calculator uses the Z-distribution (normal distribution) .

3. Baseline The baseline mean (mean under H 0 ) is the number one would expect to see if all experiment participants were assigned to the control group. It is the mean one expects to observe if the treatment has no effect whatsoever.

4. Minimum Detectable Effect . The minimum effect of interest, which is often called the minimum detectable effect ( MDE , but more accurately: MRDE, minimum reliably detectable effect) should be a difference one would not like to miss , if it existed. It can be entered as a proportion (e.g. 0.10) or as percentage (e.g. 10%). It is always relative to the mean/proportion under H 0 ± the superiority/non-inferiority or equivalence margin. For example, if the baseline mean is 10 and there is a superiority alternative hypothesis with a superiority margin of 1 and the minimum effect of interest relative to the baseline is 3, then enter an MDE of 2 , since the MDE plus the superiority margin will equal exactly 3. In this case the MDE (MRDE) is calculated relative to the baseline plus the superiority margin, as it is usually more intuitive to be interested in that value.

If entering means data, one needs to specify the mean under the null hypothesis (worst-case scenario for a composite null) and the standard deviation of the data (for a known population or estimated from a sample).

5. Type of alternative hypothesis . The calculator supports superiority , non-inferiority and equivalence alternative hypotheses. When the superiority or non-inferiority margin is zero, it becomes a classical left or right sided hypothesis, if it is larger than zero then it becomes a true superiority / non-inferiority design. The equivalence margin cannot be zero. See Types of null and alternative hypothesis below for an in-depth explanation.

6. Acceptable error rates . The type I error rate, α , should always be provided. Power, calculated as 1 - β , where β is the type II error rate, is only required when determining sample size. For an in-depth explanation of power see What is statistical power below. The type I error rate is equivalent to the significance threshold if one is doing p-value calculations and to the confidence level if using confidence intervals.

The sample size calculator will output the sample size of the single group or of all groups, as well as the total sample size required. If used to solve for power it will output the power as a proportion and as a percentage.

    Why is sample size determination important?

While this online software provides the means to determine the sample size of a test, it is of great importance to understand the context of the question, the "why" of it all.

Estimating the required sample size before running an experiment that will be judged by a statistical test (a test of significance, confidence interval, etc.) allows one to:

  • determine the sample size needed to detect an effect of a given size with a given probability
  • be aware of the magnitude of the effect that can be detected with a certain sample size and power
  • calculate the power for a given sample size and effect size of interest

This is crucial information with regards to making the test cost-efficient. Having a proper sample size can even mean the difference between conducting the experiment or postponing it for when one can afford a sample of size that is large enough to ensure a high probability to detect an effect of practical significance.

For example, if a medical trial has low power, say less than 80% (β = 0.2) for a given minimum effect of interest, then it might be unethical to conduct it due to its low probability of rejecting the null hypothesis and establishing the effectiveness of the treatment. Similarly, for experiments in physics, psychology, economics, marketing, conversion rate optimization, etc. Balancing the risks and rewards and assuring the cost-effectiveness of an experiment is a task that requires juggling with the interests of many stakeholders which is well beyond the scope of this text.

    What is statistical power?

Statistical power is the probability of rejecting a false null hypothesis with a given level of statistical significance , against a particular alternative hypothesis. Alternatively, it can be said to be the probability to detect with a given level of significance a true effect of a certain magnitude. This is what one gets when using the tool in "power calculator" mode. Power is closely related with the type II error rate: β, and it is always equal to (1 - β). In a probability notation the type two error for a given point alternative can be expressed as [1] :

β(T α ; μ 1 ) = P(d(X) ≤ c α ; μ = μ 1 )

It should be understood that the type II error rate is calculated at a given point, signified by the presence of a parameter for the function of beta. Similarly, such a parameter is present in the expression for power since POW = 1 - β [1] :

POW(T α ; μ 1 ) = P(d(X) > c α ; μ = μ 1 )

In the equations above c α represents the critical value for rejecting the null (significance threshold), d(X) is a statistical function of the parameter of interest - usually a transformation to a standardized score, and μ 1 is a specific value from the space of the alternative hypothesis.

One can also calculate and plot the whole power function, getting an estimate of the power for many different alternative hypotheses. Due to the S-shape of the function, power quickly rises to nearly 100% for larger effect sizes, while it decreases more gradually to zero for smaller effect sizes. Such a power function plot is not yet supported by our statistical software, but one can calculate the power at a few key points (e.g. 10%, 20% ... 90%, 100%) and connect them for a rough approximation.

Statistical power is directly and inversely related to the significance threshold. At the zero effect point for a simple superiority alternative hypothesis power is exactly 1 - α as can be easily demonstrated with our power calculator. At the same time power is positively related to the number of observations, so increasing the sample size will increase the power for a given effect size, assuming all other parameters remain the same.

Power calculations can be useful even after a test has been completed since failing to reject the null can be used as an argument for the null and against particular alternative hypotheses to the extent to which the test had power to reject them. This is more explicitly defined in the severe testing concept proposed by Mayo & Spanos (2006).

Computing observed power is only useful if there was no rejection of the null hypothesis and one is interested in estimating how probative the test was towards the null . It is absolutely useless to compute post-hoc power for a test which resulted in a statistically significant effect being found [5] . If the effect is significant, then the test had enough power to detect it. In fact, there is a 1 to 1 inverse relationship between observed power and statistical significance, so one gains nothing from calculating post-hoc power, e.g. a test planned for α = 0.05 that passed with a p-value of just 0.0499 will have exactly 50% observed power (observed β = 0.5).

I strongly encourage using this power and sample size calculator to compute observed power in the former case, and strongly discourage it in the latter.

    Sample size formula

The formula for calculating the sample size of a test group in a one-sided test of absolute difference is:

sample size

where Z 1-α is the Z-score corresponding to the selected statistical significance threshold α , Z 1-β is the Z-score corresponding to the selected statistical power 1-β , σ is the known or estimated standard deviation, and δ is the minimum effect size of interest. The standard deviation is estimated analytically in calculations for proportions, and empirically from the raw data for other types of means.

The formula applies to single sample tests as well as to tests of absolute difference between two samples. A proprietary modification is employed when calculating the required sample size in a test of relative difference . This modification has been extensively tested under a variety of scenarios through simulations.

    Types of null and alternative hypotheses in significance tests

When doing sample size calculations, it is important that the null hypothesis (H 0 , the hypothesis being tested) and the alternative hypothesis is (H 1 ) are well thought out. The test can reject the null or it can fail to reject it. Strictly logically speaking it cannot lead to acceptance of the null or to acceptance of the alternative hypothesis. A null hypothesis can be a point one - hypothesizing that the true value is an exact point from the possible values, or a composite one: covering many possible values, usually from -∞ to some value or from some value to +∞. The alternative hypothesis can also be a point one or a composite one.

In a Neyman-Pearson framework of NHST (Null-Hypothesis Statistical Test) the alternative should exhaust all values that do not belong to the null, so it is usually composite. Below is an illustration of some possible combinations of null and alternative statistical hypotheses: superiority, non-inferiority, strong superiority (margin > 0), equivalence.

types of statistical hypotheses

All of these are supported in our power and sample size calculator.

Careful consideration has to be made when deciding on a non-inferiority margin, superiority margin or an equivalence margin . Equivalence trials are sometimes used in clinical trials where a drug can be performing equally (within some bounds) to an existing drug but can still be preferred due to less or less severe side effects, cheaper manufacturing, or other benefits, however, non-inferiority designs are more common. Similar cases exist in disciplines such as conversion rate optimization [2] and other business applications where benefits not measured by the primary outcome of interest can influence the adoption of a given solution. For equivalence tests it is assumed that they will be evaluated using a two one-sided t-tests (TOST) or z-tests, or confidence intervals.

Note that our calculator does not support the schoolbook case of a point null and a point alternative, nor a point null and an alternative that covers all the remaining values. This is since such cases are non-existent in experimental practice [3][4] . The only two-sided calculation is for the equivalence alternative hypothesis, all other calculations are one-sided (one-tailed) .

    Absolute versus relative difference and why it matters for sample size determination

When using a sample size calculator it is important to know what kind of inference one is looking to make: about the absolute or about the relative difference, often called percent effect, percentage effect, relative change, percent lift, etc. Where the fist is μ 1 - μ the second is μ 1 -μ / μ or μ 1 -μ / μ x 100 (%). The division by μ is what adds more variance to such an estimate, since μ is just another variable with random error, therefore a test for relative difference will require larger sample size than a test for absolute difference. Consequently, if sample size is fixed, there will be less power for the relative change equivalent to any given absolute change.

For the above reason it is important to know and state beforehand if one is going to be interested in percentage change or if absolute change is of primary interest. Then it is just a matter of fliping a radio button.

    References

1 Mayo D.G., Spanos A. (2010) – "Error Statistics", in P. S. Bandyopadhyay & M. R. Forster (Eds.), Philosophy of Statistics, (7, 152–198). Handbook of the Philosophy of Science . The Netherlands: Elsevier.

2 Georgiev G.Z. (2017) "The Case for Non-Inferiority A/B Tests", [online] https://blog.analytics-toolkit.com/2017/case-non-inferiority-designs-ab-testing/ (accessed May 7, 2018)

3 Georgiev G.Z. (2017) "One-tailed vs Two-tailed Tests of Significance in A/B Testing", [online] https://blog.analytics-toolkit.com/2017/one-tailed-two-tailed-tests-significance-ab-testing/ (accessed May 7, 2018)

4 Hyun-Chul Cho Shuzo Abe (2013) "Is two-tailed testing for directional research hypotheses tests legitimate?", Journal of Business Research 66:1261-1266

5 Lakens D. (2014) "Observed power, and what to do if your editor asks for post-hoc power analyses" [online] http://daniellakens.blogspot.bg/2014/12/observed-power-and-what-to-do-if-your.html (accessed May 7, 2018)

Cite this calculator & page

If you'd like to cite this online calculator resource and information as provided on the page, you can use the following citation: Georgiev G.Z., "Sample Size Calculator" , [online] Available at: https://www.gigacalculator.com/calculators/power-sample-size-calculator.php URL [Accessed Date: 01 Apr, 2024].

Our statistical calculators have been featured in scientific papers and articles published in high-profile science journals by:

springer

The author of this tool

Georgi Z. Georgiev

     Statistical calculators

power analysis online

User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

  • Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
  • Duis aute irure dolor in reprehenderit in voluptate
  • Excepteur sint occaecat cupidatat non proident

Keyboard Shortcuts

S.5 power analysis, why is power analysis important section  .

Consider a research experiment where the p -value computed from the data was 0.12. As a result, one would fail to reject the null hypothesis because this p -value is larger than \(\alpha\) = 0.05. However, there still exist two possible cases for which we failed to reject the null hypothesis:

  • the null hypothesis is a reasonable conclusion,
  • the sample size is not large enough to either accept or reject the null hypothesis, i.e., additional samples might provide additional evidence.

Power analysis is the procedure that researchers can use to determine if the test contains enough power to make a reasonable conclusion. From another perspective power analysis can also be used to calculate the number of samples required to achieve a specified level of power.

Example S.5.1

Let's take a look at an example that illustrates how to compute the power of the test.

Let X denote the height of randomly selected Penn State students. Assume that X is normally distributed with unknown mean \(\mu\) and a standard deviation of 9. Take a random sample of n = 25 students, so that, after setting the probability of committing a Type I error at \(\alpha = 0.05\), we can test the null hypothesis \(H_0: \mu = 170\) against the alternative hypothesis that \(H_A: \mu > 170\).

What is the power of the hypothesis test if the true population mean were \(\mu = 175\)?

\[\begin{align}z&=\frac{\bar{x}-\mu}{\sigma / \sqrt{n}} \\ \bar{x}&= \mu + z \left(\frac{\sigma}{\sqrt{n}}\right) \\ \bar{x}&=170+1.645\left(\frac{9}{\sqrt{25}}\right) \\ &=172.961\\ \end{align}\]

So we should reject the null hypothesis when the observed sample mean is 172.961 or greater:

\[\begin{align}\text{Power}&=P(\bar{x} \ge 172.961 \text{ when } \mu =175)\\ &=P\left(z \ge \frac{172.961-175}{9/\sqrt{25}} \right)\\ &=P(z \ge -1.133)\\ &= 0.8713\\ \end{align}\]

and illustrated below:

Two overlapping normal distributions with means of 170 and 175. The power of 0.871 is show on the right curve.

In summary, we have determined that we have an 87.13% chance of rejecting the null hypothesis \(H_0: \mu = 170\) in favor of the alternative hypothesis \(H_A: \mu > 170\) if the true unknown population mean is, in reality, \(\mu = 175\).

Calculating Sample Size Section  

If the sample size is fixed, then decreasing Type I error \(\alpha\) will increase Type II error \(\beta\). If one wants both to decrease, then one has to increase the sample size.

To calculate the smallest sample size needed for specified \(\alpha\), \(\beta\), \(\mu_a\), then (\(\mu_a\) is the likely value of \(\mu\) at which you want to evaluate the power.

Let's investigate by returning to our previous example.

Example S.5.2

Let X denote the height of randomly selected Penn State students. Assume that X is normally distributed with unknown mean \(\mu\) and standard deviation 9. We are interested in testing at \(\alpha = 0.05\) level , the null hypothesis \(H_0: \mu = 170\) against the alternative hypothesis that \(H_A: \mu > 170\).

Find the sample size n that is necessary to achieve 0.90 power at the alternative μ = 175.

\[\begin{align}n&= \dfrac{\sigma^2(Z_{\alpha}+Z_{\beta})^2}{(\mu_0−\mu_a)^2}\\ &=\dfrac{9^2 (1.645 + 1.28)^2}{(170-175)^2}\\ &=27.72\\ n&=28\\ \end{align}\]

In summary, you should see how power analysis is very important so that we are able to make the correct decision when the data indicate that one cannot reject the null hypothesis. You should also see how power analysis can also be used to calculate the minimum sample size required to detect a difference that meets the needs of your research.

Post-hoc Power Calculator

Evaluate statistical power of an existing study, clincalc.com » statistics » post-hoc power calculator, study group design.

Two independent study groups

One study group vs. population

Primary Endpoint

Dichotomous (yes/no)

Continuous (means)

Statistical Parameters

About this calculator.

This calculator uses a variety of equations to calculate the statistical power of a study after the study has been conducted. 1

"Power" is the ability of a trial to detect a difference between two different groups. If a trial has inadequate power, it may not be able to detect a difference even though a difference truly exists. This false conclusion is called a type II error.

Just like sample size calculation , statistical power is based on the baseline incidence of an outcome, the population variance, the treatment effect size, alpha, and the sample size of a study.

The Dangers of Post-Hoc Analysis

Post-hoc power analysis has been criticized as a means of interpreting negative study results. 2 Because post-hoc analyses are typically only calculated on negative trials (p ≥ 0.05), such an analysis will produce a low post-hoc power result, which may be misinterpreted as the trial having inadequate power.

As an alternative to post-hoc power, analysis of the width and magnitude of the 95% confidence interval (95% CI) may be a more appropriate method of determining statistical power.

Sample Size Calculation

To calculate an adequate sample size for a future or planned trial, please visit the sample size calculator .

References and Additional Reading

  • Rosner B. Fundamentals of Biostatistics . 7th ed. Boston, MA: Brooks/Cole; 2011.
  • Levine M, Ensom MH. Post hoc power analysis: an idea whose time has passed? Pharmacotherapy . 2001;21(4):405-9. PMID 11310512

Related Calculators

  • Sample Size Calculator

New and Popular

Cite this page.

Show AMA citation

Sample Size Calculators

For designing clinical research.

power analysis online

  • Calculators
  • CI for proportion
  • CI for mean
  • Means - effect size
  • Means - sample size
  • Proportions - effect size
  • Proportions - sample size
  • CI for proportion - sample size
  • Survival analysis - sample size
  • CI for risk ratio
  • More calculators...
  • Calculator finder
  • About calculating sample size

If you are a clinical researcher trying to determine how many subjects to include in your study or you have another question related to sample size or power calculations, we developed this website for you. Our approach is based on Chapters 5 and 6 in the 4th edition of Designing Clinical Research (DCR-4), but the material and calculators provided here go well beyond an introductory textbook on clinical research methods.

This project was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Numbers UL1 TR000004 and UL1 TR001872. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Please cite this site wherever used in published work:

Kohn MA, Senyak J. Sample Size Calculators [website]. UCSF CTSI. 11 January 2024. Available at https://www.sample-size.net/ [Accessed 01 April 2024]

This site was last updated on January 11, 2024.

Statistical considerations for clinical trials and scientific experiments

Find sample size, power or the minimal detectable difference for parallel studies , crossover studies , or studies to find associations between variables , where the dependent variable is Success or Failure , a Quantitative Measurement , or a time to an event such as a survival time.

  • 2010-May-14: move to JavaScript-based Quantitative Measurement tools.
  • 2015-Jan-07: move to JavaScript-based Fisher Exact test.

These calculations are based on assumptions which may not be true for the clinical trial you are planning. We do not guarantee the accuracy of these calculations or their suitability for your trial. We suggest that you speak to a biostatistical consultant when planning a clinical trial. Please contact us if you have any questions or problems using this software.

The author of these tools is David A. Schoenfeld, Ph.D. ( [email protected] ), with support from the Massachusetts General Hospital Mallinckrodt General Clinical Research Center , Research Resources Division, National Institutes of Health, National Center for Advancing Translational Sciences.

Definitions

  • Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Institute for Digital Research and Education

Introduction to Power Analysis

This seminar treats power and the various factors that affect power on both a conceptual and a mechanical level. While we will not cover the formulas needed to actually run a power analysis, later on we will discuss some of the software packages that can be used to conduct power analyses.

OK, let’s start off with a basic definition of what a power is.  Power is the probability of detecting an effect, given that the effect is really there.  In other words, it is the probability of rejecting the null hypothesis when it is in fact false.  For example, let’s say that we have a simple study with drug A and a placebo group, and that the drug truly is effective; the power is the probability of finding a difference between the two groups.  So, imagine that we had a power of .8 and that this simple study was conducted many times.  Having power of .8 means that 80% of the time, we would get a statistically significant difference between the drug A and placebo groups.  This also means that 20% of the times that we run this experiment, we will not obtain a statistically significant effect between the two groups, even though there really is an effect in reality.

There are several of reasons why one might do a power analysis.  Perhaps the most common use is to determine the necessary number of subjects needed to detect an effect of a given size.  Note that trying to find the absolute, bare minimum number of subjects needed in the study is often not a good idea.  Additionally, power analysis can be used to determine power, given an effect size and the number of subjects available.  You might do this when you know, for example, that only 75 subjects are available (or that you only have the budget for 75 subjects), and you want to know if you will have enough power to justify actually doing the study.  In most cases, there is really no point to conducting a study that is seriously underpowered.  Besides the issue of the number of necessary subjects, there are other good reasons for doing a power analysis.  For example, a power analysis is often required as part of a grant proposal.  And finally, doing a power analysis is often just part of doing good research.  A power analysis is a good way of making sure that you have thought through every aspect of the study and the statistical analysis before you start collecting data.

Despite these advantages of power analyses, there are some limitations.  One limitation is that power analyses do not typically generalize very well.  If you change the methodology used to collect the data or change the statistical procedure used to analyze the data, you will most likely have to redo the power analysis.  In some cases, a power analysis might suggest a number of subjects that is inadequate for the statistical procedure.  For example, a power analysis might suggest that you need 30 subjects for your logistic regression, but logistic regression, like all maximum likelihood procedures, require much larger sample sizes.  Perhaps the most important limitation is that a standard power analysis gives you a “best case scenario” estimate of the necessary number of subjects needed to detect the effect.  In most cases, this “best case scenario” is based on assumptions and educated guesses.  If any of these assumptions or guesses are incorrect, you may have less power than you need to detect the effect.  Finally, because power analyses are based on assumptions and educated guesses, you often get a range of the number of subjects needed, not a precise number.  For example, if you do not know what the standard deviation of your outcome measure will be, you guess at this value, run the power analysis and get X number of subjects.  Then you guess a slightly larger value, rerun the power analysis and get a slightly larger number of necessary subjects.  You repeat this process over the plausible range of values of the standard deviation, which gives you a range of the number of subjects that you will need.

After all of this discussion of power analyses and the necessary number of subjects, we need to stress that power is not the only consideration when determining the necessary sample size.  For example, different researchers might have different reasons for conducting a regression analysis.  One might want to see if the regression coefficient is different from zero, while the other wants to get a very precise estimate of the regression coefficient with a very small confidence interval around it.  This second purpose requires a larger sample size than does merely seeing if the regression coefficient is different from zero.  Another consideration when determining the necessary sample size is the assumptions of the statistical procedure that is going to be used.  The number of statistical tests that you intend to conduct will also influence your necessary sample size:  the more tests that you want to run, the more subjects that you will need.  You will also want to consider the representativeness of the sample, which, of course, influences the generalizability of the results.  Unless you have a really sophisticated sampling plan, the greater the desired generalizability, the larger the necessary sample size.  Finally, please note that most of what is in this presentation does not readily apply to people who are developing a sampling plan for a survey or psychometric analyses.

Definitions

Before we move on, let’s make sure we are all using the same definitions.  We have already defined power as the probability of detecting a “true” effect, when the effect exists.  Most recommendations for power fall between .8 and .9.  We have also been using the term “effect size”, and while intuitively it is an easy concept, there are lots of definitions and lots of formulas for calculating effect sizes.  For example, the current APA manual has a list of more than 15 effect sizes, and there are more than a few books mostly dedicated to the calculation of effect sizes in various situations.  For now, let’s stick with one of the simplest definitions, which is that an effect size is the difference of two group means divided by the pooled standard deviation.  Going back to our previous example, suppose the mean of the outcome variable for the drug A group was 10 and it was 5 for the placebo group.  If the pooled standard deviation was 2.5, we would have and effect size which is equal to (10-5)/2.5 = 2 (which is a large effect size).

We also need to think about “statistically significance” versus “clinically relevant”.  This issue comes up often when considering effect sizes. For example, for a given number of subjects, you might only need a small effect size to have a power of .9.  But that effect size might correspond to a difference between the drug and placebo groups that isn’t clinically meaningful, say reducing blood pressure by two points.  So even though you would have enough power, it still might not be worth doing the study, because the results would not be useful for clinicians.

There are a few other definitions that we will need later in this seminar.  A Type I error occurs when the null hypothesis is true (in other words, there really is no effect), but you reject the null hypothesis.  A Type II error occurs when the alternative hypothesis is correct, but you fail to reject the null hypothesis (in other words, there really is an effect, but you failed to detect it).  Alpha inflation refers to the increase in the nominal alpha level when the number of statistical tests conducted on a given data set is increased.

When discussing statistical power, we have four inter-related concepts: power, effect size, sample size and alpha.  These four things are related such that each is a function of the other three.  In other words, if three of these values are fixed, the fourth is completely determined (Cohen, 1988, page 14).  We mention this because, by increasing one, you can decrease (or increase) another.  For example, if you can increase your effect size, you will need fewer subjects, given the same power and alpha level.  Specifically, increasing the effect size, the sample size and/or alpha will increase your power.

While we are thinking about these related concepts and the effect of increasing things, let’s take a quick look at a standard power graph.  (This graph was made in SPSS Sample Power, and for this example, we’ve used .61 and 4 for our two proportion positive values.)

We like these kinds of graphs because they make clear the diminishing returns you get for adding more and more subjects.  For example, let’s say that we have only 10 subjects per group.  We can see that we have a power of about .15, which is really, really low.  We add 50 subjects per group, now we have a power of about .6, an increase of .45.  However, if we started with 100 subjects per group (power of about .8) and added 50 per group, we would have a power of .95, an increase of only .15.  So each additional subject gives you less additional power.  This curve also illustrates the “cost” of increasing your desired power from .8 to .9.

Knowing your research project

As we mentioned before, one of the big benefits of doing a power analysis is making sure that you have thought through every detail of your research project.

Now most researchers have thought through most, if not all, of the substantive issues involved in their research.  While this is absolutely necessary, it often is not sufficient.  Researchers also need to carefully consider all aspects of the experimental design, the variables involved, and the statistical analysis technique that will be used.  As you will see in the next sections of this presentation, a power analysis is the union of substantive knowledge (i.e., knowledge about the subject matter), experimental or quasi-experimental design issues, and statistical analysis.  Almost every aspect of the experimental design can affect power.  For example, the type of control group that is used or the number of time points that are collected will affect how much power you have.  So knowing about these issues and carefully considering your options is important.  There are plenty of excellent books that cover these issues in detail, including Shadish, Cook and Campbell (2002); Cook and Campbell (1979); Campbell and Stanley (1963); Brickman (2000a, 2000b); Campbell and Russo (2001); Webb, Campbell, Schwartz and Sechrest (2000); and Anderson (2001).

Also, you want to know as much as possible about the statistical technique that you are going to use.  If you learn that you need to use a binary logistic regression because your outcome variable is 0/1, don’t stop there; rather, get a sample data set (there are plenty of sample data sets on our web site) and try it out.  You may discover that the statistical package that you use doesn’t do the type of analysis that need to do.  For example, if you are an SPSS user and you need to do a weighted multilevel logistic regression, you will quickly discover that SPSS doesn’t do that (as of version 25), and you will have to find (and probably learn) another statistical package that will do that analysis.  Maybe you want to learn another statistical package, or maybe that is beyond what you want to do for this project.  If you are writing a grant proposal, maybe you will want to include funds for purchasing the new software.  You will also want to learn what the assumptions are and what the “quirks” are with this particular type of analysis.  Remember that the number of necessary subjects given to you by a power analysis assumes that all of the assumptions of the analysis have been met, so knowing what those assumptions are is important deciding if they are likely to be met or not.

The point of this section is to make clear that knowing your research project involves many things, and you may find that you need to do some research about experimental design or statistical techniques before you do your power analysis.

We want to emphasize that this is time and effort well spent.  We also want to remind you that for almost all researchers, this is a normal part of doing good research.  UCLA researchers are welcome and encouraged to come by walk-in consulting at this stage of the research process to discuss issues and ideas, check out books and try out software.

What you need to know to do a power analysis

In the previous section, we discussed in general terms what you need to know to do a power analysis.  In this section we will discuss some of the actual quantities that you need to know to do a power analysis for some simple statistics.  Although we understand very few researchers test their main hypothesis with a t-test or a chi-square test, our point here is only to give you a flavor of the types of things that you will need to know (or guess at) in order to be ready for a power analysis.

– For an independent samples t-test, you will need to know the population means of the two groups (or the difference between the means), and the population standard deviations of the two groups.  So, using our example of drug A and placebo, we would need to know the difference in the means of the two groups, as well as the standard deviation for each group (because the group means and standard deviations are the best estimate that we have of those population values).  Clearly, if we knew all of this, we wouldn’t need to conduct the study.  In reality, researchers make educated guesses at these values.  We always recommend that you use several different values, such as decreasing the difference in the means and increasing the standard deviations, so that you get a range of values for the number of necessary subjects.

In SPSS Sample Power, we would have a screen that looks like the one below, and we would fill in the necessary values.  As we can see, we would need a total of 70 subjects (35 per group) to have a power of .91 if we had a mean of 5 and a standard deviation of 2.5 in the drug A group, and a mean of 3 and a standard deviation of 2.5 in the placebo group.  If we decreased the difference in the means and increased the standard deviations such that for the drug A group, we had a mean of 4.5 and a standard deviation of 3, and for the placebo group a mean of 3.5 and a standard deviation of 3, we would need 190 subjects per group, or a total of 380 subjects, to have a power of .90.  In other words, seemingly small differences in means and standard deviations can have a huge effect on the number of subjects required.

Image t-test

– For a correlation, you need to know/guess at the correlation in the population.  This is a good time to remember back to an early stats class where they emphasized that correlation is a large N procedure (Chen and Popovich, 2002).  If you guess that the population correlation is .6, a power analysis would suggest (with an alpha of .05 and for a power of .8) that you would need only 16 subjects.  There are several points to be made here.  First, common sense suggests that N = 16 is pretty low.  Second, a population correlation of .6 is pretty high, especially in the social sciences.  Third, the power analysis assumes that all of the assumptions of the correlation have been met.  For example, we are assuming that there is no restriction of range issue, which is common with Likert scales; the sample data for both variables are normally distributed; the relationship between the two variables is linear; and there are no serious outliers.  Also, whereas you might be able to say that the sample correlation does not equal zero, you likely will not have a very precise estimate of the population correlation coefficient.

Image corr

– For a chi-square test, you will need to know the proportion positive for both populations (i.e., rows and columns).  Let’s assume that we will have a 2 x 2 chi-square, and let’s think of both variables as 0/1.  Let’s say that we wanted to know if there was a relationship between drug group (drug A/placebo) and improved health.  In SPSS Sample Power, you would see a screen like this.

Image chi-square

In order to get the .60 and the .30, we would need to know (or guess at) the number of people whose health improved in both the drug A and placebo groups.

We would also need to know (or guess at) either the number of people whose health did not improve in those two groups, or the total number of people in each group.

– For an ordinary least squares regression, you would need to know things like the R 2 for the full and reduced model.  For a simple logistic regression analysis with only one continuous predictor variable, you would need to know the probability of a positive outcome (i.e., the probability that the outcome equals 1) at the mean of the predictor variable and the probability of a positive outcome at one standard deviation above the mean of the predictor variable.  Especially for the various types of logistic models (e.g., binary, ordinal and multinomial), you will need to think very carefully about your sample size, and information from a power analysis will only be part of your considerations.  For example, according to Long (1997, pages 53-54), 100 is a minimum sample size for logistic regression, and you want *at least* 10 observations per predictor.  This does not mean that if you have only one predictor you need only 10 observations.

Also, if you have categorical predictors, you may need to have more observations to avoid computational difficulties caused by empty cells or cells with few observations.  More observations are needed when the outcome variable is very lopsided; in other words, when there are very few 1s and lots of 0s, or vice versa.  These cautions emphasize the need to know your data set well, so that you know if your outcome variable is lopsided or if you are likely to have a problem with empty cells.

The point of this section is to give you a sense of the level of detail about your variables that you need to be able to estimate in order to do a power analysis. Also, when doing power analyses for regression models, power programs will start to ask for values that most researchers are not accustomed to providing.  Guessing at the mean and standard deviation of your response variable is one thing, but increments to R 2 is a metric in which few researchers are used to thinking.  In our next section we will discuss how you can guestimate these numbers.

Obtaining the necessary numbers to do a power analysis

There are at least three ways to guestimate the values that are needed to do a power analysis: a literature review, a pilot study and using Cohen’s recommendations.  We will review the pros and cons of each of these methods.  For this discussion, we will focus on finding the effect size, as that is often the most difficult number to obtain and often has the strongest impact on power.

Literature review: Sometimes you can find one or more published studies that are similar enough to yours that you can get a idea of the effect size.  If you can find several such studies, you might be able to use meta-analysis techniques to get a robust estimate of the effect size.  However, oftentimes there are no studies similar enough to your study to get a good estimate of the effect size.  Even if you can find such an study, the necessary effect sizes or other values are often not clearly stated in the article and need to be calculated (if they can) based on the information provided.

Pilot studies:  There are lots of good reasons to do a pilot study prior to conducting the actual study.  From a power analysis prospective, a pilot study can give you a rough estimate of the effect size, as well as a rough estimate of the variability in your measures.  You can also get some idea about where missing data might occur, and as we will discuss later, how you handle missing data can greatly affect your power.  Other benefits of a pilot study include allowing you to identify coding problems, setting up the data base, and inputting the data for a practice analysis.  This will allow you to determine if the data are input in the correct shape, etc.

Of course, there are some limitations to the information that you can get from a pilot study.  (Many of these limitations apply to small samples in general.)  First of all, when estimating effect sizes based on nonsignificant results, the effect size estimate will necessarily have an increased error; in other words, the standard error of the effect size estimate will be larger than when the result is significant. The effect size estimate that you obtain may be unduly influenced by some peculiarity of the small sample.  Also, you often cannot get a good idea of the degree of missingness and attrition that will be seen in the real study.  Despite these limitations, we strongly encourage researchers to conduct a pilot study.  The opportunity to identify and correct “bugs” before collecting the real data is often invaluable.  Also, because of the number of values that need to be guestimated in a power analysis, the precision of any one of these values is not that important.  If you can estimate the effect size to within 10% or 20% of the true value, that is probably sufficient for you to conduct a meaningful power analysis, and such fluctuations can be taken into account during the power analysis.

Cohen’s recommendations:  Jacob Cohen has many well-known publications regarding issues of power and power analyses, including some recommendations about effect sizes that you can use when doing your power analysis.  Many researchers (including Cohen) consider the use of such recommendations as a last resort, when a thorough literature review has failed to reveal any useful numbers and a pilot study is either not possible or not feasible.  From Cohen (1988, pages 24-27):

– Small effect:  1% of the variance; d = 0.25 (too small to detect other than statistically; lower limit of what is clinically relevant)

– Medium effect:  6% of the variance; d = 0.5 (apparent with careful observation)

– Large effect: at least 15% of the variance; d = 0.8 (apparent with a superficial glance; unlikely to be the focus of research because it is too obvious)

Lipsey and Wilson (1993) did a meta analysis of 302 meta analyses of over 10,000 studies and found that the average effect size was .5, adding support to Cohen’s recommendation that, as a last resort, guess that the effect size is .5 (cited in Bausell and Li, 2002).  Sedlmeier and Gigerenzer (1989) found that the average effect size for articles in The Journal of Abnormal Psychology was a medium effect.  According to Keppel and Wickens (2004), when you really have no idea what the effect size is, go with the smallest effect size of practical value.  In other words, you need to know how small of a difference is meaningful to you.  Keep in mind that research suggests that most researchers are overly optimistic about the effect sizes in their research, and that most research studies are under powered (Keppel and Wickens, 2004; Tversky and Kahneman, 1971).  This is part of the reason why we stress that a power analysis gives you a lower limit to the number of necessary subjects.

Factors that affect power

From the preceding discussion, you might be starting to think that the number of subjects and the effect size are the most important factors, or even the only factors, that affect power.  Although effect size is often the largest contributor to power, saying it is the only important issue is far from the truth.  There are at least a dozen other factors that can influence the power of a study, and many of these factors should be considered not only from the perspective of doing a power analysis, but also as part of doing good research.  The first couple of factors that we will discuss are more “mechanical” ways of increasing power (e.g., alpha level, sample size and effect size). After that, the discussion will turn to more methodological issues that affect power.

1.  Alpha level:  One obvious way to increase your power is to increase your alpha (from .05 to say, .1).  Whereas this might be an advisable strategy when doing a pilot study, increasing your alpha usually is not a viable option.  We should point out here that many researchers are starting to prefer to use .01 as an alpha level instead of .05 as a crude attempt to assure results are clinically relevant; this alpha reduction reduces power.

1a.  One- versus two-tailed tests:  In some cases, you can test your hypothesis with a one-tailed test.  For example, if your hypothesis was that drug A is better than the placebo, then you could use a one-tailed test.  However, you would fail to detect a difference, even if it was a large difference, if the placebo was better than drug A.  The advantage of one-tailed tests is that they put all of your power “on one side” to test your hypothesis.  The disadvantage is that you cannot detect differences that are in the opposite direction of your hypothesis.  Moreover, many grant and journal reviewers frown on the use of one-tailed tests, believing it is a way to feign significance (Stratton and Neil, 2004).

2.  Sample size:  A second obvious way to increase power is simply collect data on more subjects.  In some situations, though, the subjects are difficult to get or extremely costly to run.  For example, you may have access to only 20 autistic children or only have enough funding to interview 30 cancer survivors.  If possible, you might try increasing the number of subjects in groups that do not have these restrictions, for example, if you are comparing to a group of normal controls.  While it is true that, in general, it is often desirable to have roughly the same number of subjects in each group, this is not absolutely necessary.  However, you get diminishing returns for additional subjects in the control group:  adding an extra 100 subjects to the control group might not be much more helpful than adding 10 extra subjects to the control group.

3.  Effect size:  Another obvious way to increase your power is to increase the effect size.  Of course, this is often easier said than done. A common way of increasing the effect size is to increase the experimental manipulation.  Going back to our example of drug A and placebo, increasing the experimental manipulation might mean increasing the dose of the drug. While this might be a realistic option more often than increasing your alpha level, there are still plenty of times when you cannot do this.  Perhaps the human subjects committee will not allow it, it does not make sense clinically, or it doesn’t allow you to generalize your results the way you want to.  Many of the other issues discussed below indirectly increase effect size by providing a stronger research design or a more powerful statistical analysis.

4.  Experimental task:  Well, maybe you can not increase the experimental manipulation, but perhaps you can change the experimental task, if there is one.  If a variety of tasks have been used in your research area, consider which of these tasks provides the most power (compared to other important issues, such as relevancy, participant discomfort, and the like).  However, if various tasks have not been reviewed in your field, designing a more sensitive task might be beyond the scope of your research project.

5.  Response variable:  How you measure your response variable(s) is just as important as what task you have the subject perform.  When thinking about power, you want to use a measure that is as high in sensitivity and low in measurement error as is possible.  Researchers in the social sciences often have a variety of measures from which they can choose, while researchers in other fields may not.  For example, there are numerous established measures of anxiety, IQ, attitudes, etc.  Even if there are not established measures, you still have some choice.  Do you want to use a Likert scale, and if so, how many points should it have?  Modifications to procedures can also help reduce measurement error.  For example, you want to make sure that each subject knows exactly what he or she is supposed to be rating.  Oral instructions need to be clear, and items on questionnaires need to be unambiguous to all respondents.  When possible, use direct instead of indirect measures.  For example, asking people what tax bracket they are in is a more direct way of determining their annual income than asking them about the square footage of their house.  Again, this point may be more applicable to those in the social sciences than those in other areas of research.  We should also note that minimizing the measurement error in your predictor variables will also help increase your power.

Just as an aside, most texts on experimental design strongly suggest collecting more than one measure of the response in which you are interested. While this is very good methodologically and provides marked benefits for certain analyses and missing data, it does complicate the power analysis.

6.  Experimental design:  Another thing to consider is that some types of experimental designs are more powerful than others.  For example, repeated measures designs are virtually always more powerful than designs in which you only get measurements at one time.  If you are already using a repeated measures design, increasing the number of time points a response variable is collected to at least four or five will also provide increased power over fewer data collections.  There is a point of diminishing return when a researcher collects too many time points, though this depends on many factors such as the response variable, statistical design, age of participants, etc.

7.  Groups:  Another point to consider is the number and types of groups that you are using.  Reducing the number of experimental conditions will reduce the number of subjects that is needed, or you can keep the same number of subjects and just have more per group.  When thinking about which groups to exclude from the design, you might want to leave out those in the middle and keep the groups with the more extreme manipulations.  Going back to our drug A example, let’s say that we were originally thinking about having a total of four groups: the first group will be our placebo group, the second group would get a small dose of drug A, the third group a medium dose, and the fourth group a large dose.  Clearly, much more power is needed to detect an effect between the medium and large dose groups than to detect an effect between the large dose group and the placebo group.  If we found that we were unable to increase the power enough such that we were likely to find an effect between small and medium dose groups or between the medium and the large dose groups, then it would probably make more sense to run the study without these groups.  In some cases, you may even be able to change your comparison group to something more extreme.  For example, we once had a client who was designing a study to compare people with clinical levels of anxiety to a group that had subclinical levels of anxiety.  However, while doing the power analysis and realizing how many subjects she would need to detect the effect, she found that she needed far fewer subjects if she compared the group with the clinical levels of anxiety to a group of “normal” people (a number of subjects she could reasonably obtain).

8.  Statistical procedure:  Changing the type of statistical analysis may also help increase power, especially when some of the assumptions of the test are violated.  For example, as Maxwell and Delaney (2004) noted, “Even when ANOVA is robust, it may not provide the most powerful test available when its assumptions have been violated.”  In particular, violations of assumptions regarding independence, normality and heterogeneity can reduce power. In such cases, nonparametric alternatives may be more powerful.

9.  Statistical model:  You can also modify the statistical model.  For example, interactions often require more power than main effects.  Hence, you might find that you have reasonable power for a main effects model, but not enough power when the model includes interactions.  Many (perhaps most?) power analysis programs do not have an option to include interaction terms when describing the proposed analysis, so you need to keep this in mind when using these programs to help you determine how many subjects will be needed.  When thinking about the statistical model, you might want to consider using covariates or blocking variables.  Ideally, both covariates and blocking variables reduce the variability in the response variable.  However, it can be challenging to find such variables.  Moreover, your statistical model should use as many of the response variable time points as possible when examining longitudinal data.  Using a change-score analysis when one has collected five time points makes little sense and ignores the added power from these additional time points.  The more the statistical model “knows” about how a person changes over time, the more variance that can be pulled out of the error term and ascribed to an effect.

9a. Correlation between time points:  Understanding the expected correlation between a response variable measured at one time in your study with the same response variable measured at another time can provide important and power-saving information.  As noted previously, when the statistical model has a certain amount of information regarding the manner by which people change over time, it can enhance the effect size estimate.  This is largely dependent on the correlation of the response measure over time.  For example, in a before-after data collection scenario, response variables with a .00 correlation from before the treatment to after the treatment would provide no extra benefit to the statistical model, as we can’t better understand a subject’s score by knowing how he or she changes over time.  Rarely, however, do variables have a .00 correlation on the same outcomes measured at different times.  It is important to know that outcome variables with larger correlations over time provide enhanced power when used in a complimentary statistical model.

10.  Modify response variable:  Besides modifying your statistical model, you might also try modifying your response variable.  Possible benefits of this strategy include reducing extreme scores and/or meeting the assumptions of the statistical procedure.  For example, some response variables might need to be log transformed.  However, you need to be careful here.  Transforming variables often makes the results more difficult to interpret, because now you are working in, say, a logarithm metric instead of the metric in which the variable was originally measured. Moreover, if you use a transformation that adjusts the model too much, you can loose more power than is necessary.  Categorizing continuous response variables (sometimes used as a way of handling extreme scores) can also be problematic, because logistic or ordinal logistic regression often requires many more subjects than does OLS regression.  It makes sense that categorizing a response variable will lead to a loss of power, as information is being “thrown away.”

11.  Purpose of the study:  Different researchers have different reasons for conducting research.  Some are trying to determine if a coefficient (such as a regression coefficient) is different from zero.  Others are trying to get a precise estimate of a coefficient.  Still others are replicating research that has already been done.  The purpose of the research can affect the necessary sample size.  Going back to our drug A and placebo study, let’s suppose our purpose is to test the difference in means to see if it equals zero.   In this case, we need a relatively small sample size.  If our purpose is to get a precise estimate of the means (i.e., minimizing the standard errors), then we will need a larger sample size.  If our purpose is to replicate previous research, then again we will need a relatively large sample size.  Tversky and Kahneman (1971) pointed out that we often need more subjects in a replication study than were in the original study.  They also noted that researchers are often too optimistic about how much power they really have.  They claim that researchers too readily assign “causal” reasons to explain differences between studies, instead of sampling error. They also mentioned that researchers tend to underestimate the impact of sampling and think that results will replicate more often than is the case.

12.  Missing data:  A final point that we would like to make here regards missing data.  Almost all researchers have issues with missing data.  When designing your study and selecting your measures, you want to do everything possible to minimize missing data.  Handling missing data via imputation methods can be very tricky and very time-consuming.  If the data set is small, the situation can be even more difficult.  In general, missing data reduces power; poor imputation methods can greatly reduce power.  If you have to impute, you want to have as few missing data points on as few variables as possible.  When designing the study, you might want to collect data specifically for use in an imputation model (which usually involves a different set of variables than the model used to test your hypothesis).  It is also important to note that the default technique for handling missing data by virtually every statistical program is to remove the entire case from an analysis (i.e., listwise deletion).  This process is undertaken even if the analysis involves 20 variables and a subject is missing only one datum of the 20.  Listwise deletion is one of the biggest contributors to loss of power, both because of the omnipresence of missing data and because of the omnipresence of this default setting in statistical programs (Graham et al., 2003).

This ends the section on the various factors that can influence power.  We know that was a lot, and we understand that much of this can be frustrating because there is very little that is “black and white”.  We hope that this section made clear the close relationship between the experimental design, the statistical analysis and power.

Cautions about small sample sizes and sampling variation

We want to take a moment here to mention some issues that frequently arise when using small samples.  (We aren’t going to put a lower limit on what we mean be “small sample size.”)  While there are situations in which a researcher can either only get or afford a small number of subjects, in most cases, the researcher has some choice in how many subjects to include.  Considerations of time and effort argue for running as few subjects as possible, but there are some difficulties associated with small sample sizes, and these may outweigh any gains from the saving of time, effort or both.  One obvious problem with small sample sizes is that they have low power.  This means that you need to have a large effect size to detect anything.  You will also have fewer options with respect to appropriate statistical procedures, as many common procedures, such as correlations, logistic regression and multilevel modeling, are not appropriate with small sample sizes.  It may also be more difficult to evaluate the assumptions of the statistical procedure that is used (especially assumptions like normality).  In most cases, the statistical model must be smaller when the data set is small. Interaction terms, which often test interesting hypotheses, are frequently the first casualties.  Generalizability of the results may also be comprised, and it can be difficult to argue that a small sample is representative of a large and varied population. Missing data are also more problematic; there are a reduced number of imputations methods available to you, and these are not considered to be desirable imputation methods (such as mean imputation).  Finally, with a small sample size, alpha inflation issues can be more difficult to address, and you are more likely to run as many tests as you have subjects.

While the issue of sampling variability is relevant to all research, it is especially relevant to studies with small sample sizes.  To quote Murphy and Myors (2004, page 59), “The lack of attention to power analysis (and the deplorable habit of placing too much weight on the results of small sample studies) are well documented in the literature, and there is no good excuse to ignore power in designing studies.”  In an early article entitled The Law of Small Numbers , Tversky and Kahneman (1971) stated that many researchers act like the Law of Large Numbers applies to small numbers.  People often believe that small samples are more representative of the population than they really are.

The last two points to be made here is that there is usually no point to conducting an underpowered study, and that underpowered studies can cause chaos in the literature because studies that are similar methodologically may report conflicting results.

We will briefly discuss some of the programs that you can use to assist you with your power analysis.  Most programs are fairly easy to use, but you still need to know effect sizes, means, standard deviations, etc.

Among the programs specifically designed for power analysis, we use SPSS Sample Power, PASS and GPower.  These programs have a friendly point-and-click interface and will do power analyses for things like correlations, OLS regression and logistic regression.  We have also started using Optimal Design for repeated measures, longitudinal and multilevel designs. We should note that Sample Power is a stand-alone program that is sold by SPSS; it is not part of SPSS Base or an add-on module.  PASS can be purchased directly from NCSS at http://www.ncss.com/index.htm . GPower (please see GPower for details) and Optimal Design (please see http://sitemaker.umich.edu/group-based/home for details) are free.

Several general use stat packages also have procedures for calculating power.  SAS has proc power , which has a lot of features and is pretty nice.  Stata has the sampsi command, as well as many user-written commands, including fpower , powerreg and aipe (written by our IDRE statistical consultants).  Statistica has an add-on module for power analysis.  There are also many programs online that are free.

For more advanced/complicated analyses, Mplus is a good choice.  It will allow you to do Monte Carlo simulations, and there are some examples at http://www.statmodel.com/power.shtml and http://www.statmodel.com/ugexcerpts.shtml .

Most of the programs that we have mentioned do roughly the same things, so when selecting a power analysis program, the real issue is your comfort; all of the programs require you to provide the same kind of information.

Multiplicity

This issue of multiplicity arises when a researcher has more than one outcome of interest in a given study.  While it is often good methodological practice to have more than one measure of the response variable of interest, additional response variables mean more statistical tests need to be conducted on the data set, and this leads to question of experimentwise alpha control. Returning to our example of drug A and placebo, if we have only one response variable, then only one t test is needed to test our hypothesis.  However, if we have three measures of our response variable, we would want to do three t tests, hoping that each would show results in the same direction.  The question is how to control the Type I error (AKA false alarm) rate.  Most researchers are familiar with Bonferroni correction, which calls for dividing the prespecified alpha level (usually .05) by the number of tests to be conducted.  In our example, we would have .05/3 = .0167.  Hence, .0167 would be our new critical alpha level, and statistics with a p-value greater than .0167 would be classified as not statistically significant.  It is well-known that the Bonferroni correction is very conservative; there are other ways of adjusting the alpha level.

Afterthoughts:  A post-hoc power analysis

In general, just say “No!” to post-hoc analyses.  There are many reasons, both mechanical and theoretical, why most researchers should not do post-hoc power analyses.  Excellent summaries can be found in Hoenig and Heisey (2001) The Abuse of Power:  The Pervasive Fallacy of Power Calculations for Data Analysis and Levine and Ensom (2001) Post Hoc Power Analysis:  An Idea Whose Time Has Passed? .  As Hoenig and Heisey show, power is mathematically directly related to the p-value; hence, calculating power once you know the p-value associated with a statistic adds no new information.  Furthermore, as Levine and Ensom clearly explain, the logic underlying post-hoc power analysis is fundamentally flawed.

However, there are some things that you should look at after your study is completed.  Have a look at the means and standard deviations of your variables and see how close they are (or are not) from the values that you used in the power analysis.  Many researchers do a series of related studies, and this information can aid in making decisions in future research.  For example, if you find that your outcome variable had a standard deviation of 7, and in your power analysis you were guessing it would have a standard deviation of 2, you may want to consider using a different measure that has less variance in your next study.

The point here is that in addition to answering your research question(s), your current research project can also assist with your next power analysis.

Conclusions

Conducting research is kind of like buying a car.  While buying a car isn’t the biggest purchase that you will make in your life, few of us enter into the process lightly.  Rather, we consider a variety of things, such as need and cost, before making a purchase.  You would do your research before you went and bought a car, because once you drove the car off the dealer’s lot, there is nothing you can do about it if you realize this isn’t the car that you need.  Choosing the type of analysis is like choosing which kind of car to buy.  The number of subjects is like your budget, and the model is like your expenses.  You would never go buy a car without first having some idea about what the payments will be.  This is like doing a power analysis to determine approximately how many subjects will be needed.  Imagine signing the papers for your new Maserati only to find that the payments will be twice your monthly take-home pay.  This is like wanting to do a multilevel model with a binary outcome, 10 predictors and lots of cross-level interactions and realizing that you can’t do this with only 50 subjects.  You don’t have enough “currency” to run that kind of model.  You need to find a model that is “more in your price range.”  If you had $530 a month budgeted for your new car, you probably wouldn’t want exactly $530 in monthly payments. Rather you would want some “wiggle-room” in case something cost a little more than anticipated or you were running a little short on money that month. Likewise, if your power analysis says you need about 300 subjects, you wouldn’t want to collect data on exactly 300 subjects.  You would want to collect data on 300 subjects plus a few, just to give yourself some “wiggle-room” just in case.

Don’t be afraid of what you don’t know.  Get in there and try it BEFORE you collect your data.  Correcting things is easy at this stage; after you collect your data, all you can do is damage control.  If you are in a hurry to get a project done, perhaps the worst thing that you can do is start collecting data now and worry about the rest later.  The project will take much longer if you do this than if you do what we are suggesting and do the power analysis and other planning steps.  If you have everything all planned out, things will go much smoother and you will have fewer and/or less intense panic attacks.  Of course, some thing unexpected will always happen, but it is unlikely to be as big of a problem.  UCLA researchers are always welcome and strongly encouraged to come into our walk-in consulting and discuss their research before they begin the project.

Power analysis = planning.  You will want to plan not only for the test of your main hypothesis, but also for follow-up tests and tests of secondary hypotheses.  You will want to make sure that “confirmation” checks will run as planned (for example, checking to see that interrater reliability was acceptable).  If you intend to use imputation methods to address missing data issues, you will need to become familiar with the issues surrounding the particular procedure as well as including any additional variables in your data collection procedures.  Part of your planning should also include a list of the statistical tests that you intend to run and consideration of any procedure to address alpha inflation issues that might be necessary.

The number output by any power analysis program is often just a starting point of thought more than a final answer to the question of how many subjects will be needed.  As we have seen, you also need to consider the purpose of the study (coefficient different from 0, precise point estimate, replication), the type of statistical test that will be used (t-test versus maximum likelihood technique), the total number of statistical tests that will be performed on the data set, genearlizability from the sample to the population, and probably several other things as well.

The take-home message from this seminar is “do your research before you do your research.”

Anderson, N. H.  (2001).  Empirical Direction in Design and Analysis.  Mahwah, New Jersey:  Lawrence Erlbaum Associates.

Bausell, R. B. and Li, Y.  (2002).  Power Analysis for Experimental Research:  A Practical Guide for the Biological, Medical and Social Sciences.  Cambridge University Press, New York, New York.

Bickman, L., Editor.  (2000).  Research Design:  Donald Campbell’s Legacy, Volume 2.  Thousand Oaks, CA:  Sage Publications.

Bickman, L., Editor.  (2000).  Validity and Social Experimentation. Thousand Oaks, CA:  Sage Publications.

Campbell, D. T. and Russo, M. J.  (2001).  Social Measurement. Thousand Oaks, CA:  Sage Publications.

Campbell, D. T. and Stanley, J. C.  (1963).  Experimental and Quasi-experimental Designs for Research.  Reprinted from Handbook of Research on Teaching .  Palo Alto, CA:  Houghton Mifflin Co.

Chen, P. and Popovich, P. M.  (2002).  Correlation: Parametric and Nonparametric Measures.  Thousand Oaks, CA:  Sage Publications.

Cohen, J. (1988).  Statistical Power Analysis for the Behavioral Sciences, Second Edition.  Hillsdale, New Jersey:  Lawrence Erlbaum Associates.

Cook, T. D. and Campbell, D. T.  Quasi-experimentation:  Design and Analysis Issues for Field Settings.  (1979).  Palo Alto, CA: Houghton Mifflin Co.

Graham, J. W., Cumsille, P. E., and Elek-Fisk, E. (2003). Methods for handling missing data. In J. A. Schinka and W. F. Velicer (Eds.), Handbook of psychology (Vol. 2, pp. 87-114). New York: Wiley.

Green, S. B.  (1991).  How many subjects does it take to do a regression analysis?  Multivariate Behavioral Research, 26(3) , 499-510.

Hoenig, J. M. and Heisey, D. M.  (2001).  The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis.  The American Statistician, 55(1) , 19-24.

Kelley, K and Maxwell, S. E.  (2003).  Sample size for multiple regression:  Obtaining regression coefficients that are accurate, not simply significant.  Psychological Methods, 8(3) , 305-321.

Keppel, G. and Wickens, T. D. (2004).  Design and Analysis:  A Researcher’s Handbook, Fourth Edition.  Pearson Prentice Hall:  Upper Saddle River, New Jersey.

Kline, R. B. Beyond Significance  (2004).  Beyond Significance Testing:  Reforming Data Analysis Methods in Behavioral Research. American Psychological Association:  Washington, D.C.

Levine, M., and Ensom M. H. H.  (2001).  Post Hoc Power Analysis: An Idea Whose Time Has Passed?  Pharmacotherapy, 21(4) , 405-409.

Lipsey, M. W. and Wilson, D. B.  (1993).  The Efficacy of Psychological, Educational, and Behavioral Treatment:  Confirmation from Meta-analysis.  American Psychologist, 48(12) , 1181-1209.

Long, J. S. (1997).  Regression Models for Categorical and Limited Dependent Variables.  Thousand Oaks, CA:  Sage Publications.

Maxwell, S. E.  (2000).  Sample size and multiple regression analysis.  Psychological Methods, 5(4) , 434-458.

Maxwell, S. E. and Delany, H. D.  (2004).  Designing Experiments and Analyzing Data:  A Model Comparison Perspective, Second Edition. Lawrence Erlbaum Associates, Mahwah, New Jersey.

Murphy, K. R. and Myors, B.  (2004).  Statistical Power Analysis: A Simple and General Model for Traditional and Modern Hypothesis Tests. Mahwah, New Jersey:  Lawrence Erlbaum Associates.

Publication Manual of the American Psychological Association, Fifth Edition. (2001).  Washington, D.C.:  American Psychological Association.

Sedlmeier, P. and Gigerenzer, G.  (1989).  Do Studies of Statistical Power Have an Effect on the Power of Studies?  Psychological Bulletin, 105(2) , 309-316.

Shadish, W. R., Cook, T. D. and Campbell, D. T.  (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston:  Houghton Mifflin Co.

Stratton, I. M. and Neil, A.  (2004).  How to ensure your paper is rejected by the statistical reviewer.  Diabetic Medicine , 22, 371-373.

Tversky, A. and Kahneman, D.  (1971).  Belief in the Law of Small Numbers.  Psychological Bulletin, 76(23) , 105-110.

Webb, E., Campbell, D. T., Schwartz, R. D., and Sechrest, L.  (2000). Unobtrusive Measures, Revised Edition.  Thousand Oaks, CA:  Sage Publications.

Your Name (required)

Your Email (must be a valid email for us to receive the report!)

Comment/Error Report (required)

How to cite this page

  • © 2021 UC REGENTS

Power Analysis

Power analysis is an important aspect of experimental design. It allows us to determine the sample size required to detect an effect of a given size with a given degree of confidence. Conversely, it allows us to determine the probability of detecting an effect of a given size with a given level of confidence, under sample size constraints. If the probability is unacceptably low, we would be wise to alter or abandon the experiment.

The following four quantities have an intimate relationship:

  • sample size
  • effect size
  • significance level = P(Type I error) = probability of finding an effect that is not there
  • power = 1 - P(Type II error) = probability of finding an effect that is there

Given any three, we can determine the fourth.

Power Analysis in R

The pwr package develped by Stéphane Champely, impliments power analysis as outlined by Cohen (!988) . Some of the more important functions are listed below.

For each of these functions, you enter three of the four quantities (effect size, sample size, significance level, power) and the fourth is calculated.

The significance level defaults to 0.05. Therefore, to calculate the significance level, given an effect size, sample size, and power, use the option "sig.level=NULL".

Specifying an effect size can be a daunting task. ES formulas and Cohen's suggestions (based on social science research) are provided below. Cohen's suggestions should only be seen as very rough guidelines. Your own subject matter experience should be brought to bear.

(To explore confidence intervals and drawing conclusions from samples try this interactive course on the foundations of inference.)

For t-tests, use the following functions:

pwr.t.test(n = , d = , sig.level = , power = , type = c("two.sample", "one.sample", "paired"))

where n is the sample size, d is the effect size, and type indicates a two-sample t-test, one-sample t-test or paired t-test. If you have unequal sample sizes, use

pwr.t2n.test(n1 = , n2= , d = , sig.level =, power = )

where n1 and n2 are the sample sizes.

For t-tests, the effect size is assessed as

Cohen d

Cohen suggests that d values of 0.2, 0.5, and 0.8 represent small, medium, and large effect sizes respectively.

You can specify alternative="two.sided", "less", or "greater" to indicate a two-tailed, or one-tailed test. A two tailed test is the default.

For a one-way analysis of variance use

pwr.anova.test(k = , n = , f = , sig.level = , power = )

where k is the number of groups and n is the common sample size in each group.

For a one-way ANOVA effect size is measured by f where

Cohen f

Correlations

For correlation coefficients use

pwr.r.test(n = , r = , sig.level = , power = )

where n is the sample size and r is the correlation. We use the population correlation coefficient as the effect size measure. Cohen suggests that r values of 0.1, 0.3, and 0.5 represent small, medium, and large effect sizes respectively.

Linear Models

For linear models (e.g., multiple regression) use

pwr.f2.test(u =, v = , f2 = , sig.level = , power = )

where u and v are the numerator and denominator degrees of freedom. We use f2 as the effect size measure.

cohen f2

The first formula is appropriate when we are evaluating the impact of a set of predictors on an outcome. The second formula is appropriate when we are evaluating the impact of one set of predictors above and beyond a second set of predictors (or covariates). Cohen suggests f2 values of 0.02, 0.15, and 0.35 represent small, medium, and large effect sizes.

Tests of Proportions

When comparing two proportions use

pwr.2p.test(h = , n = , sig.level =, power = )

where h is the effect size and n is the common sample size in each group.

Cohen h

Cohen suggests that h values of 0.2, 0.5, and 0.8 represent small, medium, and large effect sizes respectively.

For unequal n's use

pwr.2p2n.test(h = , n1 = , n2 = , sig.level = , power = )

To test a single proportion use

pwr.p.test(h = , n = , sig.level = power = )

For both two sample and one sample proportion tests, you can specify alternative="two.sided", "less", or "greater" to indicate a two-tailed, or one-tailed test. A two tailed test is the default.

Chi-square Tests

For chi-square tests use

pwr.chisq.test(w =, N = , df = , sig.level =, power = )

where w is the effect size, N is the total sample size, and df is the degrees of freedom. The effect size w is defined as

Cohen w

Cohen suggests that w values of 0.1, 0.3, and 0.5 represent small, medium, and large effect sizes respectively.

Some Examples

Creating power or sample size plots.

The functions in the pwr package can be used to generate power and sample size graphs.

sample size curves

Sample Size Adjustment

0.80 Change

0.05 Change

Two sided Change

Sample Size Calculator for Comparing Paired Differences

  • Provides live interpretations.
  • Evaluates the influence of changing input values.
  • Adjusts sample sizes for continuity and clustering.

Input Values

Select one of the two options to specify input values. Hover over the sign to obtain help.

  • Click the Options button to change the default options for Power, Significance, Alternate Hypothesis and Group Sizes.
  • Click the Adjust button to adjust sample sizes for t-distribution (option applied by default), and clustering.

Note: You may change the default options for Power, Significance, Alternate Hypothesis and Group Sizes by clicking the 'Options' button. Click the 'Adjust' button below to adjust sample sizes for continuity (option applied by default), clustering and response rate.

Influence of Changing Input values on Sample Size Estimates

Customize visualisation.

Customize the plot by changing input values from here.

Note: You may change the default options for Power, Significance, Alternate Hypothesis and Group Sizes by clicking the 'Options' button. Click the 'Adjust' button below to adjust sample sizes for the t-distribution (option applied by default), and clustering.

Visualisation

This is a plot of sample sizes (number of pairs) for a range of Standard Deviations and for three values of Means of the Paired Differences. Customize the plot by changing input values from the 'Customize Visualisation' panel.

Table of Sample Sizes at a range of Expected Means and Standard Deviations

Customize table.

Customize the table by changing input values from here.

Sample Size Table

This table shows sample sizes (number of pairs) for a range of expected means and standard deviations of the paired differences. Customize the table by changing input values from the 'Customize Table' pane.

Educate 360

Set Your Location

New Horizons

  • Resources / Blog

The Power of Data Analysis in Research

The Power of Data Analysis in Research

For as long as humans have existed, we've been conducting research. We do it to solve problems, advance technology, understand diseases and develop treatments, inform policymaking, and, most fundamentally, expand human knowledge. At the heart of all research is data. Data, whether observations, statistics, surveys, or interviews, serves as the building blocks for all research.

The more we research, the more data we collect, and the more thorough our analysis will be. Data analysis illuminates patterns, trends, and relationships within the data and plays a pivotal role in shaping the outcomes and conclusions of research. In the sections below, we'll look at different data analysis methods, popular tools researchers use today, and how to make the most of your data.

On this page:

Understanding Data Analysis

The data analysis process, data analysis methods, tools and software for data analysis.

  • Choosing the Right Tools for Your Research
  • Applications for Data Analysis in Research

Challenges in Data Analysis

Future trends in data analysis, getting started with data analysis.

Data analysis is the most crucial part of any research. It allows researchers to understand the information gathered, test hypotheses, and draw conclusions. Analysis is the process of interpreting raw data through logical reasoning and applying various statistical and analytical techniques to understand patterns, relationships, and trends within the data.

Researchers must follow a methodical data analysis approach to ensure accurate results. Skipping steps can skew the outcomes and leave research findings vulnerable to scrutiny.

  • Plan and Design the Analysis: Determine the problem you want to solve before analyzing data. Defining the objectives of the analysis is crucial because it establishes the research direction and identifies the data you need to use to solve the problem.
  • Collecting and Preparing Data: Once the proper data is identified to use, it must be cleaned by checking for missing, inconsistent, and outlier data, ensuring accurate results from the analysis.
  • Analyzing Data: Once cleaned, apply statistical and mathematical to find patterns, relationships, or trends in the data.
  • Interpreting Data: After analysis, interpret the results and report actionable insights in ways non-data analysts can easily understand, e.g., using graphs or tables.

There are so many different reasons to conduct research and so many types of data that there is no one-size-fits-all approach to analysis. Instead, there are many methods, each with its unique purpose.

As digital technology has advanced, the number of data analysis tools available to researchers has exploded. Some of the most well-known data analysis tools include:

Choosing the Right Tools For Your Research.

There is no universal data analytics tool that will address all your needs. Everyone who works with data at some point needs secondary tools and plugins. Here are some things to consider when looking for data analysis software:

  • What is your budget?
  • Is there a viable free version, or do you need a subscription?
  • What is your technical expertise?
  • What is the scalability and flexibility?
  • Can the tool integrate with your existing data sources?
  • Can it handle the volume of data you’re working with?
  • Do you require a tool with modeling capabilities?

Applications of Data Analysis in Research

Data analysis is in high demand across industries, driving innovation that improves an organization's business outcomes and the lives of employees and customers. To understand how data analysis applies to different types of research, let's look at some real-world examples:

  • Environmental studies

Example #1: Healthcare

Data analysis in the medical field has dramatically improved healthcare outcomes. For example, epidemiologists investigate patterns and determinants of disease occurrence and distribution within populations. Through data analysis, they've made great strides in identifying associations between lifestyle factors (e.g., smoking, diet, and physical activity) and chronic diseases like cardiovascular disease, diabetes, and cancer.

Example #2: Finance

Data analysis plays a central role in assessing and managing financial risks. Analyzing historical data and using techniques like value-at-risk (VaR) and stress testing allows risk managers to quantify the potential impact of adverse events on investment portfolios and implement risk mitigation strategies.

Example #3: Environmental studies

The world's leading climate scientists use data analysis in their research. They analyze large datasets of temperature records, atmospheric CO2 concentrations, sea level measurements, and other climate variables to detect trends and patterns over time. This climate data allows researchers to understand global warming better, assess its impacts on ecosystems and human societies, and inform climate policy decisions.

The insights you gain from analyzing data are only as good as the data they are based on. Knowing the common challenges associated with data analytics is essential for data analysts, both new and experienced. Some of the common challenges data analysts face are:

  • Setting clear hypotheses and goals
  • Understanding the data being analyzed
  • Knowing the correct source data to use
  • Determining the integrity and trustworthiness of the source data
  • Maintaining the privacy and ethical use of data
  • Communicating data insights using easily understood methods
  • Remaining objective throughout the analysis

Any of these challenges can lead to incorrect analysis, impacting organizational decision-making. There are several ways in which data analysts can overcome these challenges , including seeking advice from fellow data analysts or taking self-paced or online training. By overcoming these data analysis challenges, data analysts can ensure they provide correct insights to improve an organization’s business outcomes.

Many future trends will impact data analysis, especially from a technology and regulatory standpoint. These trends will allow data analysts to work with more data that can provide deeper business insights for organizations while ensuring that it is used ethically and remains private and secure. Some of the future trends that will impact data analysis include:

  • Artificial intelligence (AI) and machine learning (ML) are changing data analysis by automating complex data processing tasks. These tools can identify patterns in massive data sets and provide highly accurate insights and predictions.
  • Regulation : The European Union's General Data Protection Regulation (GDPR) went into effect in 2018, heralding a new era for data privacy. It levies harsh fines against any person or organization that violates its privacy and security standards, aiming to protect consumer data. As the volume of global data increases, other international governments will follow suit. 
  • Quantum Computing : As organizations generate more significant amounts of data, the need for computers that can store it grows. Demand for more powerful computers to process vast amounts of data is rising. Quantum computing may be the answer, with its ability to store vast amounts of information using qubits and much less energy.
  • Data Democratization : As analytics platforms evolve to become more powerful and intuitive, it will allow anyone, regardless of data analysis experience, to harness and analyze data. Self-service analytics significantly reduces the time and effort required to retrieve insights from data so that data analysts can focus on more specialized work.

The typical starting point for a career in data analysis is through collegiate programs such as computer science, mathematics, and programming. However, you don't have to attend college to become a data analyst. Professional training courses on data analysis are a viable option if you want to start your data analysis career. For example, New Horizons offers online training courses in Data and Analytics , which fall into three categories:

  • No-Code is appropriate for individuals who want to improve their data analytics skills without learning a programming language.
  • Low-Code: Appropriate for those with limited programming skills or data analysis experience.
  • Traditional Data & Analytics: Appropriate for those with programming and data analysis experience looking for courses for specific job roles.

New Horizons offers several Python training courses , as well as vendor-specific data analytics courses, such as:

  • A WS Data Analytics Bootcamp : Four one-day AWS courses, including Building Data Lakes on AWS, Building Batch Data Analytics Solutions on AWS, Building Data Analytics Solutions Using Amazon Redshift, and Building Streaming Data Analytics Solutions on AWS.
  • Microsoft Power Platform : Our subject matter experts show you how to do more with your data, including manipulating, visualizing, automating, and analyzing it using PowerBI, PowerApps, Power Automate, and Power Virtual Agents.

For beginners, completing small projects using public datasets can provide a great way to gain practical data analysis experience. Platforms like Kaggle, GitHub, and Data.gov offer publicly available datasets, providing a great way to apply theoretical knowledge and develop technical skills.

Organizations will always look for ways to improve and innovate; data analysts can help define and solve problems to help the organization move forward. By pinpointing patterns and extracting actionable insights from large quantities of data, data analysts can help guide organizations toward more innovative and customer-centric solutions. As data analytics tools evolve, they will allow even those with little to no data analysis experience to work with data and make better decisions that will help their organization reach and surpass its goals.

  • Expand/Collapse Microsoft Office 38 RSS
  • Expand/Collapse Training Trends 106 RSS
  • Expand/Collapse CyberSecurity 58 RSS
  • Expand/Collapse DevOps 2 RSS
  • Expand/Collapse Modern Workplace 31 RSS
  • Expand/Collapse Cloud 14 RSS
  • Expand/Collapse Programming 9 RSS
  • Expand/Collapse Artificial Intelligence (AI) 4 RSS
  • Expand/Collapse ITIL 17 RSS
  • Expand/Collapse Data & Analytics 21 RSS
  • Expand/Collapse Business Analyst 12 RSS
  • Expand/Collapse Training By Job Role 1 RSS
  • Expand/Collapse Leadership and Professional Development 14 RSS
  • Expand/Collapse Managed Learning Services 3 RSS
  • Expand/Collapse Design & Multi-Media 1 RSS

A cellphone seen up close in a woman’s hand. She has red finger nails and is wearing a star-and-stripes shirt.

How Trump’s Allies Are Winning the War Over Disinformation

Their claims of censorship have successfully stymied the effort to filter election lies online.

Three years after Mr. Trump spread falsehoods about his defeat online, social media platforms have fewer checks on the intentional spread of lies about elections. Credit... Emily Elconin for The New York Times

Supported by

  • Share full article

Jim Rutenberg

By Jim Rutenberg and Steven Lee Myers

  • March 17, 2024

In the wake of the riot on Capitol Hill on Jan. 6, 2021, a groundswell built in Washington to rein in the onslaught of lies that had fueled the assault on the peaceful transfer of power.

Social media companies suspended Donald J. Trump, then the president, and many of his allies from the platforms they had used to spread misinformation about his defeat and whip up the attempt to overturn it. The Biden administration, Democrats in Congress and even some Republicans sought to do more to hold the companies accountable. Academic researchers wrestled with how to strengthen efforts to monitor false posts.

Mr. Trump and his allies embarked instead on a counteroffensive, a coordinated effort to block what they viewed as a dangerous effort to censor conservatives.

They have unquestionably prevailed.

Waged in the courts, in Congress and in the seething precincts of the internet, that effort has eviscerated attempts to shield elections from disinformation in the social media era. It tapped into — and then, critics say, twisted — the fierce debate over free speech and the government’s role in policing content.

Projects that were once bipartisan, including one started by the Trump administration, have been recast as deep-state conspiracies to rig elections. Facing legal and political blowback, the Biden administration has largely abandoned moves that might be construed as stifling political speech.

While little noticed by most Americans, the effort has helped cut a path for Mr. Trump’s attempt to recapture the presidency. Disinformation about elections is once again coursing through news feeds, aiding Mr. Trump as he fuels his comeback with falsehoods about the 2020 election.

“The censorship cartel must be dismantled and destroyed, and it must happen immediately,” he thundered at the start of his 2024 campaign.

The counteroffensive was led by former Trump aides and allies who had also pushed to overturn the 2020 election. They include Stephen Miller, the White House policy adviser; the attorneys general of Missouri and Louisiana, both Republicans; and lawmakers in Congress like Representative Jim Jordan, Republican of Ohio, who since last year has led a House subcommittee to investigate what it calls “the weaponization of government.”

Those involved draw financial support from conservative donors who have backed groups that promoted lies about voting in 2020. They have worked alongside an eclectic cast of characters, including Elon Musk, the billionaire who bought Twitter and vowed to make it a bastion of free speech, and Mike Benz, a former Trump administration official who previously produced content for a social media account that trafficked in posts about “white ethnic displacement.” (More recently, Mr. Benz originated the false assertion that Taylor Swift was a “psychological operation” asset for the Pentagon.)

Three years after Mr. Trump’s posts about rigged voting machines and stuffed ballot boxes went viral, he and his allies have achieved a stunning reversal of online fortune. Social media platforms now provide fewer checks against the intentional spread of lies about elections.

“The people that benefit from the spread of disinformation have effectively silenced many of the people that would try to call them out,” said Kate Starbird, a professor at the University of Washington whose research on disinformation made her a target of the effort.

It took aim at a patchwork of systems, started in Mr. Trump’s administration, that were intended to protect U.S. democracy from foreign interference. As those systems evolved to address domestic sources of misinformation, federal officials and private researchers began urging social media companies to do more to enforce their policies against harmful content.

That work has led to some of the most important First Amendment cases of the internet age, including one to be argued on Monday at the Supreme Court. That lawsuit, filed by the attorneys general of Missouri and Louisiana, accuses federal officials of colluding with or coercing the platforms to censor content critical of the government. The court’s decision, expected by June, could curtail the government’s latitude in monitoring content online.

The arguments strike at the heart of an unsettled question in modern American political life: In a world of unlimited online communications, in which anyone can reach huge numbers of people with unverified and false information, where is the line between protecting democracy and trampling on the right to free speech?

Even before the court rules, Mr. Trump’s allies have succeeded in paralyzing the Biden administration and the network of researchers who monitor disinformation.

Officials at the Department of Homeland Security and the State Department continue to monitor foreign disinformation, but the government has suspended virtually all cooperation with the social media platforms to address posts that originate in the United States.

“There’s just a chilling effect on all of this,” said Nina Jankowicz, a researcher who in 2022 briefly served as the executive director of a short-lived D.H.S. advisory board on disinformation. “Nobody wants to be caught up in it.”

Donald Trump holds a copy of the New York Post. The headline reads “The Ministry of Tweet.”

Fighting the ‘interpretive battle’

For Mr. Trump, banishment from social media was debilitating. His posts had been central to his political success, as was the army of adherents who cheered his messages and rallied behind his effort to hold onto office after he lost.

“WE have to use TIKTOK!!” read a memo prepared for Mr. Trump’s lead lawyer, Rudolph W. Giuliani, referring to a strategy to use social media to promote false messages about dead voters and vote-stealing software. “Content goes VIRAL here like no other platform!!!!! And there are MILLIONS of Trump supporters!”

After the violence on Jan. 6, Trump aides started working on how to “win the interpretive battle of the Trump history,” as one of them, Vincent Haley, had said in a previously unreported message found in the archives of the House investigation into the Jan. 6 attack. That would be crucial “for success in 2022 and 2024,” he added.

Once out of office, Mr. Trump built his own social platform, Truth Social, and his aides created a network of new organizations to advance the Trump agenda — and to prepare for his return.

Mr. Miller, Mr. Trump’s top policy adviser , created America First Legal, a nonprofit, to take on, as its mission statement put it, “an unholy alliance of corrupt special interests, big tech titans, fake news media and liberal Washington politicians.”

He solicited funding from conservative donors, drawing on a $27 million contribution from the Bradley Impact Fund , which had financed a web of groups that pushed “voter fraud” conspiracies in 2020. Another $1.3 million came from the Conservative Partnership Institute, considered the nonprofit nerve center of the Trump movement.

A key focus would be what he perceived as bias against conservatives on social media. “When you see people being banned off of Twitter and Facebook and other platforms,” he said in January 2021, “what you are seeing is the fundamental erosion of the concept of liberty and freedom in America.”

Mr. Biden’s administration was moving in the other direction. He came into office determined to take a tougher line against misinformation online — in large part because it was seen as an obstacle to bringing the coronavirus pandemic under control. D.H.S. officials were focused on bolstering defenses against election lies, which clearly had failed ahead of Jan. 6.

In one respect, that was clearer cut than matters of public health. There have long been special legal protections against providing false information about where, when and how to vote or intentionally sowing public confusion , or fear, to suppress voting.

Social media, with its pipeline to tens of millions of voters, presented powerful new pathways for antidemocratic tactics, but with far fewer of the regulatory and legal limits that exist for television, radio and newspapers.

The pitfalls were also clear: During the 2020 campaign, platforms had rushed to bury a New York Post article about Hunter Biden’s laptop out of concern that it might be tied to Russian interference. Conservatives saw it as an attempt to tilt the scales to Mr. Biden.

Administration officials said they were seeking a delicate balance between the First Amendment and social media’s rising power over public opinion.

“We’re in the business of critical infrastructure, and the most critical infrastructure is our cognitive infrastructure,” said Jen Easterly, the director of the Cybersecurity and Infrastructure Security Agency, whose responsibilities include protecting the national voting system. “Building that resilience to misinformation and disinformation, I think, is incredibly important.”

In early 2022, D.H.S. announced its first major answer to the conundrum: the Disinformation Governance Board. The board would serve as an advisory body and help coordinate anti-disinformation efforts across the department’s bureaucracy, officials said. Its director was Ms. Jankowicz, an expert in Russian disinformation.

The announcement ignited a political firestorm that killed the board only weeks after it began operating. Both liberals and conservatives raised questions about its reach and the potential for abuse.

The fury was most intense on the right. Mr. Miller, speaking on Fox News, slammed it as “something out of a dystopian sci-fi novel.”

Ms. Jankowicz said that such attacks were distorting but acknowledged that the announcement had struck a nerve.

“I think any American, when you hear, ‘Oh, the administration, the White House, is setting up something to censor Americans,’ even if that has no shred of evidence behind it, your ears are really going to prick up,” she said.

A legal assault

Among those who took note was Eric Schmitt, then the attorney general of Missouri.

He and other attorneys general had been a forceful part of Mr. Trump’s legal campaign to overturn his defeat. Now, they would lend legal firepower to block the fight against disinformation.

In May 2022, Mr. Schmitt and Jeff Landry, then the attorney general of Louisiana and now the governor, sued dozens of federal officials, including Dr. Anthony S. Fauci, the nation’s top expert on infectious diseases, who had become a villain to many conservatives.

The lawsuit picked up where others had failed. Mr. Trump and others had sued Facebook and Twitter, but those challenges stalled as courts effectively ruled that the companies had a right to ban content on their sites. The new case, known as Missouri v. Biden, argued that companies were not just barring users — they were being coerced into doing so by government officials.

The attorneys general filed the lawsuit in the Western District of Louisiana, where it fell to Judge Terry A. Doughty, a Trump appointee who had built a reputation for blocking Biden administration policies.

“A lot of these lawsuits against social media companies themselves were just dying in the graveyard in the Northern District of California,” Mr. Schmitt, who was elected to the U.S. Senate in 2022, said, referring to the liberal-leaning federal court in San Francisco. “And so our approach was a little bit different. We went directly at the government.”

The lawsuit was considered a long shot by experts, who noted that government officials were not issuing orders but urging the platforms to enforce their own policies. The decision to act was left to the companies, and more often than not, they did nothing.

Documents subpoenaed for the case showed extensive interactions between government officials and the platforms. In emails and text messages, people on both sides were alternately cooperative and confrontational. The platforms took seriously the administration’s complaints about content they said was misleading or false, but at the same time, they did not blindly carry out its bidding.

On Mr. Biden’s third day in office, a White House aide, Clarke Humphrey, wrote to Twitter flagging a post by Robert F. Kennedy Jr. falsely suggesting that the death of Hank Aaron, the baseball legend, had been caused by the Covid-19 vaccines. She asked an executive at the platform to begin the process of removing the post “as soon as possible.”

The post is still up.

Reframing the debate

In August 2022, a new organization, the Foundation for Freedom Online, posted a report on its website called “Department of Homeland Censorship: How D.H.S. Seized Power Over Online Speech.”

The group’s founder, a little-known former White House official named Mike Benz, claimed to have firsthand knowledge of how federal officials were “coordinating mass censorship of the internet.”

At the heart of Mr. Benz’s theory was the Election Integrity Partnership, a group created in the summer of 2020 to supplement government efforts to combat misinformation about the election that year.

The idea came from a group of college interns at the Cybersecurity and Infrastructure Security Agency, known as CISA. The students suggested that research institutions could help track and flag posts that might violate the platforms’ standards, feeding the information into a portal open to the agency, state and local governments and the platforms.

The project ultimately involved Stanford University, the University of Washington, the National Conference on Citizenship, the Atlantic Council’s Digital Forensic Research Lab and Graphika, a social media analytics firm. At its peak, it had 120 analysts, some of whom were college students.

It had what it considered successes, including spotting — and helping to stop — the spread of a false claim that a poll worker was burning Trump ballots in Erie, Pa. The approach could misfire, though. A separate, but related, CISA system flagged a tweet from a New York Times reporter accurately describing a printer problem at a voter center in Wisconsin, leading Twitter to affix an accuracy warning.

Decisions about whether to act remained with the platforms, which, in nearly two out of every three cases, did nothing.

In Mr. Benz’s telling, however, the government was using the partnership to get around the First Amendment, like outsourcing warfare to the private military contractor Blackwater.

Mr. Benz’s foundation for a time advertised itself as “a project of” Empower Oversight , a Republican group created by former Senate aides to support “whistle-blower” investigations.

Mr. Benz had previously lived a dual life. By day, he was a corporate lawyer in New York. In his off-hours, he toiled online under a social media avatar, Frame Game Radio, which railed against “the complete war on free speech” as it produced racist and antisemitic posts.

In videos and posts, Frame Game identified himself as a onetime member of the “Western chauvinist” group the Proud Boys, and as a Jew. Yet he blamed Jewish groups when he and others were suspended by social media companies. Warning about a looming demographic “white genocide,” Frame Game vented, “Anything pro white is called racist; anything white positive is racist.”

Mr. Benz did not respond to requests for comment. After NBC News first reported on Frame Game last fall, Mr. Benz called the account “a deradicalization project” to which he contributed in a “limited manner.” It was intended, he wrote on X, “by Jews to get people who hated Jews to stop hating Jews.”

Toward the end of 2018, Mr. Benz joined the Trump administration as a speechwriter for the housing and urban development secretary, Ben Carson. Mr. Benz’s posts were discovered by a colleague and brought to department management, according to a former official who insisted on anonymity to discuss a personnel matter.

As the election between Mr. Trump and Mr. Biden heated up, he joined Mr. Miller’s speech-writing team at the White House. He was there through the early days of the effort to keep Mr. Trump in power, and was involved in the search for statistical anomalies that could purport to show election fraud, according to testimony and records collected by House investigators, some of which were first uncovered by Kristen Ruby, a social media and public relations strategist.

In late November 2020, Mr. Benz was abruptly moved to the State Department as a deputy assistant secretary for international communications and information policy. It is unclear precisely what he did in the role. Mr. Benz has since claimed that the job, which he held for less than two months, gave him his expertise in cyberpolicy.

Mr. Benz’s report gained national attention when a conservative website, Just the News, wrote about it in September 2022. Four days later, Mr. Schmitt’s office sent requests for records to the University of Washington and others demanding information about their contacts with the government.

Mr. Schmitt soon amended his lawsuit to include nearly five pages detailing Mr. Benz’s work and asserting a new, broader claim: Not only was the government exerting pressure on the platforms, but it was also effectively deputizing the private researchers “to evade First Amendment and other legal restrictions.”

The scheme, Mr. Benz said, had “ambitious sights for 2022 and 2024.”

‘An aha moment’

In October 2022, Mr. Musk completed his purchase of Twitter and vowed to make the platform a forum for unfettered debate.

He quickly reversed the barring of Mr. Trump — calling it “morally wrong” — and loosened rules that had caused the suspensions of many of his followers.

He also set out to prove that Twitter’s previous management had too willingly cooperated with government officials. He released internal company communications to a select group of writers, among them Matt Taibbi and Michael Shellenberger.

The resulting project, which became known as the Twitter Files, began with an installment investigating Twitter’s decision to limit the reach of the Post article about Hunter Biden’s laptop.

The author of that dispatch, Mr. Taibbi, concluded that Twitter had limited the coverage amid general warnings from the F.B.I. that Russia could leak hacked materials to try to influence the 2020 election. Though he was critical of previous leadership at Twitter, he reported that he saw no evidence of direct government involvement.

In March 2023, Mr. Benz joined the fray. Both Mr. Taibbi and Mr. Benz participated in a live discussion on Twitter, which was co-hosted by Jennifer Lynn Lawrence, an organizer of the Trump rally that preceded the riot on Jan. 6.

As Mr. Taibbi described his work, Mr. Benz jumped in: “I believe I have all of the missing pieces of the puzzle.”

There was a far broader “scale of censorship the world has never experienced before,” he told Mr. Taibbi, who made plans to follow up.

Later, Mr. Shellenberger said that connecting with Mr. Benz had led to “a big aha moment.”

“The clouds parted, and the sunlight burst through the sky,” he said on a podcast. “It’s like, oh, my gosh, this guy is way, way farther down the rabbit hole than we even knew the rabbit hole went.”

A platform in Congress

A week after that online meeting, Mr. Taibbi and Mr. Shellenberger appeared on Capitol Hill as star witnesses for the Select Subcommittee on the Weaponization of the Federal Government. Mr. Benz sat behind them, listening as they detailed parts of his central thesis: This was not an imperfect attempt to balance free speech with democratic rights but a state-sponsored thought-policing system.

Mr. Shellenberger titled his written testimony, “The Censorship Industrial Complex.”

The committee had been created immediately after Republicans took control of the House in 2023 with a mandate to investigate, among other things, the actions taken by social media companies against conservatives.

It was led by Mr. Jordan, a lawmaker who helped spearhead the attempt to block certification of Mr. Biden’s victory and who has since worked closely with Mr. Miller and America First Legal.

“There are subpoenas that are going out on a daily, weekly basis,” Mr. Miller told Fox News in the first days of Republican control of the House, showing familiarity with the committee’s strategy.

Mr. Jordan’s committee soon sought documents from all those involved in the Election Integrity Partnership, as well as scores of government agencies and private researchers.

Mr. Miller followed with his own federal lawsuit on behalf of private plaintiffs in Missouri v. Biden, filing with D. John Sauer, the former solicitor general of Missouri who had led that case. (More recently, Mr. Sauer has represented Mr. Trump at the Supreme Court.)

Democrats in the House and legal experts questioned the collaboration as potentially unethical. Lawyers involved in the case have claimed that the subcommittee leaked selective parts of interviews conducted behind closed doors to America First Legal for use in its private lawsuits.

An amicus brief filed by the committee misrepresented facts and omitted evidence in ways that may have violated the Federal Rules of Civil Procedure, Representative Jerrold Nadler of New York wrote in a 46-page letter to Mr. Jordan.

A committee spokeswoman said the letter “deliberately misrepresents the evidence available to the committee to defend the Biden administration’s attacks on the First Amendment.”

The amicus brief, filed to the U.S. Court of Appeals for the Fifth Circuit, was drafted by a lawyer at Mr. Miller’s legal foundation.

Mr. Miller did not respond to requests for comment.

A chilling effect

By the summer of 2023, the legal and political effort was having an impact.

The organizations involved in the Election Integrity Partnership faced an avalanche of requests and, if they balked, subpoenas for any emails, text messages or other information involving the government or social media companies dating to 2015.

Complying consumed time and money. The threat of legal action dried up funding from donors — which had included philanthropies, corporations and the government — and struck fear in researchers worried about facing legal action and political threats online for the work.

“You had a lot of organizations doing this research,” a senior analyst at one of them said, speaking on the condition of anonymity because of fear of legal retribution. “Now, there are none.”

The Biden administration also found its hands tied. On July 4, 2023, Judge Doughty issued a sweeping injunction, saying that the government could not reach out to the platforms, or work with outside groups monitoring social media content, to address misinformation, except in a narrow set of circumstances.

The ruling went further than some of the plaintiffs in the Missouri case had expected. Judge Doughty even repeated an incorrect statistic first promoted by Mr. Benz: The partnership had flagged 22 million messages on Twitter alone, he wrote. In fact, it had flagged fewer than 5,000.

The Biden administration appealed.

While the judge said the administration could still take steps to stop foreign election interference or posts that mislead about voting requirements, it was unclear how it could without communicating “with social media companies on initiatives to prevent grave harm to the American people and our democratic processes,” the government asserted in its appeal.

In September, the U.S. Court of Appeals for the Fifth Circuit scaled the order back significantly, but still found the government had most likely overstepped the limits of the First Amendment. That sent the case to the Supreme Court, where justices recently expressed deep reservations about government intrusions in social media.

Ahead of the court’s decision, agencies across the government have virtually stopped communicating with social media companies, fearing the legal and political fallout as the presidential election approaches, according to several government officials who described the retreat on the condition of anonymity.

In a statement, Cait Conley, a senior adviser at the Cybersecurity and Infrastructure Security Agency, said the department was still strengthening partnerships to fight “risks posed by foreign actors.” She did not address online threats at home.

The platforms have also backed off. Facebook and YouTube announced that they would reverse their restrictions on content claiming that the 2020 election was stolen. The torrent of disinformation that the previous efforts had slowed, though not stopped, has resumed with even greater force.

Hailing the end of “that halcyon period of the censorship industry,” Mr. Benz has found new celebrity, sitting for interviews with Tucker Carlson and Russell Brand. His conspiracy theories, like the one about the Pentagon’s use of Taylor Swift, have aired on Fox News and become talking points for many Republicans.

The biggest winner, arguably, has been Mr. Trump, who casts himself as victim and avenger of a vast plot to muzzle his movement.

Mr. Biden is “building the most sophisticated censorship and information control apparatus in the world,” Mr. Trump said in a campaign email last week, “to crush free speech in America.”

Glenn Thrush and Luke Broadwater contributed reporting.

Jim Rutenberg is a writer at large for The Times and The New York Times Magazine and writes most often about media and politics. More about Jim Rutenberg

Steven Lee Myers covers misinformation for The Times. He has worked in Washington, Moscow, Baghdad and Beijing, where he contributed to the articles that won the Pulitzer Prize for public service in 2021. He is also the author of “The New Tsar: The Rise and Reign of Vladimir Putin.” More about Steven Lee Myers

Our Coverage of the 2024 Elections

Presidential Race

President Biden raised $25 million  campaigning alongside Barack Obama and Bill Clinton  at a Radio City Music Hall event , and held a retreat the next day  for 175 major donors.

Donald Trump pushed his law-and-order message  at a wake for a police officer killed on duty.

Trump Media, now publicly traded, could present new conflicts of interest  in a second Trump term.

Donald Trump cast Robert F. Kennedy Jr.  as a liberal democrat  in disguise  while also seeming to back the independent presidential candidate as a spoiler for the Biden campaign.

Other Key Races

Tammy Murphy, New Jersey’s first lady, abruptly ended her bid for U.S. Senate, a campaign flop that reflected intense national frustration with politics as usual .

Kari Lake, a Trump acolyte running for Senate in Arizona, is struggling to walk away from the controversial positions  that have turned off independents and alienated establishment Republicans.

Ohio will almost certainly go for Trump this November. Senator Sherrod Brown, the last Democrat holding statewide office, will need to defy the gravity of the presidential contest  to win a fourth term.

Advertisement

  • Logged in as:
  • My Analyses
  • New Analysis
  • What's new
  • Research Team

Univariate and multivariate skewness and kurtosis calculation

Turkey's resurgent opposition trounces Erdogan in pivotal local elections

  • Istanbul's CHP mayor well ahead, results show
  • Opposition keeps Ankara, claims other cities from AKP
  • Economic strain, dissatisfied Islamist voters hurt Erdogan
  • Mayor Imamoglu now 'default nemesis' of Erdogan

Local elections in Istanbul

IMAMOGLU'S RISE

The Reuters Daily Briefing newsletter provides all the news you need to start your day. Sign up here.

Additional reporting by Daren Butler, Ali Kucukgocmen and Bulent Usta in Istanbul and Ece Toksabay, Tuvan Gumrukcu and Huseyin Hayatsever in Ankara; Writing by Jonathan Spicer; Editing by Barbara Lewis, Louise Heavens, Leslie Adler and Jonathan Oatis

Our Standards: The Thomson Reuters Trust Principles. , opens new tab

Demonstration in the ultra-Orthodox neighbourhood of Mea Shearim, in Jerusalem

Two civilians have been injured in Israeli strikes on the outskirts of Damascus on Sunday, the Syrian defence ministry said, in the second such attack on the country in a few days.

Anti-government protesters launch a prolonged demonstration, in Jerusalem

Exclusive: Iran alerted Russia to security threat before Moscow attack

Iran tipped off Russia about the possibility of a major "terrorist operation" on its soil ahead of the concert hall massacre near Moscow last month, three sources familiar with the matter said.

Former Pakistani PM Imran Khan speaks with Reuters during an intervew, in Lahore

IMAGES

  1. Power Analysis

    power analysis online

  2. Power Analysis, Clearly Explained!!!

    power analysis online

  3. How To Use Power Analysis To Determine The Appropriate Sample Size Of A

    power analysis online

  4. Multiple Regression Power Analysis

    power analysis online

  5. Power analysis

    power analysis online

  6. Introducing the new PowerCenter

    power analysis online

VIDEO

  1. 1-1 Power analysis framing part 1 : counterside

  2. Power analysis in ANOVA

  3. Power Analysis and Advocacy تحليل القوة والمناصرة

  4. Securing AES Designs Against Power Analysis Attacks A Survey

  5. Tata power , jsw energy , cg Power , suzlon power analysis #tatapower #jswenergy #suzlonenergy

  6. Separation of Power: Analysis of various theories

COMMENTS

  1. Power and Sample Size

    Find out how to design experiments, studies, and surveys with optimal power and sample size. Use free, online, easy-to-use calculators for various statistical tests and scenarios.

  2. Sample Size Calculator & Statistical Power Calculator

    Calculate the sample size or power for one-sample, two-sample, or k-sample experiments with different types of outcomes and hypotheses. Enter the parameters for your experimental design and get the results for your statistical test.

  3. Sample Size Calculator

    Calculate the minimum number of subjects for adequate study power based on various parameters such as incidence, variance, treatment effect, alpha and beta. Choose from different study designs and endpoints (dichotomous or continuous).

  4. All Power Calculator

    Calculate the test power for various statistical tests based on the sample size and significance level. See the power analysis chart and learn how to interpret the regions of acceptance and rejection.

  5. Statistical Power Analysis Online

    WebPower is a free online collection of tools for conducting statistical power analysis. It provides tutorials, manual, references, and citation information for users.

  6. S.5 Power Analysis

    Learn how to perform power analysis for hypothesis testing and calculate the sample size needed to achieve a specified level of power. See examples and formulas for one-tailed and two-tailed tests.

  7. WebPower

    WebPower - Statistical Power Analysis and Sample Size Planning for Effect Size Calculator for One-way ANOVA. WebPower is a collection of tools for conducting both basic and advanced statistical power analysis including correlation, proportion, t-test, one-way ANOVA, two-way ANOVA, linear regression, logistic regression, Poisson regression, mediation analysis, longitudinal data analysis ...

  8. Post-hoc Power Calculator

    The Dangers of Post-Hoc Analysis. Post-hoc power analysis has been criticized as a means of interpreting negative study results. 2 Because post-hoc analyses are typically only calculated on negative trials (p ≥ 0.05), such an analysis will produce a low post-hoc power result, which may be misinterpreted as the trial having inadequate power.

  9. How to Calculate Sample Size Needed for Power

    Statistical power and sample size analysis provides both numeric and graphical results, as shown below. The text output indicates that we need 15 samples per group (total of 30) to have a 90% chance of detecting a difference of 5 units. The dot on the Power Curve corresponds to the information in the text output.

  10. Sample Size Calculators

    Sample Size Calculators. If you are a clinical researcher trying to determine how many subjects to include in your study or you have another question related to sample size or power calculations, we developed this website for you. Our approach is based on Chapters 5 and 6 in the 4th edition of Designing Clinical Research (DCR-4), but the ...

  11. Sample Size Calculators

    Sample Size Calculators. Statistical considerations for clinical trials and scientific experiments. Find sample size, power or the minimal detectable difference for parallel studies, crossover studies, or studies to find associations between variables, where the dependent variable is Success or Failure, a Quantitative Measurement, or a time to ...

  12. Power/Sample Size Calculator

    After making your entries, hit the calculate button at the bottom. Calculate Sample Size (for specified Power) Calculate Power (for specified Sample Size) Enter a value for mu1: Enter a value for mu2: Enter a value for sigma: 1 Sided Test. 2 Sided Test. Enter a value for α (default is .05):

  13. Introduction to Power Analysis

    Learn the basics of power analysis, the probability of detecting an effect, and the factors that affect it. This seminar covers the definitions, concepts, and limitations of power analysis, but not the formulas or software.

  14. Power T Z Calculator

    Information. Calculates the test power for the specific sample size and draw a power analysis chart. For the two-tailed test, it calculates the strict interpretation, includes the probability to reject the null assumption in the opposite tail of the true effect. Use this test for one of the following tests:

  15. Sample size calculators

    The chart shows the power per each sample size. The black bar shows the sample size that achieves the required power. In this example 102 achieves the power of 0.8. When you hover over the power chart in the calculator, you may see the sample size and the power it achieves. In this example a sample size of

  16. WebPower

    WebPower - Statistical Power Analysis and Sample Size Planning for Effect Size Calculator for Two-way ANOVA. WebPower is a collection of tools for conducting both basic and advanced statistical power analysis including correlation, proportion, t-test, one-way ANOVA, two-way ANOVA, linear regression, logistic regression, Poisson regression, mediation analysis, longitudinal data analysis ...

  17. Quick-R: Power Analysis

    Overview. Power analysis is an important aspect of experimental design. It allows us to determine the sample size required to detect an effect of a given size with a given degree of confidence. Conversely, it allows us to determine the probability of detecting an effect of a given size with a given level of confidence, under sample size ...

  18. Sample Size Calculator for Comparing Paired Differences

    Select one of the two options to specify input values. Hover over the sign to obtain help. Mean and Standard Deviation of the Differences. Effect Size. Click the Options button to change the default options for Power, Significance, Alternate Hypothesis and Group Sizes. Click the Adjust button to adjust sample sizes for t-distribution (option ...

  19. Power Analysis Sample Size Calculation Statistical Software

    Sample Size - Power Analysis Software: To find or calculate the sample size for a planned study. Power And Precision is a computer program for statistical power analysis. This software features an extremely clear interface, and it allows researchers to create reports, tables and graphs. Free trial download.

  20. Power Spectrum Analysis

    Explore math with our beautiful, free online graphing calculator. Graph functions, plot points, visualize algebraic equations, add sliders, animate graphs, and more. Power Spectrum Analysis. Save Copy. Log InorSign Up. Signal: 1. F x = sin 2 π x + 3. 2. Sampling Period and Frequency: 3. P = 5 ...

  21. The Power of Data Analysis in Research

    Any of these challenges can lead to incorrect analysis, impacting organizational decision-making. There are several ways in which data analysts can overcome these challenges, including seeking advice from fellow data analysts or taking self-paced or online training. By overcoming these data analysis challenges, data analysts can ensure they ...

  22. WebPower

    WebPower - Statistical Power Analysis and Sample Size Planning for Effect Size Calculator for Repeated-Measures ANOVA. WebPower is a collection of tools for conducting both basic and advanced statistical power analysis including correlation, proportion, t-test, one-way ANOVA, two-way ANOVA, linear regression, logistic regression, Poisson regression, mediation analysis, longitudinal data ...

  23. How Trump's Allies Are Winning the War Over Disinformation

    In August 2022, a new organization, the Foundation for Freedom Online, posted a report on its website called "Department of Homeland Censorship: How D.H.S. Seized Power Over Online Speech."

  24. The new age of naval power in the Indo-Pacific: strategy, order and

    Defense & Security Analysis Latest Articles. Submit an article Journal homepage. 0 ... CrossRef citations to date 0. Altmetric Book Review. The new age of naval power in the Indo-Pacific: strategy, order and regional security edited by Catherine L. Grant, Alessio Patalano, and James A. Russell, Washington, DC, Georgetown University Press, 2023 ...

  25. Baltimore key bridge collapse live updates: 2 bodies recovered

    Here's what to know: Victims: At least eight people went into the water.Two were rescued but the other six, part of a construction crew that had been filling potholes on the bridge, are missing and presumed dead.; Investigation: The cargo ship, called Dali, lost power and issued a mayday call shortly before it rammed the bridge. The U.S. Coast Guard boarded the vessel Wednesday to collect ...

  26. Guest Editorial: Dynamic analysis, control, and situation awareness of

    His research interests include power system analysis and control, stability of power-electronic-based power system, and large-scale renewable power generation and integration. Yongning Chi received their B.Eng. and M.Eng. degrees from Shandong University, Jinan, China, in 1995 and 2002, respectively, and their Ph.D. degree in electrical ...

  27. WebPower

    WebPower - Statistical Power Analysis and Sample Size Planning for Univariate and multivariate skewness and kurtosis calculation. WebPower is a collection of tools for conducting both basic and advanced statistical power analysis including correlation, proportion, t-test, one-way ANOVA, two-way ANOVA, linear regression, logistic regression, Poisson regression, mediation analysis, longitudinal ...

  28. The key factors that led to the Baltimore bridge collapse

    Ship reported 'power issue,' lost propulsion At the time of the collision, the 984-foot-long DALI was leaving the Port of Baltimore and bound for Colombo, Sri Lanka, officials said.

  29. Turkey's resurgent opposition trounces Erdogan in pivotal local

    It marked the worst defeat for Tayyip Erdogan and his AK Party in their more than two decades in power, and could signal a change in the country's divided political landscape. Erdogan called it a ...