Statistical analysis service

What the service covers

Statistical analysis turns the data of a study into evidence about its question. The same data can be analyzed well or badly: a test can be chosen because it gives a small p-value and not because it fits the design, assumptions can go unchecked, missing data can be ignored, and the analysis can be repeated until it produces something publishable. A sound analysis is planned in advance, fits the design, checks its assumptions, reports what was done and shows how uncertain the results are.

This service supports researchers who have collected data and need an analysis that will withstand the scrutiny of reviewers and readers. It differs from the meta-analysis service, which combines the results of several studies, and from original research support, which also covers study design and manuscript preparation, although the services can be combined. The data themselves are yours and are not generated or altered by us, as set out in the research integrity statement.

What the service includes

The analysis plan

The statistical analysis plan is the most useful single safeguard in primary research. It states, before the analysis, which outcome is primary, what comparison answers the question, which model will be used and with which covariates, how missing data and outliers will be handled, which assumptions will be checked and what happens if they fail, and which analyses are secondary or exploratory. For randomized trials, guidance exists on what such a plan should contain. The plan does not prevent exploration; it separates the analyses that were planned from those that arose from looking at the data, so that the exploratory ones are reported as such.

When the data have already been seen, which is common, the plan is still written and the report is honest that it was written after data collection and, where it applies, after a first look. A plan made after seeing the results can still document the analytic reasoning, but it carries less weight, and the report says so.

Matching the analysis to the design

The design determines which analyses are valid. A few examples show what this means in practice.

Randomized trials

Analysis follows the randomization, usually by intention to treat. Baseline covariates used in stratification are included, and the effect is estimated with an appropriate model for the outcome: linear for continuous, logistic or log-binomial for binary, and Cox or parametric models for time to event. Clustered and crossover designs need analyses that account for them.

Cohort and case-control studies

Adjustment for confounding is the main task. Covariates are chosen from knowledge of the causal structure and not by significance, and the report describes the risk of residual confounding.

Repeated measures and clustered data

Observations from the same participant, family, clinic or school are correlated. Mixed-effects models or generalized estimating equations account for this, and ignoring it gives intervals that are too narrow.

Survival data

Time-to-event outcomes need methods that handle censoring. The proportional hazards assumption is checked and alternatives are used when it fails.

Surveys and questionnaires

Scale scores need reliability checks, and complex samples need weights. Ordinal items call for ordinal models and not for treatment as continuous by default.

Diagnostic and prediction studies

Accuracy is estimated with sensitivity, specificity and discrimination and calibration measures, and prediction models are validated and reported against TRIPOD.

Common models by type of outcome

The type of outcome narrows the choice of model. The table lists the usual starting points, which are adapted to the design, the sample size and the checks of assumptions.

Rank-based tests such as the Mann-Whitney or Kruskal-Wallis are options when assumptions of a parametric model cannot be met, but they test a different hypothesis and give no direct effect size, so a model with robust inference is often preferred. Whatever model is used, the report gives an estimate and an interval and not only a test statistic.

Sample size and power

The sample size determines whether a study can answer its question. A study with too few participants is likely to miss a real effect and, when it does find one, to overestimate it. Where the data are not yet collected, a sample-size calculation is part of design and requires an assumed effect size, variability, error rates and allowance for dropout, each of which should be justified from earlier evidence or from the smallest effect that would matter in practice. Where the data already exist, a post hoc calculation of power from the observed effect adds nothing, because it is a function of the p-value. It is more useful to report the confidence interval, which shows which effects the data rule out.

Missing data

Almost every dataset has missing values, and how they are handled can change the results. The first step is to understand why they are missing, whether by design, by chance, because of the value itself or because of other observed variables. A complete-case analysis, which drops participants with any missing value, is valid only under strong conditions and can lose power and introduce bias. Multiple imputation, which creates several plausible versions of the data and combines the results, is a widely used and defensible approach when the data are missing at random conditional on the variables in the imputation model. Where data may be missing not at random, sensitivity analyses explore how conclusions change under different assumptions. The approach is set in the plan, the amount of missing data is reported, and the conclusions are checked against alternatives.

Multiple testing and p-values

Each test carries a chance of a false positive, and many tests raise the chance that something will appear significant. The primary outcome is specified in advance for this reason, and secondary and exploratory analyses are labeled and, where appropriate, adjusted. A p-value is a statement about the compatibility of the data with a null hypothesis under a model, not the probability that a hypothesis is true or a measure of the size or importance of an effect. The statistical community's guidance is to report estimates with intervals, to avoid reducing results to a significant or non-significant verdict at a threshold, and to interpret findings in light of the design and the context. The analysis and the write-up follow this guidance.

Reproducibility and code

An analysis should be repeatable by someone else. The work is therefore done in scripts and not through manual steps, and the scripts, the software versions and the settings are recorded and delivered. Data cleaning is scripted, so that every change from the raw data is traceable. Random processes such as imputation and resampling use recorded seeds. Where data cannot be shared because of privacy, the code can still be provided, and a description of the data structure allows others to see what was done. See the guide on reproducible analysis.

Reporting

Reporting guidelines exist for most designs: CONSORT for randomized trials, STROBE for observational studies, STARD for diagnostic accuracy, TRIPOD for prediction models and others catalogued by the EQUATOR Network. They specify what a reader needs in order to judge a study. For statistical content, general guidance asks for a description of every method used, the software, how missing data were handled, the exact results with confidence intervals, and a clear separation of planned and exploratory analyses. The methods section is written to be understood by a statistician reviewer and by a subject-matter reader. See reporting guidelines.

Deliverables

Optional extension: full manuscript and submission

Limitations and boundaries

Statistics cannot repair a flawed design or poor data. A confounded comparison remains confounded after adjustment, a measurement that is invalid stays invalid, and a very small sample cannot support precise conclusions. We analyze the data we are given and do not collect, generate or alter data. If the data do not support the planned analysis, we say so and discuss the alternatives, which may include a different question. We will not select analyses to reach a preferred conclusion. Statistical analysis for a manuscript is research support and does not provide clinical advice, and the results are not patient-specific guidance.

How long does it take?

The time depends on the size and condition of the dataset, the number of outcomes and models, how much cleaning is needed, how complex the design is, and how many rounds of review follow. A clean dataset with a single primary outcome is a short project. A messy dataset with repeated measures and multiple outcomes takes longer, and the cleaning is often the larger part. After reviewing the data and the plan, a schedule and its dependencies are agreed.

Frequently asked questions