Meta-analysis and evidence synthesis for organizational behavior

Evidence synthesis in organizational behavior

Organizational behavior (OB) asks how people behave in organizations and why. Typical questions are whether transformational leadership relates to team performance, how job satisfaction relates to performance and turnover, whether perceived fairness affects citizenship behavior, and how team diversity relates to outcomes. Because the field relies on survey measures of attitudes and perceptions, most of its meta-analyses pool correlations, and the field has contributed much of the methodology for correcting correlations for measurement error.

The evidence raises several problems for a reviewer. Constructs overlap: some leadership styles correlate so strongly that their separate effects are hard to identify. Many studies use one survey with one respondent, so shared method inflates correlations. Cause and effect are often unclear. Our methods follow meta-analysis and systematic review practice, extended for these features. This page builds on the general guidance for management and business and the neighboring page on organizational psychology.

Overlapping constructs and discriminant validity

A review should define each construct, state which instruments count and how mixed-content scales were handled, and report discriminant validity concerns. When two constructs correlate above about 0.7 in the primary data, an analysis that treats them as separate predictors should be interpreted with caution. Relative-weights analysis on a pooled correlation matrix is one way to ask which of several related predictors accounts for most of the explained variance, though it describes statistical contribution, not causal importance.

Measurement error and psychometric corrections

Observed correlations are attenuated by unreliability of measures and by range restriction. The psychometric approach of Hunter and Schmidt corrects for these artifacts, and estimates how much of the observed variance across studies is due to sampling error and artifacts. Corrected values give a better estimate of the relationship between constructs, but they rely on reliability estimates that are often taken from other sources, and they have wider uncertainty than the uncorrected values. A review should report both corrected and uncorrected estimates, state where reliabilities came from, and avoid using the corrected value as if it were an observed effect.

Other methods, such as those following Hedges and Olkin, do not correct for artifacts by default and work with the observed effects and their sampling variances. The two traditions can give different answers, and the choice should be stated in the protocol. Readers who care about practical effects should see the uncorrected value, since it represents what is found in real measures.

Common-method bias and source

When the same person reports on both predictor and outcome, in the same survey and at the same time, some of the correlation reflects shared method. Examples are a manager rating their own leadership and team performance, or an employee rating supervisor behavior and job attitudes. Studies that separate sources, use time lags or draw outcomes from records show different, usually smaller, associations. A review should code the source of each variable and the time between measures, and test whether effects differ by design.

Statistical remedies for common-method bias in primary studies, such as a single-factor test, are weak, and a review should not accept them as proof that the problem is absent. The more informative step is to compare studies with and without separate sources.

Causal inference in OB

Cross-sectional correlations cannot show whether satisfaction causes performance, performance causes satisfaction, or both follow from a third factor. Longitudinal and cross-lagged studies are more informative, and meta-analytic structural models can pool cross-lagged paths when studies report them. Field and laboratory experiments on leadership training, team interventions or job redesign give stronger causal evidence but are fewer, and their effects often differ from those of correlational work. We report designs separately and use them as moderators, and we avoid causal language when the data are cross-sectional.

Teams, groups and multilevel structure

Team research measures inputs (composition, diversity, leadership), processes (communication, conflict, cohesion) and outcomes (performance, viability). Data are clustered: members within teams within organizations. A review must record the level at which each variable was measured and whether it was aggregated from individuals, and should not combine individual-level and team-level correlations in one estimate. Team-level studies tend to have small samples, often fewer than 100 teams, so precision is limited.

Diversity research illustrates the problem. Effects depend on the type of diversity (demographic, functional, informational), the task and the measurement of diversity, such as variance, count or proportion. A review should keep these distinctions and examine moderators rather than seek a single estimate of the effect of diversity.

Leadership evidence

Leadership meta-analyses cover transformational, transactional, servant, ethical, authentic and other styles. Questionnaires differ across styles and across versions of the same questionnaire, and the correlation between styles is high. Outcomes include follower satisfaction, performance, citizenship behavior and team effectiveness. A review should record the instrument, the source of ratings and the outcome source, because the strongest associations tend to arise when the same followers rate both leader and outcomes. We report results by instrument family, compare them across sources, and describe the proportion of effects that come from single-source surveys.

Publication bias and selective reporting

OB journals tend to favor theory-confirming and significant results. Funnel-plot methods, selection models and comparison of published with unpublished effects (dissertations, conference papers) can show whether the published record is likely to overstate effects. Many OB studies report correlations as part of a larger table without being the focus, and such incidental effects are less likely to be selected for significance, which affects how bias tests are interpreted. We report both and discuss the limits.

Motivation, justice and well-being at work

Motivation theories (goal setting, self-determination, expectancy) and justice research have large meta-analytic literatures. Goal-setting studies include many laboratory experiments with students, while self-determination work relies more on surveys of employees. A review should keep these designs separate and should report the type of task, since effects of goals differ between simple and complex tasks. For justice, the main decision is whether to analyze overall fairness or separate procedural, distributive and interactional facets, which are correlated but can have different relationships with outcomes such as trust and citizenship behavior.

Well-being outcomes, including stress, burnout and work-family conflict, are often measured by self-report scales with different cut-offs and item sets. Reviews should name the instrument, report whether clinical thresholds were used and avoid interpreting scale scores as diagnoses. The service does not provide clinical or occupational health advice.

Reading a pooled correlation

Suppose a review finds a corrected mean correlation of 0.30 between a leadership style and follower satisfaction, with a 90 percent credibility interval from 0.10 to 0.50. These numbers are invented for illustration. The mean tells us the typical association; the credibility interval tells us the range of true values across settings, and its width shows that the relationship is not the same everywhere. A correlation of 0.30 means the predictor accounts for about 9 percent of variance in the outcome, which is a modest share. If the observed correlation before correction was 0.22, the reader should see both, with an explanation of the reliabilities used.

The question for a practitioner is whether the relationship holds in their setting. The credibility interval, the moderator analyses and the table of study settings give the evidence to judge that, and the review should say plainly when the data cannot answer it.

The same reading applies to any pooled correlation in this field. A mean value is a summary, the spread around it matters as much, and the design behind each study determines how far the value can be read as a causal effect.

Reporting and transparency standards

OB reviews follow PRISMA 2020 for reporting and the Meta-Analysis Reporting Standards of the American Psychological Association where journals request them. We recommend sharing the coded data set, the coding manual and the analysis code so that others can reproduce the results, and we document every judgment call in coding, because the choice of which effect to extract from a complex table can change results. Studies of meta-analytic practice have shown that many reviews give too little detail to reproduce their numbers, so transparency here is a practical safeguard.

Common pitfalls we look for

• Pooling closely overlapping constructs as if distinct.
• Reporting only corrected correlations without the observed values.
• Ignoring same-source and same-time measurement.
• Mixing individual-level and team-level effects.
• Causal language from cross-sectional data.
• Using a single sample multiple times across publications.

Planning an OB evidence synthesis

We help define constructs and instruments, decide between psychometric and observed-effect approaches, plan coding of source, time lag and level, and write the protocol. For theory-testing questions we set out the structural model before looking at results. See the meta-analysis service for scope and process.

How we support research projects in this area

Boundaries of this service

A synthesis of organizational behavior research describes average associations across studies and settings. It does not assess individual employees, evaluate a specific organization or provide management advice. Most data are correlational and self-reported, so causal conclusions need stronger designs. Findings from one country, industry or period may not apply elsewhere.

Frequently asked questions