Meta-analysis and evidence synthesis for human resource management

Evidence synthesis in human resource management

Human resource management (HRM) research asks whether the way organizations manage people affects results. Examples include whether high-performance work systems improve firm performance, whether training increases productivity and retention, whether pay for performance changes effort, and whether practices such as flexible working affect engagement. Meta-analyses of these questions have shaped teaching and practice, and the field has a movement toward evidence-based HR that parallels evidence-based medicine.

The evidence has features that call for careful reviewing. Practices are bundled in systems, and studies measure the bundle with different checklists. Outcomes are measured at the employee, unit or firm level, and relationships differ across levels. Most studies are cross-sectional surveys in which managers report both practices and performance, creating shared-source bias. Causal order is uncertain, because successful firms can afford more HR practices. And the context of country, industry and labor law affects how practices work.

Our methods follow meta-analysis and systematic review. This page extends the general guidance for management and business.

HR practices, systems and how they are measured

A review should define the practices it covers and the way each study measured them, code the number and type of practices, and analyze single practices and bundles separately when data allow. Meta-analyses of high-performance work systems have found positive average associations with performance, with a debate about how much depends on measurement and design. In particular, studies with performance data from independent sources typically show smaller associations than studies with manager-reported performance.

Levels of analysis and the ecological problem

HR practices are set at the organization or unit level, while attitudes and behaviors belong to employees. A positive firm-level association between HR practices and productivity does not imply that each employee reacts to the practices in the same way. Studies that use multilevel models report effects at each level, and aggregated data lose individual variation. Reviews should code the level at which each variable and each effect were measured and not mix levels. Employees' perceptions of practices often differ from managers' descriptions of them, so the perceived and intended practices need separate treatment. Where possible, the review should compare effects on outcomes using employee perceptions with those using manager reports.

Sample sizes at the firm level are small, often under 200 firms, and response rates in firm surveys are low. This limits precision and raises the risk of non-response bias. Reviews should extract the number of firms and response rates and test whether either relates to effect sizes.

Causality, endogeneity and quasi-experiments

Firms choose their HR practices, and the choice is not random. Profitable firms may invest more in training, which would create a positive association even if training had no effect on profit. Panel studies with fixed effects, instrumental variables, difference-in-differences and natural experiments address this problem, and field experiments in organizations provide the strongest evidence, though they are few. Reviews should classify designs by their ability to support causal claims and compare pooled effects across the classes. A typical finding is that associations shrink as design strength increases. Reviews should say so and avoid language of effect when the evidence is correlational.

Lagged designs, where practices at one time predict performance at a later time, help but do not remove reverse causation. The time lag chosen affects the result: training effects on productivity may take months to appear. Coding the lag and testing it as a moderator is advisable.

Context: country, industry and labor institutions

HR practices operate in institutional settings. Rules on dismissal, union presence, wage-setting and benefits differ between countries, and so does what employees expect. Practices that are associated with higher performance in one country may be neutral in another. Meta-regression on country-level variables, such as the strength of employment protection or cultural dimensions, can show whether relationships differ, though the number of countries in the data is small and country is confounded with industry and period. Industry matters too: manufacturing and service settings differ in how work is organized. Reviews should provide a table of countries and industries represented and avoid broad generalizations from samples dominated by one country, often the United States or the United Kingdom.

Firm size and sector (public, private, non-profit) are further moderators. Small firms often use informal HR practices that survey instruments, designed for large firms, may miss.

Selection, training and retention evidence

Selection and assessment evidence is covered under organizational psychology. Training evaluation meta-analyses report effects on learning and on behavior on the job, with larger effects on knowledge than on transfer. Studies with delayed follow-up are fewer, and work environment variables such as supervisor support and opportunity to use skills moderate transfer. Retention research includes studies of turnover drivers, including pay, job embeddedness and commitment, and interventions such as onboarding and mentoring. Turnover is a binary outcome in time, and survival methods are appropriate in primary studies. Reviews should state whether voluntary and involuntary turnover were distinguished, since the causes are different.

Engagement, well-being and flexible work

Many HR syntheses now include outcomes beyond productivity: employee engagement, burnout, job satisfaction and well-being. These constructs are measured by self-report scales with different item sets, so a review needs a rule for deciding which instruments count as the same construct. A narrow rule gives few studies; a broad rule pools measures that may not be interchangeable. We recommend stating the rule in the protocol, extracting the instrument name for every effect, and testing whether results differ by instrument family.

Flexible work, remote work and working-time arrangements are a common recent topic. Primary studies range from randomized or natural experiments in single firms to cross-sectional surveys. These designs answer different questions, and a review should keep them apart: an experiment in one call centre says something about causal effects in that setting, while a survey of employees who chose flexible arrangements reflects who chose them. Pooling both into one estimate without a design moderator hides the difference.

Well-being outcomes also raise the question of who reports. Employee self-reports, manager ratings and administrative records (absence days, for example) can each be pooled, but they should not be mixed in one analysis without a clear justification, because their errors differ.

How to read an association in HR research

Suppose a review finds a pooled correlation of 0.15 between a bundle of high-involvement practices and firm productivity. This is an invented illustration, not a finding. Three questions follow. First, what is the direction of the relationship? Profitable firms can afford more training and better pay, so a positive correlation is compatible with reverse causation. Second, which practices drive it? A bundle index cannot say. Third, how large is it in context? A correlation of 0.15 corresponds to a modest standardized difference, and whether that matters depends on the cost of the practices and the scale of the firm.

Good reviews answer these by reporting designs separately, by using lagged or panel studies where they exist, and by describing effects on a scale that managers can interpret. They also report the prediction interval, since an average association says little about whether a practice will work in a particular organization.

From evidence to HR decisions

Evidence-based management asks decision-makers to combine the best available research with local data, practitioner judgment and the interests of those affected. A meta-analysis is one input. It can tell an HR team which practices have a record of association with the outcome they care about and which have little supporting evidence. It cannot tell them whether the practice will fit their workforce, budget or legal context. We write the discussion section of an HR review with that boundary in mind: findings are stated with their certainty, the settings where evidence is thin are named, and recommendations are limited to what the data can support.

For organizations commissioning a review, the most useful early step is a precise question: which practice, for which employees, compared with what, and measured how. A vague question about whether HR matters produces a vague answer.

Common pitfalls we look for

• Treating a practice index as a single intervention without describing its content.
• Mixing employee-level and firm-level effects.
• Relying on manager reports of both practices and outcomes.
• Interpreting associations as causal effects.
• Generalizing from a few countries to all.
• Using turnover intention as a stand-in for actual turnover without noting the difference.

Planning and reporting

The protocol defines the practices, the levels, the outcomes, the effect size and corrections, the designs and the moderators (country, industry, size, design strength). Searches cover Business Source Complete, ABI/INFORM, PsycINFO, Web of Science, Scopus, EconLit and working paper series. A coding manual with decision rules is developed and tested. Reporting follows PRISMA 2020 and the standards of management journals, including a table of coded studies and the data and code.

Managers who use reviews need practical findings. A report that states, for each practice, the size of the average association, the range across settings, the strength of the design and the cost and feasibility where known is more useful than a single headline figure. The report should also say what the review cannot tell.

How we support research projects in this area

Boundaries of this service

A review of HRM studies describes average associations across organizations and employees. It does not provide HR consulting, legal advice or recommendations about any employee or organization, and decisions about hiring, pay or dismissal involve legal and ethical requirements that a research summary cannot address.

Frequently asked questions