the analysis and synthesis of

Analysis vs. Synthesis

What's the difference.

Analysis and synthesis are two fundamental processes in problem-solving and decision-making. Analysis involves breaking down a complex problem or situation into its constituent parts, examining each part individually, and understanding their relationships and interactions. It focuses on understanding the components and their characteristics, identifying patterns and trends, and drawing conclusions based on evidence and data. On the other hand, synthesis involves combining different elements or ideas to create a new whole or solution. It involves integrating information from various sources, identifying commonalities and differences, and generating new insights or solutions. While analysis is more focused on understanding and deconstructing a problem, synthesis is about creating something new by combining different elements. Both processes are essential for effective problem-solving and decision-making, as they complement each other and provide a holistic approach to understanding and solving complex problems.

Further Detail

Introduction.

Analysis and synthesis are two fundamental processes in various fields of study, including science, philosophy, and problem-solving. While they are distinct approaches, they are often interconnected and complementary. Analysis involves breaking down complex ideas or systems into smaller components to understand their individual parts and relationships. On the other hand, synthesis involves combining separate elements or ideas to create a new whole or understanding. In this article, we will explore the attributes of analysis and synthesis, highlighting their differences and similarities.

Attributes of Analysis

1. Focus on details: Analysis involves a meticulous examination of individual components, details, or aspects of a subject. It aims to understand the specific characteristics, functions, and relationships of these elements. By breaking down complex ideas into smaller parts, analysis provides a deeper understanding of the subject matter.

2. Objective approach: Analysis is often driven by objectivity and relies on empirical evidence, data, or logical reasoning. It aims to uncover patterns, trends, or underlying principles through systematic observation and investigation. By employing a structured and logical approach, analysis helps in drawing accurate conclusions and making informed decisions.

3. Critical thinking: Analysis requires critical thinking skills to evaluate and interpret information. It involves questioning assumptions, identifying biases, and considering multiple perspectives. Through critical thinking, analysis helps in identifying strengths, weaknesses, opportunities, and threats, enabling a comprehensive understanding of the subject matter.

4. Reductionist approach: Analysis often adopts a reductionist approach, breaking down complex systems into simpler components. This reductionist perspective allows for a detailed examination of each part, facilitating a more in-depth understanding of the subject matter. However, it may sometimes overlook the holistic view or emergent properties of the system.

5. Diagnostic tool: Analysis is commonly used as a diagnostic tool to identify problems, errors, or inefficiencies within a system. By examining individual components and their interactions, analysis helps in pinpointing the root causes of issues, enabling effective problem-solving and optimization.

Attributes of Synthesis

1. Integration of ideas: Synthesis involves combining separate ideas, concepts, or elements to create a new whole or understanding. It aims to generate novel insights, solutions, or perspectives by integrating diverse information or viewpoints. Through synthesis, complex systems or ideas can be approached holistically, considering the interconnections and interdependencies between various components.

2. Creative thinking: Synthesis requires creative thinking skills to generate new ideas, concepts, or solutions. It involves making connections, recognizing patterns, and thinking beyond traditional boundaries. By embracing divergent thinking, synthesis enables innovation and the development of unique perspectives.

3. Systems thinking: Synthesis often adopts a systems thinking approach, considering the interactions and interdependencies between various components. It recognizes that the whole is more than the sum of its parts and aims to understand emergent properties or behaviors that arise from the integration of these parts. Systems thinking allows for a comprehensive understanding of complex phenomena.

4. Constructive approach: Synthesis is a constructive process that builds upon existing knowledge or ideas. It involves organizing, reorganizing, or restructuring information to create a new framework or understanding. By integrating diverse perspectives or concepts, synthesis helps in generating comprehensive and innovative solutions.

5. Design tool: Synthesis is often used as a design tool to create new products, systems, or theories. By combining different elements or ideas, synthesis enables the development of innovative and functional solutions. It allows for the exploration of multiple possibilities and the creation of something new and valuable.

Interplay between Analysis and Synthesis

While analysis and synthesis are distinct processes, they are not mutually exclusive. In fact, they often complement each other and are interconnected in various ways. Analysis provides the foundation for synthesis by breaking down complex ideas or systems into manageable components. It helps in understanding the individual parts and their relationships, which is essential for effective synthesis.

On the other hand, synthesis builds upon the insights gained from analysis by integrating separate elements or ideas to create a new whole. It allows for a holistic understanding of complex phenomena, considering the interconnections and emergent properties that analysis alone may overlook. Synthesis also helps in identifying gaps or limitations in existing knowledge, which can then be further analyzed to gain a deeper understanding.

Furthermore, analysis and synthesis often involve an iterative process. Initial analysis may lead to the identification of patterns or relationships that can inform the synthesis process. Synthesis, in turn, may generate new insights or questions that require further analysis. This iterative cycle allows for continuous refinement and improvement of understanding.

Analysis and synthesis are two essential processes that play a crucial role in various fields of study. While analysis focuses on breaking down complex ideas into smaller components to understand their individual parts and relationships, synthesis involves integrating separate elements or ideas to create a new whole or understanding. Both approaches have their unique attributes and strengths, and they often complement each other in a cyclical and iterative process. By employing analysis and synthesis effectively, we can gain a comprehensive understanding of complex phenomena, generate innovative solutions, and make informed decisions.

Comparisons may contain inaccurate information about people, places, or facts. Please report any issues.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Published: 08 March 2018

Meta-analysis and the science of research synthesis

• Jessica Gurevitch 1 ,
• Julia Koricheva 2 ,
• Shinichi Nakagawa 3 , 4 &
• Gavin Stewart 5  

Nature volume  555 ,  pages 175–182 ( 2018 ) Cite this article

57k Accesses

933 Citations

737 Altmetric

Metrics details

• Biodiversity
• Outcomes research

Meta-analysis is the quantitative, scientific synthesis of research results. Since the term and modern approaches to research synthesis were first introduced in the 1970s, meta-analysis has had a revolutionary effect in many scientific fields, helping to establish evidence-based practice and to resolve seemingly contradictory research outcomes. At the same time, its implementation has engendered criticism and controversy, in some cases general and others specific to particular disciplines. Here we take the opportunity provided by the recent fortieth anniversary of meta-analysis to reflect on the accomplishments, limitations, recent advances and directions for future developments in the field of research synthesis.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Buy this article

• Purchase on SpringerLink
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Eight problems with literature reviews and how to fix them

The past, present and future of Registered Reports

Raiders of the lost HARK: a reproducible inference framework for big data science

Jennions, M. D ., Lortie, C. J. & Koricheva, J. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 23 , 364–380 (Princeton Univ. Press, 2013)

Article   Google Scholar  

Roberts, P. D ., Stewart, G. B. & Pullin, A. S. Are review articles a reliable source of evidence to support conservation and environmental management? A comparison with medicine. Biol. Conserv. 132 , 409–423 (2006)

Bastian, H ., Glasziou, P . & Chalmers, I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 7 , e1000326 (2010)

Article   PubMed   PubMed Central   Google Scholar  

Borman, G. D. & Grigg, J. A. in The Handbook of Research Synthesis and Meta-analysis 2nd edn (eds Cooper, H. M . et al.) 497–519 (Russell Sage Foundation, 2009)

Ioannidis, J. P. A. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 94 , 485–514 (2016)

Koricheva, J . & Gurevitch, J. Uses and misuses of meta-analysis in plant ecology. J. Ecol. 102 , 828–844 (2014)

Littell, J. H . & Shlonsky, A. Making sense of meta-analysis: a critique of “effectiveness of long-term psychodynamic psychotherapy”. Clin. Soc. Work J. 39 , 340–346 (2011)

Morrissey, M. B. Meta-analysis of magnitudes, differences and variation in evolutionary parameters. J. Evol. Biol. 29 , 1882–1904 (2016)

Article   CAS   PubMed   Google Scholar  

Whittaker, R. J. Meta-analyses and mega-mistakes: calling time on meta-analysis of the species richness-productivity relationship. Ecology 91 , 2522–2533 (2010)

Article   PubMed   Google Scholar  

Begley, C. G . & Ellis, L. M. Drug development: Raise standards for preclinical cancer research. Nature 483 , 531–533 (2012); clarification 485 , 41 (2012)

Article   CAS   ADS   PubMed   Google Scholar  

Hillebrand, H . & Cardinale, B. J. A critique for meta-analyses and the productivity-diversity relationship. Ecology 91 , 2545–2549 (2010)

Moher, D . et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6 , e1000097 (2009). This paper provides a consensus regarding the reporting requirements for medical meta-analysis and has been highly influential in ensuring good reporting practice and standardizing language in evidence-based medicine, with further guidance for protocols, individual patient data meta-analyses and animal studies.

Moher, D . et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 4 , 1 (2015)

Nakagawa, S . & Santos, E. S. A. Methodological issues and advances in biological meta-analysis. Evol. Ecol. 26 , 1253–1274 (2012)

Nakagawa, S ., Noble, D. W. A ., Senior, A. M. & Lagisz, M. Meta-evaluation of meta-analysis: ten appraisal questions for biologists. BMC Biol. 15 , 18 (2017)

Hedges, L. & Olkin, I. Statistical Methods for Meta-analysis (Academic Press, 1985)

Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36 , 1–48 (2010)

Anzures-Cabrera, J . & Higgins, J. P. T. Graphical displays for meta-analysis: an overview with suggestions for practice. Res. Synth. Methods 1 , 66–80 (2010)

Egger, M ., Davey Smith, G ., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 315 , 629–634 (1997)

Article   CAS   Google Scholar  

Duval, S . & Tweedie, R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56 , 455–463 (2000)

Article   CAS   MATH   PubMed   Google Scholar  

Leimu, R . & Koricheva, J. Cumulative meta-analysis: a new tool for detection of temporal trends and publication bias in ecology. Proc. R. Soc. Lond. B 271 , 1961–1966 (2004)

Higgins, J. P. T . & Green, S. (eds) Cochrane Handbook for Systematic Reviews of Interventions : Version 5.1.0 (Wiley, 2011). This large collaborative work provides definitive guidance for the production of systematic reviews in medicine and is of broad interest for methods development outside the medical field.

Lau, J ., Rothstein, H. R . & Stewart, G. B. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 25 , 407–419 (Princeton Univ. Press, 2013)

Lortie, C. J ., Stewart, G ., Rothstein, H. & Lau, J. How to critically read ecological meta-analyses. Res. Synth. Methods 6 , 124–133 (2015)

Murad, M. H . & Montori, V. M. Synthesizing evidence: shifting the focus from individual studies to the body of evidence. J. Am. Med. Assoc. 309 , 2217–2218 (2013)

Rasmussen, S. A ., Chu, S. Y ., Kim, S. Y ., Schmid, C. H . & Lau, J. Maternal obesity and risk of neural tube defects: a meta-analysis. Am. J. Obstet. Gynecol. 198 , 611–619 (2008)

Littell, J. H ., Campbell, M ., Green, S . & Toews, B. Multisystemic therapy for social, emotional, and behavioral problems in youth aged 10–17. Cochrane Database Syst. Rev. https://doi.org/10.1002/14651858.CD004797.pub4 (2005)

Schmidt, F. L. What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. Am. Psychol. 47 , 1173–1181 (1992)

Button, K. S . et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14 , 365–376 (2013); erratum 14 , 451 (2013)

Parker, T. H . et al. Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31 , 711–719 (2016)

Stewart, G. Meta-analysis in applied ecology. Biol. Lett. 6 , 78–81 (2010)

Sutherland, W. J ., Pullin, A. S ., Dolman, P. M . & Knight, T. M. The need for evidence-based conservation. Trends Ecol. Evol. 19 , 305–308 (2004)

Lowry, E . et al. Biological invasions: a field synopsis, systematic review, and database of the literature. Ecol. Evol. 3 , 182–196 (2013)

Article   PubMed Central   Google Scholar  

Parmesan, C . & Yohe, G. A globally coherent fingerprint of climate change impacts across natural systems. Nature 421 , 37–42 (2003)

Jennions, M. D ., Lortie, C. J . & Koricheva, J. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 24 , 381–403 (Princeton Univ. Press, 2013)

Balvanera, P . et al. Quantifying the evidence for biodiversity effects on ecosystem functioning and services. Ecol. Lett. 9 , 1146–1156 (2006)

Cardinale, B. J . et al. Effects of biodiversity on the functioning of trophic groups and ecosystems. Nature 443 , 989–992 (2006)

Rey Benayas, J. M ., Newton, A. C ., Diaz, A. & Bullock, J. M. Enhancement of biodiversity and ecosystem services by ecological restoration: a meta-analysis. Science 325 , 1121–1124 (2009)

Article   ADS   PubMed   CAS   Google Scholar  

Leimu, R ., Mutikainen, P. I. A ., Koricheva, J. & Fischer, M. How general are positive relationships between plant population size, fitness and genetic variation? J. Ecol. 94 , 942–952 (2006)

Hillebrand, H. On the generality of the latitudinal diversity gradient. Am. Nat. 163 , 192–211 (2004)

Gurevitch, J. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 19 , 313–320 (Princeton Univ. Press, 2013)

Rustad, L . et al. A meta-analysis of the response of soil respiration, net nitrogen mineralization, and aboveground plant growth to experimental ecosystem warming. Oecologia 126 , 543–562 (2001)

Adams, D. C. Phylogenetic meta-analysis. Evolution 62 , 567–572 (2008)

Hadfield, J. D . & Nakagawa, S. General quantitative genetic methods for comparative biology: phylogenies, taxonomies and multi-trait models for continuous and categorical characters. J. Evol. Biol. 23 , 494–508 (2010)

Lajeunesse, M. J. Meta-analysis and the comparative phylogenetic method. Am. Nat. 174 , 369–381 (2009)

Rosenberg, M. S ., Adams, D. C . & Gurevitch, J. MetaWin: Statistical Software for Meta-Analysis with Resampling Tests Version 1 (Sinauer Associates, 1997)

Wallace, B. C . et al. OpenMEE: intuitive, open-source software for meta-analysis in ecology and evolutionary biology. Methods Ecol. Evol. 8 , 941–947 (2016)

Gurevitch, J ., Morrison, J. A . & Hedges, L. V. The interaction between competition and predation: a meta-analysis of field experiments. Am. Nat. 155 , 435–453 (2000)

Adams, D. C ., Gurevitch, J . & Rosenberg, M. S. Resampling tests for meta-analysis of ecological data. Ecology 78 , 1277–1283 (1997)

Gurevitch, J . & Hedges, L. V. Statistical issues in ecological meta-analyses. Ecology 80 , 1142–1149 (1999)

Schmid, C. H . & Mengersen, K. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 11 , 145–173 (Princeton Univ. Press, 2013)

Eysenck, H. J. Exercise in mega-silliness. Am. Psychol. 33 , 517 (1978)

Simberloff, D. Rejoinder to: Don’t calculate effect sizes; study ecological effects. Ecol. Lett. 9 , 921–922 (2006)

Cadotte, M. W ., Mehrkens, L. R . & Menge, D. N. L. Gauging the impact of meta-analysis on ecology. Evol. Ecol. 26 , 1153–1167 (2012)

Koricheva, J ., Jennions, M. D. & Lau, J. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J . et al.) Ch. 15 , 237–254 (Princeton Univ. Press, 2013)

Lau, J ., Ioannidis, J. P. A ., Terrin, N ., Schmid, C. H . & Olkin, I. The case of the misleading funnel plot. Br. Med. J. 333 , 597–600 (2006)

Vetter, D ., Rucker, G. & Storch, I. Meta-analysis: a need for well-defined usage in ecology and conservation biology. Ecosphere 4 , 1–24 (2013)

Mengersen, K ., Jennions, M. D. & Schmid, C. H. in The Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, J. et al.) Ch. 16 , 255–283 (Princeton Univ. Press, 2013)

Patsopoulos, N. A ., Analatos, A. A. & Ioannidis, J. P. A. Relative citation impact of various study designs in the health sciences. J. Am. Med. Assoc. 293 , 2362–2366 (2005)

Kueffer, C . et al. Fame, glory and neglect in meta-analyses. Trends Ecol. Evol. 26 , 493–494 (2011)

Cohnstaedt, L. W. & Poland, J. Review Articles: The black-market of scientific currency. Ann. Entomol. Soc. Am. 110 , 90 (2017)

Longo, D. L. & Drazen, J. M. Data sharing. N. Engl. J. Med. 374 , 276–277 (2016)

Gauch, H. G. Scientific Method in Practice (Cambridge Univ. Press, 2003)

Science Staff. Dealing with data: introduction. Challenges and opportunities. Science 331 , 692–693 (2011)

Nosek, B. A . et al. Promoting an open research culture. Science 348 , 1422–1425 (2015)

Article   CAS   ADS   PubMed   PubMed Central   Google Scholar  

Stewart, L. A . et al. Preferred reporting items for a systematic review and meta-analysis of individual participant data: the PRISMA-IPD statement. J. Am. Med. Assoc. 313 , 1657–1665 (2015)

Saldanha, I. J . et al. Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial. Syst. Rev. 5 , 196 (2016)

Tipton, E. & Pustejovsky, J. E. Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression. J. Educ. Behav. Stat. 40 , 604–634 (2015)

Mengersen, K ., MacNeil, M. A . & Caley, M. J. The potential for meta-analysis to support decision analysis in ecology. Res. Synth. Methods 6 , 111–121 (2015)

Ashby, D. Bayesian statistics in medicine: a 25 year review. Stat. Med. 25 , 3589–3631 (2006)

Article   MathSciNet   PubMed   Google Scholar  

Senior, A. M . et al. Heterogeneity in ecological and evolutionary meta-analyses: its magnitude and implications. Ecology 97 , 3293–3299 (2016)

McAuley, L ., Pham, B ., Tugwell, P . & Moher, D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet 356 , 1228–1231 (2000)

Koricheva, J ., Gurevitch, J . & Mengersen, K. (eds) The Handbook of Meta-Analysis in Ecology and Evolution (Princeton Univ. Press, 2013) This book provides the first comprehensive guide to undertaking meta-analyses in ecology and evolution and is also relevant to other fields where heterogeneity is expected, incorporating explicit consideration of the different approaches used in different domains.

Lumley, T. Network meta-analysis for indirect treatment comparisons. Stat. Med. 21 , 2313–2324 (2002)

Zarin, W . et al. Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review. BMC Med. 15 , 3 (2017)

Elliott, J. H . et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med. 11 , e1001603 (2014)

Vandvik, P. O ., Brignardello-Petersen, R . & Guyatt, G. H. Living cumulative network meta-analysis to reduce waste in research: a paradigmatic shift for systematic reviews? BMC Med. 14 , 59 (2016)

Jarvinen, A. A meta-analytic study of the effects of female age on laying date and clutch size in the Great Tit Parus major and the Pied Flycatcher Ficedula hypoleuca . Ibis 133 , 62–67 (1991)

Arnqvist, G. & Wooster, D. Meta-analysis: synthesizing research findings in ecology and evolution. Trends Ecol. Evol. 10 , 236–240 (1995)

Hedges, L. V ., Gurevitch, J . & Curtis, P. S. The meta-analysis of response ratios in experimental ecology. Ecology 80 , 1150–1156 (1999)

Gurevitch, J ., Curtis, P. S. & Jones, M. H. Meta-analysis in ecology. Adv. Ecol. Res 32 , 199–247 (2001)

Lajeunesse, M. J. phyloMeta: a program for phylogenetic comparative analyses with meta-analysis. Bioinformatics 27 , 2603–2604 (2011)

CAS   PubMed   Google Scholar  

Pearson, K. Report on certain enteric fever inoculation statistics. Br. Med. J. 2 , 1243–1246 (1904)

Fisher, R. A. Statistical Methods for Research Workers (Oliver and Boyd, 1925)

Yates, F. & Cochran, W. G. The analysis of groups of experiments. J. Agric. Sci. 28 , 556–580 (1938)

Cochran, W. G. The combination of estimates from different experiments. Biometrics 10 , 101–129 (1954)

Smith, M. L . & Glass, G. V. Meta-analysis of psychotherapy outcome studies. Am. Psychol. 32 , 752–760 (1977)

Glass, G. V. Meta-analysis at middle age: a personal history. Res. Synth. Methods 6 , 221–231 (2015)

Cooper, H. M ., Hedges, L. V . & Valentine, J. C. (eds) The Handbook of Research Synthesis and Meta-analysis 2nd edn (Russell Sage Foundation, 2009). This book is an important compilation that builds on the ground-breaking first edition to set the standard for best practice in meta-analysis, primarily in the social sciences but with applications to medicine and other fields.

Rosenthal, R. Meta-analytic Procedures for Social Research (Sage, 1991)

Hunter, J. E ., Schmidt, F. L. & Jackson, G. B. Meta-analysis: Cumulating Research Findings Across Studies (Sage, 1982)

Gurevitch, J ., Morrow, L. L ., Wallace, A . & Walsh, J. S. A meta-analysis of competition in field experiments. Am. Nat. 140 , 539–572 (1992). This influential early ecological meta-analysis reports multiple experimental outcomes on a longstanding and controversial topic that introduced a wide range of ecologists to research synthesis methods.

O’Rourke, K. An historical perspective on meta-analysis: dealing quantitatively with varying study results. J. R. Soc. Med. 100 , 579–582 (2007)

Shadish, W. R . & Lecy, J. D. The meta-analytic big bang. Res. Synth. Methods 6 , 246–264 (2015)

Glass, G. V. Primary, secondary, and meta-analysis of research. Educ. Res. 5 , 3–8 (1976)

DerSimonian, R . & Laird, N. Meta-analysis in clinical trials. Control. Clin. Trials 7 , 177–188 (1986)

Lipsey, M. W . & Wilson, D. B. The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis. Am. Psychol. 48 , 1181–1209 (1993)

Chalmers, I. & Altman, D. G. Systematic Reviews (BMJ Publishing Group, 1995)

Moher, D . et al. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses. Lancet 354 , 1896–1900 (1999)

Higgins, J. P. & Thompson, S. G. Quantifying heterogeneity in a meta-analysis. Stat. Med. 21 , 1539–1558 (2002)

Download references

Acknowledgements

We dedicate this Review to the memory of Ingram Olkin and William Shadish, founding members of the Society for Research Synthesis Methodology who made tremendous contributions to the development of meta-analysis and research synthesis and to the supervision of generations of students. We thank L. Lagisz for help in preparing the figures. We are grateful to the Center for Open Science and the Laura and John Arnold Foundation for hosting and funding a workshop, which was the origination of this article. S.N. is supported by Australian Research Council Future Fellowship (FT130100268). J.G. acknowledges funding from the US National Science Foundation (ABI 1262402).

Author information

Authors and affiliations.

Department of Ecology and Evolution, Stony Brook University, Stony Brook, 11794-5245, New York, USA

Jessica Gurevitch

School of Biological Sciences, Royal Holloway University of London, Egham, TW20 0EX, Surrey, UK

Julia Koricheva

Evolution and Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, 2052, New South Wales, Australia

Shinichi Nakagawa

Diabetes and Metabolism Division, Garvan Institute of Medical Research, 384 Victoria Street, Darlinghurst, Sydney, 2010, New South Wales, Australia

School of Natural and Environmental Sciences, Newcastle University, Newcastle upon Tyne, NE1 7RU, UK

Gavin Stewart

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed equally in designing the study and writing the manuscript, and so are listed alphabetically.

Corresponding authors

Correspondence to Jessica Gurevitch , Julia Koricheva , Shinichi Nakagawa or Gavin Stewart .

Ethics declarations

Competing interests.

The authors declare no competing financial interests.

Additional information

Reviewer Information Nature thanks D. Altman, M. Lajeunesse, D. Moher and G. Romero for their contribution to the peer review of this work.

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

PowerPoint slides

Powerpoint slide for fig. 1, rights and permissions.

Reprints and permissions

About this article

Cite this article.

Gurevitch, J., Koricheva, J., Nakagawa, S. et al. Meta-analysis and the science of research synthesis. Nature 555 , 175–182 (2018). https://doi.org/10.1038/nature25753

Download citation

Received : 04 March 2017

Accepted : 12 January 2018

Published : 08 March 2018

Issue Date : 08 March 2018

DOI : https://doi.org/10.1038/nature25753

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Accelerating evidence synthesis for safety assessment through clinicaltrials.gov platform: a feasibility study.

BMC Medical Research Methodology (2024)

Investigate the relationship between the retraction reasons and the quality of methodology in non-Cochrane retracted systematic reviews: a systematic review

• Azita Shahraki-Mohammadi
• Leila Keikha
• Razieh Zahedi

Systematic Reviews (2024)

A meta-analysis on global change drivers and the risk of infectious disease

• Michael B. Mahon
• Alexandra Sack
• Jason R. Rohr

Nature (2024)

Systematic review of the uncertainty of coral reef futures under climate change

• Shannon G. Klein
• Cassandra Roch
• Carlos M. Duarte

Nature Communications (2024)

Evidence library of meta-analytical literature assessing the sustainability of agriculture – a dataset

• Andrea Schievano
• Marta Pérez-Soba
• David Makowski

Scientific Data (2024)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

A Guide to Evidence Synthesis: What is Evidence Synthesis?

• Meet Our Team
• Our Published Reviews and Protocols
• What is Evidence Synthesis?
• Types of Evidence Synthesis
• Evidence Synthesis Across Disciplines
• Finding and Appraising Existing Systematic Reviews
• 0. Develop a Protocol
• 1. Draft your Research Question
• 2. Select Databases
• 3. Select Grey Literature Sources
• 4. Write a Search Strategy
• 5. Register a Protocol
• 6. Translate Search Strategies
• 7. Citation Management
• 8. Article Screening
• 9. Risk of Bias Assessment
• 10. Data Extraction
• 11. Synthesize, Map, or Describe the Results
• Evidence Synthesis Institute for Librarians
• Open Access Evidence Synthesis Resources

What are Evidence Syntheses?

What are evidence syntheses.

According to the Royal Society, 'evidence synthesis' refers to the process of bringing together information from a range of sources and disciplines to inform debates and decisions on specific issues. They generally include a methodical and comprehensive literature synthesis focused on a well-formulated research question.  Their aim is to identify and synthesize all  of the scholarly research on a particular topic, including both published and unpublished studies. Evidence syntheses are conducted in an unbiased, reproducible way to provide evidence for practice and policy-making, as well as to identify gaps in the research. Evidence syntheses may also include a meta-analysis, a more quantitative process of synthesizing and visualizing data retrieved from various studies. 

Evidence syntheses are much more time-intensive than traditional literature reviews and require a multi-person research team. See this PredicTER tool to get a sense of a systematic review timeline (one type of evidence synthesis). Before embarking on an evidence synthesis, it's important to clearly identify your reasons for conducting one. For a list of types of evidence synthesis projects, see the next tab.

How Does a Traditional Literature Review Differ From an Evidence Synthesis?

How does a systematic review differ from a traditional literature review.

One commonly used form of evidence synthesis is a systematic review.  This table compares a traditional literature review with a systematic review.

Video: Reproducibility and transparent methods (Video 3:25)

Reporting Standards

There are some reporting standards for evidence syntheses. These can serve as guidelines for protocol and manuscript preparation and journals may require that these standards are followed for the review type that is being employed (e.g. systematic review, scoping review, etc). ​

• PRISMA checklist Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.
• PRISMA-P Standards An updated version of the original PRISMA standards for protocol development.
• PRISMA - ScR Reporting guidelines for scoping reviews and evidence maps
• PRISMA-IPD Standards Extension of the original PRISMA standards for systematic reviews and meta-analyses of individual participant data.
• EQUATOR Network The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network is an international initiative that seeks to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines. They provide a list of various standards for reporting in systematic reviews.

Video: Guidelines and reporting standards

PRISMA Flow Diagram

The  PRISMA  flow diagram depicts the flow of information through the different phases of an evidence synthesis. It maps the search (number of records identified), screening (number of records included and excluded), and selection (reasons for exclusion).  Many evidence syntheses include a PRISMA flow diagram in the published manuscript.

See below for resources to help you generate your own PRISMA flow diagram.

• PRISMA Flow Diagram Tool
• PRISMA Flow Diagram Word Template
• << Previous: Our Published Reviews and Protocols
• Next: Types of Evidence Synthesis >>
• Last Updated: Sep 25, 2024 2:24 PM
• URL: https://guides.library.cornell.edu/evidence-synthesis

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008-.

Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet].

Quantitative synthesis—an update.

Investigators: Sally C. Morton , Ph.D., M.Sc., M. Hassan Murad , M.D., M.P.H., Elizabeth O’Connor , Ph.D., Christopher S. Lee , Ph.D., R.N., Marika Booth , M.S., Benjamin W. Vandermeer , M.Sc., Jonathan M. Snowden , Ph.D., Kristen E. D’Anci , Ph.D., Rongwei Fu , Ph.D., Gerald Gartlehner , M.D., M.P.H., Zhen Wang , Ph.D., and Dale W. Steele , M.D., M.S.

Affiliations

Published: February 23, 2018 .

Quantitative synthesis, or meta-analysis, is often essential for Comparative Effective Reviews (CERs) to provide scientifically rigorous summary information. Quantitative synthesis should be conducted in a transparent and consistent way with methodologies reported explicitly. This guide provides practical recommendations on conducting synthesis. The guide is not meant to be a textbook on meta-analysis nor is it a comprehensive review of methods, but rather it is intended to provide a consistent approach for situations and decisions that are commonly faced by AHRQ Evidence-based Practice Centers (EPCs). The goal is to describe choices as explicitly as possible, and in the context of EPC requirements, with an appropriate degree of confidence.

This guide addresses issues in the order that they are usually encountered in a synthesis, though we acknowledge that the process is not always linear. We first consider the decision of whether or not to combine studies quantitatively. The next chapter addresses how to extract and utilize data from individual studies to construct effect sizes, followed by a chapter on statistical model choice. The fourth chapter considers quantifying and exploring heterogeneity. The fifth describes an indirect evidence technique that has not been included in previous guidance – network meta-analysis, also known as mixed treatment comparisons. The final section in the report lays out future research suggestions.

The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-based Practice Centers (EPCs), sponsors the development of evidence reports and technology assessments to assist public- and private-sector organizations in their efforts to improve the quality of health care in the United States. The reports and assessments provide organizations with comprehensive, science-based information on common, costly medical conditions and new health care technologies and strategies. The EPCs systematically review the relevant scientific literature on topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to developing their reports and assessments.

Strong methodological approaches to systematic review improve the transparency, consistency, and scientific rigor of these reports. Through a collaborative effort of the Effective Health Care (EHC) Program, the Agency for Healthcare Research and Quality (AHRQ), the EHC Program Scientific Resource Center, and the AHRQ Evidence-based Practice Centers have developed a Methods Guide for Comparative Effectiveness Reviews. This Guide presents issues key to the development of Systematic Reviews and describes recommended approaches for addressing difficult, frequently encountered methodological issues.

The Methods Guide for Comparative Effectiveness Reviews is a living document, and will be updated as further empiric evidence develops and our understanding of better methods improves. We welcome comments on this Methods Guide paper. They may be sent by mail to the Task Order Officer named below at: Agency for Healthcare Research and Quality, 5600 Fishers Lane, Rockville, MD 20857, or by email to vog.shh.qrha@cpe .

• Gopal Khanna, M.B.A. Director Agency for Healthcare Research and Quality
• Arlene S. Bierman, M.D., M.S. Director Center for Evidence and Practice Improvement Agency for Healthcare Research and Quality
• Stephanie Chang, M.D., M.P.H. Director Evidence-based Practice Center Program Center for Evidence and Practice Improvement Agency for Healthcare Research and Quality
• Elisabeth Kato, M.D., M.R.P. Task Order Officer Evidence-based Practice Center Program Center for Evidence and Practice Improvement Agency for Healthcare Research and Quality
• Peer Reviewers

Prior to publication of the final evidence report, EPCs sought input from independent Peer Reviewers without financial conflicts of interest. However, the conclusions and synthesis of the scientific literature presented in this report does not necessarily represent the views of individual investigators.

Peer Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals with potential non-financial conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential non-financial conflicts of interest identified.

• Eric Bass, M.D., M.P.H Director, Johns Hopkins University Evidence-based Practice Center Professor of Medicine, and Health Policy and Management Johns Hopkins University Baltimore, MD
• Mary Butler, M.B.A., Ph.D. Co-Director, Minnesota Evidence-based Practice Center Assistant Professor, Health Policy & Management University of Minnesota Minneapolis, MN
• Roger Chou, M.D., FACP Director, Pacific Northwest Evidence-based Practice Center Portland, OR
• Lisa Hartling, M.S., Ph.D. Director, University of Alberta Evidence-Practice Center Edmonton, AB
• Susanne Hempel, Ph.D. Co-Director, Southern California Evidence-based Practice Center Professor, Pardee RAND Graduate School Senior Behavioral Scientist, RAND Corporation Santa Monica, CA
• Robert L. Kane, M.D. * Co-Director, Minnesota Evidence-based Practice Center School of Public Health University of Minnesota Minneapolis, MN
• Jennifer Lin, M.D., M.C.R. Director, Kaiser Permanente Research Affiliates Evidence-based Practice Center Investigator, The Center for Health Research, Kaiser Permanente Northwest Portland, OR
• Christopher Schmid, Ph.D. Co-Director, Center for Evidence Synthesis in Health Professor of Biostatistics School of Public Health Brown University Providence, RI
• Karen Schoelles, M.D., S.M., FACP Director, ECRI Evidence-based Practice Center Plymouth Meeting, PA
• Tibor Schuster, Ph.D. Assistant Professor Department of Family Medicine McGill University Montreal, QC
• Jonathan R. Treadwell, Ph.D. Associate Director, ECRI Institute Evidence-based Practice Center Plymouth Meeting, PA
• Tom Trikalinos, M.D. Director, Brown Evidence-based Practice Center Director, Center for Evidence-based Medicine Associate Professor, Health Services, Policy & Practice Brown University Providence, RI
• Meera Viswanathan, Ph.D. Director, RTI-UNC Evidence-based Practice Center Durham, NC RTI International Durham, NC
• C. Michael White, Pharm. D., FCP, FCCP Professor and Head, Pharmacy Practice School of Pharmacy University of Connecticut Storrs, CT
• Tim Wilt, M.D., M.P.H. Co-Director, Minnesota Evidence-based Practice Center Director, Minneapolis VA-Evidence Synthesis Program Professor of Medicine, University of Minnesota Staff Physician, Minneapolis VA Health Care System Minneapolis, MN

Deceased March 6, 2017

• Introduction

The purpose of this document is to consolidate and update quantitative synthesis guidance provided in three previous methods guides. 1 – 3 We focus primarily on comparative effectiveness reviews (CERs), which are systematic reviews that compare the effectiveness and harms of alternative clinical options, and aim to help clinicians, policy makers, and patients make informed treatment choices. We focus on interventional studies and do not address diagnostic studies, individual patient level analysis, or observational studies, which are addressed elsewhere. 4

Quantitative synthesis, or meta-analysis, is often essential for CERs to provide scientifically rigorous summary information. Quantitative synthesis should be conducted in a transparent and consistent way with methodologies reported explicitly. This guide provides practical recommendations on conducting synthesis. The guide is not meant to be a textbook on meta-analysis nor is it a comprehensive review of methods, but rather it is intended to provide a consistent approach for situations and decisions that are commonly faced by Evidence-based Practice Centers (EPCs). The goal is to describe choices as explicitly as possible and in the context of EPC requirements, with an appropriate degree of confidence.

EPC investigators are encouraged to follow these recommendations but may choose to use alternative methods if deemed necessary after discussion with their AHRQ project officer. If alternative methods are used, investigators are required to provide a rationale for their choices, and if appropriate, to state the strengths and limitations of the chosen methods in order to promote consistency, transparency, and learning. In addition, several steps in meta-analysis require subjective judgment, such as when combining studies or incorporating indirect evidence. For each subjective decision, investigators should fully explain how the decision was reached.

This guide was developed by a workgroup comprised of members from across the EPCs, as well as from the Scientific Resource Center (SRC) of the AHRQ Effective Healthcare Program. Through surveys and discussions among AHRQ, Directors of EPCs, the Scientific Resource Center, and the Methods Steering Committee, quantitative synthesis was identified as a high-priority methods topic and a need was identified to update the original guidance. 1 , 5 Once confirmed as a Methods Workgroup, the SRC solicited EPC workgroup volunteers, particularly those with quantitative methods expertise, including statisticians, librarians, thought leaders, and methodologists. Charged by AHRQ to update current guidance, the workgroup consisted of members from eight of 13 EPCs, the SRC, and AHRQ, and commenced in the fall of 2015. We conducted regular workgroup teleconference calls over the course of 14 months to discuss project direction and scope, assign and coordinate tasks, collect and analyze data, and discuss and edit draft documents. After constructing a draft table of contents, we surveyed all EPCs to ensure no topics of interest were missing.

The initial teleconference meeting was used to outline the draft, discuss the timeline, and agree upon a method for reaching consensus as described below. The larger workgroup then was split into subgroups each taking responsibility for a different chapter. The larger group participated in biweekly discussions via teleconference and email communication. Subgroups communicated separately (in addition to the larger meetings) to coordinate tasks, discuss the literature review results, and draft their respective chapters. Later, chapter drafts were combined into a larger document for workgroup review and discussion on the bi-weekly calls.

Literature Search and Review

A medical research librarian worked with each subgroup to identify a relevant search strategy for each chapter, and then combined these strategies into one overall search conducted for all chapters combined. The librarian conducted the search on the ARHQ SRC Methods Library, a bibliographic database curated by the SRC currently containing more than 16,000 citations of methodological works for systematic reviews and comparative effectiveness reviews, using descriptor and keyword strategies to identify quantitative synthesis methods research publications (descriptor search=all quantitative synthesis descriptors, and the keyword search=quantitative synthesis, meta-anal*, metaanal*, meta-regression in [anywhere field]). Search results were limited to English language and 2009 and later to capture citations published since AHRQ’s previous methods guidance on quantitative synthesis. Additional articles were identified from recent systematic reviews, reference lists of reviews and editorials, and through the expert review process.

The search yielded 1,358 titles and abstracts which were reviewed by all workgroup members using ABSTRACKR software (available at http://abstrackr.cebm.brown.edu ). Each subgroup separately identified articles relevant to their own chapter. Abstract review was done by single review, investigators included anything that could be potentially relevant. Each subgroup decided separately on final inclusion/exclusion based on full text articles.

Consensus and Recommendations

Reaching consensus if possible is of great importance for AHRQ methods guidance. The workgroup recognized this importance in its first meeting and agreed on a process for informal consensus and conflict resolution. Disagreements were thoroughly discussed and if possible, consensus was reached. If consensus was not reached, analytic options are discussed in the text. We did not employ a formal voting procedure to assess consensus.

A summary of the workgroup’s key conclusions and recommendations was circulated for comment by EPC Directors and AHRQ officers at a biannual EPC Director’s meeting in October 2016. In addition, a full draft was circulated to EPC Directors and AHRQ officers prior to peer review, and the manuscript was made available for public review. All comments have been considered by the team in the final preparation of this report.

Chapter 1. Decision to Combine Trials

1.1. goals of the meta-analysis.

Meta-analysis is a statistical method for synthesizing (also called combining or pooling) the benefits and/or harms of a treatment or intervention across multiple studies. The overarching goal of a meta-analysis is generally to provide the best estimate of the effect of an intervention. As part of that aspirational goal, results of a meta-analysis may inform a number of related questions, such as whether that best estimate represents something other than a null effect (is this intervention beneficial?), the range in which the true effect likely lies, whether it is appropriate to provide a single best estimate, and what study-level characteristics may influence the effect estimate. Before tackling these questions, it is necessary to answer a preliminary but fundamental question: Is it appropriate to pool the results of the identified studies? 6

Clinical, methodological, and statistical factors must all be considered when deciding whether to combine studies in a meta-analysis. Figure 1.1 depicts a decision tree to help investigators think through these important considerations, which are discussed below.

Pooling decision tree.

1.2. Clinical and Methodological Heterogeneity

Studies must be reasonably similar to be pooled in a meta-analysis. 1 Even when the review protocol identifies a coherent and fairly narrow body of literature, the actual included studies may represent a wide range of population, intervention, and study characteristics. Variations in these factors are referred to as clinical heterogeneity and methodological heterogeneity. 7 , 8 A third form of heterogeneity, statistical heterogeneity, will be discussed later.

The first step in the decision tree is to explore the clinical and methodological heterogeneity of the included studies (Step A, Figure 1.1 ). The goal is to identify groups of trials that are similar enough that an average effect would make a sensible summary. There is no objective measure or universally accepted standard for deciding whether studies are “similar enough” to pool; this decision is inherently a matter of judgment. 6 Verbeek and colleagues suggest working through key sources of variability in sequence, beginning with the clinical variables of intervention/exposure, control condition, and participants, before moving on to methodological areas such as study design, outcome, and follow-up time. When there is important variability in these areas, investigators should consider whether there are coherent subgroups of trials, rather than the full group, that can be pooled. 6

Clinical heterogeneity refers to characteristics related to the participants, interventions, types of outcomes, and study setting. Some have suggested that pooling may be acceptable when it is plausible that the underlying effects could be similar across subpopulations and variations in interventions and outcomes. 9 For example, in a review of a lipid-lowering medication, researchers might be comfortable combining studies that target younger and middle-aged adults, but expect different effects with older adults, who have high rates of comorbidities and other medication use. Others suggest that it may be acceptable to combine interventions with likely similar mechanisms of action. 6 For example, a researcher may combine studies of depression interventions that use a range of psychotherapeutic approaches, on the logic that they all aim to change a person’s thinking and behavior in order to improve mood, but not want to combine them with trials of antidepressants, whose mechanism of action is presumed to be biochemical.

Methodological heterogeneity refers to variations in study methods (e.g., study design, measures, and study conduct). A common question regarding study design, is whether it is acceptable to combine studies that randomize individual participants with those that randomize clusters (e.g., when clinics, clinicians, or classrooms are randomized and individuals are nested within these units). We believe this is generally acceptable, with appropriate adjustment for cluster randomization as needed. 10 However, closer examination may show that the cluster randomized trials also tend to systematically differ on population or intervention characteristics from the individually-randomized trials. If so, subgroup analyses may be considered.

Outcome measures are a common source of methodological heterogeneity. First, trials may have a wide array of specific instruments and cut-points for a common outcome. For example, a review considering pooling the binary outcome of depression prevalence may find measures that range from a depression diagnosis based on a clinical interview to scores above a cut-point on a screening instrument. One guiding principle is to consider pooling only when it is plausible that the underlying relative effects are consistent across specific definitions of an outcome. In addition, investigators should take steps to harmonize outcomes to the extent possible.

Second, there is also typically substantial variability in the statistics reported across studies (e.g., odds ratios, relative risks, hazard ratios, baseline and mean followup scores, change scores for each condition, between-group differences at followup, etc.). Methods to calculate or estimate missing statistics are available, 5 however the investigators must ultimately weigh the tradeoff of potentially less accurate results (due to assumptions required to estimate missing data) with the potential advantage of pooling a more complete set of studies. If a substantial proportion of the studies require calculations that involve assumptions or estimates (rather than straightforward calculations) in order to combine them, then it may be preferable to show results in a table or forest plot without a pooled estimate

1.3. Best Evidence Versus All Evidence

Sometimes the body of evidence comprises a single trial or small number of trials that clearly represent the best evidence, along with a number of additional trials that are much smaller or with other important limitations (Step B, Figure 1.1 ). The “best evidence” trials are generally very large trials with low risk of bias and with good generalizability to the population of interest. In this case, it may be appropriate to focus on the one or few “best” trials rather than combining them with the rest of the evidence, particularly when addressing rare events that small studies are underpowered to examine. 11 , 12 For example, an evidence base of one large, multi-center trial of an intervention to prevent stroke in patients with heart disease could be preferable to a pooled analysis of 4-5 small trials reporting few events, and combining the small trials with the large trial may introduce unnecessary uncertainty to the pooled estimate.

1.4. Assessing the Risk of Misleading Meta-analysis Results

Next, reviews should explore the risk that the meta-analysis will show results that do not accurately capture the true underlying effect (Step C, Figure 1.1 ). Tables, forest plots (without pooling), and some other preliminary statistical tests are useful tools for this stage. Several patterns can arise that should lead investigators to be cautious about combining studies.

Wide-Ranging Effect Sizes

Sometimes one study may show a large benefit and another study of the same intervention may show a small benefit. This may be due to random error, especially when the studies are small. However, this situation also raises the possibility that observed effects truly are widely variable in different subpopulations or situations. Another look at the population characteristics is warranted in this situation to see if the investigators can identify characteristics that are correlated with effect size and direction, potentially explaining clinical heterogeneity.

Even if no characteristic can be identified that explains why the intervention had such widely disparate effects, there could be unmeasured features that explain the difference. If the intervention really does have widely variable impact in different subpopulations, particularly if it is benefiting some patients and harming others, it would be misleading to report a single average effect.

Suspicion of Publication or Reporting Bias

Sometimes, due to lack of effect, trial results are never published (risking publication bias), or are only published in part (risking reporting bias). These missing results can introduce bias and reduce the precision of meta-analysis. 13 Investigators can explore the risk of reporting bias by comparing trials that do and do not report important outcomes to assess whether outcomes appear to be missing at random. 13 For example, investigators may have 30 trials of weight loss interventions with only 10 reporting blood pressure, which is considered an important outcome for the review. This pattern of results may indicate reporting bias as trials finding group differences in blood pressure were more likely to report blood pressure findings. On the other hand, perhaps most of the studies limited to patients with elevated cardiovascular disease (CVD) risk factors did report blood pressure. In this case, the investigators may decide to combine the studies reporting blood pressure that were conducted in high CVD risk populations. However, investigators should be clear about the applicable subpopulation. An examination of the clinical and methodological features of the subset of trials where blood pressure was reported is necessary to make an informed judgement about whether to conduct a meta-analysis.

Small Studies Effect

If small studies show larger effects than large studies, the pooled results may overestimate the true effect size, possibly due to publication or reporting bias. 14 When investigators have at least 10 trials to combine they should examine small studies effects using standard statistical tests such as the Egger test. 15 If there appears to be a small studies effect, the investigators may decide not to report pooled results since they could be misleading. On the other hand, small studies effects could be happening for other reasons, such as differences in sample characteristics, attrition, or assessment methods. These factors do not suggest bias, but should be explored to the degree possible. See Chapter 4 for more information about exploring heterogeneity.

1.5. Special Considerations When Pooling a Small Number of Studies

When pooling a small number of studies (e.g., <10 studies), a number of considerations arise (Step E, Figure 1.1 ):

Rare Outcomes

Meta-analyses of rare binary outcomes are frequently underpowered, and tend to overestimate the true effect size, so pooling should be undertaken with caution. 11 A small difference in absolute numbers of events can result in large relative differences, usually with low precision (i.e., wide confidence intervals). This could result in misleading effect estimates if the analysis is limited to trials that are underpowered for the rare outcomes. 12 One example is all-cause mortality, which is frequently provided as part of the participant flow results, but may not be a primary outcome, may not have adjudication methods described, and typically occurs very rarely. Studies are often underpowered to detect differences in mortality if it is not a primary outcome. Investigators should consider calculating an optimal information size (OIS) when events are rare to see if the combined group of studies has sufficient power to detect group differences. This could be a concern even for a relatively large number of studies, if the total sample size is not very large. 16 See Chapter 3 for more detail on handling rare binary outcomes.

Small Sample Sizes

When pooling a relatively small number of studies, pooling should be undertaken with caution if the body of evidence is limited only to small studies. Results from small trials are less likely to be reliable than results of large trials, even when the risk of bias is low. 17 First, in small trials it is difficult to balance the proportion of patients in potentially important subgroups across interventions, and a difference between interventions of just a few patients in a subgroup can result in a large proportional difference between interventions. Characteristics that are rare are particularly at risk of being unbalanced in trials with small samples. In such situations there is no way to know if trial effects are due to the intervention or to differences in the intervention groups. In addition, patients are generally drawn from a narrower geographic range in small trials, making replication in other trials more uncertain. Finally, although it is not always the case, large trials are more likely to involve a level of scrutiny and standardization to ensure lower risk of bias than are small trials. Therefore, when the trials have small sample sizes, pooled effects are less likely to reflect the true effects of the intervention. In this case, the required or optimal information size can help the investigators determine whether the sample size is sufficient to conclude that results are likely to be stable and not due to random heterogeneity (i.e., truly significant or truly null results; not a type I or type II error). 16 , 18 An option in this case would be to pool the studies and acknowledge imprecision or other limitations when rating the strength of evidence.

What would be considered a “small” trial varies for different fields and outcomes. For addressing an outcome that only happens in 10% of the population, a small trial might be 100 to 200 per intervention arm, whereas a trial addressing a continuous quality of life measure may be small with 20 to 30 per intervention. Looking carefully at what the studies were powered to detect and the credibility of the power calculations may help determine what constitutes a “small” trial. Investigators should also consider how variable the impact of an intervention may be over different settings and subpopulations when determining how to weigh the importance of small studies. For example, the effects of a counseling intervention that relies on patients to change their behavior in order to reap health benefits may be more strongly influenced by characteristics of the patients and setting than a mechanical or chemical agent.

When the number of trials to be pooled is small, there is a heightened risk that statistical heterogeneity will be substantially underestimated, resulting in 95% confidence intervals that are inappropriately narrow and do not have 95% coverage. This is especially concerning when the number of studies being pooled is fewer than five to seven. 19 – 21

Accounting for these factors should guide an evaluation of whether it is advisable to pool the relatively small group of studies. As with many steps in the multi-stage decision to pool, the conclusion that a given investigator arrives at is subjective, although such evaluations should be guided by the criteria above. If consideration of these factors reassures investigators that the risk of bias associated with pooling is sufficiently low, then pooling can proceed. The next step of pooling, whether for a small, moderate, or large body of studies, is to consider statistical heterogeneity.

1.6. Statistical Heterogeneity

Once clinical and methodological heterogeneity and other factors described above have been deemed acceptable for pooling, investigators should next consider statistical heterogeneity (Step F, Figure 1.1 ). We discuss statistical heterogeneity in general in this chapter, and provide a deeper methodological discussion in Chapter 4 . This initial consideration of statistical heterogeneity is accomplished by conducting a preliminary meta-analysis. Next the investigator must decide if the results of the meta-analysis are valid and should be presented, rather than simply showing tables or forest plots without pooled results. If statistical heterogeneity is very high, the investigators may question whether an “average” effect is really meaningful or useful. If there is a reasonably large number of trials, the investigators may shift to exploring effect modification with high heterogeneity, however this may not be possible if few trials are available. While many would likely agree that pooling (or reporting pooled results) should be avoided when there are few studies and statistical heterogeneity is high, what constitutes “few” studies and “high” heterogeneity is a matter of judgment.

While there are a variety of methods for characterizing statistical heterogeneity, one common method is the I 2 statistic, the proportion of total variance in the pooled trials that is due to inter-study variance, as opposed to random variation. 22 The Cochrane manual proposes ranges for interpreting I 2 : 10 statistical heterogeneity associated with I 2 values of 0-40% might not be important, 30-60% may represent moderate heterogeneity, 50-90% may represent substantial heterogeneity, and 75-100% is considerable heterogeneity. Ranges overlap to reflect that other factors—such as the number and size of the trials and the magnitude and direction of the effect—must be taken into consideration. Other measures of statistical heterogeneity include Cochrane’s Q and τ 2 , but these heterogeneity statistics do not have intrinsic standardized scales that allow specific values to be characterized as “small,” “medium,” or “large” in any meaningful way. 23 However, τ 2 can be interpreted on the scale of the pooled effect, as the variance of the true effect. All these measures are discussed in more detail in Chapter 4 .

Although widely used in quantitative synthesis, the I 2 statistic has come under criticism in recent years. One important issue with I 2 is that it can be an inaccurate reflection of statistical heterogeneity when there are few studies to pool and high statistical heterogeneity. 24 , 25 For example, in random effects models (but not fixed effects models), calculations demonstrate that I 2 tends to underestimate true statistical heterogeneity when there are fewer than about 10 studies and the I 2 is 50% or more. 26 In addition, I 2 is correlated with the sample size of the included studies, generally increasing with larger samples. 27 Complicating this, meta-analyses of continuous measures tend to have higher heterogeneity than those of binary outcomes, and I 2 tends to increase as the number of studies increases when analyzing continuous outcomes, but not binary outcomes. 28 , 29 This has prompted some authors to suggest that different standards may be considered for interpreting I 2 for meta-analyses of continuous and binary outcomes, but I 2 should only be considered reliable when there are a sufficient number of studies. 29 Unfortunately there is not clear consensus regarding what constitutes a sufficient number of studies for a given amount of statistical heterogeneity, nor is it possible to be entirely prescriptive, given the limits of I 2 as a measure of heterogeneity. Thus, I 2 is one piece of information that should be considered, but generally should not be the primary deciding factor for whether to pool.

1.7. Conclusion

In the end, the decision to pool boils down to the question: will the results of a meta-analysis help you find a scientifically valid answer to a meaningful question? That is, will the meta-analysis provide something in addition to what can be understood from looking at the studies individually? Further, do the clinical, methodological, and statistical features of the body of studies permit them to be quantitatively combined and summarized in a valid fashion? Each of these decisions can be broken down into specific considerations (outlined in Figure 1.1 ) There is broad guidance to inform investigators in making each of these decisions, but generally the choices involved are subjective. The investigators’ scientific goal might factor into the evaluation of these considerations: for example, if investigators seek a general summary of the combined effect (e.g., direction only) versus an estimated effect size, the consideration of whether to pool may be weighed differently. In the end, to provide a meaningful result, the trials must be similar enough in content, procedures, and implementation to represent a cohesive group that is relevant to real practice/decision-making.

Recommendations

• Use Figure 1.1 when deciding whether to pool studies

Chapter 2. Optimizing Use of Effect Size Data

2.1. introduction.

The employed methods for meta-analysis will depend upon the nature of the outcome data. The two most common data types encountered in trials are binary/dichotomous (e.g., dead or alive, patient admitted to hospital or not, treatment failure or success, etc.) and continuous (e.g., weight, systolic blood pressure, etc.). Some outcomes (e.g., heart rate, counts of common events) that are not strictly continuous, are often treated as continuous for the purposes of meta-analysis based on assumptions of normality and the belief that statistical methods that are applied to normal distributions can be applicable to other distributions (central limit theory). Continuous outcomes are also frequently analyzed as binary outcomes when there are clinically meaningful cut-points or thresholds (e.g., a patient’s systolic blood pressure may be classified as low or high based on whether it is under or over 130mmHG). While this type of dichotomization may be more clinically meaningful it reduces statistical information, so investigators should provide their rationale for taking this approach.

Other less common data types that do not fit into either the binary or continuous categories include ordinal, categorical, rate, and time to event to data. Meta-analyzing these types of data will usually require reporting of the relevant statistics (e.g., hazard ratio, proportional odds ratio, incident rate ratio) by the study authors.

2.2. Nuances of Binary Effect Sizes

Data needed for binary effect size computation.

Under ideal circumstances, the minimal data necessary for the computation of effect sizes of binary data would be available in published trial documents or from original sources. Specifically, risk difference (RD), relative risk (RR), and odds ratios (OR) can be computed when the number of events (technically the number of cases in whom there was an event) and sample sizes are known for treatment and control groups. A schematic of one common approach to assembling binary data from trials for effect size computation is presented in Table 2.1 . This approach will facilitate conversion to analysis using commercially-available software such as Stata (College Station, TX) or Comprehensive Meta-Analysis (Englewood, NJ).

Assembling binary data for effect size computation.

In many instances, a single study (or subset of studies) to be included in the meta-analysis provides only one measure of association (an odds ratio, for example), and the sample size and event counts are not available. In that case, the meta-analytic effect size will be dictated by the available data. However, choosing the appropriate effect size is important for integrity and transparency, and every effort should be made to obtain all the data presented in Table 2.1 . Note that CONSORT guidance requires that published trial data should include the number of events and sample sizes for both treatment and control groups. 30 And, PRISMA guidance supports describing any processes for obtaining and confirming data from investigators 31 – a frequently required step.

In the event that data are only available in an effect size from the original reports, it is important to extract both the mean effect sizes and the associated 95% confidence intervals. Having raw event data available as in Table 2.1 not only facilitates the computation of various effect sizes, but also allows for the application of either binomial (preferred) or normal likelihood approaches; 32 only normal likelihood can be applied to summary statistics (e.g., an odds ratio and confidence interval in the primary study report).

Choosing Among Effect Size Options

One absolute measure and two relative measures are commonly used in meta-analyses involving binary data. The RD (an absolute measure) is a simple metric that is easily understood by clinicians, patients, and other stakeholders. The relative measures, RR or OR, are also used frequently. All three metrics should be considered additive, just on different scales. That is, RD is additive on a raw scale, RR on a log scale, and OR on a logit scale.

Risk Difference

The RD is easily understood by clinicians and patients alike, and therefore most useful to aid decision making. However, the RD tends to be less consistent across studies compared with relative measures of effect size (RR and OR). Hence, the RD may be a preferred measure in meta-analyses when the proportions of events among control groups are relatively common and similar across studies. When events are rare and/or when event rates differ across studies, however, the RD is not the preferred effect size to be used in meta-analysis because combined estimates based on RD in such instances have more conservative confidence intervals and lower statistical power. The calculation of RD and other effect size metrics using binary data from clinical trials can be performed considering the following labeling ( Table 2.2 ).

Organizing binary data for effect size computation.

Equation Set 2.1. Risk Difference

• RD = risk difference
• V RD = variance of the risk difference
• SE RD = standard error of the risk difference
• LL RD = lower limit of the 95% confidence interval of the risk difference
• UL RD = upper limit of the 95% confidence interval of the risk difference

Number Needed To Treat Related to Risk Difference

• NNT = number needed to treat

In case of a negative RD, the number needed to harm (NNH) or number needed to treat for one patient to be harmed is = − 1/RD.

The Wald method 34 is commonly used to calculate confidence intervals for NNT. It is reasonably adequate for large samples and probabilities not close to either 0 or 1, however it can be less reliable for small samples, probabilities close to either 0 or 1, or unbalanced trial designs. 35 An adjustment to the Wald method (i.e., adding pseudo-observations) helps mitigate concern about its application in small samples, 36 but it doesn’t account for other sources of limitations to this method. The Wilson method of calculating confidence intervals for NNT, as described in detail by Newcome, 37 has better coverage properties irrespective of sample size, is free of implausible results, and is argued to be easier to calculate compared with Wald confidence intervals. 35 Therefore, the Wilson method is preferable to the Wald method for calculating confidence intervals for NNT. When considering using NNT as the effect size in meta-analysis, see commentary by Lesaffre and Pledger.38 When considering using NNT as the effect size in meta-analysis, see commentary on the superior performance of combined NNT on the RD scale as opposed to the NNT scale.

It is important to note that the RR and OR are effectively equivalent for event rates below about 10%. In such cases, the RR is chosen over the OR simply for interpretability (an important consideration) and not substantive differences. A potential drawback to the use of RR over OR (or RD) is that the RR of an event is not the reciprocal of the RR for the non-occurrence of that event (e.g., using survival as the outcome instead of death). In contrast, switching between events and non-occurrence of events is reciprocal in the metric of OR and only entails a change in the sign of OR. If switching between death and survival, for example, is central to the meta-analysis, then the RR is likely not the binary effect size metric of choice unless all raw data are available and re-computation is possible. Moreover, investigators should be particularly attentive to the definition of an outcome event when using a RR.

The calculation of RR using binary data can be performed considering the labeling listed in Table 2.2 . Of particular note, the metrics of dispersion related to the RR are first computed in a natural log metric and then converted to the metric of RR.

Equation Set 2.2. Risk Ratio

• RR = risk ratio
• ln RR = natural log of the risk ratio
• V lnRR = variance of the natural log of the risk ratio
• SE lnRR = standard error of the natural log of the risk ratio
• LLlnRR = lower limit of the 95% confidence interval of the natural log of the risk ratio
• UL lnRR = upper limit of the 95% confidence interval of the natural log of the risk ratio
• LL RR = lower limit of the 95% confidence interval of the risk ratio
• UL RR = upper limit of the 95% confidence interval of the risk ratio

Therefore, while the definition of the outcome event needs to be consistent among the included studies when using any measure, the investigators should be particularly attentive to the definition of an outcome event when using an RR.

Odds Ratios

An alternative relative metric for use with binary data is the OR. Given that ORs are frequently presented in models with covariates, it is important to note that the OR is ‘non-collapsible,’ meaning that effect modification varies depending on the covariates for which control has been made; this favors the reporting of RR over OR, particularly when outcomes are common and covariates are included. 39 The calculation of OR using binary data can be performed considering the labeling listed in Table 2.2 . Similar to the computation of RR, the metrics of dispersion related to the OR are first computed in a natural log metric and then converted to the metric of OR.

Equation Set 2.3. Odds ratios

• OR = odds ratio
• Ln OR = natural log of the odds ratio
• V lnOR = variance of the natural log of the odds ratio
• SE lnoR = standard error of the natural log of the odds ratio
• LLlnOR = lower limit of the 95% confidence interval of the natural log of the odds ratio
• UL lnOR = upper limit of the 95% confidence interval of the natural log of the odds ratio
• LL OR = lower limit of the 95% confidence interval of the odds ratio
• UL OR = upper limit of the 95% confidence interval of the odds ratio

A variation on the calculation of OR is the Peto OR that is commonly referred to as the assumption-free method of calculating OR. The two key differences between the standard OR and the Peto OR is that the latter takes into consideration the expected number of events in the treatment group and also incorporates a hypergeometric variance. Because of these difference, the Peto OR is preferred for binary studies with rare events, especially when event rates are less than 1%. But in contrast, the Peto OR is biased when treatment effects are large, due to centering around the null hypothesis, and in the instance of imbalanced treatment and control groups. 40

Equation Set 2.4. Peto odds ratios

ORpeto = exp [ { A − E ( A ) } / v ] where E(A) is the expected number of events in the treatment group calculated as: E ( A ) = n 1 ( A + E ) N and v is hypergeometric variance, calculated as: v = { n 1   n 2 ( A + C ) ( B + D ) } / { N 2 ( N − 1 ) }

There is no perfect effect size of binary data to choose because each has benefits and disadvantages. Criteria used to compare and contrast these measures include consistency over a set of studies, statistical properties, and interpretability. Key benefits and disadvantages of each are presented in Table 2.3 . In the table, the term “baseline risk” is the proportion of subjects in the control group who experienced the event. The term “control rate” is sometimes used for this measure as well.

Benefits and disadvantages of binary data effect sizes.

Time-to-Event and Count Outcomes

For time to event data, the effect size measure is a hazard ratio (HR), which is commonly estimated from the Cox proportional hazards model. In the best-case scenario, HR and associated 95% confidence intervals are available from all studies, the time horizon is similar across studies, and there is evidence that the proportional hazards assumption was met in each study to be included in a meta-analysis. When these conditions are not met, an HR and associated dispersion can still be extracted and meta-analyzed. However, this approach raises concerns about reproducibility due to observer variation. 44

Incident rate ratio (IRR) is used for count data and can be estimated from a Poisson or negative binomial regression model. The IRR is a relative metric based on counts of events (e.g., number of hospitalizations, or days of length of stay) over time (i.e., per person-year) compared between trial arms. It is important to consider how IRR estimates were derived in individual studies particularly with respect to adjustments for zero-inflation and/or over-dispersion as these modeling decisions can be sources of between-study heterogeneity. Moreover, studies that include count data may have zero counts in both groups, which may require less common and more nuanced approaches to meta-analysis like Poisson regression with random intervention effects. 45

2.3. Continuous Outcomes

Assembling data needed for effect size computation.

Meta-analysis of studies presenting continuous data requires both estimated differences between the two groups being compared and estimated standard errors of those differences. Estimating the between-group difference is easiest when the study provides the mean difference. While both a standardized mean difference and ratio of means could be given by the study authors, studies more often report means for each group. Thus, a mean difference or ratio of means often must be computed.

If estimates of the standard errors of the mean are not provided studies commonly provide confidence intervals, standard deviations, p-values, z-statistics, and/or t-statistics, which make it possible to compute the standard error of the mean difference. In the absence of any of these statistics, other methods are available to estimate standard error. 45

(Weighted) Mean Difference

The mean difference (formerly known as weighted mean difference) is the most common way of summarizing and pooling a continuous outcome in a meta-analysis. Pooled mean differences can be computed when every study in the analysis measures the outcome on the same scale or on scales that can be easily converted. For example, total weight can be pooled using mean difference even if different studies reported weights in kilograms and pounds; however it is not possible to pool quality of life measured in both Self Perceived Quality of Life scale (SPQL) and the 36-item Short Form Survey Instrument (SF-36), since these are not readily convertible to one format.

Computation of the mean difference is straightforward and explained elsewhere. 5 Most software programs will require the mean, standard deviation, and sample size from each intervention group and for each study in the meta-analysis, although as mentioned above, other pieces of data may also be used.

Some studies report values as change from baseline, or alternatively present both baseline and final values. In these cases, it is possible to pool differences in final values in some studies with differences in change from baseline values in other studies, since they will be estimating the same value in a randomized control trial. If baseline values are unbalanced it may be better to perform ANCOVA analysis (see below). 5

Standardized Mean Difference

Sometimes different studies will assess the same outcome using different scales or metrics that cannot be readily converted to a common measure. In such instances the most common response is to compute a standardized mean difference (SMD) for each study and then pool these across all studies in the meta-analysis. By dividing the mean difference by a pooled estimate of the standard deviation, we theoretically put all scales in the same unit (standard deviation), and are then able to statistically combine all the studies. While the standardized mean difference could be used even when studies use the same metric, it is generally preferred to use mean difference. Interpretation of results is easier when the final pooled estimate is given in the same units as the original studies.

Several methods can compute SMDs. The most frequently used are Cohen’s d and Hedges’ g .

Cohen’s d

Cohen’s d is the simplest S. computation; it is defined as the mean difference divided by the pooled standard deviation of the treatment and control groups. 5 For a given study, Cohen’s d can be computed as: d = m T − m C s p o o l e d

Where m T and m C are the treatment and control means and spooled is essentially the square root of the weighted average of the treatment and control variances.

It has been shown that this estimate is biased in estimating the true population SMD, and the bias decreases as the sample size increases (small sample bias). 46 For this reason, Hedges g is more often used.

Hedges’ g

Hedges’ g is a transformation of Cohen’s d that attempts to adjust for small sample bias. The transformation involves multiplying Cohen’s d by a function of the total sample size.5 This generally results in a slight decrease in value of Hedges’ g compared with Cohen’s d, but the reduction lessens as the total sample size increases. The formula is: d ( 1 − 3 4 N − 9 )

Where N is the total trial sample size.

For very large sample sizes the two estimates will be very similar.

Back Transformation of Pooled SMD

One disadvantages of reporting standardized mean difference is that units of standard deviation are difficult to interpret clinically. Guidelines do exist but are often thought to be arbitrary and not applicable to all situations.47 An alternative is to back transform the pooled SMD into a scale used in the one of the analyses. In theory, by multiplying the SMD (and its upper and lower confidence bounds) by the standard deviation of the original scale, one can obtain a pooled estimate in that original scale. The difficulty is that the true standard deviation is unknown and must be estimated from available data. Alternatives for estimation include using the standard deviation from the largest study or using a pooled estimate of the standard deviations across studies.5 One should include a sensitivity analysis and be transparent about the approach used.

Ratio of Means

Ratio of Means (RoM), also known as response ratio, has been presented as an alternative to the SMD when outcomes are reported in different non-convertible scales. As the name implies the RoM divides the treatment mean by the control mean rather than taking the difference between the two. The ratio can be interpreted as the percentage change in the mean value of the treatment group relative to the control group. By meta-analyzing across studies we are making the assumption that the relative change will be homogeneous across all studies, regardless of which scale was used to measure it. Similar to the risk ratio and odds ratio, the RoM is pooled on the log scale; computational formulas are readily available. 5

For the RoM to have any clinical meaning, it is required that in the scale being used, the values are always positive (or always negative) and that a value of “zero” truly means zero. For example, if the outcome were patient temperature, RoM would be a poor choice since a temperature of 0 degrees does not truly represent what we would think of as zero.

2.4. Special Topics

Crossover trials.

A crossover trial is one where all patients receive, in sequence, both the treatment and control interventions. This results in the final data having the same group of patients represented with both their outcome values while in the treatment and control groups. When computing the standard error of the mean difference of a crossover trial, one must consider the correlation between the two groups—a result of the two measurements on different within-person treatments. 5 For most variables, the correlation will be positive, resulting in a smaller standard error than would be seen with the same values in a parallel trial.

To compute the correct pooled standard error requires an estimate of the correlation between the two groups. If correlation is available, the pooled standard error can be computed using the following formula: S E P = S E T 2 + S E C 2 + 2 r S E T S E C

Where r is the within-patient correlation and SE P , SE T , and SE C are the pooled, treatment, and control standard errors respectively

Most studies do not give the correlation or enough information to compute it, and thus it often has to be estimated based on investigator knowledge or imputed. 5 An imputation of 0.5 has been suggested as a good conservative estimate of correlation in the absence of any other information. 48

If a cross-over study reports its data by period, investigators have sometimes used first period data only when including cross-over trials in their meta-analyses—essentially treating the study as if it were a parallel design. This eliminates correlation issues, but has the disadvantage of omitting half the data from the trial.

Cluster Randomized Trials

Cluster trials occur when patients are randomized to treatment and control in groups (or clusters) rather than individually. If the units/subjects within clusters are positively correlated (as they usually are), then there is a loss of precision compared to a standard (non-clustered) parallel design of the same size. The design effect (DE) of a cluster randomized trial is the multiplicative multiplier needed to adjust the standard error computed as if the trial were a standard parallel design. Reported results from cluster trials may not reflect the design effect, and thus it will need to be computed by the investigator. The formula for computing the design effect is: D E = 1 + ( M − 1 ) I C C

Where M is the average cluster size and ICC is the intra-class correlation coefficient (see below).

Computation of the design effect involves a quantity known as the intra-class correlation coefficient (ICC), which is defined as the proportion of the total variance (i.e., within cluster variance plus between cluster variance) that is due to between cluster variance. 5 ICC’s are often not reported by cluster trials and thus a value must be obtained from external literature or a plausible value must be assumed by the investigator.

Mean Difference and Baseline Imbalance

• Use followup data.
• Use change from baseline data.
• Use an ANCOVA model that adjusts for the effects of baseline imbalance. 49

As long as trials are balanced at baseline, all three methods will give similar unbiased estimates of mean difference. 5 When baseline imbalance is present, it can be shown that using ANCOVA will give the best estimate of the true mean difference; however the parameters required to perform this analysis (mean and standard deviations of baseline, follow-up and change from baseline values) are usually not provided by the study authors. 50 If it is not feasible to perform an ANCOVA analysis, the choice of whether to use follow up or change from baseline values depends on the amount of correlation between baseline and final values. If the correlation is less than or equal to 0.5, then using the follow up values will be less biased (with respect to the estimate in the ANCOVA model) than using the change from baseline values. If the correlation is greater than 0.5, then change from baseline values will be less biased than using the follow up values. 51 There is evidence that these correlations are more often greater than 0.5, so the change from baseline means will usually be preferred if estimates of correlation are totally unobtainable. 52 A recent study 51 showed that all approaches were unbiased when there were both few trials and small sample sizes within the trials.

• The analyst should consider carefully which binary measure to analyze.
• If conversion to NNT or NNH is sought, then the risk difference is the preferred measure.
• The risk ratio and odds ratio are likely to be more consistent than the risk difference when the studies differ in baseline risk.
• The risk difference is not the preferred measure when the event is rare.
• The risk ratio is not the preferred measure if switching between occurrence and non occurrence of the event is important to the meta-analysis.
• The odds ratio can be misleading.
• The mean difference is the preferred measure when studies use the same metric.
• When calculating standardized mean difference, Hedges’ g is preferred over Cohen’s d due to the reduction in bias.
• If baseline values are unbalanced, one should perform an ANCOVA analysis. If ANCOVA cannot be performed and the correlation is greater than 0.5, change from baseline values should be used to compute the mean difference. If the correlation less than or equal to 0.5, follow-up values should be used.
• Data from clustered randomized trials should be adjusted for the design effect.

Chapter 3. Choice of Statistical Model for Combining Studies

3.1. introduction.

Meta-analysis can be performed using either a fixed or a random effects model to provide a combined estimate of effect size. A fixed effects model assumes that there is one single treatment effect across studies and any differences between observed effect sizes are due to sampling error. Under a random effects model, the treatment effects across studies are assumed to vary from study and study and follow a random distribution. The differences between observed effect sizes are not only due to sampling error, but also to variation in the true treatment effects. A random effects model usually assumes that the treatment effects across studies follow a normal distribution, though the validity of this assumption may be difficult to verify, especially when the number of studies is small. Alternative distributions 53 or distribution free models 54 , 55 have also been proposed.

Recent advances in meta-analysis include the development of alternative models to the fixed or random effects models. For example, Doi et al. proposed an inverse variance heterogeneity model (the IVhet model) for the meta-analysis of heterogeneous clinical trials that uses an estimator under the fixed effect model assumption with a quasi-likelihood based variance structure. 56 Stanley and Doucouliagosb proposed an unrestricted weighted least squares (WLS) estimator with multiplicative error for meta-analysis and claimed superiority to both conventional fixed and random effects, 57 though Mawdsley et al. 58 found modest differences when compared with the random effects model. These methods have not been fully compared with the many estimators developed within the framework of the fixed and random effects models and are not readily available in most statistical packages; thus they will not be further considered here.

General Considerations for Model Choice

Considerations for model choice include but are not limited to heterogeneity across treatment effects, the number and size of included studies, the type of outcomes, and potential bias. We recommend against choosing a statistical model based on the significance level of a heterogeneity test, for example, picking a fixed effects model when the p-value for the test of heterogeneity is more than 0.10 and a random effects model when P < 0.10, since such an approach does not take the many factors for model choice into full consideration.

In practice, clinical and methodological heterogeneity are always present across a set of included studies. Variation among studies is inevitable whether or not the test of heterogeneity detects it. Therefore, we recommend random effects models, with special considerations for rare binary outcomes (discussed below in the section on combining rare binary outcomes). For a binary outcome, when the estimate of between-study heterogeneity is zero, a fixed effects model (e.g., the Mantel-Haenszel method, inverse variance method, Peto method (for OR), or fixed effects logistic regression) provides an effect estimate similar to that produced by a random effects model. The Peto method requires that no substantial imbalance exists between treatment and control group sizes within trials and treatment effects are not exceptionally large.

When a systematic review includes both small and large studies and the results of small studies are systematically different from those of the large ones, publication bias may be present and the assumption of a random distribution of effect sizes, in particular, a normal distribution, is not justified. In this case, neither the random effects model nor the fixed effects model provides an appropriate estimate and investigators may choose not to combine all studies. 10 Investigators can choose to combine only the large studies if they are well conducted with good quality and are expected to provide unbiased effect estimates. Other potential differences between small and large studies should also be examined.

Choice of Random Effects Model and Estimator

The most commonly used random effects model for combined effect estimates is based on an estimator developed by DerSimonian and Laird (DL) due to its simplicity and ease of implementation. 59 It is well recognized that the estimator does not adequately reflect the error associated with parameter estimation, in particular, when the number of studies is small, and between-study heterogeneity is high. 40 Refined estimators have been proposed by the original authors. 19 , 60 , 61 Other estimators have also been proposed to improve the DL estimator. Sidik and Jonkman (SJ) and Hartung and Knapp (HK) independently proposed a non-iterative variant of the DL estimator using the t-distribution and an adjusted confidence interval for the overall effect. 62 – 64 We refer to this as the HKSJ method. Biggerstaff–Tweedie (BT) proposed another variant of the DL method by incorporating error in the point estimate of between-study heterogeneity into the estimation of the overall effect. 65 There are also many other likelihood based estimators such as maximum likelihood estimate, restricted maximum likelihood estimate and profile likelihood (PL) methods, which better account for the uncertainty in the estimate of between-study variance. 19

Several simulation studies have been conducted to compare the performance of different estimators for combined effect size. 19 – 21 , 66 , 67 For example, Brockwell et al. showed the PL method provides an estimate with better coverage probability than the DL method. 19 Jackson et al. showed that with a small number of studies, the DL method did not provide adequate coverage probability, in particular, when there was moderate to large heterogeneity. 20 However, these results supported the usefulness of the DL method for larger samples. In contrast, the PL estimates resulted in coverage probability closer to nominal values. IntHout et al. compared the performance of the DL and HKSJ methods and showed that the HKSJ method consistently resulted in more adequate error rates than did the DL method, especially when the number of studies was small, though they did not evaluate coverage probability and power. 67 Kontopantelis and Reeves conducted the most comprehensive simulation studies to compare the performance of nine different methods and evaluated multiple performance measures including coverage probability, power, and overall effect estimation (accuracy of point estimates and error intervals). 21 When the goal is to obtain an accurate estimate of overall effect size and the associated error interval, they recommended using the DL method when heterogeneity is low and using the PL method when heterogeneity is high, where the definition of high heterogeneity varies by the number of studies. The PL method overestimated coverage probability in the absence of between-study heterogeneity. Methods like BT and HKSJ, despite being developed to address the limitations of the DL method, were frequently outperformed by the DL method. Encouragingly, Kontopantelis and Reeves also showed that regardless of the estimation method, results are highly robust against even very severe violations of the assumption of normally distributed effect sizes.

Recently there has been a call to use alternative random-effects estimators to replace the universal use of the Dersimonian-Laird random effects model. 68 Based on the results from the simulation studies, the PL method appears to generally perform best, and provides best performance across more scenarios than other methods, though it may overestimate the confidence intervals in small studies with low heterogeneity. 21 It is appropriate to use the DL method when the heterogeneity is low. Another disadvantage of the PL method is that it does not always converge. In those situations, investigators may choose the DL method with sensitivity analyses using other methods, such as the HKSJ method. If non-convergence is due to high heterogeneity, investigators should also reevaluate the appropriateness of combining studies. The PL method (and the DL method) could be used to combine measures for continuous, count, and time to event data, as well as binary data when events are common. Note that the confidence interval produced by the PL method may not be symmetric. It is also worth noting that OR, RR, HR, and incidence rate ratio statistics should be analyzed on the logarithmic scale when the PL, DL, or HKSJ method is used. Finally, a Bayesian approach can also be used since this approach takes the variations in all parameters into account (see the section on Bayesian methods, below).

Role of Generalized Linear Mixed Effects Models

The different methods and estimators discussed above are generally used to combine effect measures directly (for example, mean difference, SMD, OR, RR, HR, and incidence rate ratio). For study-level aggregated binary data and count data, we also recommend the use of the generalized linear mixed effects model assuming random treatment effects. For aggregated binary data, a combined OR can be generated by assuming the binomial distribution with a logit link. It is also possible to generate a combined RR with the binomial distribution and a log link, though the model does not always converge. For aggregated count data, a combined rate ratio can be generated by assuming the Poisson distribution with a log link. Results from using the generalized linear models and directly combining effect measures are similar when the number of studies and/or the sample sizes are large.

3.2. A Special Case: Combining Rare Binary Outcomes

When combining rare binary outcomes (such as adverse event data), few or zero events often occur in one or both arms in some of the studies. In this case, the binomial distribution is not well-approximated by the normal approximation and choosing an appropriate model becomes complicated. The DL method does not perform well with low-event rate binary data. 43 , 69 A fixed effects model often out performs the DL method even in the presence of heterogeneity. 70 When event rates are less than 1 percent, the Peto OR method has been shown to provide the least biased, most powerful combined estimates with the best confidence interval coverage, 43 if the included studies have moderate effect sizes and the treatment and control group are of relatively similar sizes. The Peto method does not perform well when either the studies are unbalanced or the studies have large ORs (outside the range of 0.2-5). 71 , 72 Otherwise, when treatment and control group sizes are very different, effect sizes are large, or when events become more frequent (5 percent to 10 percent), the Mantel-Haenszel method (without a correction factor) or a fixed effects logistic regression provide better combined estimates.

Within the past few years, many methods have been proposed to analyze sparse data from simple averaging, 73 exact methods, 74 , 75 Bayesian approaches 76 , 77 to various parametric models (e.g., generalized linear mixed effect models, beta-binomial model, Gamma-Poisson model, bivariate Binomial-Normal model etc.). Two dominating opinions are to not use continuity corrections, and to include studies with zero events in both arms in the meta-analysis. Great efforts have been made to develop methods that can include such studies.

Bhaumik et al. proposed the simple (unweighted) average (SA) treatment affect with the 0.5 continuity correction, and found that the bias of the SA estimate in the presence of even significant heterogeneity is minimal compared with the bias of MH estimates (with 0.5 correction). 73 A simple average was also advocated by Shuster. 78 However, potential confounding remains an issue for an unweighted estimator. Spittal et al. showed that Poisson regression works better than the inverse variance method for rare events. 79 Kuss et al. conducted a comprehensive simulation of eleven methods, and recommended the use of the beta-binomial model for the three common effect measures (OR, RR, and RD) as the preferred meta-analysis methods for rare binary events with studies of zero events in one or both arms. 80 The beta-binomial model assumes that the observed events follow a binomial distribution and the binomial probabilities follow a beta distribution. In Kuss’s simulation, using a generalized linear model framework to model the treatment effect, an OR was estimated using a logit link, and an RR, using a log link. Instead of using an identity link, RD was estimated based on the estimated event probabilities from the logit model. This comprehensive simulation examined methods that could incorporate data from studies with zero events from both arms and do not need any continuity correction, and only compared the Peto and MH methods as reference methods.

Given the development of new methods that can handle studies with zero events in both arms, we advise that older methods that use continuity corrections be avoided. Investigators should use valid methods that include studies with zero events in one or both arms. For studies with zero events in one arm, or studies with sparse binary data but no zero events, an estimate can be obtained using the Peto method, the Mantel-Haenszel method, or a logistic regression approach, without adding a correction factor, when the between-study heterogeneity is small. These methods are simple to use and more readily available in standard statistical packages. When the between-study heterogeneity is large and/or there are studies with zero events in both arms, the more recently developed methods, such as beta-binomial model, could be explored and used. However, investigators should note that no method gives completely unbiased estimates when events are rare. Statistical methods can never completely solve the issue of sparse data. Investigators should always conduct sensitivity analyses 81 using alternative methods to check the robustness of results to different methods, and acknowledge the inadequacy of data sources when presenting the meta-analysis results, in particular, when the proportion of studies with zero events in both arms are high. If double-zero studies are to be excluded, they should be qualitatively summarized, by providing information on the confidence intervals for the proportion of events in each arm.

A Note on an Exact Method for Sparse Binary Data

For rare binary events, the normal approximation and asymptotic theory for large sample size does not work satisfactorily and exact inference has been developed to overcome these limitations. Exact methods do not need continuity corrections. However, simulation analyses do not identify a clear advantage of early developed exact methods 75 , 82 over a logistic regression or the Mantel-Haenszel method even in situations where these exact methods would theoretically be advantageous. 43 Recent developments of exact methods include Tian et al.’s method of combining confidence intervals 83 and Liu et al.’s method of combining p-value functions. 84 Yang et al. 85 developed a general framework for meta-analysis of rare events by combining confidence distributions (CDs), and showed that Tian’s and Liu’s methods could be unified under the CD framework. Liu showed that exact methods performed better than the Peto method (except when studies are unbalanced) and the Mantel-Haenszel method, 84 though the comparative performance of these methods has not been thoroughly evaluated. Investigators may choose to use exact methods with considerations for the interpretation of effect measures, but we do not specifically recommend exact methods over other models discussed above.

3.3. Bayesian Methods

A Bayesian framework provides a unified and comprehensive approach to meta-analysis that accommodates a wide variety of outcomes, often, using generalized linear model (GLM) with normal, binomial, Poisson and multinomial likelihoods and various link functions. 86

It should be noted that while these GLM models are routinely implemented in the frequentist framework, and are not specific to the Bayesian framework, extensions to more complex situations are most approachable using the Bayesian framework, for example, allowing for mixed treatment comparisons involving repeated measurements of a continuous outcome that varies over time. 87

There are several specific advantages inherent to the Bayesian framework. First, the Bayesian posterior parameter distributions fully incorporate the uncertainty of all parameters. These posterior distributions need not be assumed to be normal. 88 In random-effects meta-analysis, standard methods use only the most likely value of the between-study variance, 59 rather than incorporating the full uncertainty of each parameter. Thus, Bayesian credible intervals will tend to be wider than confidence intervals produced by some classical random-effects analysis such as the DL method. 89 However, when the number of studies is small, the between-study variance will be poorly estimated by both frequentist and Bayesian methods, and the use of vague priors can lead to a marked variation in results, 90 particularly when the model is used to predict the treatment effect in a future study. 91 A natural alternative is to use an informative prior distribution, based on observed heterogeneity variances in other, similar meta-analyses. 92 – 94

Full posterior distributions can provide a more informative summary of the likely value of parameters than the frequentist approach. When communicating results of meta-analysis to clinicians, the Bayesian framework allows direct probability statements to be made and provides the rank probability that a given treatment is best, second best, or worst (see the section on interpreting ranking probabilities and clinically important results in Chapter 5 below). Another advantage is that posterior distributions of functions of model parameters can be easily obtained such as the NNT. 86 Finally, the Bayesian approach allows full incorporation of parameter uncertainty from meta-analysis into decision analyses. 95

Until recently, Bayesian meta-analysis required specialized software such as WinBUGS, 96 OpenBUGS, 97 and JAGS. 98 , 99 Newer open source software platforms such as Stan 100 and Nimble 101 , 102 provide additional functionality and use BUGS-like modeling languages. In addition, there are user written commands that allow data processing in a familiar environment which then can be passed to WinBUGS, or JAGS for model fitting. 103 For example, in R, the package bmeta currently generates JAGS code to implement 22 models. 104 The R package gemtc similarly automates generation of JAGS code and facilitates assessment of model convergence and inconsistency. 105 , 106 On the other hand, Bayesian meta-analysis can be implemented in commonly used statistical packages. For example, SAS PROC MCMC can now implement at least some Bayesian hierarchical models 107 directly, as can Stata, version 14, via the bayesmh command. 108

When vague prior distributions are used, Bayesian estimates are usually similar to estimates obtained from the above frequentist methods. 90 Use of informative priors requires considerations to avoid undue influence on the posterior estimates. Investigators should provide adequate justifications for the choice of priors and conduct sensitivity analyses. Bayesian methods currently require more work in programming, MCMC simulation and convergence diagnostics.

A Note on Using a Bayesian Approach for Sparse Binary Data

It has been suggested that using a Bayesian approach might be a valuable alternative for sparse event data since Bayesian inference does not depend on asymptotic theory and takes into account all uncertainty in the model parameters. 109 The Bayesian fixed effects model provides good estimates when events are rare for binary data. 70 However, the choice of prior distribution, even when non-informative, may impact results, in particular, when a large proportion of studies have zero events in one or two arms. 80 , 90 , 110 Nevertheless, other simulation studies found that when the overall baseline rate is very small and there is moderate or large heterogeneity, Bayesian hierarchical random effect models can provide less biased estimates for the effect measures and the heterogeneity parameters. 77 To reduce the impact of the prior distributions, objective Bayesian methods have been developed 76 , 111 with special attention paid to the coherence between the prior distributions of the study model parameters and the meta-parameter, 76 though the Bayesian model was developed outside the usual hierarchical normal random effects framework. Further evaluations of these methods are required before recommendations of these objective Bayesian methods might be made.

3.4. Recommendations

• The PL method appears to generally perform best. The DL method is also appropriate when the between-study heterogeneity is low.
• For study-level aggregated binary data and count data, the use of a generalized linear mixed effects model assuming random treatment effects is also recommended.
• Methods that use continuity corrections should be avoided.
• For studies with zero events in one arm, or studies with sparse binary data but no zero events, an estimate can be obtained using the Peto method, the Mantel-Haenszel method, or a logistic regression approach, without adding a correction factor, when the between-study heterogeneity is low.
• When the between-study heterogeneity is high, and/or there are studies with zero events in both arms, more recently developed methods such as a beta-binomial model could be explored and used.
• Sensitivity analyses should be conducted with acknowledgement of the inadequacy of data.
• If investigators choose Bayesian methods, use of vague priors is supported.

Chapter 4. Quantifying, Testing, and Exploring Statistical Heterogeneity

4.1. statistical heterogeneity in meta-analysis.

Statistical heterogeneity was explained in general in Chapter 1 . In this chapter, we provide a deeper discussion from a methodological perspective. Statistical heterogeneity must be expected, quantified and sufficiently addressed in meta-analyses. 112 We recommend performing graphic and quantitative exploration of heterogeneity in combination. 113 In this chapter, it is assumed that a well-specified research question has been posed, the relevant literature has been reviewed, and a set of trials meeting selection criteria have been identified. Even when trial selection criteria are aimed toward identifying studies that are adequately homogenous, it is common for trials included in a meta-analysis to differ considerably as a function of (clinical and/or methodological) heterogeneity that was reviewed in Chapter 1 . Even when these sources of heterogeneity have been accounted for, statistical heterogeneity often remains. Statistical heterogeneity refers to the situation where estimates across studies have greater variability than expected from chance variation alone. 113 , 114

4.2. Visually Inspecting Heterogeneity

Although simple histograms, box plots, and other related graphical methods of depicting effect estimates across studies may be helpful preliminarily, these approaches do not necessarily provide insight into statistical heterogeneity. However, forest and funnel plots can be helpful in the interpretation of heterogeneity particularly when examined in combination with quantitative results. 113 , 115

Forest Plots

Forest plots can help identify potential sources and the extent of statistical heterogeneity. Meta-analyses with limited heterogeneity will produce forest plots with grossly visual overlap of study confidence intervals and the summary estimate. In contrast, a crude sign of statistical heterogeneity would be poor overlap. 115 An important recommendation is to graphically present between-study variance on forest plots of random effects meta-analyses using prediction intervals, which are on the same scale as the outcome. 93 The 95% prediction interval estimates where true effects would be expected for 95% of future studies. 93 When between-study variance is greater than zero, the prediction interval will cover a wider range than the confidence interval of the summary effect. 116 As proposed by Guddat et al. 117 and endorsed by IntHout et al., 116 including the prediction interval as a rectangle at the bottom of forest plots helps differentiate between-study variation from the confidence interval of the summary effect that is commonly depicted as a diamond.

Funnel Plots

Funnel plots are often thought of as representing bias, but they also can aid in detecting sources of heterogeneity. Funnel plots are essentially the plotting of effect sizes observed in each study (x-axis) around the summary effect size versus the degree of precision of each study (typically by standard error, variance, or precision on the y-axis). A meta-analysis that includes studies that estimate the same underlying effect across a range of precision, and has limited bias and heterogeneity would result in a funnel plot that resembles a symmetrical inverted funnel shape with increasing dispersion ranging with less precise (i.e., smaller) studies. 115 In the event of heterogeneity and/or bias, funnel plots will take on an asymmetric pattern around the summary effect size and also provide evidence of scatter outside the bounds of the 95% confidence limits. 115 Asymmetry in funnel plots can be difficult to detect visually, 118 and can be misleading due to multiple contributing factors. 113 , 119 , 120 Formal tests for funnel plot asymmetry (such as Egger’s test 15 for continuous outcomes, or the arcsine test proposed by Rucker et al., 27 for binary data) are available but should not be used with a meta-analysis involving fewer than 10 studies because of limited power. 113 Given the above cautions and considerations, funnel plots should only be used to complement other approaches in the preliminary analysis of heterogeneity.

4.3. Quantifying Heterogeneity

The null hypothesis of homogeneity in meta-analysis is that all studies are evaluating the same effect, 22 (i.e., all studies have the same true effect parameter that may or may not be equivalent to zero) and the alternative hypothesis is that at least one study has an effect that is different from the summary effect.

• Where Q is the heterogeneity statistic,
• w is the study weight based on inverse variance weighting,
• x is the observed effect size in each trial, and
• x ^ w is the summary estimate in a fixed-effect meta-analysis.

The Q statistic is assumed to have an approximate χ 2 distribution with k – 1 degrees of freedom. When Q is in excess over k – 1 and the associated p-value is low (typically, a p-value of <0.10 is used as a cut-off), the null hypothesis of homogeneity can be rejected. 22 , 122 Interpretation of a Q statistic in isolation is not advisable however, because it has low statistical power in meta-analyses involving a limited number of studies 123 , 124 and may detect unimportant heterogeneity when the number of studies included in a meta-analysis is large. Importantly, since heterogeneity is expected in meta-analyses even without statistical tests to support that claim, non-significant Q statistics must not be interpreted as the absence of heterogeneity. Moreover, the interpretation of Q in meta-analyses is more complicated than typically represented, because the actual distribution of Q is dependent on the measure of effect 125 and only approximately χ 2 in large samples. 122 Even if the null distribution of Q were χ 2 , universally interpreting all values of Q greater than the mean of k − 1 as indicating heterogeneity would be an oversimplification. 122 There are expansions to approximate Q for meta-analyses of standardized mean difference, 125 risk difference, 125 and odds ratios 126 that should be used as alternatives to Q , particularly when sample sizes of studies included in a meta-analysis are small. 122 The Q statistic and expansions thereof must be interpreted along with other heterogeneity statistics and with full consideration of their limitations.

Graphical Options for Examining Contributions to Q

Hardy and Thompson proposed using probability plots to investigate the contribution that each study makes to Q . 127 When each study is labeled, those deviating from the normal distribution in a probability plot have the greatest influence on Q . 127 Baujat and colleagues proposed another graphical method to identify studies that have the greatest impact on Q . 128 Baujat proposed plotting the contribution to the heterogeneity statistic for each study on the horizontal axis, and the squared difference between meta-analytic estimates with and without the i th study divided by the estimated variance of the meta-analytic estimate without the i th study along the vertical axis. Because of the Baujat plot presentation, studies that have the greatest influence on Q are located in the upper right corner for easy visual identification. Smaller studies have been shown to contribute more to heterogeneity than larger studies, 129 which would be visually apparent in Baujat plots. We recommend using these graphical approaches only when there is significant heterogeneity, and only when it is important to identify specific studies that are contributing to heterogeneity.

Between-Study Variance

• Where τ 2 is the parameter of between-study variance of the true effects,
• DL is the DerSimonian and Laird approach to τ 2 ,
• Q is the heterogeneity statistic (as above),
• k -1 is the degrees of freedom, and
• w is the weight applied to each study based on inverse variance weighting.

Since variance cannot be less than zero, a τ 2 less than zero is set to zero. The value of τ 2 is integrated into the weights of random-effects meta-analysis as presented in Chapter 3 . Since the DerSimonian and Laird approach to τ 2 is derived in part from Q , the problems with Q described above apply to the τ 2 parameter. 122 There are many alternatives to DerSimonian and Laird when estimating between-study variance. In a recent simulation, Veroniki and colleagues 121 compared 16 estimators of between-study variance; they argued that the Paule and Mandel 130 method of estimating between-study variance is a better alternative to the DerSimonian and Laird parameter for continuous and binary data because it less biased (i.e., yields larger estimates) when between-study variance is moderate-to-large. 121 At the time of this guidance, the Paule and Mandel method of estimating between-study variance is only provisionally recommended as an alternative to DerSimonian and Laird. 129 , 131 Moreover, Veroniki and colleagues provided evidence that the restrictive maximum likelihood estimator 132 is a better alternative to the DerSimonian and Laird parameter of between-study variance for continuous data because it yields similar values for low-to-moderate between-study variance and larger estimates in conditions of high between-study variance. 121

Inconsistency Across Studies

Another statistic that should be generated and interpreted even when Q is not statistically significant is the proportion of variability in effect sizes across studies that is explained by heterogeneity vs. random error or I 2 that is related to Q . 22 , 133

• Where Q is the estimate of between-study variance, and
• k −1 is the degrees of freedom.
• Where τ 2 is the parameter of between-study variance, and
• σ 2 is the within-study variance.

I 2 is a metric of how much heterogeneity is influencing the meta-analysis. With a range from 0% (indicating no heterogeneity) to 100% (indicating that all of the observed variance is attributable to heterogeneity), the I 2 statistic has several advantages over other heterogeneity statistics including its relative simplicity as a signal-to-noise ratio, and focus on how heterogeneity may be influencing interpretation of the meta-analysis. 59 It is important to note that I 2 increases with increasing study precision and hence is dependent on sample size. 27 By various means, confidence/uncertainty intervals can be estimated for I 2 including Higgins’ test-based method. 22 , 23 the assumptions involved in the construction of 95% confidence intervals cannot be justified in all cases, but I 2 confidence intervals based on frequentist assumptions generally provide sufficient coverage of uncertainty in meta-analyses. 133 In small meta-analyses, it has even been proposed that confidence intervals supplement or replace biased point estimates of I 2 . 26 It is important to note that since I 2 is based on Q or τ 2 , any problems that influence Q or τ 2 (most notably the number of trials included in the meta-analysis) will also indirectly interfere with the computation of I 2 . It is also important to consider that I 2 also is dependent on which between-study variance estimator is used. For example, there is a high level of agreement comparing I 2 derived from DerSimonian and Laird vs. Paul and Mandel methods of estimating between-study variance. 131 In contrast, I 2 derived from other methods of estimating between-study variance have low levels of agreement. 131

Based primarily on the observed distributions of I 2 across meta-analyses, there are ranges that are commonly used to further categorize heterogeneity. That is, I 2 values of 25%, 50%, and 75% have been proposed as working definitions of what could be considered low, moderate, and high proportions, respectively, of variability in effect sizes across studies that is explained by heterogeneity. 59 Currently, the Cochrane manual also includes ranges for interpreting I 2 (0%-40% might not be important, 30%-60% may represent moderate heterogeneity, 50-90% may represent substantial heterogeneity and 75-100% may represent considerable heterogeneity). 10 Irrespective of which categorization of I 2 is used, this statistic must be interpreted with the understanding of several nuances, including issues related to a small number of studies (i.e., fewer than 10), 24 – 26 and inherent differences in I 2 comparing binary and continuous effect sizes. 28 , 29 Moreover, I 2 of zero is often misinterpreted in published reports as being synonymous with the absence of heterogeneity despite upper confidence interval limits that most often would exceed 33% when calculated. 134 Finally, a high I 2 does not necessarily mean that dispersion occurs across a wide range of effect sizes, and a low I 2 does not necessarily mean that dispersion occurs across a narrow range of effect sizes; the I 2 is a signal-to-noise metric, not a statistic about the magnitude of heterogeneity.

4.4. Exploring Heterogeneity

Meta-regression.

Meta-regression is a common approach employed to examine the degree to which study-level factors explain statistical heterogeneity. 135 Random effects meta-regression, as compared with fixed effect meta-regression, allows for residual heterogeneity (i.e., between-study variance that is not explained by study-level factors) to incorporated into the model. 136 Because of this feature, among other benefits described below and in Chapter 3 , random effects meta-regression is recommended over fixed effect meta-regression. 137 It is the default of several statistical packages to use a modified estimator of variance in random effects meta-regression that employs a t distribution in lieu of a standard normal distribution when calculating p-values and confidence intervals (i.e., the Knapp-Hartung modification). 138 This approach is recommended to help mitigate false-positive rates that are common in meta-regression. 137 Since the earliest papers on random effects meta-regression, there has been general caution about the inherent low statistical power in analyses when there are fewer than 10 studies for each study-level factor modelled. 136 Currently, the Cochrane manual recommends that there be at least 10 studies per characteristic modelled in meta-regression 10 over the enduring concern about inflated false-positive rates with too few studies. 137 Another consideration that is reasonable to endorse is adjusting the level of statistical significance to account for making multiple comparisons in cases where more than one characteristic is being investigated in meta-regression.

Beyond statistical considerations important in meta-regression, there are also several important conceptual considerations. First, study-level characteristics to be considered in meta-regression should be pre-specified, scientifically defensible and based on hypotheses. 8 , 10 This first consideration will allow investigators to focus on factors that are believed to modify the effect of intervention as opposed to clinically meaningless study-level characteristics. Arguably, it may not be possible to identify all study-level characteristics that may modify intervention effects. The focus of meta-regression should be on factors that are plausible. Second, meta-regression should be carried out under full consideration of ecological bias (i.e., the inherent problems associated with aggregating individual-level data). 139 As classic examples, the mean study age or the proportion of study participants who were female may result in different conclusions in meta-regression as opposed to how these modifying relationships functioned in each trial. 135

Multiple Meta-regression

It may be desirable to examine the influence of more than one study-level factor on the heterogeneity observed in meta-analyses. Recalling general cautions and specific recommendations about the inherent low statistical power in analyses wherein there are fewer than 10 studies for each study-level factors modelled, 10 , 136 , 137 multiple meta-regression (that is, a meta-regression model with more than one study-level factor included) should only be considered when study-level characteristics are pre-specified, scientifically defensible, and based on hypotheses, and when there are 10 or more studies for each study-level factor included in meta-regression.

Subgroup Analysis

Subgroup analysis is another common approach employed to examine the degree to which study-level factors explain statistical heterogeneity. Since subgroup analysis is a type of meta-regression that incorporates a categorical study-level factor as opposed to a continuous study-level factor, it is similarly important that the grouping of studies to be considered in subgroup analysis be pre-specified, scientifically defensible and based on hypotheses. 8 , 10 Like other forms of meta-regression, subgroup analyses have a high false-positive rate. 137 and may be misleading when few studies are included. There are two general approaches to handling subgroups in meta-analysis. First, a common use is to perform meta-analyses within subgroups without any statistical between-group comparisons. A central problem with this approach is the tendency to misinterpret results from within separate groups as being comparative. That is, identification of groups wherein there is a significant summary effect and/or limited heterogeneity and others wherein there is no significant summary effect and/or substantive heterogeneity does not necessarily indicate that the subgroup factor explains overall heterogeneity. 10 Second, it is recommended to incorporate the subgrouping factor into a meta-regression framework. 140 Doing so allows for quantification of both within and among subgroup heterogeneity as well as well as formal statistical testing that informs whether the summary estimates are different across subgroups. Moreover, subgroup analysis in a meta-regression framework will allow for formal testing of residual heterogeneity in a similar fashion to meta-regression using a continuous study-level factor.

Detecting Outlying Studies

Under consideration that removal of one or more studies from a meta-analysis may interject bias in the results, 10 identification of outlier studies may help build the evidence necessary to justify removal. Visual examination of forest, funnel, normal probability and Baujat plots (described in detail earlier in this chapter) alone may be helpful in identifying studies with inherent outlying characteristics. Additional procedures that may be helpful in interpreting the influence of single studies are quantifying the summary effect without each study (often called one study removed), and performing cumulative meta-analyses. One study removed procedures simply involve sequentially estimating the summary effect without each study to determine if single studies are having a large influence on model results. Using cumulative meta-analysis, 141 it is possible to graph the accumulation of evidence of trials reporting at treatment effect. Simply put, this approach integrates all information up to and including each trial into summary estimates. By looking at the graphical output (from Stata’s metacum command or the R metafor cumul() function), one can examine large shifts in the summary effect that may serve as evidence for study removal. Another benefit of cumulative meta-analysis is detecting shifts in practice (e.g., guideline changes, new treatment approval or discontinuation) that would foster subgroup analysis.

Viechtbauer and Chung proposed other methods that should be considered to help identify outliers. One option is to examine extensions of linear regression residual diagnostics by using studentized deleted residuals. 142 Other options are to examine the difference between the predicted average effect with and without each study (indicating by how many standard deviations the average effect changes) or to examine what effect the deletion of each study has on the fitted values of all studies simultaneously (in a metric similar to Cook’s distance). 142 Particularly in combination, there methods serve as diagnostics that are more formal than visual inspection and both one study removed and cumulative meta-analysis procedures.

4.5. Special Topics

Baseline risk (control-rate) meta-regression.

For studies with binary outcomes, the “control rate” refers to the proportion of subjects in the control group who experienced the event. The control rate can be viewed as a surrogate for covariate differences between studies because it is influenced by illness severity, concomitant treatment, duration of follow-up, and/or other factors that may differ across studies. 143 , 144 Groups of patients with higher underlying risk for poor outcomes may experience different benefits and/or harms from treatment compared with groups of patients who have lower underlying risk. 145 Hence, the control-rate can be used to test for interactions between underlying population risk at baseline and treatment benefit.

To examine for an interaction between underlying population risk and treatment benefit, we recommend a simplified approach. First, generate a scatter plot of treatment effect against control rate to visually assess whether there may be a relation between the two. Since the RD tends to be highly correlated with the control rate, 144 we recommend using an RR or OR when examining a treatment effect against the control rate in all steps. The purpose of generating a scatterplot is simply to give preliminary insight into how differences in baseline risk (control rate) may influence the amount of observed variability in effect sizes across studies. Second, use hierarchical meta-regression 144 or Bayesian meta-regression 146 models to formally test the interaction between underlying population risk and treatment benefit. Although a weighted regression has been proposed as an intermediary step between developing a scatter plot and meta-regression, this approach identifies a significant relation between control rate and treatment effect twice as often compared with more suitable approaches (above), 144 , 146 and a negative finding would likely need to be replicated using meta-regression. Hence, the simplified two-step approach may help streamline the process.

Multivariate Meta-analysis

There are both inherent benefits and disadvantages of using meta-analysis to examine multiple outcomes simultaneously (that is, “multivariate meta-analysis”), and much methodological work has been done in both frequentist and Bayesian frameworks in recent years. 147 – 156 . Some of these methods are readily available in statistical packages (for example, Stata mvmeta ).

One of the advantages of multivariate meta-analysis is being able to incorporate multiple outcomes into one model as opposed to the conduct of multiple univariate meta-analyses wherein the outcomes are handled as being independent. 150 Another advantage of multivariate meta-analysis is being able to gain insight into relationships among study outcomes. 150 , 157 An additional advantage of multivariate meta-analysis is that different clinical conclusions may be made; 150 it may be considered easier to present results from a single multivariate meta-analysis than from several univariate analyses that may make different assumptions. Further, multivariate methods may have the potential to reduce the impact of outcome reporting bias. 150 , 158 , 159

• the disconnect between how outcomes are handled within each trial (typically in a univariate fashion) compared with a multivariate meta-analysis;
• estimation difficulties particularly around correlations between outcomes (seldom reported; see Bland 160 for additional commentary);
• overcoming assumptions of normally-distributed random effects with joint outcomes (difficult to justify with joint distributions);
• marginal model improvement in the multivariate vs. univariate case (often not sufficient trade off in effort); and
• amplification of publication bias (e.g., secondary outcomes are not published as frequently). 150

Another potential challenge is the appropriate quantification of heterogeneity in multivariate meta-analysis; but, there are newer alternatives that seem to make this less of a concern. These methods include but are not limited to the multivariate H 2 statistic (the ratio of a generalization of Q and its degrees of freedom, with an accompanying generalization of I 2 ( I H 2 ) ). 163 Finally, limitations to existing software for broad implementation and access to multivariate meta-analysis has been a long-standing barrier to this approach. With currently available add-on or base statistical packages, however, multivariate meta-analysis can be more readily performed, 150 and emerging approaches to multivariate meta-analyses are available to be integrated into standard statistical output. 153 However, the gain in precision of parameter estimates is often modest, and the conclusions from the multivariate meta-analysis are often the same as those from the univariate meta-analysis for individual outcomes, 164 which may not justify the increased complexity and difficulty.

With the exception of diagnostic testing meta-analysis (which provides a natural situation to meta-analyze sensitivity and specificity simultaneously, but which is out of scope for this report) and network meta-analysis (a special case of multivariate meta-analysis with unique challenges, see Chapter 5 ), multivariate meta-analysis has not been widely used in practice. However, we are likely to see multivariate meta-analysis approaches become more accessible to stakeholders involved with systematic reviews. 160 In the interim, however, we do not recommend this approach be used routinely.

Dose-Response Meta-analysis

Considering different exposure or treatment levels has been a longstanding consideration in meta-analyses involving binary outcomes. 165 , 166 and new methods have been developed to extend this approach to differences in means. 167 Meta-regression is commonly employed to test the relationship between exposure or treatment level and the intervention effect (i.e., dose-response). The best-case scenario for testing dose-response using meta-regression is when there are several trials that compared the dose level versus control for each dosing level. That way, subgroup analysis can be performed to provide evidence of effect similarity within groups of study-by-dose in addition to a gradient of treatment effects across groups. 10 Although incorporating study-level average dose can be considered, it should only be conducted in circumstances where there was limited-to-no variation in dosing within intervention arms of the studies included. In many instances, exposure needs to be grouped for effective comparison (e.g., ever vs. never exposed), but doing so raises the issues of non-independence and covariance between estimates. 168 Hamling et al., developed a method of deriving relative effect and precision estimates for such alternative comparisons in meta-analysis that are more reasonable compared with methods that ignore interdependence of estimates by level. 168 In the case of trials involving differences in means, dose-response models are estimated within each study in a first stage and an overall curve is obtained by pooling study-specific dose-response coefficients in a second stage. 167 A key benefit to this emerging approach to differences in means is modeling non-linear dose-response curves in unspecified shapes (including the cubic spline described in the derivation study). 167 Considering the inherent low statistical power associated with meta-regression in general, results of dose-response meta-regression should generally not be used to indicate that a dose response does not exist. 10

• Statistical heterogeneity should be expected, visually inspected and quantified, and sufficiently addressed in all meta-analyses.
• Prediction intervals should be included in all forest plots.
• Investigators should be consider evaluating multiple metrics of heterogeneity, between-study variance, and inconsistency (i.e., Q , τ 2 and I 2 along with their respective confidence intervals when possible).
• A non-significant Q should not be interpreted as the absence of heterogeneity, and there are nuances to the interpretation of Q that carry over to the interpretation of τ 2 and I 2 .
• Random effects is the preferred method for meta-regression that should be used under consideration of low power associated with limited studies (i.e., <10 studies per study-level factor) and the potential for ecological bias.
• We recommend a simplified two-step approach to control-rate meta-regression that involves scatter plotting and then hierarchical or Bayesian meta-regression.
• Routine use of multivariate meta-analysis is not recommended.

Chapter 5. Network Meta-Analysis (Mixed Treatment Comparisons/Indirect Comparisons)

5.1. rationale and definition.

Decision makers, whether patients, providers or policymakers generally want head-to-head estimates of the comparative effectiveness of the different interventions from which they have to choose. However, head-to-head trials are relatively uncommon. The majority of trials compare active agents with placebo, which has left patients and clinicians unable to compare across treatment options with sufficient certainty.

Therefore, an approach has emerged to compare agents indirectly. If we know that intervention A is better than B by a certain amount, and we know how B compares with C; we can indirectly infer the magnitude of effect comparing A with C. Occasionally, a very limited number of head-to-head trials are available (i.e., there may be a small number of trials directly comparing A with C). Such trials will likely produce imprecise estimates due to the small sample size and number of events. In this case, the indirect comparisons of A with C can be pooled with the direct comparisons, to produce what is commonly called a network meta-analysis estimate (NMA). The rationale for producing such an aggregate estimate is to increase precision, and to utilize all the available evidence for decision making.

Frequently, more than two active interventions are available and stakeholders want to compare (rank) many interventions, creating a network of interventions with comparisons accounting for all the permutations of pairings within the network. The following guidance focuses on NMA of randomized controlled trials. NMA of nonrandomized studies is statistically possible; however, without randomization, NMA assumptions would likely not be satisfied and the results would not be reliable.

5.2. Assumptions

There are three key assumptions required for network meta-analysis to be valid:

I. Homogeneity of direct evidence

When important heterogeneity (unexplained differences in treatment effect) across trials is noted, confidence in a pooled estimate decreases. 169 This is true for any meta-analysis. In an NMA, direct evidence (within each pairwise comparison) should be sufficiently homogeneous. This can be evaluated using the standard methods for evaluating heterogeneity ( I 2 statistic, τ 2 , Cochran Q test, and visual inspection of forest plots for consistency of point estimates from individual trials and overlap of confidence intervals).

II. Transitivity, similarity or exchangeability

Patients enrolled in trials of different comparisons in a network need to be sufficiently similar in terms of the distribution of effect modifiers. In other words, patients should be similar to the extent that it is plausible that they were equally likely to have received any of the treatments in the network. 170 Similarly, active and placebo controlled interventions across trials need to be sufficiently similar in order to attribute the observed change in effect size to the change in interventions.

Transitivity cannot be assessed quantitatively. However, it can be evaluated conceptually. Researchers need to identify important effect modifiers in the network and assess whether differences reported by studies are large enough to affect the validity of the transitivity assumption.

III. Consistency (Between Direct and Indirect Evidence)

Comparing direct and indirect estimates in closed loops in a network demonstrates whether the network is consistent (previously called coherent). Important differences between direct and indirect evidence may invalidate combining them in a pooled NMA estimate.

Consistency refers to the agreement between indirect and direct comparison for the same treatment comparison. If a pooled effect size for a direct comparison is similar to the pooled effect size from indirect comparison, we say the network is consistent; otherwise, the network is inconsistent or incoherent. 171 , 172 Multiple causes have been proposed for inconsistency, such as differences in patients, treatments, settings, timing, and other factors.

Statistical models have been developed to assume consistency in the network (consistency models) or account for inconsistency between direct and indirect comparison (inconsistency models). Consistency is a key assumption/prerequisite for a valid network meta-analysis and should always be evaluated. If there is substantial inconsistency between direct and indirect evidence, a network meta-analysis should not be performed. Fortunately, inconsistency can be evaluated statistically.

5.3. Statistical Approaches

The simplest indirect comparison approach is to qualitatively compare the point estimates and the overlap of confidence intervals from two direct comparisons that use a common comparator. Two treatments are likely to have comparable effectiveness if their direct effects relative to a common comparator (e.g., placebo) have the same direction and magnitude, and if there is considerable overlap in their confidence intervals. However, such qualitative comparisons have to be interpreted cautiously because the degree to which confidence intervals overlap is not a reliable substitute for formal hypothesis testing. Formal testing methods adjust the comparison of the interventions by the results of their direct comparison with a common control group and at least partially preserve the advantages of randomization of the component trials. 173

Many statistical models for network meta-analysis have been developed and applied in the literature. These models range from simple indirect comparisons to more complex mixed effects and hierarchical models, developed in both Bayesian and frequentist frameworks, and using both contrast level and arm level data.

Simple Indirect Comparisons

Simple indirect comparisons apply when there is no closed loop in the evidence network. A closed loop means that each comparison in a particular loop has both direct and indirect evidence. At least three statistical methods are available to conduct simple indirect comparisons: (1) the adjusted indirect comparison method proposed by Bucher et al, 174 (2) logistic regression, and (3) random effects meta-regression.

When there are only two sets of trials, say, A vs. B and B vs. C, Bucher‘s method is sufficient to provide the indirect estimate of A vs. C as: log(OR AC )=log(OR AB )-log(OR BC ) and

Var(Log(OR AC )) = Var(Log(OR AB )) + Var(Log(OR BC )), where OR is the odds ratio. Bucher’s method is valid only under a normality assumption on the log scale.

Logistic regression uses arm-level dichotomous outcomes data and is limited to odds ratios as the measure of effect. By contrast, meta-regression and adjusted indirect comparisons typically use contrast-level data and can be extended to risk ratios, risk differences, mean difference and any other effect measures. Under ideal circumstances (i.e., no differences in prognostic factors exist among included studies), all three methods result in unbiased estimates of direct effects. 175 Meta-regression (as implemented in Stata, metareg ) and adjusted indirect comparisons are the most convenient approaches for comparing trials with two treatment arms. A simulation study supports the use of random effects for either of these approaches. 175

Mixed Effects and Hierarchical Models

More complex statistical models are required for more complex networks with closed loops where a treatment effect could be informed by both direct and indirect evidence. These models typically assume random treatment effects and take the complex data structure into account, and may be broadly categorized as mixed effects, or hierarchical models.

Frequentist Approach

Lumley proposed the term “network meta-analysis” and the first network meta-analysis model in the frequentist framework, and constructed a random-effects inconsistency model by incorporating sampling variability, heterogeneity, and inconsistency. 176 The inconsistency follows a common random-effects distribution with mean of 0. It can use arm-level and contrast-level data and can be easily implemented in statistical software, including R’s lme package. However, studies included in the meta-analysis cannot have more than two arms.

Further development of network meta-analysis models in the frequentist framework addressed how to handle multi-armed trials as well as new methods of assessing inconsistency. 171 , 177 – 179 Salanti et al. provided a general network meta-analysis formulation with either contrast-based data or arm-based data, and defined the inconsistency in a standard way as the difference between ‘direct’ evidence and ‘indirect’ evidence. 177 In contrast, White et al. and Higgins et al. proposed to use a treatment-by-design interaction to evaluate inconsistency of evidence, and developed consistency and inconsistency models based on contrast-based multivariate random effects meta-regression. 171 , 178 These models can be implemented using network , a suite of commands in Stata with input data being either arm-level or contrast level.

Bayesian Approach

Lu and Ades proposed the first Bayesian network meta-analysis model for multi-arm studies that included both direct and indirect evidence. 180 The treatment effects are represented by basic parameters and functional parameters. Basic parameters are effect parameters that are directly compared to the baseline treatment, and functional parameters are represented as functions of basic parameters. Evidence inconsistency is defined as a function of a functional parameter and at least two basic parameters. The Bayesian model has been extended to incorporate study-level covariates in an attempt to explain between-study heterogeneity and reduce inconsistency, 181 to allow for repeated measurements of a continuous endpoint that varies over time, 87 or to appraise novelty effects. 182 A Bayesian multinomial network meta-analysis model was also developed for unordered (nominal) categorical outcomes allowing for partially observed data in which exact event counts may not be known for each category. 183 Additionally, Dias et al. set out a generalized linear model framework for the synthesis of data from randomized controlled trials, which could be applied to binary outcomes, continuous outcomes, rate models, competing risks, or ordered category outcomes. 86

Commonly, a vague (flat) prior is chosen for the treatment effect and heterogeneity parameters in Bayesian network meta-analysis. A vague prior distribution for heterogeneity however may not be appropriate when the number of studies is small. 184 An informative prior for heterogeneity can be obtained from the empirically derived predictive distributions for the degree of heterogeneity as expected in various settings (depending on the outcomes assessed and comparisons made). 185 In the NMA framework, frequentist and Bayesian approaches often provide similar results; particularly because of the common practice to use non-informative priors in the Bayesian analysis. 186 – 188 Frequentist approaches, when implemented in a statistical package, are easily applied in real-life data analysis. Bayesian approaches are highly adaptable to complex evidence structures and provide a very flexible modeling framework, but need a better understanding of the model specification and specialized programing skills.

Arm-Based Versus Contrast-Based Models

It is important to differentiate arm-based/contrast-based models from arm-level/contrast-level data. Arm-level and contrast-level data describe how outcomes are reported in the original studies. Arm-level data represent raw data per study arm (e.g., the number of events from a trial per group); while contrast-level data show the difference in outcomes between arms in the form of absolute or relative effect size (e.g., mean difference or the odds ratio of events).

Contrast-based models resemble the traditional approaches used in meta-analysis of direct comparisons. Absolute or relative effect sizes and associated variances are first estimated (per study) and then pooled to produce an estimate of the treatment comparison. Contrast-based models preserve randomization and, largely, alleviate risk of observed and unobserved imbalance between arms within a study. They use effect sizes relative to the comparison group and reduce the variability of outcomes across studies. Contrast-based models are the dominant approach used in direct meta-analysis and network meta-analysis in current practice.

Arm-based models depend on directly combining the observed absolute effect size in individual arms across studies; thereby producing a pooled rate or mean of the outcome per arm. Estimates can be compared among arms to produce a comparative effect size. Arm-based models break randomization; therefore, the comparative estimate will likely be at an increased risk of bias. Following this approach, nonrandomized studies or even noncomparative studies can be included in the analysis. Multiple models have been proposed for the arm-based approach, especially in the Bayesian framework. 177 , 189 – 192 However, the validity of arm-based methods is under debate. 178 , 193 , 194

Assessing Consistency

Network meta-analysis generates results for all pairwise comparisons; however, consistency can only be evaluated when at least one closed loop exists in the network. In other words, the network must have at least one treatment comparison with direct evidence. Many statistical methods are available to assess consistency. 173 , 174 , 176 , 195 – 200

These methods can generally be categorized into two types: (1) an overall consistency measure for the whole network; and (2) a loop-based approach in which direct and indirect estimates are compared. In the following section, we will focus on a few widely used methods in the literature.

• Single Measure for Network Consistency : These approaches use a single measure that represents consistency for the whole network. Lumley assumes that, for each treatment comparison (with or without direct evidence), there is a different inconsistency factor; and the inconsistency factor varies for all treatment comparisons and follows a common random-effects distribution. The variance of the differences, ω, also called incoherence, measures the overall inconsistency of the network. 176 A ω above 0.25 suggests substantial inconsistency and in this case, network meta-analysis may be considered inappropriate. 201
• Global Wald Test : Another approach is to use global Wald test, which tests an inconsistency factor that follows a Χ 2 distribution under the null consistency assumption. 178 A p-value less than 0.10 can be used to determine statistical significance. Rejection of the null is evidence that the model is not consistent.
• Z-test : A simple z-test can be used to compare the difference of the pooled effect sizes between direct and indirect comparisons. 174 Benefits of this approach include simplicity, ease of application, and the ability to identify specific loops with large inconsistency. Limitations include the need for multiple correlated tests.
• Side-splitting: A “node” is a treatment and a “side” (or edge) is a comparison. Dias et al. suggests that each comparison can be assessed by comparing the difference of the pooled estimate from direct evidence to the pooled estimate without direct evidence. 196 Side-splitting (sometimes referred to as node-splitting) can be implemented using the Stata network sidesplit command or R gemtc package.

Several graphical tools have been developed to describe inconsistency. One is the inconsistency plot developed by Chaimani et al. 197 Similar to a forest plot, the inconsistency plot graphically presents an inconsistency factor (the absolute difference between the direct and indirect estimates) and related confidence interval for each of the triangular and quadratic loops in the network. The Stata ifplot command can be used for this purpose.

It is important to understand the limitations of these methods. Lack of statistical significance of an inconsistency test does not prove consistency in the network. Similar to Cochran’s Q test of heterogeneity testing in traditional meta-analysis (which is often underpowered), statistical tests for inconsistency in NMA are also commonly underpowered due to the limited number of studies in direct comparisons.

• Abandon NMA and only perform traditional meta-analysis;
• Present the results from inconsistency models (that incorporate inconsistency) and acknowledge the limited trustworthiness of the NMA estimates;
• Split the network to eliminate the inconsistent nodes;
• Attempt to explain the causes of inconsistency by conducting network meta-regression to test for possible covariates causing the inconsistency: and
• Use only direct estimates for the pairwise NMA comparisons that show inconsistency (i.e., use direct estimates for inconsistent comparisons and use NMA estimates for consistent comparisons).

5.4. Considerations of Model Choice and Software

Consideration of indirect evidence.

Empirical explorations suggest that direct and indirect comparisons often agree, 174 – 176 , 202 – 204 but with notable exceptions. 205 In principle, the validity of combining direct and indirect evidence relies on the transitivity assumption. However, in practice, trials can vary in numerous ways including population characteristics, interventions, and cointerventions, length of follow-up, loss to follow-up, study quality, etc. Given the limited information in many publications and the inclusion of multiple treatments, the validity of combining direct and indirect evidence is often unverifiable. The statistical methods to evaluate inconsistency generally have low power, and are confounded by the presence of statistical heterogeneity. They often fail to detect inconsistency in the evidence network.

Moreover, network meta-analysis, like all other meta-analytic approaches, constitutes an observational study, and residual confounding can always be present. Systematic differences in characteristics among trials in a network can bias network meta-analysis results. In addition, all other considerations for meta-analyses, such as the choice of effect measures or heterogeneity, also apply to network meta-analysis. Therefore, in general, investigators should compare competing interventions based on direct evidence from head-to-head RCTs whenever possible. When head-to-head RCT data are sparse or unavailable but indirect evidence is sufficient, investigators may consider incorporating indirect evidence and network meta-analysis as an additional analytical tool. If the investigators choose to ignore indirect evidence, they should explain why.

Choice of Method

Although the development of network meta-analysis models has exploded in the last 10 years, there has been no systematic evaluation of their comparative performance, and the validity of the model assumptions in practice is generally hard to verify.

Investigators may choose a frequentist or Bayesian mode of inference based on the research team expertise, the complexity of the evidence network, and/or the research question. If investigators believe that the use of prior information is needed and that the data are insufficient to capture all the information available, then they should use a Bayesian model. On the other hand, a frequentist model is appropriate if one wants inferences to be based only on the data that can be incorporated into a likelihood.

Whichever method the investigators choose, they should assess the consistency of the direct and indirect evidence, and the invariance of treatment effects across studies and the appropriateness of the chosen method on a case-by-case basis, paying special attention to comparability across different sets of trials. Investigators should explicitly state assumptions underlying indirect comparisons and conduct sensitivity analysis to check those assumptions. If the results are not robust, findings from indirect comparisons should be considered inconclusive. Interpretation of findings should explicitly address these limitations. Investigators should also note that simple adjusted indirect comparisons are generally underpowered, needing four times as many equally sized studies to achieve the same power as direct comparisons, and frequently lead to indeterminate results with wide confidence intervals. 174 , 175

When the evidence of a network of interventions is consistent, investigators can combine direct and indirect evidence using network meta-analysis models. Conversely, they should refrain from combining multiple sources of evidence from an inconsistent (i.e., incoherent) network where there are substantial differences between direct and indirect evidence that cannot be resolved by conditioning on the known covariates. Investigators should make efforts to explain the differences between direct and indirect evidence based upon study characteristics, though little guidance and consensus exists on how to interpret the results.

Lastly, the network geometry ( Figure 5.1 ) can also affect the choice of analysis method as demonstrated in Table 5.1 .

Common network geometry (simple indirect comparison, star, network with at least one closed loop).

Impact of network geometry on choice of analysis method.

Commonly Used Software

Many statistical packages are available to implement NMA. BUGS software (Bayesian inference Using Gibbs Sampling, WINBUGS, OPENBUGS) is a popular choice for conducting Bayesian NMA 206 that offers flexible model specification including NMA meta-regression. JAGS and STAN are alternative choices for Bayesian NMA. Stata provides user-written routines ( http://www.mtm.uoi.gr/index.php/stata-routines-for-network-meta-analysis ) that can be used to conduct frequentist NMA. In particular, the Stata command network is a suite of programs for importing data for network meta-analysis, running a contrast-based network meta-analysis, assessing inconsistency, and graphing the data and results. Further, in the R environment, three packages, gemtc ( http://cran.r-project.org/web/packages/gemtc/index.html ), pcnetmeta ( http://cran.r-project.org/web/packages/pcnetmeta/index.html ), and netmeta ( http://cran.r-project.org/web/packages/netmeta/index.html ), have been developed for Bayesian ( gemtc, pcnetmeta ) or frequestist ( netmeta ) NMA. The packages also include methods to assess heterogeneity and inconsistency, and data visualizations, and allow users to perform NMA with minimal programming. 207

5.5. Inference From Network Meta-analysis

Stakeholders (users of evidence) require a rating of the strength of a body of evidence. The strength of evidence demonstrates how much certainty we should have in the estimates.

The general framework for assessing the strength of evidence used by the EPC program is described elsewhere. However; for NMA, guidance is evolving and may require some additional computations; therefore, we briefly discuss the possible approaches to rating the strength of evidence. We also discuss inference from rankings and probabilities commonly presented with a network meta-analysis.

Approaches for Rating the Strength of Evidence

The original EPC and GRADE guidance was simple and involved rating down all evidence derived from indirect comparisons (or NMA with mostly indirect evidence) for indirectness. Therefore, following this original GRADE guidance, evidence derived from most NMAs would be rated to have moderate strength at best. 208 Subsequently, Salanti et al. evaluated the transitivity assumption and network inconsistency under the indirectness and inconsistency domains of GRADE respectively. They judged the risk of bias based on a ‘contribution matrix’ which gives the percentage contribution of each direct estimate to each network meta-analysis estimate. 209 A final global judgment of the strength of evidence is made for the overall rankings in a network.

More recently, GRADE published a new approach that is based on evaluating the strength of evidence for each comparison separately rather than making a judgment on the whole network. 210 The rationale for not making such an overarching judgment is that the strength of evidence (certainty in the estimates) is expected to be different for different comparisons. The approach requires presenting the three estimates for each comparison (direct, indirect, and network estimates), then rating the strength of evidence separately for each one.

In summary, researchers conducting NMA should present their best judgment on the strength of evidence to facilitate decision-making. Innovations and newer methodology are constantly evolving in this area.

Interpreting Ranking Probabilities and Clinical Importance of Results

Network meta-analysis results are commonly presented as probabilities of being most effective and as rankings of treatments. Results are also presented as the surface under the cumulative ranking curve (SUCRA). SUCRA is a simple transformation of the mean rank that is used to provide a hierarchy of the treatments accounting both for the location and the variance of all relative treatment effects. SUCRA would be 1 when a treatment is certain to be the best and 0 when a treatment is certain to be the worst. 211 Such presentations should be interpreted with caution since they can be quite misleading.

• Such estimates are usually very imprecise. An empirical evaluation of 58 NMAs showed that the median width of the 95% CIs of SUCRA estimates was 65% (the first quartile was 38%; the third quartile was 80%). In 28% of networks, there was a 50% or greater probability that the best-ranked treatment was actually not the best. No evidence showed a difference between the best-ranked intervention and the second or third best-ranked interventions in 90% and 71% of comparisons, respectively.
• When rankings suggest superiority of an agent over others, the absolute difference between this intervention and other active agents could be trivial. Converting the relative effect to an absolute effect is often needed to present results that are meaningful to clinical practice and relevant to decision making. 212 Such results can be presented for patient groups with varying baseline risks. The source of baseline risk can be obtained from observational studies judged to be most representative of the population of interest, from the average baseline risk of the control arms of the randomized trials included in meta-analysis, or from a risk stratification tool if one is known and commonly used in practice. 213
• Rankings hide the fact that each comparison may have its own risk of bias, limitations, and strength of evidence.

5.6. Presentation and Reporting

• Rationale for conducting an NMA, the mode of inference (e.g., Bayesian, Frequentist), and the model choice (random effects vs. fixed effects; consistency vs inconsistency model, common heterogeneity assumption, etc.);
• Software and syntax/commands used;
• Choice of priors for any Bayesian analyses;
• Graphical presentation of the network structure and geometry;
• Pairwise effect sizes to allow comparative effectiveness inference; and
• Assessment of the extent of consistency between the direct and indirect estimates.
• A network meta-analysis should always be based on a rigorous a rigorous systematic review.
• Homogeneity of direct evidence
• Transitivity, similarity, or exchangeability
• Consistency (between direct and indirect evidence)
• Investigators may choose a frequentist or Bayesian mode of inference based on the research team’s expertise, the complexity of the evidence network, and the research question.
• Evaluating inconsistency is a major and mandatory component of network meta-analysis.
• Evaluating inconsistency should not be only based on a conducting a global test. A loop-based approach can identify the comparisons that cause inconsistency.
• Inference based on the rankings and probabilities of treatments being most effective should be used cautiously. Rankings and probabilities can be misleading and should be interpreted based on the magnitude of pairwise effect sizes. Differences across interventions may not be clinically important despite such rankings.
• Future Research Suggestions

The following are suggestions for directions in future research for each of the topics by chapter.

Chapter 1. Decision To Combine Trials

• Guidance regarding the minimum number of trials one can validly pool at given levels of statistical heterogeneity
• Research on ratio of means—both clinical interpretability and mathematical consistency across studies compared with standardized mean difference
• Research on use of ANCOVA models for adjusting baseline imbalance
• Software packages that more easily enable use of different information
• Methods to handle zeros in the computation of binary outcomes
• Evidence on which metrics, and language used to describe these metrics, are most helpful in conveying meta-analysis results to multiple stakeholders
• Evaluate newly developed statistical models for combining typical effect measures (e.g., mean difference, OR, RR, and/or RD) and compare with current methods
• Heterogeneity statistics for meta-analyses involving a small number of studies
• Guidance on specification of hypotheses in meta-regression
• Guidance on reporting of relationships among study outcomes to facilitate multivariate meta-analysis

Chapter 5. Network Meta-analysis (Mixed Treatment Comparisons/Indirect Comparisons)

• Methods for combining individual patient data with aggregated data
• Methods for integrating evidence from RCTs and observational studies
• Models for time-to-event data
• User friendly software similar to that available for traditional meta-analysis
• Evidence to support model choice

This report is based on research conducted by the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Centers’ 2016 Methods Workgroup. The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report.

This research was funded through contracts from the Agency for Healthcare Research and Quality to the following Evidence-based Practice Centers: Mayo Clinic (290-2015-00013-I); Kaiser Permanente (290-2015-00007-I); RAND Corporation (290-2015-00010-I); Alberta (290-2015-00001-I); Pacific Northwest (290-2015-00009-I); RTI (290-2015-00011-I); Brown (290-2015-00002-I); and the Scientific Resource Center (290-2012-00004-C).

The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policy makers, among others—make well-informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information (i.e., in the context of available resources and circumstances presented by individual patients).

This report is made available to the public under the terms of a licensing agreement between the author and the Agency for Healthcare Research and Quality. This report may be used and reprinted without permission except those copyrighted materials that are clearly noted in the report. Further reproduction of those copyrighted materials is prohibited without the express permission of copyright holders.

AHRQ or U.S. Department of Health and Human Services endorsement of any derivative products that may be developed from this report, such as clinical practice guidelines, other quality enhancement tools, or reimbursement or coverage policies may not be stated or implied.

Persons using assistive technology may not be able to fully access information in this report. For assistance, contact vog.shh.qrha@cpe .

Suggested citation: Morton SC, Murad MH, O’Connor E, Lee CS, Booth M, Vandermeer BW, Snowden JM, D’Anci KE, Fu R, Gartlehner G, Wang Z, Steele DW. Quantitative Synthesis—An Update. Methods Guide for Comparative Effectiveness Reviews. (Prepared by the Scientific Resource Center under Contract No. 290-2012-0004-C). AHRQ Publication No. 18-EHC007-EF. Rockville, MD: Agency for Healthcare Research and Quality; February 2018. Posted final reports are located on the Effective Health Care Program search page . https://doi.org/ 10 ​.23970/AHRQEPCMETHGUIDE3 .

Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services, 5600 Fishers Lane, Rockville, MD 20857, www.ahrq.gov Contract No.: 290-2012-00004-C . Prepared by: Scientific Resource Center, Portland, OR

• Cite this Page Morton SC, Murad MH, O’Connor E, et al. Quantitative Synthesis—An Update. 2018 Feb 23. In: Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008-.
• PDF version of this page (702K)

In this Page

• Decision to Combine Trials
• Optimizing Use of Effect Size Data
• Choice of Statistical Model for Combining Studies
• Quantifying, Testing, and Exploring Statistical Heterogeneity
• Network Meta-Analysis (Mixed Treatment Comparisons/Indirect Comparisons)

Other titles in these collections

• AHRQ Methods for Effective Health Care
• Health Services/Technology Assessment Text (HSTAT)

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Similar articles in PubMed

• Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas. [Cochrane Database Syst Rev. 2022] Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas. Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, et al. Cochrane Database Syst Rev. 2022 Feb 1; 2(2022). Epub 2022 Feb 1.
• Review Conducting Quantitative Synthesis When Comparing Medical Interventions: AHRQ and the Effective Health Care Program. [Methods Guide for Effectivenes...] Review Conducting Quantitative Synthesis When Comparing Medical Interventions: AHRQ and the Effective Health Care Program. Fu R, Gartlehner G, Grant M, Shamliyan T, Sedrakyan A, Wilt TJ, Griffith L, Oremus M, Raina P, Ismaila A, et al. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. 2008
• Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program. [J Clin Epidemiol. 2011] Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program. Fu R, Gartlehner G, Grant M, Shamliyan T, Sedrakyan A, Wilt TJ, Griffith L, Oremus M, Raina P, Ismaila A, et al. J Clin Epidemiol. 2011 Nov; 64(11):1187-97. Epub 2011 Apr 7.
• The future of Cochrane Neonatal. [Early Hum Dev. 2020] The future of Cochrane Neonatal. Soll RF, Ovelman C, McGuire W. Early Hum Dev. 2020 Nov; 150:105191. Epub 2020 Sep 12.
• Review Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. [Methods Guide for Effectivenes...] Review Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. Berkman ND, Lohr KN, Ansari M, McDonagh M, Balk E, Whitlock E, Reston J, Bass E, Butler M, Gartlehner G, et al. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. 2008

Recent Activity

• Quantitative Synthesis—An Update - Methods Guide for Effectiveness and Comparati... Quantitative Synthesis—An Update - Methods Guide for Effectiveness and Comparative Effectiveness Reviews

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Evidence Synthesis

• Identifying
• Collecting & Combining Data
• Explaining the Synthesis & Analysis
• Summarizing
• Types of Reviews
• Protocols & Registries
• Critical Appraisal
• Data Extraction
• Library Resources, Support, & Training

Evidence Synthesis Process: Explaining the Synthesis & Analysis

Helpful Tools

• Methods and Approaches for Synthesis and Analysis
• << Previous: Collecting & Combining Data
• Next: Summarizing >>
• Last Updated: Jul 31, 2024 9:25 AM
• URL: https://libguides.baylor.edu/evidencesynthesis

University Libraries

One Bear Place #97148 Waco, TX 76798-7148

(254) 710-6702

   Ask a Question

Copyright © Baylor® University . All rights reserved.

Report It | Title IX | Mental Health Resources | Anonymous Reporting | Legal Disclosures

Literature Reviews

• Introduction
• Tutorials and resources
• Step 1: Literature search
• Step 2: Analysis, synthesis, critique
• Step 3: Writing the review

If you need any assistance, please contact the library staff at the Georgia Tech Library Help website . 

Analysis, synthesis, critique

Literature reviews build a story. You are telling the story about what you are researching. Therefore, a literature review is a handy way to show that you know what you are talking about. To do this, here are a few important skills you will need.

Skill #1: Analysis

Analysis means that you have carefully read a wide range of the literature on your topic and have understood the main themes, and identified how the literature relates to your own topic. Carefully read and analyze the articles you find in your search, and take notes. Notice the main point of the article, the methodologies used, what conclusions are reached, and what the main themes are. Most bibliographic management tools have capability to keep notes on each article you find, tag them with keywords, and organize into groups.

Skill #2: Synthesis

After you’ve read the literature, you will start to see some themes and categories emerge, some research trends to emerge, to see where scholars agree or disagree, and how works in your chosen field or discipline are related. One way to keep track of this is by using a Synthesis Matrix .

Skill #3: Critique

As you are writing your literature review, you will want to apply a critical eye to the literature you have evaluated and synthesized. Consider the strong arguments you will make contrasted with the potential gaps in previous research. The words that you choose to report your critiques of the literature will be non-neutral. For instance, using a word like “attempted” suggests that a researcher tried something but was not successful. For example: 

There were some attempts by Smith (2012) and Jones (2013) to integrate a new methodology in this process.

On the other hand, using a word like “proved” or a phrase like “produced results” evokes a more positive argument. For example:

The new methodologies employed by Blake (2014) produced results that provided further evidence of X.

In your critique, you can point out where you believe there is room for more coverage in a topic, or further exploration in in a sub-topic.

Need more help?

If you are looking for more detailed guidance about writing your dissertation, please contact the folks in the Georgia Tech Communication Center .

• << Previous: Step 1: Literature search
• Next: Step 3: Writing the review >>
• Last Updated: Apr 2, 2024 11:21 AM
• URL: https://libguides.library.gatech.edu/litreview

Writing Resources

• Student Paper Template
• Grammar Guidelines
• Punctuation Guidelines
• Writing Guidelines
• Creating a Title
• Outlining and Annotating
• Using Generative AI (Chat GPT and others)
• Introduction, Thesis, and Conclusion
• Strategies for Citations
• Determining the Resource This link opens in a new window
• Citation Examples
• Citational Justice This link opens in a new window
• Paragraph Development
• Paraphrasing
• Inclusive Language
• International Center for Academic Integrity
• How to Synthesize and Analyze
• Synthesis and Analysis Practice
• Synthesis and Analysis Group Sessions
• Decoding the Assignment Prompt
• Annotated Bibliography
• Comparative Analysis
• Conducting an Interview
• Infographics
• Office Memo
• Policy Brief
• Poster Presentations
• PowerPoint Presentation
• White Paper
• Writing a Blog
• Research Writing: The 5 Step Approach
• Step 1: Seek Out Evidence
• Step 2: Explain
• Step 3: The Big Picture
• Step 4: Own It
• Step 5: Illustrate
• MLA Resources
• Time Management

ASC Chat Hours

ASC Chat is usually available at the following times ( Pacific Time):

If there is not a coach on duty, submit your question via one of the below methods:

  928-440-1325

  Ask a Coach

  [email protected]

Search our FAQs on the Academic Success Center's  Ask a Coach   page.

Learning about Synthesis Analysis

What D oes Synthesis and Analysis Mean?

Synthesis: the combination of ideas to

• show commonalities or patterns

Analysis: a detailed examination

• of elements, ideas, or the structure of something
• can be a basis for discussion or interpretation

Synthesis and Analysis: combine and examine ideas to

• show how commonalities, patterns, and elements fit together
• form a unified point for a theory, discussion, or interpretation
• develop an informed evaluation of the idea by presenting several different viewpoints and/or ideas

Key Resource: Synthesis Matrix

Synthesis Matrix

A synthesis matrix is an excellent tool to use to organize sources by theme and to be able to see the similarities and differences as well as any important patterns in the methodology and recommendations for future research. Using a synthesis matrix can assist you not only in synthesizing and analyzing,  but it can also aid you in finding a researchable problem and gaps in methodology and/or research.

Use the Synthesis Matrix Template attached below to organize your research by theme and look for patterns in your sources .Use the companion handout, "Types of Articles" to aid you in identifying the different article types for the sources you are using in your matrix. If you have any questions about how to use the synthesis matrix, sign up for the synthesis analysis group session to practice using them with Dr. Sara Northern!

Was this resource helpful?

• << Previous: International Center for Academic Integrity
• Next: How to Synthesize and Analyze >>
• Last Updated: Sep 26, 2024 11:11 AM
• URL: https://resources.nu.edu/writingresources

• Table of Contents
• Random Entry
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Advanced Tools
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Back to Entry
• Entry Contents
• Entry Bibliography
• Academic Tools
• Friends PDF Preview
• Author and Citation Info
• Back to Top

Supplement to Analysis

Definitions and descriptions of analysis.

The older a word, the deeper it reaches. (Wittgenstein NB , 40) { §6.5 }

This supplement collects together various definitions and descriptions of analysis that have been offered in the history of philosophy (including all the classic ones), to indicate the range of different conceptions and the issues that arise. (There are also some remarks on related topics such as analyticity, definition, and methodology more generally.) In most cases, abbreviated references are given; full details can be found in the Annotated Bibliography on Analysis, in the section mentioned in curly brackets after the relevant definition or description. Where there is more than one passage quoted from a particular author, passages are numbered in chronological order of composition (as far as that can be determined).

• Cambridge Dictionary of Philosophy , 1999, ed. Robert Audi

Concise Oxford Dictionary , 1976, ed. J. B. Sykes

• Dictionary of Philosophy and Psychology , 1925, ed. James Mark Baldwin

A Kant Dictionary , 1995, by Howard Caygill

Oxford dictionary of philosophy , 1996, by simon blackburn, philosophielexikon , 1997, ed. a. hügli and p. lübcke, routledge encyclopedia of philosophy , 1998, entry under ‘analytical philosophy’ by thomas baldwin, routledge encyclopedia of philosophy , 1998, entry under ‘conceptual analysis’ by robert hanna, alexander of aphrodisias, al-fārābī, abū naṣr muḥammad, arnauld, antoine and nicole, pierre, ayer, a. j., beaney, michael, bentham, jeremy, bergson, henri, bos, henk j. m., bradley, f. h., brandom, robert b., carnap, rudolf, cassirer, ernst, cohen, l. jonathan, collingwood, r. g., davidson, donald, de chardin, teilhard, derrida, jacques, descartes, rené.

• DeStaël, Germaine

Frege, Gottlob

Ganeri, jonardon, geertz, clifford, günderrode, karoline von, hegel, georg w.f., heidegger, martin, hobbes, thomas, hodges, wilfrid, holton, gerald, husserl, edmund, ibn rushd, abū al-walīd muḥammad ibn aḥmad, ibn sinān, ibrāhim, kant, immanuel, lakatos, imre, lambert, johann heinrich, leibniz, gottfried wilhelm, lichtenberg, georg christoph, locke, john, lodge, david, matilal, bimal krishna, mendelssohn, moses, moore, g. e., newton, isaac, nietzsche, friedrich, poincaré, jules henri, polya, george, quine, w.v.o., rorty, richard, rosen, stanley, russell, bertrand, ryle, gilbert, schiller, friedrich, sellars, wilfrid, soames, scott, staal, j. f., stebbing, l. susan.

• Strawson, F. Peter

Urmson, J. O.

Westerhoff, jan, whitehead, alfred north, wilson, john cook, wittgenstein, ludwig, 1. definitions of analysis, cambridge dictionary of philosophy , 2nd ed., 1999, ed. robert audi.

the process of breaking up a concept, proposition, linguistic complex, or fact into its simple or ultimate constituents. { §1.1 }

1. Resolution into simpler elements by analysing (opp. synthesis ); statement of result of this; … 2. (Math.) Use of algebra and calculus in problem-solving. { §1.1 }

Dictionary of Philosophy and Psychology , 1925, ed. James Mark Baldwin, Vol. I

The isolation of what is more elementary from what is more complex by whatever method. { §1.1 }

Kant combines two senses of analysis in his work, one derived from Greek geometry, the other from modern physics and chemistry. Both remain close to the original Greek sense of analysis as a ‘loosening up’ or ‘releasing’, but each proceed in different ways. The former proceeds ‘lemmatically’ by assuming a proposition to be true and searching for another known truth from which the proposition may be deduced. The latter proceeds by resolving complex wholes into their elements. { §4.5 }

The process of breaking a concept down into more simple parts, so that its logical structure is displayed. { §1.1 }

Auflösung, Zerlegung in Bestandteile, im Gegensatz zu Synthese. { §1.1 }

Philosophical analysis is a method of inquiry in which one seeks to assess complex systems of thought by ‘analysing’ them into simpler elements whose relationships are thereby brought into focus. { §1.1 }

The theory of conceptual analysis holds that concepts – general meanings of linguistic predicates – are the fundamental objects of philosophical inquiry, and that insights into conceptual contents are expressed in necessary ‘conceptual truths’ (analytic propositions). { §1.1 }

Annotated Bibliography, §1.1

2. Descriptions of Analysis

And he [Aristotle] called them Analytics because the resolution of every compound into those things out of which the synthesis [is made] is called analysis . For analysis is the converse of synthesis. Synthesis is the road from the principles to those things that derive from the principles, and analysis is the return from the end to the principles. For geometers are said to analyze when, beginning from the conclusion they go up to the principles and the problem, following the order of those things which were assumed for the demonstration of the conclusion {1}. But he also uses analysis who reduces composite bodies into simple bodies {2}, and he analyzes who divides the word into the parts of the word {3}; also he who divides the parts of the word into the syllables {4}; and he who divides these into their components {5}. And they are severally said to analyse who reduce compound syllogisms into simple ones {6}, and simple ones into the premisses out of which they get their being {7}. And further, resolving imperfect syllogisms into perfect ones is called analyzing {8}. And they call analysis the reducing of the given syllogism into the proper schemata {9}. And it is especially in this meaning of analysis that these are entitled Analytics , for he describes for us a method at the end of the first book with which we shall be able to do this. ( Commentary on Aristotle’s Prior Analytics , §1.2.1 (7, lines 11–33); tr. in Gilbert 1960, 32; the square brackets are in the original translation, the curly brackets have been added here to highlight the nine senses that Alexander distinguishes) { §2.4 , §3.2 }

• Transfer from the observed to the unobserved is of two kinds: one is by the method of Synthesis and the other is by the method of Analysis. With Analysis the reasoning starts with the unobserved, while with Synthesis it starts with the observed. If we want to use the method of Analysis to infer to the unobserved by means of the observed, we have to know the content that we are seeking [to transfer to] the unobserved thing, and then we study the question which are the sense-perceived things that satisfy that content. Then when we know something sense-perceived that satisfies that content, we use it to take those concepts that make the unobserved thing similar to the sense-perceived thing. Then we study the question which of those concepts is such that the whole of it satisfies the content that is observed in the sense-perceived thing. When we find such a concept, the content transfers necessarily from the thing observed by the senses to the unobserved thing. So therefore the inference to the unobserved by means of the observed, using this method, is in potential a question, i.e. a quaesitum , which a syllogism in the first figure is able to resolve. ( SY , 19.3; translation modified) { 3.3 }

it is not the same thing to take an argument in one’s hand and then to see and solve its faults, as it is to be able to meet it quickly while being subjected to questions; for what we know, we often do not know in a different context. Moreover, just as in other things speed or slowness is enhanced by training, so it is with arguments too, so that supposing we are unpractised, even though a point is clear to us, we are often too late for the right moment. Sometimes too it happens as with diagrams; for there we can sometimes analyse the figure, but not construct it again: so too in refutations, though we know on what the connexion of the argument depends, we still are at a loss to split the argument apart. ( SR , 16, 175a20–30) { §2.4 }

We must next explain how to reduce syllogisms to the figures previously described; this part of our inquiry still remains. For if we examine the means by which syllogisms are produced, and possess the ability to discover them, and can also analyse [ analuoimen ] the syllogisms when constructed into the figures previously described, our original undertaking will be completed. (( PrA , I, 32, 46b40–47a6; Tredennick tr. slightly modified) { §2.4 }

Thus it is evident (1) that the types of syllogism which cannot be analysed in these figures [viz., second figure syllogisms into the third figure, and third figure syllogisms into the second figure] are the same as those which we saw could not be analysed into the first figure; and (2) that when syllogisms are reduced to the first figure these alone are established per impossibile .

It is evident, then, from the foregoing account [taken as including the discussion prior to chapter 45] how syllogisms should be reduced; and also that the figures can be analysed into one another. ( PrA , I, 45, 51a40–b5; Tredennick tr., substituting ‘analysed’ for ‘resolved’) { §2.4 }

If it were impossible to prove truth from falsehood, it would be easy to make analyses [ analuein ]; for then the propositions would convert from necessity. Let A be something that is the case; and if A is the case, then these things are the case (things which I know to be the case—call them B ). From the latter, then, I shall prove that the former is the case. (In mathematics conversion is more common because mathematicians assume nothing incidental—and in this too they differ from those who argue dialectically—but only definitions.) ( PoA , I, 12, 78a6–13) { §2.4 }

We deliberate not about ends but about means. For a doctor does not deliberate whether he shall heal, nor an orator whether he shall convince, nor a statesman whether he shall produce law and order, nor does any one else deliberate about his end. Having set the end, they consider how and by what means it is to be attained; and if it seems to be produced by several means they consider by which it is most easily and best produced, while if it is achieved by one only they consider how it will be achieved by this and by what means this will be achieved, till they come to the first cause, which in the order of discovery is last. For the person who deliberates seems to inquire and analyse in the way described as though he were analysing a geometrical construction (not all inquiry appears to be deliberation—for instance mathematical inquiries—but all deliberation is inquiry), and what is last in the order of analysis seems to be first in the order of becoming. And if we come on an impossibility, we give up the search, e.g. if we need money and this cannot be got; but if a thing appears possible we try to do it. ( NE , III, 3, 1112b8–27) { §2.4 }

The art of arranging a series of thoughts properly, either for discovering the truth when we do not know it, or for proving to others what we already know, can generally be called method.

Hence there are two kinds of method, one for discovering the truth, which is known as analysis , or the method of resolution , and which can also be called the method of discovery . The other is for making the truth understood by others once it is found. This is known as synthesis , or the method of composition , and can also be called the method of instruction .

Analysis does not usually deal with the entire body of a science, but is used only for resolving some issue. ( LAT , 233–4) { §4.1 }

Now analysis consists primarily in paying attention to what is known in the issue we want to resolve. The entire art is to derive from this examination many truths that can lead us to the knowledge we are seeking.

Suppose we wondered whether the human soul is immortal, and to investigate it we set out to consider the nature of the soul. First we would notice that it is distinctive of the soul to think, and that it could doubt everything without being able to doubt whether it is thinking, since doubting is itself a thought. Next we would ask what thinking is. Since we would see nothing contained in the idea of thought that is contained in the idea of the extended substance called body, and since we could even deny of thought everything belonging to body — such as having length, width, and depth, having different parts, having a certain shape, being divisible, etc. — without thereby destroying the idea we have of thought, from this we would conclude that thought is not at all a mode of extended substance, because it is the nature of a mode not to be able to be conceived while the thing of which it is a mode is denied. From this we infer, in addition, that since thought is not a mode of extended substance, it must be the attribute of another substance. Hence thinking substance and extended substance are two really distinct substances. It follows from this that the destruction of one in no way brings about the destruction of the other, since even extended substance is not properly speaking destroyed, but all that happens in what we call destruction is nothing more than the change or dissolution of several parts of matter which exist forever in nature. Likewise it is quite easy to judge that in breaking all the gears of a clock no substance is destroyed, although we say that the clock is destroyed. This shows that since the soul is in no way divisible or composed of parts, it cannot perish, and consequently is immortal.

This is what we call analysis or resolution . We should notice, first, that in this method — as in the one called composition — we should practice proceeding from what is better known to what is less known. For there is no true method which could dispense with this rule.

Second, it nevertheless differs from the method of composition in that these known truths are taken from a particular examination of the thing we are investigating, and not from more general things as is done in the method of instruction. Thus in the example we presented, we did not begin by establishing these general maxims: that no substance perishes, properly speaking; that what is called destruction is only a dissolution of parts; that therefore what has no parts cannot be destroyed, etc. Instead we rose by stages to these general notions.

Third, in analysis we introduce clear and evident maxims only to the extent that we need them, whereas in the other method we establish them first, as we will explain below.

Fourth and finally, these two methods differ only as the route one takes in climbing a mountain from a valley differs from the route taken in descending from the mountain into the valley, or as the two ways differ that are used to prove that a person is descended from St. Louis. One way is to show that this person had a certain man for a father who was the son of a certain man, and that man was the son of another, and so on up to St. Louis. The other way is to begin with St. Louis and show that he had a certain child, and this child had others, thereby descending to the person in question. This example is all the more appropriate in this case, since it is certain that to trace an unknown genealogy, it is necessary to go from the son to the father, whereas to explain it after finding it, the most common method is to begin with the trunk to show the descendants. This is also what is usually done in the sciences where, after analysis is used to find some truth, the other method is employed to explain what has been found.

This is the way to understand the nature of analysis as used by geometers. Here is what it consists in. Suppose a question is presented to them, such as whether it is true or false that something is a theorem, or whether a problem is possible or impossible; they assume what is at issue and examine what follows from that assumption. If in this examination they arrive at some clear truth from which the assumption follows necessarily, they conclude that the assumption is true. Then starting over from the end point, they demonstrate it by the other method which is called composition . But if they fall into some absurdity or impossibility as a necessary consequence of their assumption, they conclude from this that the assumption is false and impossible.

This is what may be said in a general way about analysis, which consists more in judgment and mental skill than in particular rules. ( LAT , 236–8) { §4.1 }

It is advisable to stress the point that philosophy, as we understand it, is wholly independent of metaphysics, inasmuch as the analytic method is commonly supposed by its critics to have a metaphysical basis. Being misled by the associations of the word ‘analysis’, they assume that philosophical analysis is an activity of dissection; that it consists in ‘breaking up’ objects into their constituent parts, until the whole universe is ultimately exhibited as an aggregate of ‘bare particulars’, united by external relations. If this were really so, the most effective way of attacking the method would be to show that its basic presupposition was nonsensical. For to say that the universe was an aggregate of bare particulars would be as senseless as to say that it was Fire or Water or Experience. It is plain that no such possible observation would enable to verify such an assertion. But, so far as I know, this line of criticism is in fact never adopted. The critics content themselves with pointing out that few, if any, of the complex objects in the world are simply the sum of their parts. They have a structure, an organic unity, which distinguishes them, as genuine wholes, from mere aggregates. But the analyst, so it is said, is obliged by his atomistic metaphysics to regard an object consisting of parts a , b , c , and d , in a distinctive configuration as being simply a + b + c + d , and thus gives an entirely false account of its nature.

If we follow the Gestalt psychologists, who of all men talk most constantly about genuine wholes, in defining such a whole as one in which the properties of every part depend to some extent on its position in the whole, then we may accept it as an empirical fact that there exist genuine, or organic, wholes. And if the analytic method involved a denial of this fact, it would indeed be a faulty method. But, actually, the validity of the analytic method is not dependent on any empirical, much less any metaphysical, presupposition about the nature of things. For the philosopher, as an analyst, is not directly concerned with the physical properties of things. He is concerned only with the way in which we speak about them.

In other words, the propositions of philosophy are not factual, but linguistic in character – that is, they do not describe the behaviour of physical, or even mental, objects; they express definitions, or the formal consequences of definitions. Accordingly, we may say that philosophy is a department of logic. For we shall see that the characteristic mark of a purely logical inquiry is that it is concerned with the formal consequences of our definitions and not with questions of empirical fact.

It follows that philosophy does not in any way compete with science. The difference in type between philosophical and scientific propositions is such that they cannot conceivably contradict one another. And this makes it clear that the possibility of philosophical analysis is independent of any empirical assumptions. That it is independent of any metaphysical assumptions should be even more obvious still. For it is absurd to suppose that the provision of definitions, and the study of their formal consequences, involves the nonsensical assertion that the world is composed of bare particulars, or any other metaphysical dogma.

What has contributed as much as anything to the prevalent misunderstanding of the nature of philosophical analysis is the fact that propositions and questions which are really linguistic are often expressed in such a way that they appear to be factual. A striking instance of this is provided by the proposition that a material thing cannot be in two places at once. This looks like an empirical proposition, and is constantly invoked by those who desire to prove that it is possible for an empirical proposition to be logically certain. But a more critical inspection shows that it is not empirical at all, but linguistic. It simply records the fact that, as the result of certain verbal conventions, the proposition that two sense-contents occur in the same visual or tactual sense-field is incompatible with the proposition that they belong to the same material thing. And this is indeed a necessary fact. But it has not the least tendency to show that we have certain knowledge about the empirical properties of objects. For it is necessary only because we happen to use the relevant words in a particular way. There is no logical reason why we should not so alter our definitions that the sentence ‘A thing cannot be in two places at once’ comes to express a self-contradiction instead of a necessary truth. (1936, 75–7) { §6.7 }

From our assertion that philosophy provides definitions, it must not be inferred that it is the function of the philosopher to compile a dictionary, in the ordinary sense. For the definitions which philosophy is required to provide are of a different kind from those which we expect to find in dictionaries. In a dictionary we look mainly for what may be called explicit definitions; in philosophy, for definitions in use . ...

We define a symbol in use , not by saying that it is synonymous with some other symbol, but by showing how the sentences in which it significantly occurs can be translated into equivalent sentences, which contain neither the definiendum itself, nor any of its synonyms. A good illustration of this process is provided by Bertrand Russell’s so-called theory of descriptions, which is not a theory at all in the ordinary sense, but an indication of the way in which all phrases of the form ‘the so-and-so’ are to be defined. ( Ibid ., 80–1) { §6.7 }

[A serious mistake in my account in Language, Truth and Logic ] was my assumption that philosophical analysis consisted mainly in the provision of ‘definitions in use’. It is, indeed, true that what I describe as philosophical analysis is very largely a matter of exhibiting the inter-relationship of different types of propositions; but the cases in which this process actually yields a set of definitions are the exception rather than the rule. ...

... Thus, when Professor Moore suggests that to say that ‘existence is not a predicate’ may be a way of saying that ‘there is some very important difference between the way in which “exist” is used in such a sentence as “Tame tigers exist” and the way in which “growl” is used in “Tame tigers growl”’, he does not develop his point by giving rules for the translation of one set of sentences into another. What he does is to remark that whereas it makes good sense to say ‘All tame tigers growl’ or ‘Most tame tigers growl’ it would be nonsense to say ‘All tame tigers exist’ or ‘Most tame tigers exist’. Now this may seem a rather trivial point for him to make, but in fact it is philosophically illuminating. For it is precisely the assumption that existence is a predicate that gives plausibility to ‘the ontological argument’; and the ontological argument is supposed to demonstrate the existence of a God. Consequently Moore by pointing out a peculiarity in the use of the word ‘exist’ helps to protect us from a serious fallacy; so that his procedure, though different from that which Russell follows in his theory of descriptions, tends to achieve the same philosophical end. (1946, 31–3) { §6.7 }

In analytic geometry, the geometrical problems are solved by ‘translating’ them into the language of arithmetic and algebra. And here we can also see how ‘interpretive’ analysis plays a role. Lines, circles, curves, and so on, must first be ‘interpreted’ as equations, and the geometrical problems correspondingly reformulated, before arithmetic and algebra can be applied in solving them. The idea here can be generalized: problems need to be interpreted in some form before the resources of a relevant theory or conceptual framework can be brought to bear. And this is exactly what is involved in analytic philosophy: the propositions to be analysed—those that give rise to the philosophical problems to be solved or dissolved—need to be rephrased in a richer conceptual framework or formalized in an appropriate logical theory. Analytic philosophy, then, is ‘analytic’ much more in the sense that analytic geometry is analytic than in any crude decompositional sense. (2017, 96.)

By the word paraphrasis may be designated that sort of exposition which may be afforded by transmuting into a proposition, having for its subject some real entity, a proposition which has not for its subject any other than a fictitious entity. ( EL , 246) { §5.6 }

By intuition is meant the kind of intellectual sympathy by which one places oneself within an object in order to coincide with what is unique in it and consequently inexpressible. Analysis, on the contrary, is the operation which reduces the object to elements already known, that is, to elements common both to it and other objects. To analyse, therefore, is to express a thing as a function of something other than itself. All analysis is thus a translation, a development into symbols, a representation taken from successive points of view from which we note as many resemblances as possible between the new object which we are studying and others which we believe we know already. In its eternally unsatisfied desire to embrace the object around which it is compelled to turn, analysis multiplies without end the number of its points of view in order to complete its always incomplete representation, and ceaselessly varies its symbols that it may perfect the always imperfect translation. It goes on, therefore, to infinity. But intuition, if intuition is possible, is a simple act. (1903, 6–7) { §5.1 }

[Analysis] operates always on the immobile, whilst intuition places itself in mobility, or, what comes to the same thing, in duration. There lies the very distinct line of demarcation between intuition and analysis. The real, the experienced and the concrete are recognised by the fact that they are variability itself, the element by the fact that it is invariable. And the element is invariable by definition, being a diagram, a simplified reconstruction, often a mere symbol, in any case a motionless view of the moving reality. (1903, 40–1) { §5.1 }

Modern science is neither one nor simple. It rests, I freely admit, on ideas which in the end we find clear; but these ideas have gradually become clear through the use made of them; they owe most of their clearness to the light which the facts, and the applications to which they led, have by reflection shed on them — the clearness of a concept being scarcely anything more at bottom than the certainty, at last obtained, of manipulating the concept profitably. At its origin, more than one of these concepts must have appeared obscure, not easily reconcilable with the concepts already admitted into science, and indeed very near the borderline of absurdity. This means that science does not proceed by an orderly dovetailing together of concepts predestined to fit each other exactly. True and fruitful ideas are so many close contacts with currents of reality, which do not necessarily converge on the same point. However the concepts in which they lodge themselves manage somehow, by rubbing off each other's corners, to settle down well enough together. (1903, 74) { §5.1 }

It may help to be reminded that many philosophers who might allow themselves to be described as “analysts” have been strongly influenced by the work of Russell, Moore, and Wittgenstein. For while all three have been engaged in “clarification of meaning” they have done so in different and distinctive ways; and the resulting divergences in conceptions of philosophical method have not yet been reconciled. This makes it hard to give any simple account of what is meant today by “philosophical analysis”. (1950a, 2) { §6.1 }

A man who had to describe “philosophical analysis” might resort to talking about a climate of opinion. The weather, he might say, is congenial to empiricists, naturalists, agnostics; the well acclimatized have admired the two Principia’s and the Tractatus and have read a hundred pages of Hume for one of Kant. Here rhetoric is viewed with suspicion and enthusiasm barely tolerated; this is a land of “prose writers, hoping to be understood” [J. M. Keynes, A Treatise on Probability , 1921, preface].

... If a formula or a slogan is wanted, it is easy enough to say that these writers (like Russell, Moore, and Wittgenstein before them) are engaged in clarification of meaning . ... And if those who are best at the work of clarification might feel embarrassed to provide a satisfactory analysis of “analysis”, that is perhaps no cause for apology or alarm. For it is a mark of life to resist arbitrary confinement, and “philosohical analysis” is still much alive. (1950a, 12–13) { §6.1 }

Analysis comprises mathematical methods for finding the solutions (in geometry: the constructions) of problems or the proofs of theorems, doing so by introducing unknowns. (2001, 129) { §4.2 }

It is a very common and most ruinous superstition to suppose that analysis is no alteration, and that, whenever we distinguish, we have at once to do with divisible existence. It is an immense assumption to conclude, when a fact comes to us as a whole, that some parts of it may exist without any sort of regard for the rest. Such naive assurance of the outward reality of all mental distinctions, such touching confidence in the crudest identity of thought and existence, is worthy of the school which so loudly appeals to the name of Experience. ... If it is true in any sense (and I will not deny it) that thought in the end is the measure of things, yet at least this is false, that the divisions we make within a whole all answer to elements whose existence does not depend on the rest. It is wholly unjustifiable to take up a complex, to do any work we please upon it by analysis, and then simply predicate as an adjective of the given these results of our abstraction. These products were never there as such, and in saying, as we do, that as such they are there, we falsify the fact. You can not always apply in actual experience that coarse notion of the whole as the sum of its parts into which the school of ‘experience’ so delights to torture phenomena. If it is wrong in physiology to predicate the results, that are reached by dissection, simply and as such of the living body, it is here infinitely more wrong. The whole that is given to us is a continuous mass of perception and feeling; and to say of this whole, that any one element would be what it is there, when apart from the rest, is a very grave assertion. We might have supposed it not quite self-evident, and that it was possible to deny it without open absurdity. ( PL , §64/ WLM , 77–8) { §5.6 }

judgement is the differentiation of a complex whole, and hence always is analysis and synthesis in one. ( AR , 149/ WLM , 158) { §5.6 }

At any moment my actual experience, however relational its contents, is in the end non-relational. No analysis into relations and terms can ever exhaust its nature or fail in the end to belie its essence. What analysis leaves for ever outstanding is no mere residue, but is a vital condition of the analysis itself. Everything which is got out into the form of an object implies still the felt background against which the object comes, and, further, the whole experience of both feeling and object is a non-relational immediate felt unity. The entire relational consciousness, in short, is experienced as falling within a direct awareness. This direct awareness is itself non-relational. It escapes from all attempts to exhibit it by analysis as one or more elements in a relational scheme, or as that scheme itself, or as a relation or relations, or as the sum or collection of any of these abstractions. And immediate experience not only escapes, but it serves as the basis on which the analysis is made. Itself is the vital element within which every analysis still moves, while, and so far as, and however much, that analysis transcends immediacy. ( ETR , 176/ WLM , 280–1) { §5.6 }

I would rather now lay more stress on the logical vice of all Analysis and Abstraction – so far as that means taking any feature in the Whole of Things as ultimately real except in its union with the Whole. ( Collected Works of F.H. Bradley: Selected Correspondence 1905–1924 , Bristol, Thoemmes Press, 1999, 275)

Analysis and synthesis I take in the end to be two aspects of one principle … Every analysis proceeds from and on the basis of a unity ... The point before us is the question as to how, without separation in its existence, we can discriminate ideally in analysis. ( ETR , 300)

Socratic method is a way of bringing our practices under rational control by expressing them explicitly in a form in which they can be confronted with objections and alternatives, a form in which they can be exhibited as the conclusions of inferences seeking to justify them on the basis of premises advanced as reasons, and as premises in further inferences exploring the consequences of accepting them. (2000, 56) { §6.9 }

I think of analytic philosophy as having at its center a concern with semantic relations between what I will call ‘vocabularies’. … Its characteristic form of question is whether and in what way one can make sense of the meanings expressed by one kind of locution in terms of the meanings expressed by another kind of locution. So, for instance, two early paradigmatic projects were to show that everything expressible in the vocabulary of number-theory, and again, everything expressible using definite descriptions, is expressible already in the vocabulary of first-order quantificational logic with identity.

The nature of the key kind of semantic relation between vocabularies has been variously characterized during the history of analytic philosophy: as analysis, definition, paraphrase, translation, reduction of different sorts, truth-making, and various kinds of supervenience—to name just a few contenders. In each case, however, it is characteristic of classical analytic philosophy that logical vocabulary is accorded a privileged role in specifying these semantic relations. It has always been taken at least to be licit to appeal to logical vocabulary in elaborating the relation between analysandum and analysans —target vocabulary and base vocabulary—and, according to stronger versions of this thesis, that may be the only vocabulary it is licit to employ in that capacity. I will refer to this aspect of the analytic project as its commitment to ‘ semantic logicism ’. (2006, Lecture One, §1) { §6.9 }

What I want to call the “classical project of analysis”, then, aims to exhibit the meanings expressed by various target vocabularies as intelligible by means of the logical elaboration of the meanings expressed by base vocabularies thought to be privileged in some important respects—epistemological, ontological, or semantic—relative to those others. This enterprise is visible in its purest form in what I have called the “core programs” of empiricism and naturalism, in their various forms. In my view the most significant conceptual development in this tradition—the biggest thing that ever happened to it—is the pragmatist challenge to it that was mounted during the middle years of the twentieth century. Generically, this movement of thought amounts to a displacement from the center of philosophical attention of the notion of meaning in favor of that of use : in suitably broad senses of those terms, replacing concern with semantics by concern with pragmatics . ( Ibid ., Lecture One, §2) { §6.9 }

the analysis or, more precisely, quasi-analysis of an entity that is essentially an indivisible unit into several quasi-constituents means placing the entity in several kinship contexts on the basis of a kinship relation, where the unit remains undivided. (1928a, §71; English tr. by Rolf A. George slightly altered) { §6.7 }

The logical analysis of a particular expression consists in the setting-up of a linguistic system and the placing of that expression in this system. (1936, 143) { §6.7 }

That part of the work of philosophers which may be held to be scientific in its nature—excluding the empirical questions which can be referred to empirical science—consists of logical analysis. The aim of logical syntax is to provide a system of concepts, a language, by the help of which the results of logical analysis will be exactly formulable. Philosophy is to be replaced by the logic of science —that is to say, by the logical analysis of the concepts and sentences of the sciences, for the logic of science is nothing other than the logical syntax of the language of science . (1937, xiii) { §6.7 }

The task of making more exact a vague or not quite exact concept used in everyday life or in an earlier stage of scientific or logical development, or rather of replacing it by a newly constructed, more exact concept, belongs among the most important tasks of logical analysis and logical construction. We call this the task of explicating, or of giving an explication for, the earlier concept … (1947, 8–9) { §6.7 }

By the procedure of explication we mean the transformation of an inexact, prescientific concept, the explicandum , into a new exact concept, the explicatum . Although the explicandum cannot be given in exact terms, it should be made as clear as possible by informal explanations and examples. ...

The term ‘explicatum’ has been suggested by the following two usages. Kant calls a judgement explicative if the predicate is obtained by analysis of the subject. Husserl, in speaking about the synthesis of identification between a confused, nonarticulated sense and a subsequently intended distinct, articulated sense, calls the latter the ‘Explikat’ of the former. (For both uses see Dictionary of philosophy [1942], ed. D. Runes, p. 105). What I mean by ‘explicandum’ and ‘explicatum’ is to some extent similar to what C.H. Langford calls ‘analysandum’ and ‘analysans’: “the analysis then states an appropriate relation of equivalence between the analysandum and the analysans” [Langford 1942, 323 { §6.4 }]; he says that the motive of an analysis “is usually that of supplanting a relatively vague idea by a more precise one” ( ibid ., p. 329).

(Perhaps the form ‘explicans’ might be considered instead of ‘explicatum’; however, I think that the analogy with the terms ‘definiendum’ and ‘definiens’ would not be useful because, if the explication consists in giving an explicit definition, then both the definiens and the definiendum in this definition express the explicatum, while the explicandum does not occur.) The procedure of explication is here understood in a wider sense than the procedures of analysis and clarification which Kant, Husserl, and Langford have in mind. The explicatum (in my sense) is in many cases the result of analysis of the explicandum (and this has motivated my choice of the terms); in other cases, however, it deviates deliberately from the explicandum but still takes its place in some way; this will become clear by the subsequent examples. (1950, 3) { §6.7 }

[T]he sense of all objective judgments reduces to a final original relation, which can be expressed in different formulations as the relation of “form” to “content”, as the relation of “universal” to “particular”, as the relation of “validity [ Geltung ]” to “being [ Sein ]”. Whatever designation one may finally choose here, what is alone decisive is that the basic relation itself is to be retained as a strictly unitary relation, which can only be designated through the two opposed moments that enter into it – but never constructed out of them, as if they were independent constituents present in themselves. The original relation is not to be defined in such a way that the “universal” somehow “subsists” next to or above the “particular” – the form somehow separate from the content – so that the two are then melded with one another by means of some or another fundamental synthesis of knowledge. Rather, the unity of mutual determination constitutes the absolutely first datum, behind which one can go back no further, and which can only be analyzed via the duality of two “viewpoints” in an artificially isolating process of abstraction. It is the basic flaw of all metaphysical epistemologies that they always attempt to reinterpret this duality of “moments” as a duality of “elements”. (1913, 13–14; cited and tr. by Friedman 2000, 34) { §5.4 }

conceptual analysis typically relates one kind of reason for using a certain word to another. (1986, 51) { §6.9 }

When philosophical analysis proceeds from intuitively sanctioned premisses to a reasoned conclusion, it may be described as moving from analysandum to analysans. It seeks to ensure that any muddles or inconsistencies in our unreasoned inclinations and passive prejudices are replaced by an explicitly formulated, consciously co-ordinated, adequately reasoned, and freely adopted system of acceptable principles. (1986, 96) { §6.9 }

Socrates was essentially the inventor of a method. ... His revolt against the study of nature was essentially a revolt against observation in favour of thought; and whereas mathematical method, as an example of thought, had already been discovered by his predecessors, his own discovery was that a similar method, for which he invented an appropriate technique, could be applied to ethical questions. This technique, as he himself recognized, depended on a principle which is of great importance to any theory of philosophical method: the principle that in a philosophical inquiry what we are trying to do is not to discover something of which until now we have been ignorant, but to know better something which in some sense we knew already; not to know it better in the sense of coming to know more about it, but to know it better in the sense of coming to know it in a different and better way—actually instead of potentially, or explicitly instead of implicitly, or in whatever terms the theory of knowledge chooses to express the difference: the difference itself has been a familiar fact ever since Socrates pointed it out. (1933, 10–11) { §5.6 }

[The] work of disentangling and arranging questions, which ... I [call] analysis, may be alternatively described as the work of detecting presuppositions. ... The analysis which detects absolute presuppositions I call metaphysical analysis; but as regards procedure and the qualifications necessary to carry it out there is no difference whatever between metaphysical analysis and analysis pure and simple ... (1940, 39–40) { §5.6 }

It is only by analysis that any one can ever come to know either that he is making any absolute presuppositions at all or what absolute presuppositions he is making.

Such analysis may in certain cases proceed in the following manner. If the inquirer can find a person to experiment upon who is well trained in a certain type of scientific work, intelligent and earnest in his devotion to it, and unaccustomed to metaphysics, let him probe into various presuppositions that his ‘subject’ has been taught to make in the course of his scientific education, and invite him to justify each or alternatively to abandon it. If the ‘inquirer’ is skilful and the ‘subject’ the right kind of man, these invitations will be contemplated with equanimity, and even with interest, so long as relative presuppositions are concerned. But when an absolute presupposition is touched, the invitation wil be rejected, even with a certain degree of violence.

The rejection is a symptom that the ‘subject’, co-operating with the work of analysis, has come to see that the presupposition he is being asked to justify or abandon is an absolute presupposition; and the violence with which it is expressed is a symptom that he feels the importance of this absolute presupposition for the kind of work to which he is devoted. This is what ... I called being ‘ticklish in one’s absolute presuppositions’; and the reader will see that this ticklishness is a sign of intellectual health combined with a low degree of analytical skill. A man who is ticklish in that way is a man who knows, ‘instinctively’ as they say, that absolute presuppositions do not need justification. ( Ibid. , 43–4) { §5.6 }

metaphysical analysis, the discovery that certain presuppositions actually made are absolute presuppositions, is an integral part or an indispensable condition, you can put it whichever way you like, of all scientific work.( Ibid. , 84) { §5.6 }

“Pattern” [ lǐ ] is a term that makes reference to the close examination of things for the subtle and minute characteristics that should be distinguished in order to separate things. This is why it is called “the Pattern for separating things” [ fēnlǐ 分理]. When applied to concrete materials, it appears in the expressions “the pattern of the folds in the skin” [ jīlǐ 理], “the pattern of capillary pores” [ còulǐ 腠理] and “refined patterns” [ wénlǐ  文理].

When things are successfully separated so that the individual strands [ tiáo 條] are not intertwined, this is called “Well Ordered”[ tiáo lǐ  條理]. Mengzi declared that “Kongzi is [like] a complete orchestra”, explaining, “it is the work of wisdom to begin [a concert] in Good Order and is the work of the sage to maintain that Good Order through to the end”. To account for the supreme sageliness and wisdom of Kongzi, this description simply holds up his [capacity for] Good Order. ( An Evidential Commentary on the Meanings of Terms in the Mengzi , Section 1; tr. Justin Tiwald, in Tiwald and Van Norden 2014, 319–20) { §4.7 }

The word ‘principle’ [ lǐ 理] is a name assigned to the arrangement of the parts of anything which gives the whole its distinctive property or characteristic, and which can be observed by careful examination and analysis of the parts down to the minutest detail. This is why we speak of the principle of differentiation ( fen-li ). With reference to the substance of things, there are such expressions as the principle governing the fibres ( ji-li ), the principle governing the arrangement between skin and flesh ( cou-li ), and pattern ( wen-li ). ... When proper differentiation is made, there will be order without confusion. This is called ‘order and arrangement’ ( tiao-li ). ( An Evidential Commentary on the Meanings of Terms in the Mengzi , Section 1; tr. in Chin and Freeman 1990, 69; cited in Cheng Chung-yi 2009, 461) { §4.7 }

It has been said that there is Han classical learning and there is Song classical learning: the former emphasizes the ancient glosses ( gu-xun ) and the latter is concerned with [understanding] the reason and meaning [of things] ( yi-li ). I am greatly puzzled by this statement. If one can understand the reason and meaning [of things] by sheer speculation, then anyone can grab them out of emptiness. If that is so, what can we hope to gain from classical learning? It is precisely because sheer speculation cannot lead us to the reason and meaning [of things] as intended by the sages and worthies that one has to seek it from the ancient Classics. When seeking from the ancient Classics, we are facing the huge distance between the ancient and the present that lies in the texts, and then we have to resort to the ancient glosses [so as to fill the distance up]. Only when the ancient glosses are clear can the Classics be understood, and only when the Classics are understood can the reason and meaning [of things] as intended by the sages and worthies be grasped. ( Works of Dai Zhen , 1980, 168; tr. in Chin and Freeman 1990, 12; cited (modified) in Cheng 2009, 460) {§4.7}

In philosophy we are used to definitions, analyses, reductions. Typically these are intended to carry us from concepts better understood, or clear, or more basic epistemologically or ontologically, to others we want to understand. The method I have suggested fits none of these categories. I have proposed a looser relation between concepts to be illuminated and the relatively more basic. (‘Radical Interpretation’, 1972, Inquiries into Truth and Interpretation , Oxford: Oxford University Press, 2001, 137)

Unlike the primitives who gave a face to every moving thing, or the early Greeks who defined all the aspects and forces of nature, modern man is obsessed by the need to depersonalise (or impersonalise) all that he most admires. There are two reasons for this tendency. The first is analysis , that marvellous instrument of scientific research to which we owe all our advances but which, breaking down synthesis after synthesis, allows one soul after another to escape, leaving us confronted with a pile of dismantled machinery, and evanescent particles. The second reason lies in the discovery of the sidereal world, so vast that it seems to do away with all proportion between our own being and the dimensions of the cosmos around us. ( The Phenomenon of Man , 1955, 282; tr. Bernard Wall, Fontana, 1965; tr. first publ. 1959)

Up until now the idea of philosophy remained defined in a formal way as an idea of an infinite task theoria . Could a history of this infinite theoretical life, which merges itself in its efforts and failures with a simple realization of the self , take on the value of a genetic description? Will the history of the “transcendental motive” through all the stages of European philosophy, enlighten us at last on the genesis of transcendental subjectivity? But such a history presupposes the possibility of such a going backward, the possibility of finding again the originary sense of the former presents as such. It implies the possibility of a transcendental “regression” ( Ruckfrage ) through a history that is intelligible and transparent to consciousness, a history whose sedimentations can be unmade and remade without alteration. ( The Problem of Genesis in Husserl's Philosophy , The University of Chicago Press, 2003, 161; tr. Marian Hobson)

[discussing his ‘Rule Four’: “ We need a method if we are to investigate the truth of things ”] … the human mind has within it a sort of spark of the divine, in which the first seeds of useful ways of thinking are sown, seeds which, however neglected and stifled by studies which impede them, often bear fruit of their own accord. This is our experience in the simplest of sciences, arithmetic and geometry: we are well aware that the geometers of antiquity employed a sort of analysis which they went on to apply to the solution of every problem, though they begrudged revealing it to posterity. At the present time a sort of arithmetic called ‘algebra’ is flourishing, and this is achieving for numbers what the ancients did for figures. ( Rules for the Direction of the Mind , in PW , I, 16–17) { §4.2 }

As for the method of demonstration, this divides into two varieties: the first proceeds by analysis and the second by synthesis.

Analysis shows the true way by means of which the thing in question was discovered methodically and as it were a priori , so that if the reader is willing to follow it and give sufficient attention to all points, he will make the thing his own and understand it just as perfectly as if he had discovered it for himself. But this method contains nothing to compel belief in an argumentative or inattentive reader; for if he fails to attend even to the smallest point, he will not see the necessity of the conclusion. Moreover there are many truths which — although it is vital to be aware of them — this method often scarcely mentions, since they are transparently clear to anyone who gives them his attention.

Synthesis, by contrast, employs a directly opposite method where the search is, as it were, a posteriori (though the proof itself is often more a priori than it is in the analytic method). It demonstrates the conclusion clearly and employs a long series of definitions, postulates, axioms, theorems and problems, so that if anyone denies one of the conclusions it can be shown at once that it is contained in what has gone before, and hence the reader, however argumentative or stubborn he may be, is compelled to give his assent. However, this method is not as satisfying as the method of analysis, nor does it engage the minds of those who are eager to learn, since it does not show how the thing in question was discovered.

It was synthesis alone that the ancient geometers usually employed in their writings. But in my view this was not because they were utterly ignorant of analysis, but because they had such a high regard for it that they kept it to themselves like a sacred mystery.

Now it is analysis which is the best and truest method of instruction, and it was this method alone which I employed in my Meditations . As for synthesis, which is undoubtedly what you are asking me to use here, it is a method which it may be very suitable to deploy in geometry as a follow-up to analysis, but it cannot so conveniently be applied to these metaphysical subjects.

The difference is that the primary notions which are presupposed for the demonstration of geometrical truths are readily accepted by anyone, since they accord with the use of our senses. Hence there is no difficulty there, except in the proper deduction of the consequences, which can be done even by the less attentive, provided they remember what has gone before. Moreover, the breaking down of propositions to their smallest elements is specifically designed to enable them to be recited with ease so that the student recalls them whether he wants to or not.

In metaphysics by contrast there is nothing which causes so much effort as making our perception of the primary notions clear and distinct. Admittedly, they are by their nature as evident as, or even more evident than, the primary notions which the geometers study; but they conflict with many preconceived opinions derived from the senses which we have got into the habit of holding from our earliest years, and so only those who really concentrate and meditate and withdraw their minds from corporeal things, so far as is possible, will achieve perfect knowledge of them. Indeed, if they were put forward in isolation, they could easily be denied by those who like to contradict just for the sake of it. (‘Second Set of Replies’, in PW , II, 110–11) { §4.2 }

De Staël, Germaine

• Anatomical study cannot be undertaken on a living body without destroying it. Analysis, when one tries to apply it to indivisible truths, destroys them, because its first attempts are directed against their unity. We need to divide our mind in two so that one half may contemplate the other. However this division takes place, it deprives our being of that sublime identity without which we would lack sufficient strength to believe in that which consciousness alone can affirm. (1813, 44)

[interpolated into the text of the Elements ] What is analysis and what is synthesis. Analysis is the assumption of that which is sought as if it were admitted [and the arrival] by means of its consequences at something admitted to be true. Synthesis is an assumption of that which is admitted [and the arrival] by means of its consequences at something admitted to be true. ( E , Book XIII, Prop. 1; Vol. III, 442, where Heath comments on the interpolation) { §2.2 }

Explaining the Emptiness of Appearances

This means that the characteristic of the lion is empty. There is really only gold. There is no lion present. The Substance of the gold is never absent. This is the doctrine of “the emptiness of appearances”. Nonetheless, the emptiness has no characteristic of its own. It requires the appearance in order to become apparent. This does not prevent appearances from having an illusory existence, which is called “the emptiness of appearances”.

(Fazang, ‘Essay on the Golden Lion’, tr. by Bryan W. Van Norden, in Tiwald and Van Norden 2014, 87.)

To recap, wholeness is the building; particularity is the conditions. Identity is [building and conditions] not opposing each other. Difference is each condition considered separately. Integration is the result of the various conditions. Disintegration is each maintaining its own character. Alternatively, put in verse:

That the one is identical with the many is called wholeness.

That the many are not the same as the one is called particularity.

The various kinds are identical in constituting the whole.

Each has its particular difference manifested in the identity.

The wondrous integration is the Pattern of the dependent origination of one and many.

Disintegration is that each resides in its own character and does not create the whole.

This belongs to the sphere of wisdom, not discriminatory consciousness.

Through this expedient device one understands the one vehicle [of Huayan].

(Fazang, ‘The Rafter Dialogue’, tr. by David Elstein, in Tiwald and Van Norden 2014, 86.)

[In replying to the objections that Husserl had raised in his Philosophie der Arithmetik (1891) to Frege’s Grundlagen definitions] If words and combinations of words refer to [ bedeuten ] ideas, then for any two of them there are only two possibilities: either they designate the same idea or they designate different ideas. In the former case it is pointless to equate them by means of a definition: this is ‘an obvious circle’; in the latter case it is wrong. These are also the objections the author raises, one of them regularly. A definition is also incapable of analysing the sense, for the analysed sense just is not the original one. In using the word to be explained, I either think clearly everything I think when I use the defining expression: we then have the ‘obvious circle’; or the defining expression has a more richly articulated sense, in which case I do not think the same thing in using it as I do in using the word to be explained: the definition is then wrong. One would think that a definition was unobjectionable in the case where the word to be explained had as yet no sense at all, or where we were asked explicitly to regard its sense as non-existent so that it was first given a sense by the definition. But in the last case too, the author refutes the definition by reminding us of the difference between the ideas (p. 107). To evade all objections, one would accordingly have to create a new verbal root and form a word out of it. This reveals a split between psychological logicians and mathematicians. What matters to the former is the sense of the words, as well as the ideas which they fail to distinguish from the sense; whereas what matters to the latter is the thing itself: the Bedeutung of the words. The reproach that what is defined is not the concept but its extension actually affects all mathematical definitions. For the mathematician, it is no more right and no more wrong to define a conic as the line of intersection of a plane with the surface of a circular cone than to define it as a plane curve with an equation of the second degree in parallel coordinates. His choice of one or the other of these expressions or of some other one is guided solely by reasons of convenience and is made irrespective of the fact that the expressions have neither the same sense nor evoke the same ideas. I do not intend by this that a concept and its extension are one and the same, but that coincidence in extension is a necessary and sufficient criterion for the occurrence between concepts of the relation that corresponds to identity [ Gleichheit ] between objects. ( RH , 319–20/ FR , 225–6) { §6.2 }

We come to definitions . Definitions proper must be distinguished from elucidations [ Erläuterungen ]. In the first stages of any discipline we cannot avoid the use of ordinary words. But these words are, for the most part, not really appropriate for scientific purposes, because they are not precise enough and fluctuate in their use. Science needs technical terms that have precise and fixed Bedeutungen , and in order to come to an understanding about these Bedeutungen and exclude possible misunderstandings, we provide elucidations. Of course in so doing we have again to use ordinary words, and these may display defects similar to those which the elucidations are intended to remove. So it seems that we shall then have to provide further elucidations. Theoretically one will never really achieve one’s goal in this way. In practice, however, we do manage to come to an understanding about the Bedeutungen of words. Of course we have to be able to count on a meeting of minds, on others’ guessing what we have in mind. But all this precedes the construction of a system and does not belong within a system. In constructing a system it must be assumed that the words have precise Bedeutungen and that we know what they are. ( LM , 224/ FR , 313) { §6.2 }

We have ... to distinguish two quite different cases :

1. We construct a sense out of its constituents and introduce an entirely new sign to express this sense. This may be called a ‘constructive definition’ [‘ aufbauende Definition ’], but we prefer to call it a ‘definition’ tout court .

2. We have a simple sign with a long-established use. We believe that we can give a logical analysis [ Zerlegung ] of its sense, obtaining a complex expression which in our opinion has the same sense. We can only allow something as a constituent of a complex expression if it has a sense we recognize. The sense of the complex expression must be yielded by the way in which it is put together. That it agrees with the sense of the long established simple sign is not a matter for arbitrary stipulation, but can only be recognized by an immediate insight. No doubt we speak of a definition in this case too. It might be called an ‘analytic definition’ [‘ zerlegende Definition ’] to distinguish it from the first case. But it is better to eschew the word ‘definition’ altogether in this case, because what we should here like to call a definition is really to be regarded as an axiom. In this second case there remains no room for an arbitrary stipulation, because the simple sign already has a sense. Only a sign which as yet has no sense can have a sense arbitrarily assigned to it. So we shall stick to our original way of speaking and call only a constructive definition a definition. According to that a definition is an arbitrary stipulation which confers a sense on a simple sign which previously had none. This sense has, of course, to be expressed by a complex sign whose sense results from the way it is put together.

Now we still have to consider the difficulty we come up against in giving a logical analysis when it is problematic whether this analysis is correct.

Let us assume that A is the long-established sign (expression) whose sense we have attempted to analyse logically by constructing a complex expression that gives the analysis. Since we are not certain whether the analysis is successful, we are not prepared to present the complex expression as one which can be replaced by the simple sign A . If it is our intention to put forward a definition proper, we are not entitled to choose the sign A , which already has a sense, but we must choose a fresh sign B , say, which has the sense of the complex expression only in virtue of the definition. The question now is whether A and B have the same sense. But we can bypass this question altogether if we are constructing a new system from the bottom up; in that case we shall make no further use of the sign A – we shall only use B . We have introduced the sign B to take the place of the complex expression in question by arbitrary fiat and in this way we have conferred a sense on it. This is a definition in the proper sense, namely a constructive definition.

If we have managed in this way to construct a system for mathematics without any need for the sign A , we can leave the matter there; there is no need at all to answer the question concerning the sense in which – whatever it may be – this sign had been used earlier. In this way we court no objections. However, it may be felt expedient to use sign A instead of sign B . But if we do this, we must treat it as an entirely new sign which had no sense prior to the definition. We must therefore explain that the sense in which this sign was used before the new system was constructed is no longer of any concern to us, that its sense is to be understood purely from the constructive definition that we have given. In constructing the new system we can take no account, logically speaking, of anything in mathematics that existed prior to the new system. Everything has to be made anew from the ground up. Even anything that we may have accomplished by our analytical activities is to be regarded only as preparatory work which does not itself make any appearance in the new system itself.

Perhaps there still remains a certain unclarity. How is it possible, one may ask, that it should be doubtful whether a simple sign has the same sense as a complex expression if we know not only the sense of the simple sign, but can recognize the sense of the complex one from the way it is put together? The fact is that if we really do have a clear grasp of the sense of the simple sign, then it cannot be doubtful whether it agrees with the sense of the complex expression. If this is open to question although we can clearly recognize the sense of the complex expression from the way it is put together, then the reason must lie in the fact that we do not have a clear grasp of the sense of the simple sign, but that its outlines are confused as if we saw it through a mist. The effect of the logical analysis of which we spoke will then be precisely this – to articulate the sense clearly. Work of this kind is very useful; it does not, however, form part of the construction of the system, but must take place beforehand. Before the work of construction is begun, the building stones have to be carefully prepared so as to be usable; i.e. the words, signs, expressions, which are to be used, must have a clear sense, so far as a sense is not to be conferred on them in the system itself by means of a constructive definition.

We stick then to our original conception: a definition is an arbitrary stipulation by which a new sign is introduced to take the place of a complex expression whose sense we know from the way it is put together. A sign which hitherto had no sense acquires the sense of a complex expression by definition. ( LM , 227–9/ FR , 317–8) { §6.2 }

Whereas in Europe the emergence of the new philosophy was inextricably interwoven with the emergence of natural science, in India this was not so. The failure to appreciate that the two developments are nevertheless distinct is another important reason why there has not been a proper diagnosis of early modernity in the philosophy. Generally speaking, what we can say is that early modern forms of philosophical inquiry in India are governed by data drawn from logical form and linguistic practice rather than the microscope and distal observation of natural phenomenon. Philosophy in early modern India made the discipline rest instead on the sort of linguistic turn that characterized, much later, the origins of analytical philosophy in European thought. Bearing this point in mind, it is no surprise that profound affinities should have been discovered between early modern theory in India and twentieth century analytical philosophy. (2011, 6) { §4.6 }

The highest good is reached through an understanding of the true nature of [the distinction between] honest, dishonest and destructive debate, of false reasoning, tricks and checks in debate, of [the pattern of sound investigation, whose components are] doubt, purpose, examples, assumed principles, syllogisms, suppositional reasoning and decision, and [initially] of the ways of gaining knowledge and the knowables. (Cited in Ganeri 2011, 122) { §4.6 }

Analysis … is sorting out the structures of signification … and determining their social ground and import. ( The Interpretation of Cultures , New York: Basic Books, 1973, 9)

Cultural analysis is (or should be) guessing at meanings, assessing the guesses, and drawing explanatory conclusions from the better guesses, not discovering the Continent of Meaning and mapping out its bodiless landscape. ( Ibid ., 20)

• Now, when a person is dead, their mixture returns to the substance of the earth...But once these elements have been driven to life in the organism, they become different from what they were before they entered into this organic connection. They have become livelier and increase the earth's life in returning to the earth, like two who have increased their strength in long struggle are now stronger when the struggle has ended than they were before. ( GSW , 447; IE , 82)

The analysis of an idea, as it used to be carried out, was, in fact, nothing else than ridding it of the form in which it had become familiar. To break an idea up into its original elements is to return to its moments, which at least do not have the form of the given idea, but rather constitute the immediate property of the self. This analysis, to be sure, only arrives at thoughts which are themselves familiar, fixed, and inert determinations. But what is thus separated and non-actual is an essential moment; for it is only because the concrete does divide itself, and make itself into something non-actual, that it is self-moving. The activity of dissolution is the power and work of the Understanding , the most astonishing and mightiest of powers, or rather the absolute power. The circle that remains self-enclosed and, like substance, holds its moments together, is an immediate relationship, one therefore which has nothing astonishing about it. But that an accident as such, detached from what circumscribes it, what is bound and is actual only in its context with others, should attain an existence of its own and a separate freedom—this is the tremendous power of the negative; it is the energy of thought, of the pure ‘I’. Death, if that is what we want to call this non-actuality, is of all things the most dreadful, and to hold fast what is dead requires the greatest strength. Lacking strength, Beauty hates the Understanding for asking of her what it cannot do. But the life of Spirit is not the life that shrinks from death and keeps itself untouched by devastation, but rather the life that endures it and maintains itself in it. It wins its truth only when, in utter dismemberment, it finds itself. It is this power, not as something positive, which closes its eyes to the negative, as when we say of something that it is nothing or is false, and then, having done with it, turn away and pass on to something else; on the contrary, Spirit is this power only by looking the negative in the face, and tarrying with it. This tarrying with the negative is the magical power that converts it into being. This power is identical with what we earlier called the Subject, which by giving determinateness an existence in its own element supersedes abstract immediacy, i.e. the immediacy which barely is, and thus is authentic substance: that being or immediacy whose mediation is not outside of it but which is this mediation itself. ( PS , ‘Preface’, §32, 18–19)

[Summary of above passage offered by J.N. Findlay] The analysis of an idea is the removal of its familiarity, its reduction to elements that are the true possessions of the thinking self. In such reduction the idea itself changes and renders itself unreal. The force which effects analysis is that of the Understanding, the most remarkable and absolute of powers, the power of the thinking self and also of death. It is above all marvellous that this thinking self should be able to isolate, and to look at apart, what can only exist as an aspect or ‘moment’ in a living whole. Thinking Spirit can, however, only grasp such a whole by first tearing it into parts, each of which it must look at separately for a while, before putting them back in the whole. The thinking self must destroy an immediate, existent unity in order to arrive at a unity which includes mediation, and is in fact mediation itself. (‘Analysis of the Text’, §32, in PS , 499) { §5.2 }

What we are trying to bring to light here by means of phenomenological analysis in regard to the intentional structure of production is not contrived and fabricated but already present in the everyday, pre-philosophical productive behaviour of the Dasein. In producing, the Dasein lives in such an understanding of being without conceiving it or grasping it as such. (1927, §12, 114–15) { §5.8 }

every method by which we investigate the causes of things is either compositive, or resolutive, or partly compositive, partly resolutive. And the resolutive is usually called analytic, while the compositive is usually called synthetic. ( Logica , ‘On Method’, §1, 289) { §4.1 }

What philosophers seek to know. Philosophers seek scientific knowledge either simply or indefinitely, that is, they seek to know as much as they can when no definite question is proposed or the cause of some definite phenomenon or at least to discover something definite, such as what the cause of light is, or of heat, or gravity, of a figure which has been proposed, and similar things; or in what subject some proposed accident inheres; or which of many accidents is above all conducive to the production of some proposed effect; or in what way particular proposed causes ought to be conjoined in order to produce a definite effect. Because of the variety of the things sought for, sometimes the analytic method, sometimes the synthetic method, and sometimes both ought to be applied.

The first part, by which principles are found, is purely analytic. Seeing that the causes of all singulars are composed from the causes of universals or simples, it is necessary for those who are looking simply for scientific knowledge, which consists of the knowledge of the causes of all things insofar as this can be achieved, to know the causes of universals or those accidents which are common to all bodies, that is, to every material thing, before they know the causes of singular things, that is, of the accidents by which one thing is distinguished from another. Again, before the causes of those things can be known, it is necessary to know which things are universals. But since universals are contained in the nature of singular things, they must be unearthed by reason, that is, by resolution. For example, let any conception or idea of a singular thing be proposed, say a square. The square is resolved into: plane, bounded by a certain number of lines equal to one another, and right angles . Therefore we have these universals or components of every material thing: line, plane (in which a surface is contained), being bounded, angle, rectitude , and equality . If anyone finds the causes or origin of these, he will put them together as the cause of the square. Again, if he proposes to himself the conception of gold, the ideas of being solid, visible, and heavy (that is, of tending to the center of the earth or of motion downwards) and many others more universal than gold itself, which can be resolved further until one arrives at the most universal, will come from this by resolution. And by this same method of resolving things into other things one will know what those things are, of which, when their causes are known what those things are, of which, when their causes are known and composed one by one, the causes of all singular things are known. We thus conclude that the method of investigating the universal notions of things is purely analytic. ( Ibid ., §§ 3–4, 291–5) { §4.1 }

The method of scientific knowledge, civil as well as natural, [starting] from sense-experience and [going] to principles is analytic; while [starting] from principles is synthetic. ( Ibid ., §7, 301) { §4.1 }

it is obvious that in the investigation of causes there is a need partly for the analytic method, partly for the synthetic method. The analytic method is needed for understanding the circumstances of the effect one by one; the synthetic method for putting together those things which, single in themselves, act as one. ( Ibid ., §10, 311) { §4.1 }

that art of geometers which they call logistic is ... the method according to which by supposing that the thing asked about is true they come upon in reasoning either things known [to be true], from which they can prove the truth of the thing sought, or [they come upon] impossibilities, from which it can be understood that what was supposed [to be true] was false. ( Ibid ., §19, 329) { §4.1 }

[Logical analysis] stands somewhere between translating and paraphrasing. ( Logic , Harmondsworth: Penguin, 1977, 86)

The terms “analysis” and “synthesis” bring to mind, on the one hand, certain methodological practices in the works of Plato, Descartes, Newton, Kant, Hegel, and others and, on the other hand, techniques in fields as disparate as chemistry and logic, mathematics and psychology. The width of this spectrum of associations alerts us to the realization that at the base of these two related terms there lies a specific methodological thema-antithema ... pair. Indeed, it is one of the most pervasive and fundamental ones, in science and outside. This chapter attempts to uncover and identify this thematic content, to clarify the meanings and uses of the terms “analysis” and “synthesis”, and especially to distinguish among four general meanings: (1) Analysis and Synthesis, and particularly synthesis, used in the grand, cultural sense, (2) Analysis and Synthesis used in the reconstitutional sense (e.g., where an analysis, followed by a synthesis, re-establishes the original condition), (3) Analysis and Synthesis used in the transformational sense (e.g., where the application of Analysis and Synthesis advances one to a qualitatively new level), and (4) Analysis and Synthesis used in the judgmental sense (as in the Kantian categories and their modern critiques). (1998, 111) { §5.5 }

The point of view of function is the central one for phenomenology; the investigations radiating from it comprise almost the whole phenomenological sphere, and in the end all phenomenological analyses somehow enter into its service as component parts or preliminary stages. In place of analysis and comparison, description and classification restricted to particular experiences [ Erlebnisse ], the particulars are considered from the “teleological” point of view of their function, to make possible “synthetic unity”. ( IPP , I, §86; Kersten’s tr. modified) { §5.8 }

Explication is penetration of the internal horizon of the object by the direction of perceptual interest. In the case of the unobstructed realization of this interest, the protentional expectations fulfill themselves in the same way; the object reveals itself in its properties as that which it was anticipated to be, except that what was anticipated now attains original givenness. A more precise determination results, eventually perhaps partial corrections, or—in the case of obstruction—disappointment of the expectations, and partial modalization. ( EJ , §22, 105) { §5.8 }

The process of explication in its originality is that in which an object given at first hand is brought to explicit intuition. The analysis of its structure must bring to light how a twofold constitution of sense [ Sinngebung ] is realized in it: “object as substrate” and “determination α ...”; it must show how this constitution of sense is realized in the form of a process which goes forward in separate steps, through which, however, extends continuously a unity of coincidence —a unity of coincidence of a special kind, belonging exclusively to these sense-forms. ( EJ , §24a, 114) { §5.8 }

• What we carry out by means of these  topoi , inasmuch as they are instruments, proceeds in the following way: we place before our consideration all the  topoi  when we examine a given  quaesitum ; then we analyse this  quaesitum  into its predicate and its subject and we review inductively each of the  topoi  intended for establishing or refuting it. If we find this  quaesitum  falling under one of the  topoi , we have there and then found the syllogism which allows us to establish or refute it. Indeed it is from this action that this part of logic is called “analysis”. ( SL , §6) { 3.3 }
• Analysis is the inquiry [directed] towards finding the premisses from which the  quaesitum  follows, on condition that they include a middle term which shows that the analyst, when he reaches the goal of his analysis, has found the premisses by means of the analysis and accomplished what Aristotle in his  Analytics  called the acquisition of the premisses. When he has found the premisses in his analysis their terms are necessarily, for [analyst], existent, known and identifiable. In the analysis one must therefore name the terms, identify and describe them. { §3.3 }

§1. MATHEMATICS ARRIVES AT ALL ITS DEFINITIONS SYNTHETICALLY, WHEREAS PHILOSOPHY ARRIVES AT ITS DEFINITIONS ANALYTICALLY

There are two ways in which one can arrive at a general concept: either by the arbitrary combination of concepts, or by separating out that cognition which has been rendered distinct by means of analysis. Mathematics only ever draws up its definitions in the first way. For example, think arbitrarily of four straight lines bounding a plane surface so that the opposite sides are not parallel to each other. Let this figure be called a trapezium . The concept which I am defining is not given prior to the definition itself; on the contrary, it only comes into existence as a result of that definition. Whatever the concept of a cone may ordinarily signify, in mathematics, the concept is the product of the arbitrary representation of a right-angled triangle which is rotated on one of its sides. In this and in all other cases the definition obviously comes into being as a result of synthesis .

The situation is entirely different in the case of philosophical definitions. In philosophy, the concept of a thing is always given, albeit confusedly or in an insufficiently determinate fashion. The concept has to be analysed; the characteristic marks which have been separated out and the concept which has been given have to be compared with each other in all kinds of contexts; and this abstract thought must be rendered complete and determinate. For example, everyone has a concept of time. But suppose that that concept has to be defined. The idea of time has to be examined in all kinds of relation if its characteristic marks which have been abstracted have to be combined together to see whether they yield an adequate concept; they have to be collated with each other to see whether one characteristic mark does not partly include another within itself. If, in this case, I had tried to arrive at a definition of time synthetically, it would have had to have been a happy coincidence indeed if the concept, thus reached synthetically, had been exactly the same as that which completely expresses the idea of time which is given to us. ( IDP , 2: 276–7/ TP , 248–9) { §4.5 }

The true method of metaphysics is basically the same as that introduced by Newton into natural science and which has been of such benefit to it. Newton’s method maintains that one ought, on the basis of certain experience and, if need be, with the help of geometry, to seek out the rules in accordance with which certain phenomena of nature occur. ( IDP , 2: 286/ TP , 259) { §4.5 }

What I am chiefly concerned to establish is this: in metaphysics one must proceed analytically throughout, for the business of metaphysics is actually the analysis of confused cognitions. If this procedure is compared with the procedure which is adopted by philosophers and which is currently in vogue in all schools of philosophy, one will be struck by how mistaken the practice of philosophers is. With them, the most abstracted concepts, at which the understanding naturally arrives last of all, constitute their starting point, and the reason is that the method of the mathematicians, which they wish to imitate throughout, is firmly fixed in their minds. This is why there is a strange difference to be found between metaphysics and all other sciences. In geometry and in the other branches of mathematics, one starts with what is easier and then one slowly advances to the more difficult operations. In metaphysics, one starts with what is the most difficult: one starts with possibility, with existence in general, with necessity and contingency, and so on – all of them concepts which demand great abstraction and close attention. And the reason for this is to be sought chiefly in the fact that the signs for these concepts undergo numerous and imperceptible modifications in use; and the differences between them must not be overlooked. One is told that one ought to proceed synthetically throughout. Definitions are thus set up right at the beginning, and conclusions are confidently drawn from them. Those who practise philosophy in this vein congratulate each other for having learnt the secret of thorough thought from the geometers. What they do not notice at all is the fact that geometers acquire their concepts by means of synthesis , whereas philosophers can only acquire their concepts by means of analysis – and that completely changes the method of thought. ...

Metaphysics has a long way to go yet before it can proceed synthetically. It will only be when analysis has helped us towards concepts which are understood distinctly and in detail that it will be possible for synthesis to subsume compound cognitions under the simplest cognition, as happens in mathematics. ( IDP , 2: 289–90/ TP , 262–3) { §4.5 }

Such a system of pure (speculative) reason I hope myself to deliver under the title Metaphysics of Nature , which will be not half so extensive but will be incomparably richer in content than this critique, which had first to display the sources and conditions of its possibility, and needed to clear and level a ground that was completely overgrown. Here I expect from my reader the patience and impartiality of a judge , but there I will expect the cooperative spirit and assistance of a fellow worker ; for however completely the principles of the system may be expounded in the critique, the comprehensiveness of the system itself requires also that no derivative concepts should be lacking, which, however, cannot be estimated a priori in one leap, but must be gradually sought out; likewise, just as in the former the whole synthesis of concepts has been exhausted, so in the latter it would be additionally demanded that the same thing should take place in respect of their analysis , which would be easy and more entertainment than labor. ( CPR , Axxi) { §4.5 }

I understand by an analytic of concepts not their analysis, or the usual procedure of philosophical investigations, that of analyzing [ zergliedern ] the content of concepts that present themselves and bringing them to distinctness, but rather the much less frequently attempted analysis [ Zergliederung ] of the faculty of understanding itself, in order to research the possibility of a priori concepts by seeking them only in the understanding as their birthplace and analyzing its pure use in general; for this is the proper business of a transcendental philosophy; the rest is the logical treatment of concepts in philosophy in general. We will therefore pursue the pure concepts into their first seeds and predispositions in the human understanding, where they lie ready, until with the opportunity of experience they are finally developed and exhibited in their clarity by the very same understanding, liberated from the empirical conditions attaching to them. ( CPR , A65–6/B90–1) { §4.5 }

[in offering a refutation of Mendelssohn’s proof of the persistence of the soul] If we take the above propositions in a synthetic connection, as valid for all thinking beings, as they must be taken in rational psychology as a system, and if from the category of relation, starting with the proposition “All thinking beings are, as such, substances” we go backward through the series of propositions until the circle closes, then we finally come up against the existence of thinking beings, which in this system are conscious of themselves not only as independent of external things but also as being able to determine themselves from themselves (in regard to the persistence belonging necessarily to the character of a substance). But from this it follows that idealism , at least problematic idealism, is unavoidable in that same rationalistic system, and if the existence of external things is not at all required for the determination of one’s own existence in time, then such things are only assumed, entirely gratuitously, without a proof of them being able to be given.

If, on the contrary, we follow the analytic procedure, grounded on the “I think” given as a proposition that already includes existence in itself, and hence grounded on modality, and then we take it apart so as to cognize its content, whether and how this I determines its existence in space or time merely through it, then the propositions of the rational doctrine of the soul begin not from the concept of a thinking being in general but from an actuality; and from the way this is thought, after everything empirical has been detached from it, it is concluded what pertains to a thinking being in general ... ( CPR , B416–19) { §4.5 }

Give a philosopher the concept of a triangle, and let him try to find out in his way how the sum of its angles might be related to a right angle. He has nothing but the concept of a figure enclosed by three straight lines, and in it the concept of equally many angles. Now he may reflect on this concept as long as he wants, yet he will never produce anything new. He can analyze [ zergliedern ] and make distinct the concept of a straight line, or of an angle, or of the number three, but he will not come upon any other properties that do not already lie in these concepts. But now let the geometer take up this question. He begins at once to construct a triangle. Since he knows that two right angles together are exactly equal to all of the adjacent angles that can be drawn at one point on a straight line, he extends one side of his triangle, and obtains two adjacent angles that together are equal to two right ones. Now he divides the external one of these angles by drawing a line parallel to the opposite side of the triangle, and sees that here there arises an external adjacent angle which is equal to an internal one, etc. In such a way, through a chain of inferences that is always guided by intuition, he arrives at a fully illuminating and at the same time general solution of the question. ( CPR , A716–7/B744–5) { §4.5 }

But although a mere plan that might precede the  Critique of Pure Reason  would be unintelligible, undependable, and useless, it is by contrast all the more useful if it comes after. For one will thereby be put in the position to survey the whole, to test one by one the main points at issue in the science, and to arrange many things in the exposition better than could be done in the first execution of the work. Here then is such a  plan  subsequent to the completed work, which now can be laid out according to the  analytic method , whereas the  work  itself absolutely had to be composed according to the  synthetic method , so that the science might present all of its articulations, as the structural organization of a quite peculiar capacity of cognition, in their natural connection. ( PFM,  4: 263; translation modified)

In the Critique of Pure Reason I worked on this question [Is metaphysics possible at all?] synthetically , namely by inquiring within pure reason itself, and seeking to determine within this source both the elements and the laws of its pure use, according to principles. This work is difficult and requires a resolute reader to think himself little by little into a system that takes no foundation as given except reason itself, and that therefore tries to develop cognition out of its original seeds without relying on any fact whatever. Prolegomena should by contrast be preparatory exercises; they ought more to indicate what needs to be done in order to bring a science into existence if possible, than to present the science itself. They must therefore rely on something already known to be dependable, from which we can go forward with confidence and ascend to the sources, which are not yet known, and whose discovery not only will explain what is known already, but will also exhibit an area with many cognitions that all arise from these same sources. The methodological procedure of prolegomena, and especially of those that are to prepare for a future metaphysics, will therefore be analytic . ( PFM , 4: 274–5/ 25–6) { §4.5 }

[interpreting the method of analysis in ancient Greek geometry] Rule of analysis and synthesis: Draw conclusions from your conjecture, one after the other, assuming that it is true. If you reach a false conclusion, then your conjecture was false. If you reach an indubitably true conclusion, your conjecture may have been true. In this case reverse the process, work backwards, and try to deduce your original conjecture via the inverse route from the indubitable truth to the dubitable conjecture. If you succeed, you have proved your conjecture. (1978a, 72–3) { §2.2 }

• If a proposition is to be proven, it occurs according to either the analytic or synthetic method, which are already defined in almost all logics. According to the analytic method, one starts with a proposition. One proves it through a syllogism. If the premises are not axioms, they must be proved through new conclusions until one finally arrives at nothing but axioms, definitions, and experiences. If this happens, one considers the proposition proved. According to the synthetic method, by contrast, one starts with definitions, principles, and experiences and derives the proof of the proposition in question from them. The inferences are the same in both methods, and the difference lies exclusively in the order, which is completely reversed. (1761, §28)

Synthesis is when, beginning from principles and running through truths in order, we discover certain progressions and form tables, as it were, or sometimes even general formulae, in which the answers to what arises later can be discovered. Analysis, however, goes back to principles solely for the sake of a given problem, just as if nothing had been discovered previously, by ourselves or by others. It is better to produce a synthesis, since that work is of permanent value, whereas when we begin an analysis on account of particular problems we often do what has been done before. However, to use a synthesis which has been established by others, and theorems which have already been discovered, is less of an art than to do everything by oneself by carrying out an analysis; especially as what has been discovered by others, or even by ourselves, does not always occur to us or come to hand. There are two kinds of analysis: one is the common type proceeding by leaps, which is used in algebra, and the other is a special kind which I call ‘reductive’. This is much more elegant, but is less well-known. In practice, analysis is more necessary, so that we may solve the problems which are presented to us; but the man who can indulge in theorising will be content to practice analysis just far enough to master the art. For the rest, he will rather practise synthesis, and will apply himself readily only to those questions to which order itself leads him. For in this way he will always progress pleasantly and easily, and will never feel any difficulties, nor be disappointed of success, and in a short time he will achieve much more than he would ever have hoped for at the outset. ( USA , 16–17) { §4.4 }

Primary truths are those which either state a term of itself, or deny an opposite of its opposite. For example, ‘A is A’, or ‘A is not not-A’ ...

All other truths are reduced to primary truths by the aid of definitions—i.e. by the analysis of notions; and this constitutes a priori proof , independent of experience. ...

The predicate or consequent, therefore, is always in the subject or antecedent, and this constitutes the nature of truth in general, or, the connexion between the terms of a proposition, as Aristotle also has observed. In identities this connexion and inclusion of the predicate in the subject is express, whereas in all other truths it is implicit and must be shown through the analysis of notions, in which a priori demonstration consists. ( PT , 87–8) { §4.4 }

There are two kinds of truths , those of reason and those of fact . Truths of reason are necessary and their opposite is impossible; truths of fact are contingent and their opposite is possible. When a truth is necessary, its reason can be found by analysis, resolving it into simpler ideas and truths, until we come to those that are primitive. ( M , §33; tr. R. Latta) { §4.4 }

Our whole philosophy is rectification of colloquial linguistic usage. ( Aphorisms , 115) { §4.5 }

Writing is an excellent means of awakening in every man the system slumbering within him; and everyone who has ever written will have discovered that writing always awakens something which, though it lay within us, we failed clearly to recognize before. ( Ibid ., 119) { §4.5 }

Whichever way you look at it, philosophy is always analytical chemistry. The peasant employs all the propositions of the most abstract philosophy, only he employs them enveloped, concealed, compounded, latent, as the chemist and physicist says; the philosopher gives us the propositions pure. ( Ibid ., 162) { §4.5 }

There are therefore three ways whereby we get the complex Ideas of mixed Modes . 1. By Experience and Observation of things themselves. Thus by seeing two Men wrestle, or fence, we get the Idea of wrestling or fencing. 2. By Invention , or voluntary putting together of several simple Ideas in our own Minds: So he that first invented Printing, or Etching, had an Idea of it in his Mind, before it ever existed. 3. Which is the most usual way, by explaining the names of Actions we never saw, or Notions we cannot see; and by enumerating, and thereby, as it were, setting before our Imaginations all those Ideas which go to the making them up, and are the constituent parts of them. For having by Sensation and Reflection stored our Minds with simple Ideas , and by use got the Names, that stand for them, we can by those Names represent to another any complex Idea , we would have him conceive; so that it has in it no simple Idea , but what he knows, and has, with us, the same name for. For all our complex Ideas are ultimately resolvable into simple Ideas , of which they are compounded, and originally made up, though perhaps their immediate Ingredients, as I may so say, are also complex Ideas . Thus the mixed Mode , which the word Lye stands for, is made of these simple Ideas : 1. Articulate Sounds. 2. Certain Ideas in the Mind of the Speaker. 3. Those words the signs of those Ideas . 4. Those signs put together by affirmation or negation, otherwise than the Ideas they stand for, are in the mind of the Speaker. I think I need not go any farther in the Analysis of that complex Idea , we call a Lye : What I have said is enough to shew, that it is made up of simple Ideas : And it could not be an offensive tediousness to my Reader, to trouble him with a more minute enumeration of every particular simple Idea , that goes to this complex one; which, from what has been said, he cannot but be able to make out to himself. The same may be done in all our complex Ideas whatsoever; which however compounded, and decompounded, may at last be resolved into simple Ideas , which are all the Materials of Knowledge or Thought we have or can have. ( Essay , II, xxii, 9) { §4.3 }

Analysis has a way of unravelling the self: the longer you pull on the thread, the more flaws you find. ( Therapy , London, 31)

It is commonplace in logic to talk about the analysis of propositions. In the context of logic in Sanskrit, we have to talk about the analysis of Sanskrit propositions. A Sanskrit proposition is what is expressed in a Sanskrit sentence. It will appear that the analysis proposed by the early Sanskrit writers would not be entirely unfamiliar to one accustomed to the usual subject-predicate analysis of modern or traditional Western logic, nor is it unrelated to it. However, the logical as well as grammatical analysis of Sanskrit sentences presents some significant contrasts with the usual subject-predicate analysis. Unless these points of contrast are noted, it will be difficult to appreciate fully some of the concerns of the Sanskrit logicians.

A sentence in Sanskrit is regarded as the expression of a “thought” or what is called a cognitive state ( jñāna ), or, to be precise, a qualificative cognitive state ( viśiṣṭa-jñāna ). A simple qualificative cognitive state is one where the cognizer cognizes something (or some place or some locus, as we will have to call it) as qualified by a property or a qualifier. It is claimed by most Sanskrit writers that to say that something or some place is qualified by a qualifier is equivalent to saying that it is a locus of some property or “locatable”. (1998, 201–2)

In a specific sense, the philosophy of language was part of Indian philosophical activity from the beginning of its history. One reason was to recognize the Scriptures’( Vedas’ ) authority in certain areas of our belief system. The Indians do not always talk about ‘revelation’ in the way it is understood in the Judaeo-Christian tradition. The Scriptures were regarded by tradition as embodying certain truths derived from the supposedly ‘revealed’ insights of the sages called ‘seers’ (= rṣi ). Veda thus means a body of knowledge, in fact, a source or ‘means’ of knowledge. The Scriptures are in fact a body of statements. This linguistic nature of the Scriptures (in the case of the Buddhists, the dialogues of the Buddha fulfil the same purpose, and the same is true of Jainism and Mahāvīra) reveals gradually the fact that language or ‘verbal testimony’ is an important source of knowledge, like perception and inference. This has led to the general inquiry about how a bit of language, a word or a sentence, imparts knowledge to the hearer. Therefore, what we call the philosophy of language in India has always formed a part of the classical philosophers’ general epistemological inquiry, part of the pramāṇa-śāstra , the theory of ‘evidence’ for belief or knowledge. The question was: how does a linguistic utterance, through the communication of its meaning, impart knowledge to the hearer? For it is observed that not simply the Scriptural statements but also any ordinary statement can and does impart knowledge. Strictly speaking, most of our knowledge today is derived from reading and listening, hence we can say that it is linguistically communicated.

In particular, however, analysis of sentences and words into significant components, the relationship between word and meaning, classification of words according to semantic contribution, division of words with reference to the division of ontological categories, logical and psychological factors in knowing the meaning of a sentence, philosophical significance of grammatical analysis, and principles of linguistics—all these have been repeatedly discussed by the philosophers in India over the centuries. This discussion constitutes the vast amount of writing which we can very profitably explore to talk about the classical Indian philosophy of language. (2001, 4–5)

The certainty of mathematics is based upon the general axiom that nothing can be and not be at the same time. In this science each proposition such as, for example, “A is B”, is proven in one of two ways. Either one unpacks the concepts of A and shows “A is B”, or one unpacks the concepts of B and infers from this that not-B must also be not-A. Both types of proof are thus based upon the principle of contradiction, and since the object of mathematics in general is magnitude and that of geometry in particular extension , one can say that in mathematics in general our concepts of magnitude are unpacked and analyzed, while in geometry in particular our concepts of extension are unpacked and analyzed. In fact, since geometry lays nothing else as its basis than the abstract concept of extension and derives all its conclusions from this single source – deriving them, to be sure, in such a way that one recognizes distinctly that everything maintained in it is necessarily connected by the principle of contradiction with the abstracted concept of extension, there is no doubt that all geometric truths that geometry teaches us to unpack or untangle from the concept of extension must be encountered all tangled up in it. For what else can the profoundest inferences do but analyze a concept and make distinct what was obscure? Such inferences cannot bring in what is not to be found in the concept, and it is easy to see that it is also not possible, by means of the principle of contradiction, to derive from the concept what is not to be found in it. In the concept of extension, for example, there lies the inner possibility that a space is limited by three straight lines in such a way that two of them include a right angle. For it follows from the essence of extension that it is capable of many sorts of limitations and that the assumed sort of limitation of one of its level planes contains no contradiction. If one subsequently shows that the concept of this assumed limitation or of a right-angled triangle necessarily entails that the square of the hypotenuse is such-and-such, then it must have also been possible to find this truth originally and implicitly in the initial concept of extension. Otherwise it could never have been derived from it by means of the principle of contradiction. The idea of extension is inseparable from the idea of the possibility of such a limitation, as was previously assumed, and the limitation is in turn necessarily connected to the concept of the equality of the aforesaid square. Thus, this truth also lay tangled up, as one might say, in the original concept of extension, but it escaped our attention and could not be distinctly known and distinguished until, through analysis, we unpacked all the parts of this concept and separated them from one another. The analysis of concepts is for the understanding nothing more than what the magnifying glass is for sight. It does not produce anything that was not to be found in the object. But it spreads out the parts of the object and makes it possible for our senses to distinguish much that they would otherwise not have noticed. The analysis of concepts does nothing different from this; it makes the parts and members of these concepts, which were previously obscure and unnoticed, distinct and recognizable, but it does not introduce anything into the concepts that was not already to be found in them. (1763, §1/ PW , 257–8) { §4.5 }

It seems necessary, then, to regard the world as formed of concepts. These are the only objects of knowledge. They cannot be regarded fundamentally as abstractions either from things or from ideas; since both alike can, if anything is to be true of them, be composed of nothing but concepts. A thing becomes intelligible first when it is analysed into its constituent concepts. ( NJ , 8) { §6.4 }

It appears to me that in Ethics, as in all other philosophical studies, the difficulties and disagreements, of which its history is full, are mainly due to a very simple cause: namely to the attempt to answer questions, without first discovering precisely what question it is which you desire to answer. I do not know how far this source of error would be done away, if philosophers would try to discover what question they were asking, before they set about to answer it; for the work of analysis and distinction is often very difficult: we may often fail to make the necessary discovery, even though we make a definite attempt to do so. But I am inclined to think that in many cases a resolute attempt would be sufficient to ensure success; so that, if only this attempt were made, many of the most glaring difficulties and disagreements in philosophy would disappear. ( PE , vii) { §6.4 }

My point is that ‘good’ is a simple notion, just as ‘yellow’ is a simple notion; that, just as you cannot, by any manner of means, explain to any one who does not already know it, what yellow is, so you cannot explain what good is. Definitions of the kind that I was asking for, definitions which describe the real nature of the object or notion denoted by a word, and which do not merely tell us what the word is used to mean, are only possible when the object or notion in question is something complex. You can give a definition of a horse, because a horse has many different properties and qualities, all of which you can enumerate. But when you have enumerated them all, when you have reduced a horse to his simplest terms, then you no longer define those terms. They are simply something which you think of or perceive, and to any one who cannot think of or perceive them, you can never, by any definition, make their nature known. ( PE , 7) { §6.4 }

As in Mathematicks, so in Natural Philosophy, the Investigation of difficult Things by the Method of Analysis, ought ever to precede the Method of Composition. This Analysis consists in making Experiments and Observations, and in drawing general Conclusions from them by Induction, and admitting of no Objections against the Conclusions, but such as are taken from Experiments, or other certain Truths. For Hypotheses are not to be regarded in experimental Philosophy. And although the arguing from Experiments and Observations by Induction be no Demonstration of general Conclusions; yet it is the best way of arguing which the Nature of Things admits of, and may be looked upon as so much the stronger, by how much the Induction is more general. And if no Exception occur from Phænomena, the Conclusion may be pronounced generally. But if at any time afterwards any Exception shall occur from Experiments, it may then begin to be pronounced with such Exceptions as occur. By this way of Analysis we may proceed from Compounds to Ingredients, and from Motions to the Forces producing them; and in general, from Effects to their Causes, and from particular Causes to more general ones, till the Argument end in the most general. This is the Method of Analysis: and the Synthesis consists in assuming the Causes discover’d, and establish’d as Principles, and by them explaining the Phænomena proceeding from them, and proving the Explanations. ( Opticks , Book Three, Part I, 404–5) { §4.1 }

All concepts in which an entire process is semiotically telescoped elude definition. ( On the Genealogy of Morals , 1887, tr. Walter Kaufmann, New York: Random House, 1968, 80)

the most valuable insights are methods . ( The Antichrist , 1895, §13)

The so-called Treasury of Analysis [ analuomenos ] .. is, in short, a special body of doctrines furnished for the use of those who, after going through the usual elements, wish to obtain the power of solving theoretical problems, which are set to them, and for this purpose only is it useful. It is the work of three men, Euclid the author of the Elements , Apollonius of Perga, and Aristaeus the Elder, and proceeds by the method of analysis and synthesis.

Now analysis is the way from what is sought—as if it were admitted—through its concomitants [ akolouthôn ] in order to something admitted in synthesis. For in analysis we suppose that which is sought to be already done, and we inquire from what it results, and again what is the antecedent [ proêgoumenon ] of the latter, until we on our backward way light upon something already known and being first in order. And we call such a method analysis, as being a solution backwards [ anapalin lysin ].

In synthesis, on the other hand, we suppose that which was reached last in analysis to be already done, and arranging in their natural order as consequents [ epomena ] the former antecedents [ proêgoumena ] and linking them one with another, we in the end arrive at the construction of the thing sought. And this we call synthesis.

Now analysis is of two kinds. One seeks the truth, being called theoretical. The other serves to carry out what was desired to do, and this is called problematical. In the theoretical kind we suppose the thing sought as being and as being true, and then we pass through its concomitants [ akolouthôn ] in order, as though they were true and existent by hypothesis, to something admitted; then, if that which is admitted be true, the thing sought is true, too, and the proof will be the reverse of analysis. But if we come upon something false to admit, the thing sought will be false, too. In the problematic kind we suppose the desired thing to be known, and then we pass through its concomitants [ akolouthôn ] in order, as though they were true, up to something admitted. If the thing admitted is possible or can be done, that is, if it is what the mathematicians call given, the desired thing will also be possible. The proof will again be the reverse of analysis. But if we come upon something impossible to admit, the problem will also be impossible. ( PAC , tr. in Hintikka and Remes 1974, 8–10) { §2.2 }

For we should remember that if a person goes on analyzing names into words, and inquiring also into the elements out of which the words are formed, and keeps on always repeating this process, he who has to answer him must at last give up the inquiry in despair … But if we take a word which is incapable of further resolution, then we shall be right in saying that we at last reached a primary element, which need not be resolved any further. (‘Cratylus’, Benjamin Jowett (trans.), in Hamilton and Cairns (ed.), Collected Dialogues , New York: Pantheon Books, 421e)

Then, said I, is not dialectic the only process of inquiry that advances in this manner, doing away with hypotheses, up to the first principle itself in order to find confirmation there? And it is literally true that when the eye of the soul is sunk in the barbaric slough of the Orphic Myth, dialectic gently draws it forth and leads it up, employing as helpers and cooperators in this conversation the studies and sciences which we enumerated, which we called sciences often from habit, though they really need some other designation, connoting more clearness than opinion and more obscurity than science. ‘Understanding’ I believe was the term we employed. But, I presume we shall not dispute about the name when things of such moment lie before us for consideration. (‘Republic VII’, Paul Shorey (trans.), Ibid. , 533d)

Understand then, said I, that by the other section of the intelligible I mean that which the reason lays hold of by the power of dialectic, treating its assumptions not as absolute beginnings but literally as hypotheses, underpinnings, footings and springboards so to speak, to enable it to rise to that which requires no assumption and is the starting point of all, and after attaining to that again taking hold of the first dependencies from it, so to proceed downward to the conclusion, making no use whatever of any object of sense but only of pure ideas moving on through ideas to ideas and ending with ideas. (‘Republic VI’, Paul Shorey (trans.), Ibid ., 511b)

In mathematics logic is called analysis , and analysis means division , dissection . It can have, therefore, no tool other than the scalpel and the microscope. (‘Intuition and Logic in Mathematics’, 1900, in William Ewald, ed., From Kant to Hilbert , Oxford: Oxford University Press, 1996, 1018)

Nonmathematical illustration [of the method of analysis described by Pappus] . A primitive man wishes to cross a creek; but he cannot do so in the usual way because the water has risen overnight. Thus, the crossing becomes the object of a problem; ‘crossing the creek’ is the x of this primitive problem. The man may recall that he has crossed some other creek by walking along a fallen tree. He looks around for a suitable fallen tree which becomes his new unknown, his y . He cannot find any suitable tree but there are plenty of trees standing along he creek; he wishes that one of them would fall. Could he make a tree fall across the creek? There is a great idea and there is a new unknown; by what means could he tilt the tree over the creek?

This train of ideas ought to be called analysis if we accept the terminology of Pappus. If the primitive man succeeds in finishing his analysis he may become the inventor of the bridge and of the axe. What will be the synthesis? Translation of ideas into actions. The finishing act of the synthesis is walking along a tree across the creek.

The same objects fill the analysis and the synthesis; they exercise the mind of the man in the analysis and his muscles in the synthesis; the analysis consists in thoughts, the synthesis in acts. There is another difference; the order is reversed. Walking across the creek is the first desire from which the analysis starts and it is the last act with which the synthesis ends. (1957, 145) { §2.2 }

beauty and order are common to all branches of mathematics, as are the method of proceeding from things better known to things we seek to know and the reverse path from the latter to the former, the methods called analysis and synthesis. ( CEE , 8/6–7) { §2.2 }

as Nous is set over understanding and dispenses principles to it from above, perfecting it out of its own riches, so in the same way dialectic, the purest part of philosophy, hovers attentively over mathematics, encompasses its whole development, and of itself contributes to the special sciences their various perfecting, critical, and intellective powers—the procedures, I mean, of analysis, division, definition, and demonstration. Being thus endowed and led towards perfection, mathematics reaches some of its results by analysis, others by synthesis, expounds some matters by division, others by definition, and some of its discoveries binds fast by demonstration, adapting these methods to its subjects and employing each of them for gaining insight into mediating ideas. Thus its analyses are under the control of dialectic, and its definitions, divisions, and demonstrations are of the same family and unfold in conformity with the way of mathematical understanding. It is reasonable, then, to say that dialectic is the capstone of the mathematical sciences. It brings to perfection all the intellectual insight they contain, making what is exact in them more irrefutable, confirming the stability of what they have established and referring what is pure and incorporeal in them to the simplicity and immateriality of Nous, making precise their primary starting-points through definitions and explicating the distinctions of genera and species within their subject-matters, teaching the use of synthesis to bring out the consequences that follow from principles and of analysis to lead up to the first principles and starting-points. ( CEE , 42–3/35–6) { §2.2 }

Magnitudes, figures and their boundaries, and the ratios that are found in them, as well as their properties, their various positions and motions—these are what geometry studies, proceeding from the partless point down to solid bodies, whose many species and differences it explores, then following the reverse path from the more complex objects to the simpler ones and their principles. It makes use of synthesis and analysis, always starting from hypotheses and first principles that it obtains from the science above it and employing all the procedures of dialectic—definition and division for establishing first principles and articulating species and genera, and demonstrations and analyses in dealing with the consequences that follow from first principles, in order to show the more complex matters both as proceeding from the simpler and also conversely as leading back to them. ( CEE , 57/46) { §2.2 }

[Euclid’s Elements ] contains all the dialectical methods: the method of division for finding kinds, definitions for making statements of essential properties, demonstrations for proceeding from premises to conclusions, and analysis for passing in the reverse direction from conclusions to principles. ( CEE , 69/57) { §2.2 }

there are certain methods that have been handed down, the best being the method of analysis, which traces the desired result back to an acknowledged principle. Plato, it is said, taught this method to Leodamas, who also is reported to have made many discoveries in geometry by means of it. A second is the method of diaeresis , which divides into its natural parts the genus proposed for examination and which affords a starting-point for demonstration by eliminating the parts irrelevant for the establishment of what is proposed. This method also Plato praised as an aid in all the sciences. A third is the reduction to impossibility, which does not directly show the thing itself that is wanted but by refuting its contradictory indirectly establishes its truth. ( CEE , 211–12/165–6) { §2.2 }

for problems one common procedure, the method of analysis, has been discovered, and by following it we can reach a solution; for thus it is that even the most obscure problems are pursued. ( CEE , 242/189) { §2.2 }

In general we must understand that all mathematical arguments proceed either from or to the starting-points, as Porphyry somewhere says. Those that proceed from the starting-points are themselves of two kinds, as it happens, for they proceed either from common notions, that is, from self-evident clarity alone, or from things previously demonstrated. Those that proceed to the starting-points are either affirmative of them or destructive. But those that affirm first principles are called “analyses”, and their reverse procedures “syntheses” (for it is possible from those principles to proceed in orderly fashion to the thing sought, and this is called “synthesis”); when they are destructive, they are called “reductions to impossibility”, for it is the function of this procedure to show that something generally accepted and self-evident is overthrown. There is a kind of syllogism in it, though not the same as in analysis ... ( CEE , 255/198–9) { §2.2 }

A maxim of shallow analysis prevails: expose no more logical structure than seems useful for the deduction or other inquiry at hand. In the immortal words of Adolf Meyer, where it doesn’t itch don't scratch.

On occasion the useful degree of analysis may, conversely, be such as to cut into a simple word of ordinary language, requiring its paraphrase into a composite term in which other terms are compounded with the help of canonical notation. When this happens, the line of analysis adopted will itself commonly depend on what is sought in the inquiry at hand; again there need be no question of the uniquely right analysis, nor of synonymy. (1960, §33, 160–1) { §6.9 }

This construction [of the ordered pair as a class, such as Wiener’s identification of the ordered pair x , y > with the class {{ x }, { y , Λ}}] is paradigmatic of what we are most typically up to when in a philosophical spirit we offer an “analysis” or “explication” of some hitherto inadequately formulated “idea” or expression. We do not claim synonymy. We do not claim to make clear and explicit what the users of the unclear expression had unconsciously in mind all along. We do not expose hidden meanings, as the words ‘analysis’ or ‘explication’ would suggest; we supply lacks. We fix on the particular functions of the unclear expression that make it worth troubling about, and then devise a substitute, clear and couched in terms to our liking, that fills those functions. Beyond those conditions of partial agreement, dictated by our interests and purposes, any traits of the explicans come under the head of “don’t-cares” … Under this head we are free to allow the explicans all manner of novel connotations never associated with the explicandum. …

Philosophical analysis, explication, has not always been seen in this way. Only the reading of a synonymy claim into analysis could engender the so-called paradox of analysis, which runs thus: how can a correct analysis be informative, since to understand it we must already know the meanings of its terms, and hence already know that the terms which it equates are synonymous? The notion that analysis must consist somehow in the uncovering of hidden meanings underlies also the recent tendency of some of the Oxford philosophers to take as their business an examination of the subtle irregularities of ordinary language. And there is no mistaking the obliviousness of various writers to the point about the don’t-cares. …

... explication is elimination . We have, to begin with, an expression or form of expression that is somehow troublesome. It behaves partly like a term but not enough so, or it is vague in ways that bother us, or it puts kinks in a theory or encourages one or another confusion. But also it serves certain purposes that are not to be abandoned. Then we find a way of accomplishing those same purposes through other channels, using other and less troublesome forms of expression. The old perplexities are resolved.

According to an influential doctrine of Wittgenstein’s, the task of philosophy is not to solve problems but to dissolve them by showing that there were really none there. This doctrine has its limitations, but it aptly fits explication. For when explication banishes a problem it does so by showing it to be in an important sense unreal; viz., in the sense of proceeding only from needless usages. (1960, §53, 258–60) { §6.9 }

This brings us to the second of the five turning points, the shift from terms to sentences. The medievals had the notion of syncategorematic words, but it was a contemporary of John Horne Tooke who developed it into an explicit theory of contextual definition; namely, Jeremy Bentham. He applied contextual definition not just to grammatical particles and the like, but even to some genuine terms, categorematic ones. If he found some term convenient but ontologically embarrassing, contextual definition enabled him in some cases to continue to enjoy the services of the term while disclaiming its denotation. He could declare the term syncategorematic, despite grammatical appearances, and then could justify his continued use of it if he could show systematically how to paraphrase as wholes all sentences in which he chose to imbed it. Such was his theory of fictions: what he called paraphrasis, and what we now call contextual definition. The term, like the grammatical particles, is meaningful as a part of meaningful wholes. If every sentence in which we use a term can be paraphrased into a sentence that makes good sense, no more can be asked. (1975, 68–9) { §5.6 }

The issue is: is there such an activity as “conceptual analysis” or can philosophers do no more than describe usage and, perhaps, make recommendations for change in usage? One’s answer to this question will determine whether one thinks that Wittgenstein was wrong to give up on the idea of a systematic theory of meaning, and Quine right to suggest that the very notion of “meaning” was a hangover of Aristotelean essentialism. If they were right, it is hard to hang on to the idea that “conceptual clarity” is a goal of philosophical inquiry … Metaphilosophical issues hover in the wings of the debates over whether the content of an assertion varies from utterer to utterer and from audience to audience. If it does not, if something remains invariable – the concepts expressed by the words that make up the sentence – then perhaps there really are entities with intrinsic properties which philosophical analysis can hope to pin down. But, if content does vary in this way, then concepts are like persons — never quite the same twice, always developing, always maturing. You can change a concept by changing usage, but you cannot get a concept right, once and for all. (‘Analytic and Conversational Philosophy’, Philosophy as Cultural Politics , Cambridge: Cambridge University Press, 2007, 122–3)

Analysis, to be sure, is articulation rather than dissolution. (1980, 8) { §1.2 , §5.8 }

we must see where we are going , or what will “count” as the successful resolution to the given exercise of analysis. … Analysis is the admittedly indispensable road to our destination, but it is no more the destination than it is the intention to begin the voyage. One could perhaps say that the destination is an articulated structure. But we know that we have reached the destination only when we recognize a given articulation as the explanation of that structure. We cannot see that an analysis explains a structure by performing an additional step in the analysis. At some point we must see that we are finished. And to see an analysis is not to analyze. It is rather to see an articulated structure as a unity, whole, or synthesis. ( Ibid ., 9) { §1.2 , §5.8 }

If to understand is to possess an explanation, and if an explanation is an analysis, it remains the case that an analysis is intelligible because it is also a synthesis. Explanation may be called “recollection” in the Platonic sense because it is the process of retracing, by the method of counting and measuring, the joints of an internally articulated unity, one prefigured within the initial formulation of the entire analytical exercise. In slightly more prosaic terms, analysis is never merely the application of rules. It is also at once a seeing of which rules to apply and how to apply them. This is what it means to say that analysis is also synthesis. And this is why it is false to say, as is at least implied by so much contemporary analytical philosophy, that we begin with intuitions and then replace them with ever more sophisticated analyses. Not only is it false to say this, but strictly speaking, it is meaningless. If “to mean” is “to provide an analysis”, there is no analysis of analysis without ingredient intuition. Without intuition, there is at each stage nothing to analyze. Intuition (of syntheses or unities) without analysis is mute, but analysis without intuition is inarticulate as well as blind: the sounds it utters cannot be distinguished from noise. ( Ibid ., 9–10) { §1.2 , §5.8 }

analysis is a cognitive activity and it cannot be coherently understood except by recourse to intuition. There is a non-discursive context of analysis . ( Ibid ., 27) { §1.2 , §5.8 }

conceptual analysis is rooted in intuitions which cannot be replaced by the process of analysis but which regulate that process. ( Ibid ., 48) { §1.2 , §5.8 }

That all sound philosophy should begin with an analysis of propositions, is a truth too evident, perhaps, to demand a proof. That Leibniz’s philosophy began with such an analysis, is less evident, but seems to be no less true. ( PL , 8) { §6.3 }

It is necessary to realize that definition, in mathematics, does not mean, as in philosophy, an analysis of the idea to be defined into constituent ideas. This notion, in any case, is only applicable to concepts, whereas in mathematics it is possible to define terms which are not concepts. Thus also many notions are defined by symbolic logic which are not capable of philosophical definition, since they are simple and unanalyzable. ( POM , ch. 2, §31, 27) { §6.3 }

For the comprehension of analysis, it is necessary to investigate the notion of whole and part, a notion which has been wrapped in obscurity—though not without certain more or less valid logical reasons—by the writers who may be roughly called Hegelian. ( POM , ch. 16, §133, 137) { §6.3 }

I have already touched on a very important logical doctrine, which the theory of whole and part brings into prominence—I mean the doctrine that analysis is falsification. Whatever can be analyzed is a whole, and we have already seen that analysis of wholes is in some measure falsification. But it is important to realize the very narrow limits of this doctrine. We cannot conclude that the parts of a whole are not really its parts, nor that the parts are not presupposed in the whole in a sense in which the whole is not presupposed in the parts, nor yet that the logically prior is not usually simpler than the logically subsequent. In short, though analysis gives us the truth, and nothing but the truth, yet it can never give us the whole truth. This is the only sense in which the doctrine is to be accepted. In any wider sense, it becomes merely a cloak for laziness, by giving an excuse to those who dislike the labour of analysis. ( POM , ch. 16, §138, 141) { §6.3 }

We are sometimes told that things are organic unities, composed of many parts expressing the whole and expressed in the whole. This notion is apt to replace the older notion of substance, not, I think, to the advantage of precise thinking. The only kind of unity to which I can attach any precise sense—apart from the unity of the absolutely simple—is that of a whole composed of parts. But this form of unity cannot be what is called organic; for if the parts express the whole or the other parts, they must be complex, and therefore themselves contain parts; if the parts have been analyzed as far as possible, they must be simple terms, incapable of expressing anything except themselves. A distinction is made, in support of organic unities, between conceptual analysis and real division into parts. What is really indivisible, we are told, may be conceptually analyzable. This distinction, if the conceptual analysis be regarded as subjective, seems to me wholly inadmissible. All complexity is conceptual in the sense that it is due to a whole capable of logical analysis, but is real in the sense that it has no dependence upon the mind, but only upon the nature of the object. Where the mind can distinguish elements, there must be different elements to distinguish; though, alas! there are often different elements which the mind does not distinguish. The analysis of a finite space into points is no more objective than the analysis (say) of causality into time-sequence + ground and consequent, or of equality into sameness of relation to a given magnitude. In every case of analysis, there is a whole consisting of parts with relations; it is only the nature of the parts and the relations which distinguishes different cases. Thus the notion of an organic whole in the above sense must be attributed to defective analysis, and cannot be used to explain things.

It is also said that analysis is falsification, that the complex is not equivalent to the sum of its constituents and is changed when analyzed into these. In this doctrine, as we saw in Parts I and II, there is a measure of truth, when what is to be analyzed is a unity. A proposition has a certain indefinable unity, in virtue of which it is an assertion; and this is so completely lost by analysis that no enumeration of constituents will restore it, even though itself be mentioned as a constituent. There is, it must be confessed, a grave logical difficulty in this fact, for it is difficult not to believe that a whole must be constituted by its constituents. For us, however, it is sufficient to observe that all unities are propositions or propositional concepts, and that consequently nothing that exists is a unity. If, therefore, it is maintained that things are unities, we must reply that no things exist. ( POM , ch. 53, §439, 466–7) { §6.3 }

What we want to be clear about is the twofold method of analysis of a proposition, i.e. , first taking the proposition as it stands and analyzing it, second taking the proposition as a special case of a type of propositions. Whenever we use variables, we are already necessarily concerned with a type of propositions. E.g. “ p ⊃ q ” stands for any proposition of a certain type. When values are assigned to p and q , we reach a particular proposition by a different road from that which would have started with those values plus implication, and have so built up the particular proposition without reference to a type. This is how functions come in. (‘Fundamental Notions’, 1904, in 1994, 118) { §6.3 }

We ought to say, I think, that there are different ways of analysing complexes, and that one way of analysis is into function and argument, which is the same as type and instance. ( Ibid ., 256) { §6.3 }

The fundamental epistemological principle in the analysis of propositions containing descriptions is this: Every proposition which we can understand must be composed wholly of constituents with which we are acquainted. ( KAKD , 159) { §6.3 }

when we say ‘the author of Waverley was Scott’ we mean ‘one and only one man wrote Waverley, and he was Scott’. Here the identity is between a variable, i.e. an indeterminate subject (‘he’), and Scott; ‘the author of Waverley’ has been analysed away, and no longer appears as a constituent of the proposition. ( KAKD , 165) { §6.3 }

Analysis may be defined as the discovery of the constituents and the manner of combination of a given complex. The complex is to be one with which we are acquainted; the analysis is complete when we become acquainted with all the constituents and with their manner of combination, and know that there are no more constituents and that that is their manner of combination. We may distinguish formal analysis as the discovery of the manner of combination, and material analysis as the discovery of the constituents. Material analysis may be called descriptive when the constituents are only known by description, not by acquaintance. ( TK , 119) { §6.3 }

Philosophy, if what has been said is correct, becomes indistinguishable from logic as that word has now come to be used. The study of logic consists, broadly speaking, of two not very sharply distinguished portions. On the one hand it is concerned with those general statements which can be made concerning everything without mentioning any one thing or predicate or relation, such for example as ‘if x is a member of the class α and every member of α is a member of β , then x is a member of the class β , whatever x , α , and β may be.’. On the other hand, it is concerned with the analysis and enumeration of logical forms , i.e. with the kinds of propositions that may occur, with the various types of facts, and with the classification of the constituents of facts. In this way logic provides an inventory of possibilities, a repertory of abstractly tenable hypotheses. ( SMP , 84–5) { §6.3 }

The essence of philosophy as thus conceived is analysis, not synthesis. To build up systems of the world, like Heine’s German professor who knit together fragments of life and made an intelligible system out of them, is not, I believe, any more feasible than the discovery of the philosopher’s stone. What is feasible is the understanding of general forms, and the division of traditional problems into a number of separate and less baffling questions. ‘Divide and conquer’ is the maxim of success here as elsewhere. ( SMP , 86) { §6.3 }

Kant, under the influence of Newton, adopted, though with some vacillation, the hypothesis of absolute space, and this hypothesis, though logically unobjectionable, is removed by Occam’s razor, since absolute space is an unnecessary entity in the explanation of the physical world. Although, therefore, we cannot refute the Kantian theory of an a priori intuition, we can remove its grounds one by one through an analysis of the problem. Thus, here as in many other philosophical questions, the analytic method, while not capable of arriving at a demonstrative result, is nevertheless capable of showing that all the positive grounds in favour of a certain theory are fallacious and that a less unnatural theory is capable of accounting for the facts.

Another question by which the capacity of the analytic method can be shown is the question of realism. Both those who advocate and those who combat realism seem to me to be far from clear as to the nature of the problem which they are discussing. If we ask: ‘Are our objects of perception real and are they independent of the percipient?’ it must be supposed that we attach some meaning to the words ‘real’ and ‘independent’, and yet, if either side in the controversy of realism is asked to define these two words, their answer is pretty sure to embody confusions such as logical analysis will reveal. ( SMP , 90–1) { §6.3 }

The supreme maxim in scientific philosophizing is this:

Wherever possible, logical constructions are to be substituted for inferred entities.

Some examples of the substitution of construction for inference in the realm of mathematical philosophy may serve to elucidate the uses of this maxim. Take first the case of irrationals. In old days, irrationals were inferred as the supposed limits of series of rationals which had no rational limit; but the objection to this procedure was that it left the existence of irrationals merely optative, and for this reason the stricter methods of the present day no longer tolerate such a definition. We now define an irrational number as a certain class of ratios, thus constructing it logically by means of ratios, instead of arriving at it by a doubtful inference from them. Take again the case of cardinal numbers. Two equally numerous collections appear to have something in common: this something is supposed to be their cardinal number. But so long as the cardinal number is inferred from the collections, not constructed in terms of them, its existence must remain in doubt, unless in virtue of a metaphysical postulate ad hoc . By defining the cardinal number of a given collection as the class of all equally numerous collections, we avoid the necessity of this metaphysical postulate, and thereby remove a needless element of doubt from the philosophy of arithmetic. A similar method, as I have shown elsewhere, can be applied to classes themselves, which need not be supposed to have any metaphysical reality, but can be regarded as symbolically constructed fictions.

The method by which the construction proceeds is closely analogous in these and all similar cases. Given a set of propositions nominally dealing with the supposed inferred entities, we observe the properties which are required of the supposed entities in order to make these propositions true. By dint of a little logical ingenuity, we then construct some logical function of less hypothetical entities which has the requisite properties. The constructed function we substitute for the supposed inferred entities, and thereby obtain a new and less doubtful interpretation of the body of propositions in question. This method, so fruitful in the philosophy of mathematics, will be found equally applicable in the philosophy of physics, where, I do not doubt, it would have been applied long ago but for the fact that all who have studied this subject hitherto have been completely ignorant of mathematical logic. I myself cannot claim originality in the application of this method to physics, since I owe the suggestion and the stimulus for its application entirely to my friend and collaborator Dr Whitehead, who is engaged in applying it to the more mathematical portions of the region intermediate between sense-data and the points, instants and particles of physics.

A complete application of the method which substitutes constructions for inferences would exhibit matter wholly in terms of sense-data, and even, we may add, of the sense-data of a single person, since the sense-data of others cannot be known without some element of inference. This, however, must remain for the present an ideal, to be approached as nearly as possible, but to be reached, if at all, only after a long preliminary labour of which as yet we can only see the very beginning. ( RSDP , 115–6) { §6.3 }

In the special sciences, when they have become fully developed, the movement is forward and synthetic, from the simpler to the more complex. But in philosophy we follow the inverse direction: from the complex and relatively concrete we proceed towards the simple and abstract by means of analysis, seeking, in the process, to eliminate the particularity of the original subject-matter, and to confine our attention entirely to the logical form of the facts concerned. ( OKEW , 189–90) { §6.3 }

The nature of philosophic analysis … can now be stated in general terms. We start from a body of common knowledge, which constitutes our data. On examination, the data are found to be complex, rather vague, and largely interdependent logically. By analysis we reduce them to propositions which are as nearly as possible simple and precise, and we arrange them in deductive chains, in which a certain number of initial propositions form a logical guarantee for all the rest. ( OKEW , 214) { §6.3 }

the chief thesis that I have to maintain is the legitimacy of analysis. ( PLA , 189) { §6.3 }

it is very important to distinguish between a definition and an analysis. All analysis is only possible in regard to what is complex, and it always depends, in the last analysis, upon direct acquaintance with the objects which are the meanings of certain simple symbols. It is hardly necessary to observe that one does not define a thing but a symbol. ( PLA , 194) { §6.3 }

Analysis is not the same thing as definition. You can define a term by means of a correct description, but that does not constitute an analysis. ( PLA , 196) { §6.3 }

The business of philosophy, as I conceive it, is essentially that of logical analysis, followed by logical synthesis. ( LA , 341) { §6.3 }

Ever since I abandoned the philosophy of Kant and Hegel, I have sought solutions of philosophical problems by means of analysis; and I remain firmly persuaded, in spite of some modern tendencies to the contrary, that only by analysing is progress possible. ( MPD , 11) { §6.3 }

Philosophy must then involve the exercise of systematic restatement. But this does not mean that it is a department of philology or literary criticism.

Its restatement is not the substitution of one noun for another or one verb for another. That is what lexicographers and translators excel in. Its restatements are transmutations of syntax, and transmutations of syntax controlled not be desire for elegance or stylistic correctness but by desire to exhibit the forms of the facts into which philosophy is the enquiry.

I conclude, then, that there is, after all, a sense in which we can properly enquire and even say “what it really means to say so and so”. For we can ask what is the real form of the fact recorded when this is concealed or disguised and not duly exhibited by the expression in question. And we can often succeed in stating this fact in a new form of words which does exhibit what the other failed to exhibit. And I am for the present inclined to believe that this is what philosophical analysis is, and that this is the sole and whole function of philosophy. (1932, 100) { §6.8 }

I have no special objection to or any special liking for the fashion of describing as ‘analysis’ the sort or sorts of conceptual examination which constitute philosophizing. But the idea is totally false that this examination is a sort of garage inspection of one conceptual vehicle at a time. On the contrary, to put it dogmatically, it is always a traffic inspector’s examination of a conceptual traffic-block, involving at least two streams of vehicles hailing from the theories, or points of view or platitudes which are at cross-purposes with one another. (1953, 32) { §6.8 }

It is certain that when I wrote “Systematically Misleading Expressions” I was still under the direct influence of the notion of an “ideal language”—a doctrine according to which there were a certain number of logical forms which one could somehow dig up by scratching away at the earth which covered them. I no longer think, especially not today, that this is a good method. I do not regret having traveled that road, but I am happy to have left it behind me. (In Rorty 1967, 305) { §6.8 }

alas! intellect must first destroy the object of Inner Sense if it would make it its own. Like the analytical chemist, the philosopher can only discover how things are combined by analysing them, only lay bare the workings of spontaneous Nature by subjecting them to the torment of his own techniques. In order to lay hold of the fleeting phenomenon, he must first bind it in the fetters of rule, tear its fair body to pieces by reducing it to concepts, and preserve its living spirit in a sorry skeleton of words. Is it any wonder that natural feeling cannot find itself again in such an image, or that in the account of the analytical thinker truth should appear as paradox? ( AE , I, 4) { §5.2 }

analysis without synopsis must be blind. (‘Time and the World Order’, in Herbert Feigl and Grover Maxwell, (eds.), Minnesota Studies in the Philosophy of Science III , Minneapolis: University of Minnesota Press, 1962, 527)

[in discussing Ryle 1953 { Quotation }] Personally, I have no axe to grind about what it takes to analyze a concept. Very likely, there are different sorts of cases. It may well be that sometimes what we want from an analysis is the tracing of the sort of intricate web of conceptual relations in which Ryle delights. But there is little reason for thinking that this is always so—at least, if analysis is construed as whatever it is that philosophers do to solve their problems. What strikes me as worrisome is Ryle’ tendency to use the web metaphor as a rationale for rejecting the old, Russellian conception of analysis, with its emphasis on precisely formulated logical forms, and replacing it with methodology which, in some cases, may degenerate into a recipe for generating a conceptual fog. It is all well and good to recognize that sometimes the concepts philosophers deal with will be vague, imprecise, and open-ended, with close conceptual connections to other concepts of the same sort. We do have to be able to deal with such cases—perhaps along the lines Ryle suggests. What is not good is a prior ideological commitment to blurred edges, indirectness, and an unwillingness to separate tangential from central issues. Sometimes Ryle and other ordinary language philosophers seem to go too far in this direction; substituting one confining orthodoxy about analysis for another. When this happens, central philosophical points get missed ... (2003, II, 80–1) { §6.1 }

Philosophical analysis is a term of art. At different times in the twentieth century, different authors have used it to mean different things. What is to be analyzed (e.g., words and sentences versus concepts and propositions), what counts as a successful analysis, and what philosophical fruits come from analysis are questions that have been vigorously debated since the dawn of analysis as a self-conscious philosophical approach. Often, different views of analysis have been linked to different views of the nature of philosophy, the sources of philosophical knowledge, the role of language in thought, the relationship between language and the world, and the nature of meaning—as well to more focused questions about necessary and apriori truth. Indeed the variety of positions is so great as to make any attempt to extract a common denominator from the multiplicity of views sterile and not illuminating.

Nevertheless analytic philosophy—with its emphasis on what is called “philosophical analysis”—is a clear and recognizable tradition. Although the common core of doctrine uniting its practitioners scarcely exceeds the platitudinous, a pattern of historical influence is not hard to discern. The tradition begins with G.E. Moore, Bertrand Russell, and Ludwig Wittgenstein (as well as Gottlob Frege, whose initial influence was largely filtered through Russell and Wittgenstein). These philosophers set the agenda, first, for logical positivists such as Rudolf Carnap, Carl Hempel, and A.J. Ayer and then later for Wittgenstein, who in turn ushered in the ordinary language school led by Gilbert Ryle and J.L. Austin. More recently the second half of the twentieth century has seen a revival of Russellian and Carnapian themes in the work of W.V. Quine, Donald Davidson, and Saul Kripke. Analytic philosophy, with its changing views of philosophical analysis, is a trail of influence ... (2005, 144) { §6.1 }

the mathematical method is characteristic of much of Western philosophy, whereas the grammatical method is characteristic of much of Indian philosophy. (1965, 99) { §2.6 }

Discoveries of this kind [of the dependence of systems of categories on the language in which they are formulated] open up new vistas. They emphasize the desirability that philosophers should take full account of linguistics. With the help of linguistics, philosophy is in a position to enter a fruitful area of research. In this respect, philosophers are in a more favourable position that mathematicians—for the latter investigate what they have first created, while the former face the richness and variety of natural languages, where reality surpasses the boldest imagination. Mathematicians can never enter other spaces than the one in which they were born, not even with the advancement of space travel. At most, they can propose to physicists that they should describe physical space with the help of another geometry. Philosophers, on the other hand, can learn a language and thereby enter a new world of experience: the linguistic categories of a newly learned language may not segment reality in the same way as do the categories Western philosophers are accustomed to. Philosophers obtain passports for non-Aristotelian worlds as soon as they begin to study the syntax of a language which is sufficiently different from Greek. Of course, the fruitfulness of such research increases if the language studied belongs to a civilization which has also produced philosophy, logic, and linguistics or related fields of study. (1965, 105) { §2.6 }

Historically speaking, Pāṇini’s method has occupied a place comparable to that held by Euclid’s method in Western thought. Scientific developments have therefore taken different directions in India and in the West. Pāṇini’s system produced at an early date such logical distinctions as those between language and metalanguage, theorem and metatheorem, use and mention, which were discovered much later in Europe. In other Indian sciences, e.g., in mathematics and astronomy, as well as in later grammatical systems of Sanskrit, Prakrit, and Tamil, systematic abbreviations are used which not only are ingenious but also constitute new adaptations of the same method. In India, Pāṇini’s perfection and ingenuity have rarely been matched outside the realm of linguistics. In the West, this corresponds to the belief that mathematics is the most perfect among the sciences. Just as Plato reserved admission to his Academy for geometricians, Indian scholars and philosophers are expected to have first undergone a training in scientific linguistics. In India, grammar was called the Veda of the Vedas, the science of sciences. Renou declares: ‘To adhere to Indian thought means first of all to think like a grammarian’ ... This has determined the form and method of a large part of Indian philosophy, an important feature which is generally lost when Sanskrit originals are translated into Western languages. It seems almost unavoidable that translations of an accurate original should therefore appear vague. (1965, 114) { §2.6 }

In my opinion Logical Positivism fails in its treatment of analysis. Wittgenstein and the other Logical Positivists talk much about analysis, but they do not consider the various kinds of analysis, nor do they show in what sense philosophy is the analysis of facts. They make use of analytic definition of a symbolic expression, and of the analytic clarification of a concept, but they do not distinguish between them. They also employ postulational analysis. But they do not seem to understand directional analysis, and, accordingly, they fail to apprehend the need for it. In this way they depart, in my opinion, from the practice of Moore. Not only is their conception of analysis defective, but, further, their conception of the kinds of facts to be analysed is inadequate. They treat all facts as linguistic facts . Hence, they suppose that the first problem of philosophy is to determine the principles of symbolism, and from these principles to draw limits with regard to what we can think. This assumption has two important consequences. First, it leads to the view that philosophy is ‘the activity of finding meaning’, to quote Schlick’s statement. The second consequence is that they are apt to place too much reliance upon the construction of postulational systems. (1933b, 82–3) { §6.6 }

Strawson, Peter F.

An analysis, I suppose, may be thought of as a kind of breaking down or decomposing of something. So we have the picture of a kind of intellectual taking to pieces of ideas or concepts; the discovering of what elements a concept or idea is composed and how they are related. Is this the right picture or the wrong one—or is it partly right and partly wrong? That is a question which calls for a considered response … ( Analysis and Metaphysics , Oxford: Oxford University Press, 1992, 2)

If we took this notion [of analysis as decomposition] completely seriously for the case of conceptual analysis—analysis of ideas—we should conclude that our task was to find ideas that were completely simple, that were free from internal conceptual complexity; and then to demonstrate how the more or less complex ideas that are of interest to philosophers could be assembled by a kind of logical or conceptual construction out of these simple elements. The aim would be to get a clear grasp of complex meanings by reducing them, without remainder, to simple meanings. Thus baldly stated, this may seem a rather implausible project. And so it is. Nevertheless it, or some close relation of it, has been, and is, taken seriously. Even when not taken to the lengths I have just described, it continues to exercise a certain influence on the philosophical mind. ( Ibid. 18)

Among the philosophers who were most influential in England in the period between the two world wars were the analysts. Their analytic theories were sometimes associated with the metaphysical view which Russell called logical atomism, sometimes with the supposedly anti-metaphysical doctrines of logical positivism, and sometimes, as in the case of G. E. Moore, the analytic practice had no clearly defined dogmatic background at all. But they were united at least in the view that analysis was at least one of the most important tasks of the philosopher; and by analysis they meant something which, whatever precise description of it they chose, at least involved the attempt to rewrite in different and in some way more appropriate terms those statements which they found philosophically puzzling. (1956, vii) { §6.1 }

analysis is a familiar philosophical method. I shall not attempt to offer you a complete historical account of analytic philosophy. Even the minute examination of a particular analytic philosopher, or group of analytic philosophers, would not be of great interest. I propose rather to sketch, in broad strokes, four major forms of philosophical analysis which I think important to distinguish carefully from one another. I shall call the first of these: classical analysis. It corresponds, roughly, to the traditional method of analysis used by English philosophers, a method which Russell did so much to develop. I shall then examine three other, more recent forms of philosophical analysis: (1) the type of analysis which involves the construction of artificial languages; (2) the type of analysis practiced by Wittgenstein in his later period; (3) the type of analysis which characterizes present-day Oxford Philosophy.

The fundamental notion of classical analysis is that propositions couched in ordinary language are correct, in the sense that they are not objectionable in principle. They are neither logically nor metaphysically absurd. On the other hand, insofar as the form of these propositions of ordinary language hides their true meaning, they are neither metaphysically nor logically satisfactory. The task of the analyst is, therefore, to reformulate them so that this meaning will be clearly and explicitly presented, rather then to reject them. To analyze, is to reformulate,—to translate into a better wording. (1962, 294–5) { §6.1 }

The logical positivism of the Vienna Circle did not modify the methodology of classical analysis. However, because of the anti-metaphysical standpoint which was characteristic of positivism, it could not accept the notion of the goal of analysis as metaphysical discovery. For the positivists of this school, the goal of philosophical analysis is to clarify the language of science, a clarification which would result from, for example, elucidating the relationships between observation and theory, or between scientific concepts at different levels of abstraction. ( Ibid ., 296) { §6.1 }

A second school [or third school, after ‘classical analysis’ and logical positivism] was inspired (largely, but not entirely) by the thought of Wittgenstein in his later period. Wittgenstein had himself been led to this new point of view in his criticism of his own Tractatus Logico-Philosophicus ( Logische-Philosophische Abhandlung ), a book which itself espoused implicitly a certain form of classical analysis. According to Wittgenstein, classical analysis rested upon a false conception of language and of thought. ...

... for an analyst of this sort, philosophical problems do not result from ignorance of the precise meaning of a concept, but from an entirely false conception of its function. ... Such a false conception is what Ryle calls a “category mistake”. To resolve a philosophical problem, one should exhibit the generic character of the concepts involved in it, rather than attempting to give a perfect definition or explication of these concepts. ...

This conception of philosophical analysis—of analysis as the resolution of conceptual enigmas—has sometimes been condescendingly called “therapeutic positivism”. ( Ibid ., 297–9) { §6.1 }

The fourth method of analysis ... is that of Oxford Philosophy. ...

The analytic philosophers of the Cambridge School—for example, Russell and Wittgenstein—came to philosophy after considerable work in the sciences and in mathematics. Philosophy of mathematics was the first topic to which Russell applied his classical method of analysis. But the Oxford philosophers came to their subject, almost without exception, after extensive study of classics. Thus they were naturally interested in words, in syntax, and in (idioms. They did not wish to use linguistic analysis simply to resolve philosophical problems; they were interested in the study of language for its own sake. Therefore these philosophers are, perhaps, both more given to making linguistic distinctions, and better at finding such distinctions, than most. Ibid ., 299) { §6.1 }

Many English philosophers (including many who owe allegiance to Oxford Philosophy) would place themselves at a position between that of Wittgenstein and the view I have just sketched. It may therefore be in point to indicate briefly the principal differences between the two schools:

(1) Wittgensteinian analysis has, for its sole end, the resolution of philosophical enigmas. If there were no such enigmas, there would be no need for analysis. For Oxford, on the other hand, analysis has an intrinsic value.

(2) According to Wittgenstein and his disciples, all that is necessary is to exhibit the generic character of the concepts which we analyze. For Oxford, a minute analysis is indispensable.

(3) For Wittgenstein, analysis is the only useful method in philosophy. For Oxford, it is only one among others, and no one claims that it is sufficient, by itself, to resolve all philosophical problems. ( Ibid ., 301) { §6.1 }

It is not sensible to ask for the method of making one‘s fortune (or of ruining oneself); there are many. It is no more sensible to ask “What is the analytical method?” There is not one “analytic philosophy”. There are several. ( Ibid ., 301 [closing sentences]) { §6.1 }

For explaining Dharmakīrti’s approach it is necessary to distinguish four levels of philosophical analysis in ascending order of sophistication. At the lowest level we begin with the perspective of ordinary, unenlightened beings . Their view of the world is not to be faulted to the extent to which it is largely pragmatically successful: it allows them to successfully interact with the world. However, from a philosophical perspective it leaves much to be desired, as it is characterized by the chief fault of satkāyadr ṣ ṭi , the mistaken superimposition of a substantial self where there is none, both in the case of persons, as well as in the case of other phenomena. At the second level of the scale we come to the reductionist view that we find exemplified in the Abhidharma. Both persons as well as other partite objects are analysed and found to be nothing but convenient verbal designations sitting on top of what is ultimately real, namely conglomerations of fundamentally existing dharmas . At this level, some elements are still characterized by spatial, temporal, or conceptual extension. Some objects, such as colours, are spread out in space, some objects have temporal extension, and most importantly, some qualities of objects are shared across different instances of them: all earth-atoms are solid, all water-atoms wet, and so on. By and large this perspective accords with the Sarvāstivāda view we find in Vasubandhu’s Abhidharmakośa . At the third level the reductionist perspective is further refined into a form of particularism . According to this position all three forms of extension are given up because they are considered to be the products of cognitive errors. We perceive objects as spatially extended because we confuse qualities of the mental image of the object with the qualities of what gives rise to the mental image. The assumption of temporal extension is an artefact of the slowness of our perceptual system. Because we cannot keep up with the rapid succession that marks the change of things, we simply group together various successive phenomena that form part of a single causal chain and construe it as one temporally persisting object. The same happens in the case of shared objects in general or object-types. Even though every particular is different from any other particular, we are often not able to register the differences between distinct things. As in the case of temporal resolution, the comparative coarseness of our conceptual resolution causes us to lump together various distinct, though similar things. So despite the fact that all there is out there in the world is a variety of things such as earth-atoms that are distinct from one another, on account of some similarity we put them all together and argue that they all instantiate the same object-type of solidity. This view is often referred to as a Sautrāntika position, and the emphasis on the extremely short-lived nature of all objects seems to justify this, even though, as noted before, it is difficult to be precise about the distinction between this form of Sautrāntika, the form that we find in Vasubandhu, and those coming from sources preceding Vasubandhu. This particularist stance is the philosophical position from which Dharmakīrti constructs most of his arguments. This is a curious fact, since it does not represent his final view, the position he wants to endorse after discussing various other positions that are all in some way defective. For if we push our philosophical analysis yet further we get to a fourth level, an idealist theory , according to which the duality between the perceiving subject and the non-material perceived object is illusory. All phenomena have only one nature, and this nature is mental. The affinity of this view with Yogācāra positions is obvious. Despite the fact that this is the position Dharmakīrti wants to endorse, in the end it does not dominate his philosophical exposition. In fact there is only one substantial section of the Pramāṇavārttika where he employs it consistently as a background for his argumentation.

This sequence of four positions along the sliding scale of analysis is interesting for a number of reasons. On the one hand it mirrors the historical development of Buddhist thought in India, from the confrontation with non-Buddhist believers in a substantial ātman through Abhidharma reductionism, a thoroughgoing form of particularism, up to the idealism of Yogācāra. Yet this sequence is at the same time considered to be a conceptual hierarchy, an ascent to better and better philosophical theories or, what amounts to the same thing in the Buddhist context, a hierarchy of views that result in less and less erroneous superimposition ( samāropa ). It is obvious how the Abhidharma reductionism is supposed to remove clinging to the mistaken belief in a substantial self where there is none. Yet, as the partcularist stage argues, the reductionist is still bound by superimposing spatial, temporal, and conceptual extension to a world consisting of non-extended, momentary, and utterly distinct particulars. Removing those frees us from further superimpositions, and thereby from the potential for further clinging, clinging that in turn leads to suffering and continuing entanglement in cyclic existence. Further superimposition takes place when the appearance of external objects is superimposed on some purely mental phenomena, thereby causing the particularist picture in the first place. A thoroughgoing removal of superimpositions must also dispense with the erroneous distinction between perceiving subject and perceived object.

A single argumentative pattern can be understood as the driving force behind the movement through the four levels. This is the neither-one-nor-many argument, well known throughout the history of Buddhist philosophy. When applied to the perspective of ordinary beings, this argument begins with the question whether an object and its parts are identical or different. It appears that they cannot be identical (since the object is one and the parts are many, and one thing cannot have contradictory properties), and that they cannot be different (as the whole is never found as a separate entity distinct from the parts). The reductionist argues that we should conclude from this that wholes are not real in the first place, but merely conceptually constructed pseudo-entities. the same considerations can then be applied to particulars and properties they supposedly share (here a key argument is that distinct shared properties would have to be permanent, conflicting with the principle of momentariness), and to the perceiving object and perception (if they are distinct, why do we never encounter one without the other?). ( The Golden Age of Indian Buddhist Philosophy , Oxford: Oxford University Press, 2018, 253–5)

The primary weapon is analysis. And analysis is the evocation of insight by the hypothetical suggestions of thought, and the evocation of thought by the activities of direct insight. In this process the composite whole, the interrelations, and the things related, concurrently emerge into clarity. ( Essays in Science and Philosophy , New York: Philosophical Library, 1947, 157)

Analysis is often understood to imply a whole of which the parts are explicitly known before the analysis; but logical elements are for our ordinary consciousness only implicit: we use them without reflecting on them, just as we use grammatical distinctions long before we have any knowledge of grammar. Logic does not merely analyse: it makes explicit what was implicit. ( Statement and Inference , Oxford: Oxford University Press, 1926, 49)

The hypothetical process therefore combines in itself both the method of discovery and the proof, and is the proper scientific exposition. The non-hypothetical proof to which we are accustomed is a sort of scientific pedantry, and it is consequently a great mistake first to give what is called analysis, which corresponds to the hypothetical process, and then to follow it by a synthesis, which is the non-hypothetical part, thus putting aside analysis as if it were a sort of accident. It is an error because it conceals the true process of thinking. ( Ibid. , 560)

I have changed my views on “atomic” complexes: I now think that qualities, relations (like love) etc. are all copulae! That means I for instance analyse a subject-predicate proposition, say, “Socrates is human” into “Socrates” and “something is human”, (which I think is not complex). The reason for this is a very fundamental one. I think that there cannot be different Types of things! In other words whatever can be symbolized by a simple proper name must belong to one type. And further: every theory of types must be rendered superfluous by a proper theory of symbolism: For instance if I analyse the proposition Socrates is mortal into Socrates, mortality and (∃x,y) ∈ 1 (x,y) I want a theory of types to tell me that “mortality is Socrates” is nonsensical, because if I treat “mortality” as a proper name (as I did) there is nothing to prevent me to make the substitution the wrong way round. But if I analyse (as I do now) into Socrates and (∃x).x is mortal or generally into x and (∃x) φx it becomes impossible to substitute the wrong way round because the two symbols are now of a different kind themselves. What I am most certain of is not however the correctness of my present way of analysis, but of the fact that all theory of types must be done away with by a theory of symbolism showing that what seem to be different kinds of things are symbolized by different kinds of symbols which cannot possibly be substituted in one another’s places. I hope I have made this fairly clear!

Propositions which I formerly wrote ∈ 2 (a,R,b) I now write R(a,b) and analyse them into a,b and (∃x,y)R(x,y) [with (∃x,y)R(x,y) marked in the text as “not complex”] ( NB , 121–2) { §6.5 }

How is it reconcilable with the task of philosophy, that logic should take care of itself? If, for example, we ask: Is such and such a fact of the subject-predicate form?, we must surely know what we mean by “subject-predicate form”. We must know whether there is such a form at all. How can we know this? “From the signs”. But how? For we haven’t got any signs of this form. We may indeed say: We have signs that behave like signs of the subject-predicate form, but does that mean that there really must be facts of this form? That is, when those signs are completely analysed? And here the question arises again: Does such a complete analysis exist? And if not : then what is the task of philosophy?!!? ( NB , 2) { §6.5 }

Our difficulty now lies in the fact that to all appearances analysability, or its opposite, is not reflected in language. That is to say: We can not , as it seems, gather from language alone whether for example there are real subject-predicate facts or not. But how COULD we express this fact or its opposite? This must be shewn . ( NB , 10) { §6.5 }

The trivial fact that a completely analysed proposition contains just as many names as there are things contained in its reference [ Bedeutung ]; this fact is an example of the all-embracing representation of the world through language. ( NB , 11) { §6.5 }

The completely analysed proposition must image its reference [ Bedeutung ]. ( NB , 18) { §6.5 }

A question: can we manage without simple objects in LOGIC?

Obviously propositions are possible which contain no simple signs, i.e. no signs which have an immediate reference [ Bedeutung ]. And these are really propositions making sense, nor do the definitions of their component parts have to be attached to them.

But it is clear that components of our propositions can be analysed by means of a definition, and must be, if we want to approximate to the real structure of the proposition. At any rate, then, there is a process of analysis . And can it not now be asked whether this process comes to an end? And if so: What will the end be?

If it is true that every defined sign signifies via its definitions then presumably the chain of definitions must some time have an end. [Cf. TLP 3.261.]

The analysed proposition mentions more than the unanalysed.

Analysis makes the proposition more complicated than it was, but it cannot and must not make it more complicated than its meaning [ Bedeutung ] was from the first.

When the proposition is just as complex as its reference [ Bedeutung ], then it is completely analysed.

But the reference [ Bedeutung ] of our propositions is not infinitely complicated. ( NB , 46) { §6.5 }

But it also seems certain that we do not infer the existence of simple objects from the existence of particular simple objects, but rather know them—by description, as it were—as the end-product of analysis, by means of a process that leads to them. ( NB , 50) { §6.5 }

Let us assume that every spatial object consists of infinitely many points, then it is clear that I cannot mention all these by name when I speak of that object. Here then would be a case in which I cannot arrive at the complete analysis in the old sense at all; and perhaps just this is the usual case.

But this is surely clear: the propositions which are the only ones that humanity uses will have a sense just as they are and do not wait upon a future analysis in order to acquire a sense.

Now, however, it seems to be a legitimate question: Are–e.g.–spatial objects composed of simple parts; in analysing them, does one arrive at parts that cannot be further analysed, or is this not the case?

—But what kind of question is this?—

Is it , A PRIORI, clear that in analysing we must arrive at simple components—is this, e.g., involved in the concept of analysis— , or is analysis ad infinitum possible?—Or is there in the end even a third possibility? ( NB , 62) { §6.5 }

In a proposition a thought can be so expressed that to the objects of the thought correspond elements of the propositional sign.

I call these elements ‘simple signs’ and the proposition ‘completely analysed’. ( TLP , 3.2, 3.201) { §6.5 }

There is one and only one complete analysis of a proposition. ( TLP , 3.25) { §6.5 }

It is obvious that, in the analysis of propositions, we must arrive at elementary propositions, which consist of names in immediate combination.

This raises the question as to how the propositional unity comes about. ( TLP , 4.221) { §6.5 }

If we know on purely logical grounds that there must be elementary propositions, then everyone who understands propositions in their unanalysed form must know it. ( TLP , 5.5562) { §6.5 }

The correct method in philosophy would really be this: to say nothing except what can be said, that is, propositions of natural science—that is, something that has nothing to do with philosophy; and then, whenever someone else wanted to say something metaphysical, to demonstrate to them that they had not given a meaning to certain signs in their propositions. This method would be unsatisfying to them—they would not have the feeling that we were teaching them philosophy—but  this  would be the only strictly correct one. ( TLP , 6.53) { §6.5 }

My propositions elucidate when someone who understands me finally recognizes them as nonsensical by using them to climb up, over, and out of them. (They must throw away the ladder, so to speak, having used it to climb up.) They must get over these propositions, and then they see the world correctly. ( TLP , 6.54) { §6.5 }

A proposition is completely logically analysed if its grammar is made completely clear: no matter what idiom it may be written or expressed in. ( PR , 51; cf. BT , 308) { §6.5 }

Logical analysis is the analysis of something we have, not of something we don’t have. Therefore it is the analysis of propositions as they stand . ( PR , 52) { §6.5 }

a mathematical proof is an analysis of the mathematical proposition. ( PR , 179) { §6.5 }

Complex is not like fact. For I can, e.g., say of a complex that it moves from one place to another, but not of a fact.

But that this complex is now situated here is a fact. ...

A complex is composed of its parts, the things of a kind which go to make it up. (This is of course a grammatical proposition concerning the words ‘complex’, ‘part’ and ‘compose’.)

To say that a red circle is composed of redness and circularity, or is a complex with these component parts, is a misuse of these words and is misleading. (Frege was aware of this and told me.) It is just as misleading to say the fact that this circle is red (that I am tired) is a complex whose component parts are a circle and redness (myself and tiredness).

Neither is a house a complex of bricks and their spatial relations; i.e. that too goes against the correct use of the word. ( PR , 301–2) { §6.5 }

When I say: “My broom is in the corner”,—is this really a statement about the broomstick and the brush? Well, it could at any rate be replaced by a statement giving the position of the stick and the position of the brush. And this statement is surely a further analysed form of the first one.—But why do I call it “further analysed”?—Well, if the broom is there, that surely means that the stick and brush must be there, and in a particular relation to one another; and this was as it were hidden in the sense of the first sentence, and is expressed in the analysed sentence. Then does someone who says that the broom is in the corner really mean: the broomstick is there, and so is the brush, and the broomstick is fixed in the brush?—If we were to ask anyone if he meant this he would probably say that he had not thought specially of the broomstick or specially of the brush at all. And that would be the right answer, for he meant to speak neither of the stick nor of the brush in particular. Suppose that, instead of saying “Bring me the broom”, you said “Bring me the broomstick and the brush which is fitted on to it.”!—Isn’t the answer: “DO you want the broom? Why do you put it so oddly?”——Is he going to understand the further analysed sentence better?—This sentence, one might say, achieves the same as the ordinary one, but in a more roundabout way.— Imagine a language-game in which someone is ordered to bring certain objects which are composed of several parts, to move them about, or something else of that kind. And two ways of playing it: in one (a) the composite objects (brooms, chairs, tables, etc.) have names, as in (15); in the other (b) only the parts are given names and the wholes are described by means of them.—In what sense is an order in the second game an analysed form of an order in the first? Does the former lie concealed in the latter, and is it now brought out by analysis?—True, the broom is taken to pieces when one separates broomstick and brush; but does it follow that the order to bring the broom also consists of corresponding parts? ...

To say, however, that a sentence in (b) is an ‘analysed’ form of one in (a) readily seduces us into thinking that the former is the more fundamental form; that it alone shews what is meant by the other, and so on. For example, we think: If you have only the unanalysed form you miss the analysis; but if you know the analysed form that gives you everything.—But can I not say that an aspect of the matter is lost on you in the latter case as well as the former? ( PI , §§ 60, 63) { §6.5 }

Our investigation is therefore a grammatical one. Such an investigation sheds light on our problem by clearing misunderstandings away. Misunderstandings concerning the use of words, caused, among other things, by certain analogies between the forms of expression in different regions of language.—Some of them can be removed by substituting one form of expression for another; this may be called an “analysis” of our forms of expression, for the process is sometimes like one of taking a thing apart.

But now it may come to look as if there were something like a final analysis of our forms of language, and so a single completely resolved form of every expression. That is, as if our usual forms of expression were, essentially, unanalysed; as if there were something hidden in them that had to be brought to light. When this is done the expression is completely clarified and our problem solved.

It can also be put like this: we eliminate misunderstandings by making our expressions more exact; but now it may look as if we were moving towards a particular state, a state of complete exactness; and as if this were the real goal of our investigation. ( PI , §§ 90–1) { §6.5 }

We are not analysing a phenomenon (e.g. thought) but a concept (e.g. that of thinking), and therefore the use of a word. ( PI , §383) { §6.5 }

A list of key works on analysis (monographs and collections) can be found in the

Annotated Bibliography, §1.2 .

Copyright © 2024 by Michael Beaney < michael . beaney @ hu-berlin . de > Thomas Raysmith < t . h . raysmith @ gmail . com >

• Accessibility

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2024 by The Metaphysics Research Lab , Department of Philosophy, Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

Systematic Reviews and Other Evidence Synthesis Types Guide

• Systematic Review and Other Evidence Synthesis Types
• Types of Evidence Synthesis
• Evidence Synthesis Comparison
• Are You Ready to Conduct an Evidence Synthesis?
• UT Southwestern Evidence Synthesis Services
• Task 1 - Find Articles
• Task 2 - Formulate Question
• Task 3 - Select Reporting Guideline
• Task 4 - Write and Register Protocol
• Evidence Synthesis - Search (Task 5)
• Screen and Appraise (Tasks 6 – 11)
• Synthesize (Tasks 12 – 15)
• Write Up Review (Task 16)
• Systematic Review or Meta-Analysis
• Integrative Review
• Narrative/Literature Review
• Rapid Review
• Scoping Review
• Umbrella Review

Request UT Southwestern Library Evidence Synthesis/Systematic Review Services

The UT Southwestern Librarians provide two levels of Evidence Synthesis/Systematic Review (ES/SR) support.

Level 1 – Education (No Cost)

• A librarian will provide training about the systematic review process.
• Use the Training Request Form .

Level 2 – Librarian As ES/SR Team Member and Co-Author (Fee-Based)

• The librarian is an active contributor.
• UT Southwestern faculty
• UT Southwestern residents or fellows
• UT Southwestern Medical Center and University Hospitals clinicians
• Begin by completing the Evidence Synthesis/Systematic Review Request Form . For more information on the fees ($1,250 per PICO or equivalent question), see the "Costs" section in the form.
• If a Librarian joins the ES/SR Team, the ES/SR Team will complete the Evidence Synthesis/Systematic Review Library Services Agreement .
• Contact LibAsk Schedule an appointment with UT Southwestern librarians.

What Is Evidence Synthesis?

Definitions of evidence synthesis.

Evidence synthesis is a general term that captures a widening universe of methodologies….Unlike these traditional narrative reviews, evidence synthesis aims to reduce biases in the process of selecting the studies that will be included in a review. Evidence synthesis uses transparent and reproducible methods to exhaustively search for information on a topic and select studies based on well-defined predetermined criteria. Depending on the type and purpose of the evidence synthesis, results from the studies that meet the criteria may be assessed for quality and bias and the results combined in a meta-analysis to reach a more complete understanding. Other evidence synthesis methods use systematic searching approaches but skip a quality assessment step and instead map, describe, or distill the existing knowledge to guide future research directions and provide decision support tools for policy-makers and practitioners. Due to the thorough nature of searching and combing through the literature, evidence synthesis typically takes longer to complete than narrative reviews do, though exceptions may exist. Eldermire, E., & Young, Y., 2022, p. 17-18

Evidence Synthesis International Position Statement

Evidence synthesis can be defined as the review of what is known from existing research using systematic and explicit methods in order to clarify the evidence base and is the main focus of this journal. The research questions addressed and the methods used by such syntheses vary considerably, but they are all based on the principles of rigour and transparency. When reviews of research evidence are based on expert advice, expert panels and unsystematic methods, then the basis for the claims made by the reviews are uncertain. If reviews are not explicit and transparent in reporting their methods of review, then one cannot assess whether they have used rigorous methods that can justify the findings that they report. Grough, Davies, Jamtvedt, et al. 2020

WHO Regional Office for Europe

Evidence synthesis is an approach to integrating findings from peer-reviewed and grey literature to summarize a substantive and diverse body of evidence. Evidence synthesis is characterized by its systematic and transparent (i.e. replicable and observable) approach to formulating questions and searching, appraising, synthesizing and packaging the body of evidence to provide a more comprehensive picture than a single study could do. This means that the methodology used (e.g. search terms, sources/databases, inclusion/exclusion criteria) is explicitly documented to leave a trail for others to replicate the search, make updates easier and assist readers to be aware of any potential bias. Karlsson, L. E., & Takahashi, R. 2017

The explosion of the evidence synthesis methods can lead to confusion. Before starting an evidence synthesis, it is essential to select the right approach to synthesize the evidence based on your question or topic. The next tab reviews types of evidence synthesis.

• Gough, D., Davies, P., Jamtvedt, G., Langlois, E., Littell, J., Lotfi, T., Masset, E., Merlin, T., Pullin, A. S., Ritskes-Hoitinga, M., Røttingen, J.-A., Sena, E., Stewart, R., Tovey, D., White, H., Yost, J., Lund, H., & Grimshaw, J. (2020). Evidence Synthesis International (ESI): position statement. Systematic Reviews , 9 (1), 155. https://doi.org/10.1186/s13643-020-01415-5
• Karlsson, L. E., & Takahashi, R. (2017). Introduction. In A Resource for Developing an Evidence Synthesis Report for Policy-Making [Internet] . WHO Regional Office for Europe. https://www.ncbi.nlm.nih.gov/books/NBK453544/
• << Previous: Systematic Review and Other Evidence Synthesis Types
• Next: Types of Evidence Synthesis >>
• Last Updated: Sep 24, 2024 12:06 PM
• URL: https://utsouthwestern.libguides.com/sres

Jump to navigation

Cochrane Training

Chapter 12: synthesizing and presenting findings using other methods.

Joanne E McKenzie, Sue E Brennan

Key Points:

• Meta-analysis of effect estimates has many advantages, but other synthesis methods may need to be considered in the circumstance where there is incompletely reported data in the primary studies.
• Alternative synthesis methods differ in the completeness of the data they require, the hypotheses they address, and the conclusions and recommendations that can be drawn from their findings.
• These methods provide more limited information for healthcare decision making than meta-analysis, but may be superior to a narrative description where some results are privileged above others without appropriate justification.
• Tabulation and visual display of the results should always be presented alongside any synthesis, and are especially important for transparent reporting in reviews without meta-analysis.
• Alternative synthesis and visual display methods should be planned and specified in the protocol. When writing the review, details of the synthesis methods should be described.
• Synthesis methods that involve vote counting based on statistical significance have serious limitations and are unacceptable.

Cite this chapter as: McKenzie JE, Brennan SE. Chapter 12: Synthesizing and presenting findings using other methods [last updated October 2019]. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.5. Cochrane, 2024. Available from www.training.cochrane.org/handbook .

12.1 Why a meta-analysis of effect estimates may not be possible

Meta-analysis of effect estimates has many potential advantages (see Chapter 10 and Chapter 11 ). However, there are circumstances where it may not be possible to undertake a meta-analysis and other statistical synthesis methods may be considered (McKenzie and Brennan 2014).

Some common reasons why it may not be possible to undertake a meta-analysis are outlined in Table 12.1.a . Legitimate reasons include limited evidence; incompletely reported outcome/effect estimates, or different effect measures used across studies; and bias in the evidence. Other commonly cited reasons for not using meta-analysis are because of too much clinical or methodological diversity, or statistical heterogeneity (Achana et al 2014). However, meta-analysis methods should be considered in these circumstances, as they may provide important insights if undertaken and interpreted appropriately.

Table 12.1.a Scenarios that may preclude meta-analysis, with possible solutions

12.2 Statistical synthesis when meta-analysis of effect estimates is not possible

A range of statistical synthesis methods are available, and these may be divided into three categories based on their preferability ( Table 12.2.a ). Preferable methods are the meta-analysis methods outlined in Chapter 10 and Chapter 11 , and are not discussed in detail here. This chapter focuses on methods that might be considered when a meta-analysis of effect estimates is not possible due to incompletely reported data in the primary studies. These methods divide into those that are ‘acceptable’ and ‘unacceptable’. The ‘acceptable’ methods differ in the data they require, the hypotheses they address, limitations around their use, and the conclusions and recommendations that can be drawn (see Section 12.2.1 ). The ‘unacceptable’ methods in common use are described (see Section 12.2.2 ), along with the reasons for why they are problematic.

Compared with meta-analysis methods, the ‘acceptable’ synthesis methods provide more limited information for healthcare decision making. However, these ‘acceptable’ methods may be superior to a narrative that describes results study by study, which comes with the risk that some studies or findings are privileged above others without appropriate justification. Further, in reviews with little or no synthesis, readers are left to make sense of the research themselves, which may result in the use of seemingly simple yet problematic synthesis methods such as vote counting based on statistical significance (see Section 12.2.2.1 ).

All methods first involve calculation of a ‘standardized metric’, followed by application of a synthesis method. In applying any of the following synthesis methods, it is important that only one outcome per study (or other independent unit, for example one comparison from a trial with multiple intervention groups) contributes to the synthesis. Chapter 9 outlines approaches for selecting an outcome when multiple have been measured. Similar to meta-analysis, sensitivity analyses can be undertaken to examine if the findings of the synthesis are robust to potentially influential decisions (see Chapter 10, Section 10.14 and Section 12.4 for examples).

Authors should report the specific methods used in lieu of meta-analysis (including approaches used for presentation and visual display), rather than stating that they have conducted a ‘narrative synthesis’ or ‘narrative summary’ without elaboration. The limitations of the chosen methods must be described, and conclusions worded with appropriate caution. The aim of reporting this detail is to make the synthesis process more transparent and reproducible, and help ensure use of appropriate methods and interpretation.

Table 12.2.a Summary of preferable and acceptable synthesis methods

12.2.1 Acceptable synthesis methods

12.2.1.1 summarizing effect estimates.

Description of method Summarizing effect estimates might be considered in the circumstance where estimates of intervention effect are available (or can be calculated), but the variances of the effects are not reported or are incorrect (and cannot be calculated from other statistics, or reasonably imputed) (Grimshaw et al 2003). Incorrect calculation of variances arises more commonly in non-standard study designs that involve clustering or matching ( Chapter 23 ). While missing variances may limit the possibility of meta-analysis, the (standardized) effects can be summarized using descriptive statistics such as the median, interquartile range, and the range. Calculating these statistics addresses the question ‘What is the range and distribution of observed effects?’

Reporting of methods and results The statistics that will be used to summarize the effects (e.g. median, interquartile range) should be reported. Box-and-whisker or bubble plots will complement reporting of the summary statistics by providing a visual display of the distribution of observed effects (Section 12.3.3 ). Tabulation of the available effect estimates will provide transparency for readers by linking the effects to the studies (Section 12.3.1 ). Limitations of the method should be acknowledged ( Table 12.2.a ).

12.2.1.2 Combining P values

Description of method Combining P values can be considered in the circumstance where there is no, or minimal, information reported beyond P values and the direction of effect; the types of outcomes and statistical tests differ across the studies; or results from non-parametric tests are reported (Borenstein et al 2009). Combining P values addresses the question ‘Is there evidence that there is an effect in at least one study?’ There are several methods available (Loughin 2004), with the method proposed by Fisher outlined here (Becker 1994).

Fisher’s method combines the P values from statistical tests across k studies using the formula:

One-sided P values are used, since these contain information about the direction of effect. However, these P values must reflect the same directional hypothesis (e.g. all testing if intervention A is more effective than intervention B). This is analogous to standardizing the direction of effects before undertaking a meta-analysis. Two-sided P values, which do not contain information about the direction, must first be converted to one-sided P values. If the effect is consistent with the directional hypothesis (e.g. intervention A is beneficial compared with B), then the one-sided P value is calculated as

In studies that do not report an exact P value but report a conventional level of significance (e.g. P<0.05), a conservative option is to use the threshold (e.g. 0.05). The P values must have been computed from statistical tests that appropriately account for the features of the design, such as clustering or matching, otherwise they will likely be incorrect.

Reporting of methods and results There are several methods for combining P values (Loughin 2004), so the chosen method should be reported, along with details of sensitivity analyses that examine if the results are sensitive to the choice of method. The results from the test should be reported alongside any available effect estimates (either individual results or meta-analysis results of a subset of studies) using text, tabulation and appropriate visual displays (Section 12.3 ). The albatross plot is likely to complement the analysis (Section 12.3.4 ). Limitations of the method should be acknowledged ( Table 12.2.a ).

12.2.1.3 Vote counting based on the direction of effect

Description of method Vote counting based on the direction of effect might be considered in the circumstance where the direction of effect is reported (with no further information), or there is no consistent effect measure or data reported across studies. The essence of vote counting is to compare the number of effects showing benefit to the number of effects showing harm for a particular outcome. However, there is wide variation in the implementation of the method due to differences in how ‘benefit’ and ‘harm’ are defined. Rules based on subjective decisions or statistical significance are problematic and should be avoided (see Section 12.2.2 ).

To undertake vote counting properly, each effect estimate is first categorized as showing benefit or harm based on the observed direction of effect alone, thereby creating a standardized binary metric. A count of the number of effects showing benefit is then compared with the number showing harm. Neither statistical significance nor the size of the effect are considered in the categorization. A sign test can be used to answer the question ‘is there any evidence of an effect?’ If there is no effect, the study effects will be distributed evenly around the null hypothesis of no difference. This is equivalent to testing if the true proportion of effects favouring the intervention (or comparator) is equal to 0.5 (Bushman and Wang 2009) (see Section 12.4.2.3 for guidance on implementing the sign test). An estimate of the proportion of effects favouring the intervention can be calculated ( p = u / n , where u = number of effects favouring the intervention, and n = number of studies) along with a confidence interval (e.g. using the Wilson or Jeffreys interval methods (Brown et al 2001)). Unless there are many studies contributing effects to the analysis, there will be large uncertainty in this estimated proportion.

Reporting of methods and results The vote counting method should be reported in the ‘Data synthesis’ section of the review. Failure to recognize vote counting as a synthesis method has led to it being applied informally (and perhaps unintentionally) to summarize results (e.g. through the use of wording such as ‘3 of 10 studies showed improvement in the outcome with intervention compared to control’; ‘most studies found’; ‘the majority of studies’; ‘few studies’ etc). In such instances, the method is rarely reported, and it may not be possible to determine whether an unacceptable (invalid) rule has been used to define benefit and harm (Section 12.2.2 ). The results from vote counting should be reported alongside any available effect estimates (either individual results or meta-analysis results of a subset of studies) using text, tabulation and appropriate visual displays (Section 12.3 ). The number of studies contributing to a synthesis based on vote counting may be larger than a meta-analysis, because only minimal statistical information (i.e. direction of effect) is required from each study to vote count. Vote counting results are used to derive the harvest and effect direction plots, although often using unacceptable methods of vote counting (see Section 12.3.5 ). Limitations of the method should be acknowledged ( Table 12.2.a ).

12.2.2 Unacceptable synthesis methods

12.2.2.1 vote counting based on statistical significance.

Conventional forms of vote counting use rules based on statistical significance and direction to categorize effects. For example, effects may be categorized into three groups: those that favour the intervention and are statistically significant (based on some predefined P value), those that favour the comparator and are statistically significant, and those that are statistically non-significant (Hedges and Vevea 1998). In a simpler formulation, effects may be categorized into two groups: those that favour the intervention and are statistically significant, and all others (Friedman 2001). Regardless of the specific formulation, when based on statistical significance, all have serious limitations and can lead to the wrong conclusion.

The conventional vote counting method fails because underpowered studies that do not rule out clinically important effects are counted as not showing benefit. Suppose, for example, the effect sizes estimated in two studies were identical. However, only one of the studies was adequately powered, and the effect in this study was statistically significant. Only this one effect (of the two identical effects) would be counted as showing ‘benefit’. Paradoxically, Hedges and Vevea showed that as the number of studies increases, the power of conventional vote counting tends to zero, except with large studies and at least moderate intervention effects (Hedges and Vevea 1998). Further, conventional vote counting suffers the same disadvantages as vote counting based on direction of effect, namely, that it does not provide information on the magnitude of effects and does not account for differences in the relative sizes of the studies.

12.2.2.2 Vote counting based on subjective rules

Subjective rules, involving a combination of direction, statistical significance and magnitude of effect, are sometimes used to categorize effects. For example, in a review examining the effectiveness of interventions for teaching quality improvement to clinicians, the authors categorized results as ‘beneficial effects’, ‘no effects’ or ‘detrimental effects’ (Boonyasai et al 2007). Categorization was based on direction of effect and statistical significance (using a predefined P value of 0.05) when available. If statistical significance was not reported, effects greater than 10% were categorized as ‘beneficial’ or ‘detrimental’, depending on their direction. These subjective rules often vary in the elements, cut-offs and algorithms used to categorize effects, and while detailed descriptions of the rules may provide a veneer of legitimacy, such rules have poor performance validity (Ioannidis et al 2008).

A further problem occurs when the rules are not described in sufficient detail for the results to be reproduced (e.g. ter Wee et al 2012, Thornicroft et al 2016). This lack of transparency does not allow determination of whether an acceptable or unacceptable vote counting method has been used (Valentine et al 2010).

12.3 Visual display and presentation of the data

Visual display and presentation of data is especially important for transparent reporting in reviews without meta-analysis, and should be considered irrespective of whether synthesis is undertaken (see Table 12.2.a for a summary of plots associated with each synthesis method). Tables and plots structure information to show patterns in the data and convey detailed information more efficiently than text. This aids interpretation and helps readers assess the veracity of the review findings.

12.3.1 Structured tabulation of results across studies

Ordering studies alphabetically by study ID is the simplest approach to tabulation; however, more information can be conveyed when studies are grouped in subpanels or ordered by a characteristic important for interpreting findings. The grouping of studies in tables should generally follow the structure of the synthesis presented in the text, which should closely reflect the review questions. This grouping should help readers identify the data on which findings are based and verify the review authors’ interpretation.

If the purpose of the table is comparative, grouping studies by any of following characteristics might be informative:

• comparisons considered in the review, or outcome domains (according to the structure of the synthesis);
• study characteristics that may reveal patterns in the data, for example potential effect modifiers including population subgroups, settings or intervention components.

If the purpose of the table is complete and transparent reporting of data, then ordering the studies to increase the prominence of the most relevant and trustworthy evidence should be considered. Possibilities include:

• certainty of the evidence (synthesized result or individual studies if no synthesis);
• risk of bias, study size or study design characteristics; and
• characteristics that determine how directly a study addresses the review question, for example relevance and validity of the outcome measures.

One disadvantage of grouping by study characteristics is that it can be harder to locate specific studies than when tables are ordered by study ID alone, for example when cross-referencing between the text and tables. Ordering by study ID within categories may partly address this.

The value of standardizing intervention and outcome labels is discussed in Chapter 3, Section 3.2.2 and Section 3.2.4 ), while the importance and methods for standardizing effect estimates is described in Chapter 6 . These practices can aid readers’ interpretation of tabulated data, especially when the purpose of a table is comparative.

12.3.2 Forest plots

Forest plots and methods for preparing them are described elsewhere ( Chapter 10, Section 10.2 ). Some mention is warranted here of their importance for displaying study results when meta-analysis is not undertaken (i.e. without the summary diamond). Forest plots can aid interpretation of individual study results and convey overall patterns in the data, especially when studies are ordered by a characteristic important for interpreting results (e.g. dose and effect size, sample size). Similarly, grouping studies in subpanels based on characteristics thought to modify effects, such as population subgroups, variants of an intervention, or risk of bias, may help explore and explain differences across studies (Schriger et al 2010). These approaches to ordering provide important techniques for informally exploring heterogeneity in reviews without meta-analysis, and should be considered in preference to alphabetical ordering by study ID alone (Schriger et al 2010).

12.3.3 Box-and-whisker plots and bubble plots

Box-and-whisker plots (see Figure 12.4.a , Panel A) provide a visual display of the distribution of effect estimates (Section 12.2.1.1 ). The plot conventionally depicts five values. The upper and lower limits (or ‘hinges’) of the box, represent the 75th and 25th percentiles, respectively. The line within the box represents the 50th percentile (median), and the whiskers represent the extreme values (McGill et al 1978). Multiple box plots can be juxtaposed, providing a visual comparison of the distributions of effect estimates (Schriger et al 2006). For example, in a review examining the effects of audit and feedback on professional practice, the format of the feedback (verbal, written, both verbal and written) was hypothesized to be an effect modifier (Ivers et al 2012). Box-and-whisker plots of the risk differences were presented separately by the format of feedback, to allow visual comparison of the impact of format on the distribution of effects. When presenting multiple box-and-whisker plots, the width of the box can be varied to indicate the number of studies contributing to each. The plot’s common usage facilitates rapid and correct interpretation by readers (Schriger et al 2010). The individual studies contributing to the plot are not identified (as in a forest plot), however, and the plot is not appropriate when there are few studies (Schriger et al 2006).

A bubble plot (see Figure 12.4.a , Panel B) can also be used to provide a visual display of the distribution of effects, and is more suited than the box-and-whisker plot when there are few studies (Schriger et al 2006). The plot is a scatter plot that can display multiple dimensions through the location, size and colour of the bubbles. In a review examining the effects of educational outreach visits on professional practice, a bubble plot was used to examine visually whether the distribution of effects was modified by the targeted behaviour (O’Brien et al 2007). Each bubble represented the effect size (y-axis) and whether the study targeted a prescribing or other behaviour (x-axis). The size of the bubbles reflected the number of study participants. However, different formulations of the bubble plot can display other characteristics of the data (e.g. precision, risk-of-bias assessments).

12.3.4 Albatross plot

The albatross plot (see Figure 12.4.a , Panel C) allows approximate examination of the underlying intervention effect sizes where there is minimal reporting of results within studies (Harrison et al 2017). The plot only requires a two-sided P value, sample size and direction of effect (or equivalently, a one-sided P value and a sample size) for each result. The plot is a scatter plot of the study sample sizes against two-sided P values, where the results are separated by the direction of effect. Superimposed on the plot are ‘effect size contours’ (inspiring the plot’s name). These contours are specific to the type of data (e.g. continuous, binary) and statistical methods used to calculate the P values. The contours allow interpretation of the approximate effect sizes of the studies, which would otherwise not be possible due to the limited reporting of the results. Characteristics of studies (e.g. type of study design) can be identified using different colours or symbols, allowing informal comparison of subgroups.

The plot is likely to be more inclusive of the available studies than meta-analysis, because of its minimal data requirements. However, the plot should complement the results from a statistical synthesis, ideally a meta-analysis of available effects.

12.3.5 Harvest and effect direction plots

Harvest plots (see Figure 12.4.a , Panel D) provide a visual extension of vote counting results (Ogilvie et al 2008). In the plot, studies based on the categorization of their effects (e.g. ‘beneficial effects’, ‘no effects’ or ‘detrimental effects’) are grouped together. Each study is represented by a bar positioned according to its categorization. The bars can be ‘visually weighted’ (by height or width) and annotated to highlight study and outcome characteristics (e.g. risk-of-bias domains, proximal or distal outcomes, study design, sample size) (Ogilvie et al 2008, Crowther et al 2011). Annotation can also be used to identify the studies. A series of plots may be combined in a matrix that displays, for example, the vote counting results from different interventions or outcome domains.

The methods papers describing harvest plots have employed vote counting based on statistical significance (Ogilvie et al 2008, Crowther et al 2011). For the reasons outlined in Section 12.2.2.1 , this can be misleading. However, an acceptable approach would be to display the results based on direction of effect.

The effect direction plot is similar in concept to the harvest plot in the sense that both display information on the direction of effects (Thomson and Thomas 2013). In the first version of the effect direction plot, the direction of effects for each outcome within a single study are displayed, while the second version displays the direction of the effects for outcome domains across studies . In this second version, an algorithm is first applied to ‘synthesize’ the directions of effect for all outcomes within a domain (e.g. outcomes ‘sleep disturbed by wheeze’, ‘wheeze limits speech’, ‘wheeze during exercise’ in the outcome domain ‘respiratory’). This algorithm is based on the proportion of effects that are in a consistent direction and statistical significance. Arrows are used to indicate the reported direction of effect (for either outcomes or outcome domains). Features such as statistical significance, study design and sample size are denoted using size and colour. While this version of the plot conveys a large amount of information, it requires further development before its use can be recommended since the algorithm underlying the plot is likely to have poor performance validity.

12.4 Worked example

The example that follows uses four scenarios to illustrate methods for presentation and synthesis when meta-analysis is not possible. The first scenario contrasts a common approach to tabulation with alternative presentations that may enhance the transparency of reporting and interpretation of findings. Subsequent scenarios show the application of the synthesis approaches outlined in preceding sections of the chapter. Box 12.4.a summarizes the review comparisons and outcomes, and decisions taken by the review authors in planning their synthesis. While the example is loosely based on an actual review, the review description, scenarios and data are fabricated for illustration.

Box 12.4.a The review

12.4.1 Scenario 1: structured reporting of effects

We first address a scenario in which review authors have decided that the tools used to measure satisfaction measured concepts that were too dissimilar across studies for synthesis to be appropriate. Setting aside three of the 15 studies that reported on the birth partner’s satisfaction with care, a structured summary of effects is sought of the remaining 12 studies. To keep the example table short, only one outcome is shown per study for each of the measurement periods (antenatal, intrapartum or postpartum).

Table 12.4.a depicts a common yet suboptimal approach to presenting results. Note two features.

• Studies are ordered by study ID, rather than grouped by characteristics that might enhance interpretation (e.g. risk of bias, study size, validity of the measures, certainty of the evidence (GRADE)).
• Data reported are as extracted from each study; effect estimates were not calculated by the review authors and, where reported, were not standardized across studies (although data were available to do both).

Table 12.4.b shows an improved presentation of the same results. In line with best practice, here effect estimates have been calculated by the review authors for all outcomes, and a common metric computed to aid interpretation (in this case an odds ratio; see Chapter 6 for guidance on conversion of statistics to the desired format). Redundant information has been removed (‘statistical test’ and ‘P value’ columns). The studies have been re-ordered, first to group outcomes by period of care (intrapartum outcomes are shown here), and then by risk of bias. This re-ordering serves two purposes. Grouping by period of care aligns with the plan to consider outcomes for each period separately and ensures the table structure matches the order in which results are described in the text. Re-ordering by risk of bias increases the prominence of studies at lowest risk of bias, focusing attention on the results that should most influence conclusions. Had the review authors determined that a synthesis would be informative, then ordering to facilitate comparison across studies would be appropriate; for example, ordering by the type of satisfaction outcome (as pre-defined in the protocol, starting with global measures of satisfaction), or the comparisons made in the studies.

The results may also be presented in a forest plot, as shown in Figure 12.4.b . In both the table and figure, studies are grouped by risk of bias to focus attention on the most trustworthy evidence. The pattern of effects across studies is immediately apparent in Figure 12.4.b and can be described efficiently without having to interpret each estimate (e.g. difference between studies at low and high risk of bias emerge), although these results should be interpreted with caution in the absence of a formal test for subgroup differences (see Chapter 10, Section 10.11 ). Only outcomes measured during the intrapartum period are displayed, although outcomes from other periods could be added, maximizing the information conveyed.

An example description of the results from Scenario 1 is provided in Box 12.4.b . It shows that describing results study by study becomes unwieldy with more than a few studies, highlighting the importance of tables and plots. It also brings into focus the risk of presenting results without any synthesis, since it seems likely that the reader will try to make sense of the results by drawing inferences across studies. Since a synthesis was considered inappropriate, GRADE was applied to individual studies and then used to prioritize the reporting of results, focusing attention on the most relevant and trustworthy evidence. An alternative might be to report results at low risk of bias, an approach analogous to limiting a meta-analysis to studies at low risk of bias. Where possible, these and other approaches to prioritizing (or ordering) results from individual studies in text and tables should be pre-specified at the protocol stage.

Table 12.4.a Scenario 1: table ordered by study ID, data as reported by study authors

* All scales operate in the same direction; higher scores indicate greater satisfaction. CI = confidence interval; MD = mean difference; OR = odds ratio; POR = proportional odds ratio; RD = risk difference; RR = risk ratio.

Table 12.4.b Scenario 1: intrapartum outcome table ordered by risk of bias, standardized effect estimates calculated for all studies

* Outcomes operate in the same direction. A higher score, or an event, indicates greater satisfaction. ** Mean difference calculated for studies reporting continuous outcomes. † For binary outcomes, odds ratios were calculated from the reported summary statistics or were directly extracted from the study. For continuous outcomes, standardized mean differences were calculated and converted to odds ratios (see Chapter 6 ). CI = confidence interval; POR = proportional odds ratio.

Figure 12.4.b Forest plot depicting standardized effect estimates (odds ratios) for satisfaction

Box 12.4.b How to describe the results from this structured summary

12.4.2 Overview of scenarios 2–4: synthesis approaches

We now address three scenarios in which review authors have decided that the outcomes reported in the 15 studies all broadly reflect satisfaction with care. While the measures were quite diverse, a synthesis is sought to help decision makers understand whether women and their birth partners were generally more satisfied with the care received in midwife-led continuity models compared with other models. The three scenarios differ according to the data available (see Table 12.4.c ), with each reflecting progressively less complete reporting of the effect estimates. The data available determine the synthesis method that can be applied.

• Scenario 2: effect estimates available without measures of precision (illustrating synthesis of summary statistics).
• Scenario 3: P values available (illustrating synthesis of P values).
• Scenario 4: directions of effect available (illustrating synthesis using vote-counting based on direction of effect).

For studies that reported multiple satisfaction outcomes, one result is selected for synthesis using the decision rules in Box 12.4.a (point 2).

Table 12.4.c Scenarios 2, 3 and 4: available data for the selected outcome from each study

* All scales operate in the same direction. Higher scores indicate greater satisfaction. ** For a particular scenario, the ‘available data’ column indicates the data that were directly reported, or were calculated from the reported statistics, in terms of: effect estimate, direction of effect, confidence interval, precise P value, or statement regarding statistical significance (either statistically significant, or not). CI = confidence interval; direction = direction of effect reported or can be calculated; MD = mean difference; NS = not statistically significant; OR = odds ratio; RD = risk difference; RoB = risk of bias; RR = risk ratio; sig. = statistically significant; SMD = standardized mean difference; Stand. = standardized.

12.4.2.1 Scenario 2: summarizing effect estimates

In Scenario 2, effect estimates are available for all outcomes. However, for most studies, a measure of variance is not reported, or cannot be calculated from the available data. We illustrate how the effect estimates may be summarized using descriptive statistics. In this scenario, it is possible to calculate odds ratios for all studies. For the continuous outcomes, this involves first calculating a standardized mean difference, and then converting this to an odds ratio ( Chapter 10, Section 10.6 ). The median odds ratio is 1.32 with an interquartile range of 1.02 to 1.53 (15 studies). Box-and-whisker plots may be used to display these results and examine informally whether the distribution of effects differs by the overall risk-of-bias assessment ( Figure 12.4.a , Panel A). However, because there are relatively few effects, a reasonable alternative would be to present bubble plots ( Figure 12.4.a , Panel B).

An example description of the results from the synthesis is provided in Box 12.4.c .

Box 12.4.c How to describe the results from this synthesis

12.4.2.2 Scenario 3: combining P values

In Scenario 3, there is minimal reporting of the data, and the type of data and statistical methods and tests vary. However, 11 of the 15 studies provide a precise P value and direction of effect, and a further two report a P value less than a threshold (<0.001) and direction. We use this scenario to illustrate a synthesis of P values. Since the reported P values are two-sided ( Table 12.4.c , column 6), they must first be converted to one-sided P values, which incorporate the direction of effect ( Table 12.4.c , column 7).

Fisher’s method for combining P values involved calculating the following statistic:

The combination of P values suggests there is strong evidence of benefit of midwife-led models of care in at least one study (P < 0.001 from a Chi 2 test, 13 studies). Restricting this analysis to those studies judged to be at an overall low risk of bias (sensitivity analysis), there is no longer evidence to reject the null hypothesis of no benefit of midwife-led model of care in any studies (P = 0.314, 3 studies). For the five studies reporting continuous satisfaction outcomes, sufficient data (precise P value, direction, total sample size) are reported to construct an albatross plot ( Figure 12.4.a , Panel C). The location of the points relative to the standardized mean difference contours indicate that the likely effects of the intervention in these studies are small.

An example description of the results from the synthesis is provided in Box 12.4.d .

Box 12.4.d How to describe the results from this synthesis

12.4.2.3 Scenario 4: vote counting based on direction of effect

In Scenario 4, there is minimal reporting of the data, and the type of effect measure (when used) varies across the studies (e.g. mean difference, proportional odds ratio). Of the 15 results, only five report data suitable for meta-analysis (effect estimate and measure of precision; Table 12.4.c , column 8), and no studies reported precise P values. We use this scenario to illustrate vote counting based on direction of effect. For each study, the effect is categorized as beneficial or harmful based on the direction of effect (indicated as a binary metric; Table 12.4.c , column 9).

Of the 15 studies, we exclude three because they do not provide information on the direction of effect, leaving 12 studies to contribute to the synthesis. Of these 12, 10 effects favour midwife-led models of care (83%). The probability of observing this result if midwife-led models of care are truly ineffective is 0.039 (from a binomial probability test, or equivalently, the sign test). The 95% confidence interval for the percentage of effects favouring midwife-led care is wide (55% to 95%).

The binomial test can be implemented using standard computer spreadsheet or statistical packages. For example, the two-sided P value from the binomial probability test presented can be obtained from Microsoft Excel by typing =2*BINOM.DIST(2, 12, 0.5, TRUE) into any cell in the spreadsheet. The syntax requires the smaller of the ‘number of effects favouring the intervention’ or ‘the number of effects favouring the control’ (here, the smaller of these counts is 2), the number of effects (here 12), and the null value (true proportion of effects favouring the intervention = 0.5). In Stata, the bitest command could be used (e.g. bitesti 12 10 0.5 ).

A harvest plot can be used to display the results ( Figure 12.4.a , Panel D), with characteristics of the studies represented using different heights and shading. A sensitivity analysis might be considered, restricting the analysis to those studies judged to be at an overall low risk of bias. However, only four studies were judged to be at a low risk of bias (of which, three favoured midwife-led models of care), precluding reasonable interpretation of the count.

An example description of the results from the synthesis is provided in Box 12.4.e .

Box 12.4.e How to describe the results from this synthesis

Figure 12.4.a Possible graphical displays of different types of data. (A) Box-and-whisker plots of odds ratios for all outcomes and separately by overall risk of bias. (B) Bubble plot of odds ratios for all outcomes and separately by the model of care. The colours of the bubbles represent the overall risk of bias judgement (green = low risk of bias; yellow = some concerns; red = high risk of bias). (C) Albatross plot of the study sample size against P values (for the five continuous outcomes in Table 12.4.c , column 6). The effect contours represent standardized mean differences. (D) Harvest plot (height depicts overall risk of bias judgement (tall = low risk of bias; medium = some concerns; short = high risk of bias), shading depicts model of care (light grey = caseload; dark grey = team), alphabet characters represent the studies)

12.5 Chapter information

Authors: Joanne E McKenzie, Sue E Brennan

Acknowledgements: Sections of this chapter build on chapter 9 of version 5.1 of the Handbook , with editors Jonathan J Deeks, Julian PT Higgins and Douglas G Altman.

We are grateful to the following for commenting helpfully on earlier drafts: Miranda Cumpston, Jamie Hartmann-Boyce, Tianjing Li, Rebecca Ryan and Hilary Thomson.

Funding: JEM is supported by an Australian National Health and Medical Research Council (NHMRC) Career Development Fellowship (1143429). SEB’s position is supported by the NHMRC Cochrane Collaboration Funding Program.

12.6 References

Achana F, Hubbard S, Sutton A, Kendrick D, Cooper N. An exploration of synthesis methods in public health evaluations of interventions concludes that the use of modern statistical methods would be beneficial. Journal of Clinical Epidemiology 2014; 67 : 376–390.

Becker BJ. Combining significance levels. In: Cooper H, Hedges LV, editors. A handbook of research synthesis . New York (NY): Russell Sage; 1994. p. 215–235.

Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA 2007; 298 : 1023–1037.

Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Meta-Analysis methods based on direction and p-values. Introduction to Meta-Analysis . Chichester (UK): John Wiley & Sons, Ltd; 2009. pp. 325–330.

Brown LD, Cai TT, DasGupta A. Interval estimation for a binomial proportion. Statistical Science 2001; 16 : 101–117.

Bushman BJ, Wang MC. Vote-counting procedures in meta-analysis. In: Cooper H, Hedges LV, Valentine JC, editors. Handbook of Research Synthesis and Meta-Analysis . 2nd ed. New York (NY): Russell Sage Foundation; 2009. p. 207–220.

Crowther M, Avenell A, MacLennan G, Mowatt G. A further use for the Harvest plot: a novel method for the presentation of data synthesis. Research Synthesis Methods 2011; 2 : 79–83.

Friedman L. Why vote-count reviews don’t count. Biological Psychiatry 2001; 49 : 161–162.

Grimshaw J, McAuley LM, Bero LA, Grilli R, Oxman AD, Ramsay C, Vale L, Zwarenstein M. Systematic reviews of the effectiveness of quality improvement strategies and programmes. Quality and Safety in Health Care 2003; 12 : 298–303.

Harrison S, Jones HE, Martin RM, Lewis SJ, Higgins JPT. The albatross plot: a novel graphical tool for presenting results of diversely reported studies in a systematic review. Research Synthesis Methods 2017; 8 : 281–289.

Hedges L, Vevea J. Fixed- and random-effects models in meta-analysis. Psychological Methods 1998; 3 : 486–504.

Ioannidis JP, Patsopoulos NA, Rothstein HR. Reasons or excuses for avoiding meta-analysis in forest plots. BMJ 2008; 336 : 1413–1415.

Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012; 6 : CD000259.

Jones DR. Meta-analysis: weighing the evidence. Statistics in Medicine 1995; 14 : 137–149.

Loughin TM. A systematic comparison of methods for combining p-values from independent tests. Computational Statistics & Data Analysis 2004; 47 : 467–485.

McGill R, Tukey JW, Larsen WA. Variations of box plots. The American Statistician 1978; 32 : 12–16.

McKenzie JE, Brennan SE. Complex reviews: methods and considerations for summarising and synthesising results in systematic reviews with complexity. Report to the Australian National Health and Medical Research Council. 2014.

O’Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis DA, Haynes RB, Harvey EL. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2007; 4 : CD000409.

Ogilvie D, Fayter D, Petticrew M, Sowden A, Thomas S, Whitehead M, Worthy G. The harvest plot: a method for synthesising evidence about the differential effects of interventions. BMC Medical Research Methodology 2008; 8 : 8.

Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ 2011; 342 : d549.

Schriger DL, Sinha R, Schroter S, Liu PY, Altman DG. From submission to publication: a retrospective review of the tables and figures in a cohort of randomized controlled trials submitted to the British Medical Journal. Annals of Emergency Medicine 2006; 48 : 750–756, 756 e751–721.

Schriger DL, Altman DG, Vetter JA, Heafner T, Moher D. Forest plots in reports of systematic reviews: a cross-sectional study reviewing current practice. International Journal of Epidemiology 2010; 39 : 421–429.

ter Wee MM, Lems WF, Usan H, Gulpen A, Boonen A. The effect of biological agents on work participation in rheumatoid arthritis patients: a systematic review. Annals of the Rheumatic Diseases 2012; 71 : 161–171.

Thomson HJ, Thomas S. The effect direction plot: visual display of non-standardised effects across multiple outcome domains. Research Synthesis Methods 2013; 4 : 95–101.

Thornicroft G, Mehta N, Clement S, Evans-Lacko S, Doherty M, Rose D, Koschorke M, Shidhaye R, O’Reilly C, Henderson C. Evidence for effective interventions to reduce mental-health-related stigma and discrimination. Lancet 2016; 387 : 1123–1132.

Valentine JC, Pigott TD, Rothstein HR. How many studies do you need?: a primer on statistical power for meta-analysis. Journal of Educational and Behavioral Statistics 2010; 35 : 215–247.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Working with sources
• Synthesizing Sources | Examples & Synthesis Matrix

Synthesizing Sources | Examples & Synthesis Matrix

Published on July 4, 2022 by Eoghan Ryan . Revised on May 31, 2023.

Synthesizing sources involves combining the work of other scholars to provide new insights. It’s a way of integrating sources that helps situate your work in relation to existing research.

Synthesizing sources involves more than just summarizing . You must emphasize how each source contributes to current debates, highlighting points of (dis)agreement and putting the sources in conversation with each other.

You might synthesize sources in your literature review to give an overview of the field or throughout your research paper when you want to position your work in relation to existing research.

Table of contents

Example of synthesizing sources, how to synthesize sources, synthesis matrix, other interesting articles, frequently asked questions about synthesizing sources.

Let’s take a look at an example where sources are not properly synthesized, and then see what can be done to improve it.

This paragraph provides no context for the information and does not explain the relationships between the sources described. It also doesn’t analyze the sources or consider gaps in existing research.

Research on the barriers to second language acquisition has primarily focused on age-related difficulties. Building on Lenneberg’s (1967) theory of a critical period of language acquisition, Johnson and Newport (1988) tested Lenneberg’s idea in the context of second language acquisition. Their research seemed to confirm that young learners acquire a second language more easily than older learners. Recent research has considered other potential barriers to language acquisition. Schepens, van Hout, and van der Slik (2022) have revealed that the difficulties of learning a second language at an older age are compounded by dissimilarity between a learner’s first language and the language they aim to acquire. Further research needs to be carried out to determine whether the difficulty faced by adult monoglot speakers is also faced by adults who acquired a second language during the “critical period.”

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

To synthesize sources, group them around a specific theme or point of contention.

As you read sources, ask:

• What questions or ideas recur? Do the sources focus on the same points, or do they look at the issue from different angles?
• How does each source relate to others? Does it confirm or challenge the findings of past research?
• Where do the sources agree or disagree?

Once you have a clear idea of how each source positions itself, put them in conversation with each other. Analyze and interpret their points of agreement and disagreement. This displays the relationships among sources and creates a sense of coherence.

Consider both implicit and explicit (dis)agreements. Whether one source specifically refutes another or just happens to come to different conclusions without specifically engaging with it, you can mention it in your synthesis either way.

Synthesize your sources using:

• Topic sentences to introduce the relationship between the sources
• Signal phrases to attribute ideas to their authors
• Transition words and phrases to link together different ideas

To more easily determine the similarities and dissimilarities among your sources, you can create a visual representation of their main ideas with a synthesis matrix . This is a tool that you can use when researching and writing your paper, not a part of the final text.

In a synthesis matrix, each column represents one source, and each row represents a common theme or idea among the sources. In the relevant rows, fill in a short summary of how the source treats each theme or topic.

This helps you to clearly see the commonalities or points of divergence among your sources. You can then synthesize these sources in your work by explaining their relationship.

If you want to know more about ChatGPT, AI tools , citation , and plagiarism , make sure to check out some of our other articles with explanations and examples.

• ChatGPT vs human editor
• ChatGPT citations
• Is ChatGPT trustworthy?
• Using ChatGPT for your studies
• What is ChatGPT?
• Chicago style
• Paraphrasing

 Plagiarism

• Types of plagiarism
• Self-plagiarism
• Avoiding plagiarism
• Academic integrity
• Consequences of plagiarism
• Common knowledge

Prevent plagiarism. Run a free check.

Synthesizing sources means comparing and contrasting the work of other scholars to provide new insights.

It involves analyzing and interpreting the points of agreement and disagreement among sources.

You might synthesize sources in your literature review to give an overview of the field of research or throughout your paper when you want to contribute something new to existing research.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

Topic sentences help keep your writing focused and guide the reader through your argument.

In an essay or paper , each paragraph should focus on a single idea. By stating the main idea in the topic sentence, you clarify what the paragraph is about for both yourself and your reader.

At college level, you must properly cite your sources in all essays , research papers , and other academic texts (except exams and in-class exercises).

Add a citation whenever you quote , paraphrase , or summarize information or ideas from a source. You should also give full source details in a bibliography or reference list at the end of your text.

The exact format of your citations depends on which citation style you are instructed to use. The most common styles are APA , MLA , and Chicago .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Ryan, E. (2023, May 31). Synthesizing Sources | Examples & Synthesis Matrix. Scribbr. Retrieved September 23, 2024, from https://www.scribbr.com/working-with-sources/synthesizing-sources/

Is this article helpful?

Eoghan Ryan

Other students also liked, signal phrases | definition, explanation & examples, how to write a literature review | guide, examples, & templates, how to find sources | scholarly articles, books, etc., get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

How to Synthesize Written Information from Multiple Sources

Shona McCombes

Content Manager

B.A., English Literature, University of Glasgow

Shona McCombes is the content manager at Scribbr, Netherlands.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

On This Page:

When you write a literature review or essay, you have to go beyond just summarizing the articles you’ve read – you need to synthesize the literature to show how it all fits together (and how your own research fits in).

Synthesizing simply means combining. Instead of summarizing the main points of each source in turn, you put together the ideas and findings of multiple sources in order to make an overall point.

At the most basic level, this involves looking for similarities and differences between your sources. Your synthesis should show the reader where the sources overlap and where they diverge.

Unsynthesized Example

Franz (2008) studied undergraduate online students. He looked at 17 females and 18 males and found that none of them liked APA. According to Franz, the evidence suggested that all students are reluctant to learn citations style. Perez (2010) also studies undergraduate students. She looked at 42 females and 50 males and found that males were significantly more inclined to use citation software ( p < .05). Findings suggest that females might graduate sooner. Goldstein (2012) looked at British undergraduates. Among a sample of 50, all females, all confident in their abilities to cite and were eager to write their dissertations.

Synthesized Example

Studies of undergraduate students reveal conflicting conclusions regarding relationships between advanced scholarly study and citation efficacy. Although Franz (2008) found that no participants enjoyed learning citation style, Goldstein (2012) determined in a larger study that all participants watched felt comfortable citing sources, suggesting that variables among participant and control group populations must be examined more closely. Although Perez (2010) expanded on Franz’s original study with a larger, more diverse sample…

Step 1: Organize your sources

After collecting the relevant literature, you’ve got a lot of information to work through, and no clear idea of how it all fits together.

Before you can start writing, you need to organize your notes in a way that allows you to see the relationships between sources.

One way to begin synthesizing the literature is to put your notes into a table. Depending on your topic and the type of literature you’re dealing with, there are a couple of different ways you can organize this.

Summary table

A summary table collates the key points of each source under consistent headings. This is a good approach if your sources tend to have a similar structure – for instance, if they’re all empirical papers.

Each row in the table lists one source, and each column identifies a specific part of the source. You can decide which headings to include based on what’s most relevant to the literature you’re dealing with.

For example, you might include columns for things like aims, methods, variables, population, sample size, and conclusion.

For each study, you briefly summarize each of these aspects. You can also include columns for your own evaluation and analysis.

The summary table gives you a quick overview of the key points of each source. This allows you to group sources by relevant similarities, as well as noticing important differences or contradictions in their findings.

Synthesis matrix

A synthesis matrix is useful when your sources are more varied in their purpose and structure – for example, when you’re dealing with books and essays making various different arguments about a topic.

Each column in the table lists one source. Each row is labeled with a specific concept, topic or theme that recurs across all or most of the sources.

Then, for each source, you summarize the main points or arguments related to the theme.

The purposes of the table is to identify the common points that connect the sources, as well as identifying points where they diverge or disagree.

Step 2: Outline your structure

Now you should have a clear overview of the main connections and differences between the sources you’ve read. Next, you need to decide how you’ll group them together and the order in which you’ll discuss them.

For shorter papers, your outline can just identify the focus of each paragraph; for longer papers, you might want to divide it into sections with headings.

There are a few different approaches you can take to help you structure your synthesis.

If your sources cover a broad time period, and you found patterns in how researchers approached the topic over time, you can organize your discussion chronologically .

That doesn’t mean you just summarize each paper in chronological order; instead, you should group articles into time periods and identify what they have in common, as well as signalling important turning points or developments in the literature.

If the literature covers various different topics, you can organize it thematically .

That means that each paragraph or section focuses on a specific theme and explains how that theme is approached in the literature.

Source Used with Permission: The Chicago School

If you’re drawing on literature from various different fields or they use a wide variety of research methods, you can organize your sources methodologically .

That means grouping together studies based on the type of research they did and discussing the findings that emerged from each method.

If your topic involves a debate between different schools of thought, you can organize it theoretically .

That means comparing the different theories that have been developed and grouping together papers based on the position or perspective they take on the topic, as well as evaluating which arguments are most convincing.

Step 3: Write paragraphs with topic sentences

What sets a synthesis apart from a summary is that it combines various sources. The easiest way to think about this is that each paragraph should discuss a few different sources, and you should be able to condense the overall point of the paragraph into one sentence.

This is called a topic sentence , and it usually appears at the start of the paragraph. The topic sentence signals what the whole paragraph is about; every sentence in the paragraph should be clearly related to it.

A topic sentence can be a simple summary of the paragraph’s content:

“Early research on [x] focused heavily on [y].”

For an effective synthesis, you can use topic sentences to link back to the previous paragraph, highlighting a point of debate or critique:

“Several scholars have pointed out the flaws in this approach.” “While recent research has attempted to address the problem, many of these studies have methodological flaws that limit their validity.”

By using topic sentences, you can ensure that your paragraphs are coherent and clearly show the connections between the articles you are discussing.

As you write your paragraphs, avoid quoting directly from sources: use your own words to explain the commonalities and differences that you found in the literature.

Don’t try to cover every single point from every single source – the key to synthesizing is to extract the most important and relevant information and combine it to give your reader an overall picture of the state of knowledge on your topic.

Step 4: Revise, edit and proofread

Like any other piece of academic writing, synthesizing literature doesn’t happen all in one go – it involves redrafting, revising, editing and proofreading your work.

Checklist for Synthesis

•   Do I introduce the paragraph with a clear, focused topic sentence?
•   Do I discuss more than one source in the paragraph?
•   Do I mention only the most relevant findings, rather than describing every part of the studies?
•   Do I discuss the similarities or differences between the sources, rather than summarizing each source in turn?
•   Do I put the findings or arguments of the sources in my own words?
•   Is the paragraph organized around a single idea?
•   Is the paragraph directly relevant to my research question or topic?
•   Is there a logical transition from this paragraph to the next one?

Further Information

How to Synthesise: a Step-by-Step Approach

Help…I”ve Been Asked to Synthesize!

Learn how to Synthesise (combine information from sources)

How to write a Psychology Essay

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Synthesizing Sources

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

When you look for areas where your sources agree or disagree and try to draw broader conclusions about your topic based on what your sources say, you are engaging in synthesis. Writing a research paper usually requires synthesizing the available sources in order to provide new insight or a different perspective into your particular topic (as opposed to simply restating what each individual source says about your research topic).

Note that synthesizing is not the same as summarizing.  

• A summary restates the information in one or more sources without providing new insight or reaching new conclusions.
• A synthesis draws on multiple sources to reach a broader conclusion.

There are two types of syntheses: explanatory syntheses and argumentative syntheses . Explanatory syntheses seek to bring sources together to explain a perspective and the reasoning behind it. Argumentative syntheses seek to bring sources together to make an argument. Both types of synthesis involve looking for relationships between sources and drawing conclusions.

In order to successfully synthesize your sources, you might begin by grouping your sources by topic and looking for connections. For example, if you were researching the pros and cons of encouraging healthy eating in children, you would want to separate your sources to find which ones agree with each other and which ones disagree.

After you have a good idea of what your sources are saying, you want to construct your body paragraphs in a way that acknowledges different sources and highlights where you can draw new conclusions.

As you continue synthesizing, here are a few points to remember:

• Don’t force a relationship between sources if there isn’t one. Not all of your sources have to complement one another.
• Do your best to highlight the relationships between sources in very clear ways.
• Don’t ignore any outliers in your research. It’s important to take note of every perspective (even those that disagree with your broader conclusions).

Example Syntheses

Below are two examples of synthesis: one where synthesis is NOT utilized well, and one where it is.

Parents are always trying to find ways to encourage healthy eating in their children. Elena Pearl Ben-Joseph, a doctor and writer for KidsHealth , encourages parents to be role models for their children by not dieting or vocalizing concerns about their body image. The first popular diet began in 1863. William Banting named it the “Banting” diet after himself, and it consisted of eating fruits, vegetables, meat, and dry wine. Despite the fact that dieting has been around for over a hundred and fifty years, parents should not diet because it hinders children’s understanding of healthy eating.

In this sample paragraph, the paragraph begins with one idea then drastically shifts to another. Rather than comparing the sources, the author simply describes their content. This leads the paragraph to veer in an different direction at the end, and it prevents the paragraph from expressing any strong arguments or conclusions.

An example of a stronger synthesis can be found below.

Parents are always trying to find ways to encourage healthy eating in their children. Different scientists and educators have different strategies for promoting a well-rounded diet while still encouraging body positivity in children. David R. Just and Joseph Price suggest in their article “Using Incentives to Encourage Healthy Eating in Children” that children are more likely to eat fruits and vegetables if they are given a reward (855-856). Similarly, Elena Pearl Ben-Joseph, a doctor and writer for Kids Health , encourages parents to be role models for their children. She states that “parents who are always dieting or complaining about their bodies may foster these same negative feelings in their kids. Try to keep a positive approach about food” (Ben-Joseph). Martha J. Nepper and Weiwen Chai support Ben-Joseph’s suggestions in their article “Parents’ Barriers and Strategies to Promote Healthy Eating among School-age Children.” Nepper and Chai note, “Parents felt that patience, consistency, educating themselves on proper nutrition, and having more healthy foods available in the home were important strategies when developing healthy eating habits for their children.” By following some of these ideas, parents can help their children develop healthy eating habits while still maintaining body positivity.

In this example, the author puts different sources in conversation with one another. Rather than simply describing the content of the sources in order, the author uses transitions (like "similarly") and makes the relationship between the sources evident.

Welcome to the new OASIS website! We have academic skills, library skills, math and statistics support, and writing resources all together in one new home.

• Walden University
• Faculty Portal

Using Evidence: Synthesis

Synthesis video playlist.

Note that these videos were created while APA 6 was the style guide edition in use. There may be some examples of writing that have not been updated to APA 7 guidelines.

Basics of Synthesis

As you incorporate published writing into your own writing, you should aim for synthesis of the material.

Synthesizing requires critical reading and thinking in order to compare different material, highlighting similarities, differences, and connections. When writers synthesize successfully, they present new ideas based on interpretations of other evidence or arguments. You can also think of synthesis as an extension of—or a more complicated form of—analysis. One main difference is that synthesis involves multiple sources, while analysis often focuses on one source.

Conceptually, it can be helpful to think about synthesis existing at both the local (or paragraph) level and the global (or paper) level.

Local Synthesis

Local synthesis occurs at the paragraph level when writers connect individual pieces of evidence from multiple sources to support a paragraph’s main idea and advance a paper’s thesis statement. A common example in academic writing is a scholarly paragraph that includes a main idea, evidence from multiple sources, and analysis of those multiple sources together.

Global Synthesis

Global synthesis occurs at the paper (or, sometimes, section) level when writers connect ideas across paragraphs or sections to create a new narrative whole. A literature review , which can either stand alone or be a section/chapter within a capstone, is a common example of a place where global synthesis is necessary. However, in almost all academic writing, global synthesis is created by and sometimes referred to as good cohesion and flow.

Synthesis in Literature Reviews

While any types of scholarly writing can include synthesis, it is most often discussed in the context of literature reviews. Visit our literature review pages for more information about synthesis in literature reviews.

Didn't find what you need? Email us at [email protected] .

• Previous Page: Analysis
• Next Page: Citing Sources Properly
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Cost of Attendance
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Analysis and Synthesis

• First Online: 18 February 2020

Cite this chapter

• Patricia A. Dwyer 3  

5427 Accesses

14 Citations

Data analysis is a challenging stage of the integrative review process as it requires the reviewer to synthesize data from diverse methodological sources. Although established approaches to data analysis and synthesis of integrative review findings continue to evolve, adherence to systematic methods during this stage is essential to mitigating potential bias. The use of rigorous and transparent data analysis methods facilitates an evidence synthesis that can be confidently incorporated into practice. This chapter discusses strategies for data analysis including creating a data matrix and presents inductive analysis approaches to support the integration and interpretation of data from a body of literature. This chapter also discusses the presentation of results and includes examples of narrative and thematic syntheses from recently published integrative reviews.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

What do meta-analysts need in primary studies? Guidelines and the SEMI checklist for facilitating cumulative knowledge

Systematic reviews and meta-analysis: a guide for beginners, a guide to conducting a meta-analysis.

Alexis O, Worsley A (2018) An integrative review exploring black men of African and Caribbean backgrounds, their fears of prostate cancer and their attitudes towards screening. Health Educ Res 33(2):155–166. https://doi.org/10.1093/her/cyy001

Article   PubMed   Google Scholar  

Beyea SC, Nicoll LH (1998) Writing an integrative review. AORN J 67(4):877–880

Article   CAS   Google Scholar  

Blondy LC, Blakeslee AM, Scheffer BK, Rubenfeld MG, Cronin BM, Luster-Turner R (2016) Understanding synthesis across disciplines to improve nursing education. West J Nurs Res 38(6):668–685

Article   Google Scholar  

Booth A (2012) Synthesizing included studies. In: Booth A, Papaioannou D, Sutton A (eds) Systematic approaches to a successful literature review. Sage, London, pp 125–169

Google Scholar  

Brady S, Lee N, Gibbons K, Bogossian F (2019) Woman-centred care: an integrative review of the empirical literature. Int J Nurs Stud 94:107–119

Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77–101. https://doi.org/10.1191/1478088706qp063oa

Cameron J, Roxburgh M, Taylor J, Lauder W (2011) An integrative literature review of student retention in programmes of nursing and midwifery education: why do students stay? J Clin Nurs. 20:1372–1382. https://doi.org/10.1111/j.1365-2702.2010.03336.x

Cooper H (1998) Synthesizing research: a guide for literature reviews, 3rd edn. Sage, Thousand Oaks, CA

Coughlin MB, Sethares KA (2017) Chronic sorrow in parents of children with a chronic illness or disability: an integrative literature review. J Pediatr Nurs 37:108–116

Elo S, Kynga SH (2008) The qualitative content analysis process. J Adv Nurs 62(1):107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x

Garrard J (2017) Health sciences literature review made easy: the matrix method. In: Chapter 5, Review matrix folder: how to abstract the research literature, 4th edn. Jones & Bartlett Learning, Burlington, MA, pp 139–160

Harstade CW, Blomberg K, Benzein E, Ostland U (2018) Dignity-conserving care actions in palliative care: an integrative review of Swedish research. Scand J Caring Sci 32(1):8–23. https://doi.org/10.1111/scs.12433

Hopia H, Latvala E, Liimatainen L (2016) Reviewing the methodology of an integrative review. Scand J Caring Sci 30:662–669

Knafl K, Whittemore R (2017) Top 10 tips for undertaking synthesis research. Res Nurs Health 40:189–193

Miles MB, Huberman AM (1994a) Chapter 1, Introduction. In: Qualitative data analysis: an expanded sourcebook, 2nd edn. Sage, Thousand Oaks, CA, pp 1–11

Miles MB, Huberman AM (1994b) Chapter 7, Cross-case displays: exploring and describing. In: Qualitative data analysis: An expanded sourcebook, 2nd edn. Sage, Thousand Oaks, CA, pp 172–205

Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, Britten N (2006) Chapter 3, Guidance on narrative synthesis: an overview. In: Guidance on the conduct of narrative synthesis in systematic reviews: a product from the ESRC methods programme. ESRC, pp 11–24

Sandelowski M (1995) Qualitative analysis: what it is and how to begin. Res Nurs Health 18:371–375. https://doi.org/10.1002/nur.4770180411

Article   CAS   PubMed   Google Scholar  

Sandelowski M (2000) Focus on research methods: whatever happened to qualitative description? Res Nurs Health 23:334–340

Tobiano G, Marshall A, Bucknall T, Chaboyer W (2015) Patient participation in nursing care on medical wards: an integrative review. Int J Nurs Stud 52:1107–1120. https://doi.org/10.1016/j.ijnurstu.2015.02.010

Toronto CE, LaRocco SA (2019) Family perception of and experience with family presence during cardiopulmonary resuscitation: an integrative review. J Clin Nurs 28(1):32–46

Toronto CE, Quinn B, Remington R (2018) Characteristics of reviews published in nursing literature: a methodological review. ANS Adv Nurs Sci 41(1):30–40. https://doi.org/10.1097/ANS.0000000000000180

Torraco RJ (2005) Writing integrative literature reviews: guidelines and examples. Hum Resour Dev Rev 4(3):356–367

Torraco RJ (2016) Writing integrative literature reviews: using the past and present to explore the future. Hum Resour Dev Rev 15(4):404–428. https://doi.org/10.1177/1534484316671606

Whittemore R (2005) Combining evidence in nursing research: methods and implications. Nurs Res 54(1):56–62

Whittemore R, Knafl K (2005) The integrative review: updated methodology. J Adv Nurs 52(5):546–553. https://doi.org/10.1111/j.1365-2648.2005.03621.x

Download references

Author information

Authors and affiliations.

Boston Children’s Hospital, Waltham, MA, USA

Patricia A. Dwyer

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

School of Nursing, Curry College, Milton, MA, USA

Coleen E. Toronto

Department of Nursing, Framingham State University, Framingham, MA, USA

Ruth Remington

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Dwyer, P.A. (2020). Analysis and Synthesis. In: Toronto, C., Remington, R. (eds) A Step-by-Step Guide to Conducting an Integrative Review. Springer, Cham. https://doi.org/10.1007/978-3-030-37504-1_5

Download citation

DOI : https://doi.org/10.1007/978-3-030-37504-1_5

Published : 18 February 2020

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-37503-4

Online ISBN : 978-3-030-37504-1

eBook Packages : Medicine Medicine (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Organizations
• Planning & Activities
• Product & Services
• Structure & Systems
• Career & Education
• Entertainment
• Fashion & Beauty
• Political Institutions
• SmartPhones
• Protocols & Formats
• Communication
• Web Applications
• Household Equipments
• Career and Certifications
• Diet & Fitness
• Mathematics & Statistics
• Processed Foods
• Vegetables & Fruits

Difference Between Analysis and Synthesis

• Categorized under Science | Difference Between Analysis and Synthesis

Analysis Vs Synthesis

Analysis is like the process of deduction wherein you cut down a bigger concept into smaller ones. As such, analysis breaks down complex ideas into smaller fragmented concepts so as to come up with an improved understanding. Synthesis, on the other hand, resolves a conflict set between an antithesis and a thesis by settling what truths they have in common. In the end, the synthesis aims to make a new proposal or proposition.

Derived from the Greek word ‘analusis’ which literally means ‘a breaking up,’ analysis is, by far, mostly used in the realm of logic and mathematics even before the time of the great philosopher Aristotle. When learners are asked to analyze a certain concept or subject matter, they are encouraged to connect different ideas or examine how each idea was composed. The relation of each idea that connects to the bigger picture is studied. They are also tasked to spot for any evidences that will help them lead into a concrete conclusion. These evidences are found by discovering the presence of biases and assumptions.

Synthesizing is different because when the learners are asked to synthesize, they already try to put together the separate parts that have already been analyzed with other ideas or concepts to form something new or original. It’s like they look into varied resource materials to get insights and bright ideas and from there, they form their own concepts.

Similar definitions of synthesis (from other sources) state that it is combining two (or even more) concepts that form something fresh. This may be the reason why synthesis in chemistry means starting a series of chemical reactions in order to form a complex molecule out of simpler chemical precursors. In botany, plants perform their basic function of photosynthesis wherein they use the sunlight’s energy as catalyst to make an organic molecule from a simple carbon molecule. In addition, science professors use this term like bread and butter to denote that something is being made. When they mention about amino acid (the building blocks of proteins) synthesis, then it is the process of making amino acids out of its many basic elements or constituents. But in the field of Humanities, synthesis (in the case of philosophy) is the end product of dialectic (i.e. a thesis) and is considered as a higher process compared to analysis.

When one uses analysis in Chemistry, he will perform any of the following: (quantitative analysis) search for the proportionate components of a mixture, (qualitative analysis) search for the components of a specific chemical, and last is to split chemical processes and observe any reactions that occur between the individual elements of matter.

1. Synthesis is a higher process that creates something new. It is usually done at the end of an entire study or scientific inquiry. 2. Analysis is like the process of deduction wherein a bigger concept is broken down into simpler ideas to gain a better understanding of the entire thing.

• Recent Posts
• Difference Between Plant Protein and Animal Protein - March 7, 2024
• Difference Between Crohn’s and Colitis - March 7, 2024
• Difference Between Expression and Equation - March 7, 2024

Sharing is caring!

Search DifferenceBetween.net :

• Difference Between Hydrolysis and Dehydration Synthesis
• Difference Between Idea and Concept
• Difference Between Anticodon and Codon
• Difference Between Deep Learning and Surface Learning
• Difference Between Compound and Mixture

Cite APA 7 , . (2011, March 19). Difference Between Analysis and Synthesis. Difference Between Similar Terms and Objects. http://www.differencebetween.net/science/difference-between-analysis-and-synthesis/. MLA 8 , . "Difference Between Analysis and Synthesis." Difference Between Similar Terms and Objects, 19 March, 2011, http://www.differencebetween.net/science/difference-between-analysis-and-synthesis/.

It’s very useful to understand the science and other subjects. Thanks

It was insightful

Thanks so much…. You explained so beautifully and simply….. Thanks again a lot

Thank you sir for your good explanation

Leave a Response

Name ( required )

Email ( required )

Please note: comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.

Notify me of followup comments via e-mail

Written by : Julita. and updated on 2011, March 19 Articles on DifferenceBetween.net are general information, and are not intended to substitute for professional advice. The information is "AS IS", "WITH ALL FAULTS". User assumes all risk of use, damage, or injury. You agree that we have no liability for any damages.

Advertisments

More in 'science'.

• Difference Between Rumination and Regurgitation
• Difference Between Pyelectasis and Hydronephrosis 
• Difference Between Cellulitis and Erysipelas
• Difference Between Suicide and Euthanasia
• Difference Between Vitamin D and Vitamin D3

Top Difference Betweens

Get new comparisons in your inbox:, most emailed comparisons, editor's picks.

• Difference Between MAC and IP Address
• Difference Between Platinum and White Gold
• Difference Between Civil and Criminal Law
• Difference Between GRE and GMAT
• Difference Between Immigrants and Refugees
• Difference Between DNS and DHCP
• Difference Between Computer Engineering and Computer Science
• Difference Between Men and Women
• Difference Between Book value and Market value
• Difference Between Red and White wine
• Difference Between Depreciation and Amortization
• Difference Between Bank and Credit Union
• Difference Between White Eggs and Brown Eggs

Maintenance work is planned from 09:00 BST to 12:00 BST on Saturday 28th September 2024.

During this time the performance of our website may be affected - searches may run slowly, some pages may be temporarily unavailable, and you may be unable to access content. If this happens, please try refreshing your web browser or try waiting two to three minutes before trying again.

We apologise for any inconvenience this might cause and thank you for your patience.

RSC Advances

Thymol and carvacrol derivatives as anticancer agents; synthesis, in vitro activity, and computational analysis of biological targets †.

* Corresponding authors

a Department of Pharmacology, College of Pharmacy, Prince Sattam Bin Abdulaziz University, Al-Kharj 16273, Saudi Arabia E-mail: [email protected]

b Department of Pharmacognosy, College of Pharmacy, Prince Sattam Bin Abdulaziz University, Al-Kharj 16273, Saudi Arabia

c Department of Pharmaceutical Chemistry, College of Pharmacy, Prince Sattam Bin Abdulaziz University, Al-Kharj 16273, Saudi Arabia

Various thymol and carvacrol derivatives have been synthesized to test the anticancer activity potential. Computational methods including network pharmacology and molecular docking approaches were utilized to identify and assess the potential biological targets relating to cancer. Amongst the synthesized derivatives the ethoxy-cyclohexyl analogues were consistently the most active against a panel of 10 different cancer cell lines covering a variety of origins. Biological target predictions revealed the AKT1 protein to be a core and central target of the most active compounds. Molecular docking identified a binding pocket within this protein in which the most active compounds bind. The incorporation of computational analysis methods and conventional structure–activity approaches identified analogues of thymol and carvacrol with the highest anticancer potential, and analyzed their possible biological targets in a comprehensive manner.

Supplementary files

• Supplementary information PDF (2728K)

Article information

Download Citation

Permissions.

Thymol and carvacrol derivatives as anticancer agents; synthesis, in vitro activity, and computational analysis of biological targets

M. A. Alamri, M. S. Abdel-Kader, M. A. Salkini and M. A. Alamri, RSC Adv. , 2024,  14 , 30662 DOI: 10.1039/D4RA03941F

This article is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported Licence . You can use material from this article in other publications, without requesting further permission from the RSC, provided that the correct acknowledgement is given and it is not used for commercial purposes.

To request permission to reproduce material from this article in a commercial publication , please go to the Copyright Clearance Center request page .

If you are an author contributing to an RSC publication, you do not need to request permission provided correct acknowledgement is given.

If you are the author of this article, you do not need to request permission to reproduce figures and diagrams provided correct acknowledgement is given. If you want to reproduce the whole article in a third-party commercial publication (excluding your thesis/dissertation for which permission is not required) please go to the Copyright Clearance Center request page .

Read more about how to correctly acknowledge RSC content .

Social activity

Search articles by author, advertisements.