list of systematic literature review

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters and Hedges
• Grey Literature
• Managing and Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Jul 23, 2024 3:40 PM
• URL: https://guides.library.ucla.edu/systematicreviews

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Prevent plagiarism. Run a free check.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved September 27, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.

• Duke NetID Login
• 919.660.1100
• Duke Health Badge: 24-hour access
• Accounts & Access
• Databases, Journals & Books
• Request & Reserve
• Training & Consulting
• Request Articles & Books
• Renew Online
• Reserve Spaces
• Reserve a Locker
• Study & Meeting Rooms
• Course Reserves
• Pay Fines/Fees
• Recommend a Purchase
• Access From Off Campus
• Building Access
• Computers & Equipment
• Wifi Access
• My Accounts
• Mobile Apps
• Known Access Issues
• Report an Access Issue
• All Databases
• Article Databases
• Basic Sciences
• Clinical Sciences
• Dissertations & Theses
• Drugs, Chemicals & Toxicology
• Grants & Funding
• Interprofessional Education
• Non-Medical Databases
• Search for E-Journals
• Search for Print & E-Journals
• Search for E-Books
• Search for Print & E-Books
• E-Book Collections
• Biostatistics
• Global Health
• MBS Program
• Medical Students
• MMCi Program
• Occupational Therapy
• Path Asst Program
• Physical Therapy
• Population Health
• Researchers
• Community Partners

Conducting Research

• Archival & Historical Research
• Black History at Duke Health
• Data Analytics & Viz Software
• Data: Find and Share
• Evidence-Based Practice
• NIH Public Access Policy Compliance
• Publication Metrics
• Qualitative Research
• Searching Animal Alternatives

Systematic Reviews

• Test Instruments

Using Databases

• JCR Impact Factors
• Web of Science

Finding & Accessing

• COVID-19: Core Clinical Resources
• Health Literacy
• Health Statistics & Data
• Library Orientation

Writing & Citing

• Creating Links
• Getting Published
• Reference Mgmt
• Scientific Writing

Meet a Librarian

• Request a Consultation
• Find Your Liaisons
• Register for a Class
• Request a Class
• Self-Paced Learning

Search Services

• Literature Search
• Systematic Review
• Animal Alternatives (IACUC)
• Research Impact

Citation Mgmt

• Other Software

Scholarly Communications

• About Scholarly Communications
• Publish Your Work
• Measure Your Research Impact
• Engage in Open Science
• Libraries and Publishers
• Directions & Maps
• Floor Plans

Library Updates

• Annual Snapshot
• Conference Presentations
• Contact Information
• Gifts & Donations
• What is a Systematic Review?

Types of Reviews

• Manuals and Reporting Guidelines
• Our Service
• 1. Assemble Your Team
• 2. Develop a Research Question
• 3. Write and Register a Protocol
• 4. Search the Evidence
• 5. Screen Results
• 6. Assess for Quality and Bias
• 7. Extract the Data
• 8. Write the Review
• Additional Resources
• Finding Full-Text Articles

Review Typologies

There are many types of evidence synthesis projects, including systematic reviews as well as others. The selection of review type is wholly dependent on the research question. Not all research questions are well-suited for systematic reviews.

• Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice and information is widely applicable.

Review the table to peruse review types and associated methodologies. Librarians can also help your team determine which review type might be appropriate for your project. 

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108.  doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: What is a Systematic Review?
• Next: Manuals and Reporting Guidelines >>
• Last Updated: Jun 18, 2024 9:41 AM
• URL: https://guides.mclibrary.duke.edu/sysreview
• Duke Health
• Duke University
• Duke Libraries
• Medical Center Archives
• Duke Directory
• Seeley G. Mudd Building
• 10 Searle Drive
• [email protected]

Systematic Reviews and Other Evidence Synthesis Types Guide

• Systematic Review and Other Evidence Synthesis Types
• Types of Evidence Synthesis
• Evidence Synthesis Comparison
• Are You Ready to Conduct an Evidence Synthesis?
• UT Southwestern Evidence Synthesis Services
• Task 1 - Find Articles
• Task 2 - Formulate Question
• Task 3 - Select Reporting Guideline
• Task 4 - Write and Register Protocol
• Evidence Synthesis - Search (Task 5)
• Screen and Appraise (Tasks 6 – 11)
• Synthesize (Tasks 12 – 15)
• Write Up Review (Task 16)

Systematic Review or Meta-Analysis

• Integrative Review
• Narrative/Literature Review
• Rapid Review
• Scoping Review
• Umbrella Review

Request UT Southwestern Library Evidence Synthesis/Systematic Review Services

The UT Southwestern Librarians provide two levels of Evidence Synthesis/Systematic Review (ES/SR) support.

Level 1 – Education (No Cost)

• A librarian will provide training about the systematic review process.
• Use the Training Request Form .

Level 2 – Librarian As ES/SR Team Member and Co-Author (Fee-Based)

• The librarian is an active contributor.
• UT Southwestern faculty
• UT Southwestern residents or fellows
• UT Southwestern Medical Center and University Hospitals clinicians
• Begin by completing the Evidence Synthesis/Systematic Review Request Form . For more information on the fees ($1,250 per PICO or equivalent question), see the "Costs" section in the form.
• If a Librarian joins the ES/SR Team, the ES/SR Team will complete the Evidence Synthesis/Systematic Review Library Services Agreement .
• Contact LibAsk Schedule an appointment with UT Southwestern librarians.

• Public Health Systematic Review Guidelines
• Electronic Books

Systematic Review – seeks to systematically search for, appraise and synthesize research evidence on a specific question, often adhering to guidelines on the conduct of a review.

Meta-analysis – a technique that statistically combines the results of quantitative studies to provide a more precise effect of the results. A good systematic review is essential to a meta-analysis of the literature.

Standards (see the Books tab) and guidelines have been developed on how to conduct and report systematic reviews and meta analyses.

Guidelines and Best Practices

• Cochrane Handbook for Systematic Reviews of Interventions, Current Version While this Handbook focuses on systematic reviews of interventions, Cochrane publishes five main types of systematic reviews , and has developed a rigorous approach to the preparation of each of the following: ❖ Effects of interventions ❖ Diagnostic test accuracy ❖ Prognosis ❖ Reviews of reviews (umbrella reviews) ❖ Reviews of methodology Part 3 provides considerations for tackling systematic reviews from different perspectives, such as when thinking about specific populations, or complex interventions, or particular types of outcomes. It comprises the following chapters: 16. Equity 17. Intervention complexity 18. Patient-reported outcomes 19. Adverse effects 20. Economic evidence 21. Qualitative evidence
• MECIR Manual The MECIR Standards present a guide to the conduct of new Cochrane Intervention Reviews, and the planning and conduct of updates. This online version will be kept up to date;a PDF of each section can be generated. All substantive changes will be noted here .
• Campbell Collaboration An international social science research network that produces high quality, open and policy-relevant evidence syntheses, plain language summaries and policy briefs.

Reporting Guidelines

• PRISMA 2020 Statement An evidence-based minimum set of items for reporting in systematic reviews and meta-analyses, PRISMA primarily focuses on the reporting of reviews evaluating the effects of interventions, but can also be used as a basis for reporting systematic reviews with objectives other than evaluating interventions (e.g. evaluating etiology, prevalence, diagnosis or prognosis). The PRISMA 2020 Statement is accompanied by the PRISMA 2020 Explanation and Elaboration paper.
• PRISMA 2020 Checklist The 27 checklist items pertain to the content of a systematic review and meta-analysis, which include the title, abstract, methods, results, discussion and funding. Note: As a member of the ES/SR Team, the UT Southwestern Librarian completes Item 7 (Search Strategy) in the checklist.
• PRISMA Flow Diagram The flow diagram depicts the flow of information through the different phases of a systematic review. It maps out the number of records identified, included and excluded, and the reasons for exclusions. Different templates are available depending on the type of review (new or updated) and sources used to identify studies.
• PRISMA for Searching Published in 2021, the checklist includes 16 reporting items, each of which is detailed with exemplar reporting and rationale. The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and therefore reproducible. For additional information, refer to the PRISMA for searching statement/exploratory paper .

Protocol Guidelines

• PRISMA for Systematic Review Protocols (PRISMA-P) PRISMA-P, published in 2015, includes a 17-item checklist intended to facilitate the preparation and reporting of a robust protocol for the systematic review. The developers note that there are many review types outside of this scope. They recommend that due to the general lack of protocol guidance for other types of reviews, reviewers preparing any type of review protocol make use of PRISMA-P as applicable.

Protocol Registration

• PROSPERO An international prospective register of systematic reviews. Key details from new Cochrane protocols are automatically uploaded into PROSPERO. It is produced by the Centre of Reviews and Dissemination, University of York, United Kingdom.

The Cochrane Library includes:

• Cochrane Database of Systematic Reviews – peer-reviewed systematic reviews and protocols)
• Cochrane Central Register of Controlled Trials (CENTRAL) – reports of randomized and quasi-randomized controlled trials
• Cochrane Clinical Answers (CCAs) – developed to inform point-of-care decision-making each CCA contains a clinical question, a short answer, and relevant outcomes data for the clinician
• JBI Systematic Review Register Members of the JBI Collaboration can register their review titles with JBI via completion of the online Systematic Review Title Registration Form. Once titles become registered with JBI, they are listed on the website. Titles are subsequently removed when the full protocol is publicly available, either published or posted to an accessible website.
• Cumpston, M. S., McKenzie, J. E., Welch, V. A., & Brennan, S. E. (2022). Strengthening systematic reviews in public health: guidance in the Cochrane Handbook for Systematic Reviews of Interventions, 2nd edition. J Public Health (Oxf), 44(4), e588-e592. https://doi.org/10.1093/pubmed/fdac036
• Jackson, N., & Waters, E. (2005). Criteria for the systematic review of health promotion and public health interventions. Health Promotion International, 20(4), 367-374. https://doi.org/10.1093/heapro/dai022
• Thomas, B. H., Ciliska, D., Dobbins, M., & Micucci, S. (2004). A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions. Worldviews on Evidence-Based Nursing, 1(3), 176-184. https://doi.org/10.1111/j.1524-475X.2004.04006.x

3 Should I undertake a scoping review or a systematic review? (Ask JBI) on YouTube (12:43).

Agency for Healthcare Research and Quality

• Training Modules for the Systematic Reviews Methods Guide (Agency for Healthcare Research and Quality)

Campbell Collaboration and the Open Learning Initiative

• Systematic Reviews and Meta-Analysis Open & Free (Carnegie Mellon University) Provides an overview of the steps involved in conducting a systematic (scientific) review of results of multiple quantitative studies.
• Cochrane Collaboration Online Training Includes links to learning resources relevant to systematic reviews and evidence-based medicine
• Cochrane Methodology Learning resources on key areas of Cochrane review methodology.

Joanna Briggs Institute

• JBI SUMARI Knowledge Base

Johns Hopkins University/Coursera

• Introduction to Systematic Review and Meta-Analysis (Johns Hopkins University)

University of North Carolina Health Sciences Library

• Introduction to Conducting a Systematic Review Workshop (University of North Carolina Health Sciences Library) Used with permission from the Systematic Reviews LibGuide developed by the University of North Carolina Health Sciences Library.
• Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., … PRISMA-P Group (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic reviews, 4(1), 1. https://doi.org/10.1186/2046-4053-4-1
• Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A. C., Welch, V. A., Whiting, P., & McKenzie, J. E. (2021). PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ, 372, n160. https://doi.org/10.1136/bmj.n160
• Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., & PRISMA-S Group (2021). PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic reviews, 10(1), 39. https://doi.org/10.1186/s13643-020-01542-z
• Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, the PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;349:g7647. https://doi.org/10.1136/bmj.g7647
• << Previous: Evidence Synthesis - Resources and Guidelines
• Next: Integrative Review >>
• Last Updated: Sep 24, 2024 12:06 PM
• URL: https://utsouthwestern.libguides.com/sres

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

How to Write a Systematic Review of the Literature

Affiliations.

• 1 1 Texas Tech University, Lubbock, TX, USA.
• 2 2 University of Florida, Gainesville, FL, USA.
• PMID: 29283007
• DOI: 10.1177/1937586717747384

This article provides a step-by-step approach to conducting and reporting systematic literature reviews (SLRs) in the domain of healthcare design and discusses some of the key quality issues associated with SLRs. SLR, as the name implies, is a systematic way of collecting, critically evaluating, integrating, and presenting findings from across multiple research studies on a research question or topic of interest. SLR provides a way to assess the quality level and magnitude of existing evidence on a question or topic of interest. It offers a broader and more accurate level of understanding than a traditional literature review. A systematic review adheres to standardized methodologies/guidelines in systematic searching, filtering, reviewing, critiquing, interpreting, synthesizing, and reporting of findings from multiple publications on a topic/domain of interest. The Cochrane Collaboration is the most well-known and widely respected global organization producing SLRs within the healthcare field and a standard to follow for any researcher seeking to write a transparent and methodologically sound SLR. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), like the Cochrane Collaboration, was created by an international network of health-based collaborators and provides the framework for SLR to ensure methodological rigor and quality. The PRISMA statement is an evidence-based guide consisting of a checklist and flowchart intended to be used as tools for authors seeking to write SLR and meta-analyses.

Keywords: evidence based design; healthcare design; systematic literature review.

PubMed Disclaimer

Similar articles

• The future of Cochrane Neonatal. Soll RF, Ovelman C, McGuire W. Soll RF, et al. Early Hum Dev. 2020 Nov;150:105191. doi: 10.1016/j.earlhumdev.2020.105191. Epub 2020 Sep 12. Early Hum Dev. 2020. PMID: 33036834
• Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas. Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, Moraleda C, Rogers L, Daniels K, Green P. Crider K, et al. Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.
• Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data: the PRISMA-IPD Statement. Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, Tierney JF; PRISMA-IPD Development Group. Stewart LA, et al. JAMA. 2015 Apr 28;313(16):1657-65. doi: 10.1001/jama.2015.3656. JAMA. 2015. PMID: 25919529
• Systematic Reviews in Sports Medicine. DiSilvestro KJ, Tjoumakaris FP, Maltenfort MG, Spindler KP, Freedman KB. DiSilvestro KJ, et al. Am J Sports Med. 2016 Feb;44(2):533-8. doi: 10.1177/0363546515580290. Epub 2015 Apr 21. Am J Sports Med. 2016. PMID: 25899433 Review.
• The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. Liberati A, et al. J Clin Epidemiol. 2009 Oct;62(10):e1-34. doi: 10.1016/j.jclinepi.2009.06.006. Epub 2009 Jul 23. J Clin Epidemiol. 2009. PMID: 19631507
• A systematic review and meta-analysis of balance training in patients with chronic ankle instability. Guo Y, Cheng T, Yang Z, Huang Y, Li M, Wang T. Guo Y, et al. Syst Rev. 2024 Feb 12;13(1):64. doi: 10.1186/s13643-024-02455-x. Syst Rev. 2024. PMID: 38347564 Free PMC article.
• Association between infection and the onset of giant cell arteritis and polymyalgia rheumatica: a systematic review and meta-analysis. Pacoureau L, Barde F, Seror R, Nguyen Y. Pacoureau L, et al. RMD Open. 2023 Nov;9(4):e003493. doi: 10.1136/rmdopen-2023-003493. RMD Open. 2023. PMID: 37949615 Free PMC article.
• From Social Rejection to Welfare Oblivion: Health and Mental Health in Juvenile Justice in Brazil, Colombia and Spain. Carbonell Á, Georgieva S, Navarro-Pérez JJ, Botija M. Carbonell Á, et al. Int J Environ Res Public Health. 2023 May 29;20(11):5989. doi: 10.3390/ijerph20115989. Int J Environ Res Public Health. 2023. PMID: 37297594 Free PMC article. Review.
• Why is didactic transposition in disaster education needed by prospective elementary school teachers? Noviana E, Syahza A, Putra ZH, Hadriana, Yustina, Erlinda S, Putri DR, Rusandi MA, Biondi Situmorang DD. Noviana E, et al. Heliyon. 2023 Apr 18;9(4):e15413. doi: 10.1016/j.heliyon.2023.e15413. eCollection 2023 Apr. Heliyon. 2023. PMID: 37128333 Free PMC article. Review.
• Comparative analysis of efficacy of different combination therapies of α-receptor blockers and traditional Chinese medicine external therapy in the treatment of chronic prostatitis/chronic pelvic pain syndrome: Bayesian network meta-analysis. Zhang K, Zhang Y, Hong S, Cao Y, Liu C. Zhang K, et al. PLoS One. 2023 Apr 20;18(4):e0280821. doi: 10.1371/journal.pone.0280821. eCollection 2023. PLoS One. 2023. PMID: 37079509 Free PMC article.
• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

Other Literature Sources

• scite Smart Citations

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Systematic Literature Reviews: An Introduction

• Proceedings of the Design Society International Conference on Engineering Design 1(1):1633-1642
• CC BY-NC-ND 4.0
• Conference: International Conference on Engineering Design 2019

• CentraleSupélec

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• Ozge Ceylin Yıldırım

• Hatice Kübra Saraoğlu Yumni
• Octaviani Gita Putri
• Disast Prev Manag
• Tsitsi Trina Magadza

• Stanishkar Thevaruparambil Sunil
• Akriti Sharma
• Girish Kumar Jha
• COMPUT NETW
• Haopeng Huang
• Qingqing Huang

• Liandro Antonio Tiongson Tabora
• Datu Sajid Islam Sinsuat Ampatuan
• Marian Angelique Chamen Castaneda
• Eury Ellyn Manaloto Zulueta
• DESIGN STUD

• BMC MED RES METHODOL

• Martin Stacey

• L. Shamseer
• David Moher
• Alessandro Liberati

• David Jones

• J CLEAN PROD

• Alissa Kendall
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Systematic Literature Review or Literature Review?

• 3 minute read
• 61.9K views

Table of Contents

As a researcher, you may be required to conduct a literature review. But what kind of review do you need to complete? Is it a systematic literature review or a standard literature review? In this article, we’ll outline the purpose of a systematic literature review, the difference between literature review and systematic review, and other important aspects of systematic literature reviews.

What is a Systematic Literature Review?

The purpose of systematic literature reviews is simple. Essentially, it is to provide a high-level of a particular research question. This question, in and of itself, is highly focused to match the review of the literature related to the topic at hand. For example, a focused question related to medical or clinical outcomes.

The components of a systematic literature review are quite different from the standard literature review research theses that most of us are used to (more on this below). And because of the specificity of the research question, typically a systematic literature review involves more than one primary author. There’s more work related to a systematic literature review, so it makes sense to divide the work among two or three (or even more) researchers.

Your systematic literature review will follow very clear and defined protocols that are decided on prior to any review. This involves extensive planning, and a deliberately designed search strategy that is in tune with the specific research question. Every aspect of a systematic literature review, including the research protocols, which databases are used, and dates of each search, must be transparent so that other researchers can be assured that the systematic literature review is comprehensive and focused.

Most systematic literature reviews originated in the world of medicine science. Now, they also include any evidence-based research questions. In addition to the focus and transparency of these types of reviews, additional aspects of a quality systematic literature review includes:

• Clear and concise review and summary
• Comprehensive coverage of the topic
• Accessibility and equality of the research reviewed

Systematic Review vs Literature Review

The difference between literature review and systematic review comes back to the initial research question. Whereas the systematic review is very specific and focused, the standard literature review is much more general. The components of a literature review, for example, are similar to any other research paper. That is, it includes an introduction, description of the methods used, a discussion and conclusion, as well as a reference list or bibliography.

A systematic review, however, includes entirely different components that reflect the specificity of its research question, and the requirement for transparency and inclusion. For instance, the systematic review will include:

• Eligibility criteria for included research
• A description of the systematic research search strategy
• An assessment of the validity of reviewed research
• Interpretations of the results of research included in the review

As you can see, contrary to the general overview or summary of a topic, the systematic literature review includes much more detail and work to compile than a standard literature review. Indeed, it can take years to conduct and write a systematic literature review. But the information that practitioners and other researchers can glean from a systematic literature review is, by its very nature, exceptionally valuable.

This is not to diminish the value of the standard literature review. The importance of literature reviews in research writing is discussed in this article . It’s just that the two types of research reviews answer different questions, and, therefore, have different purposes and roles in the world of research and evidence-based writing.

Systematic Literature Review vs Meta Analysis

It would be understandable to think that a systematic literature review is similar to a meta analysis. But, whereas a systematic review can include several research studies to answer a specific question, typically a meta analysis includes a comparison of different studies to suss out any inconsistencies or discrepancies. For more about this topic, check out Systematic Review VS Meta-Analysis article.

Language Editing Plus

With Elsevier’s Language Editing Plus services , you can relax with our complete language review of your systematic literature review or literature review, or any other type of manuscript or scientific presentation. Our editors are PhD or PhD candidates, who are native-English speakers. Language Editing Plus includes checking the logic and flow of your manuscript, reference checks, formatting in accordance to your chosen journal and even a custom cover letter. Our most comprehensive editing package, Language Editing Plus also includes any English-editing needs for up to 180 days.

How to Make a PowerPoint Presentation of Your Research Paper

Step-by-Step Guide: How to Craft a Strong Research Hypothesis

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

Penn State University Libraries

• Home-Articles and Databases
• Asking the clinical question
• PICO & Finding Evidence
• Evaluating the Evidence
• Systematic Review vs. Literature Review
• Fall 2024 Workshops
• Nursing Library Instruction Course
• Ethical & Legal Issues for Nurses
• Useful Nursing Resources
• Writing Resources
• LionSearch and Finding Articles
• The Catalog and Finding Books

Know the Difference! Systematic Review vs. Literature Review

It is common to confuse systematic and literature reviews as both are used to provide a summary of the existent literature or research on a specific topic.  Even with this common ground, both types vary significantly.  Please review the following chart (and its corresponding poster linked below) for the detailed explanation of each as well as the differences between each type of review.

• What's in a name? The difference between a Systematic Review and a Literature Review, and why it matters by Lynn Kysh, MLIS, University of Southern California - Norris Medical Library
• << Previous: Evaluating the Evidence
• Next: Fall 2024 Workshops >>
• Last Updated: Aug 27, 2024 1:21 PM
• URL: https://guides.libraries.psu.edu/nursing

• AI Tools for Systematic Literature Reviews

Sep 25, 2024 | News and Trends

Polly Field , Tomas Rees and Richard White , Oxford PharmaGenesis, UK

Email your questions and comments on this article to  [email protected] .

A systematic literature review (SLR) allows us to find and evaluate existing evidence to answer a specific research question. But with increasing interest in reviews that are rapidly updated and cover a wide evidence-base – an evidence base that is increasing with the exponential growth in the volume of scientific literature – we see increasing demands on the resources needed to develop these reviews, with multiple people required to ensure that the methods are reproducible, and the evidence is accurate. This is one reason that many people are turning to artificial intelligence (AI), especially when considering large and complex SLRs, which would otherwise not be feasible without AI because of cost or turnaround time.

Here we consider how and when AI should be used to develop SLRs for publication, drawing on our experience as AI users, SLR experts and publication professionals. We provide guidance on the potential applications of AI, and the issues that should be considered when choosing an AI-based approach, deliberately avoiding describing specific tools, as these are evolving daily. 

What advantages can AI bring?

AI can help with the scale, efficiency, quality and understanding of some SLRs.

• Scale/volume of evidence: AI can enable literature reviews at scales not previously considered feasible. The larger the SLR, the greater the benefit.
• Efficiency: AI can learn from initial work, speeding up subsequent updates, for example by using LLM screening questions (prompts) optimized in the initial review or by fine-tuning ML algorithms on the original data set.
• Quality: use of AI, with appropriate checks, has the potential to improve accuracy and reduce researcher bias.
• Understanding: AI can help summarize, group and visualize data, revealing patterns and trends in the underlying information.

How can AI help in the different stages of an SLR for publication?

AI can help across the full workflow of SLR development if used correctly and with people to check and adjust the output. 2 In Figure 1, we describe current major uses by stage, deliberately not describing specific tools.

Considerations for evaluating AI tools for use in SLRs for publication

Before using a tool or platform, it should be critically appraised to understand potential advantages and risks. This includes testing of usability – whether the team can easily interact with the AI and incorporate it into the SLR workflow – and testing of performance, including sensitivity, specificity, time and resource use. There are important ethical, practical, business and legal considerations when using AI relating to the following risks.

• Any data, information or files uploaded into an AI tool may essentially become public domain or be used to train a provider’s AI model, potentially breaching confidentiality and/or copyright. Choice of provider and careful assessment of how they use any data uploaded to the tool is key.
• LLMs are trained on massive data sets of existing text. By analyzing these data, the AI learns the patterns and relationships within that content, which, following more training, gives them the ability to perform specific tasks like answering questions about a document. AI does not understand what it is creating and cannot judge the quality, usefulness or correctness of what it generates.
• In general use, LLMs tend to adhere to text patterns that occur in their training set. They can easily generate material that is grammatically correct and convincingly written, but factually or logically incorrect. In the SLR context, this is mitigated by using retrieval-augmented generation, in which contextual content is provided to the LLM for use in developing the output. Many tools enable the user to easily validate LLM responses by highlighting the source text. This helps with fact-checking, but errors of omission remain a challenge.

Fundamental principles for the use of AI in SLRs for publication

The use of AI in any project should be aligned with relevant formal institutional and corporate policies for all parties involved, assuring a consistent and ethical approach. A fundamental principle of any publication is that the authors of an SLR are responsible for the quality and accuracy of their work, whether developed entirely by themselves or with assistance from AI; the use of AI does not alter the authors’ ownership of quality.

• Explainability and reproducibility – these are key challenges with SLRs, even when conducted by humans. LLMs add to the problem because they are by nature stochastic. Furthermore, differences in training sets and decisions in model development are often not adequately disclosed by the developers (this affects some tools more than others).
• Appropriateness of AI to the question being asked and the available evidence – the potential benefits of using AI in the various stages of the SLR process will differ with every project. For example, a novel therapeutic modality being used in a previously untreatable rare disease may have a small corpus of literature utilizing a broad range of terminology and new endpoints, which will be challenges for the use of existing LLMs or training of new ML approaches.
• Transparency in the use of AI and its reporting – everyone involved in an SLR should agree to the use of AI, and the approaches taken should be clearly described in subsequent publications in line with relevant journal or conference guidelines.
• Assuring quality and accuracy – humans providing expert knowledge and oversight are crucial; AI should be considered an augmentation, not a human replacement. A rigorous methodology with multiple layers of human validation should be developed and agreed by all authors involved in the SLR, noting that reviewing AI-generated material requires not just more checking, but a different focus while checking.

What guidelines exist for the use of AI in SLRs for publication?

The use of AI is captured in many guidelines related to publication and SLR methodology, and it is important to ensure compliance. The International Committee of Medical Journal Editors recommendations note that authors using AI to conduct a study (which we take to include AI use in literature reviews) should describe its use in the methods section in sufficient detail to enable replication of the approach. 3 It further notes that authors using AI for writing assistance should report this in the acknowledgments section. 3   Many journals also have policies about the use of generative AI, and authors should check the policy of the specific journal ahead of submitting a review to ensure compliance. Overall, policies accept that authors can use generative AI, providing that they maintain responsibility for the content and accuracy of the work. This can mean that AI is used to help with exploring ideas, searching and classifying literature, and improving the language used, but that authors are accountable for the originality, validity and integrity of their work. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines are widely used for the reporting of SLRs, and the 2020 statement includes guidance on how the use of AI should be reported. 4 This is limited to the early stage of the review, with authors asked to report the number of records that are removed before screening. 4

Conclusions

AI is here to stay and can help medical publication professionals develop high-quality work, including SLRs for publication. It is up to all of us to keep engaging, to be alert to new approaches and applications, and to evaluate, upskill and start to use these new and evolving tools. Above all, we need to stay transparent and acknowledge our use of AI, making clear that humans remain responsible for the quality of SLRs.

Acknowledgments

This article was critically reviewed by Kim Wager, Martin Callaghan, Gemma Carter and Jacob Willet from Oxford PharmaGenesis; Jody Filkowski from Medlior contributed to the original planning and content of the article.

Bibliography

• Higgins JPT, Thomas J, Chandler J et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .
• Teperikidis E, Boulmpou A, Potoupni V et al. Does the long-term administration of proton pump inhibitors increase the risk of adverse cardiovascular outcomes? A ChatGPT powered umbrella review. Acta Cardiol 2023;78:980–8.
• International Committee of Medical Journal Editors. Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals (updated January 2024). Available from https://www.icmje.org/news-and-editorials/icmje-recommendations_annotated_jan24.pdf (Accessed 12 August 2024).
• Page MJ, McKenzie JE, Bossuyt PM et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev 2021;10:89.

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)
• News and Trends

Recent Posts

• Publication Planning for Medical Devices: The Opportunities in an Underserved Field
• Proceed with Data & Author Changes to a 2-year-old Manuscript?
• Mastering the Art of Meeting Facilitation: Part 2
• Mastering the Art of Meeting Facilitation: Part 1
• September 2024
• February 2024
• January 2024
• December 2023
• November 2023
• October 2023
• September 2023
• August 2023
• February 2023
• January 2023
• December 2022
• November 2022
• October 2022
• August 2022
• February 2022
• January 2022
• December 2021
• November 2021
• October 2021
• September 2021
• August 2021
• February 2021
• January 2021
• December 2020
• November 2020
• October 2020
• September 2020
• August 2020
• February 2020
• January 2020
• December 2019
• November 2019
• October 2019
• September 2019
• August 2019
• February 2019
• January 2019
• December 2018
• November 2018
• October 2018
• September 2018
• August 2018
• February 2018
• January 2018
• December 2017
• November 2017
• October 2017
• September 2017
• August 2017
• February 2017
• January 2017
• August 2016

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PLoS Comput Biol
• v.9(7); 2013 Jul

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

• interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
• an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
• a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

• keep track of the search items you use (so that your search can be replicated [10] ),
• keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
• use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
• define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
• do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

• discussing in your review the approaches, limitations, and conclusions of past reviews,
• trying to find a new angle that has not been covered adequately in the previous reviews, and
• incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

• be thorough,
• use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
• look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

• the major achievements in the reviewed field,
• the main areas of debate, and
• the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

The Evolution of Emojis for Sharing Emotions: A Systematic Review of the HCI Literature

With the prevalence of instant messaging and social media platforms, emojis have become important artifacts for expressing emotions and feelings in our daily lives. We ask how HCI researchers have examined the role and evolution of emojis in sharing emotions over the past 10 years. We conducted a systematic literature review of papers addressing emojis employed for emotion communication between users. After screening more than 1,000 articles, we identified 42 articles of studies analyzing ways and systems that enable users to share emotions with emojis. Two main themes described how these papers have (1) improved how users select the right emoji from an increasing emoji lexicon, and (2) employed emojis in new ways and digital materials to enhance communication. We also discovered an increasingly broad scope of functionality across appearance, medium, and affordance. We discuss and offer insights into potential opportunities and challenges emojis will bring for HCI research.

1. Introduction

With the prevalence of instant messaging and social media platforms, emojis have become important artifacts for expressing emotions and feelings in computer-mediated communication. According to the Unicode Consortium, 92% of the world’s online population uses emojis (Greg, [n. d.] ) . In Adobe Research’s’ 2022 U.S. Emoji Trend Report, 91% of respondents said emojis make it easier to express themselves, and 75% of them felt more connected to people who use emojis (Fonts team, 2022 ) . While emojis began as a set of simple icons in the 1990s, they experienced a rapid expansion in variety and complexity due to the advancement of smartphones and the growing number of social media platforms in the world (noa, 2023 ) . Furthermore, the increasing personalization and incorporation of multimedia components have transformed emojis into new interactive objects, such as stickers, avatars, and memes (Konrad et al . , 2020 ) . As such, emojis have significantly transformed and enhanced online communication over the past decades, and become a cultural staple in and of itself (Alonso, 2017 ) .

One of the main reasons for emoji’s popularity is their capability to provide nonverbal markers of emotion to text messages (Wiseman and Gould, 2018 ; Kelly and Watts, 2015 ) . Previous research has shown that emojis can strengthen the meaning of a message, helping readers better understand its sentiment by providing visual and emotional context (Chairunnisa and A.S., 2017 ) . Their designs and pictorial elements enable nonverbal cues from text and enable users to express emotions outside of words (Gawne and McCulloch, 2019 ; Novak et al . , 2015 ; Boutet et al . , 2021 ) . As a result, users can convey different tones and emotions that might be lost or hard to express using plain text (Highfield and Leaver, 2016 ) . Furthermore, emojis are used in highly personalized ways, where they may take on multiple functions depending on the relationship between their users (Wiseman and Gould, 2018 ) . People can make their own meaning out of emojis to create a “shared uniqueness,” where an emoji can completely change its meaning depending on who the emoji is sent to, as well as the sender’s relationship with that person. Emojis have also become more customizable, allowing users to select different skin tones, genders, and body characteristics to present themselves more authentically (Robertson et al . , 2018 ) . Because of these customizable and non-textual properties, social media platforms and messaging apps have adapted emojis for creating stickers and reactions, increasing their already widespread ubiquity as well as providing an effort-free way to use them (Seta, 2018 ; Konrad et al . , 2020 ) . Emojis are now capable of eliciting joy among users and supporting more meaningful communications across a broader range of spaces (Chen and Siu, 2017 ) .

Despite the popular and ubiquitous use of emojis in modern online communication, there is still a gap in the literature surrounding the way emojis have evolved. As emojis continue to shape online communication between millions of users, HCI researchers are called to understand how emojis’ unique roles, designs, and functions have evolved in online communication systems over the past years (Bai et al . , 2019 ) . Mapping the existing research on emojis can help HCI researchers transform emoji’s design and application, as well as envision more effective, engaging, and interactive online communication systems. Prior literature has identified different ways in which people, platforms, and systems employ emojis, taking a broader view of emoji to include usage as communication markers, pictures, and even as lingua franca for worldwide users (Alshenqeeti, 2016 ; Gawne and McCulloch, 2019 ; Maier, 2023 ; Dresner and Herring, 2010 ) . As platforms push forward with novel introductions to the emoji space, aiming to garner the attention of users as emoji originally did, they begin to differ wildly from traditional emojis (Bai et al . , 2019 ; Seta, 2018 ) . However, these systems are still primarily used to fill a similar role to emojis—to express emotions, clarify intents, and for fun (Tang and Hew, 2019 ) . Therefore, examining the emojis’ visual appearance, functionality, and mode of communication will be fundamental to understanding their design, development, and use in the future. In this paper, we seek to answer the following research questions:

RQ1: How have emojis evolved to better communicate users’ emotions?

RQ2: How do the changes in emojis affect their ability to communicate emotion?

To answer the research questions, we conducted a systematic literature review to map how the HCI community has studied, designed, and employed emojis for sharing emotions in online communication. After screening over 1,100 articles from the ACM Digital Library, IEEExplore, and Web of Science, we identified 42 relevant articles that described methods, systems, models, and studies employing emojis to share emotions, reactions, and feelings. From this corpus, we conducted a thematic analysis to identify enduring themes that described the findings and contributions of these articles. We found that our corpus revolved around two different themes: (1) improving how users find and select emojis to represent their emotions and (2) employing emojis in new ways and digital materials to enhance their communication. The first theme included methods to improve recommendations, input selection, and emojis themselves. The second theme included enhancements and new characteristics of emojis to expand their communicative space.

Our contributions are as follows. First, we provide a comprehensive review of the HCI literature on emojis for sharing emotions in computer-mediated communication. We focus on their function as tools for facilitating communication between different people and their evolution over the past few years. We present our results, which indicate that emojis’ designs and purposes have drastically changed over time, as researchers seek to expand the domain of emojis through different mediums of communication, ways of usage, and means of emotion conveyance. We highlight an increasing design divergence within the themes generated from our corpus, with one branch continuing to further improve how users interact with modern emoji, and the other branch exploring more experimental methods to connect user meaning with emoji communicative function. Finally, we discuss and offer insights into the potential opportunities and challenges that emojis will bring for future HCI research.

2. Background

In this section, we review relevant studies covering emotions in computer-mediated communication (CMC). We then revise the evolution of emojis and their use, including describing their increasing affordances. Lastly, we conclude this section with our research question.

2.1. Sharing Emotions in Computer-Mediated Communication

One of the main challenges in online communication is overcoming the lack of non-verbal cues (Archer and Akert, 1977 ) . While in face-to-face (F2F) communication individuals can employ nonverbal cues that significantly contribute to the understanding of their message, such as facial, tonal, and physical expressions, those elements are missing in online communication. This is due to the tools and adaptations that communicators have developed in response to the inherent lack of nonverbal cues in CMC. One such adaptation is making nonverbal cues verbal by expressing their intent directly (Walther et al . , 2005 ) .

According to media richness theories, the effectiveness of online communication technologies relies on the amount and quality of transmitted information (Ishii et al . , 2019 ) . While F2F interactions allow individuals to use their bodies, other materials, and the space to communicate, the same information cannot be transmitted through low-richness mediums such as documents, notes, or memos. People prefer either lean or rich communication channels depending on the media sender’s relation and motives (Kwak, 2012 ) , as there are benefits and drawbacks to both. While the most common reason that people choose F2F over CMC is because of the ability to use nonverbal cues, the most common reason for using CMC over F2F is to transmit information remotely and shield themself from the message recipient (Riordan and Kreuz, 2010 ) .

In the context of CMC, the most well-researched tools for conveying emotion are emoticons and emojis (Riordan, 2017 ) . They can be used as nonverbal markers of emotion, skepticism, politeness, and more within CMC communication (Vandergriff, 2013 ) . The use of such icons for visualizing emotions enables ‘leaner’ ways to communicate emotions using CMC, approaching the ‘richer’ expressiveness of F2F communication (Ahn et al . , 2011 ; Derks et al . , 2008 ) .

2.2. Emojis Evolution

Emojis are defined as colorful and cartoon pictographs embedded in text messages posted on websites and online devices (noa, 2016 ) . The word ‘emoji’ comes from the Japanese words picture (‘e’) and written character (‘moji’) (noa, 2016 ) . They were initially created to provide visual representations and fill in emotional cues that are hard to express or convey using only text (Miller et al . , 2016 ) . Over the past three decades, computer science researchers developed solutions to enable representations of these nonverbal cues in textual communication. One of the earlier forms was ‘emoticons,’ which are letters and punctuation marks arranged within the text to represent an expressive face (e.g., :D ) (Bai et al . , 2019 ) . Nass posited that emoticons provide a mechanism to transmit emotion when individuals do not have voice (noa, 2007 ) . By the end of the 1990s, the first set of dedicated pictographs was released by Japanese phone carrier SoftBank (Burge, 2019 ) . While this is thought to be the first set of emoji, it was not until Shigetaka Kurita created a set of 176 emojis for DoCoMo’s ‘i–mode’ in 1999 that emoji began to see widespread use. The popularity of i–mode emojis led other platforms worldwide (including MSN Messenger, BlackBerry, and Apple) to create their own emoji sets, each slightly different than the other (Evers, 2003 ; Alonso, 2017 ) .

While unique emoji sets across platforms served to draw engagement to their respective messaging platforms, the lack of cross–platform standardization meant that emoji was restricted from broader internet usage. Recognizing the need for standardization, there was a proposal as early as 2000 to encode the i–mode emojis into Unicode (noa, 2016 ) . In 2006, Google worked on converting the various Japanese emojis into Unicode, which continued through 2007. Finally, in 2010, the Unicode Consortium accepted the proposal to adopt 625 emoji characters. Since then, the Unicode Consortium has been responsible for decisions on new emoji characters as well as generally ensuring their equivalence across platforms (noa, 2024 ) . Modern emojis are characterized by a defined set of agreed-upon expressions, which are determined by the Unicode Consortium. According to Emojipedia , emojis appear differently across platforms since the “artwork is specific to which fonts are included on the system” (noa, [n. d.] ) . This may reflect different design philosophies and artistic styles, as well as satisfy customers’ needs for uniqueness to draw them to their platform as opposed to competitors (Abosag et al . , 2020 ) .

The integration of emojis into major tech platforms increased their worldwide popularity. In 2011, Apple added an emoji keyboard to its mobile operative system (iOS), making it easy for iPhone users to access and use emojis (Alonso, 2017 ) . Google followed this decision and integrated emojis into Android (Alonso, 2017 ) . The widespread use of these operating systems helped propel emojis into mainstream usage. Moreover, social media platforms like Facebook, Twitter, WhatsApp, and Instagram adopted and promoted emojis (Alonso, 2017 ) .

In more recent years, scholars have discussed the evolution of emoji into other forms of communication artifacts, most notably stickers (Konrad et al . , 2020 ) . Stickers are pictograph “images, usually larger than graphical emoticons and emojis, offered as thematic sets in the communication interfaces of instant messaging apps and social networking services, often organized in tabs and personalized collections” (Seta, 2018 ) . While emojis can be used in conjunction with text or other media providing context (McCulloch, [n. d.] ) , stickers can only be sent as the sole message, with no additional context provided within the message (Bai et al . , 2019 ) . Moreover, stickers are highly contextual and are not fully designed to convey emotions. While emojis are simple and generalizable tools to share emotions, stickers can transmit other kinds of information, such as jokes, memes, impressions, or animations that require users’ social context to be understood (Wang et al . , 2019a ; Cha et al . , 2018 ) . In addition, stickers are tied directly to the platform that the sender is on. While similar stickers may exist across platforms, they lack the intrinsic ubiquity of emoji (Lee et al . , 2016 ) . The unfettered access to creating and using stickers, alongside the fact that stickers are unique to each platform, has led to a greater diversity of stickers overall. As of 2024, there are less than 4,000 total emojis, while back in 2017, there were nearly 500 thousand individual stickers on one messaging app (iMessage) alone (List, 2017 ) .

2.3. Transmitting emotions through emojis

To understand the role of emojis in communicating emotions, we build on Derks et al. (Derks et al . , 2008 ) ’s emotion communication framework. According to this framework, transmitting emotions through CMC faces reduces social presence when expressing emotions and reduces visibility when recognizing emotions. As a consequence, people can feel more comfortable expressing more intense and frequent positive and negative emotions when compared to F2F.

Researchers have extensively discussed the effectiveness of emojis to communicate emotions. Alshenqeeti (Alshenqeeti, 2016 ) shows that emojis convey tone, intent, and feelings to fill their role as nonverbal cues. The nonverbal information that emoji conveys in CMC has been likened to physical gestures (Gawne and McCulloch, 2019 ) and facial expressions (Grosz et al . , 2023 ) in F2F communication. These nonverbal cues reduce ambiguity as well as alter the emotional intensity within digital messages (Archer and Akert, 1977 ; Lee and Wagner, 2002 ) . Meanwhile, Dresner et al. (Dresner and Herring, 2010 ) highlight the ability of emoji to communicate pragmatically, without necessarily attaching emotion. As a ‘pragmatic indicator’, emojis can be used to replace words or ideas in a sentence without changing the emotional context (Dresner and Herring, 2010 ) . The effect of emoji in a message depends heavily on the context in which it is used, how it is used, and how it is interpreted by the reader (Alshenqeeti, 2016 ; Dresner and Herring, 2010 ) .

In sum, HCI researchers have delved into solutions and experimented with diverse systems to support and examine the effectiveness of communicating emotions through emojis. Consequently, our goal is to investigate how emojis have been employed to facilitate this purpose over the past years. By conducting a systematic literature review, we aim to delineate the solutions, insights, and recommendations that emojis have already provided and could offer to advance CMC communication further.

3. Methodology

To answer our research questions, we followed a scoping review methodology (Arksey and O’Malley, 2005 ; Levac et al . , 2010 ) to synthesize existing research and identify trends in research employing emojis. This methodology allows researchers to map the literature and identify gaps in a specific area of research. Scoping reviews are transparent, comprehensive, less prone to bias, and make it easier to reproduce the detailed information reported about each step of the review and how it is conducted (Harris et al . , 2019 ) . They have gained prominence in the HCI community as they allow synthesizing and comprehending the development of different research directions (Shibuya et al . , 2022 ; Rogers et al . , 2022 ; Hirzle et al . , 2023 ; Cosio et al . , 2023 ) . We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to report the methods and results (Page et al . , 2021 ) . PRISMA covers four stages in the review process: Identification, Screening, Eligibility, and Inclusion. We collected papers and tabulated them in a shared Google Spreadsheet, capturing metadata such as publication year, abstract, and keywords. Once the metadata was recorded, we hid the articles’ authors to avoid potential bias during the coding phase.

3.1. Eligibility Criteria for Inclusion of Articles

Inclusion. This review contains articles that pertain to emojis or systems centered on emojis. Articles were eligible for inclusion if they focused on emojis as they pertain to communicating, expressing, or sharing emotions. We included papers that were written starting in 2004 as we want to focus on developments in emoji use.

Exclusion. We excluded articles that were written in a language other than English or could not be downloaded from their websites. We also excluded articles that did not study any form of communicating emotions, moods, or feelings. We removed studies that only analyzed current norms of emoji usage and perception, rather than their communicative purposes. We decided not to include derivate terms from emojis, such as “sticker,” “avatars,” or “emoticons,” to keep our focus exclusively on the evolution of emojis. Lastly, we removed meta-analysis or literature review articles as our goal was to analyze research articles.

3.2. Data Sources

We chose three data sources highly employed by HCI researchers: the ACM Digital Library, the IEEE Xplore Digital Library, and the Web of Science database (WoS). We built search queries to find papers examining emojis as a form of non-verbal/non-textual communication for emotion. We iterated through several rounds of search queries before deciding which to use. The final search query was: “emotion” AND “communication” AND “emoji” . We chose these search terms as they allowed us to find emoji-related studies that focus on how to communicate emotions through emojis. The same search query was used in all three databases.

We conducted the search in June 2024 and gathered 1,122 articles through the ACM Digital Library, 80 through the IEEE Xplore Digital Library, and 187 through the Web of Science. In total, we collected 1,389 articles that included duplicates. From this search, only two articles were published before 2014. The other 1,120 articles from ACM DL were published from 2015 onwards. We exported the results of the IEEE Xplore and Web of Science searches to CSV files, and the results of the ACM Digital Library search were exported to BibTeX as citations through their website. This BibTeX file was then converted to a CSV file. We then tabulated the articles into a Google Spreadsheet. We then removed 120 duplicates using the articles’ DOIs, resulting in 1,149 articles.

3.3. Article Selection

The first author (henceforth referred to as the coder) manually screened all retrieved articles according to the Inclusion and Exclusion criteria in a three-stage process. In the first stage, the coder reviewed the articles’ titles and determined their eligibility for inclusion according to the criteria previously set. This produced 203 articles that were deemed eligible. In the second stage, the coder reviewed the articles’ abstracts in addition to their titles. If the coder found that the articles met the eligibility criteria, then they were retained. After the second stage of review, 72 articles were selected. In the third stage, the coder reviewed the full text of the eligible articles in order to determine eligibility. The coder then reviewed all selections made and finalized the corpus. In total, 42 articles were deemed eligible.

3.4. Data Extraction and Synthesis

Once the final corpus was established, the coder extracted data to synthesize their characteristics, methods, and findings. Using a Google Spreadsheet, the coder extracted data pertaining to:

Year of publication

Type of System’s Innovation (e.g., using biosignals, using voice)

Article Type: Research paper, short paper (four pages or fewer), and posters.

Evaluation type: User Study, Deployment Study, Benchmark Testing

Methodology: Quantitative, Qualitative, or Mixed Methods.

Using these 42 articles, we conducted a thematic analysis to create cohesive themes that described this final corpus (Braun and Clarke, 2006 ) . The goal of thematic analysis is to discover themes from the data while minimizing confirmation bias. We began by familiarizing ourselves with the articles and taking notes on the main contributions of the paper, as well as the ideas mentioned throughout. These notes were compiled in a Google spreadsheet and used to inductively generate codes without trying to fit these codes into preexisting coding frameworks or theories (Nowell et al . , 2017 ; Braun and Clarke, 2006 ) . The coder generated codes by revising each paper and used these codes to recognize patterns throughout the corpus. The codes were not mutually exclusive since the articles could address multiple codes. The coder iterated the initial codes through several rounds of adjustments until no new ideas emerged from them. Following this, the codes were collected into themes, which also underwent iterative rounds of renaming to create more accurate descriptions. Through several meetings, all authors discussed the themes to ensure they both agreed upon the definition and boundaries of each theme. After several iterations, the authors arrived at the three crucial themes that will be discussed in the following sections.

Our final corpus considers 42 articles that addressed methods and systems employing emojis to communicate and share emotions among users. Figure 1 shows the filtering process of the papers in IEEE Xplore, ACM, and Web of Science databases. After screening more than 1,300 articles, the most frequent reason to exclude articles in the screening stage was that they did not focus on enabling users to communicate their emotions using emojis. Of the records assessed for eligibility, 20 passed through the initial screening but were not included in the final review. Many of the excluded articles did not present a system or design pertaining to communicating emotions using emojis.

PRISMA Flow Diagram showing the article selection process

4.1. Description of the Articles

Figure 2 shows the publication dates of the articles included in the review. Most of the articles were published in the ACM Digital Library (n=36), followed by four articles published in the Web of Science database, and two articles in the IEEE Digital Library. All of the reviewed papers were published from 2016 onwards. We hypothesize that there were no papers from before 2016 included in the final corpus due to the lack of extensive emoji research, which has been steadily increasing over the past 10 years. Out of the 42 articles reviewed, 30 were evaluated through user studies or deployment studies. Of those that did not include a user evaluation, six were short papers, three were posters, one contributed a theoretical framework, and one contributed a gesture set. Five articles included benchmark comparisons or some method of verifying model performance. Of the 30 papers that performed user studies, ten involved a deployment study (ranging from one to four weeks) while one article was deployed as a full application. Twenty-nine papers had users participate by themselves (13) or in pairs (16). Three of these articles were lenient on the group size, and allowed a group of three. Two articles studied groups of three or more. Thirteen articles encouraged participants to sign up as pairs or groups, and one study randomly assigned participants to pairs. These results are also shown in Table 1 .

4.2. Article Classification

The following article classifications emerged from the thematic analysis described above. We divided the papers into two main thematic categories: “Emotion Discovery with Emojis Made Simple” and “Augmenting emojis’ affordances and characteristics.” We also further identify sub-categories based on how these themes are implemented within the papers. Papers could belong to one or more categories as they addressed several of these themes. Details are listed in Table 2 .

4.2.1. Emotion Discovery with Emojis Made Simple

This theme describes how systems enable users to determine and chose emojis that best express their emotions. The articles include research on input methods to find emojis, implementing changes to existing emojis, and creating new emojis in real time. We found 17 articles on this theme, which encompasses the following sub-themes: connecting users to emojis, guiding emoji recommendation, and emoji expressiveness.

Connecting Users to Emojis ( n = 10 𝑛 10 n=10 italic_n = 10 ).

With the continuous addition of new emojis, the list has expanded significantly from its original size and makes it harder for users to find the appropiate emoji to communicate their emotions. To alleviate some of these design limitations, the articles of this sub-theme introduce alternative and improved methods to identify emojis based on the user’s desired emotion. For example, Pohl et al. (Pohl et al . , 2016 ) presented one of the first articles to research emoji entry: a novel keyboard built around zooming rather than scrolling. This design allows for exploratory interaction in two dimensions rather than one, utilizing spatial memory more effectively and allowing users to enter emojis up to 18% faster. In a subsequent article, Pohl et al. (Pohl et al . , 2017 ) presented two further emoji keyboard models based on semantic and annotated information, while simultaneously highlighting the need for further development in emoji entry. Rathod et al. presented a system for recommending emojis by predicting emoji usage based on the context of the conversation, including to whom the speaker is messaging (Rathod et al . , 2023 ) . This explores the findings from Wiseman et al. (Wiseman and Gould, 2018 ) of personalized communication through emojis. Research also shows more multimodal alternatives to keyboard-based emoji entry, such as video and voice. Zhang et al. (Zhang et al . , 2021 ) presented an application allowing visually impaired users to issue verbal commands that correspond to relevant emojis. Koh et al. (Koh et al . , 2019 ) created a hand gesture set and recognition system that maps to emojis. Other systems use facial recognition to map user’s displayed emotions to emojis (Liu et al . , 2018 ; Henriques et al . , 2018 ; Liu et al . , 2019b ) . These systems aim to reduce the mental effort required in the technical usage of emojis, which can help users select emojis that better represent their emotions (Pohl et al . , 2017 ; Henriques et al . , 2018 ) .

Enriching Expressions through Emojis ( n = 3 𝑛 3 n=3 italic_n = 3 ).

These articles presented research on new formats, images, or icons that expand the emotional scope of emoji. Originally, emojis were conceived as simple and pretty recognizable symbols that resemble objects or emotions from everyday life. However, emojis were not originally designed with the explicit intent to be as expressive as possible. Articles from this corpus reveal that the set of emojis has gradually become more nuanced with the addition of new emojis and allowing users to customize pre-existing emojis. More expressive emojis could benefit users with visual impairments or difficulty in recognizing emotions. Many articles of this theme focused on increasing the accessibility of emojis, either in recognizing the emotions they convey or for those users with disabilities. Cherbonnier et al. (Cherbonnier and Michinov, 2021 ) designed a new set of emojis that conveyed six basic emotions with greater intensity and recognition when compared to Facebook emojis. In one of the studies they performed, participants were able to recognize emotions conveyed by the ’new emoticons’ more accurately and successfully than other emojis or even images of faces. Further studies confirmed this advantage but emphasized it for certain emojis, primarily disgust and sadness. Choi et al. (Choi et al . , 2020 ) , introduced tactile emojis for visually impaired individuals that were recognized correctly 81% of the time and were able to improve the clarity of the sentence. This study shows that image-based tactile emojis could be beneficial for visually impaired individuals, and begin to minimize the gap in the communication environment between sighted and visually impaired individuals. Finally, Nishimori et al. presented a method to customize the expressiveness of an emoji on the fly (Nishimori and Mukai, 2023 ) . Users can swipe up or down on an emoji to change some accessory elements on the emoji (such as the size of a heart, or tears), which in turn alters the emotional expressiveness of the emoji.

Guiding Emoji Recommendation ( n = 4 𝑛 4 n=4 italic_n = 4 ).

One simple way to recommend emojis is by replacing words or emoticons in a text conversation with their corresponding emojis (e.g. the word fire becoming a fire emoji). While this is suitable for emoji and emoticon’s function as a literal pictorial representation of words, it falls flat when it comes to more complex interpretations of emojis. Articles in this category focused on contributions towards enhancing these recommendations to users. These articles used mostly natural language processing and machine learning techniques to improve recommendations’ accuracy and variety of emojis. Guibon et al. (Guibon et al . , 2018 ) introduced a model using sentiment-related features, allowing them to predict the emoji used with an 84% F1 score and 95% precision in a corpus of private instant messages. Kim and Gong et al. (Kim et al . , 2020b ) presented a model that uses similar features across a larger scope of the chat, allowing for more context across multiple sentences. This allowed users to choose emojis up to 38% faster than baseline and reportedly more suited to the conversation. In another study, Hong et al. (Hong et al . , 2024 ) presented a keyboard interface using query expansion and emoji prediction to better suggest emojis for users. Lastly, Gao et al. (Gao et al . , 2020 ) introduced a similar model for sticker recommendation, learning representations without labels and considering context across multiple turns of dialog. The model achieved state-of-the-art performance across all metrics and paved the way for a personalized sticker response selection system.

4.2.2. Augmenting emojis’ affordances and characteristics

Within the corpus, we see some papers whose contributions deviate greatly from the traditional definition of emoji. However, the authors of these papers use emojis as a reference or a comparison to their systems. We include these papers because they are important to understanding the current state of HCI literature in emotion communication. The second emerging theme was focused on studies and systems that combined emerging technologies with emojis in innovative ways, drastically diverging from their common usage. It included 20 articles that introduced new properties, affordances, and characteristics to emojis in order to enhance their ability to communicate emotions. The sub-categories are: (1) Connecting the Body and Emojis, (2) Employing Augmented Reality (AR) with Emoji, (3) Color and Shape, (4) Emoji in Context, and (5) Leveraging Physical Space. One article (Buschek et al . , 2018 ) was classified into three sub-categories, as the authors studied three different systems. Two other articles (Chen et al . , 2021 ; Semertzidis et al . , 2020 ) were classified into two sub-categories, as their implications are relevant for both.

Emojis as Artifacts of Users’ Data ( n = 3 𝑛 3 n=3 italic_n = 3 )

As some of the systems identified in this corpus have allowed users to customize the search and use of emojis more than before, emojis are inherently embedding more user information, such as their identity, relationships, and emotions. Many of the papers reviewed in our corpus utilize data from the user in order to facilitate their goals, be it text data from the conversation, audio/visual data, or raw biosignal data. This trend can potentially contain information about users’ appearance, location, and potentially even physical state. This information is not only transferred through the emoji but also recorded by the emoji within the context of the conversation and the user. Several articles in the review presented some form of automatic emotion detection, such as image capturing or facial recognition. Across these papers, many participants voiced some negative sentiment toward the automatic function. For example, Liu et al. (Liu et al . , 2018 ) presented a system that automatically attaches reactions to Slack messages by detecting facial expressions. While this enhanced the genuine expression of emotion, it also led some to worry about their portrayal of emotion. Participants showed concern about leaking emotion. Furthermore, contrary to their hypothesis, ReactionBot reduced the social presence felt by users. Lee et al. (Lee et al . , 2023 ) created ‘ARWand’, a system to enable users to create asynchronous augmented reality (AR) messages. ‘ARWand’ featured an automatic reaction-capturing video, which helped users be more authentic but also led to privacy concerns about their surroundings. Poguntke et al. (Poguntke et al . , 2019 ) created four visualizations, representing emotions captured automatically by a webcam. The automaticity of the reactions highlighted users’ desire to maintain control over their emotional reactions online. This connects with the hypothesis posed by Derks et al. (Derks et al . , 2008 ) , which states that the reduced spontaneity typically associated with CMC allows for more regulation of emotions.

Connecting the User’s Body with Emojis ( n = 11 ) 𝑛 11 (n=11) ( italic_n = 11 ) .

This theme focuses on furthering the connection between the users’ bodies and the emotions they want to portray through emojis and other visualizations. The articles within this category use a variety of means to achieve this effect. Some systems transform physical bio-information into emotions through the use of existing biosensory technologies (such as smartwatches) or by supplementing text messaging with data from extraneous sensors. Liu et al. (Liu et al . , 2017b ) and Buschek et al. (Buschek et al . , 2018 ) investigated the effects of sharing heart rate with a partner alongside text messages, finding that the information was expressive and promoted mindfulness of their own as well as their partners’ heart rate. Expanding on this work, Liu et al. (Liu et al . , 2019a ) created a smartwatch app with a shareable set of animations that played according to the emotional state of the wearer, serving as a lightweight social connection tool. These animations were highly abstract, consisting of a shape, color, and motion that changes according to the biosignal data. In another study, Liu et al. (Liu et al . , 2021 ) used an animated otter, and found that allowing the animations to ’sense’ biosignals enabled a more genuine emotional connection than just the animation without biosignal sensing. Other papers discussed how brain activity visualization can also affect interpersonal emotions. In Liu et al. (Liu et al . , 2017a ) , different visualizations of brain waves and the associated emotional state were compared. Participants were willing to use the expressive biosignals to form impressions about others. Semertzidis et al. (Semertzidis et al . , 2020 ) introduces a brain-computer interface that utilizes artificial intelligence to read users’ emotional state. Their system, Neo-Noumena, used emotional valence and arousal to create 3D fractals and dynamically display them to their partners in mixed reality. Using the system resulted in significant improvement in measures of emotion regulation and challenged users’ perceptions of emotion representation.

Other papers classified in this theme explored how vibrations, heat, and other haptic motions can communicate emotions like emojis do. Haptic feedback is widely used to draw users’ attention to incoming information or notifications. While vibrations are the most common, other modalities of haptic feedback are less commonplace. The articles within this theme expanded upon the different types of haptic feedback, and combined it with text-based messaging to replace emoji’s role in nonverbal emotion communication. Wilson et al. (Wilson and Brewster, 2017 ) systematically combined multiple different haptic modalities across three studies (vibrotactile, thermal, and vibrotactile + thermal) to show that the combination of vibrotactile, thermal, and visual increases the affective emotional range of messages the greatest. An et al. (An et al . , 2022 ) developed VibEmoji, offering users a system to create multi-modal emojis using animation and vibration. Participants reported that the animation and vibrotactile effects enhanced the meanings of emojis, and even helped them create new meanings associated with the emojis. Participants also found that the emojis were able to set and reset the atmosphere of the conversation. Araujo de Aguiar et al. (Araujo De Aguiar et al . , 2023 ) proposes a system that incorporates light, vibrations, tactile interactions, and a digital display with the aim of improving emotion communication. The system combines a wearable device to transfer multi-modal haptic feedback with an app that enables text messaging as well as some social media.

Finally, other papers have used audio to augment the expression of emotions through emojis. Voice messaging provides an alternative to text messaging, allowing users to utilize voice inflections and sound to convey their message with a greater affect. These articles explored different ways to add emotions to voice messages. Chen et al. (Chen et al . , 2021 ) studied how coloring the background of a voice message had an intensifying effect on emotional arousal. The coloring of the background was based on the emotion conveyed in the voice message, and was found to intensify emotions on average. However, users were less willing to use the different colors in the case of negative emotions. In another example, Haas et al. (Haas et al . , 2020 ) created a system that allows users to augment voice messages with other audio such as background sounds and voice changers. Users found the system more expressive and personal compared to traditional voice messages. Extending previous work (VibEmoji (An et al . , 2022 ) , discussed above), An et al. (An et al . , 2024 ) created a system that allows users to choose emotional teasers on voice messages, that significantly improved the communication experience for both senders and receivers.

Employing Augmented Reality (AR) with Emojis ( n = 7 ) 𝑛 7 (n=7) ( italic_n = 7 ) .

These articles explored ways in which AR technologies can employ emojis to overlay reality and communicate emotion more intimately. Namikawa et al. (Namikawa et al . , 2021 ) proposed the design of an AR system that covers users’ faces with emojis over video calls, either automatically or through manual input. This system allowed users to control emotional delivery more accurately. This design also allowed users to read the emotions of the speaker and audience members more easily. In another study, Zhang et al. (Zhang et al . , 2022 ) proposed a digital avatar that can be dressed, posed, animated, and sent as a personal handcrafted message. Their system, “Auggie,” highlights personal efforts while minimizing procedural efforts in order to allow users to create meaningful and personal messages in a lightweight format. Lee et al. (Lee et al . , 2023 ) explored this idea more, prototyping a messaging system centered around AR messages and capturing video reactions. Participants found that AR messages allowed for communication and connection in an immersive and “fun” way. Leong et al. (Leong et al . , 2022 ) presented an app for using AR effects in a shared physical space to add another dimension to the conversation. This process also enabled the creation of design reflections for future mixed reality emoji systems. Bhatia et al. borrowed visual language from comics to investigate the different effects on emotion and sensations (Bhatia et al . , 2024 ) . The comic annotations, which are visual effects derived from comics, were found to enhance the emotion associated with an action or object similar to how emojis influence the meanings of a sentence. Though this area is still in its infancy, the common theme reported is that AR may be a novel way to interact more immersively with each other. Semertzidis et al. (Semertzidis et al . , 2020 ) introduced their system, Neo-Noumena, which used VR as a way to actualize the emotions portrayed by users in space and time. Users found the system was able to augment their communication, not just facilitate it. Finally, Sun et al. (Sun et al . , 2019 ) presented a system utilizing optical see-through, overlaying participants’ facial expressions with emoji. The goal of the system was to help understand how children with autism spectrum disorders perceive emotions.

Color and Shape ( n = 5 ) 𝑛 5 (n=5) ( italic_n = 5 ) .

These five articles studied how modifying the colors and shapes of text bubbles has an effect on communicating emotions, in comparison and conjunction with emojis. In particular, alterations were made to the outlined form and background color of text bubbles as well as other objects within the interface in order to produce an emotional effect. For example, Buschek et al. (Buschek et al . , 2018 ) studied the effects of various changes to the style, size, and form of text, and concluded that this allowed for more recognizable and individual typing compared to standard. The system studied, TapScript, included a emoji drawing feature that participants used to emphasize the personal expression offered by the system. Aoki et al. (Aoki et al . , 2022 ) created a system that used speech input to create text with a corresponding speech balloon with different shape according to the intensity of emotion. This study suggested that using these emotional speech balloons results in a lesser misunderstanding between senders’ and receivers’ emotional arousal when compared to using emojis. An et al. (An et al . , 2023 ) presents “AniBalloons,” which imbues chat balloons with affective animations, designed with frequently used emojis in mind. Participants responses emphasized the ability of the “AniBalloons” to be an additional source of emotion expression alongside emojis, as well as being more abstract and less personalized when compared to emojis. As discussed above, Chen et al. (Chen et al . , 2021 ) studied how coloring the background of a voice message had an intensifying effect on emotional arousal. The background color intensified the emotional effect in the ‘excited,’ ‘sad,’ and ‘angry’ emotions. However, the ‘serene’ emotion was unaffected. Lastly, Bhatia et al. (Bhatia et al . , 2024 ) utilized elements from comics, such as speed or scent lines to emphasize different sensations. Users reported having more positive emotions associated with the elements presented in the article.

Emoji and Context ( n = 2 ) 𝑛 2 (n=2) ( italic_n = 2 ) .

In this sub-theme, two articles discussed and examined how leveraging additional users’ contextual information can enhance the use of emojis to share emotions. Their goal is to transmit more information about the senders’ context that is unknown to the recipients. Buschek et al. (Buschek et al . , 2018 ) studied the effects of contextual information such as weather, location, and activity being added to a text conversation, comparing this information to emoji’s role in providing additional context to a message. This direct contextual information reflects the authors’ design principle of minimizing ambiguity in interpretation. Through three systems and studies, the paper found that users were able to infer better the meaning of the messages by incorporating additional contextual information. These studies also emphasized the importance of facilitating shared understanding between users. In particular, Khandekar et al. (Khandekar et al . , 2019 ) created an emoji-only social media site that utilized location-based posts. Through a full deployment, surveys, and interviews, their system “Opico” showed that though messages comprised solely of emoji required more mental effort to interpret than straightforward text, there were benefits in the speed of sending messages, users’ enjoyment, and the greater amount of creativity expressed. Users strongly relied on the contextual data of location and time to decipher the meaning of the emoji messages.

5. Discussion

In this study, we explored how the HCI literature has addressed the evolution of emojis for communicating emotions. After identifying 42 papers that examined, employed, and extended the use of emojis, we identified relevant themes that described how emojis’ affordances, design, and capabilities have evolved over the past decade. These articles have introduced new ways for users to communicate their emotions through emojis, which have been constantly reshaped over the past years. In the following subsections, we elaborate on the key findings of this systematic literature review, focusing on how these articles delineate the use and design of emojis for sharing emotions, identifying gaps in this corpus, and outlining potential future research directions and opportunities.

5.1. RQ1: How have emojis evolved to better communicate users’ emotions?

The corpus shows that emojis have evolved from text-messaging icons to more complex multimodal forms. These modalities include AR/VR/MR, biosignals, haptics, and speech bubble form. These subthemes have little to do with emoji as Unicode-based icons, and more to do with the role in which emoji plays as a nonverbal communicative marker of emotion. Notably, the identified articles frequently made direct comparisons to or cited emojis as inspiration for the emotion communication systems presented. In this way, we posit that the articles discussed represent the next step in the evolution of emojis for emotion communication.

In contrast to previous emoji studies, the corpus suggests that the scope of emojis has broadened, and emphasizes the increased affordances of emoji research. In more established systems, we can see a divergence in vocabulary following the different affordances provided, from emoticons and emojis to stickers, avatars, and reactions. Within the selected articles, however, the vocabulary is not yet divergent. These articles still represent their contributions in comparison to emoji — systems such as Multi-Moji (Wilson and Brewster, 2017 ) and VibEmoji (An et al . , 2022 ) clearly reference them in their title. Participants even compare some of the presented systems to using emojis (Liu et al . , 2019a ) . Other articles directly present their systems as evolutions of emojis (Liu et al . , 2021 ) . As the modalities discussed within the corpus gain popularity, the vocabulary associated with each theme may vary as well. This divergence may be indicative of a larger trend within online communication platforms and technologies, as users employ different systems, devices, and technologies to satisfy their communication needs (Gonzales et al . , 2015 ) .

As Wiseman et al. (Wiseman and Gould, 2018 ) described, emojis can take on meaning outside of their direct visual interpretations. The selected articles shed light on further extensions of this meaning-making enabled by these different modalities. For example, ‘Auggie’ (Zhang et al . , 2022 ) , which focused on creating extremely personalized, effortful experiences. Other articles employed biosignals to interpret users’ emotions and generate specific emojis (Semertzidis et al . , 2020 ; Liu et al . , 2017b ; An et al . , 2022 ; Liu et al . , 2021 ) . The abstract visualizations did not deter participants and their communication partners from creating their own meaning (Zhang et al . , 2022 ; Liu et al . , 2021 , 2017b ) . In fact, Liu et al. identifies this as an area where future work might focus on deliberately designing systems around subjective interpretations (Liu et al . , 2017a ) .

Finally, the articles suggest that there is a relationship between the amount of customization and the complexity of an emoji. The relationship between these two is not necessarily directly correlated. As a natural development of the technology, there is a trend towards greater customization for emotion communication. Mainstream emoji is still relatively limited in function compared to the systems presented in this literature review. The customization options of Unicode emoji are limited to visual appearance, and cross-platform emojis are not one-to-one copies of each other. The innovations presented in the corpus represent a variety of different affordances for emojis, and the customization — complexity relationship reflects that as a trend broadly towards the upper right. While there is an associated amount of complexity as customization increases, the amount of effort put in does not necessarily need to increase linearly, as we depict in Figure 3 . Some articles deal with increasing or decreasing the effort required to use the system. A system that supports an easier way for users to send messages may reduce the mental burden associated with using it, and allow users to focus more on the content of the message being sent, such as by detecting cues from the body (Pohl et al . , 2017 ; Liu et al . , 2021 , 2017b ) . However, a system that intentionally requires more effort may be a benefit, especially by emphasizing personal efforts for greater communicative affect (Zhang et al . , 2022 ) .

In summary, emojis have evolved into a diverse set of modalities to enhance their ability to communicate emotion. Research in AR/VR/MR, biosignals, and haptics represent the potential future of emojis. Though current vocabulary delineates emoji from these modalities, the work represented in corpus of articles suggests that this distinction may not exist in the future. Furthermore, we see a push towards designing for greater personal meaning and customization within emojis.

Customization and Complexity are positively correlated, but not necessarily linear

5.2. RQ2: How do the changes in emojis affect their ability to communicate emotion?

The corpus shows that emoji research has supported users’ ability to create more personal, interpretable, and effective communications. The articles explore different levels of expressiveness over a broad range of emotions. The different modalities in which this research was conducted also had an effect on users’ emotion communication. Moreover, the corpus suggests a trend in emoji research towards emphasizing the ability for users to create custom emojis. These changes show how emojis have become more personal and open to interpretation, similar to F2F communication.

The corpus suggests that emoji research has made emojis more interpretable for users through customization and contextual information. Emojis are traditionally icons depicting faces with an expression, which is relatively straightforward. Even so, there is still some amount of ambiguity surrounding emoji meanings, stemming from popular culture usage, context, and personalized meanings. The articles increase the contextual information provided alongside communication channels. Articles utilized data such as heart rate or input from voice and video to increase the information with which receivers can interpret the emotion communicated. Another way that articles were used is through the personal nature of communications. Abstract visualizations resulted in more ways for users to interpret the emotions, as long as they were consistent and not random. Similarly, customization of emojis or other communication objects allowed users to reframe their conversations depending on who the conversation was with. This emphasizes the ability for users to assign personal meanings to emojis depending on the conversation partner (Wiseman and Gould, 2018 ) .

The multiple modalities provided by emoji research have afforded users a greater communicative affect. With the variety of modalities discussed in the corpus, emoji research has afforded users more ways to convey information. Across the different modalities, users were additionally able to accurately convey emotion despite their relative unfamiliarity with the systems. Some studies investigated the differences between positive, negative, calm, or angry emotions, with varying willingness to use. Participants were more willing to portray positive emotions in some studies (An et al . , 2023 ; Chen et al . , 2021 ) , and negative emotions in others (Semertzidis et al . , 2020 ) .

Lastly, the corpus also provided evidence of how emoji research has allowed users to align their emotion communication intention with the emoji selection. Firstly, users can select an emoji that is more accurate to their emotions, such as by using facial recognition to map user’s displayed emotions to emojis as seen in Liu et al. (Liu et al . , 2018 ) and Henriques et al. (Henriques et al . , 2018 ) . Secondly, users can select an appropriate emoji faster, such as through keyboard layouts such as EmojiZoom (Pohl et al . , 2016 ) . In fact, these ways are intertwined, and many articles discuss ways to improve both the accuracy of the emoji’s emotion as well as speed, such as in recommendation systems (Kim et al . , 2020b ; Gao et al . , 2020 ; Guibon et al . , 2018 ; Hong et al . , 2024 ) or reducing the mental effort required to select an emoji (Koh et al . , 2019 ) . Finally, the set of emojis can be modified to have a greater communicative effect. Articles such as (Nishimori and Mukai, 2023 ) allow users to directly control the expressiveness displayed by an emoji. Additionally, as new emojis are added by the Unicode Consortium over time, there are more options for users to select appropriate emojis.

In summary, we found that emoji research has led users towards emotion communication that is similar to F2F communication. The new channels of communication are marked by ambiguity, interpretability, and personalization, characteristics shared by F2F communication. This challenges some established theories of media richness and CMC (Ishii et al . , 2019 ) .

5.3. Implications

5.3.1. types of relational conversations.

Of the articles presented in the corpus, only some articles deal with users in groups rather than solo users. Due to emoji’s heavily personalizable meaning, emphasizing dyadic relationships when evaluating work may be important. The articles discussed in the theme “Augmenting emojis’ affordances and characteristics” are primarily concerned with more intimate conversations. This is likely due to the fact that they are more experimental and depart from established emojis and uses. The more bespoke systems are more difficult to deploy, and haptic / biosignal devices are not ubiquitous. The first theme, “Emotion Discovery with Emojis Made Simple” includes articles that deal with or could be extended to workplace environments, such as ReactionBot (Liu et al . , 2018 ) . This is likely due to the fact that it works around established norms of what emojis are, what they can do, and what they mean. However, this research is not specific to emotion communication or emoji in the context of the workplace or professional relationships. We identify that there is space for research to be done on the use of emojis for relationships between friends or colleagues.

5.3.2. Automaticity and Recommendations

The first category, ”Emotion Discovery with Emoji” is focused on aligning users’ emotions with the appropriate emoji. As discussed above, this alignment is a focus for emoji research. However, as emojis are regularly added by the Unicode Consortium, there becomes a greater diversity of expression available for users. The gradual increase in the set of emojis provides a sort of benchmark increase in the emotion alignment for users. Emoji research should aim to provide contributions other than adding new emojis to the set. It is also worth noting that a greater number of available emoji may come contrary to the goal of finding the appropriate emoji. This can be due to decision paralysis or the increasing time it takes to find the emoji on the keyboard. However, this again can be remedied through recommendation, which theoretically allows the system to align itself with the user’s intended emotion and predict which emoji should be used. While recommendation algorithms are not relevant for every type of research work, designers should be aware of the findings and capabilities presented by such papers.

Across several articles in the corpus, many participants voiced some negative sentiment toward the automatic function. For example, Liu et al. (Liu et al . , 2018 ) presented a system that automatically attaches reactions to Slack messages by detecting facial expressions. While this enhanced the genuine expression of emotion, it also led some to worry about their portrayal of emotion. Participants showed concern about leaking emotion. Similarly, Lee et al. (Lee et al . , 2023 ) featured an automatic reaction-capturing video that led to privacy concerns, as participants may not want to share information about their surroundings. Even just the webcam requiring permissions in Poguntke et al. (Poguntke et al . , 2019 ) was enough to give participants pause and an associated feeling of unease. This connects with the hypothesis posed by Derks et al. (Derks et al . , 2008 ) , which states that the reduced spontaneity typically associated with CMC allows for more regulation of emotions. Designers should be aware of the dangers of automaticity in CMC, and consider the benefits that CMC provides.

5.3.3. Accessibility

The affordances of emoji lend themselves more generally to sighted individuals, as they visually represent faces and icons. Within the corpus, we see articles that focus on making emojis more accessible to the visually impaired (Choi et al . , 2020 ) . Several articles (Zhang et al . , 2021 ) dealt with techniques that deliberately seek to enhance accessibility, either through voice entry (Zhang et al . , 2021 ) or for emotion recognition (Choi et al . , 2020 ) . However, this is a space that the corpus still is lacking in. Several articles introduce new modalities for emotion communication that are inaccessible to those outside of visual impairments. Effects that change colors in order to affect emotion may not work as well for those who are colorblind, and different entry techniques (i.e. swiping) or haptic feedback may be inaccessible for those with physical impairments. As more improvements are made towards the accessibility of emoji, designers should consider implications for those with different physical abilities outside of the visual.

While aimed at those with a disability, accessible research has the potential to benefit those without disability (Microsoft, 2003 ) In general, technologies designed for users have often proved to be beneficial for a wide range of users (Mott et al . , 2016 ; Wang et al . , 2019b ) . Conversely, articles that do not have a focus on accessibility may prove to be effective for some, such as non-verbal emoji entry systems (Koh et al . , 2019 ; El Ali et al . , 2017 ; Liu et al . , 2018 ) or more affective emojis (Kim et al . , 2020a ) . Biosignals have been adopted for use in assisting those with disabilities in communicating with caretakers (Schultz et al . , 2017 ; Pinheiro et al . , 2011 ) , but not for more casual use. We identify this as a space for future work, to investigate how multiple modalities may be incorporated in designs for those with disabilities to communicate their emotions better.

5.3.4. The Increasing Divergence Across CMC platforms

Lastly, the increasing number of platforms and services is another main explanation for emojis’ divergence. With the increasing number of social platforms competing for users’ attention and experiences, designers have created more tailored emojis that respond to the specific experiences that these platforms aim to provide. Allowing users to use emojis to quickly react to other users’ stories, as well as the choices of these emotions, proves that the future of emojis relies profoundly on how these platforms promote, curate, and trigger certain emotions. Future research should explore the consequences of this increasing partitioning and divergence of emojis across operative systems, platforms, and devices, as the choices will shape users’ communication in the long term.

Furthermore, HCI designers and practitioners should discuss new ways to standardize these multiple families of emojis, as well as when they became part of the Unicode system. With standardization and consensus among emojis’ purposes and characteristics, there will need to be a discussion about implementation, affordances, and visual congruity across platforms. Alternatively, if emojis do not converge into a few solutions, HCI designers may need to provide new ways to facilitate interoperability for these digital objects. For example, emojis created in one system such as “Opico” (Khandekar et al . , 2019 ) cannot be used on Facebook. In contrast, Snapchat’s ‘Bitmojis’ can currently be sent as images in emails and SMS messages, which allows them to essentially be used on any platform that allows images to be sent. While future efforts to unify emojis among online platforms and systems will remain uncertain, characterizing the evolution of emojis can provide guidance for future considerations.

The divergence in emoji research reflects a broader trend in the field of CMC technologies, where new tools and social platforms continually emerge and perish, each with unique features and designs. Understanding this divergence is essential for HCI researchers as they navigate the dynamic landscape of digital communication and strive to create more cohesive and user-friendly experiences. This is still a relatively unexplored space in HCI, and it is one that will continue to develop alongside the maturation of the technologies. While it is still too early to tell how emotion communication will be affected by the technologies of tomorrow, for now, we can be aware of some potential opportunities and challenges that will present themselves over time.

5.4. Limitations and Future Work

This study is not exempt from limitations. Firstly, excluding ‘stickers,’ ‘avatars’, or ‘emoticons’ in our search query limited the potential scope of these articles. We decided not to include these terms as our research goal was to examine the evolution of emojis. Studies with stickers and emoticons that acknowledged emojis were still captured by our search query. Therefore, the terms used by our search query were sufficient for our research goal. Future systematic reviews should consider including these terms and also exploring how those pictograms have been employed to communicate emotions online. Secondly, one coder primarily conducted the eligibility and inclusion stages, which could have affected the reliability of the coding process. To mitigate the bias in this coding process, the authors met frequently to discuss the findings and selected papers. New themes and structures emerged from their conversations. We provide the spreadsheet with its notes and decisions as Supplemental Materials to promote the reproducibility and analysis of our study. Lastly, systematic literature reviews are limited by the data sources and queries established by the research team. We did not include articles indexed in other research platforms (e.g., Google Scholar, Scopus), pre-printed versions (e.g., arXiv, SSRN), or non-academic outlets (e.g., blogs, industry reports). Including these datasets could provide more insights into the role of emojis in communicating emotions.

6. Conclusion

We conducted a systematic literature review to study how HCI researchers have examined emojis to communicate emotions in the past 10 years. After identifying 42 articles, we found several themes motivated by both purpose and method, with the purpose of improving the search and selection of emojis, as well as enhancing emojis to provide new communicative affordances in a distinct manner. The literature provides different methods and systems that include improving recommendations, input selection, and emojis themselves. The review also shows how systems employ different signals and environments to enhance emojis’ purposes and emotion communication. In conclusion, we hope this systematic review sheds light on the role of emojis for communicating emotions, providing HCI researchers and practitioners with discussions and insights into emojis’ future affordances and components.

• noa ([n. d.]) [n. d.]. FAQ. https://emojipedia.org/faq
• noa (2007) 2007. Digital ‘smiley face’ turns 25. https://www.nbcnews.com/id/wbna20829611
• noa (2016) 2016. UTR #51: Unicode Emoji. http://unicode.org/reports/tr51/tr51-7.html
• noa (2023) 2023. FAQ. https://emojipedia.org/faq
• noa (2024) 2024. Unicode Consortium. https://unicode.org/consortium/consort.html
• Abosag et al . (2020) Ibrahim Abosag, Zahy B. Ramadan, Tom Baker, and Zhongqi Jin. 2020. Customers’ need for uniqueness theory versus brand congruence theory: The impact on satisfaction with social network sites. Journal of Business Research 117 (Sept. 2020), 862–872. https://doi.org/10.1016/j.jbusres.2019.03.016
• Ahn et al . (2011) Wonmi Ahn, Jeeyea Park, and Kwang-hee Han. 2011. Emoticons convey emotion in CMC. https://doi.org/10.14236/ewic/HCI2011.11
• Alonso (2017) Daniel Hånberg Alonso. 2017. Emoji Timeline - An overview of the history of emojis. https://emojitimeline.com/
• Alshenqeeti (2016) Hamza Alshenqeeti. 2016. Are Emojis Creating a New or Old Visual Language for New Generations? A Socio-semiotic Study. Advances in Language and Literary Studies 7, 6 (Dec. 2016), 56–69. http://www.journals.aiac.org.au/index.php/alls/article/view/2823 Number: 6.
• Alvina et al . (2019) Jessalyn Alvina, Chengcheng Qu, Joanna McGrenere, and Wendy E. Mackay. 2019. MojiBoard: Generating Parametric Emojis with Gesture Keyboards. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems . ACM, Glasgow Scotland Uk, 1–6. https://doi.org/10.1145/3290607.3312771
• An et al . (2023) Pengcheng An, Chaoyu Zhang, Haichen Gao, Ziqi Zhou, Linghao Du, Che Yan, Yage Xiao, and Jian Zhao. 2023. Affective Affordance of Message Balloon Animations: An Early Exploration of AniBalloons. In Computer Supported Cooperative Work and Social Computing . ACM, Minneapolis MN USA, 138–143. https://doi.org/10.1145/3584931.3607017
• An et al . (2022) Pengcheng An, Ziqi Zhou, Qing Liu, Yifei Yin, Linghao Du, Da-Yuan Huang, and Jian Zhao. 2022. VibEmoji: Exploring User-authoring Multi-modal Emoticons in Social Communication. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22) . Association for Computing Machinery, New York, NY, USA, 1–17. https://doi.org/10.1145/3491102.3501940
• An et al . (2024) Pengcheng An, Jiawen Stefanie Zhu, Zibo Zhang, Yifei Yin, Qingyuan Ma, Che Yan, Linghao Du, and Jian Zhao. 2024. EmoWear: Exploring Emotional Teasers for Voice Message Interaction on Smartwatches. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24) . Association for Computing Machinery, New York, NY, USA, 1–16. https://doi.org/10.1145/3613904.3642101
• Aoki et al . (2022) Toshiki Aoki, Rintaro Chujo, Katsufumi Matsui, Saemi Choi, and Ari Hautasaari. 2022. EmoBalloon - Conveying Emotional Arousal in Text Chats with Speech Balloons. In CHI Conference on Human Factors in Computing Systems . ACM, New Orleans LA USA, 1–16. https://doi.org/10.1145/3491102.3501920
• Araujo De Aguiar et al . (2023) Carlos Henrique Araujo De Aguiar, Zezhi Guo, and Yuhe Cui. 2023. TOUCH: A Multi-sensory Communication System that Communicates Emotions. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments . ACM, Corfu Greece, 347–356. https://doi.org/10.1145/3594806.3594860
• Archer and Akert (1977) Dane Archer and Robin M. Akert. 1977. Words and everything else: Verbal and nonverbal cues in social interpretation. Journal of Personality and Social Psychology 35, 6 (June 1977), 443–449. https://doi.org/10.1037/0022-3514.35.6.443 Publisher: American Psychological Association.
• Arksey and O’Malley (2005) Hilary Arksey and Lisa O’Malley. 2005. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology 8, 1 (Feb. 2005), 19–32. https://doi.org/10.1080/1364557032000119616 Publisher: Routledge _eprint: https://doi.org/10.1080/1364557032000119616.
• Bai et al . (2019) Qiyu Bai, Qi Dan, Zhe Mu, and Maokun Yang. 2019. A Systematic Review of Emoji: Current Research and Future Perspectives. Frontiers in Psychology 10 (2019). https://www.frontiersin.org/articles/10.3389/fpsyg.2019.02221
• Bhatia et al . (2024) Arpit Bhatia, Henning Pohl, Teresa Hirzle, Hasti Seifi, and Kasper Hornbæk. 2024. Using the Visual Language of Comics to Alter Sensations in Augmented Reality. In Proceedings of the CHI Conference on Human Factors in Computing Systems . ACM, Honolulu HI USA, 1–17. https://doi.org/10.1145/3613904.3642351
• Boutet et al . (2021) Isabelle Boutet, Megan LeBlanc, Justin A. Chamberland, and Charles A. Collin. 2021. Emojis influence emotional communication, social attributions, and information processing. Computers in Human Behavior 119 (June 2021), 106722. https://doi.org/10.1016/j.chb.2021.106722
• Braun and Clarke (2006) Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (Jan. 2006), 77–101. https://doi.org/10.1191/1478088706qp063oa Publisher: Routledge _eprint: https://www.tandfonline.com/doi/pdf/10.1191/1478088706qp063oa.
• Burge (2019) Jeremy Burge. 2019. Correcting the Record on the First Emoji Set. https://blog.emojipedia.org/correcting-the-record-on-the-first-emoji-set/
• Buschek et al . (2018) Daniel Buschek, Mariam Hassib, and Florian Alt. 2018. Personal Mobile Messaging in Context: Chat Augmentations for Expressiveness and Awareness. ACM Transactions on Computer-Human Interaction 25, 4 (Aug. 2018), 23:1–23:33. https://doi.org/10.1145/3201404
• Cha et al . (2018) Yoonjeong Cha, Jongwon Kim, Sangkeun Park, Mun Yong Yi, and Uichin Lee. 2018. Complex and Ambiguous: Understanding Sticker Misinterpretations in Instant Messaging. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (Nov. 2018), 1–22. https://doi.org/10.1145/3274299
• Chairunnisa and A.S. (2017) Sabrina Chairunnisa and Benedictus A.S. 2017. Analysis of Emoji and Emoticon Usage in Interpersonal Communication of Blackberry Messenger and WhatsApp Application User. International Journal of Social Sciences and Management 4, 2 (April 2017), 120–126. https://doi.org/10.3126/ijssm.v4i2.17173
• Chen et al . (2021) Qinyue Chen, Yuchun Yan, and Hyeon-Jeong Suk. 2021. Bubble Coloring to Visualize the Speech Emotion. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems . ACM, Yokohama Japan, 1–6. https://doi.org/10.1145/3411763.3451698
• Chen and Siu (2017) Xin Chen and Kin Wai Michael Siu. 2017. Exploring user behaviour of emoticon use among Chinese youth. Behaviour & Information Technology 36, 6 (June 2017), 637–649. https://doi.org/10.1080/0144929X.2016.1269199 Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/0144929X.2016.1269199.
• Cherbonnier and Michinov (2021) Anthony Cherbonnier and Nicolas Michinov. 2021. The recognition of emotions beyond facial expressions: Comparing emoticons specifically designed to convey basic emotions with other modes of expression. Computers in Human Behavior 118 (May 2021), 106689. https://doi.org/10.1016/j.chb.2021.106689
• Choi et al . (2020) Yuri Choi, Kyung Hoon Hyun, and Ji-Hyun Lee. 2020. Image-Based Tactile Emojis: Improved Interpretation of Message Intention and Subtle Nuance for Visually Impaired Individuals. Human–Computer Interaction 35, 1 (Jan. 2020), 40–69. https://doi.org/10.1080/07370024.2017.1324305 Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/07370024.2017.1324305.
• Cosio et al . (2023) Laura D Cosio, Oğuz ’Oz’ Buruk, Daniel Fernández Galeote, Isak De Villiers Bosman, and Juho Hamari. 2023. Virtual and Augmented Reality for Environmental Sustainability: A Systematic Review. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23) . Association for Computing Machinery, New York, NY, USA, 1–23. https://doi.org/10.1145/3544548.3581147
• Derks et al . (2008) Daantje Derks, Agneta H. Fischer, and Arjan E. R. Bos. 2008. The role of emotion in computer-mediated communication: A review. Computers in Human Behavior 24, 3 (May 2008), 766–785. https://doi.org/10.1016/j.chb.2007.04.004
• Dresner and Herring (2010) Eli Dresner and Susan C. Herring. 2010. Functions of the Nonverbal in CMC: Emoticons and Illocutionary Force. Communication Theory 20, 3 (Aug. 2010), 249–268. https://doi.org/10.1111/j.1468-2885.2010.01362.x
• El Ali et al . (2017) Abdallah El Ali, Torben Wallbaum, Merlin Wasmann, Wilko Heuten, and Susanne Cj Boll. 2017. Face2Emoji: Using Facial Emotional Expressions to Filter Emojis. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems . ACM, Denver Colorado USA, 1577–1584. https://doi.org/10.1145/3027063.3053086
• Evers (2003) Joris Evers. 2003. MSN Messenger 6 looks for emotion. https://www.infoworld.com/article/2234898/msn-messenger-6-looks-for-emotion.html
• Fonts team (2022) Adobe Fonts team. 2022. The Future of Creativity: 2022 U.S. Emoji Trend Report: How Americans are Using Emoji | Adobe Blog. https://blog.adobe.com/en/publish/2022/09/13/emoji-trend-report-2022
• Gao et al . (2020) Shen Gao, Xiuying Chen, Chang Liu, Li Liu, Dongyan Zhao, and Rui Yan. 2020. Learning to Respond with Stickers: A Framework of Unifying Multi-Modality in Multi-Turn Dialog. In Proceedings of The Web Conference 2020 (WWW ’20) . Association for Computing Machinery, New York, NY, USA, 1138–1148. https://doi.org/10.1145/3366423.3380191
• Gawne and McCulloch (2019) Lauren Gawne and Gretchen McCulloch. 2019. Emoji as digital gestures. Language@Internet 17, 2 (June 2019). https://www.languageatinternet.org/articles/2019/gawne
• Gonzales et al . (2015) Joseph A. Gonzales, Casey Fiesler, and Amy Bruckman. 2015. Towards an Appropriable CSCW Tool Ecology: Lessons from the Greatest International Scavenger Hunt the World Has Ever Seen. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW ’15) . Association for Computing Machinery, New York, NY, USA, 946–957. https://doi.org/10.1145/2675133.2675240
• Greg ([n. d.]) Greg. [n. d.]. Emoji Frequency. https://home.unicode.org/emoji/emoji-frequency/
• Grosz et al . (2023) Patrick Georg Grosz, Gabriel Greenberg, Christian De Leon, and Elsi Kaiser. 2023. A semantics of face emoji in discourse. Linguistics and Philosophy 46, 4 (Aug. 2023), 905–957. https://doi.org/10.1007/s10988-022-09369-8
• Guibon et al . (2018) Gaël Guibon, Magalie Ochs, and Patrice Bellot. 2018. Emoji recommendation in private instant messages. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing (SAC ’18) . Association for Computing Machinery, New York, NY, USA, 1821–1823. https://doi.org/10.1145/3167132.3167430
• Haas et al . (2020) Gabriel Haas, Jan Gugenheimer, and Enrico Rukzio. 2020. VoiceMessage++: Augmented Voice Recordings for Mobile Instant Messaging. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’20) . Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3379503.3403560
• Harris et al . (2019) Alexa M. Harris, Diego Gómez-Zará, Leslie A. DeChurch, and Noshir S. Contractor. 2019. Joining Together Online: The Trajectory of CSCW Scholarship on Group Formation. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (Nov. 2019), 1–27. https://doi.org/10.1145/3359250
• Henriques et al . (2018) Tiago Henriques, Samuel Silva, Susana Brás, Sandra C. Soares, Nuno Almeida, and António Teixeira. 2018. Emotionally-Aware Multimodal Interfaces: Preliminary Work on a Generic Affective Modality. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion (DSAI ’18) . Association for Computing Machinery, New York, NY, USA, 80–87. https://doi.org/10.1145/3218585.3218589
• Highfield and Leaver (2016) Tim Highfield and Tama Leaver. 2016. Instagrammatics and digital methods: studying visual social media, from selfies and GIFs to memes and emoji. Communication Research and Practice 2, 1 (Jan. 2016), 47–62. https://doi.org/10.1080/22041451.2016.1155332 Publisher: Routledge _eprint: https://doi.org/10.1080/22041451.2016.1155332.
• Hirzle et al . (2023) Teresa Hirzle, Florian Müller, Fiona Draxler, Martin Schmitz, Pascal Knierim, and Kasper Hornbæk. 2023. When XR and AI Meet - A Scoping Review on Extended Reality and Artificial Intelligence. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23) . Association for Computing Machinery, New York, NY, USA, 1–45. https://doi.org/10.1145/3544548.3581072
• Hong et al . (2024) Yoo Jin Hong, Hye Soo Park, Eunki Joung, and Jihyeong Hong. 2024. MOJI: Enhancing Emoji Search System with Query Expansions and Emoji Recommendations. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems . ACM, Honolulu HI USA, 1–8. https://doi.org/10.1145/3613905.3650838
• Ishii et al . (2019) Kumi Ishii, Mary Madison Lyons, and Sabrina A. Carr. 2019. Revisiting media richness theory for today and future. Human Behavior and Emerging Technologies 1, 2 (2019), 124–131. https://doi.org/10.1002/hbe2.138 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/hbe2.138.
• Jiang et al . (2023) Yanqi Jiang, Xianghua(Sharon) Ding, Xiaojuan Ma, Zhida Sun, and Ning Gu. 2023. IntimaSea: Exploring Shared Stress Display in Close Relationships. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23) . Association for Computing Machinery, New York, NY, USA, 1–19. https://doi.org/10.1145/3544548.3581000
• Kelly and Watts (2015) Ryan Kelly and Leon Watts. 2015. Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships: Experiences of Technology Appropriation: Unanticipated Users, Usage, Circumstances, and Design.
• Khandekar et al . (2019) Sujay Khandekar, Joseph Higg, Yuanzhe Bian, Chae Won Ryu, Jerry O. Talton Iii, and Ranjitha Kumar. 2019. Opico: A Study of Emoji-first Communication in a Mobile Social App. In Companion Proceedings of The 2019 World Wide Web Conference (WWW ’19) . Association for Computing Machinery, New York, NY, USA, 450–458. https://doi.org/10.1145/3308560.3316547
• Kim et al . (2020a) Joongyum Kim, Taesik Gong, Kyungsik Han, Juho Kim, JeongGil Ko, and Sung-Ju Lee. 2020a. Messaging Beyond Texts with Real-time Image Suggestions. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’20) . Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3379503.3403553
• Kim et al . (2020b) Joongyum Kim, Taesik Gong, Bogoan Kim, Jaeyeon Park, Woojeong Kim, Evey Huang, Kyungsik Han, Juho Kim, Jeonggil Ko, and Sung-Ju Lee. 2020b. No More One Liners: Bringing Context into Emoji Recommendations. ACM Transactions on Social Computing 3, 2 (April 2020), 9:1–9:25. https://doi.org/10.1145/3373146
• Koh et al . (2019) Jung In Koh, Josh Cherian, Paul Taele, and Tracy Hammond. 2019. Developing a Hand Gesture Recognition System for Mapping Symbolic Hand Gestures to Analogous Emojis in Computer-Mediated Communication. ACM Transactions on Interactive Intelligent Systems 9, 1 (March 2019), 1–35. https://doi.org/10.1145/3297277
• Konrad et al . (2020) Artie Konrad, Susan C Herring, and David Choi. 2020. Sticker and Emoji Use in Facebook Messenger: Implications for Graphicon Change. Journal of Computer-Mediated Communication 25, 3 (May 2020), 217–235. https://doi.org/10.1093/jcmc/zmaa003
• Kwak (2012) Hyokjin Kwak. 2012. Self-disclosure in online media: An active audience perspective. International Journal of Advertising 31, 3 (Jan. 2012), 485–510. https://doi.org/10.2501/IJA-31-3-485-510 Publisher: Routledge _eprint: https://doi.org/10.2501/IJA-31-3-485-510.
• Lee et al . (2016) Joon Young Lee, Nahi Hong, Soomin Kim, Jonghwan Oh, and Joonhwan Lee. 2016. Smiley face: why we use emoticon stickers in mobile messaging. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct . ACM, Florence Italy, 760–766. https://doi.org/10.1145/2957265.2961858
• Lee et al . (2023) Kyungjun Lee, Hong Li, Muhammad Rizky Wellyanto, Yu Jiang Tham, Andrés Monroy-Hernández, Fannie Liu, Brian A. Smith, and Rajan Vaish. 2023. Exploring Immersive Interpersonal Communication via AR. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (April 2023), 50:1–50:25. https://doi.org/10.1145/3579483
• Lee and Wagner (2002) Victoria Lee and Hugh Wagner. 2002. The Effect of Social Presence on the Facial and Verbal Expression of Emotion and the Interrelationships Among Emotion Components. Journal of Nonverbal Behavior 26, 1 (March 2002), 3–25. https://doi.org/10.1023/A:1014479919684
• Leong et al . (2022) Joanne Leong, Olivia Seow, Cathy Mengying Fang, Benny J. Tang, Rajan Vaish, and Pattie Maes. 2022. Wemoji: Towards Designing Complementary Communication Systems in Augmented Reality. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST ’22 Adjunct) . Association for Computing Machinery, New York, NY, USA, 1–3. https://doi.org/10.1145/3526114.3558699
• Levac et al . (2010) Danielle Levac, Heather Colquhoun, and Kelly K. O’Brien. 2010. Scoping studies: advancing the methodology. Implementation Science 5, 1 (Sept. 2010), 69. https://doi.org/10.1186/1748-5908-5-69
• List (2017) Sticker List. 2017. StickerStats — State of the Stickers. https://medium.com/@stickerlist/state-of-the-stickers-2017-46c9dc744e6
• Liu et al . (2017a) Fannie Liu, Laura Dabbish, and Geoff Kaufman. 2017a. Can Biosignals be Expressive? How Visualizations Affect Impression Formation from Shared Brain Activity. Proceedings of the ACM on Human-Computer Interaction 1, CSCW (Dec. 2017), 71:1–71:21. https://doi.org/10.1145/3134706
• Liu et al . (2017b) Fannie Liu, Laura Dabbish, and Geoff Kaufman. 2017b. Supporting Social Interactions with an Expressive Heart Rate Sharing Application. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (Sept. 2017), 77:1–77:26. https://doi.org/10.1145/3130943
• Liu et al . (2019a) Fannie Liu, Mario Esparza, Maria Pavlovskaia, Geoff Kaufman, Laura Dabbish, and Andrés Monroy-Hernández. 2019a. Animo: Sharing Biosignals on a Smartwatch for Lightweight Social Connection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 1 (March 2019), 18:1–18:19. https://doi.org/10.1145/3314405
• Liu et al . (2021) Fannie Liu, Chunjong Park, Yu Jiang Tham, Tsung-Yu Tsai, Laura Dabbish, Geoff Kaufman, and Andrés Monroy-Hernández. 2021. Significant Otter: Understanding the Role of Biosignals in Communication. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21) . Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3411764.3445200
• Liu et al . (2019b) Li Liu, Dragos Guta, and Shuo Niu. 2019b. Integration of User Emotion and Self-Awareness in Text Messaging. In 2019 IEEE 5th International Conference on Collaboration and Internet Computing (CIC) . 178–183. https://doi.org/10.1109/CIC48465.2019.00030
• Liu et al . (2018) Miki Liu, Austin Wong, Ruhi Pudipeddi, Betty Hou, David Wang, and Gary Hsieh. 2018. ReactionBot: Exploring the Effects of Expression-Triggered Emoji in Text Messages. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (Nov. 2018), 110:1–110:16. https://doi.org/10.1145/3274379
• Maier (2023) Emar Maier. 2023. Emojis as Pictures. https://ling.auf.net/lingbuzz/006025 LingBuzz Published In: To appear in Ergo.
• McCulloch ([n. d.]) Gretchen McCulloch. [n. d.]. The Linguistic Secrets Found in Billions of Emoji - SXSW 2016 presentation. https://www.slideshare.net/SwiftKey/the-linguistic-secrets-found-in-billions-of-emoji-sxsw-2016-presentation-59956212
• Microsoft (2003) Microsoft. 2003. The Wide Range of Abilities and Its Impact on Computer Technology . Technical Report. Microsoft. https://www.microsoft.com/en-us/download/details.aspx?id=18446
• Miller et al . (2016) Hannah Miller, Jacob Thebault-Spieker, Shuo Chang, Isaac Johnson, Loren Terveen, and Brent Hecht. 2016. “Blissfully Happy” or “Ready toFight”: Varying Interpretations of Emoji. Proceedings of the International AAAI Conference on Web and Social Media 10, 1 (2016), 259–268. https://doi.org/10.1609/icwsm.v10i1.14757 Number: 1.
• Mott et al . (2016) Martez E. Mott, Radu-Daniel Vatavu, Shaun K. Kane, and Jacob O. Wobbrock. 2016. Smart Touch: Improving Touch Accuracy for People with Motor Impairments with Template Matching. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16) . Association for Computing Machinery, New York, NY, USA, 1934–1946. https://doi.org/10.1145/2858036.2858390
• Namikawa et al . (2021) Kosaku Namikawa, Ippei Suzuki, Ryo Iijima, Sayan Sarcar, and Yoichi Ochiai. 2021. EmojiCam: Emoji-Assisted Video Communication System Leveraging Facial Expressions. In Human-Computer Interaction. Design and User Experience Case Studies (Lecture Notes in Computer Science) , Masaaki Kurosu (Ed.). Springer International Publishing, Cham, 611–625. https://doi.org/10.1007/978-3-030-78468-3_42
• Nishimori and Mukai (2023) Chizu Nishimori and Tomohiko Mukai. 2023. On-the-fly Editing of Emoji Elements for Mobile Messaging. In Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology . ACM, San Francisco CA USA, 1–2. https://doi.org/10.1145/3586182.3616630
• Novak et al . (2015) Petra Kralj Novak, Jasmina Smailović, Borut Sluban, and Igor Mozetič. 2015. Sentiment of Emojis. PLOS ONE 10, 12 (Dec. 2015), e0144296. https://doi.org/10.1371/journal.pone.0144296 Publisher: Public Library of Science.
• Nowell et al . (2017) Lorelli S. Nowell, Jill M. Norris, Deborah E. White, and Nancy J. Moules. 2017. Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods 16, 1 (Dec. 2017), 1609406917733847. https://doi.org/10.1177/1609406917733847 Publisher: SAGE Publications Inc.
• Olshevsky et al . (2021) Vyacheslav Olshevsky, Ivan Bondarets, Andrii Kozyr, Oleksandr Trunov, Artem Shcherbina, Igor Tolmachov, and Svitlana Alkhimova. 2021. Touchless Gestures for Interactive Messaging: Gesture Interface for Sending Emoji. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’20) . Association for Computing Machinery, New York, NY, USA, 1–4. https://doi.org/10.1145/3406324.3410535
• Page et al . (2021) Matthew J Page, David Moher, Patrick M Bossuyt, Isabelle Boutron, Tammy C Hoffmann, Cynthia D Mulrow, Larissa Shamseer, Jennifer M Tetzlaff, Elie A Akl, Sue E Brennan, Roger Chou, Julie Glanville, Jeremy M Grimshaw, Asbjørn Hróbjartsson, Manoj M Lalu, Tianjing Li, Elizabeth W Loder, Evan Mayo-Wilson, Steve McDonald, Luke A McGuinness, Lesley A Stewart, James Thomas, Andrea C Tricco, Vivian A Welch, Penny Whiting, and Joanne E McKenzie. 2021. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ (March 2021), n160. https://doi.org/10.1136/bmj.n160
• Pinheiro et al . (2011) Carlos G. Pinheiro, Eduardo LM Naves, Pierre Pino, Etienne Losson, Adriano O. Andrade, and Guy Bourhis. 2011. Alternative communication systems for people with severe motor disabilities: a survey. BioMedical Engineering OnLine 10, 1 (April 2011), 31. https://doi.org/10.1186/1475-925X-10-31
• Poguntke et al . (2019) Romina Poguntke, Tamara Mantz, Mariam Hassib, Albrecht Schmidt, and Stefan Schneegass. 2019. Smile to Me: Investigating Emotions and their Representation in Text-based Messaging in the Wild. In Proceedings of Mensch und Computer 2019 (MuC ’19) . Association for Computing Machinery, New York, NY, USA, 373–385. https://doi.org/10.1145/3340764.3340795
• Pohl et al . (2017) Henning Pohl, Christian Domin, and Michael Rohs. 2017. Beyond Just Text: Semantic Emoji Similarity Modeling to Support Expressive Communication. ACM Transactions on Computer-Human Interaction 24, 1 (Feb. 2017), 1–42. https://doi.org/10.1145/3039685
• Pohl et al . (2016) Henning Pohl, Dennis Stanke, and Michael Rohs. 2016. EmojiZoom: emoji entry via large overview maps. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services . ACM, Florence Italy, 510–517. https://doi.org/10.1145/2935334.2935382
• Rathod et al . (2023) Jinendra Rathod, K Neha, Harshvardhan S Purohit, Joya Verma, and Savitha Hiremath. 2023. Emoji Recommendation System Using Deep Learning Algorithms. In 2023 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) . 1–5. https://doi.org/10.1109/CONECCT57959.2023.10234737 ISSN: 2766-2101.
• Riordan (2017) Monica A. Riordan. 2017. Emojis as Tools for Emotion Work: Communicating Affect in Text Messages. Journal of Language and Social Psychology 36, 5 (Oct. 2017), 549–567. https://doi.org/10.1177/0261927X17704238 Publisher: SAGE Publications Inc.
• Riordan and Kreuz (2010) Monica A. Riordan and Roger J. Kreuz. 2010. Emotion encoding and interpretation in computer-mediated communication: Reasons for use. Computers in Human Behavior 26, 6 (Nov. 2010), 1667–1673. https://doi.org/10.1016/j.chb.2010.06.015
• Robertson et al . (2018) Alexander Robertson, Walid Magdy, and Sharon Goldwater. 2018. Self-Representation on Twitter Using Emoji Skin Color Modifiers. Proceedings of the International AAAI Conference on Web and Social Media 12, 1 (June 2018). https://doi.org/10.1609/icwsm.v12i1.15055 Number: 1.
• Rogers et al . (2022) Katja Rogers, Sukran Karaosmanoglu, Maximilian Altmeyer, Ally Suarez, and Lennart E. Nacke. 2022. Much Realistic, Such Wow! A Systematic Literature Review of Realism in Digital Games. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22) . Association for Computing Machinery, New York, NY, USA, 1–21. https://doi.org/10.1145/3491102.3501875
• Schultz et al . (2017) Tanja Schultz, Michael Wand, Thomas Hueber, Dean J. Krusienski, Christian Herff, and Jonathan S. Brumberg. 2017. Biosignal-Based Spoken Communication: A Survey. IEEE/ACM Transactions on Audio, Speech, and Language Processing 25, 12 (Dec. 2017), 2257–2271. https://doi.org/10.1109/TASLP.2017.2752365
• Semertzidis et al . (2020) Nathan Semertzidis, Michaela Scary, Josh Andres, Brahmi Dwivedi, Yutika Chandrashekhar Kulwe, Fabio Zambetta, and Florian Floyd Mueller. 2020. Neo-Noumena: Augmenting Emotion Communication. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20) . Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376599
• Seta (2018) Gabriele de Seta. 2018. Biaoqing: The circulation of emoticons, emoji, stickers, and custom images on Chinese digital media platforms. First Monday (Sept. 2018). https://doi.org/10.5210/fm.v23i9.9391
• Shibuya et al . (2022) Yuya Shibuya, Andrea Hamm, and Teresa Cerratto Pargman. 2022. Mapping HCI research methods for studying social media interaction: A systematic literature review. Computers in Human Behavior 129 (April 2022), 107131. https://doi.org/10.1016/j.chb.2021.107131
• Sun et al . (2019) Ran Sun, Harald Haraldsson, Yuhang Zhao, and Serge Belongie. 2019. Anon-Emoji: An Optical See-Through Augmented Reality System for Children with Autism Spectrum Disorders to promote Understanding of Facial Expressions and Emotions. In 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) . 448–450. https://doi.org/10.1109/ISMAR-Adjunct.2019.00052
• Tang and Hew (2019) Ying Tang and Khe Foon Hew. 2019. Emoticon, Emoji, and Sticker Use in Computer-Mediated Communication: A Review of Theories and Research Findings. International Journal of Communication 13, 0 (May 2019), 27. https://ijoc.org/index.php/ijoc/article/view/10966 Number: 0.
• Vandergriff (2013) Ilona Vandergriff. 2013. Emotive communication online: A contextual analysis of computer-mediated communication (CMC) cues. Journal of Pragmatics 51 (May 2013), 1–12. https://doi.org/10.1016/j.pragma.2013.02.008
• Walther et al . (2005) Joseph B. Walther, Tracy Loh, and Laura Granka. 2005. Let Me Count the Ways: The Interchange of Verbal and Nonverbal Cues in Computer-Mediated and Face-to-Face Affinity. Journal of Language and Social Psychology 24, 1 (2005), 36–65. https://doi.org/10.1177/0261927X04273036 Place: US Publisher: Sage Publications.
• Wang et al . (2019b) Ruolin Wang, Chun Yu, Xing-Dong Yang, Weijie He, and Yuanchun Shi. 2019b. EarTouch: Facilitating Smartphone Use for Visually Impaired People in Mobile and Public Scenarios. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19) . Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300254
• Wang et al . (2019a) Yuan Wang, Yukun Li, Xinning Gui, Yubo Kou, and Fenglian Liu. 2019a. Culturally-Embedded Visual Literacy: A Study of Impression Management via Emoticon, Emoji, Sticker, and Meme on Social Media in China. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (Nov. 2019), 1–24. https://doi.org/10.1145/3359170
• Wilson and Brewster (2017) Graham Wilson and Stephen A. Brewster. 2017. Multi-moji: Combining Thermal, Vibrotactile & Visual Stimuli to Expand the Affective Range of Feedback. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems . ACM, Denver Colorado USA, 1743–1755. https://doi.org/10.1145/3025453.3025614
• Wiseman and Gould (2018) Sarah Wiseman and Sandy J. J. Gould. 2018. Repurposing Emoji for Personalised Communication: Why means “I love you”. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . ACM, Montreal QC Canada, 1–10. https://doi.org/10.1145/3173574.3173726
• Zhang et al . (2022) Lei Zhang, Tianying Chen, Olivia Seow, Tim Chong, Sven Kratz, Yu Jiang Tham, Andrés Monroy-Hernández, Rajan Vaish, and Fannie Liu. 2022. Auggie: Encouraging Effortful Communication through Handcrafted Digital Experiences. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2 (Nov. 2022), 1–25. https://doi.org/10.1145/3555152
• Zhang et al . (2021) Mingrui Ray Zhang, Ruolin Wang, Xuhai Xu, Qisheng Li, Ather Sharif, and Jacob O. Wobbrock. 2021. Voicemoji: Emoji Entry Using Voice for Visually Impaired People. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . ACM, Yokohama Japan, 1–18. https://doi.org/10.1145/3411764.3445338

Efficacy and safety of dexamethasone sparing for the prevention of nausea and vomiting associated with highly emetogenic risk antineoplastic agents: a systematic review and meta-analysis of the Clinical Practice Guidelines for Antiemesis 2023 from the Japan Society of Clinical Oncology

• Special Article
• Published: 28 September 2024

Cite this article

• Ayako Yokomizo   ORCID: orcid.org/0000-0002-8255-794X 1 ,
• Kazuhisa Nakashima 2 ,
• Arisa Iba 3 ,
• Kenji Okita 4 ,
• Makoto Wada 5 ,
• Keiko Iino 6 ,
• Tatsuo Akechi 7 ,
• Hirotoshi Iihara 8 ,
• Chiyo K. Imamura 9 ,
• Ayako Okuyama 10 ,
• Keiko Ozawa 11 ,
• Yong-Il Kim 12 ,
• Hidenori Sasaki 13 ,
• Eriko Satomi 14 ,
• Masayuki Takeda 15 ,
• Ryuhei Tanaka 16 ,
• Takako Eguchi Nakajima 1 ,
• Naoki Nakamura 17 ,
• Junichi Nishimura 18 ,
• Mayumi Noda 19 ,
• Kazumi Hayashi 20 ,
• Takahiro Higashi 21 ,
• Narikazu Boku 22 ,
• Koji Matsumoto 23 ,
• Yoko Matsumoto 24 ,
• Nobuyuki Yamamoto 25 ,
• Kenjiro Aogi 26 &
• Masakazu Abe 27  

Chemotherapy-induced nausea and vomiting (CINV) are common side effects, classified according to timing and severity. Conventional agents such as dexamethasone are effective but have various side effects. For moderately emetogenic chemotherapy, dexamethasone-sparing antiemetic therapies have been developed to minimize these side effects. This systematic review evaluated the efficacy and safety of dexamethasone-sparing antiemetic therapy for highly emetogenic chemotherapy (HEC).

We performed a thorough literature search for studies related to dexamethasone-sparing antiemetic therapy with neurokinin-1 antagonists (NK 1 RA) for HEC using the PubMed, Cochrane Library, and Ichushi-Web databases. A qualitative analysis of the combined data was performed and risk differences with confidence intervals were calculated.

Two reviewers independently assessed the 425 records and 12 full-text articles were evaluated for eligibility. Two studies were included in the qualitative and meta-analyses. These studies included anthracycline-cyclophosphamide (AC) regimens and cisplatin-based regimens, with palonosetron as the serotonin receptor antagonist. In the two studies, no difference was found in the prevention of vomiting (delayed complete response). However, non-inferiority was not demonstrated in the subgroup that received cisplatin-containing regimens. Delayed complete control showed different results for nausea prevention; however, there was no significant difference in the meta-analysis. Only one report has shown non-inferiority for delayed total control. Although the strength of evidence for individual outcomes varied, there was no difference in the duration of dexamethasone administration.

Conclusions

This systematic review and meta-analysis revealed that dexamethasone-sparing antiemetic therapy with NK 1 RA and palonosetron can be used to prevent CINV in HEC, limited to AC combination therapy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data availability

The data associated with this systematic review can be accessed from the corresponding author upon reasonable request.

Iihara H, Abe M, Wada M et al (2024) 2023 Japan Society of Clinical Oncology Clinical practice guidelines update for antiemesis. Int J Clin Oncol 29:873–888. https://doi.org/10.1007/s10147-024-02535-x

Article   PubMed   PubMed Central   Google Scholar  

Hesketh PJ, Kris MG, Basch E et al (2020) Antiemetics: ASCO guideline update. J Clin Oncol 38:2782–2797

Article   CAS   PubMed   Google Scholar  

Herrstedt J, Celio L, Hesketh PJ et al (2023) 2023 updated MASCC/ESMO consensus recommendations: prevention of nausea and vomiting following high-emetic-risk antineoplastic agents. Support Care Cancer 32:47

National Comprehensive Cancer Network. Antiemesis, version 1.2024. https://www.nccn.org/professionals/physician_gls/pdf/antiemesis.pdf . Accessed April 2024

Vardy J, Chiew KS, Galica J et al (2006) Side effects associated with the use of dexamethasone for prophylaxis of delayed emesis after moderately emetogenic chemotherapy. Br J Cancer 94:1011–1015

Article   CAS   PubMed   PubMed Central   Google Scholar  

Nakamura M, Ishiguro A, Muranaka T et al (2017) A prospective observational study on effect of short-term periodic steroid premedication on bone metabolism in gastrointestinal cancer (ESPRESSO-01). Oncologist 22:592–600

Jeong Y, Han HS, Lee HD et al (2016) A pilot study evaluating steroid-induced diabetes after antiemetic dexamethasone therapy in chemotherapy-treated cancer patients. Cancer Res Treat 48:1429–1437

Celio L, Bonizzoni E, Zattarin E et al (2019) Impact of dexamethasone-sparing regimens on delayed nausea caused by moderately or highly emetogenic chemotherapy: a meta-analysis of randomised evidence. BMC Cancer 19:1268

Scotté F, Schwartzberg L, Iihara H et al (2023) 2023 updated MASCC/ESMO Consensus recommendations: prevention of nausea and vomiting following moderately emetic risk antineoplastic agents. Support Care Cancer 32:45

Article   PubMed   Google Scholar  

Kojimahara N, Nakayama T, Morizane T, et al. (2017) Minds manual for guideline development. Japan Council for Quality Health Care, Tokyo

Shea BJ, Hamel C, Wells GA et al (2009) AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 62:1013–1020

Jaeschke R, Guyatt GH, Dellinger P et al (2008) Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. BMJ 337:a744

Ito Y, Tsuda T, Minatogawa H et al (2018) Placebo-controlled, double-blinded phase III study comparing dexamethasone on day 1 with dexamethasone on days 1 to 3 with combined neurokinin-1 receptor antagonist and palonosetron in high-emetogenic chemotherapy. J Clin Oncol 36:1000–1006

Kosaka Y, Tanino H, Sengoku N et al (2016) Phase II randomized, controlled trial of 1 day versus 3 days of dexamethasone combined with palonosetron and aprepitant to prevent nausea and vomiting in Japanese breast cancer patients receiving anthracycline-based chemotherapy. Support Care Cancer 24:1405–1411

Minatogawa H, Izawa N, Shimomura K et al (2024) Dexamethasone-sparing on days 2–4 with combined palonosetron, neurokinin-1 receptor antagonist, and olanzapine in cisplatin: a randomized phase III trial (SPARED Trial). Br J Cancer 130:224–232

Celio L, Cortinovis D, Cogoni AA et al (2021) Dexamethasone-sparing regimens with oral netupitant and palonosetron for the prevention of emesis caused by high-dose cisplatin: a randomized noninferiority study. Oncologist 26:e1854–e1861

Celio L, Bonizzoni E, Montani E et al (2022) Efficacy of the dexamethasone-sparing triplet regimen for preventing cisplatin-induced emesis: a combined analysis. Future Oncol 18:3389–3397

Zhang L, Lu S, Feng J et al (2018) A randomized phase III study evaluating the efficacy of single-dose NEPA, a fixed antiemetic combination of netupitant and palonosetron, versus an aprepitant regimen for prevention of chemotherapy-induced nausea and vomiting (CINV) in patients receiving highly emetogenic chemotherapy (HEC). Ann Oncol 29:452–458

Hata A, Okamoto I, Inui N et al (2022) Randomized, double-blind, Phase III study of fosnetupitant versus fosaprepitant for prevention of highly emetogenic chemotherapy-induced nausea and vomiting: CONSOLE. J Clin Oncol 40:180–188

Hashimoto H, Abe M, Tokuyama O et al (2020) Olanzapine 5 mg plus standard antiemetic therapy for the prevention of chemotherapy-induced nausea and vomiting (J-FORCE): a multicentre, randomised, double-blind, placebo-controlled, phase 3 trial. Lancet Oncol 21:242–249

Download references

Acknowledgements

The authors are grateful to Mr. Naohiko Yamaguchi and Ms. Yuko Mitsuoka for their contributions in the initial literature search. The authors thank Ms. Kyoko Hamada and Ms. Natsuki Fukuda for their valuable comments and recommendations. We would like to thank Editage ( http://www.editage.jp ) for the English language editing.

This study was supported by Japan Society of Clinical Oncology.

Author information

Authors and affiliations.

Department of Early Clinical Development, Kyoto University Graduate School of Medicine, 54 Kawaharacho, Shogoin, Sakyo-ku, Kyoto, 606-8507, Japan

Ayako Yokomizo & Takako Eguchi Nakajima

Faculty of Medicine, Department of Internal Medicine, Division of Medical Oncology & Respiratory Medicine, Shimane University, 89-1 Enya-Cho, Izumo, Shimane, 693-8501, Japan

Kazuhisa Nakashima

Institute for Global Health Policy Research, Bureau of International Health Cooperation, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjuku, Tokyo, 162-8655, Japan

Department of Surgery, Otaru Ekisaikai Hospital, 1-4-1, Inaho, Otaru, Hokkaido, 047-0032, Japan

Kenji Okita

Department of Psycho-Oncology and Palliative Medicine, Osaka International Cancer Institute, Otemae 3-1-69, Chuo-ku, Osaka, 541-8567, Japan

Makoto Wada

School of Nursing, National College of Nursing School of Nursing, National College of Nursing, Japan, 1-2-1, Umezono, Kiyose, Tokyo, 204-8575, Japan

Department of Psychiatry and Cognitive-Behavioral Medicine, Nagoya City University Graduate School of Medical Sciences, 1 Kawasumi, Mizuho-cho, Mizuho-ku, Nagoya, 467-8601, Japan

Tatsuo Akechi

Department of Pharmacy, Gifu University Hospital, 1-1 Yanagido, Gifu, Gifu, 501-1194, Japan

Hirotoshi Iihara

Advanced Cancer Translational Research Institute, Showa University, 1-5-8 Hatanodai, Shinagawa-ku, Tokyo, 142-8555, Japan

Chiyo K. Imamura

Graduate School of Nursing Science, St. Luke’s International University, 10-1 Akashi-cho, Chuo-ku, Tokyo, 104-0044, Japan

Ayako Okuyama

Division of Survivorship, Institute for Cancer Control, National Cancer Center, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan

Keiko Ozawa

Division of Medical Oncology, Yodogawa Christian Hospital, 1-7-50 Kunijima, Higasiyodogawa-ku, Osaka, 533-0024, Japan

Yong-Il Kim

Division of Medical Oncology, Hematology and Infectious Disease, Fukuoka University Hospital, 7-45-1, Nanakuma, Jonan-ku, Fukuoka, Fukuoka, 814-0180, Japan

Hidenori Sasaki

Department of Palliative Medicine, National Cancer Center Hospital, 5-1-1, Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan

Eriko Satomi

Department of Cancer Genomics and Medical Oncology, Nara Medical University, 840 Shijo-cho, Kashihara, Nara, 634-8521, Japan

Masayuki Takeda

Department of Pediatric Hematology/Oncology, International Medical Center, Saitama Medical University, 1398-1 Yamane, Hidaka, Saitama, 350-1298, Japan

Ryuhei Tanaka

Department of Radiation Oncology, St. Marianna University, 2-16-1, Sugao, Miyamae, Kawasaki, Kanagawa, 216-8511, Japan

Naoki Nakamura

Department of Gastroenterological Surgery, Oisana International Cancer Institute, 3-1-69, Otemae, Chuoku, Osaka, 541-8567, Japan

Junichi Nishimura

Non-Profit Organization Sasaeau-Kai “Alpha”, 518-7 Kawado-cho, Chuo-ku, Chiba-shi, Chiba, 260-0802, Japan

Mayumi Noda

Department of Clinical Oncology and Hematology, The Jikei University School of Medicine, 3-25-8 Nishi-Shinbashi Minato-ku, Tokyo, 105-8461, Japan

Kazumi Hayashi

Department of Public Health and Health Policy, The University of Tokyo, 7-3-1 Hongo, Bunkyoku, Tokyo, 113-0033, Japan

Takahiro Higashi

Department of Oncology and General Medicine, IMSUT Hospital, Institute of Medical Science, University of Tokyo, 4-6-1, Shiroganedai, Minatoku, Tokyo, 108-8639, Japan

Narikazu Boku

Division of Medical Oncology, Hyogo Cancer Center, 13-70 Kitaoji-cho, Akashi, Hyogo, 673-0021, Japan

Koji Matsumoto

Non-Profit Organization Ehime Cancer Support “Orange-no-kai”, 3-8-24 FurukawaMinami, Matsuyama, Ehime, 790-0943, Japan

Yoko Matsumoto

Internal Medicine III, Wakayama Medical University, Kimiidera 811-1, Wakayama, Wakayama, 541-8509, Japan

Nobuyuki Yamamoto

Department of Breast Surgery, National Hospital Organization Shikoku Cancer Center, 160 Kou, Minamiumemoto-matchi, Matsuyama, Ehime, 791-0280, Japan

Kenjiro Aogi

Department of Obstetrics and Gynecology, Hamamatsu University School of Medicine, 1-20-1 Handayama, Chuo-ku, Hamamatsu, Shizuoka, 431-3192, Japan

Masakazu Abe

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the conception and design of this study. A.Y. wrote the draft of the manuscript and all authors reviewed and commented on the manuscript. All the authors have read and approved the final manuscript.

Corresponding author

Correspondence to Kazuhisa Nakashima .

Ethics declarations

Conflict of interest.

Kazuhisa Nakashima received honoraria from Taiho Pharmaceutical Co., Ltd., Chugai Pharmaceutical Co., Ltd., AstraZeneca K.K., and Eli Lilly Japan K.K. Eriko Satomi received honoraria from Shionogi & Co. Ltd. Masayuki Takeda received honoraria from Chugai Pharmaceutical Co., Ltd.; AstraZeneca K.K.; Novartis Pharma K.K.; Ono Pharmaceutical Co., Ltd.; and Bayer. Takako Eguchi Nakajima received research funding from KBBM, Inc. and Takeda Pharmaceutical Co., Ltd. Junichi Nishimura received honoraria from Taiho Pharmaceutical Co. Ltd. Narikazu Boku received honoraria from Ono Pharmaceutical Co., Ltd., Bristol Myers Squibb, Daiichi Sankyo Co., Ltd., Taiho Pharmaceutical Co., Ltd., and Eli Lilly Japan K.K. Koji Matsumoto received honoraria from MSD K.K., Kyowa Kirin Co., Ltd., and Chugai Pharmaceutical Co., Ltd. as well as research funding from Daiichi Sankyo Co., Ltd., MSD K.K., Gilead Sciences, Inc., and Eli Lilly Japan K.K. Nobuyuki Yamamoto received honoraria from MSD K.K., Accuray Japan K.K., AstraZeneca K.K., Abbvie Inc., Amgen Inc., Ono Pharmaceutical Co., Ltd., Guardant Health Japan Corp., Daiichi Sankyo Co., Ltd., Taiho Pharmaceutical Co., Ltd., Takeda Pharmaceutical Co., Ltd., Chugai Pharmaceutical Co., Ltd., Chugai Foundation for Innovative Drug Discovery Science, Lao Tsumura Co., Ltd., Terumo Corporation, Eli Lilly Japan K.K., Nippon Kayaku Co., Ltd., Novartis AG, Pfizer Global Supply Japan Inc., Merck Biopharma Co., Ltd, Pfizer Global Supply Japan Inc., Merck Biopharma Co., Ltd., Janssen Pharmaceutical K.K., and USACO Corporation, as well as legal fees in case of lawsuit from Taiho Pharmaceutical Co., Ltd., Boehringer Ingelheim Japan, Chugai Pharmaceutical Co., Ltd., Kyowa Kirin Co., Ltd., Nippon Kayaku Co., Ltd., Prime Research Institute for Medical RWD, Inc., AstraZeneca K.K., and A2 Healthcare Corporation. The authors declare that there are no conflicts of interest.

Ethical approval

Not applicable.

Informed consent

Formal consent was not required for this type of study. Not applicable.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Yokomizo, A., Nakashima, K., Iba, A. et al. Efficacy and safety of dexamethasone sparing for the prevention of nausea and vomiting associated with highly emetogenic risk antineoplastic agents: a systematic review and meta-analysis of the Clinical Practice Guidelines for Antiemesis 2023 from the Japan Society of Clinical Oncology. Int J Clin Oncol (2024). https://doi.org/10.1007/s10147-024-02624-x

Download citation

Received : 17 July 2024

Accepted : 01 September 2024

Published : 28 September 2024

DOI : https://doi.org/10.1007/s10147-024-02624-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Dexamethasone
• Highly emetogenic chemotherapy
• Chemotherapy-induced nausea and vomiting
• Meta-analysis
• Find a journal
• Publish with us
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Instructions for Authors
• BMJ Journals

You are here

• Online First
• Digital health for cancer symptom management in palliative medicine: systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-4123-9432 Meryem Hamdoune ,
• http://orcid.org/0000-0002-1256-6031 Khaoula Jounaidi ,
• Nada Ammari and
• Abdellah Gantare
• Hassan First University of Settat, Higher Institute of Health Sciences, Laboratory of Health Sciences and Technologies , Settat , Morocco
• Correspondence to Meryem Hamdoune; m.hamdoune{at}uhp.ac.ma

Background Digital health technologies (DHTs) play a crucial role in symptom management, particularly in palliative care, by providing patients with accessible tools to monitor and manage their symptoms effectively. The aim of this systematic review was to examine and synthesise the scientific literature on DHTs for symptom management in palliative oncology care.

Methods A systematic review was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for systematic reviews and meta-analyses from 2 June to 20 June 2024. Databases including Scopus, Web of Science, ScienceDirect, PubMed and the Cochrane Library were searched. Data were extracted using a standardised form based on the PICOTT (Population, Intervention, Comparison, Outcome, Type and Technology) framework. The quality of the included studies was assessed using the Appraisal of Guidelines for Research & Evaluation (AGREE) II tool during the selection process.

Results The systematic review included seven articles describing six DHTs from five countries: the UK, Kenya, Tanzania, the Netherlands and the USA. The findings of this comprehensive literature review elucidate four principal themes: the specific types of DHTs used for symptom management in palliative cancer care, their roles and advantages, as well as the factors that limit or promote their adoption by patients and healthcare professionals.

Conclusion The findings of this review give valuable insights into the ongoing discourse on integrating digital health solutions into palliative care practices, highlighting its potential role in enhancing symptom management within palliative cancer care and showcasing its possible benefits while also identifying key factors influencing their adoption among patients and healthcare professionals.

• Palliative care
• Symptoms and symptom management

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/spcare-2024-005107

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Digital health is an expanding field that provides innovative solutions to enhance palliative care delivery through improved efficiency, accessibility and effectiveness.

WHAT THIS STUDY ADDS

This systematic review provides an overview of what digital health technologies are used for symptom management in palliative oncology care, their functions, benefits, efficiency as well as potential factors that influence their adoption by patients and healthcare professionals.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

The results of this review could be instrumental for policymakers, clinicians and health technology developers in the design, implementation and evaluation of appropriate and effective digital solutions for managing bothersome symptoms in palliative care patients with cancer.

Introduction

Digital health is a swiftly expanding field that presents dynamic opportunities for innovation and enhancements in health services. 1 It aims to enhance the efficiency, accessibility and effectiveness of healthcare by using digital technology to gather, analyse, manage and distribute health data. 1 Digital health technologies (DHTs) encompasses mobile health applications, electronic health records, telehealth, human-machine interaction, wearable sensors, artificial intelligence and other types of DHTs. 1 2

Digital health interventions are increasingly crucial for adults, children and young people undergoing cancer treatment and palliative care (PC). 2 3 Indeed, the WHO has advocated for 80% availability of accessible technologies to address non-communicable diseases, such as cancer. 4

According to the WHO’s Global Cancer Statistics report, 20 million new cancer cases and 9.7 million cancer deaths were recorded worldwide in 2022. 5 About 53.5 million people were estimated to be alive 5 years post-diagnosis. 6 Regional variations exist, with Africa and Asia experiencing higher death rates due to late-stage diagnosis. Europe reports 22.4% of global cancer cases and 20.4% of deaths, despite having only 9.6% of the world’s population. 5 Future projections indicate over 35 million new cancer cases will occur in 2050, with a 77% increase from the 20 million cases estimated in 2022. 5 Regionally, the most significant rise in incidence will occur in high and very high Human Development Index (HDI) countries, including China, with an additional 4.8 and 3.9 million cases, respectively. Lower HDI regions will see a 142% increase, from 0.8 million cases in 2022 to 2 million in 2050, while medium HDI countries, like India, will witness nearly a 100% increase, doubling from 2.4 million to 4.8 million cases. 5

In this regard, provision and accessibility of pain management and PC becomes highly required as pain is reported to be experienced by 55% of patients undergoing anticancer treatment and by 66% of patients who have advanced, metastatic or terminal disease. 7 Patients with advanced palliative cancer experience a range of symptoms, with the most distressing being pain, fatigue and anxiety. 8 Other common symptoms include loss of appetite, dyspnoea, constipation and nausea. 9

Pain is a frequently experienced and poorly managed symptom in patients with cancer. 10 11 More than two-thirds of patients will suffer from pain in the final stages of their cancer. 12 Pain significantly contributes to suffering and adversely affects their quality of life, often resulting in unplanned hospital admissions due to unmanaged symptoms. 10 13

PC aims to improve the quality of life for patients dealing with serious or life-threatening illnesses, such as cancer. 14 The WHO has reported that each year an estimated 56.8 million people are in need of PC, most of whom live in low- and middle-income countries. 15 While availability and access to this care varies. Overall, high-income countries, especially in Europe, over two-thirds of the countries, offer PC in both community or home-based care settings and in primary healthcare settings. 15 In contrast, in low-income countries, only 19% have PC available in primary healthcare settings and 10% in community or home settings. 15 A similar disparity exists in the Southeast Asia region, where 55% of countries report PC availability in primary healthcare settings, compared with 36% in community or home-based settings. 15

To improve accessibility and effective PC and pain management for patients with cancer, the use of technologies to facilitate this process is a growing area of interest for patients, caregivers and policymakers. Research suggests that remote symptom monitoring significantly enhances care for patients with palliative needs, leading to improved outcomes in cancer symptom management. 16 17 Electronic self-monitoring enables patients to track pain fluctuations and their connections to factors such as medication intake and daily activities. 11 16 Healthcare technologies facilitate effective self-management by providing organised data access for both patients and healthcare professionals. 18 These approaches not only aid in monitoring and controlling symptoms but also promote treatment adherence and expand healthcare accessibility across diverse patient populations. 19 No evidence to our knowledge has explored these technologies and their benefits in managing pain and different symptoms in palliative cancer care. This systematic review aims to fill this knowledge gap and offers a review of the adoption of these technologies and the evidence surrounding their effectiveness.

This systematic review aims to:

Identify the DHTs used to manage symptoms, particularly pain, in palliative cancer care.

Evaluate the advantages of DHTs in managing symptoms in patients with cancer requiring PC.

Explore the factors influencing the adoption of DHTs in palliative oncology care among patients and healthcare professionals.

Material and methods

Study design.

This systematic review aimed to identify DHTs used for symptoms management (especially pain) in PC patients with cancer, focusing on their effectiveness and the factors influencing their adoption by patients and healthcare professionals. The study adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, 2020 edition.

Search strategy

We conducted a comprehensive search through Scopus, Web of Science, ScienceDirect, PubMed and the Cochrane Library between 2 June 2024 and 20 June 2024, using the following keywords: (‘Palliative care’ OR ‘palliative cancer care’) AND (‘eHealth’ OR ‘digital health’) AND (‘pain management’ OR ‘symptom monitoring’). The PRISMA flow diagram illustrating the flow of information through the different phases of the review is included in figure 1 .

• Download figure
• Open in new tab
• Download powerpoint

Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram flow.

Inclusion and exclusion criteria

Inclusion criteria encompassed peer-reviewed articles employing qualitative, quantitative and mixed methods, and research focusing on DHTs for symptoms management in palliative oncology care. The search was limited to articles published (full text available) between 2013 and 2023 and restricted to English-language publications.

Exclusion criteria included non-peer-reviewed articles, conference abstracts, reviews, meta-analysis, research programmes and study protocols and articles not focused on DHTs used to manage symptoms in palliative oncology care.

Selection process

After searching all the databases and removing duplicates, we first screened the titles and abstracts of the identified studies to check their relevance to the topic, excluding those deemed irrelevant. Second, we obtained the full texts of studies that appeared to meet the inclusion criteria. The researchers then examined all the retrieved articles separately for relevance. Discrepancies were resolved through discussion and consensus.

Data extraction and quality assessment

Data were extracted using a standardised form based on the PICOT framework (Population, Intervention, Comparison, Outcome and Type). 20 To meet our research objective, we included a second ‘T’ (PICOTT) to denote the technology used.

The research team maintained rigour and avoided biases during data collection through the strict following of the clearly defined inclusion and exclusion criteria to curb selection bias, and the use of multiple databases to ensure comprehensive coverage and identification of all relevant studies. Additionally, to eliminate interpretation bias, a reading committee consisting of four members evaluated and synthesised the studies to guarantee an objective interpretation. Furthermore, to ensure intercoder reliability of data extraction, the research team members independently and concurrently analysed the included studies, and any discrepancies were resolved through discussion and consensus.

The final step of the selection process involved assessing the quality of the included studies with the Appraisal of Guidelines for Research & Evaluation (AGREE)II tool. Originally developed by Brouwers et al and updated in 2016 to enhance accuracy and neutrality, this tool includes 23 items distributed across six domains: scope and purpose, stakeholder involvement, rigour of development, clarity of presentation, applicability and editorial independence. Every item is rated on a Likert scale from 1 (strongly disagree) to 7 (strongly agree). 21 The scores of the quality appraisal results from the selected studies are illustrated in table 1 .

• View inline

The scores of the quality appraisal results from the selected studies

Article selection

A total of seven papers adhered to the selection criteria and were included and examined in this study. The studies included originated from five countries: the UK, Kenya, Tanzania, the Netherlands and the USA. With two articles originating from the UK, two from the USA, two articles from the African region and one from the Netherlands. The studies were published in various years, including one from 2016 (Hochstenbach et al ), three from 2020 (Hackett et al , Ingram et al and Wilkie et al ), one from 2022 (Ho et al ) and one from 2023 (Cornetta et al ). Most papers focused on the development and testing of DHTs. 11 22–26 . Table 2 presents study characteristics relevant to the purpose of this review.

Characteristics of included studies

The results of this systematic literature review highlight the following themes: types of DHTs used to manage symptoms in palliative cancer care, their functions, their benefits and the factors influencing their adoption by patients and healthcare professionals.

Types of DHTs reported in the studies included

Mobile/web health applications.

PainCheck is an information and communication technology system designed to regularly assess and monitor pain in patients in the late stages of cancer. The system is the outcome of a significant research initiative (IMPACCT (Improving the Management of Pain From Advanced Cancer in the Community; ISRCTN registry No. 18281 271)) conducted in the UK. The software development team comprised three developers and a business analyst, all proficient in agile methodologies. Development adhered to the Disciplined Agile Delivery framework, a structured approach used by software developers to direct the creation of information and communication technology systems from initial conception through implementation to eventual retirement. The application questions are derived from the Brief Pain Inventory and the Coping Strategies Questionnaire. 10

Mobile Palliative Care Link

Mobile Palliative Care Link (mPCL) is a user-focused mobile/web application developed to enhance symptom management and quality of life for patients with cancer by facilitating remote, real-time symptom assessment and interdisciplinary care coordination. The mPCL was developed based on the African Palliative Care Outcome Scale and uses CommCare, a secure, cloud-based, open-source platform. It can be accessed via a native Android application or a web browser and supports both online and offline data collection in areas with limited connectivity. 23

A mobile application connected to a web application

It is a mobile phone and web-based self-management support application for patients and nurses that uses the Numerical Rating Scale. The application was developed as part of an iterative co-creative development process involving healthcare workers, patients, researchers and technical experts. 11

PAINRelieveIt

PAINRelieveIt is a web-based application for pain management that offers a systematic approach to improving cancer pain outcomes. The application equips patients with suitable language for reporting pain and provides clinicians with a clinical decision aid to facilitate the prescription of analgesics. It incorporates the McGill Pain Questionnaire, Pain Management Index and the Barriers Questionnaire-13 to enhance its effectiveness. 22

Digital platform

The palliative/end-of-life/assessment/care coordination/evidence-based programme.

Palliative/End-of-Life/Assessment/Care Coordination/Evidence-Based Programme (PEACE) is an interactive platform designed to improve PC by monitoring symptoms and supporting patients through technology-assisted condition management and care pathway protocols. It also alerts the PC team when intervention is required. 24

Cornetta et al 27 established a weekly follow-up programme to evaluate physical symptoms and distress in patients and their careers. Participants’ symptoms were initially evaluated at enrolment and then assessed weekly basis, through questions from the validated Africa Palliative Care Association Palliative Outcome Scale. 27

Key features of DHTs in palliative cancer care

The various functions of DHTs identified in this review are summarised in table 3 , which provides a detailed overview of each technology and its functions to managing symptoms in palliative cancer care.

Benefits of DHTs for symptoms management

Pain management.

The patients stressed the substantial value of the applications developed by Hochstenbach, Allsop and their team in assisting them with self-managing their pain and pain medication. 11 25 26 Furthermore, Cornetta et al 27 stated that a weekly monitoring programme makes it possible to adjust medication and facilitate refills. 27

Promote interdisciplinary care coordination

Ho and colleagues 23 mentioned that the mPCL seems to enhance interdisciplinary care coordination by allowing carers to interact with patients and their caregivers. It offers quick access to shared medical records and real-time responses at the point of care, ensuring timely symptom monitoring. It also records clinical interactions with patients after discharge and tracks longitudinal treatment decisions. 23

Improving care and support

The mPCL and PainCheck enhanced the speed and efficiency of care while tackling financial, transportation and other patient-specific challenges that are often present in traditional care settings. 23 Patients perceived these applications as a supportive resource. They no longer felt alone; instead, they felt engaged and integrated into the services. This led to a sense of reassurance, knowing that assistance was readily available when needed. 25 26

Remote symptom control

Having a common platform (mPCL) ensures that patients with milder symptoms can be triaged and treated more effectively as situations arise. On the other hand, it facilitates assessing symptoms, maintaining contact and remotely communicating with patients after their discharge from the hospital, which represents a significant improvement compared with the previous lack or scarcity of symptom management resources in the community. 23 In the same vein, PEACE has enabled the proactive and continuous monitoring of symptoms for hospice patients at home without increasing nurse visits, resulting in high overall satisfaction among patients, their families and staff. 24 While, Cornetta et al 27 have shown that telehospice can offer a temporary solution for PC to patients who lack access to PC services at home. 27

Factors influencing adoption of DHTs in palliative cancer care

Familiarity with technology.

Healthcare professionals’ knowledge, confidence, understanding and familiarity with information and communication technologies affected their own engagement, which subsequently influenced how much they encouraged and facilitated patient engagement. 23 25 26 One barrier to understanding and becoming familiar with the system was the workload, as patients in the trial were distributed across different clinical nurse specialists. Consequently, these healthcare professionals had limited opportunities to use and familiarise themselves with the system. 26 In this respect, Ingram et al 24 have pointed out that to improve the use of new technologies, it is imperative to employ just-in-time training methods for staff. 24

Availability of technological resources

Patients’ and healthcare providers’ access to e-Health technologies has also been influenced by limited access to technological resources such as phones, SIM cards, internet connection at home and not having a computer and handheld. Indeed, people who rarely used technology or computers, often lacked an internet connection or struggled with digital technology, found its use stressful and, as a result, were reluctant to incorporate it into their daily routine. 23 25 Therefore, leveraging patient technology platforms including tablets, televisions and smartphones could simplify this process and enhance adherence. 24

Ease of access and use

Participants in studies by Allsop et al 10 (2019) and Hackett et al 26 characterised the PainCheck system as straightforward, user-friendly, efficient and minimally intrusive. 25 26 Similarly, Hochstenbach et al reported that patients appreciated the mobile application’s ease of use and its range of features. 11 Additionally, Ho et al 23 found that despite some reported challenges, most providers deemed mPCL ‘easy to use’ and expressed a commitment to continue using it if it remained available. 23

Health professionals’ commitment

Patient engagement with PainCheck was further shaped by healthcare providers responsible for facilitating and monitoring patient interaction with the information and communication system. 25 In fact, some patients mentioned that nurses did not introduce them to PainCheck to avoid adding an unnecessary burden. 25 26 According to Hackett et al, 26 effective implementation and improved engagement of professionals and patients necessitate proactive support and collaboration between the research team and healthcare workers managing information and communication systems. 26 Moreover, the study by Hochstenbach et al (2023) emphasised that the follow-up and guidance provided by nurses, combined with their collaboration with attending physicians, were essential for improving patients’ experiences. 11 Similarly, Ho et al 23 reported that most clinicians demonstrated a keen enthusiasm in continuing to use mPCL and in sharing the application with their peers. They indicated that extending the application’s availability to patients across Tanzania could significantly enhance access to PC, especially in rural regions where services are scarce or non-existent. 23

Perceived usefulness of DHTs

Although digital technology enables healthcare professionals to contact patients online, they appeared sceptical about its ability to deliver the same level of personalised care as their traditional methods, notably telephone and face-to-face consultations. 26 Moreover, healthcare professionals voiced concerns about the scalability of their engagement with PainCheck, noting that using the system might add to their workload if they were managing a larger number of patients. 26 Wilkie et al 22 found that nurses who did not use the application mentioned that they had limited time to access the information, as they perceived it as optional since they were already familiar with their patients’ needs and preferences. 22 Additionally, participants in a study by Hochstenbach et al expressed similar concerns about the effectiveness of digital tools compared with traditional care methods. Despite their enthusiasm for the application, nurses viewed their new approach to work as a genuine challenge. It introduced new tasks, different responsibilities and unfamiliar technologies, which required an adjustment period. 11

Connectivity and software problems

Healthcare professionals have noted usability challenges such as the complexity of generating a clinical record for a newly registered patient and the requirement to promptly address real-time reminders or alerts for routine, non-urgent updates on a patient’s condition. 23 Patients also encountered several primary technological issues, including not receiving a diary, inability to record medications, lack of an updated graph and difficulty accessing information. 11 Furthermore, the application failed to document the initial pain assessment, which was essential for facilitating patient transfer and maintaining continuity of care. 11

This systematic review aimed to identify DHTs used for symptom management, especially pain, in PC for patients with cancer. It specifically focused on their effectiveness and the factors that influence their adoption by patients and healthcare professionals.

The literature review identified different types of DHTs used to manage bothersome symptoms, particularly pain, in palliative cancer care. The DHTs reported in the reviewed studies encompassed mobile/web health applications, digital platforms and telehospice. The mobile/web health applications consisted of PainCheck, 25 26 mPCL, 23 a mobile application connected to a web application, 11 and PAINRelieveIt. 22 PainCheck operated in the UK, mPCL was used in Tanzania, the mobile application connected to a web application was developed in the Netherlands and PainRelievIt was created in the USA.

The only digital platform identified was the PEACE 24 and was used in USA, while telehealth 27 was implemented in Kenya to evaluate physical symptoms and distress in patients and their careers.

The existing literature underscores the availability of a diverse array of DHTs for the effective management of symptoms in PC, with a particular emphasis on pain relief. For instance, Bhargava et al 16 created RELIEF, a remote self-reporting application designed for community patients requiring PC. The pilot feasibility study showed that RELIEF is a practical and well-accepted tool for monitoring patients care patients remotely by enabling regular symptom self-reporting. 16

The studies reviewed highlighted several key features of mobile/web health applications, including personalised pain management advice and real-time assessment of symptoms and quality of life. 11 23 25 26 These applications facilitate the documentation of users’ clinical interactions with patients and caregivers post-discharge, and they provide short message service capabilities for direct contact between patients or caregivers and clinicians during emergencies. 11 23 Additionally, they include a diary function that enables patients to monitor pain and side effects, allowing for regular reporting and sharing of pain data with health professionals. 11 22 23 25 26 Educational sessions within the applications offer essential information on pain management, while a clinical decision support tool assists clinicians in prescribing analgesics effectively. 11 22 23

Telehospice enables the remote assessment of physical symptoms and distress for both patients and caregivers through remote patient monitoring. 27 The digital platform ‘PEACE’ offers several key features, including video consultations with a nurse at the hospice centre, which facilitate direct communication and support for patients and their families. Additionally, this platform provides telephone interactions, allowing for convenient and timely access to care and information, ensuring that patients receive the assistance they need in a flexible manner. 24

The potential of DHTs for symptom management, particularly pain, in palliative patients with cancer was demonstrated in all reviewed studies. 11 22–27 The benefits included pain management, promoting interdisciplinary care coordination, improving care and support and remote symptom control. A systematic integrative review revealed that e-Health improved individualised care for patients in PC, increased their sense of security, enhanced symptom management and boosted their participation in care. 28 Zhou et al found that telephone follow-up is a practical alternative to hospital visits for patients with advanced cancer seeking symptom relief, significantly reducing the burden of travel. 29 Additionally, the use of web-based platforms to gather patient-reported outcomes has been linked to enhancements in health-related quality of life and overall survival, facilitated by proactive management of emerging symptoms. 30

The factors influencing the adoption of DHTs in palliative cancer care, identified in the articles selected for the systematic review, were familiarity with technology, availability of technological resources, ease of access and use, health professionals’ commitment, perceived usefulness of interventions and connectivity and software problems. A study carried out among digitally lagging nurses, aimed at identifying the factors influencing their adoption of health information technology, revealed that a negative attitude toward computer use and a lack of digital skills contributed to feelings of increased incompetence, leading to the postponement or avoidance of health information technologies both privately and professionally. 31 Similarly, Wicki et al 2 found that barriers to the acceptance of DHTs among PC patients included unfamiliarity, concerns about data security, errors in data interpretation and the loss of personal interaction due to artificial intelligence. 2

Strengths and limitations

Our systematic review exhibited several strengths, including rigorous adherence to well-defined inclusion and exclusion criteria to minimise selection bias, and the use of multiple databases to ensure comprehensive coverage of relevant studies. To reduce interpretation bias, a four-member reading committee provided objective analysis and synthesis of the data. Additionally, independent and concurrent data extraction by team members, with discrepancies resolved through discussion, ensured high intercoder reliability.

This study has also several limitations to consider. The systematic review misses relevant studies in non-English languages or those in less common databases. Additionally, it excludes studies published before 2013, potentially omitting important research. Moreover, the review does not include grey literature, such as reports, theses and conference papers, which can provide valuable insights and data not available in peer-reviewed journals. This omission introduces bias, as grey literature often contains unique findings and alternative perspectives that might be crucial for a comprehensive understanding of the topic.

Recommendations

Encourage the systematic integration of proven digital technologies into symptom management protocols for patients with cancer in PC.

Provide ongoing training for healthcare professionals in the use of digital technologies for symptom management, focusing on adapting these technologies to their clinical practices and providing the necessary technical support.

Ensure equitable access to digital technologies for all PC patients, considering possible economic and technological barriers.

Promote ongoing evaluation of the effectiveness, safety, and acceptability of DHTs for symptom management, considering feedback from patients and healthcare professionals.

Implications of the results for practice, policy and future research

For practice:

Our study provides a comprehensive list of available digital health technologies, assisting healthcare professionals in selecting the most suitable tools for managing symptoms in palliative patients with cancer. By identifying the specific functions of each technology, caregivers can adopt a more personalised and effective approach to monitoring and treating symptoms.

For policy:

Our research results can help policymakers develop strategies and policies to enhance the adoption of digital health technologies in PC. By gaining a deeper understanding of the factors that hinder adoption, policymakers can design targeted interventions to address these challenges.

For research:

Our study identifies not only current technologies and their functions, but also the barriers to their adoption. This information is invaluable in guiding future research, highlighting areas requiring technological innovation or in-depth study.

In conclusion, this systematic review emphasises the important role of DHTs in improving symptom management in palliative cancer care. The results highlighted the various DHTs and their advantages while identifying key factors that limit or promote their adoption by patients and healthcare professionals. This research offers valuable insights about incorporating digital health solutions into PC practices, highlighting the need for targeted strategies to promote adoption and optimise the integration of these technologies for better patient outcomes. By addressing both the potential benefits and the barriers to implementation, this study lays the groundwork for future initiatives aimed at enhancing the quality and accessibility of PC through innovative digital tools.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

• Sasi R , et al
• Amann M , et al
• Begovic D ,
• Salifu Y , et al
• World Health Organization
• Laversanne M ,
• Sung H , et al
• Gilbertson-White S ,
• Aouizerat BE ,
• Jahan T , et al
• Van Lancker A ,
• Van Hecke A , et al
• Allsop MJ ,
• Bennett MI , et al
• Hochstenbach LMJ ,
• Zwakhalen SMG ,
• Courtens AM , et al
• van den Beuken-van Everdingen MHJ ,
• Joosten EAJ , et al
• Haddou Rahou B , et al
• Bhargava R ,
• Keating B ,
• Isenberg SR , et al
• Vrancken Peeters NJMC ,
• Koppert LB ,
• Jager A , et al
• Kuijpers W ,
• Aaronson NK , et al
• Lacey C , et al
• Brouwers MC ,
• Browman GP , et al
• Wilkie DJ ,
• Ezenwa MO , et al
• Lambden K , et al
• Shadbolt EL , et al
• Johnson O ,
• Taylor S , et al
• Hackett J ,
• Cornetta K ,
• Nyariki S ,
• Manji I , et al
• Widberg C ,
• Wiklund B ,
• Zhou M , et al
• Kris MG , et al
• De Leeuw JA ,
• Woltjer H ,

Contributors Designing the systematic review protocol and developing the search strategy: MH and KJ. Conducting the literature search, screening and data extraction: All authors. Performing the quality assessment of included studies: All authors. Contributing to the synthesis of results: All authors. Providing critical revisions and overseeing the writing and finalisation of the manuscript: All authors. MH is responsible for the overall content of the manuscript and acts as the guarantor.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF: