Effective use of research-based information is critical for informed decision- and policy-making. However, assessing the use and influence of such information can pose significant challenges due to the complex interactions between scientific knowledge, policy processes, and stakeholder interests (Bogenschneider et al., 2021; Englund et al., 2022; Soomai et al., 2016). In this blog post, we explore the importance of communication between researchers and policymakers when measuring the use and influence of research-based information and we highlight the benefits and challenges for enhancing collaboration.
Importance of Research-Based Information
Understanding how research informs policy decisions can improve the effectiveness and accountability of management efforts, ultimately contributing to the conservation and sustainable use of natural resources (Bogenschneider et al., 2021; Soomai et al., 2016). Despite the importance of measuring information use and influence, many challenges and complexities characterize the use and influence concepts. The often-interdisciplinary nature of research and policymaking requires collaboration between researchers from various fields, policymakers in different levels of government, stakeholders, and practitioners (Soomai et al., 2016). Diversities in perspectives can lead to differing interpretations of research findings and priorities, complicating efforts to assess use and influence of research (Bogenschneider et al., 2021; Cvitanovic et al., 2021).
Measuring the use and influence of research-based information requires a multifaceted approach. Quantitative indicators, such as citation analysis and usage statistics, provide insights into the distribution of research findings among researchers, policymakers, and, to a lesser extent, the public (Soomai et al., 2016). Qualitative methods, including interviews and case studies, can extend understanding into how research informs decision-making processes and influences policy outcomes. By combining quantitative and qualitative approaches, researchers can develop a fuller understanding of how research-based information shapes policy and management actions (Soomai et al., 2016).
Collaboration Through Communication
Communication between researchers and policymakers facilitates better integration of scientific knowledge into policies (Bogenschneider et al., 2021; Soomai et al., 2016). Policymakers rely on research-based information to develop evidence-based policies that address pressing issues (Cvitanovic et al, 2021). Effective communication enables the exchange of ideas, data, and insights between scientists and policymakers, thereby negotiating a balance between scientific understanding and real-world considerations and policy priorities (Bogenschneider et al., 2021; Soomai et al., 2016).
Enhancing collaboration between researchers and policymakers requires proactive efforts to build trust and mutual understanding (Bogenschneider et al., 2021; Cvitanovic et al., 2021). Regular engagement through stakeholder meetings, workshops, and advisory panels provides venues for researchers to share findings, strategies, and uncertainties directly with policymakers, which can ensure scientific evidence is accessible and comprehensible (Cvitanovic et al., 2021). In exchange, these forums offer an opportunity for policymakers to present valuable insights into policy needs, priorities, and implementation challenges, which can guide researchers in framing research questions and methodologies to align with real-world policy contexts (Bogenschneider et al., 2021).
Despite the benefits of communication between researchers and policymakers, several challenges must be addressed to achieve effective collaboration (Bogenschneider et al., 2021; Cvitanovic et al., 2021). Differences in language and values between the scientific and policy communities can hinder communication/understanding. Also, the fast-paced nature of policy decision-making processes places a requirement on researchers to deliver timely and relevant information that meets policymakers’ needs and constraints (Bogenschneider et al., 2021). Overcoming these challenges involves ongoing dialogue, relationship-building, and development of communication strategies specific to the needs and preferences of both researchers and policymakers.
Research Value vs. Research Use
While citations have traditionally been the most common indicator for measuring impact, many researchers suggest that this metric may not capture a holistic picture of research value. Williams (2022) developed a novel method of assessing research value, one that considers both citations and altmetrics. Altmetrics are, in essence, a gauge of the online “buzz” created by research papers. This measurement includes mentions in social media (like Facebook, Instagram, etc.), as well as bookmarks and other internet-based engagement tools (Williams, 2022). This approach highlights that research value appears in many forms. Just because a paper has not have received many citations does not mean it is of no importance as value can be revealed by other indicators of engagement of broader non-academic audiences in important conversations about science.
Arnott and Lemos (2021) also studied measurements of research use. Their paper examined research programs where collaboration between researchers and the “users” of knowledge was required as part of research funding. They found that researchers struggled to define use of research, predict who the users would be, and articulate whether such users would translate the research into action (Arnott & Lemos, 2021).
Arnott’s and Lemos’s (2021) examination of the creation of practical, immediately-useful research, and Williams’s (2022) description of how research is valued by citations and altmetrics suggests there is a challenge in overall evaluation of research. Should researchers be encouraged to produce outputs that will be shared widely, both within academia and social media, or should they co-produce research to create tangible utility? If research achieves high altmetric indicators and/or citations, does this imply that it also includes usable scientific knowledge? These questions deal with the issue of defining research value and use and highlight the fact that there are many varying perspectives on what research “value” really means.
Different Perspectives on Value of Research-Based Information
A lack of consensus among researchers and stakeholders on the value of research-based information makes it challenging to specify how researchers should measure its use and influence. As noted by Cvitanovic et al. (2021), the heavy emphasis on traditional forms of research impact should be modified to consider a wider range of “success” in terms of how research influences multiple sectors of society. The methods used to measure research impact or value have consequences for policy priorities. Policymakers must figure out how to balance competing perspectives/priorities when developing policies and management strategies. Without consensus on the value of research-based information, policymakers may have difficulty identifying the best available information for creating effective policies that adequately address the diverse needs and challenges present in complex societal problems.
However, the lack of consensus between measuring values of various stakeholders and research-based information can be mitigated through a knowledge co-production approach, which involves the production of knowledge through shared research processes of individuals with differing backgrounds. Englund et al. (2022) identified co-production as a potential method for intertwining different values among stakeholders and producing stronger understanding of problems and solutions. They highlight important participatory approaches for involving stakeholders in policymaking, such as adaptive learning through developmental evaluation practices, adaptive management, building and refining the Theory of Change approach, participatory evaluation methods, and combining varyious data collection methods (Englund et al., 2022). With these guidelines, handling the complexity of differentiated values of research-based information can be simplified, external factors and commonalities become identifiable, and production of stronger data sets can be made available to stakeholders for better understanding of the problem and differentiating values of research-based information (Englund et al., 2022).
Conclusion
Understanding of the use and influence of research-based information is essential for enhancing decision-making. Despite the challenges and complexities involved, efforts to measure information use and influence are crucial for improving the effectiveness, transparency, and sustainability of social policy in today’s increasingly complex and dynamic environment.
References
Arnott, J. C., & Lemos, M. C. (2021). Understanding knowledge use for sustainability. Environmental Science & Policy, 120, 222-230. https://doi.org/10.1016/j.envsci.2021.02.016
Bogenschneider, K., Normandin, H., Onaga, E., Bowman, S., Wadsworth, S. M., & Settersten, R. A. (Jr.). (2021). Evaluating efforts to communicate research to policy makers: A theory of change in action. In K. Bogenschneider & T. J. Corbett. Evidence-based policymaking: Envisioning a new era of theory, research, and practice (2nd ed., Chapter 8, pp. 195-231). New York: Routledge.
Cvitanovic, C., Mackay, M., Keenan, R. J., van Putten, E. I., Karcher, D. B., & Dickey-Collas, M. (2021). Understanding and evidencing a broader range of “successes” that can occur at the interface of marine science and policy. Marine Policy, 134, 104802. https://doi.org/10.1016/j.marpol.2021.104802
Englund, M., André, K., Gerger Swartling, Å., & Iao-Jörgensen, J. (2022). Four methodological guidelines to evaluate the research impact of co-produced climate services. Frontiers in Climate, 4, 909422. https://doi.org/10.3389/fclim.2022.909422
Soomai, S. S., Wells, P. G., MacDonald, B. H., De Santo, E. M., & Gruzd, A. (2016). Measuring awareness, use, and influence of information: Where theory meets practice. In B. H. MacDonald, S. S. Soomai, E. M. De Santo, & P. G. Wells (Eds.), Science, information, and policy interface for effective coastal and ocean management (Chapter 11, pp. 253-279). Boca Raton, FL: CRC Press. https://www.taylorfrancis.com/chapters/oa-edit/10.1201/b21483-22/measuring-awareness-use-influence-information-theory-meets-practice-suzuette-soomai-peter-wells-bertrum-macdonald-elizabeth-de-santo-anatoliy-gruzd
Williams, K. (2022). What counts: Making sense of metrics of research value. Science and Public Policy, 49(3), 518-531. https://doi.org/10.1093/scipol/scac004
Authors: Katie Mansvelt, Raven Freeman-Wink, and Logan Collins
This blog post is part of a series of posts authored by students in the graduate course “Information in Public Policy and Decision Making” offered at Dalhousie University.
