Measuring and promoting research impacts using alternative metrics

Measuring and promoting research impacts using alternative metrics

Measuring and promoting research impacts using alternative metrics as a means towards SMART research in African Universities By Malekani, A. W (PhD) Sokoine University of Agriculture (SUA, Sokoine National Agricultural Library (SNAL), P. O Box 3022, Morogoro, Tanzania E-mail: [email protected] Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities Outline:

Introduction Methodology of the study Global presence of African research output Measuring scholarly output traditional methods alternative methods Altmetrics What is Altmetrics; what are the advantages of altmetrics? What is measured? Who is doing it? PlumAnalytics, Inside Story, PLOS, KUDOS, Altmetric.com Controversy / limitations/conclusions Carnegie CPD programme

Enhancing librarians' ICT skills for research enablement in African universities 2 Introduction LIS workplace and profession has changed Then Now Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 3

and so has the sources. Then Now Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 4 And so has the library user. Then

Now Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 5 Introduction Quantifying scholarly output via traditional citation metrics is the time-honoured method to gauge academic success. These methods are used by Universities for promotion purposes, etc. Assessing the impact of a scholars work can be measured by evaluating several factors including the number of peer-reviewed publications, citations to these publications, and the influence of the

publications. However, these metrics take a relatively long time to accumulate. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 6 Introduction cont This paper outlines alternative metrics for measuring research output as opposed to traditional citation metrics like impact factors (IF) and Hindex which are what a lot of researchers are privy to. It presents the alternative metrics (altmetrics) that can be promoted by LIS profession that measure the impact of a publication by looking at several factors like the number of times the

paper has been downloaded, viewed, shared, liked and mentioned on social media. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 7 Methodology of the study The paper uses mainly literature review as a method for the study. The paper highlights first global presence of African research output and the traditional metrics and their limitations and then explores the alternative metrics that researchers should opt to publish their research articles for Carnegie CPD programme Enhancing librarians' ICT skills for research

enablement in African universities 8 Global presence research output of African In 2006, there was low global presence of African research output. According to Dick (2013) and Schmaus (2008), SubSaharan Africa contributed 0.7% to world science output. There was not just a challenge of raising the quality and quantity of scientific output in Africa, but also of raising the visibility of research in Africa.

At the same time, 20% of African researchers carried out most of their research outside the walls of academia and traditional research bodies, usually for international organisations, foreign intergovernmental bodies and NGOs. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 9 From 1996 to 2012, number of research papers in scientific journals with at least one African author quadrupled (12, 500 to 52, 000). For the same period, the share of worlds articles with African authors doubled (1.2% to 2.3%). The continents share of global research has increased from 2.31% in 2010 to 3.24% in 2016 (Green, 2018).

The reasons for emerging on world stage are attributed to: -increased funding; -policy changes; -improved research infrastructure; -ICT resources; -open, free and low-cost access to peer-reviewed literature; -research capacity building and training (Schemm, 2013). Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 10 African research output and global share of articles, December 2013 (Source: Scopus) Carnegie CPD programme Enhancing librarians' ICT skills for research

enablement in African universities 11 And there is still room for improvement!!! HOW? According to Schemm (2013), while African authors have nearly doubled their article share over the past decade, the returns could be many times greater over the next decade if awareness, usage and research capacity are tackled in a collaborative and integrated manner

by African institutions, access programmes and publishers. Above all, with the use of altmetrics, the quantity, quality + impact may increase tremendously beyond the present level. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 12 Traditional methods There are a number of traditional methods that the scientific community uses to measure academic success.

These assessments are tied to the scholarly publishing process and the peer review procedure. Some of the traditional methods to gauge academic success are tied to the journal title itself. The first, and most well-known of these metrics, is the Impact Factor. The Impact Factor is a measure of the frequency with which the "average article" in a journal has been cited in a particular year or period (Thomson Reuters 2013) Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 13 To calculate the Impact Factor (IF), Thomson Reuters averages the number of times articles from the journal in the past two years have been cited in the past year.

This information is available through their Journal Citation Reports (Thomson Reuters 2012). Because it is a standard metric, it is frequently discussed as the only metric to evaluate journal quality. It is important to recognize that the only titles that are included in the Journal Citation Reports and given an IF are titles indexed by the Web of Knowledge database. Not only does this exclude some journal titles, it also excludes book chapters and conference proceedings, which are important scholarly output in some disciplines (Cameron 2005). Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 14 Additional methods of gauging academic success

focus on individual articles rather than the journal in which an article is published Simply counting the number of later articles that cite the original article is one of those metrics. While the metric is not complicated, gathering the data to count the citations is. Both Scopus and Web of Knowledge databases include citation linking, and with that linked data one can create reports of the number of times an article, or all articles by an author, have been cited. As discussed earlier, neither set of data is complete. Google Scholar also includes a cited by count in its display of an individual article. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 15

That information can be collected for a single author by using Google Scholar Citations, which allows an author to create a profile displaying various citation indices. Criticisms of Google Scholars citation counting. The numbers may be inflated if citations to articles are counted twice; for example, if the citing paper is published in a journal and also posted to a researchers website. Google Scholar does cover many more conference papers and book chapters than either Scopus or Web of Knowledge (Jacso 2006). Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

16 The h-index is another method for evaluating the scholarly output of a researcher. This metric does not just evaluate one journal or one article but tries to give a broader view of the impact of a researchers work. The h-index is the number of a scholars papers, h, that have been cited at least h times by other publications (Hirsch 2005). The h-index is frequently represented on a graph with the number of papers along the x axis and the number of citations along the y axis. H is found where the papers equal the number of citations (Figure 2). Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

17 Figure 2: the h-index Carnegie CPD programme Enhancing librarians' ICT skills for research Carnegie CPD universities programme enablement in African Enhancing librarians' ICT skills for research enablement in African universities 19

18 While the metrics described above capture a works influence in a particular discipline, this narrow approach to gauging impact discounts the many new ways in which users engage with scholarly information. As scholars communicate more frequently and more meaningfully with social media tools, the importance of quantifying these interactions is evident. For as long as scholarly output has been measured, scholars have thought about ways to increase their impact. One way to do so is by maintaining an online presence and by active involvement in social media. These forms of engagement offer scholars a method to disseminate their research, increase their professional profile, and communicate their findings with the public. Carnegie CPD programme

Enhancing librarians' ICT skills for research enablement in African universities 19 From Bibliometrics to Altmetrics Whole Internet, cyberspace Informetrics Bibliographies largely references Science of Science

Bibliometrics Cybermetrics Web presence, visibility and impact links, pages, documents Scientometrics Altmetrics Webometrics Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

Alternative metrics views, downloads, web citations, etc Altmetrics: Altmetrics are new metrics proposed as an alternative to the widely used journal impact factor and personal citation indices like the h-index. The term altmetrics was proposed in 2010, as a generalization of article level metrics. (not alternative metrics although many authors now refer to it as such) Although altmetrics are often thought of as metrics about articles, they can be applied to people, journals, books, data sets, presentations, videos, source code repositories, web pages, etc. Carnegie CPD programme

Enhancing librarians' ICT skills for research enablement in African universities 21 Altmetrics: Altmetrics cover not just citation counts, but also other aspects of the impact of a work, such as how many data and knowledge bases refer to it, article views, downloads, or mentions in social media and news media Various websites and projects are calculating altmetrics, including ImpactStory, Altmetric.com, Plum Analytics, and CitedIn. Several publishers have started providing such information to readers / authors, including BioMed Central, Public Library of Science, Frontiers, Nature Publishing Group, and Elsevier. Carnegie CPD programme

Enhancing librarians' ICT skills for research enablement in African universities 22 Advantages of altmetrics There are four potential advantages according to Piwowar (2013): -a more in depth understanding of impact, showing us which scholarly products are read, discussed, saved and recommended as well as cited; -often more timely data, showing evidence of impact in days instead of years; -a window on the impact of web-native scholarly products like datasets, software, blog posts, videos and more; -indications of impacts on diverse audiences including scholars but also practitioners, clinicians, educators and the general public.

Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 23 What is measured? How many times an output article, website, blog, dataset, grey literature, software, etc has been: Viewed (Publisher websites, Dryad) Downloaded (Slideshare, publisher websites, Dryad) Cited (PubMed, CrossRef, Scopus, Wikipedia, DOI, Web of Science) Reused/Adapted (Github)

Shared (Facebook, Twitter) Bookmarked (Mendeley, CiteULike, Delicious) Commented upon (Twitter, Mendeley, blogs, publisher websites, Wikipedia, Faculty of 1000) Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 24 AltMetrics - Who Does It Plum Analytics - Service provider with API. Profile page and analytics. Say they cover more Metric sources than anyone else Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

AltMetrics - Who Does It Impact Story - Displays data in a CV like format. Could become the Linked-In for Scientists. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities Impactstory.org Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 27

Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 28 Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 29 AltMetrics - Who Does It PLOS - One of the first publishers to use article level metrics. One of the current leaders. Metrics i. ii.

iii. iv. Views Cited Saved Discussed Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities AltMetrics - Who Does It Altmetric.com - Service provider for publishers, bibliographic websites and more. also offer bookmarklet. Carnegie CPD programme

Enhancing librarians' ICT skills for research enablement in African universities AltMetrics - Who Does It Institutional repositories An online archive for collecting, preserving, and disseminating digital copies of the intellectual output of an institution, particularly a research institution Metrics - Total views - Country views

- Cities views - Trend of views Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities Controversy / limitations: The usefulness of metrics for estimating impact is controversial, but the community shows a clear need: funders demand measurables on the impact of their spending. Like other metrics, altmetrics are prone to self-citation, gaming, and other mechanisms to boost one's apparent impact. However, it should be kept in mind that the metrics are only one of the outcomes of tracking how research is

used. Even more informative than knowing how often a paper is cited, is which papers are citing it. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 33 Controversy / limitations: Altmetrics can be gamed: for example, likes and mentions can be bought. Altmetrics can be more difficult to standardize than citations. One example is the number of tweets linking to a paper where the number can vary widely depending on how the tweets are collected. Another source of objections against altmetrics, or any metrics, is how universities are using metrics to rank

their employees. The score on each field does not directly tell you anything about the quality or impact of the paper. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 34 Controversy / limitations: Still more valuable sources to cover - classes, seminars, lectures, posters, journal clubs and more. Aggregation is difficult - data in many formats, stored in different ways, in many places, some not accessible (NISO standard being developed). Author names are not standard and multiple copies of media under different names - ORCID, Data One, BioShare

Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities 35 Conclusions Alternative metrics or Altmetrics (besides usage statistics and citation counts) can be used as a value-added service to showcase content impact and give some hints on how this tool can be used to supplement the traditional research performance assessment exercise in an institution. Altmetrics can provide a measure of impact for all non-journal scholarly works available in Open Access (OA), like usage statistics, but it can go further by contextualizing the readership of an authors

research output. Carnegie CPD programme Enhancing librarians' ICT skills for research enablement in African universities

Recently Viewed Presentations

  • Fish 424: Parasitology - University of Idaho

    Fish 424: Parasitology - University of Idaho

    Fish at surface gulping or piping. Suggests parasites on gills. Fish rolling/flashing. suggests protozoan or worm infestation (internal or external) Lethargy or listlessness. Suggests gill parasite. Fish at bottom. Suggests gill parasite, especially "Ich" Fin erosion/Lesions. Indicative of external parasite....
  • Development, Characterization, and Optimization of a Thermoelectric Generator

    Development, Characterization, and Optimization of a Thermoelectric Generator

    Development, Characterization, and Optimization of a Thermoelectric Generator System. Lindsey Bunte, Jonny Hoskins, Tori Johnson, Shane McCauley
  • Folie 1 - Voltimum

    Folie 1 - Voltimum

    Benefits to society e.g. if using the world's resources to profit - give something back. Use less resources to achieve more . Reduce emissions and pollution. Reduce hazardous waste. Summary of any sustainable initiative!A holistic approach . Economic.
  • Lean Six Sigma Reducing Pothole Response Time Bob

    Lean Six Sigma Reducing Pothole Response Time Bob

    Lean Six Sigma Reducing Pothole Response Time Bob Kennedy, Street Department Pothole Formation Team Members Black Belt: Bob Kennedy Champion: Ted Rhinehart Brad Baumgartner - Street Department Jill Morgan, Kim Jones - Data Input Brooks Beatty - Street Department Main...
  • Louis Zukofsky - Yola

    Louis Zukofsky - Yola

    Further, rather than establishing a rhetorical argument by which a reader can be persuaded, Zukofsky creates an enactment of the poem's subject material (an experience of traveling through the confusions of ideological abstraction) wherein the "objects" of poems perception carry...
  • Figure 5.8b Skin appendages: Cutaneous glands. Sebaceous gland

    Figure 5.8b Skin appendages: Cutaneous glands. Sebaceous gland

    Protection (cont.) Biological barriers. Epidermis contains phagocytic cells. Dendritic cells of epidermis engulf foreign antigens (invaders) and present to white blood cells, activating the immune response
  • راهنمای نقد تحقيق پدیده شناسی

    راهنمای نقد تحقيق پدیده شناسی

    The emphasis in phenomenology research is on the meaning of lived experience. (Van Manen, 1984) تاکيد و تمرکز پديدار شناسي بر معني تجارب زنده است. The goal of phenomenology is to describe human experience as it is lived. ... OSCE...
  • Introductions What is Professional Development?  Acquisition of knowledge

    Introductions What is Professional Development? Acquisition of knowledge

    Academic Partner. CEBS is a university-based curriculum, which distinguishes it from other designations. Since CEBS began, the International Foundation of Employee Benefit Plans has partnered with the Wharton School of the University of Pennsylvania - one of the leading business...