View source for Bibliometrics

{{Short description|Statistical analysis of written publications}}
{{Use dmy dates|date=April 2022}}
[[File:A synthesized document co-citation network.png|thumb|upright=1.15|An example of a "cocitation network" commonly used in bibliometrics analysis]]
{{Information science}}
{{Research}}

'''Bibliometrics'''  is the application of statistical methods to the study of bibliographic data, especially in scientific and [[library and information science]] contexts, and is closely associated with [[scientometrics]] (the analysis of scientific metrics and indicators) to the point that both fields largely overlap.

Bibliometrics studies first appeared in the late 19th century. They have known a significant development after the [[Second World War]] in a context of "periodical crisis" and new technical opportunities offered by computing tools. In the early 1960s, the [[Science Citation Index]] of [[Eugene Garfield]] and the citation network analysis of [[Derek John de Solla Price]] laid the fundamental basis of a structured research program on bibliometrics.

[[Citation analysis]] is a commonly used bibliometric method based on constructing the [[citation graph]],{{sfn|Hutchins et al.|2019}} a network or graph representation of the citations shared by documents. Many research fields use bibliometric methods to explore the impact of their field, the impact of a set of researchers, the impact of a particular paper, or to identify particularly impactful papers within a specific field of research. Bibliometrics tools have been commonly integrated in [[descriptive linguistics]], the development of [[thesauri]], and evaluation of reader usage. Beyond specialized scientific use, popular web search engines, such as the [[pagerank]] algorithm implemented by Google have been largely shaped by bibliometrics methods and concepts.

The emergence of the Web and the open science movement has gradually transformed the definition and the purpose of "bibliometrics." In the 2010s historical proprietary infrastructures for citation data such as the [[Web of Science]] or [[Scopus]] have been challenged by new initiatives in favor of open citation data. The ''[[Leiden Manifesto for Research Metrics]]'' (2015) opened a wide debate on the use and transparency of metrics.

==Definition==
[[File:Bibliometrics definition.svg|thumb|Definitions of the different fields associated with bibliometrics]]
The term ''bibliométrie'' was first used by [[Paul Otlet]] in 1934,{{sfn|Otlet|1934}}<ref name=ip>{{Cite journal |last=Passas |first=Ioannis |date=June 2024 |title=Bibliometric Analysis: The Main Steps |journal=Encyclopedia |language=en |volume=4 |issue=2 |pages=1014–1025 |doi=10.3390/encyclopedia4020065 |doi-access=free |issn=2673-8392}}</ref> and defined as "the measurement of all aspects related to the publication and reading of books and documents."{{sfn|Rousseau|2014}} The [[Anglicisation|anglicized]] version ''bibliometrics'' was first used by Alan Pritchard in a paper published in 1969, titled "Statistical Bibliography or Bibliometrics?"{{sfn|Pritchard|1969}} He defined the term as "the application of mathematics and statistical methods to books and other media of communication." ''Bibliometrics'' was conceived as a replacement for ''statistical bibliography'', the main label used by publications in the field until then: for Pritchard, statistical bibliography was too "clumsy" and did not make it very clear what was the main object of study.{{sfn|Hertzel|2003|p=288}}

The concept of bibliometrics "stresses the material aspect of the undertaking: counting books, articles, publications, citations".{{sfn|Bellis|2009|p=3}} In theory, bibliometrics is a distinct field from ''scientometrics'' (from the Russian ''naukometriya''),{{sfn|Bellis|2009|p=12}} which relies on the analysis of non-bibliographic indicators of scientific activity. In practice, bibliometrics and scientometrics studies tend to use similar data sources and methods, as citation data has become the leading standard of quantitative scientific evaluation during the mid-20th century: "insofar as bibliometric techniques are applied to scientific and technical literature, the two areas of scientometrics and bibliometrics overlap to a considerable degree."{{sfn|Bellis|2009|p=3}} The development of the web and the expansion of bibliometrics approach to non-scientific production has entailed the introduction of broader labels in the 1990s and the 2000s: infometrics, webometrics or cybermetrics.{{sfn|Bellis|2009|p=4}} These terms have not been extensively adopted, as they partly overlap with pre-existing research practices, such as information retrieval.

==History==
Scientific works, studies and researches that have a bibliometric character can be identified, depending on the definition, already for the 12th century in the form of Jewish indexes.{{sfn|Jovanovic|2012}}

===Early experiments (1880–1914)===
Bibliometric analysis appeared at the turn of the 19th and the 20th century.{{sfn|Bellis|2009|p=2}}{{sfn|Godin|2006}}{{sfn|Danesh|Mardani-Nejad|2020}}{{sfn|Hertzel|2003|p=292}} These developments predate the first occurrence of the concept of ''bibiometrics'' by several decades. Alternative label were commonly used: ''bibliography statistics'' became especially prevalent after 1920 and continued to remain in use until the end of the 1960s.{{sfn|Hertzel|2003|p=292}} Early statistical studies of scientific metadata were motivated by the significant expansion of scientific output and the parallel development of indexing services of databases that made this information more accessible in the first place.{{sfn|Bellis|2009|p=6}} Citation index were first applied to case law in the 1860s and their most famous example, ''[[Shepard's Citations]]'' (first published in 1873) will serve as a direct inspiration for the [[Science Citation Index]] one century later.{{sfn|Bellis|2009|p=23}}

The emergence of [[social sciences]] inspired new speculative research on the ''[[Metascience|science of science]]'' and the possibility of studying science itself as a scientific object: "The belief that social activities, including science, could be reduced to quantitative laws, just as the trajectory of a cannonball and the revolutions of the heavenly bodies, traces back to the positivist sociology of [[Auguste Comte]], [[William Ogburn]], and [[Herbert Spencer]]."{{sfn|Bellis|2009|p=1}} Bibliometric analysis was not conceived as a separate body studies but one of the available methods for the quantitative analysis of scientific activity in different fields of research: [[science history]] (''Histoire des sciences et des savants depuis deux siècles'' of [[Alphonse de Candolle]] in 1885, ''The history of comparative anatomy, a statistical analysis of the literature'' by [[Francis Joseph Cole]] and [[Nellie B. Eales]] in 1917), [[bibliography]] (''The Theory of National and International Bibliography'' of [[Francis Burburry Campbell]] in 1896) or [[Sociology of scientific knowledge|sociology of science]] (''Statistics of American Psychologists'' of [[James McKeen Cattell]] in 1903).

Early bibliometrics and scientometrics work were not simply descriptive but expressed normative views of what science should be and how it could progress. The measurement of the performance of individual researchers, scientific institutions or entire countries was a major objective.{{sfn|Bellis|2009|p=6}} The statistical analysis of James McKeen Cattell acted as a preparatory work for a large scale evaluation of American researchers with [[eugenism|eugenicists]] undertones: ''[[American Men of Science]]'' (1906), "with its astoundingly simplistic rating system of asterisks attached to individual entries in proportion to the estimated eminence of the starred scholar."{{sfn|Bellis|2009|p=2}}

===Development of bibliography statistics (1910–1945)===
[[File:Statistical evolution of scientific literature on anatomy.jpg|thumb|upright=2|An early example of bibliometric analysis of a scientific corpus on [[anatomy]] by [[Francis Joseph Cole]] and [[Nellie B. Eales]] in 1917, with a breakdown by topics and countries]]
After 1910, bibliometrics approach increasingly became the main focus in several study of scientific performance rather than one quantitative method among others.{{sfn|Bellis|2009|p=7}} In 1917, Francis Joseph Cole and Nellie B. Eales argued in favor of the primary statistical value of publications as a publication "is an isolated and definite piece of work, it is permanent, accessible, and may be judged, and in most cases it is not difficult to ascertain when, where, and by whom it was done, and to plot the results on squared paper."{{sfn|Cole|Eales|1917|p=578}} Five years later, [[Edward Wyndham Hulme]] expanded this argument to the point that publications could be considered as the standard measure of an entire civilization: "If civilization is but the product of the human mind operating upon a shifting platform of its environment, we may claim for bibliography that it is not only a pillar in the structure of the edifice, but that it can function as a measure of the varying forces to which this structure is continuously subjected."{{sfn|Hulme|1923|p=43}} This shift toward publication had a limited impact: well until the 1970s, national and international evaluation of scientific activities "disdained bibliometric indicators" which were deemed too simplistic, in favor of socological and economic measures.{{sfn|Bellis|2009|p=14}}

Both the enhanced value attached to scientific publications as a measure of knowledge and the difficulties met by libraries to manage the growing flow of academic periodicals entailed the development of the first citation indexes.{{sfn|Bellis|2009|p=9}} In 1927, P. Gross and E. M. Gross compiled the 3,633 references quoted by the ''Journal of the American Chemical Society'' during the year 1926 and ranked journals depending on their level of citation. The two authors created a set of tools and methods still commonly used by academic search engines, including attributing a bonus to recent citations since "the ''present trend'' rather than the ''past performance'' of a journal should be considered first."{{sfn|Gross|Gross|1927|p=387}} Yet the academic environment measured was markedly different: German rather than English ranked by far the main [[language of science]] of chemistry with more than 50% of all references.{{sfn|Gross|Gross|1927|p=388}}

In the same period, fundamental algorithms, metrics and methods of bibliometrics were first identified in several unrelated projects,{{sfn|Bellis|2009|p=75}} most of them being related to the structural inequalities of scientific production. In [[Alfred Lotka]] introduced its law of productivity from an analysis of the authored publications in the ''[[Chemical Abstracts]]'' and the ''[[Geschichtstafeln der Physik]]'': the number of authors producing an n number of contributions is equal to the 1/n^2 number of authors that only produced one publication.{{sfn|Bellis|2009|p=92}} In, the chief librarian of the [[London Science Museum]], [[Samuel Bradford]] derived a ''[[Bradford's law|law of scattering]]'' from his experience in bibliographic indexing: there are exponentially diminishing returns of searching for references in science journals, as more and more work need to be consulted to find relevant work. Both the Lotka and Bradford law have been criticized as they are far from universal and rather uncovers a rough [[power law]] relationship rendered by deceivingly precise equations.{{sfn|Bellis|2009|p=99}}

===Periodical crisis, digitization and citation index (1945–1960)===
After the [[Second World War]], the growing challenge in managing and accessing scientific publications turned into a full-fledged "periodical crisis": existing journals could not keep up with the rapidly increasing scientific output spurred by the [[big science]] projects.{{sfn|Wouters|1999|p=61}}{{sfn|Bellis|2009|p=12}} The issue became politically relevant after the successful launch of [[Sputnik]] in 1957: "The Sputnik crisis turned the librarians' problem of bibliographic control into a national information crisis.."{{sfn|Wouters|1999|p=62}} In a context of rapid and dramatic change, the emerging field of bibliometrics was linked to large scale reforms of academic publishing and nearly utopian visions of the future of science. In 1934, [[Paul Otlet]] introduced under the concept of ''bibliométrie'' or ''bibliology'' an ambitious project of measuring the impact of texts on society. In contrast with the bounded definition of ''bibliometrics'' that will become prevalent after the 1960s, the vision of Otlet was not limited to scientific publication nor in fact to ''publication'' as a fundamental unit: it aimed for "by the resolution of texts into atomic elements, or ideas, which he located in the single paragraphs (alinéa, verset, articulet) composing a book."{{sfn|Bellis|2009|p=10}} In 1939 [[John Desmond Bernal]] envisioned a network of scientific archives, which was briefly considered by the [[Royal Society]] in 1948: "The scientific paper sent to the central publication office, upon approval by an editorial board of referees, would be microfilmed, and a sort of print-on-demand system set in action thereafter."{{sfn|Bellis|2009|p=52}} While not using the concept of ''bibliometrics'', Bernal had a formative influence of leading figures of the field such as Derek John de Solla Price.

The emerging computing technologies were immediately considered as a potential solution to make a larger amount of scientific output readable and searchable. During the 1950s and 1960s, an uncoordinated wave of experiments in indexing technologies resulted in the rapid development of key concepts of computing research retrieval.{{sfn|Bellis|2009|p=27}} In 1957, IBM engineer [[Hans Peter Luhn]] introduced an influential paradigm of statistical-based analysis of word frequencies, as "communication of ideas by means of words is carried out on the basis of statistical probability."{{sfn|Luhn|1957}} Automated translation of non-English scientific work has also significantly contributed to fundamental research on natural language processing of bibliographic references, as in this period a significant amount of scientific publications [[Languages of Science|were not still available in English]], especially the one coming from the Soviet bloc. Influent members of the [[National Science Foundation]] like [[Joshua Ledeberg]] advocated for the creation of a "centralized information system", [[SCITEL]], partly influenced by the ideas of John Desmond Bernal. This system would at first coexist with printed journals and gradually replace them altogether on account of its efficiency.{{sfn|Wouters|1999|p=60}} In the plan laid out by Ledeberg to Eugen Garfield in November 1961, a centralized deposit would index as much as 1,000,000 scientific articles per year. Beyond full-text searching, the infrastructure would also ensure the indexation of citation and other metadata, as well as the automated translation of foreign language articles.{{sfn|Wouters|1999|p=64}}

The first working prototype on an online retrieval system developed in 1963 by [[Douglas Engelbart|Doug Engelbart]] and Charles Bourne at the Stanford Research Institute proved the feasibility of these theoretical assumptions, although it was heavily constrained by memory issues: no more than 10,000 words of a few documents could be indexed.{{sfn|Bourne|Hahn|2003|p=16}} The early scientific computing infrastructures were focused on more specific research areas, such as [[MEDLINE]] for medicine, NASA/RECON for space engineering or OCLC Worldcat for library search: "most of the earliest online retrieval system provided access to a bibliographic database and the rest used a file containing another sort of information—encyclopedia articles, inventory data, or chemical compounds."{{sfn|Bourne|Hahn|2003|p=12}} Exclusive focus on text analysis proved limitative as the digitized collections expanded: a query could yield a large number results and it was difficult to evaluate the relevancy and the accuracy of the results.{{sfn|Bellis|2009|p=30}}

The ''periodical crisis'' and the limitations of index retrieval technologies motivated the development of bibliometric tools and large citation index like the [[Science Citation Index]] of [[Eugene Garfield]]. Garfield's work was initially primarily concerned with the automated analysis of text work. In contrast with ongoing work largely focused on internal semantic relationship, Garfield highlighted "the importance of metatext in discourse analysis", such as introductory sentences and bibliographic references.{{sfn|Bellis|2009|p=34}} Secondary forms of scientific production like literature reviews and bibliographic notes became central to Garfield's vision as they have already been to [[John Desmond Bernal]]'s vision of scientific archives.{{sfn|Bellis|2009|p=53}} By 1953, Garfield's attention was permanently shifted to citation analysis: in a private letter to [[William C. Adair]], the vice-president of the publisher of the [[Shepard's Citation]] index, "he suggested a well tried solution to the problem of automatic indexing, namely to "shepardize" biomedical literature, to untangle the skein of its content by following the thread of citation links in the same way the legal citator did with court sentences."{{sfn|Bellis|2009|p=35}} In 1955, Garfield published his seminal article "Citation Indexes for Science", that both laid out the outline of the Science Citation Index and had a large influence on the future development of bibliometrics.{{sfn|Bellis|2009|p=35}} The general citation index envisioned by Garfield was originally one of the building block of the ambitious plan of Joshua Lederberg to computerize scientific literature.{{sfn|Bellis|2009|p=36}} Due to lack of funding, the plan was never realized.{{sfn|Bellis|2009|p=37}} In 1963, Eugene Garfield created the [[Institute for Scientific Information]] that aimed to transform the projects initially envisioned with Lederberg into a profitable business.

===Bibliometric reductionism, metrics, and structuration of a research field (1960–1990)===
The field of bibliometrics coalesced in parallel to the development of the Science Citation Index, that was to become its fundamental infrastructure and data resource:{{sfn|Bellis|2009|p=49}} "while the early twentieth century contributed methods that were necessary for measuring research, the mid-twentieth century was characterized by the development of institutions that motivated and facilitated research measurement."{{sfn|Sugimoto|Larivière|2018|p=8}} Significant influences of the nascent field included along with John Desmond Bernal, Paul Otlet the sociology of science of [[Robert K. Merton]], that was re-interpreted in a non-ethic manner: the [[Matthew Effect]], that is the increasing concentration of attention given to researchers that were already notable, was no longer considered ''as a derive''(?) but a feature of normal science.{{sfn|Bellis|2009|p=57}}

A follower of Bernal, the British historian of science [[Derek John de Solla Price]] has had a major impact on the disciplinary formation of bibliometrics: with "the publication of ''Science Since Babylon'' (1961), ''Little Science, Big Science'' (1963), and ''Networks of Scientific Papers'' (1965) by Derek Price, scientometrics already had a sound empirical and conceptual toolkit available."{{sfn|Bellis|2009|p=49}} Price was a proponent of ''bibliometric reductionism''.{{sfn|Bellis|2009|p=62}} As Francis Joseph Cole and Nellie B. Eales in 1917, he argued that a publication is the best possible standard to lay out a quantitative study of science: they "resemble a pile of bricks (…) to remain in perpetuity as an intellectual edifice built by skill and artifice, resting on primitive foundation."{{sfn|Price|1975|p=162}} Price doubled down on this reductionist approach by limiting in turn the large set of existing bibliographic data to citation data.

Price's framework, like Garfield's, takes for granted the structural inequality of science production, as a minority of researchers creates a large share of publication and an even smaller share have a real measurable impact on subsequent research (with as few as 2% of papers having 4 citations or more at the time).{{sfn|Bellis|2009|p=65}} Despite the unprecedented growth of post-war science, Price claimed for the continued existence of an ''invisible college'' of elite scientists that, as in the time of [[Robert Boyle]] undertook the most valuable work.{{sfn|Bellis|2009|p=67}} While Price was aware of the power relationships that ensured the domination of such an elite, there was a fundamental ambiguity in the bibliometrics studies, that highlighted the concentration of academic publishing and prestige but also created tools, models and metrics that normalized pre-existing inequalities.{{sfn|Bellis|2009|p=67}} The central position of the [[Scientific Citation Index]] amplified this performative effect. In the end of the 1960s Eugene Garfield formulated a ''law of concentration'' that was formally a reinterpretation of the [[Samuel Bradford]]'s ''law of scattering'', with a major difference: while Bradford talked for the perspective of a specific research project, Garfield drew a generalization of the law to the entire set of scientific publishing: "the core literature for all scientific disciplines involves a group of no more than 1000 journals, and may involve as few as 500." Such law was also a justification of the practical limitation of the citation index to a limited subset of ''core'' journals, with the underlying assumption that any expansion into second-tier journals would yield diminishing returns.{{sfn|Bellis|2009|p=103}} Rather than simply observing structural trends and patterns, bibliometrics tend to amplify and stratify them even further: "Garfield's citation indexes would have brought to a logical completion, the story of a stratified scientific literature produced by (…) a few, high-quality, "must-buy" international journals owned by a decreasing number of multinational corporations ruling the roost in the global information market."{{sfn|Bellis|2009|p=104}}

Under the impulsion of Garfield and Price, bibliometrics became both a research field and a testing ground for quantitative policy evaluation of research. This second aspect was not a major focus of the Science Citation Index has been a progressive development: the famous [[Impact Factor]] was originally devised in the 1960s by Garfield and Irving Sher to select the core group of journals that were to be featured in ''Current Contents'' and the Science Citation Index and was only regularly published after 1975.{{sfn|Bellis|2009|p=187}} The metric itself is a very simple ratio between the total count of citation received by the journal on the past year and its productivity on the past two years, to ponderate the prolificity of some publications.{{sfn|Bellis|2009|p=186}} For example, [[Nature (journal)|''Nature'']] had an impact factor of 41.577 in 2017:<ref>{{cite report |year=2018 |chapter=Nature |title=2017 Journal Citation Reports |publisher=[[Thomson Reuters]] |edition=Science |series=[[Web of Science]] |url=https://admin.umt.edu.pk/Media/Site/lrc/FileManager/hecrecjournals/JCR_2017.pdf}}{{verify source|date=August 2023|reason=Could not find the value in this report. I added the URL of my best guess of the source.}}</ref>

<math display=block>\text{IF}_{2017} = \frac{\text{Citations}_{2017}}{\text{Publications}_{2016} + \text{Publications}_{2015}} = \frac{74090}{880 + 902} = 41.577.</math>

The simplicity of the impact factor has likely been a major factor in its wide adoption by scientific institutions, journals, funders or evaluators: "none of the revised versions or substitutes of ISI IF has gained general acceptance beyond its proponents, probably because the alleged alternatives lack the degree of interpretability of the original measure."{{sfn|Bellis|2009|p=194}}

Alongside these simplified measurements, Garfield continued to support and fund fundamental research in science history and sociology of science. First published 1964, ''The Use of Citation Data in Writing the History of Science'' compiles several experimental case studies relying on the citation network of the Science Citation Index, including a quantitative reconstruction of the discovery of the DNA.{{sfn|Bellis|2009|p=153}} Interest in this area persisted well after the sell of the Index to Thomson Reuters: as late as 2001, Garfield unveiled ''HistCite'', a software for "algorithmic historiography" created in collaboration with Alexander Pudovkin, and Vladimir S. Istomin.{{sfn|Bellis|2009|p=173}}

===The Web turn (1990–present)===
[[File:Wijt a 1997691 f0005 c.jpg|thumb|upright=1.5|A three-fields plot that shows the relationship of authors between their institutions and cited sources within the retrieved literature. Created with [[List of bibliometrics software|Biblioshiny]] - an online bibliometrics data visualisation tool.]]
The development of the [[World Wide Web]] and the [[Digital Revolution]] had a complex impact on bibliometrics.

The web itself and some of its key components (such as search engines) were partly a product of bibliometrics theory. In its original form, it was derived from a bibliographic scientific infrastructure commissioned to [[Tim Berners-Lee]] by the [[CERN]] for the specific needs of high energy physics, [[ENQUIRE]]. The structure of ENQUIRE was closer to an internal web of data: it connected "nodes" that "could refer to a person, a software module, etc. and that could be interlined with various relations such as made, include, describes and so forth."{{sfn|Hogan|2014|p=20}} Sharing of data and data documentation was a major focus in the initial communication of the World Wide Web when the project was first unveiled in August 1991 : "The WWW project was started to allow high energy physicists to share data, news, and documentation. We are very interested in spreading the web to other areas, and having gateway servers for other data."<ref>Tim Berners-Lee, "[https://www.w3.org/People/Berners-Lee/1991/08/art-6484.txt Qualifiers on Hypertext Links]", mail sent on 6 August 1991 to the alt.hypertext</ref> The web rapidly superseded pre-existing online infrastructure, even when they included more advanced computing features.{{sfn|Star|Ruhleder|1996|p=131}} The core value attached to hyperlinking in the design of the web seem to validate the intuitions of the funding figures of bibliometrics: "The onset of the World Wide Web in the mid-1990s made Garfield's citationist dream more likely to come true. In the world network of hypertexts, not only is the bibliographic reference one of the possible forms taken by a hyperlink inside the electronic version of a scientific article, but the Web itself also exhibits a citation structure, links between web pages being formally similar to bibliographic citations."{{sfn|Bellis|2009|p=285}} Consequently, bibliometrics concepts have been incorporated in major communication technologies the search algorithm of Google: "the citation-driven concept of relevance applied to the network of hyperlinks between web pages would revolutionize the way Web search engines let users quickly pick useful materials out of the anarchical universe of digital information."{{sfn|Bellis|2009|pp=31–32}}

While the web expanded the intellectual influence of bibliometrics way beyond specialized scientific research, it also shattered the core tenets of the field. In contrast with the wide utopian visions of Bernal and Otlet that partly inspired it, the Science Citation Index was always conceived as a closed infrastructure, not only from the perspective of their users but also from the perspective of the collection index: the logical conclusion of Price's theory of ''invisible college'' and Garfield's law of concentration was to focus exclusively on a limited set of core scientific journals. With the rapid expansion of the Web, numerous forms of publications (notably preprints), scientific activities and communities suddenly became visible and highlighted by contrast the limitations of applied bibliometrics.{{sfn|Bellis|2009|p=289}} The other fundamental aspect of bibliometric reductionism, the exclusive focus on citation, has also been increasingly fragilized by the multiplication of alternative data sources and the unprecedented access to full text corpus that made it possible to revive the large scale semantic analysis first envisioned by Garfield in the early 1950s: "Links alone, then, just like bibliographic citations alone, do not seem sufficient to pin down critical communication patterns on the Web, and their statistical analysis will probably follow, in the years to come, the same path of citation analysis, establishing fruitful alliances with other emerging qualitative and quantitative outlooks over the web landscape."{{sfn|Bellis|2009|p=322}}

The close relationship between bibliometrics and commercial vendors of citation data and indicators has become more strained since the 1990s. Leading scientific publishers have diversified their activities beyond publishing and moved "from a content-provision to a data analytics business."{{sfn|Aspesi et al.|2019|p=5}} By 2019, Elsevier has either acquired or built a large portofolio platforms, tools, databases and indicators covering all aspects and stages of scientific research: "the largest supplier of academic journals is also in charge of evaluating and validating research quality and impact (e.g., Pure, Plum Analytics, Sci Val), identifying academic experts for potential employers (e.g., Expert Lookup5), managing the research networking platforms through which to collaborate (e.g., SSRN, Hivebench, Mendeley), managing the tools through which to find funding (e.g., Plum X, Mendeley, Sci Val), and controlling the platforms through which to analyze and store researchers' data (e.g., Hivebench, Mendeley)."{{sfn|Chen et al.|2019|loc=par. 25}} Metrics and indicators are key components of this vertical integration: "Elsevier's further move to offering metrics-based decision making is simultaneously a move to gain further influence in the entirety of the knowledge production process, as well as to further monetize its disproportionate ownership of content."{{sfn|Chen et al.|2019|loc=par. 29}} The new market for scientific publication and scientific data has been compared with the business models of social networks, search engines and other forms of ''platform capitalism''{{sfn|Moore|2019|p=156}}{{sfn|Chen et al.|2019}}{{sfn|Wainwright|Bervejillo|2021}} While content access is free, it is indirectly paid through data extraction and surveillance.{{sfn|Wainwright|Bervejillo|2021|p=211}} In 2020, Rafael Ball envisioned a bleak future for bibliometricians where their research contribute to the emerge of a highly invasive form of "surveillance capitalism":scientists "be given a whole series of scores which not only provide a more comprehensive picture of the academic performance, but also the perception, behaviour, demeanour, appearance and (subjective) credibility (…) In China, this kind of personal data analysis is already being implemented and used simultaneously as an incentive and penalty system."{{sfn|Ball|2020|p=504}}

The ''Leiden manifesto for research metrics'' (2015) highlighted the growing rift between the commercial providers of scientific metrics and bibliometric communities. The signatories stressed the potential social damage of uncontrolled metric-based evaluation and surveillance: "as scientometricians, social scientists and research administrators, we have watched with increasing alarm the pervasive misapplication of indicators to the evaluation of scientific performance."{{sfn|Hicks et al.|2015|p=430}} Several structural reforms of bibliometric research and research evaluation are proposed, including a stronger reliance on qualitative assessment and the reliance on "open, transparent and simple" [[data collection]].{{sfn|Hicks et al.|2015|p=430}} The Leiden Manifesto has stirred an important debate in bibliometrics/scientometrics/infometrics with some critics arguing that the elaboration of quantitative metrics bears no responsibility on their misuse in commercial platforms and research evaluation.{{sfn|David|Frangopol|2015}}

==Usage==
Historically, bibliometric methods have been used to trace relationships amongst [[academic journal]] [[citation]]s. [[Citation analysis]], which involves examining an item's referring documents, is used in searching for materials and analyzing their merit.{{sfn|Schaer|2013}} [[Citation index|Citation indices]], such as [[Institute for Scientific Information]]'s [[Web of Science]], allow users to search forward in time from a known article to more recent publications which cite the known item.<ref name=ip/>

Data from citation indexes can be analyzed to determine the popularity and impact of specific articles, authors, and publications.<ref>{{Cite web|title=Library Guides: Citation & Research Management: Best Practice: Bibliometrics, Citation Analysis |publisher=Berkeley Libraries |url=https://guides.lib.berkeley.edu/citation_management/bibliometrics |access-date=30 May 2020 |archive-url=https://web.archive.org/web/20200727011717/https://guides.lib.berkeley.edu/citation_management/bibliometrics |archive-date=27 July 2020 |url-status=dead}}</ref><ref>{{Cite web |title=Bibliometrics and Citation Analysis: Home |publisher=University of Wisconsin-Madison Libraries |url=https://researchguides.library.wisc.edu/c.php?g=178134&p=1169079 |access-date=30 May 2020 |website=Research Guides}}</ref> Using citation analysis to gauge the importance of one's work, for example, has been common in hiring practices of the late 20th century.<ref>{{cite web |author=Steve Kolowich |date=15 December 2009 |title=Tenure-o-meter |website=Inside Higher Ed |url=http://www.insidehighered.com/news/2009/12/15/tenure}} This article refers to the bibliometrics tool now known as [http://scholarometer.indiana.edu/ Scholarometer].</ref>{{sfn|Hoang|Kaur|Menczer|2010}} Information scientists also use citation analysis to quantitatively assess the core journal titles and watershed publications in particular disciplines; interrelationships between authors from different institutions and schools of thought; and related data about the sociology of academia. Some more pragmatic applications of this information includes the planning of retrospective [[bibliographies]], "giving some indication both of the age of material used in a discipline, and of the extent to which more recent publications supersede the older ones"; indicating through high frequency of citation which documents should be archived; comparing the coverage of secondary services which can help publishers gauge their achievements and competition, and can aid librarians in evaluating "the effectiveness of their stock."{{sfn|Nicholas|Ritchie|1978|pp=12–28}} There are also some limitations to the value of citation data. They are often incomplete or biased; data has been largely collected by hand (which is expensive), though citation indexes can also be used; incorrect citing of sources occurs continually; thus, further investigation is required to truly understand the rationale behind citing to allow it to be confidently applied.{{sfn|Nicholas|Ritchie|1978|pp=28–29}}

Bibliometrics can be used for understanding the research hot topics, for example, in housing Bibliometrics, the results show that Keywords such as influencing factors of housing prices, supply and demand analysis, policy impact on housing prices, and regional city trends are commonly found in housing price research literature. Recent popular keywords include regression analysis and house price predictions. The USA has been a pioneer in housing price research, with well-established means and methods leading the way in this field. Developing countries, on the other hand, need to adopt innovative research approaches and focus more on sustainability in their housing price studies. Research indicates a strong correlation between housing prices and the economy, with keywords like gross domestic product, interest rates, and currency frequently appearing in economic-related cluster analyses.<ref>Li, N. and Li, R.Y.M. (2024), "A bibliometric analysis of six decades of academic research on housing prices", International Journal of Housing Markets and Analysis, Vol. 17 No. 2, pp. 307-328. https://doi.org/10.1108/IJHMA-05-2022-0080</ref>

Bibliometrics are now used in quantitative research assessment exercises of academic output which is starting to threaten practice based research.{{sfn|Henderson|Shurville|Fernstrom|2009}} The UK government has considered using bibliometrics as a possible auxiliary tool in its [[Research Excellence Framework]], a process which will assess the quality of the research output of UK universities and on the basis of the assessment results, allocate research funding.<ref>{{cite web |author=Higher Education Funding Council for England |date=3 July 2009 |title=Research Excellence Framework |url=http://www.hefce.ac.uk/Research/ref/ |access-date=20 July 2009 |website=www.hefce.ac.uk |url-status=dead|archive-url=https://web.archive.org/web/20090704051347/http://www.hefce.ac.uk/Research/ref/ |archive-date=4 July 2009}}</ref> This has met with significant skepticism and, after a pilot study, looks unlikely to replace the current peer review process.<ref>{{Cite web |author=Higher Education Funding Council for England |date=8 July 2015 |title=Metrics cannot replace peer review in the next REF |url=http://www.hefce.ac.uk/news/newsarchive/2015/Name,104464,en.html |website=www.hefce.ac.uk|access-date=20 March 2016|archive-url= https://web.archive.org/web/20180719024409/http://www.hefce.ac.uk/news/newsarchive/2015/Name,104464,en.html |archive-date=19 July 2018|url-status=dead}}</ref> Furthermore, excessive usage of bibliometrics in assessment of value of academic research encourages [[gaming the system]] in various ways including publishing large quantity of works with low new content (see [[least publishable unit]]), publishing premature research to satisfy the numbers, focusing on popularity of the topic rather than scientific value and author's interest, often with detrimental role to research.{{sfn|Biagioli|2020|p=18}} Some of these phenomena are addressed in a number of recent initiatives, including the [[San Francisco Declaration on Research Assessment]].

Guidelines have been written on the using of bibliometrics in academic research, in disciplines such as Management,{{sfn|Linnenluecke|Marrone|Singh|2020}} Education,{{sfn|Diem|Wolter|2012}} and Information Science.{{sfn|Kurtz|Bollen|2010}} Other bibliometrics applications include: creating thesauri; measuring term frequencies; as metrics in [[scientometrics|scientometric]] analysis, exploring [[grammar|grammatical]] and [[syntax|syntactical]] structures of texts; measuring usage by readers; quantifying value of online media of communication; in the context of technological trend analyses;{{sfn|Jovanovic|2020}} measuring Jaccard distance cluster analysis and text mining based on binary logistic regression.{{sfn|Hovden|2013}}{{sfn|Aristovnik|Ravšelj|Umek|2020}}

In the context of the [[big deal (subscription model)|big deal]] cancellations by several library systems in the world,{{sfn|Fernández-Ramos et al.|2019}} data analysis tools like [[Unpaywall Journals]] are used by libraries to assist with big deal cancellations: libraries can avoid subscriptions for materials already served by instant [[open access]] via [[open archive]]s like PubMed Central.<ref>{{cite web |author=Denise Wolfe |date=7 April 2020 |title=SUNY Negotiates New, Modified Agreement with Elsevier |website=Libraries News Center |publisher=[[University at Buffalo]] Libraries |url=https://library.buffalo.edu/news/2020/04/07/suny-negotiates-new-modified-agreement-with-elsevier/ |access-date=18 April 2020}}</ref>


==Bibliometrics and open science==
[[File:When science goes viral - The research response during three months of the COVID-19 outbreak - Fig. 4.jpg|thumb|upright=1.35 |Distribution of corresponding authors of scholarly articles on [[SARS-CoV-2]] and [[COVID-19]] between January and March 2020]]
The [[open science]] movement has been acknowledged as the most important transformation faced by bibliometrics since the emergence of the field in the 1960s.{{sfn|Bellis|2009|p=288 sq.}}{{sfn|Heck|2020}} The free sharing of a wide variety of scientific outputs on the web affected the practice of bibliometrics at all levels: the definition and the collection of the data, infrastructure, and metrics.

Before the crystallization of the field around the Science Citation Index and the reductionist theories of Derek de Solla Price, bibliometrics has been largely influenced by utopian projects of enhanced knowledge sharing beyond specialized academic communities. The scientific networks envisioned by Paul Otlet or John Desmond Bernal have gained a new relevancy with the development of the Web: "The philosophical inspiration of the pioneers in pursuing the above lines of inquiry, however, faded gradually into the background (…) Whereas Bernal's input would eventually find an ideal continuation in the open access movement, the citation machine set into motion by Garfield and Small led to the proliferation of sectorial studies of a fundamentally empirical nature."{{sfn|Bellis|2009|p=336}}

===From altmetrics to open metrics===
{{Further|Altmetrics}}
In the early developments, the open science movement partly co-opted the standard tools of bibliometrics and quantitative evaluation: "the fact that no reference was made to metadata in the main OA declarations (Budapest, Berlin, Bethesda) has led to a paradoxical situation (…) it was through the use of the Web of Science that OA advocates were eager to show how much accessibility led to a citation advantage compared to paywalled articles."{{sfn|Torny|Capelli|Danjean|2019|p=1}} After 2000, an important bibliometric literature was devoted to the citation advantage of open access publications.{{sfn|Sugimoto|Larivière|2018|p=70}}

By the end of the 2000s, the impact factor and other metrics have increasingly held responsible a systemic ''locked-in'' of prestigious non-accessible sources. Key figures of the open science movement like [[Stevan Harnad]] called for the creation of "open access scientometrics" that would take "advantage of the wealth of usage and impact metrics enabled by the multiplication of online, full-text, open access digital archives."{{sfn|Bellis|2009|p=300}} As the public of open science expanded beyond academic circles, new metrics should aim for "measuring the broader societal impacts of scientific research."{{sfn|Wilsdon et al.|2017|p=9}}

The concept of ''alt-metrics'' was introduced in 2009 by [[Cameron Neylon]] and [[Shirly Wu]] as ''article-level metrics''.{{sfn|Neylon|Wu|2009}} In contrast with the focus of leading metrics on journals (impact factor) or, more recently, on individual researchers (h-index), the article-level metrics makes it possible to track the circulation of individual publications: "(an) article that used to live on a shelf now lives in [[Mendeley]], [[CiteULike]], or [[Zotero]] – where we can see and count it"{{sfn|Priem et al.|2011|p=3}} As such they are more compatible with the diversity of publication strategies that has characterized open science: preprints, reports or even non-textual outputs like dataset or software may also have associated metrics.{{sfn|Wilsdon et al.|2017|p=9}} In their original research proposition, Neylon and Wu favored the use of data from [[reference management software]] like Zotero or Mendeley.{{sfn|Neylon|Wu|2009}} The concept of ''altmetrics'' evolved and came to encover data extracted "from social media applications, like blogs, Twitter, ResearchGate and Mendeley.".{{sfn|Wilsdon et al.|2017|p=9}} Social media sources proved especially to be more reliable on a long-term basis, as specialized academic tools like Mendeley came to be integrated into a proprietary ecosystem developed by leading scientific publishers. Major altmetrics indicators that emerged in the 2010s include [[Altmetric.com]], [[PLUMx]] and [[ImpactStory]].

As the meaning of altmetrics shifted, the debate over the positive impact of the metrics evolved toward their redefinition in an open science ecosystem: "Discussions on the misuse of metrics and their interpretation put metrics themselves in the center of open science practices."{{sfn|Heck|2020|p=513}} While altmetrics were initially conceived for open science publications and their expanded circulation beyond academic circles, their compatibility with the emerging requirements for open metrics has been brought into question: social network data, in particular, is far from transparent and readily accessible.{{sfn|Bornmann|Haunschild|2016}}{{sfn|Tunger|Meier|2020}} In 2016, Ulrich Herb published a systematic assessment of the leading publications' metrics in regard to open science principles and concluded that "neither citation-based impact metrics nor alternative metrics can be labeled open metrics. They all lack scientific foundation, transparency and verifiability."{{sfn|Herb|2016|p=60}}

{| class="wikitable" style="margin: auto;"
|+Assessment of leading publication metrics and altmetrics in regard to open science principles{{sfn|Herb|2016}}
|-
! Metric
! Provider
! Sources
! Free<br/>access
! Data<br/>access
! Open<br/>data
! Open<br/>software
|-
| [[Journal Impact Factor]]
| [[Clarivate]]
| Citations ([[Web of Science]])
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| [[SCImago Journal Rank]]
| [[Elsevier]]
| Citations ([[Scopus]])
| style="background:#caedbd"|Yes
| style="background:#caedbd"|Yes
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| [[Source normalized impact per paper|SNIP]]
| Elsevier
| Citations (Scopus)
| style="background:#caedbd"|Yes
| style="background:#caedbd"|Yes
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| [[Eigenfactor]]
| Clarivate
| Citations (Web of Science)
| style="background:#caedbd"|Yes
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| [[Google Journal Ranking]]
| [[Google Scholar]]
| Citations (Google)
| style="background:#caedbd"|Yes
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| [[h-index]]
| Clarivate
| Citations (Web of Science)
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| h-index
| Elsevier
| Citations (Scopus)
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| h-index
| Google Scholar
| Citations (Google)
| style="background:#caedbd"|Yes
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| [[Altmetrics]]
| [[PLUM Analytics]]
| Varied sources
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| Altmetrics
| Altmetric (Macmillan)
| Varied sources
| style="background:#f8e0cb"| Partial
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|style="background:#fecccc"| No
|-
| Altmetrics
| [[PLOS]]
| Varied sources
| style="background:#caedbd"|Yes
| style="background:#caedbd"|Yes
| style="background:#f8e0cb"| Partial (include proprietary data)
| style="background:#caedbd"|Yes
|-
| Altmetrics
| [[ImpactStory]]
| Varied sources
| style="background:#caedbd"|Yes
| style="background:#caedbd"|Yes
| style="background:#f8e0cb"| Partial (include proprietary data)
| style="background:#caedbd"|Yes
|-
| Open Citation Data
| [[OpenCitations#OpenCitations Corpus|OpenCitations Corpus]]
| Varied sources
| style="background:#caedbd"|Yes
| style="background:#caedbd"|Yes
| style="background:#caedbd"|Yes
| style="background:#caedbd"|Yes
|}

Herb laid an alternative program for open metrics that have yet to be developed.{{sfn|Herb|2012|p=29}}{{sfn|Herb|2016|p=70}} The main criteria included:
* A large selection of publication items (journal articles, books, dataset, software) that agree with the writing and reading practices of scientific communities.{{sfn|Herb|2016|p=70}}
* Fully documented data sources.{{sfn|Herb|2016|p=70}}
* Transparent and reproducible process for the calculation of the metrics and other indices.{{sfn|Herb|2016|p=70}}
* Open software.{{sfn|Herb|2016|p=70}}
* Promotion of reflexive and interpretive uses of the metrics, to prevent their misuse in quantitative assessments.{{sfn|Herb|2016|p=70}}

This definition has been implemented in research programs, like ROSI (''Reference implementation for open scientometric indicators'').{{sfn|Hauschke et al.|2018}} In 2017, the European Commission Expert Group on Altmetrics expanded the open metrics program of Ulrich Herb under a new concept, the ''Next-generation metrics''. These metrics should be managed by "open, transparent and linked data infrastructure".{{sfn|Wilsdon et al.|2017|p=15}} The expert group underline that not everything should be measured and not all metrics are relevants: "Measure what matters: the next generation of metrics should begin with those qualities and impacts that European societies most value and need indices for, rather than those which are most easily collected and measure".{{sfn|Wilsdon et al.|2017|p=15}}

===Infrastructure for open citation data===
{{Further|Initiative for Open Citations}}
Until the 2010s, the impact of open science movement was largely limited to scientific publications: it "has tended to overlook the importance of social structures and systemic constraints in the design of new forms of knowledge infrastructures."{{sfn|Okune et al.|2018|p=13}} In 1997, Robert D. Cameron called for the development of an open databases of citation that would completely alter the condition of science communication: "Imagine a universal bibliographic and citation database linking every scholarly work ever written—no matter how published—to every work that cites and every work that cites it. Imagine that such a citation database was freely available over the Internet and was updated every day with all the new works published that day, including papers in traditional and electronic journals, conference papers, theses, technical reports, working papers, and preprints."{{sfn|Cameron|1997}} Despite the development of specific indexes focused on open access works like [[CiteSeer]], a large open alternative to the Science Citation Index failed to materialize. The collection of citation data, remained dominated by large commercial structure such as the direct descendant of the Scientific Citation Index, the [[Web of Science]]. This had the effect of maintaining the emerging ecosystem of open resources at the periphery of academic networks: "common pool of resources is not governed or managed by the current scholarly commons initiative. There is no dedicated hard infrastructure and though there may be a nascent community, there is no formal membership."{{sfn|Bosman et al.|2018|p=19}}

Since 2015, open science infrastructures, platforms and journals have converged to the creation of digital academic commons, increasingly structured around a shared ecosystem of services and standards has emerged through the network of dependencies from one infrastructure to another. This movement stem from an increasingly critical stance toward leading proprietary databases. In 2012, the San Francisco Declaration on Research Assessment (DORA) called for "ending the use of journal impact factors in funding, hiring and promotion decisions."{{sfn|Wilsdon et al.|2017|p=7}} The Leiden Manifesto for research metrics (2015) encouraged the development of "open, transparent and simple" data collection.{{sfn|Hicks et al.|2015|p=430}}

Collaborations between academic and non-academic actors collectively committed in the creation and maintenance of [[knowledge commons]] has been a determining factor in the creation of new infrastructure for open citation data. Since 2010, a dataset of open citation data, the ''Open Citation Corpus'', has been collected by several researchers from a variety of open access sources (including PLOS and Pubmed).{{sfn|Peroni|Dutton|Gray|Shotton|2015}} This collection was the initial kernel of the [[Initiative for OpenCitations]], incepted in 2017 in response to issues of data accessibility faced by a [[Wikimedia]] project, [[Wikidata]]. A conference, given by Dario Taraborelli, head of research at the [[Wikimedia Foundation]] showed that only 1% of papers in [[Crossref]] had citations metadata that were freely available and references stored on Wikidata were unable to include the very large segment of non-free data. This coverage expanded to more than half of the recorded papers, when Elsevier finally joined the initiative in January 2021.<ref name="leid_Q&Aa">{{Cite web |title=Q&A about Elsevier's decision to open its citations |last=Waltman |first=Ludo |work=Leiden Madtrics |publisher=[[Universiteit Leiden]] |date=22 December 2020 |access-date=11 June 2021 |url= https://leidenmadtrics.nl/articles/q-a-about-elseviers-decision-to-open-its-citation }}</ref>

Since 2021, [[OpenAlex]] has become a major [[Open Science Infrastructure|open infrastructure]] for scientific metadata. Initially created as a replacement for the discontinued ''Microsoft Academic Graph'', OpenAlex indexed in 2022 209 millions of scholarly works from 213 millions authors as well as their associated institutions, venues and concepts in a knowledge graph integrated into the semantic web (and [[Wikidata]]).{{sfn|Priem|Piwowar|Orr|2022|p=1-2}} Due to its large coverage and large amount of data properly migrated from the Microsoft Academic Graph (MAG), OpenAlex "seems to be at least as suited for bibliometric analyses as MAG for publication years before 2021."{{sfn|Scheidsteger|Haunschild|2022|p=10}} In 2023, a study on the coverage of [[data journal]]s in scientific indexes found that OpenAlex, along with Dimensions, "enjoy a strong advantage over the two more traditional databases, WoS and Scopus" {{sfn|Jiao|Li|Fang|2023|p=14}} and is overall especially suited for the indexation of non-journal publications like books{{sfn|Laakso|2023|p=166}} or from researchers in non-western countries{{sfn|Akbaritabar|Theile|Zagheni|2023}}

The opening of science data has been a major topic of debate in the bibliometrics and scientometrics community and had wide range social and intellectual consequences. In 2019, the entire scientific board of the ''Journal of Infometrics'' resigned and created a new open access journals, ''Quantitative Science Studies''. The journal was published by [[Elsevier]] since 2007 and the members of the board were increasingly critical of the lack of progress in the open sharing of open citation data: "Our field depends on high-quality scientific metadata. To make our science more robust and reproducible, these data must be as open as possible. Therefore, our editorial board was deeply concerned with the refusal of Elsevier to participate in the Initiative for Open Citations (I4OC)."{{sfn|Waltman et al.|2020|p=1}}

===Bibliometrics without evaluation: the shift to quantitative science studies===
The unprecedented availability of a wide range of scientific productions (publications, data, software, conference, reviews...) has entailed a more dramatic redefinition of the bibliometrics project. For new alternative works anchored in the open science landscape, the principles of bibliometrics as defined by Garfield and Price in the 1960s need to be rethought. The pre-selection of a limited corpus of important journals seem neither necessary nor appropriate. In 2019, the proponents of the Matilda project, "do not want to just "open" the existing closed information, but wish to give back a fair place to the whole academic content that has been excluded from such tools, in a "all texts are born equal" fashion."{{sfn|Torny|Capelli|Danjean|2019|p=2}} They aim to "redefine bibliometrics tools as a technology" by focusing on the exploration and mapping of scientific corpus.{{sfn|Torny|Capelli|Danjean|2019|p=7}}

Issues of inclusivity and more critical approach of structural inequalities in science have become more prevalent in scientometrics and bibliometrics, especially in relation to gender imbalance.{{sfn|Larivière et al.|2013}}{{sfn|Torny|Capelli|Danjean|2019}}{{sfn|Chary|Amrein|Soeteman|Mehta|2021}} After 2020, one of the most heated debate in the field{{sfn|Gingras|2022}} revolved around the reception of a study on the gender imbalance in fundamental physics.{{sfn|Strumia|2021}}

The structural shift in the definition of bibliometrics, scientometrics or infometrics has entailed the need for alternative labels. The concept of ''Quantitative Science Studies'' was originally introduced in the late 2000s in the context of a renewed critical assessment of classic bibliometric findings.{{sfn|Glänzel|2008}} It has become more prevalent in the late 2010s. After leaving [[Elsevier]], the editors of the ''Journal of Infometrics'' opted for this new label and created a journal for ''Quantitative Science Studies''. The first editorial removed all references to metric and aimed for a wider inclusion of quantitative and qualitative research on the science of science:

{{Blockquote|We hope that those who identify under labels such as scientometrics, science of science, and metascience will all find a home in QSS. We also recognize the diverse range of disciplines for whom science is an object of study: We welcome historians of science, philosophers of science, and sociologists of science to our journal. While we bear the moniker of quantitative, we are inclusive of a breadth of epistemological perspectives. Quantitative science studies cannot operate in isolation: Robust empirical work requires the integration of theories and insights from all metasciences.{{sfn|Waltman et al.|2020}}}}

==See also==
* [[Altmetrics]]
* [[Author-level metrics]]
* [[Bibliomining]]
* [[Citation impact]]
* [[Citation graph]]
* [[Hirsch number]] (or ''h''-index)
* [[Impact factor]]
* [[Informetrics]]
* [[Journalology]]
* [[List of bibliometrics software]]
* [[Rankings of academic publishers]]
* [[Webometrics]]

==References==
{{Reflist|30em}}

==Bibliography==

===Books & thesis===
* {{cite book |last1=Bourne |first1=Charles P. |last2=Hahn |first2=Trudi Bellardo |title=A History of Online Information Services, 1963–1976 |date=1 August 2003 |publisher=MIT Press |isbn=978-0-262-26175-3 |url=https://books.google.com/books?id=LTTvmUU8rskC}}
* {{Cite journal| publisher = M. Georg| last = Candolle| first = Alphonse de| title = Histoire des sciences et des savants depuis deux siècles| journal = Lilliad Université de Lille – Sciences et Technologies| location = Genève|access-date= 16 March 2022| date = 1885| url = https://iris.univ-lille.fr/handle/1908/4172}}
* {{Cite book| publisher = Library Bureau| last1 = Campbell| first1 = Francis Bunbury Fitzgerald| last2 = Campbell| first2 = Frank| title = The Theory of National and International Bibliography: With Special Reference to the Introduction of System in the Record of Modern Literature| date = 1896}}
* {{Cite book| publisher = London : Hulme| last = Hulme| first = Edward Wyndham| title = Statistical bibliography in relation to the growth of modern civilization : two lectures delivered in the University of Cambridge in May, 1922|access-date= 16 March 2022| date = 1923| url = http://archive.org/details/statisticalbibli00hulmuoft}}
* {{cite book |last=Otlet |first=Paul |title=Traité de Documentation: Le Livre sur le Livre. Théorie et Pratique |publisher=Editiones Mundaneum |place=Bruxelles |year=1934 |url=https://archive.org/details/OtletTraitDocumentationUgent}}
* {{Cite book| publisher = Yale University Press| isbn = 978-0-300-01798-4| last = Price| first = Derek John de Solla| title = Science Since Babylon| date = 1975}}
* {{cite book |last1=Nicholas|first1=David |last2=Ritchie|first2=Maureen |year=1978 |title=Literature and Bibliometrics |place=London |publisher=Clive Bingley}}
* {{Cite thesis |last=Wouters |first=Paul F. |title=The citation culture |date=1999 |url=https://dare.uva.nl/search?identifier=b101b769-100f-43e5-b8d2-cac6c11e5bbf |degree=PhD Thesis |publisher=Royal Netherlands Academy of Arts and Sciences |place=Amsterdam, Netherlands |format=PDF}}
* {{Cite book| publisher = Institut national de la recherche scientifique, Centre Urbanisation Culture Société| last = Godin| first = Benoît| title = The Making of science, technology and innovation policy: conceptual frameworks as narratives, 1945–2005| location = Montréal|access-date= 16 March 2022| date = 2009| url = https://espace.inrs.ca/id/eprint/9915/}}
* {{Cite book| publisher = Scarecrow Press| isbn = 978-0-8108-6714-7| last = Bellis| first = Nicola De| title = Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics| date = 9 March 2009}}
* {{Cite book| publisher = MIT Press| isbn = 978-0-262-32329-1| last1 = Cronin| first1 = Blaise| author-link = Blaise Cronin | last2 = Sugimoto| first2 = Cassidy R.| author2-link = Cassidy Sugimoto | title = Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact| date = 16 May 2014}}
* {{Cite book| publisher = IOS Press| isbn = 978-1-61499-383-4| last = Hogan| first = A.| title = Reasoning Techniques for the Web of Data| date = 2014-04-09}}
* {{Cite book| publisher = MIT Press| isbn = 978-0-262-33766-3| last = Gingras| first = Yves| title = Bibliometrics and Research Evaluation: Uses and Abuses| date = 30 September 2016}}
* {{Cite book| publisher = Chandos Publishing| isbn = 978-0-08-102151-4| last = Ball| first = Rafael| title = An Introduction to Bibliometrics: New Development and Trends| date = 18 September 2017}}
* {{Cite book| publisher = Oxford University Press| isbn = 978-0-19-064011-8| last1 = Sugimoto| first1 = Cassidy R.| last2 = Larivière| first2 = Vincent| title = Measuring Research: What Everyone Needs to Know| date = 2018}}
* {{Cite thesis| publisher = King's College London| last = Moore| first = Samuel| title = Common Struggles: Policy-based vs. scholar-led approaches to open access in the humanities (thesis deposit)| accessdate = 2021-12-11| date = 2019| url = https://www.samuelmoore.org/2019/05/02/common-struggles-policy-based-vs-scholar-led-approaches-to-open-access-in-the-humanities-thesis-deposit/}}
* {{cite book |last=Biagioli |first=Mario |title=Gaming the metrics: misconduct and manipulation in academic research |publisher=The MIT Press |place=Cambridge, Massachusetts |year=2020 |isbn=978-0-262-53793-3 |oclc=1130310967}}
* {{Cite book| publisher = ENS Éditions| isbn = 979-10-362-0262-9| last = Gozlan| first = Clémentine| title = Les valeurs de la science: Enquête sur les réformes de l'évaluation de la recherche en France| date = 6 April 2021}}

===Journal articles===
* {{Cite journal |doi= 10.2307/1412321| issn = 0002-9556| volume = 14| issue = 3/4| pages = 310–328| last = Cattell| first = J. McKeen| title = Statistics of American Psychologists| journal = The American Journal of Psychology| date = 1903| jstor = 1412321}}
* {{Cite journal| issn = 2059-495X| volume = 11| issue = 44| pages = 578–596| last1 = Cole| first1 = F. J.| last2 = Eales| first2 = Nellie B.| title = The history of comparative anatomy – A statistical analysis of the literature| journal = Science Progress | date = 1917| jstor = 43426882}}
* {{Cite journal| volume = 16| issue = 12| pages = 8| last = Lotka| first = Alfred J.| title = The frequency distribution of scientific productivity| journal = Journal of the Washington Academy of Sciences| date = 1926}}
* {{Cite journal |doi= 10.1126/science.66.1713.385| last1 = Gross| first1 = P. L. K.| last2 = Gross| first2 = E. M.| title = College Libraries and Chemical Education| journal = Science| date = 28 October 1927| volume = 66| issue = 1713| pages = 385–389| pmid = 17782476| bibcode = 1927Sci....66..385G}}
* {{Cite journal |doi= 10.1147/rd.14.0309| issn = 0018-8646| volume = 1| issue = 4| pages = 309–317| last = Luhn| first = H. P.| title = A Statistical Approach to Mechanized Encoding and Searching of Literary Information| journal = IBM Journal of Research and Development| date = October 1957| s2cid = 15879823}}
* {{Cite journal| volume = 25| pages = 348–349| last = Pritchard| first = Alan| title = Statistical Bibliography or Bibliometrics?| journal = Journal of Documentation| date = 1969| url = https://www.academia.edu/598618| access-date = 6 January 2015}}{{Dead link|date=October 2023 |bot=InternetArchiveBot |fix-attempted=yes }}
* {{Cite journal| doi = 10.1287/isre.7.1.111| issn = 1047-7047| volume = 7| issue = 1| pages = 111–134| last1 = Star| first1 = Susan Leigh| last2 = Ruhleder| first2 = Karen| title = Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces| journal = Information Systems Research| accessdate = 2021-12-22| date = 1996-03-01| s2cid = 10520480| url = https://pubsonline.informs.org/doi/abs/10.1287/isre.7.1.111}}
* {{Cite journal |doi= 10.5210/fm.v2i4.522| issn = 1396-0466| last = Cameron| first = Robert D.| title = A Universal Citation Database| journal = First Monday| accessdate = 21 March 2022| date = 7 April 1997| url = https://journals.uic.edu/ojs/index.php/fm/article/view/522| doi-access = free}}
* {{Cite journal |doi= 10.1007/s11192-006-0086-0| issn = 0138-9130 | volume = 68| issue = 1| pages = 109–133| last = Godin| first = Benoît| title = On the origins of bibliometrics| journal = Scientometrics | date = 26 July 2006 | s2cid = 32674188 |url=https://akjournals.com/view/journals/11192/68/1/article-p109.xml|access-date=16 March 2022}}
* {{Cite journal| last1 = Godin| first1 = Benoît| title = From Eugenics to Scientometrics: Galton, Cattell and Men of Science| date = 1 October 2007 |journal=Social Science Information |volume=37 |issue=5 |pages=691–728 |doi=10.1177/0306312706075338 |jstor=25474544| pmid = 18348397| s2cid = 13166578}} {{cite web |title=Original |year=2006 |url=http://www.csiic.ca/PDF/Godin_32.pdf |access-date=26 March 2022}}
* {{Cite journal| last = Godin| first = Benoît| title = The Value of Science: Changing Conceptions of Scientific Productivity, 1869-circa 1970 | date = 17 November 2009 |journal=Social Science Information |volume=48 |issue=4 |pages=547–586 |doi=10.1177/0539018409344475| s2cid = 145245489}} {{cite web |title=Original |year=2006 |url=http://www.csiic.ca/PDF/Godin_34.pdf |access-date=26 March 2022}}
* {{Cite journal| doi = 10.1080/09737766.2008.10700836| issn = 0973-7766| volume = 2| issue = 1| pages = 9–17| last = Glänzel| first = Wolfgang| title = Seven Myths in Bibliometrics About facts and fiction in quantitative science studies| journal = COLLNET Journal of Scientometrics and Information Management| date = 2008-06-01| s2cid = 44206336}}
* {{Cite journal |doi=10.1108/10650740910967348| title=The quantitative crunch| journal=Campus-Wide Information Systems| volume=26| issue=3| pages=149–167| year=2009| last1=Henderson| first1=Michael| last2=Shurville| first2=Simon| last3=Fernstrom| first3=Ken}}
* {{Cite journal| doi = 10.1371/journal.pbio.1000242| issn = 1545-7885| volume = 7| issue = 11| pages = –1000242| last1 = Neylon| first1 = Cameron| last2 = Wu| first2 = Shirley| title = Article-Level Metrics and the Evolution of Scientific Impact| journal = PLOS Biology| date = 2009-11-17| pmid = 19918558| pmc = 2768794| doi-access = free}}
* {{Cite journal |last1=Kurtz |first1=Michael J. |last2=Bollen |first2=Johan |date=2010 |title=Usage bibliometrics |journal=Annual Review of Information Science and Technology |volume=44 |issue=1 |page=3 |doi=10.1002/aris.2010.1440440108 |arxiv=1102.2891 |bibcode=2010ARIST..44....3K |s2cid=484831 |issn=1550-8382}}
* {{cite conference |vauthors=Hoang D, Kaur J, Menczer F |author3-link=Filippo Menczer |date=April 26–27, 2010 |title=Crowdsourcing Scholarly Data |book-title=Proceedings of the WebSci10: Extending the Frontiers of Society On-Line |location=Raleigh, NC |url=http://journal.webscience.org/321/2/websci10_submission_107.pdf |access-date=25 May 2010 |archive-date=8 April 2015 |archive-url=https://web.archive.org/web/20150408222105/http://journal.webscience.org/321/2/websci10_submission_107.pdf |url-status=dead}}
* {{Cite web | ref = {{harvid|Priem et al.|2011}} | pages = 5| last1 = Priem| first1 = Jason| last2 = Taraborelli| first2 = Dario| last3 = Groth| first3 = Paul| last4 = Neylon| first4 = Cameron| title = altmetrics: a manifesto |date = 2011 |publisher=University of Nebraska |location= Lincoln |url=http://altmetrics.org/manifesto/}}
* {{cite journal |last=Jovanovic |first=Milos |date=April 5, 2012 |title=Eine kleine Frühgeschichte der Bibliometrie |trans-title=A short history of early bibliometrics |language=de |journal=Information – Wissenschaft & Praxis |volume=63 |issue=2 |pages=71–80 |s2cid=32450731 |doi=10.1515/iwp-2012-0017 |doi-access=free}}
* {{Cite journal |last1=Diem |first1=Andrea |last2=Wolter |first2=Stefan C. |date=6 June 2012 |title=The Use of Bibliometrics to Measure Research Performance in Education Sciences |journal=Research in Higher Education |volume=54 |issue=1 |pages=86–114 |issn=0361-0365 |s2cid=144986574 |url=http://repec.business.uzh.ch/RePEc/iso/leadinghouse/0066_lhwpaper.pdf |doi=10.1007/s11162-012-9264-5}}
* {{cite journal |doi=10.1002/asi.22936 |arxiv=1303.0766 |title=Bibliometrics for Internet media: Applying the h-index to YouTube |year=2013 |last1=Hovden |first1=R. |journal=Journal of the American Society for Information Science and Technology |volume=64 |issue=11 |pages=2326–2331 |s2cid=38708903}}
* {{cite journal |last=Schaer |first=Philipp |year=2013 |title=Applied Informetrics for Digital Libraries: An Overview of Foundations, Problems and Current Approaches |journal=Historical Social Research |volume=38 |issue=3 |pages=267–281 |doi=10.12759/hsr.38.2013.3.267-281}}
* {{Cite journal| ref={{harvid|Larivière et al.|2013}} |doi= 10.1038/504211a| issn = 1476-4687| volume = 504| issue = 7479| pages = 211–213| last1 = Larivière| first1 = Vincent| last2 = Ni| first2 = Chaoqun| last3 = Gingras| first3 = Yves| last4 = Cronin| first4 = Blaise| last5 = Sugimoto| first5 = Cassidy R.| title = Bibliometrics: Global gender disparities in science| journal = Nature| date = December 2013|pmid= 24350369|s2cid= 38026327| doi-access = free}}
* {{cite journal |last=Rousseau |first=Ronald |title=Library Science: Forgotten Founder of Bibliometrics. |journal=Nature |year=2014 |volume=510 |issue=7504 |pages=218 |doi=10.1038/510218e |pmid=24919911 |bibcode=2014Natur.510..218R |doi-access=free}}
* {{Cite journal |doi= 10.1002/phar.1586| issn = 1875-9114| volume = 35| issue = 6| pages = 551–559| last1 = Thompson| first1 = Dennis F.| last2 = Walker| first2 = Cheri K.| title = A Descriptive and Historical Review of Bibliometrics with Applications to Medical Sciences| journal = Pharmacotherapy | date = 2015| pmid = 25940769| s2cid = 206358632| doi-access = free}}
* {{Cite journal|doi= 10.1108/JD-12-2013-0166| issn = 0022-0418| volume = 71| issue = 2| pages = 253–277| last1 = Peroni| first1 = Silvio| last2 = Dutton| first2 = Alexander| last3 = Gray| first3 = Tanya| last4 = Shotton| first4 = David| title = Setting our bibliographic references free: towards open citation data| journal = Journal of Documentation| date = 2015}}
* {{Cite journal| ref={{harvid|Hicks et al.|2015}} |doi= 10.1038/520429a| issn = 1476-4687| volume = 520| issue = 7548| pages = 429–431| last1 = Hicks| first1 = Diana| last2 = Wouters| first2 = Paul| last3 = Waltman| first3 = Ludo| last4 = de Rijcke| first4 = Sarah| last5 = Rafols| first5 = Ismael| title = Bibliometrics: The Leiden Manifesto for research metrics| journal = Nature| date = April 2015 |pmid= 25903611|bibcode= 2015Natur.520..429H|s2cid= 4462115| doi-access = free| hdl = 10261/132304| hdl-access = free}}
* {{Cite journal |doi= 10.1007/s11192-015-1634-2| issn = 1588-2861| volume = 105| issue = 3| pages = 2255–2257| last1 = David| first1 = Daniel| last2 = Frangopol| first2 = Petre| title = The lost paradise, the original sin, and the Dodo bird: a scientometrics Sapere Aude manifesto as a reply to the Leiden manifesto on scientometrics| journal = Scientometrics| date = 1 December 2015| s2cid = 28104352}}
* {{Cite journal| doi = 10.5281/zenodo.153831| volume = 1| pages = 59–79| last = Herb| first = Ulrich| title = Impactmessung, Transparenz & Open Science| journal = Young Information Scientist| date = 2016-01-01}}
* {{Cite journal |doi= 10.1108/OIR-09-2015-0314| issn = 1468-4527| volume = 40| issue = 4| pages = 529–543| last1 = Bornmann| first1 = Lutz| last2 = Haunschild| first2 = Robin| title = To what extent does the Leiden manifesto also apply to altmetrics? A discussion of the manifesto against the background of research into altmetrics| journal = Online Information Review| date = 2016| url = https://figshare.com/articles/journal_contribution/1464981}}
* {{Cite journal |doi= 10.1017/S0007087417000012| issn = 0007-0874 | volume = 50| issue = 1| pages = 23–60| last = Csiszar| first = Alex| title = How lives became lists and scientific papers became data: cataloguing authorship during the nineteenth century| journal = The British Journal for the History of Science| date = March 2017| pmid = 28202102 | s2cid = 41853820 | doi-access = free}}
* {{Cite book| ref={{harvid|Okune et al.|2018}}| doi=10.4000/proceedings.elpub.2018.31| last1=Chan| first1=Leslie| last2=Posada| first2=Alejandro| last3=Albornoz| first3=Denisse| last4=Hillyer| first4=Rebecca| last5=Okune| first5=Angela | title=22nd International Conference on Electronic Publishing| chapter=Whose Infrastructure? Towards Inclusive and Collaborative Knowledge Infrastructures in Open Science| date=June 2018| s2cid=65726844 |chapter-url=https://elpub.episciences.org/4619/pdf| accessdate=2021-12-22| doi-access=free}}
* {{Cite web| ref={{harvid|Bosman et al.|2018}}| publisher = OSF Preprints| last1 = Bosman| first1 = Jeroen| last2 = Bruno| first2 = Ian| last3 = Chapman| first3 = Chris| last4 = Tzovaras| first4 = Bastian Greshake| last5 = Jacobs| first5 = Nate| last6 = Kramer| first6 = Bianca| last7 = Martone| first7 = Maryann Elizabeth| last8 = Murphy| first8 = Fiona| last9 = O'Donnell| first9 = Daniel Paul| last10 = Bar-Sinai| first10 = Michael| last11 = Hagstrom| first11 = Stephanie| last12 = Utley| first12 = Josh| last13 = Veksler| first13 = Lusia Ludmila| title = The Scholarly Commons – principles and practices to guide research communication| accessdate = 2022-01-07| date = 2017-09-15| url = https://osf.io/6c2xt/}}
* {{Cite journal| ref={{harvid|Hauschke et al.|2018}}| doi = 10.3897/rio.4.e31656| issn = 2367-7163| volume = 4| pages = 31656| last1 = Hauschke| first1 = Christian| last2 = Cartellieri| first2 = Simone| last3 = Heller| first3 = Lambert| title = Reference implementation for open scientometric indicators (ROSI)| journal = Research Ideas and Outcomes| accessdate = 2022-04-24| date = 2018-11-15| s2cid = 70336574| url = https://riojournal.com/article/31656/| doi-access = free}}
* {{Cite conference| publisher = OpenEdition Press|doi= 10.4000/proceedings.elpub.2019.22| conference = ELPUB 2019 23d International Conference on Electronic Publishing| last1 = Torny| first1 = Didier| last2 = Capelli| first2 = Laurent| last3 = Danjean| first3 = Lydie|title= ELPUB 2019 23d International Conference on Electronic Publishing| chapter = Matilda: Building a bibliographic/metric tool for open citations and open science| book-title = ELPUB 2019 23d International Conference on Electronic Publishing| date = 14 June 2019| chapter-url = https://hal.archives-ouvertes.fr/hal-02141839| doi-access = free}}
* {{cite journal |ref={{harvid|Hutchins et al.|2019}} |vauthors=Hutchins BI, Baker KL, Davis MT, Diwersy MA, Haque E, Harriman RM, Hoppe TA, Leicht SA, Meyer P, Santangelo GM |title=The NIH Open Citation Collection: A public access, broad coverage resource. |journal=PLOS Biology |date=October 2019 |volume=17 |issue=10 |at=e3000385 |doi=10.1371/journal.pbio.3000385 |doi-access=free |pmid=31600197 |pmc=6786512}}
* {{cite journal|ref={{harvid|Fernández-Ramos et al.|2019}} |last1=Fernández-Ramos |first1=Andrés |last2=Rodríguez Bravo |first2=María Blanca |last3=Alvite Díez |first3=María Luisa |last4=Santos de Paz |first4=Lourdes |last5=Morán Suárez |first5=María Antonia |last6=Gallego Lorenzo |first6=Josefa |last7=Olea Merino |first7=Isabel |title=Evolución del uso de los big deals en las universidades públicas de Castilla y León |trans-title=Evolution of the big deals use in the public universities of the Castile and Leon region, Spain |language=es |journal=El Profesional de la Información |date=2019 |volume=28 |issue=6 |doi=10.3145/epi.2019.nov.19 |doi-access=free |url=https://buleria.unileon.es/handle/10612/11498|hdl=10612/11498 |hdl-access=free }}
* {{Cite journal| ref={{harvid|Waltman et al.|2020}}|doi= 10.1162/qss_e_00025| issn = 2641-3337| volume = 1| issue = 1| pages = 1–3| last1 = Waltman| first1 = Ludo| last2 = Larivière| first2 = Vincent| last3 = Milojević| first3 = Staša| last4 = Sugimoto| first4 = Cassidy R.| title = Opening science: The rebirth of a scholarly journal| journal = Quantitative Science Studies| date = 1 February 2020|s2cid= 211212402| doi-access = free| hdl = 1866/23210| hdl-access = free}}
* {{Cite journal |last1=Linnenluecke |first1=Martina K. |last2=Marrone |first2=Mauricio |last3=Singh |first3=Abhay K. |date=May 2020 |title=Conducting systematic literature reviews and bibliometric analyses |journal=Australian Journal of Management |volume=45 |issue=2 |pages=175–194 |doi=10.1177/0312896219877678 |s2cid=211378937 |issn=0312-8962|doi-access=free }}
* {{cite journal |vauthors=Aristovnik A, Ravšelj D, Umek L |date=November 2020 |title=A Bibliometric Analysis of COVID-19 across Science and Social Science Research Landscape |journal=Sustainability |volume=12 |issue=21 |at=9132 |doi=10.3390/su12219132 |doi-access=free|bibcode=2020Sust...12.9132A }}
* {{Cite journal | doi = 10.1016/j.geoforum.2020.04.012| issn = 0016-7185| volume = 118| pages = 210–212| last1 = Wainwright| first1 = Joel| last2 = Bervejillo| first2 = Guillermo| title = Leveraging monopoly power up the value chain: Academic publishing in an era of surveillance capitalism| journal = Geoforum| date = January 2021| s2cid = 234328559}}
* {{Cite journal |doi= 10.1162/qss_a_00114| issn = 2641-3337| volume = 2| issue = 1| pages = 225–253| last = Strumia| first = Alessandro| title = Gender issues in fundamental physics: A bibliometric analysis| journal = Quantitative Science Studies| date = 8 April 2021| s2cid = 233874607| doi-access = free}}
* {{Cite journal |last1=Chary |first1=Sowmya |last2=Amrein |first2=Karin |last3=Soeteman |first3=Djøra I. |last4=Mehta |first4=Sangeeta |last5=Christopher |first5=Kenneth B. |date=2021-07-02 |title=Gender disparity in critical care publications: a novel Female First Author Index |journal=Annals of Intensive Care |volume=11 |issue=1 |pages=103 |doi=10.1186/s13613-021-00889-3 |issn=2110-5820 |pmc=8253865 |pmid=34213685 |doi-access=free}}
* {{Cite journal| doi = 10.1162/qss_c_00178| issn = 2641-3337| volume = 3| issue = 1| pages = 315–318| last = Gingras| first = Yves| title = Towards a moralization of bibliometrics? A response to Kyle Siler| journal = Quantitative Science Studies| date = 2022-04-12| s2cid = 248102517| doi-access = free}}
* {{Cite journal| last1 = Akbaritabar| first1 = Aliakbar| last2 = Theile| first2 = Tom| last3 = Zagheni| first3 = Emilio| title = Global flows and rates of international migration of scholars| journal = MPIDR Working Papers| accessdate = 2023-09-07| date = 2023| url = https://ideas.repec.org//p/dem/wpaper/wp-2023-018.html}}
* {{Cite journal |pages=1–19 |arxiv=2307.09704 |last1=Jiao |first1=Chenyue |last2=Li |first2=Kai |last3=Fang |first3=Zhichao |title=How are exclusively data journals indexed in major scholarly databases? An examination of the Web of Science, Scopus, Dimensions, and OpenAlex |journal= |date=2023-07-18}}
* {{Cite journal| doi = 10.1108/JD-02-2023-0016| issn = 0022-0418| volume = 79| issue = 7| pages = 157–177| last = Laakso| first = Mikael| title = Open access books through open data sources: assessing prevalence, providers, and preservation| journal = Journal of Documentation| accessdate = 2023-09-07| date = 2023-01-01| s2cid = 259300771| url = https://doi.org/10.1108/JD-02-2023-0016}}

===Book sections===
* {{Cite book| publisher=CRC Press| isbn=978-0-8247-2077-3| last=Hertzel| first=Dorothy H.| title=Encyclopedia of Library and Information Science |edition=2nd | chapter=Bibliometrics history| date=20 May 2003}}
* {{Cite book| publisher=Kluwer| isbn=978-1-4020-2702-4| pages=19–50| last=Raan| first=Ton| title=Handbook of quantitative science and technology research: the use of publication and patent statistics in studies of S&T systems| chapter=Measuring Science Capita Selecta of current main issues.| location=Dordrecht| date=2004}}
* {{Citation | publisher = universaar| isbn = 978-3-86223-061-7| pages = 11–44| editor = Ulrich Herb | last = Herb| first = Ulrich| title = Open Initiatives : Offenheit in der digitalen Welt und Wissenschaft| chapter = Offenheit und wissenschaftliche Werke: Open Access, Open Review, Open Metrics, Open Science & Open Knowledge| location = Saarbrücken| series = Saarbrücker Schriften zur Informationswissenschaft| date = 2012}}
* {{Cite book| ref={{harvid|Chen et al.|2019}} | publisher = OpenEdition Press| isbn = 979-10-365-3802-5| editor = Pierre Mounier | last1 = Chen| first1 = George| last2 = Posada| first2 = Alejandro| last3 = Chan| first3 = Leslie| title = Connecting the Knowledge Commons – From Projects to Sustainable Infrastructure : The 22nd International Conference on Electronic Publishing – Revised Selected Papers| chapter = Vertical Integration in Academic Publishing : Implications for Knowledge Inequality| location = Marseille| series = Laboratoire d'idées| accessdate = 2022-02-26| date = 2019-06-02| chapter-url = http://books.openedition.org/oep/9068}}
* {{Cite book|ref=none| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| editor-last=Ball| editor-first=Rafael| title=Handbook Bibliometrics| date=7 December 2020 |doi=10.1515/9783110646610| s2cid=243656614}}
* {{Cite book| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| pages=7–18| last1=Danesh| first1=Farshid| editor-last=Ball| editor-first=Rafael| last2=Mardani-Nejad| first2=Ali| title=Handbook Bibliometrics| chapter=1.1 A Historical Overview of Bibliometrics| date=7 December 2020 |doi=10.1515/9783110646610-003| s2cid=235861940}}
* {{Cite book| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| pages=19–26| last=Taubert| first=Niels| editor-last=Ball| editor-first=Rafael| title=Handbook Bibliometrics| chapter=1.2 Institutionalization and Professionalization of Bibliometrics| date=7 December 2020 |doi=10.1515/9783110646610-004| s2cid=235860965}}
* {{Cite book| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| pages=27–40| last=Pendlebury| first=David A.| editor-last=Ball| editor-first=Rafael| title=Handbook Bibliometrics| chapter=1.3 Eugene Garfield and the Institute for Scientific Information| date=7 December 2020 |doi=10.1515/9783110646610-005| s2cid=235861751}}
* {{Cite book|ref=none| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| pages=41–52| last1=Danesh| first1=Farshid| last2=Mardani-Nejad| first2=Ali| editor-last=Ball| editor-first=Rafael| title=Handbook Bibliometrics| chapter=1.4 Derek De Solla Price: The Father of Scientometrics| date=7 December 2020 |doi=10.1515/9783110646610-006| s2cid=235871925}}
* {{Cite book| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| pages=181–190| last1=Tunger| first1=Dirk| last2=Meier| first2=Andreas| editor-last=Ball| editor-first=Rafael| title=Handbook Bibliometrics| chapter=4.1 The Future Has Already Begun: Origin, Classification, and Applications of Altmetrics in Scholarly Communication| date=7 December 2020 |doi=10.1515/9783110646610-019| s2cid=235879114}}
* {{Cite book| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| pages=499–506| last=Ball| first=Rafael| editor-last=Ball| editor-first=Rafael| title=Handbook Bibliometrics| chapter=8.1 The Future of Bibliometrics: Where is Bibliometrics Heading?| date=7 December 2020 |doi=10.1515/9783110646610-045| s2cid=235878741}}
* {{Cite book| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| pages=507–516| last=Heck| first=Tamara| editor-last=Ball| editor-first=Rafael| title=Handbook Bibliometrics| chapter=8.2 Open Science and the Future of Metrics| date=7 December 2020 |doi=10.1515/9783110646610-046| s2cid=235861958}}
* {{Cite book| publisher=De Gruyter Saur| isbn=978-3-11-064661-0| pages=311–318| last=Jovanovic| first=Milos| editor-last=Ball| editor-first=Rafael| title=Handbook Bibliometrics| chapter=5.3 Technological Trend Analysis| date=7 December 2020 |doi=10.1515/9783110646610-031| s2cid=242988619}}

=== Reports ===
* {{Cite report |last1=Wilsdon |first1=James |last2=Allen |first2=Liz |isbn=978-1902369273 |last3=Belfiore |first3=Eleonora |last4=Campbell |first4=Philip |last5=Curry |first5=Stephen |last6=Hill |first6=Steven |last7=Jones |first7=Richard |last8=Kain |first8=Roger |last9=Kerridge |first9=Simon |access-date=11 December 2023 |last10=Thelwall |first10=Mike |last11=Tinkler |first11=Jane |last12=Viney |first12=Ian |last13=Wouters |first13=Paul |last14=Hill |first14=Jude |last15=Johnson |first15=Ben |url=https://kar.kent.ac.uk/81123/1/Metric_Tide_main_report.pdf |title=The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management |date=2015-07-09 |publisher=HEFCE |doi=10.13140/RG.2.1.4929.1363 |publication-date=2015 |pages=176}}
* {{Cite report| ref={{harvid|Wilsdon et al.|2017}}| publisher = European Commission Publications Office| last1 = Wilsdon| first1 = James| last2 = Bar Ilan| first2 = Judit| last3 = Frodeman| first3 = Robert| last4 = Lex| first4 = Elisabeth| last5 = Peters| last6 = Wouters.| first6 = Paul| title = Next-generation metrics: responsible metrics and evaluation for open science.| location = LU| accessdate = 2022-04-24| date = 2017| doi = 10.2777/337729| url = https://data.europa.eu/doi/10.2777/337729}}
* {{Cite report| publisher = Publications Office| last = European Commission. Directorate General for Research and Innovation.| title = Evaluation of research careers fully acknowledging Open Science practices: rewards, incentives and/or recognition for researchers practicing Open Science.| location = LU| accessdate = 2022-04-24| date = 2017| doi = 10.2777/75255| url = https://data.europa.eu/doi/10.2777/75255}}
* {{Cite report |ref={{harvid|Aspesi et al.|2019}} | publisher = LIS Scholarship Archive| last1 = Aspesi| first1 = Claudio| last2 = Allen| first2 = Nicole Starr| last3 = Crow| first3 = Raym| last4 = Daugherty| first4 = Shawn| last5 = Joseph| first5 = Heather| last6 = McArthur| first6 = Joseph| last7 = Shockey| first7 = Nick| title = SPARC Landscape Analysis: The Changing Academic Publishing Industry – Implications for Academic Institutions| accessdate = 2022-01-05| date = 2019-04-03| url = https://osf.io/preprints/lissa/58yhb/}}
* {{Cite report| last1 = Bauer| first1 = Ingrid| last2 = Bohmert| first2 = David| last3 = Czernecka| first3 = Alexandra| last4 = Eichenberger| first4 = Thomas| last5 = Garbajosa| first5 = Juan| last6 = Iovu| first6 = Horia| last7 = Kinnaird| first7 = Yvonne| last8 = Madeira| first8 = Ana Carla| last9 = Nygard| first9 = Mads| last10 = Per-Anders Östling| last11 = Räder| first11 = Susanne| last12 = Ravera| first12 = Mario| last13 = Per-Eric Thörnström| last14 = Wit| first14 = Kurt De| title = Next Generation Metrics| accessdate = 2022-04-24| date = 2020-06-10| url = https://zenodo.org/record/3874801}}

=== Conferences ===
*{{Cite conference| eprint = 2205.01833| conference = STI 2022| last1 = Priem| first1 = Jason| last2 = Piwowar| first2 = Heather| last3 = Orr| first3 = Richard| title = OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts| date = 2022-06-16}}
*{{Cite conference| doi = 10.5281/zenodo.6975102| conference = STI 2022| last1 = Scheidsteger| first1 = Thomas| last2 = Haunschild| first2 = Robin| title = Comparison of metadata with relevance for bibliometrics between Microsoft Academic Graph and OpenAlex until 2020| date = 2022| arxiv = 2206.14168}}

==External links==
*  The [https://www.the-research-hub.org/ Research HUB] published a full-length freely accessible [https://www.youtube.com/playlist?list=PLQ45kJKYgpbmhyO9dIXe2ur2evFRQA5_U&disable_polymer=true/ Bibliometric Analysis video tutorial] playlist.

{{Science and technology studies}}
{{Academic publishing}}

{{Authority control}}

[[Category:Bibliometrics| ]]