View source for Explicit semantic analysis

In [[natural language processing]] and [[information retrieval]], '''explicit semantic analysis''' ('''ESA''') is a [[Vector space model|vectoral]] representation of text (individual words or entire documents) that uses a document corpus as a [[knowledge base]]. Specifically, in ESA, a word is represented as a column vector in the [[tf*idf|tf–idf]] matrix of the text corpus and a document (string of words) is represented as the [[centroid]] of the vectors representing its words. Typically, the text corpus is [[English Wikipedia]], though other corpora including the [[Open Directory Project]] have been used.<ref name="tois">{{cite journal|last1=Egozi|first1=Ofer|last2=Markovitch|first2=Shaul|last3=Gabrilovich|first3=Evgeniy|year=2011|title=Concept-Based Information Retrieval using Explicit Semantic Analysis|url=http://www.cs.technion.ac.il/~shaulm/papers/pdf/Egozi-Gabrilovich-Markovitch-TOIS2011.pdf|journal=ACM Transactions on Information Systems|volume=29|issue=2|pages=1–34|doi=10.1145/1961209.1961211|s2cid=743663|access-date=January 3, 2015}}</ref>

ESA was designed by [[Evgeniy Gabrilovich]] and Shaul Markovitch as a means of improving [[document classification|text categorization]]<ref>{{cite conference |first1=Evgeniy |last1=Gabrilovich |first2=Shaul |last2=Markovitch |title=Overcoming the brittleness bottleneck using Wikipedia: enhancing text categorization with encyclopedic knowledge |conference=Proc. 21st National Conference on Artificial Intelligence (AAAI) |pages=1301–1306 |year=2006 |url=http://www.aaai.org/Papers/AAAI/2006/AAAI06-204.pdf}}</ref>
and has been used by this pair of researchers to compute what they refer to as "[[Semantics|semantic]] relatedness" by means of [[cosine similarity]] between the aforementioned vectors, collectively interpreted as a space of "concepts explicitly defined and described by humans", where Wikipedia articles (or ODP entries, or otherwise titles of documents in the knowledge base corpus) are equated with concepts. The name "explicit semantic analysis" contrasts with [[latent semantic analysis]] (LSA), because the use of a knowledge base makes it possible to assign human-readable labels to the concepts that make up the vector space.{{r|tois}}<ref name="ijcai">{{cite conference |first1=Evgeniy |last1=Gabrilovich |first2=Shaul |last2=Markovitch |title=Computing semantic relatedness using Wikipedia-based Explicit Semantic Analysis |conference=Proc. 20th Int'l Joint Conf. on Artificial Intelligence (IJCAI) |pages=1606–1611 |year=2007 |url=https://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf}}</ref>

==Model==
To perform the basic variant of ESA, one starts with a collection of texts, say, all Wikipedia articles; let the number of documents in the collection be {{mvar|N}}. These are all turned into "[[Bag of words assumption|bags of words]]", i.e., term frequency histograms, stored in an [[inverted index]]. Using this inverted index, one can find for any word the set of Wikipedia articles containing this word; in the vocabulary of Egozi, Markovitch and Gabrilovitch, "each word appearing in the Wikipedia corpus can be seen as triggering each of the concepts it points to in the inverted index."{{r|tois}}

The output of the inverted index for a single word query is a list of indexed documents (Wikipedia articles), each given a score depending on how often the word in question occurred in them (weighted by the total number of words in the document). Mathematically, this list is an {{mvar|N}}-dimensional vector of word-document scores, where a document not containing the query word has score zero. To compute the relatedness of two words, one compares the vectors (say {{math|'''u'''}} and {{math|'''v'''}}) by computing the cosine similarity,

:<math>\mathsf{sim}(\mathbf{u}, \mathbf{v})
    = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\| \|\mathbf{v}\|}
    = \frac{\sum_{i=1}^N u_i v_i}{\sqrt{\sum_{i=1}^N u_i^2}\sqrt{\sum_{i=1}^N v_i^2}}
</math>

and this gives a numeric estimate of the semantic relatedness of the words. The scheme is extended from single words to multi-word texts by simply summing the vectors of all words in the text.{{r|ijcai}}

==Analysis==
ESA, as originally posited by Gabrilovich and Markovitch, operates under the assumption that the knowledge base contains topically [[Orthogonality|orthogonal]] concepts. However, it was later shown by Anderka and Stein that ESA also improves the performance of [[information retrieval]] systems when it is based not on Wikipedia, but on the [[List of datasets for machine-learning research#News articles|Reuters corpus of newswire articles]], which does not satisfy the orthogonality property; in their experiments, Anderka and Stein used newswire stories as "concepts".<ref>Maik Anderka and Benno Stein. [http://www.uni-weimar.de/medien/webis/publications/papers/stein_2009c.pdf The ESA retrieval model revisited] {{Webarchive|url=https://web.archive.org/web/20120610024831/http://www.uni-weimar.de/medien/webis/publications/papers/stein_2009c.pdf |date=2012-06-10 }}. Proceedings of the 32nd International ACM Conference on Research and Development in Information Retrieval (SIGIR), pp. 670-671, 2009.</ref> To explain this observation, links have been shown between ESA and the [[generalized vector space model]].<ref>Thomas Gottron, Maik Anderka and Benno Stein. [http://www.uni-weimar.de/medien/webis/publications/papers/stein_2011o.pdf Insights into explicit semantic analysis] {{Webarchive|url=https://web.archive.org/web/20120610024853/http://www.uni-weimar.de/medien/webis/publications/papers/stein_2011o.pdf |date=2012-06-10 }}. Proceedings of the 20th ACM International Conference on Information and Knowledge Management (CIKM), pp. 1961-1964, 2011.</ref> Gabrilovich and Markovitch replied to Anderka and Stein by pointing out that their experimental result was achieved using "a single application of ESA (text similarity)" and "just a single, extremely small and homogenous test collection of 50 news documents".{{r|tois}}

==Applications==
=== Word relatedness ===
ESA is considered by its authors a measure of semantic relatedness (as opposed to [[semantic similarity]]). On datasets used to benchmark relatedness of words, ESA outperforms other algorithms, including [[WordNet]] semantic similarity measures and skip-gram Neural Network Language Model ([[Word2vec]]).<ref>Kliegr, Tomáš, and Ondřej Zamazal. [https://www.sciencedirect.com/science/article/pii/S0169023X17301325 Antonyms are similar: Towards paradigmatic association approach to rating similarity in SimLex-999 and WordSim-353.] Data & Knowledge Engineering 115 (2018): 174-193. (source may be paywalled, [https://www.researchgate.net/publication/324459370_Antonyms_are_similar_Towards_paradigmatic_association_approach_to_rating_similarity_in_SimLex-999_and_WordSim-353 mirror])</ref>

===Document relatedness===
ESA is used in commercial software packages for computing relatedness of documents.<ref>{{cite web|url=https://blogs.oracle.com/r/explicit-semantic-analysis-esa-for-text-analytics|access-date=31 March 2023|website=blogs.oracle.com|title=Explicit Semantic Analysis (ESA) for Text Analytics|date=November 17, 2017|author=Marc Hornick}}</ref> Domain-specific restrictions on the ESA model are sometimes used to provide more robust document matching.<ref>Luca Mazzola, Patrick Siegfried, Andreas Waldis, Michael Kaufmann, Alexander Denzler. [https://doi.org/10.1109/IS.2018.8710507 A Domain Specific ESA Inspired Approach for Document Semantic Description]. Proceedings of the 9th IEEE Conf. on Intelligent Systems 2018 (IS), pp. 383-390, 2018.</ref>

==Extensions==
Cross-language explicit semantic analysis (CL-ESA) is a multilingual generalization of ESA.<ref>Martin Potthast, Benno Stein, and Maik Anderka. [http://www.uni-weimar.de/medien/webis/publications/papers/stein_2008b.pdf A Wikipedia-based multilingual retrieval model] {{Webarchive|url=https://web.archive.org/web/20120610024823/http://www.uni-weimar.de/medien/webis/publications/papers/stein_2008b.pdf |date=2012-06-10 }}. Proceedings of the 30th European Conference on IR Research (ECIR), pp. 522-530, 2008.</ref> CL-ESA exploits a document-aligned multilingual reference collection (e.g., again, Wikipedia) to represent a document as a language-independent concept vector. The relatedness of two documents in different languages is assessed by the cosine similarity between the corresponding vector representations.

== See also ==
* [[Topic model]]

== References ==
{{Reflist|30em}}

== External links ==
* [https://gabrilovich.com/resources/code/esa/esa.html Explicit semantic analysis] on Evgeniy Gabrilovich's homepage; has links to implementations

{{Natural language processing}}

[[Category:Natural language processing]]
[[Category:Vector space model]]