Also known as: Bag of words Model
Модель мешка слов представляет текст через содержащиеся в нем слова, игнорируя грамматику и порядок слов. Каждый документ или фраза обычно кодируется как разреженный вектор счетчиков словаря, индикаторов наличия или взвешенных значений. Этот подход прост и полезен для многих задач классификации текста и retrieval, но теряет информацию о последовательности и контексте.
A representation of the words in a phrase or passage, irrespective of order. For example, bag of words represents the following three phrases identically: - the dog jumps - jumps the dog - dog jumps the Each word is mapped to an index in a sparse vector, where the vector has an index for every word in the vocabulary. For example, the phrase the dog jumps is mapped into a feature vector with non-zero values at the three indexes corresponding to the words the , dog , and jumps. The non-zero value can be any of the following: - A 1 to indicate the presence of a word. - A count of the number of times a word appears in the bag. For example, if the phrase were the maroon dog is a dog with maroon fur , then both maroon and dog would be represented as 2, while the other words would be represented as 1. - Some other value, such as the logarithm of the count of the number of times a word appears in the bag.
A simplifying representation used in natural language processing and information retrieval (IR). In this model, a text (such as a sentence or a document) is represented as the bag (multiset) of its words, disregarding grammar and even word order but keeping multiplicity. The bag-of-words model has also been used for bag-of-words model in computer vision. The bag-of-words model is commonly used in methods of document classification where the (frequency of) occurrence of each word is used as a feature for training a classifier.