Show simple item record

dc.contributor.authorEisenstein, Jacob
dc.date.accessioned2016-08-10T12:39:50Z
dc.date.available2016-08-10T12:39:50Z
dc.date.issued2017
dc.identifier.urihttp://hdl.handle.net/1853/55424
dc.descriptionTo be presented at the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), February 4–9, 2017, San Francisco, California, USA.en_US
dc.description.abstractIn lexicon-based classification, documents are assigned labels by comparing the number of words that appear from two opposed lexicons, such as positive and negative sentiment. Creating such words lists is often easier than labeling instances, and they can be debugged by non-experts if classification performance is unsatisfactory. However, there is little analysis or justification of this classification heuristic. This paper describes a set of assumptions that can be used to derive a probabilistic justification for lexicon-based classification, as well as an analysis of its expected accuracy. One key assumption behind lexicon-based classification is that all words in each lexicon are equally predictive. This is rarely true in practice, which is why lexicon-based approaches are usually outperformed by supervised classifiers that learn distinct weights on each word from labeled instances. This paper shows that it is possible to learn such weights without labeled data, by leveraging co-occurrence statistics across the lexicons. This offers the best of both worlds: light supervision in the form of lexicons, and data-driven classification with higher accuracy than traditional word-counting heuristics.en_US
dc.publisherGeorgia Institute of Technologyen_US
dc.subjectDocument classificationen_US
dc.subjectLexicon-based classificationen_US
dc.subjectLexiconsen_US
dc.titleUnsupervised Learning for Lexicon-Based Classificationen_US
dc.typePre-printen_US
dc.typeProceedingsen_US
dc.contributor.corporatenameGeorgia Institute of Technology. School of Interactive Computingen_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record