Of.SrSSKrSSK7 SSK7 \R"S5r\R"S5r\R"S5r\R"S5r"SS 5r "S S 5r "S S \ 5r g)a CorpusReader for reviews corpora (syntax based on Customer Review Corpus). Customer Review Corpus information ================================== Annotated by: Minqing Hu and Bing Liu, 2004. Department of Computer Science University of Illinois at Chicago Contact: Bing Liu, liub@cs.uic.edu https://www.cs.uic.edu/~liub Distributed with permission. The "product_reviews_1" and "product_reviews_2" datasets respectively contain annotated customer reviews of 5 and 9 products from amazon.com. Related papers: - Minqing Hu and Bing Liu. "Mining and summarizing customer reviews". Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-04), 2004. - Minqing Hu and Bing Liu. "Mining Opinion Features in Customer Reviews". Proceedings of Nineteeth National Conference on Artificial Intelligence (AAAI-2004), 2004. - Xiaowen Ding, Bing Liu and Philip S. Yu. "A Holistic Lexicon-Based Appraoch to Opinion Mining." Proceedings of First ACM International Conference on Web Search and Data Mining (WSDM-2008), Feb 11-12, 2008, Stanford University, Stanford, California, USA. Symbols used in the annotated reviews: :[t]: the title of the review: Each [t] tag starts a review. :xxxx[+|-n]: xxxx is a product feature. :[+n]: Positive opinion, n is the opinion strength: 3 strongest, and 1 weakest. Note that the strength is quite subjective. You may want ignore it, but only considering + and - :[-n]: Negative opinion :##: start of each sentence. Each line is a sentence. :[u]: feature not appeared in the sentence. :[p]: feature not appeared in the sentence. Pronoun resolution is needed. :[s]: suggestion or recommendation. :[cc]: comparison with a competing product from a different brand. :[cs]: comparison with a competing product from the same brand. Note: Some of the files (e.g. "ipod.txt", "Canon PowerShot SD500.txt") do not provide separation between different reviews. This is due to the fact that the dataset was specifically designed for aspect/feature-based sentiment analysis, for which sentence-level annotation is sufficient. For document- level classification and analysis, this peculiarity should be taken into consideration. N)*z ^\[t\](.*)$z%((?:(?:\w+\s)+)?\w+)\[((?:\+|\-)\d)\]z\[(?!t)(p|u|s|cc|cs)\]z##(.*)$c:\rSrSrSrS SjrSrSrSrSr S r g) ReviewMz6 A Review is the main block of a ReviewsCorpusReader. Nc2XlUc/UlgX lg)zt :param title: the title of the review. :param review_lines: the list of the ReviewLines that belong to the Review. N)title review_lines)selfrr s >> from nltk.corpus import product_reviews_1 >>> camera_reviews = product_reviews_1.reviews('Canon_G3.txt') >>> review = camera_reviews[0] >>> review.sents()[0] # doctest: +NORMALIZE_WHITESPACE ['i', 'recently', 'purchased', 'the', 'canon', 'powershot', 'g3', 'and', 'am', 'extremely', 'satisfied', 'with', 'the', 'purchase', '.'] >>> review.features() # doctest: +NORMALIZE_WHITESPACE [('canon powershot g3', '+3'), ('use', '+2'), ('picture', '+2'), ('picture quality', '+1'), ('picture quality', '+1'), ('camera', '+2'), ('use', '+2'), ('feature', '+1'), ('picture quality', '+3'), ('use', '+1'), ('option', '+1')] We can also reach the same information directly from the stream: >>> product_reviews_1.features('Canon_G3.txt') [('canon powershot g3', '+3'), ('use', '+2'), ...] We can compute stats for specific product features: >>> n_reviews = len([(feat,score) for (feat,score) in product_reviews_1.features('Canon_G3.txt') if feat=='picture']) >>> tot = sum([int(score) for (feat,score) in product_reviews_1.features('Canon_G3.txt') if feat=='picture']) >>> mean = tot / n_reviews >>> print(n_reviews, tot, mean) 15 24 1.6 utf8cJ[RXX$5 X0lSUlg)a4 :param root: The root directory for the corpus. :param fileids: a list or regexp specifying the fileids in the corpus. :param word_tokenizer: a tokenizer for breaking sentences or paragraphs into words. Default: `WordPunctTokenizer` :param encoding: the encoding that should be used to read the corpus. z README.txtN) CorpusReaderr _word_tokenizer_readme)r rootfileidsword_tokenizerencodings r r ReviewsCorpusReader.__init__s! d'<-# rNc Uc URnO[U[5(aU/n[UR US5VVs/sHup#UR X R US9PM! snn5$s snnf)a= Return a list of features. Each feature is a tuple made of the specific item feature and the opinion strength about that feature. :param fileids: a list or regexp specifying the ids of the files whose features have to be returned. :return: all features for the item(s) in the given file(s). :rtype: list(tuple) Tr@)_fileidsrstrconcatabspaths CorpusView_read_featuresr r>fileidencs r rReviewsCorpusReader.featuressu ?mmG  % %iG&*]]7D%A %AMV(;(;cJ%A    s&A1 c Uc URn[URUS5VVs/sHup#URX RUS9PM! snn5$s snnf)a# Return all the reviews as a list of Review objects. If `fileids` is specified, return all the reviews from each of the specified files. :param fileids: a list or regexp specifying the ids of the files whose reviews have to be returned. :return: the given file(s) as a list of reviews. TrC)rDrFrGrH_read_review_blockrJs r reviewsReviewsCorpusReader.reviewssa ?mmG&*]]7D%A %AMV(?(?#N%A    s&A c [URUSS5VVVs/sH up#nURX RUS9PM" snnn5$s snnnf)a! Return all sentences in the corpus or in the specified files. :param fileids: a list or regexp specifying the ids of the files whose sentences have to be returned. :return: the given file(s) as a list of sentences, each encoded as a list of word strings. :rtype: list(list(str)) TrC)rFrGrH_read_sent_blockr r>pathrLrKs r rReviewsCorpusReader.sentsX,0==$+M +M'T&;&;cJ+M    'A c [URUSS5VVVs/sH up#nURX RUS9PM" snnn5$s snnnf)a Return all words and punctuation symbols in the corpus or in the specified files. :param fileids: a list or regexp specifying the ids of the files whose words have to be returned. :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) TrC)rFrGrH_read_word_blockrTs r wordsReviewsCorpusReader.wordsrWrXc/n[S5HHnUR5nU(dUs $UR[R"[ U55 MJ U$)N)rangereadlinerrefindallFEATURES)r streamrilines r rI"ReviewsCorpusReader._read_featuressGrA??$D OOBJJx6 7  rcUR5nU(d/$[R"[U5nU(a'[ UR S5R 5S9nOMdUR5nUR5nU(dU/$[R"[U5(aURU5 U/$[R"[U5n[R"[U5n[R"[U5nU(aURRUS5n[XUS9n UR!U 5 M)N)rrr0)r`ramatchTITLErgroupstriptellseekrbrcNOTESSENTr;tokenizerr) r rdrf title_matchreviewoldposfeatsr1rrs r rO&ReviewsCorpusReader._read_review_blocks ??$D ((5$/K%++A.446[[]F??$Dxxxt$$ F#xJJx.EJJud+E::dD)D++44T!W=$$eLK OOK (%rc/nURU5H2nURUR5Vs/sHoDPM sn5 M4 U$s snfr/)rOrr)r rdrrtrs r rS$ReviewsCorpusReader._read_sent_block>sG--f5F LL6<<>:>4$>: ;6 ;s A c/n[S5HdnUR5n[R"[U5nU(dM7UR UR RUS55 Mf U$)Nr^r)r_r`rarbrqrr;rr)r rdr[rerfrs r rZ$ReviewsCorpusReader._read_word_blockDs^rA??$D::dD)Dt T11::47CD   r)r<r;r/)r%r&r'r(r)StreamBackedCorpusViewrHWordPunctTokenizerr rrPrr[rIrOrSrZr*r+rr r6r6sF>(J-?,@6 $ * $ " ")B rr6) r)ranltk.corpus.reader.api nltk.tokenizecompilerkrcrprqrrr:r6r+rr rsw6p $ >" ::,  ,- zz*2 2 j  0q,qr