OfxZSrSSKrSSKrSSKJr SSKJrJrJr SSK J r "SS\5r g)z{ A reader for corpora that consist of Tweets. It is assumed that the Tweets have been serialised into line-delimited JSON. N) CorpusReader)StreamBackedCorpusViewZipFilePathPointerconcat)TweetTokenizerc\\rSrSrSr\rS\"5S4SjrS Sjr S Sjr S Sjr S r S r g) TwitterCorpusReadera Reader for corpora that consist of Tweets represented as a list of line-delimited JSON. Individual Tweets can be tokenized using the default tokenizer, or by a custom tokenizer specified as a parameter to the constructor. Construct a new Tweet corpus reader for a set of documents located at the given root directory. If you made your own tweet collection in a directory called `twitter-files`, then you can initialise the reader as:: from nltk.corpus import TwitterCorpusReader reader = TwitterCorpusReader(root='/path/to/twitter-files', '.*\.json') However, the recommended approach is to set the relevant directory as the value of the environmental variable `TWITTER`, and then invoke the reader as follows:: root = os.environ['TWITTER'] reader = TwitterCorpusReader(root, '.*\.json') If you want to work directly with the raw Tweets, the `json` library can be used:: import json for tweet in reader.docs(): print(json.dumps(tweet, indent=1, sort_keys=True)) Nutf8c[R"XX$5 URUR5HLn[ U[ 5(aM[ RRU5S:XdM?[SUS35e X0l g)z :param root: The root directory for this corpus. :param fileids: A list or regexp specifying the fileids in this corpus. :param word_tokenizer: Tokenizer for breaking the text of Tweets into smaller units, including but not limited to words. rzFile z is emptyN) r__init__abspaths_fileids isinstancerospathgetsize ValueError_word_tokenizer)selfrootfileidsword_tokenizerencodingrs `_ :return: the given file(s) as a list of dictionaries deserialised from JSON. :rtype: list(dict) T)r)rr CorpusView _read_tweets)rrrencfileids rdocsTwitterCorpusReader.docsNsX,0==$+M +M'T&7&7#F+M    s'A cURU5n/nUHJnUSn[U[5(aURUR5nUR U5 ML U$![ a M]f=f)z{ Returns only the text content of Tweets in the file(s) :return: the given file(s) as a list of Tweets. :rtype: list(str) text)r#rbytesdecoderappendKeyError)rr fulltweetstweetsjsonor&s rstringsTwitterCorpusReader.strings_suYYw' E V}dE**;;t}}5D d#     sAA&& A43A4cURU5nURnUVs/sHoCRU5PM sn$s snf)z :return: the given file(s) as a list of the text content of Tweets as as a list of words, screenanames, hashtags, URLs and punctuation symbols. :rtype: list(list(str)) )r.rtokenize)rrr, tokenizerts r tokenizedTwitterCorpusReader.tokenizedrs>g&(( /56v!""1%v666s?c/n[S5HEnUR5nU(dUs $[R"U5nUR U5 MG U$)zC Assumes that each line in ``stream`` is a JSON-serialised object. )rangereadlinejsonloadsr))rstreamr,ilinetweets rr TwitterCorpusReader._read_tweets}sLrA??$D JJt$E MM%   r)r)N)__name__ __module__ __qualname____firstlineno____doc__rrrr r#r.r4r __static_attributes__rrr r s8>(J !1AF.( "& 7 rr ) rEr:rnltk.corpus.reader.apirnltk.corpus.reader.utilrrr nltk.tokenizerr rGrrrKs,  /VV(s,sr