g:SrSSKrSSKrSSKrSSKJrJrJr SSKJ r SSK J r SSK J r JrJr SSKJr SS KJr SS KJr SS KJr SS KJr SS KJr SSKJr "SS5rg)a Module containing the UniversalDetector detector class, which is the primary class a user of ``chardet`` should use. :author: Mark Pilgrim (initial port to Python) :author: Shy Shalom (original C code) :author: Dan Blanchard (major refactoring for 3.0) :author: Ian Cordasco N)ListOptionalUnion)CharSetGroupProber) CharSetProber) InputStateLanguageFilter ProbingState)EscCharSetProber) Latin1Prober)MacRomanProber)MBCSGroupProber) ResultDict)SBCSGroupProber) UTF1632Proberc f\rSrSrSrSr\R"S5r\R"S5r \R"S5r SSS S S S S SS.r SSSS SSSS.r \ RS4S\ S\SS4Sjjr\S\4Sj5r\S\4Sj5r\S\\4Sj5rS#SjrS\\\4SS4S jrS\4S!jrS"rg)$UniversalDetector8aI The ``UniversalDetector`` class underlies the ``chardet.detect`` function and coordinates all of the different charset probers. To get a ``dict`` containing an encoding and its confidence, you can simply run: .. code:: u = UniversalDetector() u.feed(some_bytes) u.close() detected = u.result g?s[-]s(|~{)s[-]z Windows-1252z Windows-1250z Windows-1251z Windows-1256z Windows-1253z Windows-1255z Windows-1254z Windows-1257) iso-8859-1z iso-8859-2z iso-8859-5z iso-8859-6z iso-8859-7z iso-8859-8 iso-8859-9z iso-8859-13z ISO-8859-11GB18030CP949UTF-16)asciirztis-620rgb2312zeuc-krzutf-16leF lang_filtershould_rename_legacyreturnNcSUlSUl/UlSSSS.UlSUlSUl[ RUlSUl Xl [R"[5UlSUlX lUR#5 g)Nencoding confidencelanguageF)_esc_charset_prober_utf1632_prober_charset_probersresultdone _got_datar PURE_ASCII _input_state _last_charrlogging getLogger__name__logger_has_win_bytesrreset)selfrrs ;/usr/lib/python3/dist-packages/chardet/universaldetector.py__init__UniversalDetector.__init__ds @D 8<57#   &11&''1 #$8! r&cUR$N)r.r6s r7 input_stateUniversalDetector.input_state{s   r&cUR$r;)r4r<s r7 has_win_bytesUniversalDetector.has_win_bytess"""r&cUR$r;)r)r<s r7charset_probers!UniversalDetector.charset_proberss$$$r&cnSSSS.UlSUlSUlSUl[R UlSUlUR(aURR5 UR(aURR5 URHnUR5 M g)z Reset the UniversalDetector and all of its probers back to their initial states. This is called by ``__init__``, so you only need to call this directly in between analyses of different documents. Nr!r"Fr&) r*r+r,r4r r-r.r/r'r5r(r))r6probers r7r5UniversalDetector.resets $(sM  #&11  # #  $ $ * * ,     & & (++F LLN,r&byte_strc UR(agU(dg[U[5(d [U5nUR(GdUR [ R 5(a SSSS.UlOUR [ R[ R45(a SSSS.UlOUR S5(a SSSS.UlOaUR S 5(a S SSS.UlO?UR [ R[ R45(a S SSS.UlS UlURS bS UlgUR[R:XaURR!U5(a[R"Ul O`UR[R:XaBUR$R!UR&U-5(a[R(Ul USSUlUR*(d[-5UlUR*R.[0R2:XalUR*R5U5[0R6:Xa?UR*R8UR*R;5SS.UlS UlgUR[R(:XaUR<(d[?UR@5UlUR<R5U5[0R6:XaSUR<R8UR<R;5UR<RBS.UlS UlggUR[R":XGaLURD(d[GUR@5/Ul"UR@[HRJ-(a#URDRM[O55 URDRM[Q55 URDRM[S55 URDH\nUR5U5[0R6:XdM(UR8UR;5URBS.UlS Ul O URTR!U5(aS Ul+ggg)a Takes a chunk of a document and feeds it through all of the relevant charset probers. After calling ``feed``, you can check the value of the ``done`` attribute to see if you need to continue feeding the ``UniversalDetector`` more data, or if it has made a prediction (in the ``result`` attribute). .. note:: You should always call ``close`` when you're done feeding in your document if ``done`` is not already ``True``. Nz UTF-8-SIG?r"zUTF-32szX-ISO-10646-UCS-4-3412szX-ISO-10646-UCS-4-2143rTr#),r+ isinstance bytearrayr, startswithcodecsBOM_UTF8r* BOM_UTF32_LE BOM_UTF32_BEBOM_LEBOM_BEr.r r-HIGH_BYTE_DETECTORsearch HIGH_BYTE ESC_DETECTORr/ ESC_ASCIIr(rstater DETECTINGfeedFOUND_IT charset_nameget_confidencer'r rr%r)rr NON_CJKappendrr rWIN_BYTE_DETECTORr4)r6rHrFs r7r]UniversalDetector.feeds 99  (I.. *H~~~""6??33!,"% " $$f&9&96;N;N%OPP,43TVW $$%899!9"% "  $$%899!9"% "  $$fmmV]]%CDD,43TVW !DN{{:&2      5 5 5&&--h77$.$8$8!!!Z%:%::%%,,T__x-GHH$.$8$8!"23-###0?D    % %)?)? ?##((2l6K6KK $ 4 4 A A"&"6"6"E"E"G " !     4 4 4+++;D//;;x(L,A,AA$*$7$7&,&;&;&=$*OO#DK !%DI0%%,,X66&*#7#7r&c UR(a UR$SUlUR(dURR S5 GOiUR [ R:Xa SSSS.UlGO>UR [ R:XGaSnSnSnURH'nU(dM UR5nX:dM#UnUnM) U(aX R:aURnUceUR5nUR5nURS 5(a,UR(aUR R#Xe5nUR$(a3UR&R#U=(d SR5U5nUUUR(S.UlURR+5[,R.::aURS cURR S 5 URHnU(dM [1U[25(aUUR4HCnURR S URUR(UR55 ME MvURR S URUR(UR55 M UR$) z Stop analyzing the current document and come up with a final prediction. :returns: The ``result`` attribute, a ``dict`` with the keys `encoding`, `confidence`, and `language`. Tzno data received!rrJrKr"Nr!ziso-8859r#z no probers hit minimum thresholdz%s %s confidence = %s)r+r*r,r3debugr.r r-rXr)r`MINIMUM_THRESHOLDr_lowerrOr4 ISO_WIN_MAPgetr LEGACY_MAPr%getEffectiveLevelr0DEBUGrMrprobers) r6prober_confidencemax_prober_confidence max_proberrFr_lower_charset_namer$ group_probers r7closeUniversalDetector.closes` 99;;  ~~ KK  1 2  *"7"7 7'.crRDK  *"6"6 6 $ $' !J//$*$9$9$;!$<,=)!'J 047M7MM)66 #///%1%7%7%9"'668 &00<<**'+'7'7';';.( ,,#'??#6#6%+224l$L!-", * 3 3  ;; ( ( *gmm ;{{:&. !!"DE$($9$9L' !,0BCC&2&:&:F KK-- 7 & 3 3 & & 5 5 7 '; ))3(55(11(779 %:${{r&) r)r'r,r4r.r/r(r+rr3r*r)rN) r2 __module__ __qualname____firstlineno____doc__rgrecompilerVrYrcrirkr ALLboolr8propertyintr=r@rrrCr5rbytesrNr]rrt__static_attributes__r&r7rr8s1 N3::l+L >2$$$$$$$% K $ $J'5&8&8%*##  .!S!!#t##%m!4%%&A+U5)#34A+A+FMzMr&r)ryrPr0rztypingrrrcharsetgroupproberr charsetproberrenumsr r r escproberr latin1proberr macromanproberrmbcsgroupproberr resultdictrsbcsgroupproberr utf1632proberrrrr&r7rsH8 ((2(;;'&*,",(rrr&