Of{SrSSKrSSKrSSKJr SSKJr SSKJr SSK J r SSK J r "SS 5r SS jr"S S \5rS rSSjrSrSrSrSrSrSr\S:Xa\"5 gg)z Tools for reading and writing dependency trees. The input is assumed to be in Malt-TAB format (https://stp.lingfil.uu.se/~nivre/research/MaltXML.html). N) defaultdict)chain)pformat) find_binary)Treec\rSrSrSrSSjrSrSrSrSr S r S r S r S r S rSr\S Sj5rSrSrSrS!SjrS"SjrSrSrS#SjrSrSrSrSrSrSrSr g)$DependencyGraphzI A container for the nodes and labelled edges of a dependency structure. Nc[S5UlURSRSSSS.5 SUlU(aUR UUUUUS9 gg)aTDependency graph. We place a dummy `TOP` node with the index 0, since the root node is often assigned 0 as its head. This also means that the indexing of the nodes corresponds directly to the Malt-TAB format, which starts at 1. If zero-based is True, then Malt-TAB-like input with node numbers starting at 0 and the root node assigned -1 (as produced by, e.g., zpar). :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. c 4SSSSSSS[[5SS. $)N) addresswordlemmactagtagfeatsheaddepsrel)rlist*DependencyGraph.__init__..=s'#D) rrTOP)rrr N)cell_extractor zero_basedcell_separatortop_relation_label)rnodesupdateroot_parse)selftree_strrrrr s r__init__DependencyGraph.__init__$sc0!     1 eEaHI  KK-%-#5   rcURU g)z_ Removes the node with the given address. References to this node in others will still exist. Nr!)r%r s rremove_by_address!DependencyGraph.remove_by_addressWs JJw rcURR5H>n/nUSH,nXQ;aURU5 MURU5 M. XCS'M@ g)zX Redirects arcs to any of the nodes in the originals list to the redirect node address. rN)r!valuesappend)r% originalsredirectnodenew_depsdeps r redirect_arcsDependencyGraph.redirect_arcs^sU JJ%%'DHF|#OOH-OOC( $ $L(rcURUSnURUSRU/5 URUSURU5 g)z_ Adds an arc from the node specified by head_address to the node specified by the mod address. rrN)r! setdefaultr/)r% head_address mod_addressrelations radd_arcDependencyGraph.add_arclsV ::k*51 < (33HbA < (299+Frc"URR5HqnURR5HPnUSUS:wdMUSS:wdMUSnUSRU/5 USURUS5 MR Ms g)zZ Fully connects all non-root nodes. All nodes are set to be dependents of the root node. r rrrN)r!r.r8r/)r%node1node2r;s r connect_graphDependencyGraph.connect_graphvs ZZ&&(E**,#uY'77E%LE>> dg = DependencyGraph( ... 'John N 2\n' ... 'loves V 0\n' ... 'Mary N 2' ... ) >>> print(dg.to_dot()) digraph G{ edge [dir=forward] node [shape=plaintext] 0 [label="0 (None)"] 0 -> 2 [label="ROOT"] 1 [label="1 (John)"] 2 [label="2 (loves)"] 2 -> 1 [label=""] 2 -> 3 [label=""] 3 [label="3 (Mary)"] } z digraph G{ zedge [dir=forward] znode [shape=plaintext] c US$Nr r)vs rr(DependencyGraph.to_dot..sa lr)keyz {} [label="{} ({})"]r rrz {} -> {} [label="{}"]z {} -> {} z })sortedr!r.formatitems)r%sr2rrr4s rto_dotDependencyGraph.to_dots0  ## ''4::,,.4JKD )00YYV  A "&\//1 C6==d9osXX]11$y/3GG  2 L U rc8UR5n[U5$)a@Show SVG representation of the transducer (IPython magic). >>> from nltk.test.setup_fixt import check_binary >>> check_binary('dot') >>> dg = DependencyGraph( ... 'John N 2\n' ... 'loves V 0\n' ... 'Mary N 2' ... ) >>> dg._repr_svg_().split('\n')[0] '' )rUdot2img)r% dot_strings r _repr_svg_DependencyGraph._repr_svg_s[[] z""rc,[UR5$N)rr!r%s r__str__DependencyGraph.__str__stzz""rc4S[UR5S3$)Nz)lenr!r^s r__repr__DependencyGraph.__repr__s'DJJ'8@@rc [U5nUR5RS5Vs/sHn[UUUUS9PM snsSSS5 $s snf!,(df  g=f)a :param filename: a name of a file in Malt-TAB format :param zero_based: nodes in the input file are numbered starting from 0 rather than 1 (as produced by, e.g., zpar) :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. :return: a list of DependencyGraphs  )rrr N)openreadsplitr )filenamerrr infiler&s rloadDependencyGraph.loadsd (^v!' 3 3F ;!# UHoT:dM Sv M g7fNr.0cindexs r 0DependencyGraph.left_children..4he)11h  r from_iterabler!r.sumr% node_indexchildrenrus @r left_childrenDependencyGraph.left_childrenP &&tzz*'=f'E'L'L'NO :&y14h444rc^[R"URUSR55nURUSm[ U4SjU55$)zU Returns the number of right children under the node specified by the given address. rr c36># UHoT:dM Sv M g7frprrrs rrv1DependencyGraph.right_children..rxryrzr}s @rright_childrenDependencyGraph.right_childrenrrczURUS5(d"URUSRU5 ggrM)rIr!r")r%r2s radd_nodeDependencyGraph.add_nodes7$$T)_55 JJtI ' . .t 46rc SnSnSnSn UUUU S.n [U[5(aSURS55nSU5n S U 5n S n [U S S 9HupURU5nU c [ U5n OU [ U5:XdeUcXnU"X5un nnnnnnnUS:XaMX[U5nU(aUS - nURU RU UUUUUUUS.5 U S:XaUS:XaUnURUSURU 5 M URSSU(a3URSSUSnURUUl XPlg [R "S5 g ![ a n[ S RU 55UeS nAff=f![[ 4a U"U5unnnnnnnGN0f=f)a}Parse a sentence. :param extractor: a function that given a tuple of cells returns a 7-tuple, where the values are ``word, lemma, ctag, tag, feats, head, rel``. :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. cUup#nXX#USUS4$Nr)cellsrurrrs rextract_3_cells/DependencyGraph._parse..extract_3_cellss#ODt3D"< .extract_4_cellss#( Dt3D= =rcVUup#pEpgn[U5nXXEUSXx4$![a Nf=frint ValueError) rru line_indexrrr_rrs rextract_7_cells/DependencyGraph._parse..extract_7_cells sE9> 6Je!3 JCT> >  s  ((c XUu p#pEpgp n [U5nXXEXgX4$![a Nf=fr]r) rrurrrrrrrrrs rextract_10_cells0DependencyGraph._parse..extract_10_cells)sGIN FJe3t!Q JSB B  s  ))) c3$# UHov M g7fr]r)rslines rrv)DependencyGraph._parse..:s:'9td'9s c3@# UHoR5v M g7fr])rstriprsls rrvr<s,VVsc36# UHo(dM Uv M g7fr]rrs rrvr=s'EqQEs  Nrq)startTNumber of tab-delimited fields ({}) not supported by CoNLL(10) or Malt-Tab(4) formatr)r rrrrrrrrrrzBThe graph doesn't contain a node that depends on the root element.) isinstancestrri enumeraterbKeyErrorrrR TypeErrorrr!r"r/r#r warningswarn)r%input_rrrr rrrr extractorslines cell_numberrurrerrrrrrr root_addresss rr$DependencyGraph._parses/, = > ? C   fc " ":v||D'9:F,V,'E' $U!4KEJJ~.E"!%j "c%j000%%/% gN 7f)z[ Extract dependency triples of the form: ((head word, head tag), rel, (dep word, dep tag)) rrrr)r2N)r#rQrr{r.rFtriples)r%r2rrr4s rrDependencyGraph.tripless 99DV d6l+++DL,?,?,ABCA%%a(CUc&k3v;%?@ @|||- - -D .sBBB BcHURUS$![a gf=f)Nrr! IndexErrorr%rs r_hdDependencyGraph._hds, ::a=( (    !!cHURUS$![a gf=f)Nrrrs r_relDependencyGraph._rels, ::a=' '  rc0nURR5H#nUSHn[USU/5nSX'M M% URHn0nUH8nUH/nUSUS:XdM[USUS/5nXX-Xd'M1 M: UHBn XiX'U SU S:XdMURUR U S5U S5n U s s $ M g)aCheck whether there are cycles. >>> dg = DependencyGraph(treebank_data) >>> dg.contains_cycle() False >>> cyclic_dg = DependencyGraph() >>> top = {'word': None, 'deps': [1], 'rel': 'TOP', 'address': 0} >>> child1 = {'word': None, 'deps': [2], 'rel': 'NTOP', 'address': 1} >>> child2 = {'word': None, 'deps': [4], 'rel': 'NTOP', 'address': 2} >>> child3 = {'word': None, 'deps': [1], 'rel': 'NTOP', 'address': 3} >>> child4 = {'word': None, 'deps': [3], 'rel': 'NTOP', 'address': 4} >>> cyclic_dg.nodes = { ... 0: top, ... 1: child1, ... 2: child2, ... 3: child3, ... 4: child4, ... } >>> cyclic_dg.root = top >>> cyclic_dg.contains_cycle() [1, 2, 4, 3] rr rqrF)r!r.tupleget_cycle_pathrF) r% distancesr2r4rPr new_entriespair1pair2pairpaths rcontains_cycleDependencyGraph.contains_cycles4 JJ%%'DF|T)_c23!" $( AK"&EQx58+#U1XuQx$89+4+;i>N+N ('# $"-"3 7d1g%..t/B/B47/KTRSWUDK $rcUSHnX2:XdM US/s $ USHLnURURU5U5n[U5S:dM5URSUS5 Us $ /$)Nrr r)rrFrbinsert)r% curr_nodegoal_node_indexr4rs rrDependencyGraph.get_cycle_paths}V$C%!),--%V$C&&t':':3'?QD4y1} Ay34 %  rc^US:XaSmO,US:XaSmO#US:XaSmO[SRU55eSRU4S j[URR 5555$) z The dependency graph in CoNLL format. :param style: the style to use for the format (3, 4, 10 columns) :type style: int :rtype: str rz{word} {tag} {head} rz{word} {tag} {head} {rel} rz9{i} {word} {lemma} {ctag} {tag} {feats} {head} {rel} _ _ rrc3f># UH&upUSS:wdMTR"SSU0UD6v M( g7f)rrrNr)rR)rsrr2templates rrv+DependencyGraph.to_conll.. s9 5E{e# )HOO (a (4 (5s11)rrRjoinrQr!rS)r%stylers @rto_conllDependencyGraph.to_conlls{ A:0H aZ7H b[U 228&-  ww !$**"2"2"45   rcSSKn[[S[UR555nUVs/sH<o0R U5(dMX0R U5UR U54PM> nn0UlUH"nURUSURU'M$ UR5nURU5 URU5 U$s snf)zJConvert the data in a ``nodelist`` into a networkx labeled directed graph.rNrqr) networkxrrangerbr!rr nx_labels MultiDiGraphadd_nodes_fromadd_edges_from)r%r nx_nodelistn nx_edgelistgs rnx_graphDependencyGraph.nx_graphs5C O45 4? 4?q88A; *Q TYYq\ *K  A $ 1 f 5DNN1   ! ! # % % s C&C)r!rr#r )NNFNROOT)FNr)NFNr)Tr])!__name__ __module__ __qualname____firstlineno____doc__r'r+r5r<rArFrIrUrZr_rc staticmethodrlrrrr$rrrrrrrrrr__static_attributes__rrrr r s ! 1f  $G E(*+Z# #ALR4555! xt  = . 0d  :rr c8[S5 US;a[R"SSU-/SUSS9nO"[R"SSU-/[USS9S9nUR$! [ S R U55e=f![an[ S 5UeS nAff=f) a Create image representation fom dot_string, using the 'dot' program from the Graphviz package. Use the 't' argument to specify the image file format, for ex. 'jpeg', 'eps', 'json', 'png' or 'webp' (Running 'dot -T:' lists all available formats). Note that the "capture_output" option of subprocess.run() is only available with text formats (like svg), but not with binary image formats (like png). dot)rdot_jsonjsonsvgz-T%sT)capture_outputinputtextutf8)encoding)r zACannot create image representation by running dot from string: {}z0Cannot find the dot binary from Graphviz packageN)r subprocessrunbytesstdout ExceptionrROSError)rYtprocrs rrXrX&sSE 66!~~FQJ'#'$ "~~FQJ' V<;;  6*%  SJKQRRSs) A>AAA;;A>> B BBc\rSrSrSrSrg)DependencyGraphErroriKzDependency graph exception.rN)rrrrrrrrrrrKs%rrcT[5 [5 [5 [5 gr]) malt_demo conll_democonll_file_democycle_finding_demorrrdemorOs KLrc[S5nUR5nUR5 U(aSSKnSSKJn UR 5nUR5 URUSS9nURXVSS9 URXVUR5 UR/5 UR/5 URS 5 UR5 gg) zk A demonstration of the result of reading a dependency version of the first sentence of the Penn Treebank. Pierre NNP 2 NMOD Vinken NNP 8 SUB , , 2 P 61 CD 5 NMOD years NNS 6 AMOD old JJ 2 NMOD , , 2 P will MD 0 ROOT join VB 8 VC the DT 11 NMOD board NN 9 OBJ as IN 9 VMOD a DT 15 NMOD nonexecutive JJ 15 NMOD director NN 12 PMOD Nov. NNP 9 VMOD 29 CD 16 NMOD . . 9 VMOD rN)pylabrq)dim2) node_sizeztree.png)r rpprintr matplotlibr rinfo spring_layoutdraw_networkx_nodesdraw_networkx_labelsrxticksytickssavefigshow)nxdgrrr rposs rrrVs   B* 779DKKM $ KKM $$QA$.$$Qr$:%%abll; R R j!  rc[[5nUR5nUR5 [ U5 [ UR S55 g)z[ A demonstration of how to read a string representation of a CoNLL format dependency tree. rN)r conll_data1rr$printr)r/rs rrrs9  %B 779DKKM "I "++a.rc[S5 [RS5Vs/sHo(dM [U5PM nnUH.nUR 5n[S5 UR 5 M0 gs snf)NzMass conll_read demo...rfr)r3 conll_data2rir rr$)entrygraphsgraphrs rrrsa #$2=2C2CF2K U2Ku$oe$2KF Uzz| d  Vs A7A7c[[5n[UR55 [5nUR SS/SSS.5 UR SS/SSS.5 UR SS/SSS.5 UR SS/SSS.5 UR SS/SSS.5 [UR55 g) Nrqrr)rrrr NTOPrr)r treebank_datar3rr)r/ cyclic_dgs rrrs  'B "   !I qc%ANO qc&QOP qc&QOP qc&QOP qc&QOP ) " " $%rra/ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 had heb V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 met met Prep Prep voor 8 mod _ _ 4 haar haar Pron Pron bez|3|ev|neut|attr 5 det _ _ 5 moeder moeder N N soort|ev|neut 3 obj1 _ _ 6 kunnen kan V V hulp|ott|1of2of3|mv 2 vc _ _ 7 gaan ga V V hulp|inf 6 vc _ _ 8 winkelen winkel V V intrans|inf 11 cnj _ _ 9 , , Punc Punc komma 8 punct _ _ 10 zwemmen zwem V V intrans|inf 11 cnj _ _ 11 of of Conj Conj neven 7 vc _ _ 12 terrassen terras N N soort|mv|neut 11 cnj _ _ 13 . . Punc Punc punt 12 punct _ _ a1 Cathy Cathy N N eigen|ev|neut 2 su _ _ 2 zag zie V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 hen hen Pron Pron per|3|mv|datofacc 2 obj1 _ _ 4 wild wild Adj Adj attr|stell|onverv 5 mod _ _ 5 zwaaien zwaai N N soort|mv|neut 2 vc _ _ 6 . . Punc Punc punt 5 punct _ _ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 had heb V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 met met Prep Prep voor 8 mod _ _ 4 haar haar Pron Pron bez|3|ev|neut|attr 5 det _ _ 5 moeder moeder N N soort|ev|neut 3 obj1 _ _ 6 kunnen kan V V hulp|ott|1of2of3|mv 2 vc _ _ 7 gaan ga V V hulp|inf 6 vc _ _ 8 winkelen winkel V V intrans|inf 11 cnj _ _ 9 , , Punc Punc komma 8 punct _ _ 10 zwemmen zwem V V intrans|inf 11 cnj _ _ 11 of of Conj Conj neven 7 vc _ _ 12 terrassen terras N N soort|mv|neut 11 cnj _ _ 13 . . Punc Punc punt 12 punct _ _ 1 Dat dat Pron Pron aanw|neut|attr 2 det _ _ 2 werkwoord werkwoord N N soort|ev|neut 6 obj1 _ _ 3 had heb V V hulp|ovt|1of2of3|ev 0 ROOT _ _ 4 ze ze Pron Pron per|3|evofmv|nom 6 su _ _ 5 zelf zelf Pron Pron aanw|neut|attr|wzelf 3 predm _ _ 6 uitgevonden vind V V trans|verldw|onverv 3 vc _ _ 7 . . Punc Punc punt 6 punct _ _ 1 Het het Pron Pron onbep|neut|zelfst 2 su _ _ 2 hoorde hoor V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 bij bij Prep Prep voor 2 ld _ _ 4 de de Art Art bep|zijdofmv|neut 6 det _ _ 5 warme warm Adj Adj attr|stell|vervneut 6 mod _ _ 6 zomerdag zomerdag N N soort|ev|neut 3 obj1 _ _ 7 die die Pron Pron betr|neut|zelfst 6 mod _ _ 8 ze ze Pron Pron per|3|evofmv|nom 12 su _ _ 9 ginds ginds Adv Adv gew|aanw 12 mod _ _ 10 achter achter Adv Adv gew|geenfunc|stell|onverv 12 svp _ _ 11 had heb V V hulp|ovt|1of2of3|ev 7 body _ _ 12 gelaten laat V V trans|verldw|onverv 11 vc _ _ 13 . . Punc Punc punt 12 punct _ _ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 hadden heb V V trans|ovt|1of2of3|mv 0 ROOT _ _ 3 languit languit Adv Adv gew|geenfunc|stell|onverv 11 mod _ _ 4 naast naast Prep Prep voor 11 mod _ _ 5 elkaar elkaar Pron Pron rec|neut 4 obj1 _ _ 6 op op Prep Prep voor 11 ld _ _ 7 de de Art Art bep|zijdofmv|neut 8 det _ _ 8 strandstoelen strandstoel N N soort|mv|neut 6 obj1 _ _ 9 kunnen kan V V hulp|inf 2 vc _ _ 10 gaan ga V V hulp|inf 9 vc _ _ 11 liggen lig V V intrans|inf 10 vc _ _ 12 . . Punc Punc punt 11 punct _ _ 1 Zij zij Pron Pron per|3|evofmv|nom 2 su _ _ 2 zou zal V V hulp|ovt|1of2of3|ev 7 cnj _ _ 3 mams mams N N soort|ev|neut 4 det _ _ 4 rug rug N N soort|ev|neut 5 obj1 _ _ 5 ingewreven wrijf V V trans|verldw|onverv 6 vc _ _ 6 hebben heb V V hulp|inf 2 vc _ _ 7 en en Conj Conj neven 0 ROOT _ _ 8 mam mam V V trans|ovt|1of2of3|ev 7 cnj _ _ 9 de de Art Art bep|zijdofmv|neut 10 det _ _ 10 hare hare Pron Pron bez|3|ev|neut|attr 8 obj1 _ _ 11 . . Punc Punc punt 10 punct _ _ 1 Of of Conj Conj onder|metfin 0 ROOT _ _ 2 ze ze Pron Pron per|3|evofmv|nom 3 su _ _ 3 had heb V V hulp|ovt|1of2of3|ev 0 ROOT _ _ 4 gewoon gewoon Adj Adj adv|stell|onverv 10 mod _ _ 5 met met Prep Prep voor 10 mod _ _ 6 haar haar Pron Pron bez|3|ev|neut|attr 7 det _ _ 7 vriendinnen vriendin N N soort|mv|neut 5 obj1 _ _ 8 rond rond Adv Adv deelv 10 svp _ _ 9 kunnen kan V V hulp|inf 3 vc _ _ 10 slenteren slenter V V intrans|inf 9 vc _ _ 11 in in Prep Prep voor 10 mod _ _ 12 de de Art Art bep|zijdofmv|neut 13 det _ _ 13 buurt buurt N N soort|ev|neut 11 obj1 _ _ 14 van van Prep Prep voor 13 mod _ _ 15 Trafalgar_Square Trafalgar_Square MWU N_N eigen|ev|neut_eigen|ev|neut 14 obj1 _ _ 16 . . Punc Punc punt 15 punct _ _ __main__)r)F)rrr collectionsr itertoolsrr$rnltk.internalsr nltk.treerr rXrrrrrrrr<r2r5rrrrrCs #&DDN"SJ&9&*Z  & ( T l zFr