ó 0 Dc@sÝdZyeWnek r-dZdZnXddlZddlZddlZddlZddlm Z ddl m Z m Z m Z mZmZdZdZejd ƒZd „Ze eefZd „Zd „Zddeeedd „Zdd$d„ƒYZdd%d„ƒYZdd&d„ƒYZdd'd„ƒYZdd(d„ƒYZdefd„ƒYZdefd„ƒYZ dd)d„ƒYZ!de"fd„ƒYZ#d e"fd!„ƒYZ$d"ejfd#„ƒYZ%dS(*s\HTML table parsing. ClientTable is a Python module for generic HTML table parsing. It is most useful when used in conjunction with other parsers (htmllib or HTMLParser, regular expressions, etc.), to divide up the parsing work between your own code and ClientTable. RFC 1866: HTML 2.0 RFC 1942: HTML Tables HTML 4.01 Specification, W3C Recommendation 24 December 1999 Requires Python 2.2. Copyright 2002-2003 John J. Lee This code is free software; you can redistribute it and/or modify it under the terms of the MIT License (see the file COPYING included with the distribution). iiiÿÿÿÿN(t entitydefs(t StringTypet UnicodeTypetIntTypetLongTypet FloatTypes0.0.1ais\s+cCstjd|ƒS(Nt (t WHITESPACE_REtsub(ttext((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytcollapse_whitespaceOscCs;y |dWn(ttfk r&dStk r6nXdS(s"Test whether object is a sequence.ii(t TypeErrortKeyErrort IndexError(tobject((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt issequenceSs  cCs&t|tƒst|tƒr"tStS(N(t isinstanceRRtTruetFalse(R((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt isstringlike]sc Cs‘t||||||ƒ}xN|jtƒ}y|j|ƒWntk rRPnXt|ƒtkrPqqx|jD]} | jƒqvW|jS(sƒParse HTML tables and return a list of HTMLTable objects. file: file object nr_to_parse: stop after parsing this many tables; negative means parse all of them single_span: return cells that span multiple columns or rows only once, rather than once for every row / column they span strip_tags: remove HTML tags from cell contents collapse_whitespace: collapse consecutive whitespace characters (anything matching r"\s")to a single space recode_entities: recode HTML entities according to this dict Note that passing in the return value of urllib2.urlopen here as the file argument is fine. (t TableParsertreadtCHUNKtfeedt ParseFinishedtlenttablestfixup( tfilet nr_to_parsetnr_toplevel_to_parset single_spant strip_tagsR trecode_entitiesttptdatattable((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt ParseFilebs"  t HTMLTablecBs§eZdZdZed„Zd„Zd„Zd„Z d„Z d„Z d„Z d„Z d „Zd „Zd „Zd „Zd „Zd„Zd„Zd„ZRS(s€Represents (surprise!) an HTML table. At the moment, tables may not be nested. HTMLTable instances are iterators over TableRows. This includes any header rows. Public attributes: headers_row, headers_col. headers_row is a TableRow instance used to index columns in the table. headers_col is a TableColumn instance used to index columns in the table. cCs||_g|_g|_dS(N(t _single_spant_datat _sub_tables(tselfR((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt__init__œs  cCs|jj|ƒdS(N(R(tappend(R*trow((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytpush_row¡scCs|jj|ƒdS(N(R(R,(R*R$((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt push_table¤scCs ||_dS(N(t _max_cols(R*tmax_cols((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt set_max_cols§sc Csx|jD]}|jƒq Wi}t|jƒ}dg|}xôt|ƒD]æ}|j|}t|dƒr›|}tƒ}|j|ƒ||j|R4R RRR((R*trownameRC((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytget_row_by_names cCsatƒ}xG|jD]<}||}t|dƒrB|j|ƒq|j|ƒqW|jƒ|S(sGet column by integer index.R3(RMR(R6R/R9R(R*RKRNR-R?((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt get_col_by_nr s   cCs |j|S(sGet row by integer index.(R((R*trownr((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt get_row_by_nrscCs t|jƒS(s&Return iterator over columns of table.(titeratorRV(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytcol_iterscCs t|jƒS(N(RYRX(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt__iter__!scCsKg}x"|jD]}|jd|ƒqWd|jjtj|dƒfS(Ns %ss%s[ %s]s (R(R,t __class__t__name__tstringtjoin(R*trepR-((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt__str__#scCs t|jƒS(N(RR((R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt__len__)sN(R]t __module__t__doc__R4R3RR+R.R/R2RR<R=RQRORURVRXRZR[RaRb(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR&s$      7       RYcBs#eZd„Zd„Zd„ZRS(cCsd|_||_dS(s index_fn: function behaving like __getitem__ for simple sequence object (ie. taking integer argument and returning a corresponding object, and raising IndexError if argument is out-of-bounds; valid indices must be consecutive) iN(t _iterator__it_iterator__index_fn(R*tindex_fn((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR+.s cCs|S(N((R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR[7scCsCy|j|jƒ}Wntk r/t‚nX|jd7_|S(Ni(RfReR t StopIteration(R*tr((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytnext8s   (R]RcR+R[Rj(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRY-s tCellholdcBs>eZdddd„Zd„Zd„Zd„Zd„ZRS(ttdicCsvt|ƒstd‚nt|tƒs3td‚nt|tƒsNtd‚n||_||_||_||_dS(Ns)a string-like object is required for datas"an integer is required for rowspans"an integer is required for colspan(RR Rt NumericTypesR#RItrowspantcolspan(R*R#RIRnRo((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR+Bs       cCs?t|tƒr(|j|jkr;dSn|j|kr;dSdS(Nii(RRkR#(R*tother((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt__cmp__Ns cCs|jS(N(R#(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRaTscCsN|jdks|jdkr7d|j|jf}nd}d||jfS(Nis%dx%d ts<%sCellhold[%s]>(RnRoR#(R*tspan((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt__repr__UscCs t|ƒS(N(tid(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt__hash__\s(R]RcR+RqRaRtRv(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRkAs    tCellcBsDeZdZdddd„Zd„Zd„Zd„Zd„ZRS(s²A single cell of an HTML table. Note that a single cell may span many rows or columns (or both). Public readable attributes: data, rowspan, colspan, element_type. RlicCsvt|ƒstd‚nt|tƒs3td‚nt|tƒsNtd‚n||_||_||_||_dS(s data: string-lke object: contents of cell element_type: HTML element type; should be "td" or "th" rowspan: horizontal span of cell (nr. of rows occupied by cell) rowspan: horizontal span of cell (nr. of columns occupied by cell) s)a string-like object is required for datas"an integer is required for rowspans"an integer is required for colspanN(RR RRmR#RIRnRo(R*R#RIRnRo((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR+gs       cCs?t|tƒr(|j|jkr;dSn|j|kr;dSdS(Nii(RRwR#(R*Rp((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRq{s cCs|jS(N(R#(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRascCsN|jdks|jdkr7d|j|jf}nd}d||jfS(Nis%dx%d Rrs <%sCell[%s]>(RnRoR#(R*Rs((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRt‚scCs t|ƒS(N(Ru(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRv‰s(R]RcRdR+RqRaRtRv(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRw_s   tTableSeqcBsteZdZed„Zd„Zd„Zd„Zd„Zd„Z d„Z d„Z d „Z d „Z d „ZRS( s1Abstract base class for TableRow and TableColumn.cCs1|rt|_n t|_d|_g|_dS(N(RR;RR4theadersR((R*R;((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR+Žs    cCsdS(N((R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR”scCs5t|tƒs!t|jjƒ‚|jj|ƒdS(N(RR&tAssertionErrorR\R]R(R,(R*R$((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR/–s!cCsJg}x$|jD]}|jt|ƒƒqWd|jjdj|ƒfS(Ns%s[%s]s, (R(R,treprR\R]R_(R*R`tline((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRašscCs ||jkS(N(R((R*titem((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt __contains__ŸscCs t|jƒS(N(RR((R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRb scCs |j|S(N(R((R*RC((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRQ¡scCs8t|tƒs!t|jjƒ‚|jj||ƒdS(N(RRwRzR\R]R(tinsert(R*RCR}((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR¢s!cCsAt|tƒs!t|jjƒ‚|t_|jj||ƒdS(N(RRwRzR\R]R#R(R(R*RCR}R^((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt insertdata¥s! cCs|jj|ƒS(N(R(tcount(R*R}((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRªscCs|jj|ƒS(N(R(RR(R*R}((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRR«s(R]RcRdRR+RR/RaR~RbRQRR€RRR(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRxŒs          R7cBs>eZdZed„Zd„Zd„Zd„Zd„ZRS(sŽRow of an HTML table. Indexing with a string gets a cell using the headers_row attribute. Indexing with an integer gets a cell. cCstj||ƒd|_dS(N(RxR+R4R:(R*R;((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR+¸scCsf|dk r@t|tƒs-t|jjƒ‚|jj|ƒn"|dksRt‚|jj|ƒdS(N(R4RRwRzR\R]R(R,(R*RE((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR9¼s  !cCs$t|ƒr|j|ƒS|j|S(N(Rtget_cell_by_nameR((R*RP((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRQÅs  cCs |j|S(sGet cell by integer index.(R((R*RK((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytget_cell_by_nrÊscCs<|jdkrtd‚n|jj|ƒ}|j|SdS(s9Get row by finding its name in the headers_row attribute.sno header row has been setN(R:R4R RRR((R*RSRC((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR‚Îs ( R]RcRdRR+R9RQRƒR‚(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR7±s    RMcBs>eZdZed„Zd„Zd„Zd„Zd„ZRS(sŽRow of an HTML table. Indexing with a string gets a cell using the headers_col attribute. Indexing with an integer gets a cell. cCstj||ƒd|_dS(N(RxR+R4R>(R*R;((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR+ÞscCs|jj|ƒdS(N(R(R,(R*RE((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR9âscCs$t|ƒr|j|ƒS|j|S(N(RR‚R((R*RP((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRQæs  cCs |j|S(sGet cell by integer index.(R((R*RW((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRƒëscCs<|jdkrtd‚n|jjtƒ}|j|SdS(s<Get column by finding its name in the headers_col attribute.sno header column has been setN(R>R4R RRRTR((R*RSRC((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR‚ïs ( R]RcRdRR+R9RQRƒR‚(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyRM×s     t ParseStatecBseZd„ZRS(cCs¦g|_d|_d|_t|_t|_t|_t|_t|_ d|_ d|_ t|_ t|_ d|_d|_d|_d|_d|_d|_dS(Nii(t cell_dataR4R$ttablerowRtin_tabletin_trtin_thtin_tdt in_colgrouptcolR-t in_header_rowthave_colt tx_colspant tx_rowspanttx_colst colgroup_spant colgroup_colstcol_cols(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR+ s$                 (R]RcR+(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR„ st ParseErrorcBseZRS((R]Rc(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR•-sRcBseZRS((R]Rc(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR.sRcBs¶eZdZdZddeeedd„Zd „Zd „Zd „Z d „Z d „Z d„Z d„Z d„Zd„Zd„Zd„Zd„Zd„Zd„Zd„ZRS(sHTML Table parser.R$ttrRlRHRŒtcolgroupiÿÿÿÿcCs×tjj|ƒ|dkr(td‚n|tkr@td‚n||_||_|rdt|_n t|_|rt|_n t|_|ršt|_ n t|_ ||_ g|_ g|_ d|_ tƒ|_dS(s nr_to_parse: only parse this number of tables, then stop; if negative, parse all tables in the document single_span: return cells that span multiple columns or rows only once, rather than once for every row / column they span collapse_whitespace: convert all consecutive whitespace characters to a single space # XXX name recode_entities is poor if this is a dict recode_entities: recode HTML entities according to this dict depth: internal use only is(nr_toplevel_to_parse not yet implementedssingle_span not yet implementedN(t HTMLParserR+tNotImplementedErrorRt _nr_to_parset_nr_toplevel_to_parseRR't _strip_tagst_collapse_whitespacet_recode_entitiesRt_stackt_depthR„t_ps(R*RRRR R R!((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR+8s.                cCsõ|jjr%|dkr%|jƒn|jst|jjsF|jjr‡||jkr‡|jjj|j ƒƒq‡n|jjjdƒyt |d|ƒ}WnFt k ræyt |d|ƒ}Wnt k rØqñX||ƒn X||ƒdS(NRŒRtstart_tdo_( R¡R‹t end_colgroupRœRŠR‰t table_tagsR…R,tget_starttag_texttgetattrRJ(R*ttagtattrstmethod((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pythandle_starttagms     cCsÊ|jjr%|dkr%|jƒn|jsr|jjsF|jjr…||jkr…|jjjd|ƒq…n|jjjdƒyt |d|ƒ}Wnt k rµd}nX|rÆ|ƒndS(NRŒsRtend_( R¡R‹R¤RœRŠR‰R¥R…R,R§RJR4(R*R¨Rª((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt handle_endtag‹s    cCs¼|jjrA|jjr%|jƒn|jjrA|jƒqAn|jdkrYt‚n|jd7_|j j |jƒt ƒ|_t ƒ|j_ t|j_d|j_|j_dS(Nii(R¡RˆRŠtend_tdR‰tend_thRšRR RŸR,R„R&R$RR‡R-RŒ(R*R©((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt start_tableŸs      cCs.|jjstd‚n|jjrc|jjr=|jƒn|jjrV|jƒn|jƒn|jj }|jj rŠ|jj }n |jj }|j |ƒ|jd8_|jdkrÔ|jj|ƒnG|jjƒ|_|jjr|jjj|ƒn|jj j|ƒ|jd8_dS(Nsend of TABLE before startii(R¡R‡R•RˆRŠR®R‰R¯tend_trR$RŽR”R‘R2R RR,RŸtpopR†R/Rš(R*R$R1((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt end_table°s*          cCsu|jjstd‚nd}x2|D]*\}}|dkr%t|dƒ}q%q%Wt|j_|jj|7_dS(NsCOL outside of TABLEiRsi (R¡R‡R•tintRRŽR“(R*R©Rstktv((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt start_colËs   cCs‡|jjstd‚nt|j_t|j_d|j_d}x2|D]*\}}|dkrIt|dƒ}qIqIW||j_dS(NsCOL outside of TABLEiiRsi ( R¡R‡R•RR‹RŽR“R´R’(R*R©RsRµR¶((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytstart_colgroupÓs     cCsm|jjstd‚nt|j_|jjdkrQ|jj|jj7_n|jj|jj7_dS(Nsend of COLGROUP before starti(R¡R‹R•RR“R”R’(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR¤Üs    cCsŽ|jjstd‚n|jjrc|jjr=|jƒn|jjrV|jƒn|jƒnt ƒ|j_ t |j_d|j_ dS(Ns$start of TR element outside of TABLEi( R¡R‡R•RˆRŠR®R‰R¯R±R7R†RRŒ(R*R©((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytstart_trås       cCsÍ|jjstd‚n|jjr1|jƒn|jjrJ|jƒnt|jjt |jj ƒƒ|j_|jj |jj _ |jj j|jj ƒt|j_ t|j_|jjd7_dS(Ns"end of TR element outside of TABLEi(R¡R‡R•RŠR®R‰R¯tmaxR‘RR†RR;R$R.RRˆR-(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR±ðs    '  cCs§|jr3dj|jjƒjƒ}t|ƒ}ndj|jjƒ}|jjr]d}nd}t|||jj|jj ƒ}|jj j |ƒg|j_dS(s<Consolidate data for single table entry (including headers).RrRHRlN( RR_R¡R…tstripR R‰RwRRR†R9(R*R#RLRE((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt_process_cell_dataýs  !cCsÛ|jjstd‚n|jjr1|jƒn|jjrJ|jƒnt|j_t|j_d|j_ |j_ x\|D]T\}}|dkr¯t |dƒ|j_ n|dkrt |dƒ|j_ qqWdS(NsTH outside of TABLEiRoi Rn( R¡R‡R•R‰R¯RŠR®RRRRR´(R*R©RµR¶((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytstart_th s       cCsV|jjstd‚n|jjs*t‚|jƒt|j_|jjd7_dS(Nsend of TH before starti(R¡R‰R•R‡RzR¼RRŒ(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR¯s    cCsÏ|jjstd‚n|jjr1|jƒn|jjrJ|jƒnt|j_d|j_|j_ x\|D]T\}}|dkr£t |dƒ|j_n|dkrst |dƒ|j_ qsqsWdS(NsTD outside of TABLEiRoi Rn( R¡R‡R•RŠR®R‰R¯RRRR´(R*R©RµR¶((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pytstart_td s      cCsV|jjstd‚n|jjs*t‚|jƒt|j_|jjd7_dS(Nsend of TD before starti(R¡RŠR•R‡RzR¼RRŒ(R*((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR®*s    cCsB|jjsdS|jjs(|jjr>|jjj|ƒndS(N(R¡R‡RŠR‰R…R,(R*R#((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyt handle_data2s (stablestrstdsthscolR—N(R]RcRdR¥RR4R+R«R­R°R³R·R¸R¤R¹R±R¼R½R¯R¾R®R¿(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyR5s. 0        (((((((&RdRt NameErrorRR˜tretcopyR^thtmlentitydefsRttypesRRRRRtVERSIONRtcompileRR RmRRR4R%R&RYRkRwRxR7RMR„t ExceptionR•RR(((s9/opt/htdocs/wiki/en/extensions/wikipdf/src/ClientTable.pyts@,  0(   % -%&3#