Web Scraping using Diazo!

•

5 gostaram•4,280 visualizações

pythonchile

Web Scraping using Diazo! Talk given at the StarTechConf 2011 Santiago, Chile www.startechconf.com

Tecnologia

Web Scraping
@alvaro_aguirre

Saturday, November 5, 2011

In search of our cosmic origins...

Saturday, November 5, 2011

Data Scraping
vs
Web Scraping

Saturday, November 5, 2011

Data Scraping
<html>
<header></header>
<body>
.....
</body>
</html>

Saturday, November 5, 2011

Web Scraping

Saturday, November 5, 2011

Deliverance
XDV
Diazo

Saturday, November 5, 2011

<replace css:content=”h1” css:theme=”#main” />

Saturday, November 5, 2011

<drop css:content=”h1” />

<drop css:theme=”breadcrumbs” />

Saturday, November 5, 2011

<replace css:theme=”#header” content=”#header-
element” if-content=”” />

Saturday, November 5, 2011

<drop css:theme="#info-box" if-path="/news"/>

Saturday, November 5, 2011

<theme/>
<notheme/>
<replace/>
<before/>
<after/>
<drop/>
<strip/>
<merge/>
<copy/>

Saturday, November 5, 2011

<replace css:theme="#details">
<dl id="details">
<xsl:for-each css:select="table#details > tr">
<dt><xsl:copy-of select="td[1]/text()" /></dt>
<dd><xsl:copy-of select="td[2]/node()"/></dd>
</xsl:for-each>
</dl>
</replace>/></dt>

<table id="details">
<dl id="details">
<tr>
<dt>One</dt>
<td>One</td>
<dd>1</dd>
<td>1</td>
<dt>Two</dt>
</tr>
<dd>2</dd>
<tr>
</dl>
<td>Two</td>
<td>2</td>
</tr>
</table>

Saturday, November 5, 2011

External Content

Saturday, November 5, 2011

• development of web & mobile interfaces
• legacy apps integrations
• prototypes
• low coupling

Saturday, November 5, 2011

from diazo.compiler import compile_theme
from lxml import etree
from diazo.compiler import compile_theme

absolute_prefix = "/static"

rules = "rules.xml"
theme = "theme.html"

compiled_theme = compile_theme(rules, theme,
absolute_prefix=absolute_prefix)

transform = etree.XSLT(compiled_theme)
content = etree.parse(some_content)
transformed = transform(content)

output = etree.tostring(transformed)

Saturday, November 5, 2011

github/aaguirre

Saturday, November 5, 2011

Mais conteúdo relacionado

Semelhante a Web Scraping using Diazo!

Semantic chopPascal Rettig

What's new in HTML5, CSS3 and JavaScript, James PearceSencha

Moving to Dojo 1.7 and the path to 2.0James Thomas

international PHP2011_ilia alshanetsky_Hidden Features of PHPsmueller_sandsmedia

JS Tooling in Rails 3.1Duda Dornelles

Javascript - How to avoid the bad partsMikko Ohtamaa

잘 알려지지 않은 Php 코드 활용하기형우 안

Sequel @ madrid-rbFernando Blat

永不止步的“重构”Kejun Zhang

A Tour Through the Groovy EcosystemLeonard Axelsson

让开发也懂前端lifesinger

Parser combinatorslifecoder

Xmlbaabtra.com - No. 1 supplier of quality freshers

Clouds against the Floods (RubyConfBR2011) Leonardo Borges

Ext GWT 3.0 Advanced TemplatesSencha

HTML5 & CSS3 in Drupal (on the Bayou)Mediacurrent

Rails ORM De-mystifying Active Record has_manyBlazing Cloud

The Solar Framework for PHPConFoo

The Easy Way - Plone Conference 2011Mikko Ohtamaa

Xml presentationMiguel Angel Teheran Garcia

Semelhante a Web Scraping using Diazo! (20)

Semantic chop

What's new in HTML5, CSS3 and JavaScript, James Pearce

Moving to Dojo 1.7 and the path to 2.0

international PHP2011_ilia alshanetsky_Hidden Features of PHP

JS Tooling in Rails 3.1

Javascript - How to avoid the bad parts

잘 알려지지 않은 Php 코드 활용하기

Sequel @ madrid-rb

永不止步的“重构”

A Tour Through the Groovy Ecosystem

让开发也懂前端

Parser combinators

Xml

Clouds against the Floods (RubyConfBR2011)

Ext GWT 3.0 Advanced Templates

HTML5 & CSS3 in Drupal (on the Bayou)

Rails ORM De-mystifying Active Record has_many

The Solar Framework for PHP

The Easy Way - Plone Conference 2011

Xml presentation

Último

Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm

Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software

Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst

Story boards and shot lists for my a level piececharlottematthew16

My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar

The Future of Software Development - Devin AI Innovative Approach.pdfSeasiaInfotech2

Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106

What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett

WordPress Websites for Engineers: Elevate Your Brandgvaughan

SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero

Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited

Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity

My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer

Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski

Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz

Commit 2024 - Secret Management made easyAlfredo García Lavilla

Vector Databases 101 - An introduction to the world of Vector DatabasesZilliz

Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos

Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge

E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxnull - The Open Security Community

Web Scraping using Diazo!

1. Web Scraping @alvaro_aguirre Saturday, November 5, 2011

2. In search of our cosmic origins... Saturday, November 5, 2011

3. Saturday, November 5, 2011

4. Saturday, November 5, 2011

5. Saturday, November 5, 2011

6. Saturday, November 5, 2011

7. Saturday, November 5, 2011

8. Saturday, November 5, 2011

9. Data Scraping vs Web Scraping Saturday, November 5, 2011

10. Data Scraping <html> <header></header> <body> ..... </body> </html> Saturday, November 5, 2011

11. Web Scraping Saturday, November 5, 2011

12. Saturday, November 5, 2011

13. Saturday, November 5, 2011

14. Deliverance XDV Diazo Saturday, November 5, 2011

15. Diazo Saturday, November 5, 2011

16. Saturday, November 5, 2011

17. <replace css:content=”h1” css:theme=”#main” /> Saturday, November 5, 2011

18. <drop css:content=”h1” /> <drop css:theme=”breadcrumbs” /> Saturday, November 5, 2011

19. <replace css:theme=”#header” content=”#header- element” if-content=”” /> Saturday, November 5, 2011

20. <drop css:theme="#info-box" if-path="/news"/> Saturday, November 5, 2011

21. <theme/> <notheme/> <replace/> <before/> <after/> <drop/> <strip/> <merge/> <copy/> Saturday, November 5, 2011

22. <replace css:theme="#details"> <dl id="details"> <xsl:for-each css:select="table#details > tr"> <dt><xsl:copy-of select="td[1]/text()" /></dt> <dd><xsl:copy-of select="td[2]/node()"/></dd> </xsl:for-each> </dl> </replace>/></dt> <table id="details"> <dl id="details"> <tr> <dt>One</dt> <td>One</td> <dd>1</dd> <td>1</td> <dt>Two</dt> </tr> <dd>2</dd> <tr> </dl> <td>Two</td> <td>2</td> </tr> </table> Saturday, November 5, 2011

23. Saturday, November 5, 2011

24. Saturday, November 5, 2011

25. Saturday, November 5, 2011

26. Tools Saturday, November 5, 2011

27. External Content Saturday, November 5, 2011

28. Saturday, November 5, 2011

29. • development of web & mobile interfaces • legacy apps integrations • prototypes • low coupling Saturday, November 5, 2011

30. from diazo.compiler import compile_theme from lxml import etree from diazo.compiler import compile_theme absolute_prefix = "/static" rules = "rules.xml" theme = "theme.html" compiled_theme = compile_theme(rules, theme, absolute_prefix=absolute_prefix) transform = etree.XSLT(compiled_theme) content = etree.parse(some_content) transformed = transform(content) output = etree.tostring(transformed) Saturday, November 5, 2011

31. github/aaguirre Saturday, November 5, 2011

32. diazo.org Saturday, November 5, 2011

33. plone.org Saturday, November 5, 2011

34. gracias! Saturday, November 5, 2011

Web Scraping using Diazo!

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Web Scraping using Diazo!

Semelhante a Web Scraping using Diazo! (20)

Último

Último (20)

Web Scraping using Diazo!