Scraping Data: PHP Simple HTML DOM Parser

Posted on December 12, 2008, Filled under PHP, Bookmark it

PHP Simple HTML DOM Parser, written in PHP5+, allows you to manipulate HTML in a very easy way. Supporting invalid HTML, this parser is better then other PHP scripts that use complicated regexes to extract information from web pages.

Before getting the necessary info, a DOM should be created from either URL or file. The following script extracts links & images from a website:

// Create DOM from URL or file
$html = file_get_html('http://www.microsoft.com/');

// Extract links
foreach($html->find('a') as $element)
       echo $element->href . '<br>'; 

// Extract images
foreach($html->find('img') as $element)
       echo $element->src . '<br>';

The parser can also be used to modify HTML elements:

// Create DOM from string
$html = str_get_html('<div id="simple">Simple</div><div id="parser">Parser</div>');

$html->find('div', 1)->class = 'bar';

$html->find('div[id=simple]', 0)->innertext = 'Foo';

// Output: <div id="simple">Foo</div><div id="parser" class="bar">Parser</div>
echo $html;

Do you wish to retrieve content without any tags?

echo file_get_html('http://www.yahoo.com/')->plaintext;

In the package files of this parser (http://simplehtmldom.sourceforge.net/) you can find some scraping examples from digg, imdb, slashdot. Let’s create one that extracts the first 10 results (titles only) for the keyword “php” from Google:

$url = 'http://www.google.com/search?hl=en&q=php&btnG=Search';

// Create DOM from URL
$html = file_get_html($url);

// Match all 'A' tags that have the class attribute equal with 'l'
foreach($html->find('a[class=l]') as $key => $info)
{
echo ($key + 1).'. '.$info->plaintext."<br />\n";
}

NOTE Make sure to include the parser before using any functions of it:

include 'simple_html_dom.php';

For more information regarding the usage of this function consider checking the ‘PHP Simple HTML Dom Parser’ Manual. To download the package files use the following URL: http://sourceforge.net/project/showfiles.php?group_id=218559.

Do you wish to receive the latest updates as soon as they are posted? Get our RSS Feed or Subscribe to the Newsletter!

December 12, 2008
article by Gabriel C.
21 comments

Tags: web fetch

Scraping Data: PHP Simple HTML DOM Parser

Sponsors

PHP: Creating a simple web data (spider) extractor
at September 14, 2008 with 9 comments
A very Simple and Easy to Customize XML based CMS: GetSimple
at August 5, 2009 with 4 comments
How to extract images from an URL in PHP
at August 30, 2008 with 1 comment
Create a PHP Script that Logins in to a Password Protected Area
at December 17, 2008 with 14 comments
Extract URL(s) from Link(s) with PHP
at September 4, 2008 with 1 comment

21 Replies to "Scraping Data: PHP Simple HTML DOM Parser"

New Fresh PHP Tutorials | Pocko.org
January 14, 2009 at 1:47 PM

[...] 14. Scraping data with PHP Simple HTML DOM Parser [...]
tedz
January 21, 2009 at 11:57 PM

Reply

how about when do you want to scrape data hve a search result with a post method
Gabriel
January 22, 2009 at 6:10 AM

Reply

In this case, I recommend you the following post: Create a PHP script that logins in to a password protected area.
Popescu
January 23, 2009 at 7:38 AM

Reply

I can not make it work. I simply put the files on my server and it gave me the error:

"Parse error: parse error, expecting `T_OLD_FUNCTION' or `T_FUNCTION' or `T_VAR' or `'}'' in c:\easyphp\www\r2\3\simple_html_dom.php on line 84 "

how can I make it to work?
Gabriel
January 29, 2009 at 7:09 AM

Reply

It should work 100%. Make sure you didn't make any changes on simple_html_dom.php and you have PHP5 installed on your server.
Simon Quizno
February 1, 2009 at 6:21 PM

Reply

Is php 5 required for simple_html_dom – says on the website that it's php4 and there is no mention of minimum requirements but I'm getting the same parse error as popescu is getting. It works on my local test server php5 but not on production php4
Simon Quizno
February 2, 2009 at 7:08 AM

Reply

Woooops..nevermind. I read it wrong. It does say php5 required.

Go on about your normal daily business.
Search
February 3, 2009 at 1:24 PM

Reply

I see "file_get_html('http://www.yahoo.com/')->plaintext; "
return a big plain string. Is there a way I can get an array of string stead?
anonymous
March 26, 2009 at 10:56 PM

Reply

How to scrape page with different charset than utf-8? In my case I want to scrape page with charset=windows-1257 but in the results I get unknown symbols for some non latin letters
anonymous
March 28, 2009 at 10:58 AM

Reply

Ok I found the solution function iconv can solve the problem http://uk.php.net/manual/en/function.iconv.php
lina
April 25, 2009 at 3:54 PM

Reply

I cannot run file_get_html. whats are the requirement for this html dom parser.
1. Gabriel
  April 26, 2009 at 8:54 PM
  
  Reply
  
  PHP5+ is required for the PHP HTML Dom Parser.
Bob Brodie
May 13, 2009 at 5:07 PM

Reply

Is there an easy way to get more than 10 results from Google? Lets say you want 50. The url would get &start=10 for page 2, &start=20 for page 3, etc.
1. Randall
  February 11, 2010 at 12:45 PM
  
  Reply
  
  add &num=50 to the google url or any umber u want between 1-100
Teil einer Seite auslesen - PHP @ tutorials.de: Forum, Tutorial, Anleitung, Schulung & Hilfe
July 20, 2009 at 11:49 AM

[...] [...]
fl3x7
July 20, 2009 at 5:03 PM

Reply

this looks great. i just had a go but couldnt really get it working

would really appreciate some help.

what im trying to do is parse some information within a particular div with all tags intact. Eg grab everything as it is within the

The particular div has a table in it. i want it to copy parse it all intact. i dont need a for loop i dont think as im not repeating the search. just want it to copy the contents of the first id test it hits (as there is only one on the page anyway)

thank you for your help, really appreciate it

fl3×7
fl3x7
July 20, 2009 at 5:06 PM

Reply

ahhh sorry your comments box seems to have stripped some of the code i put in
Kasper
September 14, 2009 at 9:33 AM

Reply

This plugins is just brilliant so easy to sort out large amounts of data <3 this plugin!
Vasile
November 16, 2009 at 8:29 PM

Reply

What about if I want to find tag with multiple attributes. let say that I have a table with width attribute, colspan, cellpading, border… And I want to find that particular table. In xpath this can be done with conjunction, what is the conjunction in simple_html_dom?b
toan
January 26, 2010 at 9:15 PM

Reply

i can’t user. Because for read anothet url i need Curl,in the file, i can insert it
Hawknut
February 9, 2010 at 2:59 PM

Reply

This DOM creator is fantastic! I just spent the last several hours messing around with very complex regex, curl, and preg_match code trying to accomplish only 1/2 of what I was able to accomplish with your script in 2 minutes! Thank you so much!