PHP: Best way to fetch content over internet
The comment in PHP manual under curl() function, by trucex, made me think - is there really a huge difference between PHP functions, with which we fetch data from the internet.
I’ve written a small PHP script, with microtime(true) function to measure the time, that each function needs to fetch data from several local and world wide web sites (NYTimes, CNN, RtvSlo, 24ur, Wikipedia, ..). All functions were tested under the same conditions - server (3.2GHz, 1.5GB ram), line (5Mbit dl, 1.5Mbit ul) and as i said, same web sites.
Compared functions: fopen(), file_get_contents(), file(), fsckopen() and curl_init().
The results were quite surprising as the difference between functions is quite big.
file_get_contents - 78.052254915237
file - 37.172894954681
fopen - 35.497438192368
fsckopen - 20.492045879364
curl_init - 20.141245126724
As we can see from the results, the best way to fetch data is by using functions curl_init() or fsckopen() - as the worst was is by a very popular function file_get_contents().
The difference shouldn’t affect small scripts with several fetch functions, but for heavily usage of fetch functions I would recommend curl_init(). It can be used in PHP versions 4.x and 5.x, not only to get but to post data as well.
Bah, for simple scripts use file_get_contents, cURL is waste of time … or, i have simple cURl finction, soooo one include and simple call and i have data :) wiii .
As I said .. file_get_contents() is OK, even fopen() for that matter. But if you have a script which fetches thousands of files, every second is important.
In the future i might write a simple class with curl_init() to fetch data from the web into a string.
Very nice, I am thinking about using CURL too. Thanks for performing the comparison.
________
Daniel
Can you show us your test code examples? I’m asking you this since three of five functions mentioned don’t do all the work required to fetch a document from the web. I’m just not experiencing such differences between mentioned methods but it’s true that socket and cURL methods normally perform fastest.
I used fopen/fsockopen with stream_get_contents() .. if you’re fetching ~ 20 feeds, all this functions work OK and the difference isn’t that big .. but if num(feeds) > 100, every milisecond is important ..