php get all the images from url which width and height >=200 more quicker

Solution 1:

I think what you use do is run curl requests in parallel using curl_multi_init please see http://php.net/manual/en/function.curl-multi-init.php for more information. This way it will load much faster and escape all bandwidth issue that can also affect speed.

Save the image into a local temp directory not run getimagesize() on the local directly which is much faster than running it over http://

I hope this helps

Edit 1

Note***

A. Not all Images start with http

B. Not all images are valid

C. Create temp folder where the images needs to be stored

Prove of Concept

require 'simple_html_dom.php';
$url = 'http://www.huffingtonpost.com';
$html = file_get_html ( $url );
$nodes = array ();
$start = microtime ();
$res = array ();

if ($html->find ( 'img' )) {
    foreach ( $html->find ( 'img' ) as $element ) {
        if (startsWith ( $element->src, "/" )) {
            $element->src = $url . $element->src;
        }
        if (! startsWith ( $element->src, "http" )) {
            $element->src = $url . "/" . $element->src;
        }
        $nodes [] = $element->src;
    }
}

echo "<pre>";
print_r ( imageDownload ( $nodes, 200, 200 ) );
echo "<h1>", microtime () - $start, "</h1>";

function imageDownload($nodes, $maxHeight = 0, $maxWidth = 0) {

    $mh = curl_multi_init ();
    $curl_array = array ();
    foreach ( $nodes as $i => $url ) {
        $curl_array [$i] = curl_init ( $url );
        curl_setopt ( $curl_array [$i], CURLOPT_RETURNTRANSFER, true );
        curl_setopt ( $curl_array [$i], CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)' );
        curl_setopt ( $curl_array [$i], CURLOPT_CONNECTTIMEOUT, 5 );
        curl_setopt ( $curl_array [$i], CURLOPT_TIMEOUT, 15 );
        curl_multi_add_handle ( $mh, $curl_array [$i] );
    }
    $running = NULL;
    do {
        usleep ( 10000 );
        curl_multi_exec ( $mh, $running );
    } while ( $running > 0 );

    $res = array ();
    foreach ( $nodes as $i => $url ) {
        $curlErrorCode = curl_errno ( $curl_array [$i] );

        if ($curlErrorCode === 0) {
            $info = curl_getinfo ( $curl_array [$i] );
            $ext = getExtention ( $info ['content_type'] );
            if ($info ['content_type'] !== null) {
                $temp = "temp/img" . md5 ( mt_rand () ) . $ext;
                touch ( $temp );
                $imageContent = curl_multi_getcontent ( $curl_array [$i] );
                file_put_contents ( $temp, $imageContent );
                if ($maxHeight == 0 || $maxWidth == 0) {
                    $res [] = $temp;
                } else {
                    $size = getimagesize ( $temp );
                    if ($size [1] >= $maxHeight && $size [0] >= $maxWidth) {
                        $res [] = $temp;
                    } else {
                        unlink ( $temp );
                    }
                }
            }
        }
        curl_multi_remove_handle ( $mh, $curl_array [$i] );
        curl_close ( $curl_array [$i] );

    }

    curl_multi_close ( $mh );
    return $res;
}

function getExtention($type) {
    $type = strtolower ( $type );
    switch ($type) {
        case "image/gif" :
            return ".gif";
            break;
        case "image/png" :
            return ".png";
            break;

        case "image/jpeg" :
            return ".jpg";
            break;

        default :
            return ".img";
            break;
    }
}

function startsWith($str, $prefix) {
    $temp = substr ( $str, 0, strlen ( $prefix ) );
    $temp = strtolower ( $temp );
    $prefix = strtolower ( $prefix );
    return ($temp == $prefix);
}

Output

Array
(
    [0] => temp/img8cdd64d686ee6b925e8706fa35968da4.gif
    [1] => temp/img5811155f8862cd0c3e2746881df9cd9f.gif
    [2] => temp/imga597bf04873859a69373804dc2e2c27e.jpg
    [3] => temp/img0914451e7e5a6f4c883ad7845569029e.jpg
    [4] => temp/imgb1c8c4fa88d0847c99c6f4aa17a0a457.jpg
    [5] => temp/img36e5da68a30df7934a26911f65230819.jpg
    [6] => temp/img068c1aa705296b38f2ec689e5b3172b9.png
    [7] => temp/imgfbeca2410b9a9fb5c08ef88dacd46895.png
)
0.076347

Thanks :)

Solution 2:

getimagesize() will download the ENTIRE image file first, then do the analysis. generally you only need the first couple hundred bytes of the file to get type/resolution details. Plus, it'll be using a separate http request for each image.

A properly optimized system would use a partial-get requests to fetch only the first chunk of the image, and take advantage of http keep-alives to keep TCP connection overhead down to a mininum.

Solution 3:

Reference

Use imagecreatefromstring, imagesx and imagesy, This should be run in 30 seconds. a bit faster than getimagesize()

function ranger($url){
    $headers = array( "Range: bytes=0-32768" );
    $curl = curl_init($url);
    curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
    return curl_exec($curl);
    curl_close($curl);
}
require dirname(__FILE__) . '/simple_html_dom.php';
$url = 'http://www.huffingtonpost.com/';
$html = file_get_html($url);
if($html->find('img')){
    foreach($html->find('img') as $element) {
        $raw = ranger($element->src);
        $im = @imagecreatefromstring($raw);
        $width = @imagesx($im);
        $height = @imagesy($im);
        if($width>=200&&$height>=200){
            echo $element;
        }
    }
}

Solution 4:

And what about reading width and height from html? I know some of the images may not have this attributes, but maybe you can just skip images with this attributes smaller than 200px.

It is just an idea to way around but maybe not usable for you.