A crawler to get all a tags of a page. It only crawls the targets domain. If you crawl example.com which has a link to whatever.com, whatever.com will not be crawled.
$crawl = new Crawler();
$crawl->crawl('https://example.com');
Gets all the crawled links of that domain as a one dimensional array.
$crawl->getCrawledLinks();
- http
- https
- html
- htm
You can set and get allowed schemes and file extensions.
$crawl->allowed('set', 'allowedFiles', '.pdf', '.png');
$crawl->allowed('remove', 'allowedFiles', '.pdf', '.png');
$crawl->allowed('remove', 'allowedSchemes', 'http');
$crawl->allowed('get', 'allowedFiles');
$crawl->allowed('get', 'allowedSchemes');