Apify Actor Web Crawler
SkillWeb & browsingThis skill lets your AI run large-scale website crawls by deploying custom crawlers on Apify. Once added, your AI can gather data from many pages at once, keep the collected results organized in Apify datasets, and apply proxy settings for bigger jobs. It is aimed at collecting web data at scale rather than reading a single page.
Available today. Use it from your connected AI after setup.
No other account needed.
After adding it, tell your AI which website to crawl and what data you want from it. Your AI will set up the crawler on Apify and run the crawl for you.
Then ask your AI: use the Apify Actor Web Crawler skill
What your AI can do with it
- Deploy custom crawlers on Apify through its API
- Crawl large numbers of web pages in one job
- Save collected page data in Apify datasets for later use
- Manage the list of pages to visit during a crawl
- Configure proxies for large crawl jobs
What this skill tells your AI
The instructions your AI receives, as published by agentskillexchange/skills in skills/apify-actor-web-crawler/SKILL.md and read by ahel’s review.
Deploys custom Apify Actors via the Apify API v2 for large-scale web crawling using CrawleeJS. Leverages Apify dataset storage, RequestQueue, and proxy configuration for distributed scraping at scale.
Installation
Use the upstream install or setup path that matches your environment:
- npm install apify
Requirements and caveats from upstream:
- Apify SDK requires Node.js 16 or later. Add Apify SDK to any Node.js project by running:
Basic usage or getting-started notes:
-
This short tutorial will set you up to start using Apify SDK in a minute or two.
-
If you want to learn more, proceed to the Apify Platform
-
guide that will take you step by step through running your Actor on Apify's platform.
-
Extracted from upstream docs: https://raw.githubusercontent.com/apify/apify-sdk-js/HEAD/README.md
Source
Signals
- GitHub stars
- 38
- Forks
- 53
- Last commit
- Sep 2026
Advanced
- Catalog kind
- skill
- Gateway key
apify-actor-web-crawler- Source
- github.com/agentskillexchange/skills