Scrapling 0.4.14
SkillWeb & browsingWrite scrapers with the official Scrapling 0.4.14 API (Fetcher, StealthyFetcher, DynamicFetcher, sessions, Spider templates, adaptive CSS). Default framework in the Scraping module. Fall back to the scrape tool or browser only when Scrapling is the wrong tool.
Available today. Use it from your connected AI after setup.
No other account needed.
Connect ahel once, and every AI you use reads what you have installed.
Then ask your AI: use the Scrapling 0.4.14 skill
What this skill tells your AI
The instructions your AI receives, as published by navinspire-ia/navin in navin/skills/scrapling/SKILL.md and read by ahel’s review.
Official adaptive scraping framework (D4Vinci). Pin scrapling==0.4.14. Use only this API. Do not invent methods or mix BeautifulSoup / raw Playwright unless the user named that stack.
Docs: https://scrapling.readthedocs.io/en/latest/
In the Scraping module this is the default, the same way Code uses the Dev stack and Leads uses prospecting scripts.
Choose a layer
| Need | Use |
|---|---|
| Static HTML, TLS impersonation | Fetcher / FetcherSession |
| Cloudflare Turnstile / stealth | StealthyFetcher / StealthySession (solve_cloudflare=True) |
| Full JS browser (Playwright Chromium / Chrome) | DynamicFetcher / DynamicSession |
| Multi-page crawl, pause/resume, export | Spider (or a template below) |
| Parse HTML you already have | Selector from scrapling.parser |
| One-shot corpus, no new Python | Navin scrape tool (Rust / httpx) |
| Forms, multi-tab, upload, human walls in Navin UI | Navin browser tool |
Read references/api.md for sessions, spiders, templates, CLI, and MCP.
Install (required before fetchers/spiders)
pip install scrapling is parser only. from scrapling.fetchers or from scrapling.spiders fails without extras.
pip install "scrapling[fetchers]==0.4.14"
scrapling install
Repo extra: pip install -e ".[scraping]" then scrapling install.
- MCP:
pip install "scrapling[ai]==0.4.14" - Shell /
scrapling extract:pip install "scrapling[shell]==0.4.14" - Everything:
pip install "scrapling[all]==0.4.14"thenscrapling install
If import fails, install via exec, then retry. Do not silently switch stack.
Canonical fetch + adaptive select
from scrapling.fetchers import Fetcher, AsyncFetcher, StealthyFetcher, DynamicFetcher
StealthyFetcher.adaptive = True
page = StealthyFetcher.fetch(
"https://example.com",
headless=True,
network_idle=True,
)
products = page.css(".product", auto_save=True)
# After a redesign:
products = page.css(".product", adaptive=True)
HTTP with session + Chrome TLS:
from scrapling.fetchers import Fetcher, FetcherSession
with FetcherSession(impersonate="chrome") as session:
page = session.get("https://quotes.toscrape.com/", stealthy_headers=True)
quotes = page.css(".quote .text::text").getall()
page = Fetcher.get("https://quotes.toscrape.com/")
Stealth / Cloudflare:
from scrapling.fetchers import StealthyFetcher, StealthySession
with StealthySession(headless=True, solve_cloudflare=True) as session:
page = session.fetch("https://nopecha.com/demo/cloudflare", google_search=False)
data = page.css("#padded_content a").getall()
page = StealthyFetcher.fetch("https://nopecha.com/demo/cloudflare")
Full browser:
from scrapling.fetchers import DynamicFetcher, DynamicSession
with DynamicSession(headless=True, disable_resources=False, network_idle=True) as session:
page = session.fetch("https://quotes.toscrape.com/", load_dom=False)
data = page.xpath('//span[@class="text"]/text()').getall()
page = DynamicFetcher.fetch("https://quotes.toscrape.com/")
Canonical spider
from scrapling.spiders import Spider, Request, Response
class QuotesSpider(Spider):
name = "quotes"
start_urls = ["https://quotes.toscrape.com/"]
concurrent_requests = 10
async def parse(self, response: Response):
for quote in response.css(".quote"):
yield {
"text": quote.css(".text::text").get(),
"author": quote.css(".author::text").get(),
"url": response.url,
}
next_page = response.css(".next a")
if next_page:
yield response.follow(next_page[0].attrib["href"])
result = QuotesSpider().start()
result.items.to_json("scrape/quotes.json")
Pause / resume: QuotesSpider(crawldir="./scrape/crawl_data").start(). Ctrl+C saves; same crawldir resumes.
Ready-made templates (prefer these over a blank spider when they fit): CrawlSpider, SitemapSpider, XMLFeedSpider, CSVFeedSpider, ShopifySpider.
from scrapling.spiders import ShopifySpider
class MyStore(ShopifySpider):
target_website = "example.com"
result = MyStore().start()
Parse without fetching
from scrapling.parser import Selector
page = Selector("<html>...</html>")
quotes = page.css(".quote")
quotes = page.xpath('//div[@class="quote"]')
quotes = page.find_all("div", class_="quote")
quotes = page.find_by_text("quote", tag="div")
Selection: CSS, XPath, find_all, find_by_text, chained .css(), ::text / ::attr(href) like Scrapy/Parsel. Navigation: .parent, .next_sibling, .find_similar(), .below_elements(). Adaptive: auto_save=True then later adaptive=True.
Delivery in Navin
- Write scripts under
scrape/build/. Pinscrapling==0.4.14in any requirements file. - Deliver the exported dataset and the report, not the spider. Hand over scraper code only when the user asked for it.
- Export with
result.items.to_json()/to_jsonl()/to_csv()/to_xml(), or Navinscrapeexport. - Keep a source URL on every row. Never dump corpora into chat.
robots_txt_obeyon spiders when the user cares about compliance.- Captcha / paywall / login that still blocks after StealthyFetcher: pause and use the Navin
browsertool with the user. Do not sell bypass as a guarantee.
Fallback (do not skip)
- Scrapling 0.4.14 for code the agent writes or runs.
- Navin
scrapetool for a ready-made fetch/crawl/export (Rust, else httpx). - Navin
browserfor interactive UI, empty JS shells after Scrapling, or assisted walls.
Signals
- GitHub stars
- 22
- Forks
- 4
- Last commit
- Sep 2026
ahel review
K1binfo
installs-packagesK1binfo
installs-packages (in references/api.md)
Automated review, not a security audit. Ruleset v1+k2.
Advanced
- Catalog kind
- skill
- Gateway key
scrapling-navinspire-ia- Source
- github.com/navinspire-ia/navin