SkillPedia Skills Encyclopedia
中文 EN
Today's updates 2026-11-17 UTC

No new commits yet today

Open the update feed

scraperaiscraperai/scraperai

breaks web scraping into reusable skill modules so an Agent can pick a scraping strategy by page type and output structu…

No ratings yet Rate this skill
Stars486
Forks67
Watchers486
Open Issues4

InstallOne-line install

Unconfirmed

Whether this repository follows the Agent Skills spec (that is, whether it contains a SKILL.md) has not been verified yet. Until that is settled, clone it and use what you need:

git clone https://github.com/scraperai/scraperai.git

OverviewOverview

Key Features

breaks web scraping into reusable skill modules so an Agent can pick a scraping strategy by page type and output structured data. Key capabilities include: 1. Catalog listing-page scraping: identifies repeating elements on listing pages and batch-extracts fields for every record in one pass; 2. Details detail-page parsing: locates and extracts specified fields from a single record's detail page to fill gaps left by the listing page; 3. Captcha handling: triggers a handling flow when a page is blocked by a captcha so the scraping task does not stall; 4. xpath targeting: uses XPath expressions to pinpoint the exact elements to scrape on pages with unstable structure; 5. scroll loading: automatically scrolls infinite-scroll pages and keeps collecting newly loaded content; 6. urls link discovery: bulk-extracts and organizes target URLs from a page as an entry list for later scraping.

Use Cases

web data collection tasks that need batch collection from listing pages, field completion from detail pages, continued scraping on login or captcha pages, complete scraping of scroll-loaded pages, and link discovery before batch scraping.

Target Roles

Data Analyst / Data Engineer, AI / Algorithm Engineer, Backend Developer, Test / QA Engineer.

TopicsTopics

RatingUser ratings

Nobody has rated this skill yet. Yours would be the first on the site — your vote becomes the initial average.

Rate this skill

Fill it in to be included in the per-role averages below.

One vote per IP,and it cannot be changed after submitting, so choose carefully the first time. No account is collected and no sign-in is required.

RepositoryRepository

Skill ID1879
Data sourceGitHub
Repositoryscraperai/scraperai
CategoryAI & Agent Skills / Data Collection & Web Parsing
Primary languageHTML
LicenseGNU General Public License v3.0
Default branchmain
GitHub repository ID710781843
Node IDR_kgDOKl2rkw
Rank#1294 (position within the source ranking)
Site-wide popularity rank#1471 (computed live from Stars, across all 5833 entries)

GrowthStar / Fork growth

One snapshot per day, tracking how Stars and Forks changed

02004006002026-09-272026-10-112026-09-27 · 4842026-10-11 · 486
  • Now486
  • Previous point · 09-27 484
  • vs previous point +2
  • 2 data points 14 days
0204060802026-09-272026-10-112026-09-27 · 672026-10-11 · 67
  • Now67
  • Previous point · 09-27 67
  • vs previous point ±0
  • 2 data points 14 days

TimelineTimeline

Repository created2023-10-27 12:31:462023-10-27
Data last updated2026-10-26 12:11:5822 days ago
Last commit2025-09-18 19:02:562025-09-18

CommitLatest commit

Version [latest] committed by iakov-kaiumov. Update Category: Bug Fixes & Stability Improvements. Commit details: Merge pull request #6 from AnnaLisachenko/main. This update resolves known runtime errors and enhances system reliability.

5d914ef vlatest iakov-kaiumov 2025-09-18

Next stepsNext steps