Web scraping

OpenSERP is a self-hosted SERP API and CLI that aggregates search results from Google, Bing, Yandex, Baidu, DuckDuckGo, and Ecosia with browser...

BrowserAct Skills is a browser automation CLI designed specifically for AI agents to perform web tasks while bypassing anti-bot protections, handling...

A TypeScript-based template that automates website cloning by using AI coding agents (primarily Claude Code) to reverse-engineer any website into a...

invisible_playwright is a modified Firefox browser with anti-detection patching at the C++ source level, designed as a drop-in Playwright replacement...

Firecrawl is a web scraping and search API that converts any URL into clean Markdown, structured JSON, or screenshots suitable for LLM consumption,...

webclaw is a Rust-based web content extraction tool that converts URLs into clean markdown, JSON, and LLM-ready structured data. It serves AI agents,...

httpcloak is a Go HTTP client library that emulates browser TLS/HTTP2/HTTP3 fingerprints to bypass bot detection systems. It replicates Chrome,...

Camofox-browser is a REST API wrapper around Camoufox (a Firefox fork with C++-level fingerprint spoofing) designed for AI agents to browse the web...

Firecrawl MCP Server is an official Model Context Protocol integration that enables LLM clients (Cursor, Claude, etc.) to scrape websites, search the...

Fredy is a self-hosted real estate search automation tool for the German market that scrapes listings from ImmoScout24, Immowelt, Immonet, eBay...

CloakBrowser is a stealth Chromium browser with source-level C++ fingerprint patches designed to evade bot detection systems. It serves developers...

Scrapling is an adaptive Python web scraping framework that handles everything from single HTTP requests to full browser-based crawling with stealth...

Maxun is an open-source no-code platform that converts websites into structured APIs through web scraping, crawling, and AI-powered data extraction....

A curated collection of public APIs organized by category, designed for developers building integrations, automations, and data-driven applications....

A Go-based web scraper for extracting business data from Google Maps, including names, addresses, phone numbers, ratings, reviews, and coordinates....

curl_cffi is a Python binding for curl-impersonate that enables HTTP clients to mimic browser TLS/JA3/HTTP2 fingerprints, designed specifically for...

Trafilatura is a Python package and command-line tool for web crawling, scraping, and extracting structured text and metadata from HTML. It...

ScrapeGraphAI is a Python library that uses LLMs and graph logic to automate web scraping and data extraction from websites and local documents...

Crawlee is a comprehensive Node.js library for web scraping and browser automation that helps developers build reliable, human-like crawlers...

Spider is a high-performance web crawler and scraper written in Rust, designed to stream pages as they arrive and render JavaScript only when...

changedetection.io is a self-hosted website monitoring platform that detects content changes and sends alerts via multiple channels (Discord, Email,...

This is a Python SDK for Oxylabs' AI Studio API, enabling structured data extraction from websites using AI-powered scraping, crawling, and browser...

Miasma is a lightweight Rust server designed to trap AI web scrapers by serving them poisoned training data and self-referential links in an endless...

Skill Seekers is a Python tool that extracts documentation from websites, GitHub repositories, PDFs, and 10+ other sources, transforming them into...

Patchright is a patched version of Playwright that aims to evade bot detection while automating browser interactions. It serves web scraping,...

Helium is a Python library for web browser automation that wraps Selenium with a higher-level, user-friendly API. It specializes in simplifying web...

ScrapFly Scrapers is a collection of 40+ educational Python web scraping examples for popular domains (Amazon, eBay, Aliexpress, etc.), built on the...

PinchTab is a lightweight HTTP server written in Go that enables AI agents and automation tools to control headless Chrome instances through a...

wreq is a Python HTTP client library built in Rust that specializes in evading detection systems through precise TLS and HTTP/2 fingerprint control,...

AgentQL is a suite of tools that enables AI agents and LLMs to interact with and extract data from web pages using natural language queries, with...