Enterprise AI Recursive Web Scraper
- When
- Nov 2024 – Oct 2025
- Length
- 11 months
- Status
- Completed / active maintenance
Visit npmjs.
Advanced AI-powered recursive web scraper utilizing Groq LLMs, Playwright, and Puppeteer for intelligent, structured content extraction and risk assessment.
Where it sits
EarlierElysiumOSSNov 2024LaterAI-BITSDec 2024The project
A high-performance, enterprise-ready recursive web scraper designed for intelligent content extraction.
It features concurrent multi-threaded crawling, token bucket rate limiting, and a robust content analysis pipeline using Google Gemini LLMs for topic-based and semantic chunking. The system uses Playwright/