Aggregator is a proxy pool construction tool that crawls, validates, and aggregates free proxies from multiple sources into formats compatible with various proxy clients.
The tool solves the problem of manually sourcing and managing proxy lists by automating the entire workflow. It crawls proxies from platforms including Telegram, GitHub, Google, Yandex, and Twitter, then validates their activity and quality through intelligent checking. The aggregated proxies are converted into formats supported by clients like Clash, V2Ray, and SingBox, and can be stored on multiple backends including GitHub Gist, PasteGG, and Imperial. The project includes a plugin system for extending crawling capabilities and uses multithreading for efficient batch processing.
The tool offers two usage modes suited to different needs. The process.py mode provides full functionality with support for complex configurations, multiple crawling sources, custom rules, and scheduled automation, making it appropriate for users who need fine-grained control over proxy collection and multi-group output management. The collect.py mode is a simplified version designed for quick proxy gathering without complex configuration, suitable for personal daily use and rapid free proxy acquisition. Adoption is straightforward for users comfortable with Python configuration files and environment variables.
Development activity shows significant architectural ambition. The project has planned a comprehensive refactoring to introduce core interfaces like ICrawler, IStorage, and IConverter with corresponding base classes and factory patterns to reduce module coupling. A plugin architecture is under development to modularize crawlers, storage backends, and converters with automatic discovery and registration systems. The configuration system is being enhanced with Pydantic-based type-safe models, improved validation with better error messaging, and standardized templates for common scenarios.
