Photon is a web crawler designed for open-source intelligence gathering that prioritizes speed and data extraction.
The tool addresses the need to rapidly collect intelligence from websites by crawling pages and extracting structured information including URLs, email addresses, social media accounts, API keys, JavaScript files, DNS records, and custom pattern matches. It uses smart thread management and refined crawling logic to achieve high performance while remaining resource-conscious. The tool offers flexible control through options for timeouts, delays, URL filtering, and seed management. It can also leverage archived versions of pages from archive.org as crawl seeds to reduce the need for live requests.
Photon suits security researchers, penetration testers, and OSINT practitioners who need to systematically gather intelligence from target domains. The tool extracts both in-scope and out-of-scope URLs along with sensitive data like credentials and API keys, organizing results in a structured format or exporting them as JSON. It includes plugins for DNS enumeration via dnsdumpster and integration with archive.org's Wayback Machine. The project provides a Docker image for containerized deployment and maintains a wiki with detailed usage documentation.
The project shows active development with regular updates addressing bug fixes, performance optimization, and new features. A public development board tracks ongoing work and planned improvements. The tool is distributed through standard Python package management channels and includes continuous integration testing.
