Torbot is an open source intelligence tool for crawling and analyzing the dark web.
The tool addresses the challenge of gathering information from Tor network sites by providing automated crawling capabilities that discover and map .onion domains. It works by traversing onion sites, extracting links and metadata, and building a structured representation of the discovered content. The crawler resolves relative links against the current page, checks whether links are live, and can output results in multiple formats including JSON representations of link trees and HTML files.
Torbot suits security researchers, penetration testers, and investigators who need to systematically explore dark web content for intelligence gathering. The tool is designed for those conducting evidence-based analysis where versioned crawl results need to be preserved and organized. It distinguishes itself through its ability to generate visual trees of link relationships and save structured link hierarchies, making it useful for understanding site topology rather than simple link collection. The project requires Tor and Python 3.10 or later, with pip for dependency management.
The project maintains active continuous integration with automated testing and code quality checks. Development activity shows responsiveness to improving the command-line interface, with recent work enabling version and update operations without requiring URL arguments. The crawler has been refined to handle edge cases more gracefully, including better resolution of relative links and more robust handling of malformed hrefs. The team has expanded the tool's accessibility by enabling a desktop UI option that can be launched from the command line.
