Newspaper3k is a Python library for extracting article text, metadata, and images from web pages.
The tool solves the problem of reliably extracting article content from news websites, which often contain navigation, advertisements, and other noise alongside the actual article text. It downloads HTML from a given URL, parses the content using lxml for performance, and extracts the article body, authors, publication date, top image, and embedded videos. The library also includes natural language processing capabilities to generate keyword lists and summaries from the extracted text. It can work with individual articles or build a collection by crawling an entire news site to discover article URLs and category pages.
Newspaper3k suits projects that need to aggregate news content, monitor specific publications, or build datasets from article text. It handles multiple languages with automatic detection or explicit specification. The tool is designed for simplicity in its API, drawing inspiration from the requests library. Anyone building a news aggregator, content monitoring system, or article analysis pipeline would find this library applicable. The README does not compare it to alternative libraries, so no comparative guidance can be offered.
The project maintains active development with regular updates and bug fixes applied to the codebase. The tool has established test coverage tracked through continuous integration. Documentation is available beyond the README through an advanced docs link provided in the repository.
