pgloader is a data loading tool for PostgreSQL that automates bulk data migration from multiple source databases and file formats.
The tool solves the problem of rigid transactional behavior in PostgreSQL's native COPY command, which halts the entire load operation when encountering a single erroneous row. pgloader instead maintains a separate file of rejected records and continues loading valid data, allowing partial success on problematic datasets. It also performs automatic data reformatting, such as converting MySQL's invalid zero-dates to PostgreSQL NULL values, eliminating manual preprocessing steps.
The tool suits teams migrating entire databases from MySQL, SQLite, SQL Server, or other sources, particularly when data quality is uncertain. It handles schema migration including tables, indexes, foreign keys, and comments, and supports parallel loading for performance. The current version is a complete rewrite in Clojure distributed as a single JAR file requiring only Java, eliminating the native dependencies and Common Lisp runtime that characterized earlier versions. This makes deployment simpler and removes memory exhaustion issues on large migrations, which can be tuned with standard JVM flags. Connection strings use standard JDBC format with driver-specific parameters passed through unchanged, and SSL configuration moves to URI parameters rather than command-line flags.
Development activity shows active maintenance with a major architectural shift underway. The project maintains backward compatibility with the previous version's load file syntax and command-line interface despite the complete rewrite. Docker images are built automatically on every push to the development branch, indicating continuous integration practices. A Debian package for the new version is planned but not yet available, suggesting the team is stabilizing the rewrite before packaging for distribution.
