Creating PostgreSQL bulk import files with Python

PostgreSQL supports COPY FROM for bulk loading tables from CSV and binary files on the server's filesystem, and the equivalent \copy to load the file through psql. Turn off indexes, constraints, and WAL, and COPY FROM is pretty fast. I don't know if it's the fastest way to bulk load into postgres but it sure is fast enough for me - we loaded 49 million records totalling over a terabyte in about 8 hours. It's reasonably portable across versions, too, especially if you minimize the use of binary import files.

Generating the CSV and binary formats from Python is easy enough. Here are samples for each.