using sqlalchemy to load csv file into a database

In case your CSV is quite large, using INSERTS is very ineffective. You should use a bulk loading mechanisms, which differ from base to base. E.g. in PostgreSQL you should use “COPY FROM” method: with open(csv_file_path, ‘r’) as f: conn = create_engine(‘postgresql+psycopg2://…’).raw_connection() cursor = conn.cursor() cmd = ‘COPY tbl_name(col1, col2, col3) FROM STDIN WITH (FORMAT … Read more