Uploading large files with Python/Django

by

Django will by default, put uploaded file data into memory if it is less than 2.5MB. Anything larger will be written to the server’s /tmp directory and then copied across when the transfer completes. Many of Django’s file upload settings can be customised, details are available in the documentation. You can also customise the file handling and you’ll certainly want to do this.

Before we consider any technical constraints, uploading such large files with the browser will give the user a very poor experience. There is no feedback about how the transfer is going (although google chrome does display the upload status as a percentage) and no way to pause or resume transfers.

You are also likely to run into problems on the server. Apart from the extremely long time that each thread will be taken with dealing with the streamed data, you have the time it takes for the system to copy the resulting file from /tmp to its correct location.

Unless you are very confident that you can foresee any problem that the server might have with the uploads, I would suggest that this is a bad idea. It’s pretty hard to find any information on this via google and there do seem to be a lot of hits that describe problems with large file uploads.

While Django is technically capable of receiving uploaded files this large, the very poor user experience and technical difficulties mean this may not be the best approach. Have you considered using dedicated software to handle the file transfer?

More Related Contents:

  1. Django add brackets to django models field
  2. Execute code when Django starts ONCE only?
  3. Programmatically saving image to Django ImageField
  4. Does SQLAlchemy have an equivalent of Django’s get_or_create?
  5. Custom Filter in Django Admin on Django 1.3 or below
  6. Django Aggregation: Summation of Multiplication of two fields
  7. OperationalError: database is locked
  8. How do you log server errors on django sites
  9. Install mysql-python (Windows)
  10. Django: How to manage development and production settings?
  11. Django migration strategy for renaming a model and relationship fields
  12. What is the most efficient way to store a list in the Django models?
  13. Can’t connect to local MySQL server through socket ‘/tmp/mysql.sock
  14. Django Server Error: port is already in use
  15. Django – CSRF verification failed
  16. Making a Django form class with a dynamic number of fields
  17. Accepting email address as username in Django
  18. Composite primary key in django
  19. Serialize datetime to JSON
  20. How to make/use a custom database function in Django
  21. Microsoft Visual C++ 14.0 is required. Get it with “Microsoft Visual C++ Build Tools”: http://landinghub.visualstudio.com/visual-cpp-build-tools [duplicate]
  22. How to display uploaded images in “Change List” page in Django Admin?
  23. Get local timezone in django
  24. How to make an Inner Join in django?
  25. How do I call a Django function on button click?
  26. Send message using Django Channels from outside Consumer class
  27. How to use Matplotlib in Django?
  28. creating a MIME email template with images to send with python / django
  29. Heroku: ModuleNotFoundError :No module named ‘requests’
  30. How to delete a record in Django models?

Leave a Comment