Scrapy image download how to use custom filename

by

This is just actualization of the answer for scrapy 0.24 (EDITED), where the image_key() is deprecated

class MyImagesPipeline(ImagesPipeline):

    #Name download version
    def file_path(self, request, response=None, info=None):
        #item=request.meta['item'] # Like this you can use all from item, not just url.
        image_guid = request.url.split("https://stackoverflow.com/")[-1]
        return 'full/%s' % (image_guid)

    #Name thumbnail version
    def thumb_path(self, request, thumb_id, response=None, info=None):
        image_guid = thumb_id + response.url.split("https://stackoverflow.com/")[-1]
        return 'thumbs/%s/%s.jpg' % (thumb_id, image_guid)

    def get_media_requests(self, item, info):
        #yield Request(item['images']) # Adding meta. I don't know, how to put it in one line :-)
        for image in item['images']:
            yield Request(image)

More Related Contents:

  1. Scraping dynamic content using python-Scrapy
  2. Scraping: SSL: CERTIFICATE_VERIFY_FAILED error for http://en.wikipedia.org
  3. ReactorNotRestartable error in while loop with scrapy
  4. Scrapy – Reactor not Restartable [duplicate]
  5. How to pass a user defined argument in scrapy spider
  6. Scrapy Very Basic Example
  7. “OSError: [Errno 1] Operation not permitted” when installing Scrapy in OSX 10.11 (El Capitan) (System Integrity Protection)
  8. Scrapy – how to manage cookies/sessions
  9. Scraping ajax pages using python
  10. How to integrate Flask & Scrapy?
  11. Crawling with an authenticated session in Scrapy
  12. passing selenium response url to scrapy
  13. Click a Button in Scrapy
  14. Access django models inside of Scrapy
  15. How can I get all the plain text from a website with Scrapy?
  16. how to handle 302 redirect in scrapy
  17. Using Scrapy with authenticated (logged in) user session
  18. Running Scrapy spiders in a Celery task
  19. Scrapy crawl from script always blocks script execution after scraping
  20. crawl site that has infinite scrolling using python
  21. How can I use different pipelines for different spiders in a single Scrapy project
  22. TypeError: Object of type ‘bytes’ is not JSON serializable
  23. getting Forbidden by robots.txt: scrapy
  24. Force my scrapy spider to stop crawling
  25. Python Scrapy: Convert relative paths to absolute paths
  26. Get document DOCTYPE with BeautifulSoup
  27. Scrapy: ImportError: No module named items
  28. Locally run all of the spiders in Scrapy
  29. Creating a generic scrapy spider
  30. find a word on a website and get its page link

Leave a Comment