How to split dictionary into multiple dictionaries fast

by

Since the dictionary is so big, it would be better to keep all the items involved to be just iterators and generators, like this

from itertools import islice

def chunks(data, SIZE=10000):
    it = iter(data)
    for i in range(0, len(data), SIZE):
        yield {k:data[k] for k in islice(it, SIZE)}

Sample run:

for item in chunks({i:i for i in xrange(10)}, 3):
    print(item)

Output

{0: 0, 1: 1, 2: 2}
{3: 3, 4: 4, 5: 5}
{8: 8, 6: 6, 7: 7}
{9: 9}

More Related Contents:

  1. Creating multiple python dictionaries from a file
  2. Remove string in python dictionary value
  3. Converting dictionary to JSON
  4. How do you create nested dict in Python?
  5. python filter list of dictionaries based on key value
  6. How to pass dictionary items as function arguments in python? [duplicate]
  7. How to create a Python dictionary with double quotes as default quote format?
  8. Why does updating one dictionary object affect other?
  9. filter items in a python dictionary where keys contain a specific string
  10. How to completely traverse a complex dictionary of unknown depth?
  11. Why does dict(k=4, z=2).update(dict(l=1)) return None in Python? [duplicate]
  12. pprint dictionary on multiple lines
  13. Why dict.get(key) instead of dict[key]?
  14. Remove duplicate dict in list in Python
  15. How can I selectively escape percent (%) in Python strings?
  16. How to get a random value from dictionary?
  17. Python MySQLdb TypeError: not all arguments converted during string formatting
  18. Insert a Pandas Dataframe into mongodb using PyMongo
  19. Unable to install Python libraries
  20. How to specify where a Tkinter window opens?
  21. Permission problems when creating a dir with os.makedirs in Python
  22. Unable to install sasl-0.1.3 python package on windows machine
  23. Screen Capture with OpenCV and Python-2.7
  24. Python dictionary replace values
  25. How to get multiple dictionary values?
  26. Sort a list of tuples by second value, reverse=True and then by key, reverse=False
  27. Is the right-hand side of an assignment always evaluated before the assignment?
  28. getattr and setattr on nested subobjects / chained properties?
  29. pydev breakpoints not working
  30. How do I send list of dictionary as Body parameter together with Files in FastAPI?

Leave a Comment