SQLite Performance Benchmark — why is :memory: so slow…only 1.5X as fast as disk?

by

It has to do with the fact that SQLite has a page cache. According to the Documentation, the default page cache is 2000 1K pages or about 2Mb. Since this is about 75% to 90% of your data, it isn’t surprising that the two number are very similar. My guess is that in addition to the SQLite page cache, the rest of the data is still in the OS disk cache. If you got SQLite to flush the page cache (and the disk cache) you would see some really significant differences.

More Related Contents:

  1. Variable table name in sqlite
  2. Importing a CSV file into a sqlite3 database table using Python
  3. OperationalError: database is locked
  4. Is there a way to get a list of column names in sqlite?
  5. sqlite3.OperationalError: unable to open database file
  6. How to open and convert sqlite database to pandas dataframe
  7. Python sqlite3.OperationalError: no such table:
  8. Why does Python code run faster in a function?
  9. Is it possible to dereference variable id’s?
  10. Python out of memory on large CSV file (numpy)
  11. Parameter substitution for a SQLite “IN” clause
  12. Why is looping over range() in Python faster than using a while loop?
  13. Access memory address in python
  14. How to limit the heap size?
  15. Sqlite3, OperationalError: unable to open database file
  16. How to increase Jupyter notebook Memory limit?
  17. How to read datetime back from sqlite as a datetime instead of string in Python?
  18. Why is matrix multiplication faster with numpy than with ctypes in Python?
  19. How do you join two tables on a foreign key field using django ORM?
  20. Bulk insert huge data into SQLite using Python
  21. How to clear memory completely of all matplotlib plots
  22. How to make/use a custom database function in Django
  23. What is an efficient way of inserting thousands of records into an SQLite table using Django?
  24. Race conditions in django
  25. How can I check the memory usage of objects in iPython?
  26. Problem with regexp python and sqlite
  27. Python 3 sqlite parameterized SQL-query
  28. Convert JSON to SQLite in Python – How to map json keys to database columns properly?
  29. In Python, what is `sys.maxsize`?
  30. Limit Python VM memory

Leave a Comment