Parsing SQL with Python

by

I have looked into this issue quite extensively. Python-sqlparse is a non validating parser which is not really what you need. The examples in antlr need a lot of work to convert to a nice ast in python. The sql standard grammars are here, but it would be a full time job to convert them yourself and it is likely that you would only need a subset of them i.e no joins. You could try looking at the gadfly (a Python SQL database) as well, but I avoided it as they used their own parsing tool.

For my case, I only essentially needed a where clause. I tried booleneo (a boolean expression parser) written with pyparsing but ended up using pyparsing from scratch. The first link in the reddit post of Mark Rushakoff gives a SQL example using it. Whoosh a full text search engine also uses it but I have not looked at the source to see how.

Pyparsing is very easy to use and you can very easily customize it to not be exactly the same as SQL (most of the syntax you will not need). I did not like ply as it uses some magic using naming conventions.

In short give pyparsing a try, it will most likely be powerful enough to do what you need and the simple integration with python (with easy callbacks and error handling) will make the experience pretty painless.

More Related Contents:

  1. Split string at commas except when in bracket environment
  2. how to parse the response in python?
  3. Why can’t Python parse this JSON data? [closed]
  4. Pandas read_csv low_memory and dtype options
  5. How to convert comma-delimited string to list in Python?
  6. How to put parameterized sql query into variable and then execute in Python?
  7. URL query parameters to dict python
  8. What is the fastest way to parse large XML docs in Python?
  9. What’s the best practice using a settings file in Python? [closed]
  10. Splitting on last delimiter in Python string?
  11. When splitting an empty string in Python, why does split() return an empty list while split(‘\n’) returns [”]?
  12. Python parsing bracketed blocks
  13. How can I translate dates and times from natural language to datetime? [closed]
  14. Why is json.loads an order of magnitude faster than ast.literal_eval?
  15. Insert list into my database using Python [duplicate]
  16. Complex foreign key constraint in SQLAlchemy
  17. parsing nested parentheses in python, grab content by level
  18. Using ConfigParser to read a file without section name
  19. Is this Python code vulnerable to SQL injection? (SQLite3)
  20. Python SQL query string formatting
  21. Regular expression to return all characters between two special characters
  22. SQLAlchemy: how to filter date field?
  23. How to execute “left outer join” in SqlAlchemy
  24. Python “safe” eval (string to bool/int/float/None/string)
  25. Python SQLite3 SQL Injection Vulnerable Code
  26. Getting random row through SQLAlchemy
  27. Select NULL Values in SQLAlchemy
  28. How to check the existence of a row in SQLite with Python?
  29. How to quote a string value explicitly (Python DB API/Psycopg2)
  30. Pandas analogue to SQL MINUS / EXCEPT operator, using multiple columns

Leave a Comment