How to decode/deserialize Avro with Python from Kafka

by

If you use Confluent Schema Registry and want to deserialize avro messages, just add message_bytes.seek(5) to the decode function, since Confluent adds 5 extra bytes before the typical avro-formatted data. 

def decode(msg_value):
    message_bytes = io.BytesIO(msg_value)
    message_bytes.seek(5)
    decoder = BinaryDecoder(message_bytes)
    event_dict = reader.read(decoder)
    return event_dict

More Related Contents:

  1. Pyspark 2.4.0, read avro from kafka with read stream – Python
  2. SyntaxError on “self.async” when running python kafka producer
  3. Remove string in python dictionary value
  4. How to prompt for user input and read command-line arguments [closed]
  5. How to compare two dates?
  6. How to connect to Tor browser using Python
  7. What is the simplest way to SSH using Python?
  8. How to unzip a list of tuples into individual lists? [duplicate]
  9. How to stop flask application without using ctrl-c
  10. How can I initialize a dictionary of distinct empty lists in Python?
  11. Selenium not able to click on Get Data button on using Python
  12. Most efficient way to forward-fill NaN values in numpy array
  13. Retrieve top n in each group of a DataFrame in pyspark
  14. How to re import an updated package while in Python Interpreter? [duplicate]
  15. Getting file size in Python? [duplicate]
  16. Difference between python3 and python3m executables
  17. json.loads allows duplicate keys in a dictionary, overwriting the first value
  18. Python/Selenium incognito/private mode
  19. Efficiently create sparse pivot tables in pandas?
  20. Pass existing Webdriver object to custom Python library for Robot Framework
  21. Convert RGB to black OR white
  22. How to remove or hide x-axis labels from a seaborn / matplotlib plot
  23. Running a Python script for a user-specified amount of time
  24. Most efficient way to calculate radial profile
  25. all permutations of a binary sequence x bits long
  26. Remove NaN ‘Cells’ without dropping the entire ROW (Pandas,Python3)
  27. Large Dataset Polynomial Fitting Using Numpy
  28. Numpy find indices of groups with same value
  29. Passing a list of kwargs?
  30. Download PDF file using pdfkit and FastAPI

Leave a Comment