Python Script using ExecuteStreamCommand

by

From your question you say you need to invoke the Python script without using the InvokeScriptedProcessor or ExecuteScript processors because you can’t use Jython. Given that requirement, you should still be able to accomplish your goal. While it requires some familiarity with the framework, all of this information is from the ExecuteStreamCommand documentation.

Your “I currently don’t understand” section:

  • How to call the python script (from the ExecuteStreamCommand Processor)

    • In your ExecuteStreamCommand processor, configure the Command Arguments and Command Path properties with the following:

      • Command Arguments: any flags or args, delimited by ; (i.e. /path/to/my_script.py)
      • Command Path: /path/to/python3

  • How to load up the FlowFile from within Python

    • The flowfile content will be passed via STDIN, so in your Python script, process that data the same way you would normally process STDIN.
  • How to update or create a new FlowFile from within Python
    • NiFi handles the flowfile creation in the framework. Any data passed by your Python script to STDOUT will be populated into the content of the resulting flowfile passed to the output stream relationship of the ExecuteStreamCommand processor. Your script does not need to have any awareness of “flowfiles” in this instance. If you were instead using the ISP or ES processors, you could use the NiFi scripting API which is automatically injected into the scripts to create or update the flowfile object.
  • How to output the updated FlowFile from Python back to NiFi.
    • Again, simply write the desired flowfile content to STDOUT from your script, and (given a return status code of 0) NiFi will generate a new flowfile with that content. If you set the Output Destination Attribute property of ESC to a non-null value, NiFi will instead update the existing flowfile with a new attribute of the same name containing the output of the script.

More Related Contents:

  1. Why is the output of my function printing out “None”?
  2. Convert a String representation of a Dictionary to a dictionary?
  3. How do you fix “runtimeError: package fails to pass a sanity check” for numpy and pandas?
  4. Getting list of parameter names inside python function [duplicate]
  5. Pandas Groupby Range of Values
  6. How do I build a numpy array from a generator?
  7. What is the cause of the Bad Request Error when submitting form in Flask application?
  8. Open file in a relative location in Python
  9. Assignment inside lambda expression in Python
  10. Return value from button click
  11. How to get JSON from webpage into Python script
  12. How to find the average colour of an image in Python with OpenCV?
  13. Flask Gunicorn app can’t get __name__ to equal ‘__main__’
  14. Dynamically serving a matplotlib image to the web using python
  15. How to upgrade Python version to 3.7? [closed]
  16. Iterating over arbitrary dimension of numpy.array
  17. conda environment has access to system modules, how to prevent?
  18. How to have a Python script for a Django app that accesses models without using the manage.py shell?
  19. How do I check if an object has an attribute?
  20. How to return data in JSON format using FastAPI?
  21. How do I extract all the values of a specific key from a list of dictionaries?
  22. Python: ‘Private’ module in a package
  23. Getting PIL/Pillow 4.2.1 to upload properly to AWS Lambda Py3.6
  24. Proper way to consume data from RESTFUL API in django
  25. pandas, melt, unmelt preserve index
  26. Conditional row read of csv in pandas
  27. pymysql fetchall() results as dictionary?
  28. Getting HEAD content with Python Requests
  29. What does In [*] in IPython Notebook mean and how to turn it off? [duplicate]
  30. NumPy library ImportError: DLL load failed: The specified procedure could not be found

Leave a Comment