Setting smaller buffer size for sys.stdin?

by

You can completely remove buffering from stdin/stdout by using python’s -u flag:

-u     : unbuffered binary stdout and stderr (also PYTHONUNBUFFERED=x)
         see man page for details on internal buffering relating to '-u'

and the man page clarifies:

   -u     Force stdin, stdout and stderr to  be  totally  unbuffered.   On
          systems  where  it matters, also put stdin, stdout and stderr in
          binary mode.  Note that there is internal  buffering  in  xread-
          lines(),  readlines()  and  file-object  iterators ("for line in
          sys.stdin") which is not influenced by  this  option.   To  work
          around  this, you will want to use "sys.stdin.readline()" inside
          a "while 1:" loop.

Beyond this, altering the buffering for an existing file is not supported, but you can make a new file object with the same underlying file descriptor as an existing one, and possibly different buffering, using os.fdopen. I.e.,

import os
import sys
newin = os.fdopen(sys.stdin.fileno(), 'r', 100)

should bind newin to the name of a file object that reads the same FD as standard input, but buffered by only about 100 bytes at a time (and you could continue with sys.stdin = newin to use the new file object as standard input from there onwards). I say “should” because this area used to have a number of bugs and issues on some platforms (it’s pretty hard functionality to provide cross-platform with full generality) — I’m not sure what its state is now, but I’d definitely recommend thorough testing on all platforms of interest to ensure that everything goes smoothly. (-u, removing buffering entirely, should work with fewer problems across all platforms, if that might meet your requirements).

More Related Contents:

  1. How do you read from stdin?
  2. How do I pass a string into subprocess.Popen (using the stdin argument)?
  3. How do I write to a Python subprocess’ stdin?
  4. How to open every file in a folder
  5. Interactive input/output using Python
  6. Multiple inputs and outputs in python subprocess communicate
  7. How do I read from stdin?
  8. file.tell() inconsistency
  9. Running an interactive command from within Python
  10. How do I check if stdin has some data?
  11. Python 3: How to specify stdin encoding
  12. Using stdin in a child Process
  13. Can you upload to S3 using a stream rather than a local file?
  14. Read from File, or STDIN
  15. Pipe input to script and later get input from user
  16. How do you read from stdin in python from a pipe which has no ending
  17. How Postgresql COPY TO STDIN With CSV do on conflic do update?
  18. Python random.choice same output
  19. How to convert string to binary?
  20. How to strip decorators from a function in Python
  21. Does anybody know how to identify shadow dom web elements using selenium webdriver?
  22. How to replicate tee behavior in Python when using subprocess?
  23. Python Method overriding, does signature matter?
  24. Is there a ‘multimap’ implementation in Python?
  25. Can’t install psycopg2 with pip in virtualenv on Mac OS X 10.7
  26. Pandas – dataframe groupby – how to get sum of multiple columns
  27. How to make numpy.argmax return all occurrences of the maximum?
  28. How is unicode represented internally in Python?
  29. Pandas Plotting Display all date values on x-axis (matplolib only displays few values) formatted as MMM-YYYY [duplicate]
  30. Python regex split without empty string

Leave a Comment