How does the python socket.recv() method know that the end of the message has been reached?

by

It depends on the protocol. Some protocols like UDP send messages and exactly 1 message is returned per recv. Assuming you are talking about TCP specifically, there are several factors involved. TCP is stream oriented and because of things like the amount of currently outstanding send/recv data, lost/reordered packets on the wire, delayed acknowledgement of data, and the Nagle algorithm (which delays some small sends by a few hundred milliseconds), its behavior can change subtly as a conversation between client and server progresses.

All the receiver knows is that it is getting a stream of bytes. It could get anything from 1 to the fully requested buffer size on any recv. There is no one-to-one correlation between the send call on one side and the recv call on the other.

If you need to figure out message boundaries its up to the higher level protocols to figure that out. Take HTTP for example. It starts with a \r\n delimited header and then has a count of the remaining bytes the client should expect to receive. The client knows how to read the header because of the \r\n then knows exactly how many bytes are coming next. Part of the charm of RESTful protocols is that they are HTTP based and somebody else already figured this stuff out!

Some protocols use NUL to delimit messages. Others may have a fixed length binary header that includes a count of any variable data to come. I like zeromq which has a robust messaging system on top of TCP.

More details on what happens with receive…

When you do recv(1024), there are 6 possibilities

  1. There is no receive data. recv will wait until there is receive data. You can change that by setting a timeout.

  2. There is partial receive data. You’ll get that part right away. The rest is either buffered or hasn’t been sent yet and you just do another recv to get more (and the same rules apply).

  3. There is more than 1024 bytes available. You’ll get 1024 of that data and the rest is buffered in the kernel waiting for another receive.

  4. The other side has shut down the socket. You’ll get 0 bytes of data. 0 means you will never get more data on that socket. But if you keep asking for data, you’ll keep getting 0 bytes.

  5. The other side has reset the socket. You’ll get an exception.

  6. Some other strange thing has gone on and you’ll get an exception for that.

More Related Contents:

  1. What does Python’s socket.recv() return for non-blocking sockets if no data is received until a timeout occurs?
  2. Python socket not receiving without sending
  3. TypeError: a bytes-like object is required, not ‘str’
  4. Python Socket Receive Large Amount of Data
  5. Problem with multi threaded Python app and socket connections
  6. How to set timeout on python’s socket recv method?
  7. On localhost, how do I pick a free port number?
  8. Errno 10061 : No connection could be made because the target machine actively refused it ( client – server )
  9. Python socket receive – incoming packets always have a different size
  10. Python simple socket client/server using asyncio
  11. Python Socket Flush
  12. Opening a SSL socket connection in Python
  13. Python server “Only one usage of each socket address is normally permitted”
  14. Python Error: io.UnsupportedOperation: fileno
  15. How to connect to a remote Windows machine to execute commands using python?
  16. Why am I getting the error “connection refused” in Python? (Sockets)
  17. Can Python select what network adapter when opening a socket?
  18. 104, ‘Connection reset by peer’ socket error, or When does closing a socket result in a RST rather than FIN?
  19. Send a file through sockets in Python
  20. Python socket.accept nonblocking?
  21. Sending a reset in TCP/IP Socket connection
  22. Imported module not found in PyInstaller
  23. Running Sudo Command with paramiko
  24. Python [Errno 98] Address already in use
  25. Python socket bind to any IP?
  26. Python socket.error: [Errno 111] Connection refused
  27. How do I determine all of my IP addresses when I have multiple NICs?
  28. Making moves w/ websockets and python / django ( / twisted? )
  29. What is “backlog” in TCP connections?
  30. How to close a socket left open by a killed program?

Leave a Comment