Socket Programming Python: How to make sure entire message is received?

by

First of all, to send all the bytes you don’t need a loop because python sockets provide a simple method: socket.sendall().

Now to your questions:

  1. Yes, even to receive just 4 bytes you should have a receive loop that calls recv() on the socket until 4 bytes are read.

  2. You can, if you can guarantee that such characters will not appear in the message itself. However, you’d still need to search every character that you read in for the magic delimiter, so it seems inferior to simply prefixing the message body with a length.

  3. When you call recv(n) that is only guaranteed to return at most n bytes, not exactly n bytes.

Here are three different recvall() methods to compare:

def recvall(sock, size):
    received_chunks = []
    buf_size = 4096
    remaining = size
    while remaining > 0:
        received = sock.recv(min(remaining, buf_size))
        if not received:
            raise Exception('unexpected EOF')
        received_chunks.append(received)
        remaining -= len(received)
    return b''.join(received_chunks)

and the much shorter

def recvall2(sock, size):
    return sock.recv(size, socket.MSG_WAITALL)

and finally another version that is a little shorter than the first but lacks a couple of features:

def recvall3(sock, size):
    result = b''
    remaining = size
    while remaining > 0:
        data = sock.recv(remaining)
        result += data
        remaining -= len(data)
    return result

The second one is nice and short, but it relies on a socket option socket.MSG_WAITALL that I do not believe is guaranteed to exist on every platform. The first and third ones should work everywhere. I haven’t really benchmarked any to compare and contrast.

More Related Contents:

  1. Python socket.accept nonblocking?
  2. Sending a file over TCP sockets in Python
  3. Socket Programming Python: How to make sure entire message is received?
  4. What is “backlog” in TCP connections?
  5. How to close a socket left open by a killed program?
  6. Python socket not receiving without sending
  7. TypeError: a bytes-like object is required, not ‘str’
  8. Python Socket Receive Large Amount of Data
  9. Problem with multi threaded Python app and socket connections
  10. How to set timeout on python’s socket recv method?
  11. On localhost, how do I pick a free port number?
  12. What does Python’s socket.recv() return for non-blocking sockets if no data is received until a timeout occurs?
  13. socket.shutdown vs socket.close
  14. Python simple socket client/server using asyncio
  15. Python Socket Flush
  16. Opening a SSL socket connection in Python
  17. Python server “Only one usage of each socket address is normally permitted”
  18. Python Error: io.UnsupportedOperation: fileno
  19. How to connect to a remote Windows machine to execute commands using python?
  20. Why am I getting the error “connection refused” in Python? (Sockets)
  21. How does the python socket.recv() method know that the end of the message has been reached?
  22. Can Python select what network adapter when opening a socket?
  23. 104, ‘Connection reset by peer’ socket error, or When does closing a socket result in a RST rather than FIN?
  24. Send a file through sockets in Python
  25. Discovering public IP programmatically
  26. How to change tcp keepalive timer using python script?
  27. Python [Errno 98] Address already in use
  28. Python socket.error: [Errno 111] Connection refused
  29. Making moves w/ websockets and python / django ( / twisted? )
  30. How to parse packets in a python library? [closed]

Leave a Comment