Sending multiple files through a TCP socket

by

TCP is a streaming protocol with no concept of message boundaries, so if you print msgp you will see it received more than you expected, probably folder name, number of files, and part of the binary file data. Since that data isn’t UTF-8 encoded, you get a UnicodeDecodeError.

You have to define a protocol and buffer data from the socket until it satisfies the protocol (read to a newline character, for example). Also see socket.makefile which wraps a socket in a file-like object so you can treat it more like a file. Methods like .readline() and .read(n) exist, so you could define a protocol like:

  1. Send Folder Name + newline
  2. Send number of files + newline
  3. Send filename #1 + newline
  4. send file size + newline
  5. send binary data of exactly “file size” bytes.
  6. Repeat 3-5 for remaining files.

Example implementing the above protocol (no error handling if a client breaks the protocol). Prepare a folder or two to send, then start the server, in another terminal, run client.py <folder> to transmit <folder> to a Downloads folder.

server.py

import socket
import os

s = socket.socket()
s.bind(('', 8000))
s.listen()

while True:
    client, address = s.accept()
    print(f'{address} connected')

    # client socket and makefile wrapper will be closed when with exits.
    with client, client.makefile('rb') as clientfile:
        while True:
            folder = clientfile.readline()
            if not folder:  # When client closes connection folder == b''
                break
            folder = folder.strip().decode()
            no_files = int(clientfile.readline())
            print(f'Receiving folder: {folder} ({no_files} files)')
            # put in different directory in case server/client on same system
            folderpath = os.path.join('Downloads', folder)
            os.makedirs(folderpath, exist_ok=True)
            for i in range(no_files):
                filename = clientfile.readline().strip().decode()
                filesize = int(clientfile.readline())
                data = clientfile.read(filesize)
                print(f'Receiving file: {filename} ({filesize} bytes)')
                with open(os.path.join(folderpath, filename), 'wb') as f:
                    f.write(data)

client.py

import socket
import sys
import os

def send_string(sock, string):
    sock.sendall(string.encode() + b'\n')

def send_int(sock, integer):
    sock.sendall(str(integer).encode() + b'\n')

def transmit(sock, folder):
    print(f'Sending folder: {folder}')
    send_string(sock, folder)
    files = os.listdir(folder)
    send_int(sock, len(files))
    for file in files:
        path = os.path.join(folder, file)
        filesize = os.path.getsize(path)
        print(f'Sending file: {file} ({filesize} bytes)')
        send_string(sock, file)
        send_int(sock, filesize)
        with open(path, 'rb') as f:
            sock.sendall(f.read())

s = socket.socket()
s.connect(('localhost', 8000))
with s:
    transmit(s, sys.argv[1])

I prepared two folders then ran “client Folder1” and “client Folder2”. Client terminal output:

C:\test>client Folder1
Sending folder: Folder1
Sending file: file1 (13 bytes)
Sending file: file2 (13 bytes)
Sending file: file3 (13 bytes)
Sending file: file4 (13 bytes)

C:\test>client Folder2
Sending folder: Folder2
Sending file: file5 (13 bytes)
Sending file: file6 (13 bytes)

Output (server.py):

C:\test>server
('127.0.0.1', 2303) connected
Receiving folder: Folder1 (4 files)
Receiving file: file1 (13 bytes)
Receiving file: file2 (13 bytes)
Receiving file: file3 (13 bytes)
Receiving file: file4 (13 bytes)
('127.0.0.1', 2413) connected
Receiving folder: Folder2 (2 files)
Receiving file: file5 (13 bytes)
Receiving file: file6 (13 bytes)

Other Examples:

More Related Contents:

  1. TypeError: a bytes-like object is required, not ‘str’
  2. Print elements of list horizontally [closed]
  3. How do i get my code to return the the answer without the extra '' and ()
  4. Removing Capitalized Strings from Input File in Python3.4?
  5. Python eval() not multiplying properly
  6. Get raw key value
  7. Import error: No module name urllib2
  8. Download file from web in Python 3
  9. Fixed digits after decimal with f-strings
  10. Print multiple arguments in Python
  11. Python, TypeError: unhashable type: ‘list’
  12. zip variable empty after first use
  13. How to convert ‘binary string’ to normal string in Python3?
  14. How to redirect with post data (Django)
  15. ImportError: No module named ‘django.core.urlresolvers’
  16. Recursively iterate through all subdirectories using pathlib
  17. Python 3: How to specify stdin encoding
  18. How to make a custom object iterable?
  19. Identifier normalization: Why is the micro sign converted into the Greek letter mu?
  20. How to create virtual env with python3
  21. asterisk in tuple, list and set definitions, double asterisk in dict definition
  22. How to get the previous URL from a POST in Django
  23. How to encode Python 3 string using \u escape code?
  24. User input boolean in python
  25. Nested Loop Python
  26. TypeError: ufunc ‘add’ did not contain a loop with signature matching types
  27. AttributeError: module ‘numpy’ has no attribute ‘__version__’
  28. Install mod_wsgi on Ubuntu with Python 3.6, Apache 2.4, and Django 1.11
  29. Python double underscore mangling
  30. Get the object with the max attribute’s value in a list of objects

Leave a Comment