Load just part of an image in python

Save your files as uncompressed 24-bit BMPs. These store pixel data in a very regular way. Check out the “Image Data” portion of this diagram from Wikipedia. Note that most of the complexity in the diagram is just from the headers:

For example, let’s say you are storing this image (here shown zoomed in):

This is what the pixel data section looks like, if it’s stored as a 24-bit uncompressed BMP. Note that the data is stored bottom-up, for some reason, and in BGR form instead of RGB, so the first line in the file is the bottom-most line of the image, the second line is the second-bottom-most, etc:

00 00 FF    FF FF FF    00 00
FF 00 00    00 FF 00    00 00

That data is explained as follows:

           |  First column  |  Second Column  |  Padding
-----------+----------------+-----------------+-----------
Second Row |  00 00 FF      |  FF FF FF       |  00 00
-----------+----------------+-----------------+-----------
First Row  |  FF 00 00      |  00 FF 00       |  00 00
-----------+----------------+-----------------+-----------

or:

           |  First column  |  Second Column  |  Padding
-----------+----------------+-----------------+-----------
Second Row |  red           |  white          |  00 00
-----------+----------------+-----------------+-----------
First Row  |  blue          |  green          |  00 00
-----------+----------------+-----------------+-----------

The padding is there to pad the row size to a multiple of 4 bytes.

---

So, all you have to do is implement a reader for this particular file format, and then calculate the byte offset of where you have to start and stop reading each row:

def calc_bytes_per_row(width, bytes_per_pixel):
    res = width * bytes_per_pixel
    if res % 4 != 0:
        res += 4 - res % 4
    return res

def calc_row_offsets(pixel_array_offset, bmp_width, bmp_height, x, y, row_width):
    if x + row_width > bmp_width:
        raise ValueError("This is only for calculating offsets within a row")

    bytes_per_row = calc_bytes_per_row(bmp_width, 3)
    whole_row_offset = pixel_array_offset + bytes_per_row * (bmp_height - y - 1)
    start_row_offset = whole_row_offset + x * 3
    end_row_offset = start_row_offset + row_width * 3
    return (start_row_offset, end_row_offset)

Then you just have to process the proper byte offsets. For example, say you want to read the 400×400 chunk starting at position 500×500 in a 10000×10000 bitmap:

def process_row_bytes(row_bytes):
    ... some efficient way to process the bytes ...

bmpf = open(..., "rb")
pixel_array_offset = ... extract from bmp header ...
bmp_width = 10000
bmp_height = 10000
start_x = 500
start_y = 500
end_x = 500 + 400
end_y = 500 + 400

for cur_y in xrange(start_y, end_y):
    start, end = calc_row_offsets(pixel_array_offset, 
                                  bmp_width, bmp_height, 
                                  start_x, cur_y, 
                                  end_x - start_x)
    bmpf.seek(start)
    cur_row_bytes = bmpf.read(end - start)
    process_row_bytes(cur_row_bytes)

Note that it’s important how you process the bytes. You can probably do something clever using PIL and just dumping the pixel data into it but I’m not entirely sure. If you do it in an inefficient manner then it might not be worth it. If speed is a huge concern, you might consider writing it with pyrex or implementing the above in C and just calling it from Python.