This is a common use case, and most implementations use TensorFlow’s queues to decouple the preprocessing code from the training code. There is a tutorial on how to use queues, but the main steps are as follows: Define a queue, q, that will buffer the preprocessed data. TensorFlow supports the simple tf.FIFOQueue that produces elements … Read more
As Margaret points out, one of the args is rw. Baseline x86-64 (SSE2) does not include write-prefetch instructions, but they exist as ISA extensions. As usual, compilers won’t use them unless you tell them you’re compiling for a target that supports it. (But they will safely run as a NOP on any non-ancient CPU.) The … Read more
This question isn’t really about compilers as they’re just providing some hook to insert prefetch instructions into your assembly code / binary. Different compilers may provide different intrinsic formats but you can just ignore all these and (carefully) add it directly in assembly code. Now the real question seems to be “when are prefetches useful”, … Read more
Yes, some recent versions of GCC (e.g. 4.9 in march 2015) are able to issue some PREFETCH instruction when optimizing with -O3 (even without any explicit __builtin_prefetch) We don’t know what get_neighbor is doing, and what are the types of v and neigh_val. And prefetching is not always profitable. Adding explicit __builtin_prefetch can slow down … Read more
Jumping directly from block to block is often a win for branch prediction, vs. returning up to one parent indirect-branch, especially on CPUs older than Intel Haswell. With jumps from the tail of each block, each branch has a different branch-predictor history. It’s probably common for a given block to usually jump to the same … Read more
In Django 1.6 and earlier, it is not possible to avoid the extra queries. The prefetch_related call effectively caches the results of a.photoset.all() for every album in the queryset. However, a.photoset.filter(format=1) is a different queryset, so you will generate an extra query for every album. This is explained in the prefetch_related docs. The filter(format=1) is … Read more
Here’s an actual piece of code that I’ve pulled out of a larger project. (Sorry, it’s the shortest one I can find that had a noticable speedup from prefetching.) This code performs a very large data transpose. This example uses the SSE prefetch instructions, which may be the same as the one that GCC emits. … Read more
You can enable or disable the hardware prefetchers using msr-tools http://www.kernel.org/pub/linux/utils/cpu/msr-tools/. The following enables the hardware prefetcher (by unsetting bit 9): [root@… msr-tools-1.2]# ./wrmsr -p 0 0x1a0 0x60628e2089 [root@… msr-tools-1.2]# ./rdmsr 0x1a0 60628e2089 The following disables the hardware prefetcher (by enabling bit 9): [root@… msr-tools-1.2]# ./wrmsr -p 0 0x1a0 0x60628e2289 [root@… msr-tools-1.2]# ./rdmsr 0x1a0 60628e2289 … Read more