You have to calculate the distance to each color, and pick the smallest.
There are a few ways to do this. A simple method would be to calculate the distance would be:
sqrt((r-r1)^2+(g-g1)^2+(b-b1)^2)
A better method might be to incorporate the weighted values to calculate a distance, for instance the values used when converting RGB->YUV:
Y = 0.299 * R + 0.587 * G + 0.114 * B
in that case you would use
sqrt(((r - r1) * .299)^2 + ((g - g1) * .587)^2 + ((b - b1) * .114)^2)
Of course, since you don’t need the exact distances, just a comparison, you can and probably should just skip the square root, making the last calculation:
((r - r1) * .299)^2 + ((g - g1) * .587)^2 + ((b - b1) * .114)^2