I believe this is because the compiler doesn’t know what type of operation it needs to perform. IIRC, you can get it to work if you introduce boxing:
if (typeof(T) == typeof(int)) return (T)(object)map.GetInt(key);
but that’s not ideal in terms of performance.
I think it’s just a limitation of generics, unfortunately.