This combination (and values near to these) seems to “magically” work for me to keep the colorbar scaled to the plot, no matter what size the display.
plt.colorbar(im,fraction=0.046, pad=0.04)
It also does not require sharing the axis which can get the plot out of square.