OK, found the answer here:
Refreshing the display from a widget?
Basically, have to make a transparent activity that processes the brightness change. What’s not mentioned in the post is that you have to do:
Settings.System.putInt(y.getContentResolver(),Settings.System.SCREEN_BRIGHTNESS_MODE, 0);
Settings.System.putInt(y.getContentResolver(),Settings.System.SCREEN_BRIGHTNESS, brightnessLevel);
then do
WindowManager.LayoutParams lp = getWindow().getAttributes();
lp.screenBrightness = brightness;
getWindow().setAttributes(lp);
And if you call finish() right after applying the changes, brightness will never actually change because the layout has to be created before the brightness settings is applied. So I ended up creating a thread that had a 300ms delay, then called finish().