For Android Studio 3.1, select the icon below the Build one in the Build window.
By Android Studio 3.3 (possibly in 3.2.1), the icon has changed, though the location is the same:
The build window should open when you run a build action (e.g. from the Build menu). If you don’t see it, you can try the “Build” button along the bottom of the window (also visible in the above screenshots), or through the menus View → Tool Windows → Build.