I would say that multiple Activities almost always makes more sense. I just don’t think Android is designed for constantly switching its own views – you miss out on so much. You have to implement Back yourself, you don’t get any inter-Activity transitions, you have to implement a lot of internal logic to resume an application in the correct state. If you don’t partition your app into Activities, it makes it a lot more difficult later on to change the flow of your application. It also results in one mega-Activity that can be a lot harder to handle than a lot of smaller pieces of code.
I have trouble imagining that speed is really an issue; if it is then there’s something wrong with the way you’re initializing each Activity. For example, I used try to pass Serializable objects between Activities, and that proved to be incredibly slow; when I switched to a faster method of passing objects, the speed of launching Activities increased immensely.
Also, I think it’s telling that the Android guidelines for Activity and Task Design don’t mention switching Views at all; it’s centered around an Activity-as-View design.