After checking back with RxJava developer Dávid Karnok I’d like to propose a full explanation of what was going on here.
share() is defined as publish().refCount(), i. e. the source Observable is first transformed to a ConnectableObservable by publish() but instead of having to call connect() “manually” that part is handled by refCount(). In particular, refCount will call connect() on the ConnectableObservable when it itself receives the first subscription; then, as long as there is at least one subscriber it will stay subscribed; and, finally, when the number of subscribers drops to 0 it will unsubscribe upwards. With cold Observables, like the ones returned by Retrofit, this will stop any running computations.
If, after one of these cycles another subscriber comes along, refCount will again call connect and thus trigger a new subscription to the source Observable. In this case, it will trigger another network request.
Now, this usually did not become apparent with Retrofit 1 (and indeed any version before this commit), because these older versions of Retrofit by default moved all network requests to another thread. This usually meant that all your subscribe() calls would happen while the first request/Observable was still running and therefore the new Subscribers would simply be added to the refCount and therefore would not trigger additional requests/Observables.
Newer versions of Retrofit, however, do not by default move the work to another thread anymore – you have to do that explicitly by calling, for example, subscribeOn(Schedulers.io()). If you don’t, everything will just stay on the current thread, meaning that the second subscribe() will only happen after the first Observable has called onCompleted and therefore after all Subscribers have unsubscribed and everything is shut down. Now, as we saw in the first paragraph, when the second subscribe() is called, share() has no choice but to cause another Subscription to the source Observable and trigger another network request.
So, to go back to the behavior you are used to from Retrofit 1, just add subscribeOn(Schedulers.io()).
This should result in only network request being executed – most of the time. In principle though, you could still get multiple requests (and you always could have with Retrofit 1), but only if your network requests are extremely fast and/or the subscribe() calls happen with considerable delay, so that, again, the first request is finished when the second subscribe() happens.
Therefore, Dávid suggests to either use cache() (but it has the drawbacks you mentioned) or replay().autoConnect(). According to these release notes, autoConnect works like only the first half of refCount, or more precisely, it is
similar in behavior to refCount(), except that it doesn’t disconnect
when subscribers are lost.
This means the request would only be triggered when the first subscribe() happens but then all later Subscribers would receive all emitted items, regardless of whether there were, at any time in between, 0 subscribers.