You’ve answered your own question. In fact you really want to do conda install ...
instead of using pip if you can.
You can install a conda package also without activating the environment. Just use conda install -n <env_name> <package>
or conda install -p <path/to/env> <package>
.