If you really want to replace the fragment then use replace() methode instead of doing a remove() and an add().
FragmentManager fragmentManager = getFragmentManager();
FragmentTransaction fragmentTransaction = fragmentManager.beginTransaction();
fragmentTransaction.replace(..............);
fragmentTransaction.addToBackStack(null);
fragmentTransaction.commit();
Don’t forget to do the addToBackStack(null) so your previous state will be added to the backstack allowing you to go back with the back button.
Another good source is http://developer.android.com/guide/components/fragments.html (search for replace() function).