How do I remove default markers?

The only markers that should show up on the map are those you add yourself. Care to share your code or a page where we can see this happening?

Update: ok, these aren’t really ‘markers’ in the normal sense of the word, they’re just points of interest, which happen to behave like markers in that you can click on them and see infowindows. It seems to me that these might be of the class MapTypeStyleFeatureType, probably of types like poi.medical, poi.park, transit.station.rail and so on. I wonder if you could use the MapTypeStyle. Maybe something like this:

var myStyles =[
    {
        featureType: "poi",
        elementType: "labels",
        stylers: [
              { visibility: "off" }
        ]
    }
];

var myOptions = {
    zoom: 10,
    center: homeLatlng,
    mapTypeId: google.maps.MapTypeId.ROADMAP,
    styles: myStyles 
};

You might also want to look at the Styled Map Wizard

Update, July 2016: The Maps API also now has an option you can specify in the MapOptions, clickableIcons, which if you set to false, the icons for these POIs will appear but clicking them doesn’t open Google’s infowindows. This saves you having to set the styles to hide the icons unless you want to, if all you need to do is prevent the clicks opening the infowindows.
Just set clickableIcons: false in the options you initialise the Map with.