creating a DialogPreference from XML

Here is an example of how to use the dialog preference (subclassing as you mentioned).

package dk.myapp.views;

import android.content.Context;
import android.preference.DialogPreference;
import android.util.AttributeSet;

/**
 * The OptionDialogPreference will display a dialog, and will persist the
 * <code>true</code> when pressing the positive button and <code>false</code>
 * otherwise. It will persist to the android:key specified in xml-preference.
 */
public class OptionDialogPreference extends DialogPreference {

    public OptionDialogPreference(Context context, AttributeSet attrs) {
        super(context, attrs);
    }

    @Override
    protected void onDialogClosed(boolean positiveResult) {
        super.onDialogClosed(positiveResult);
        persistBoolean(positiveResult);
    }

}

The preferences.xml should contain

<dk.myapp.views.OptionDialogPreference
            android:key="@string/prefKeyResetQuests"
            android:dialogIcon="@android:drawable/ic_dialog_alert"
            android:title="Reset Quests"
            android:summary="Reset all quest-progress."
            android:dialogMessage="Are you sure you wish to reset your quest progress? This action cannot be undone!"
            android:positiveButtonText="Clear Quests"
            android:negativeButtonText="Cancel"/>

I have a res/value containing (though the key-name can also be declared explicitly above).

<string name="prefKeyResetQuests">resetQuests</string>  

My PreferenceActivity does the actual resetting from onPause. Note that onStop may be too late since the onStop will not always be called immediately after pressing back:

@Override
public void onPause() {
    SharedPreferences prefs = android.preference.PreferenceManager.
        getDefaultSharedPreferences(getBaseContext());
    if(prefs.getBoolean(
        getResources().getString(R.string.prefKeyResetQuests), false)) {
        // apply reset, and then set the pref-value back to false
    }
}

Or equivalently since we are still in the PreferenceActivity:

@Override
protected void onPause() {
    Preference prefResetQuests =
        findPreference(getResources().getString(R.string.prefKeyResetQuests));
    if(prefResetQuests.getSharedPreferences().
        getBoolean(prefResetQuests.getKey(), false)){
        // apply reset, and then set the pref-value back to false 
    }
}