I love using the built-in Application Settings. Then you have built in support for using the settings designer if you want at design-time, or at runtime to use:
// read setting
string setting1 = (string)Settings.Default["MySetting1"];
// save setting
Settings.Default["MySetting2"] = "My Setting Value";
// you can force a save with
Properties.Settings.Default.Save();
It does store the settings in a similar folder structure as you describe (with the version in the path). However, with a simple call to:
Properties.Settings.Default.Upgrade();
The app will pull all previous versions settings in to save in.