Here is a trick to access image in Resource file:
Accessing image from Resource File in XAML markup
First you need to add reference to project properties like this:
xmlns:properties="clr-namespace:MyProject.Properties"
And then access it via XAML like this:
<image source="{Binding Source={x:Static properties:Resources.ImageName}}" />
You can use PNG/JPG/BMP as well as ICO file but everyone recommend PNG.