I was missing the AspNetCoreModule from IIS->Modules. After I installed that I no longer had the 500.19 error.
To find out more this blog was great: https://weblog.west-wind.com/posts/2016/Jun/06/Publishing-and-Running-ASPNET-Core-Applications-with-IIS
Here is the page from Microsoft to find the download: https://docs.microsoft.com/en-us/aspnet/core/publishing/iis?tabs=aspnetcore2x