This answer solved my problem:
First of all, commit all pending changes.
Then run this command:
git rm -r --cached .
This removes everything from the index, then just run:
git add .
Commit it:
git commit -m ".gitignore is now working"
Please be careful, when you push this to a repository and pull from somewhere else into a state where those files are still tracked, the files will be DELETED