The following solution works for both proxied and unproxied servers. So if you are using CloudFlare, AWS Elastic Load Balancing, Heroku, OpenShift or any other Cloud/PaaS solution and you are experiencing redirect loops with normal HTTPS redirects, give it a try.
RewriteEngine On
# If we receive a forwarded http request from a proxy...
RewriteCond %{HTTP:X-Forwarded-Proto} =http [OR]
# ...or just a plain old http request directly from the client
RewriteCond %{HTTP:X-Forwarded-Proto} =""
RewriteCond %{HTTPS} !=on
# Redirect to https version
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
# Put the rest of your rewrite rules here