This is a very easy fix. I use cloudflare myself and use this to convert http traffic to ssl.. it's ready to go as is.
Code:
RewriteEngine on
#Options +FollowSymLinks
################https##############
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
################https##############
<ifModule mod_headers.c>
Header always set Content-Security-Policy "upgrade-insecure-requests;"
</IfModule>