As of writing this post it was not possible to add a CSP in the http header for github.io pages. As an alternativ you can add a CSP in the HTML meta tag. The only limitation is that you can’t use frame-ancestors, report-uri, or sandbox features of CSP.
Add an folder _includes to your jekyll blog. Then add a head.html file in that folder. Get the original head.html of the jekyll theme you are using with the gem path. You can find the name of your theme in your _config.yml file.
Once you you got the path navigate into the _include folder and copy the contents of the head.html into the head.html file your created earlier. Then just add you CSP in as a meta tag.