To change the color scheme/fonts of the website modify the
theme in the _site.yml file (see here for
options):

The small image that shows up on the browser can also be changed.
You can make a small image to replace the existing one by going to https://favicon.io/favicon-converter/ and uploading an image that you would like.
Next, simply replace the image called favicon.ico in the
images directory within the resources
directory with the image you just created and downloaded from the
favicon converter website.
To make additional changes to the style, you can modify the
styles.css file with css code. This website has great information
about css code.
As an example if you wanted to change the color of the blue line to
green you could change where it says lightblue to
lightgreen in the styles.css file. You can
also use a hex color code like those that can be found at this website, such as
#00FF9E to get a specific shade.

Note that if you change the css file with a new element that is not
already defined like body then you would need to do it as
done with the banner element. This was then added to the
index.Rmd file by using:
<div class = "banner">
Banner text!
</div>
Also checkout the R Markdown cheatsheet for more customization of the pages.