Hi there – the easiest way to bring a new font into a WordPress site is by using a Google Fonts plugin.
Caveat is available via Google Fonts:
https://www.google.com/fonts/specimen/Caveat
You can then target elements with custom CSS to override the default font with your new one. Here are some very helpful posts that will help you customize your site with CSS:
https://dailypost.wordpress.com/2013/07/25/css-selectors/
http://dailypost.wordpress.com/2013/06/21/css-intro/
http://dailypost.wordpress.com/2013/08/29/css-matched-rule-pane/
http://en.support.wordpress.com/custom-design/how-to-find-your-themes-css/
https://thewc.co/articles/view/web-inspector-tutorial
https://cssworkshop.wordpress.com/
If you get stuck targeting a particular element with CSS just let us know, and provide a link to your site so we can better assist.
When you’re ready to add your custom CSS overrides, don’t edit the theme files directly, otherwise your changes will be overwritten every time the theme is updated to the latest version.
An easy way to add custom CSS is to install the Jetpack plugin and activate the Custom CSS module. You’ll then add your custom CSS in the new stylesheet editor that’ll appear in your dashboard, under Appearance > Edit CSS.
As alternatives, you could either install a standalone custom CSS plugin, or create a child theme.