It only happens on my first load, but doesn't flicker after loading since the font files are now cached on my browser. Just like any other file, font files are needed to be downloaded and that's is why it's happening. It's greatly affected by internet speed but once it's cached on your device, it will stay permanent unless you clear your cache again. https://www.dropbox.com/s/ivhhztmwtdigam5/bandicam%202019-01-06%2020-55-42-234.mp4?dl=0
With system font, it doesn't need to be downloaded so there is no flickering.
And that's all normal, but if you really wish not to see the flickering, then you would need a site preloader. Which will download all the resources including font files before it renders your site. And if you'll implement that feature then I recommend contacting a developer.