Hi there,
Thanks for writing in! This is normal behavior for any caching program. What it does is saves parts of your site in a cache and loads it from there to improve performance. When making changes and your caching plugin or scripts doesn’t recognize the change it wont refresh the cache. When you site loads it will load the parts on the cache with priority. It is best to clear or purge your cache, then deactivate you plugin when working on your site. This is to avoid such incidents you are experiencing. You can turn caching back on when you are done editing your site.
Hope this helps – thanks!