Hey Godfather,
The theme is highly likely not causing this issue. I can’t replicate this in my test site.
I suspect 1 of the third party plugins is altering the URL of images. I understand that you tested for a plugin conflict before but please deactivate your photo gallery plugin and then recreate your images with caption again. I suspect there’s a setting in the third party plugin that causes this. The setting most probably is Enable href attribute.

If that does not help, please deactivate all third party plugins and then create a test page with images with captions and monitor if the theme is that is really causing the issue.
Thanks.