Hey Noelle,
We have replicated what you described. The under-construction page though is a special page. That page just houses or outputs the page (e.g. The custom Under Construction page in your site.) you assign to it. That means though you’re seeing your custom under construction page, the address is not actually the under construction page. Hope that makes sense.
The Header and Footer assignments work with the page ID and your Under Construction page, when viewed in incognito, does not use its page ID since it’s housed or displayed only by the Under Construction plugin.
There’s currently no assignment for the Under Construction page so it will use the Global or the Front Page assignment. Because this is a problem for you, I’d recommend removing the Header and Footer assignments to your Under Construction page and set your UC Page Template to Blank No Container | No Header No Footer and recreate the footer within the Content Builder. Don’t assign a Header to the Front Page or Globally though as I just tested the Header is still showing up even the No Header template is used. I’ll report this case along with the bug I’ve discovered.
Alternatively, if you’re designing your home page, don’t use it as your Front Page just yet so you can assign a different Header and Footer to it. Then, just assign the Footer that’s supposed to be for you UC page to the temporary Front Page. The only downside with this is you don’t actually get to use the root domain for the front page.
Hope that helps.