Can't press portal button after boss fight

After fighting Sheliak's Yoke and collecting rewards, I am unable to press the glowing blue button that should activate the portal out. It shows a white dot like interactable objects should when I get close, but never get the option to activate it with the interaction button. Used quick travel to progress, so not actually stuck there at the moment.