While it looks like magic offhand items affect the magic score, the effect vanishes when casting spells. I have a strong magical umbrella, but whether I have it equipped or not, my spell damage on enemies remains unchanged. I also see my stamina and health bars briefly reduced to a much lower values while casting.
