I can't receive my outlook email when plugged into AT&T Modem.

About 4 months ago we added AT&T Fiber Internet to our home. (we still have comcast/xfinity) My wife works from home and Xfinity's reliability in my area is less than desired. 

I've never had any issues with my "Outlook" comcast e-mail account. It's always sent & received as it should. BUT.. when I'm plugged into the internet via the AT&T modem, my outlook can't receive my comcast mail. I can access it from the web, but it doesn't store anything in my outlook like I prefer. I can plug my laptop into the Xfinity modem directly and it works correct again.

I've barely messed with e-mail settings (mail, pop, pop3, etc), but changed them back when it didn't fix the issue.

We plan on keeping both services for now, so it would be nice to have to switch modems to download e-mail into Outlook.