Community Forum
I am having the very same issue, doesn't matter the time of day or night. It's like it drops off to nothing, but it is very annoying. Didn't used to have this issue until we added TV service. Something is definitely swirly.
Start here;
https://forums.xfinity.com/t5/Your-Home-Network/Internet-Troubleshooting-Tips/m-p/3310447#M316652