Connection Speed Question - Why is it much slower at different times of the day?

I recently changed plans to have up to 600 Meg download, and also replaced the XFinity Xfi router with a Nighthawk CM1150V. I've been testing the connection speed at different times of the day, ensuring that all factors are similar -- during the test ensuring the same number of devices are connected, there's no heavy bandwidth usage, and similar distance from the router. I also have quite a few 2.4 mgz smart devices connected (mostly wi fi bulbs and plugs). 

Download speeds have been as high as 460 Mbps, and Upload Speeds vary from about 12 to 18 Mbps. I usually get those results when I test in the afternoon...

But when I test in the morning, around 5:30 AM, I get download speeds varying from about 35 to 60 Mbps. 

I've tried restarting the modem, but that doesn't solve the issue, and I have consistent results similar to the above over about the past week or two.

I could still return (exchange) the modem, if that could be an issue. 

Any ideas/suggestions on why this may be occurring, and how to address the inconsistent speeds I'm  getting? Thanks.