Random ping spikes and packet loss during the day

Ever since late March my connection speeds have slowly gotten worse. When I run a pingplotter my round trip ms is way over 100ms, latency over 400ms and huge packet loss. I normally woud think it's my hardware but during the times from 1-5 am I get normal connection (round trip ms sub 30, latency 90ms and zero packet loss. 

I have a 600mbps download and 35 mbps upload package and those speeds are perfectly normal, these aren't the problem. I have called customer service mulitple times for them to tell me it's all good when clearly it's not. I asked if someone could come to my house but due to COVID-19 I was told that service is unavailable. I hope someone can help me get to the bottom of this because it's getting worse before it gets better.