Random High Latency Spikes

I've been getting lots of random latency spikes when playing Overwatch 2 or Team Fortress 2. In OW, I have latency shown and it will randomly go from sitting at 50 ms then spike up to 200 ms. This happens in other games as well. I've done ping tests and seen no packet loss or anything strange. I'm not too knowledgeable in this but I have no idea what could be causing this. Also not on wifi, I have an ethernet cable connected to my pc to the XB6 modem.