Not getting anywhere close to internet speeds I am paying for from 12p-12a

I live in an off campus University housing neighborhood so I’m assuming lots of people here are using Xfinity, but ever since school started up and everyone moved back in, my wifi speeds have gotten worse and worse. 

I pay for 500mbps download and I am often getting <50mbps between 12p-12a and <3mbps upload. My latency has also been frequently spiking to 1000-2000ms making playing games or streaming virtually impossible. 

I’m wondering if there’s a fix for this besides “upgrading my plan” because I don’t even get what I pay for now so why would I want to pay more.