Unusually high data usage megathread

I got a call from the corporate office as a result of the BBB complaint.  I got nowhere.  The representative was a very nice person, but I wasn't going to be conviced with tech jargon and generalizations.  Something is wrong.  They basically look at data transfer, packets, etc., they claim the data came from my modem but cannot prove it.  They said that 1000gb of data in 3 days is not alarming...  They said they see it all of the time...  They said a lot of irrelevant things. 

I pointed out that of course they see it all of the time, because that's what these calls respond to.  I emphasized that they are not supposed to tell me whether there is an anomaly based on their experience with other customers because that is generalizing.  They are supposed to perform individualized analysis, look at my history and find the anomaly or issue.  I have steadily, historyically, had 20-30gb of use per day.  I have never used 443gb in one day or 1000gb in three days.

The kicker is he could not explain how I was allegedly using data when my modem was unplugged. My router and all devices were disconnected from the modem overnight for about 10 hours the first night, and completely powered off for about 9 hours the following night, yet registered over 220GB of use during that period.  Impossible.  He said he cannot see when my modem was offline because they don't store that data.  I asked for the report he is looking at to validate the data use, but he said Comcast has a strict privacy policy of not sharing customer data...  That makes no sense because it's my alleged data use.  There is no privacy issue when they are the service provider and I'm asking to see my alleged use.  

He also said Comcast cannot manipulate or alter data use, everything is automated, they have no control... but then said Comcast regularly underreports data use for the benefit of customers...  That makes no sense.  How does an automated system where Comcast has no control over the data use totals, and sends auto warnings and auto bills us for alleged overages, underreport our use?  If Comcast can program their monitoring system to underreport with a math equation, it can certainly work the other way around.

Toward the end of the call, he couldn't come up with any more excuses.  Basically, Comcast can report what they want and don't feel they have to prove it to you, so they can get away with it.

Next step is filing the FCC complaint, which will likely result in a call from the corporate office again.  I will also send an email to as many executives at Comcast that I can find.  In the meantime, we bit the bullet, downgraded speed to lower that cost, enrolled for unlimited data to avoid charges, and we are staying month to month because AT&T fiber is not too far from us.  Some of my neighbors and I are trying to get them to expand their plant to our area ASAP so we can switch.  

@amir32 

Sorry to hear you got the usual "double speak", "stonewalling" treatment. This sounds more like the tactics employed by lower level CCSA.  Hopefully the FCC complaint will net better results since this is the agency that regulates them and not just a  "third party" arbitration agency.

AS to their ability to manipulate the data stream, or reporting there of, how were they able to "throttle back" usage befor they employed data caps? Their conten tion , according to @ComcastTed, is their data recording is handled by  an independent third party , NetForecast. Who does NetForecast get paid by? Why Comcast, of course! Typical Comcast "bull"!

Was coming here to post a new thread about why Xfinity says my data usage is so much higher than my Unifi stats are saying but I see everyone is having the same issue.

As of this writing, it's 16 days into my billing cycle. Xfinity says I've used 525 GB but on only 418 GB have passed through the WAN port between my Unifi UDM and the cable modem. 

39.2 GB UP

379 GB down

The DPI WAN traffic stats to each device add up to the total that have passed through the WAN port. There's no doubt the Unifi stats are accurate. My data usage as reported by Xfinity is inflated by 25.6%!!!!!!!!!

If Xfinity is ALSO correct, then the only way that could happen is if there's more than a 100 GB of data going throgh the modem that never passes through the UDM. ICMP is disabled so it can't be millions of pings taking bandwidth.

I guess the next thing to try is to disconnect the UDM from the modem and leave the modem on and freestanding and see if the data counter incrments on Comcasts side.

And please no replies about disabling port forwarding or anything like. Those suggestions will help reduce data usage. The issue here is that the data used is not what's actually reported by Comcast. Comcast is reporting more data used than is actually used.

Suggestions that prevent data from getting into the cable modem that DON'T get into your router like potentially pings, are useful.

@alanwaterman

1. Hardware/link issues generating traffic from reconnects/rebroadcasts

2. Missing traffic / traffic you may be blind to:

From a remote location on a windows/linux/bsd/mac box, I find nmap to be an extremely useful utility.   Find out how to use it:  https://nmap.org/book/man.html  Attack your modem/gateway's address.  There are many options, and more information on various forums with examples about how to use it with a little searching.  There are many types of scans and some scripting knowledge can be really useful to automate it.

The concern is that your modem/gateway isn't responding correctly to a SYN/ACK or similar type of attack and/or not counting the traffic that hits your firewall if it was blocked. Or even perhaps it doesn't count all protocols (doesn't count UDP??). Traffic counters aren't all that accurate on these cable modems/gateways.  Varies with vendor.

Traffic still might actually exist even though it wasn't counted because your firewall responded incorrectly with "something" as opposed to "nothing" (how it should work).  A proper firewall shouldn't tell a remote location anything.  They should just hit a timeout on their end.  If you find something different, it's a problem.

Try every option.  Try bad tcp/ip flags. Try mangled packets even.  You might find the 26%

@alanwaterman

Maybe they've improved, but you can find quite a few complaints about the inaccuracy of the unifi traffic metering on the web.  To be fair, you can find complaints about the traffic metering in a lot of routers.  My netgear router will seem to track the PCs fine, but do anything on the Roku and it goes crazy.  For example, it says my Roku UPloaded 175,000,000 gigabytes (175 Petabytes) while watching a 30-minute show last night.  (Fortunately, Comcast's meter says otherwise. :)) The Unifi routers also have a feature that causes them to run their own speed tests and they don't include that in their metering.

My automatic speed tests are turned off. I've already gone through the various lists of things to check on Unifi gateways. Even then the speed test data counts toward the WAN port counter even though it doesn't for DPI. The WAN port is not DPI. It's strictly bytes that cross that port. The inaccuracies on traffic stats are relatd to DPI. In my case they happen to match. I know in for many they haven't. 

The real issue here is that the WAN port traffic doesn't match what Xfinity says. That count IS accurate. Even if my gateway is responding with something rather than nothing  from some sort of attack, it will STILL count on the WAN port on my UDM. What won't count is any kind of attack that can pass into the modem that doesn't make it to the UDM via the WAN port. 

Well, here we go again. Recieved a call yesterday from a Comcast agent, Brian. Not sure what department he was in . Sounded like , an IT person, not sales. Must say up front he was extremely courteous, professional, and patient. However he started by questioning me as to what actions I had taken to resolve my sudden surge in data use. I relayed to him, as best as I could remeber, the numerous actions and countless hours I had spent checking settings, running diagnosis, monotoring processes' (both OS and network related), and analyzing data as  to the best of my limited ability. The end result, however, was that he would "open a ticket", I assume to run diagnosis from their end. Fair enough!

For those of you reading this post who have more tech knowledge than me  here is a list of actions I have , or currently , employed:

Modem related

Did a factory reset on modem and reset password ( for the third time this week)

Made sure firewalls were up and operating.

Made sure alerts were set and operating to notify when any device connected to my wifi network, as wll as high traffic alerts.

Checked security protocols ( to the best of my ability) for intrusions, hacks or malware

Checked incoming cable specs, which I really had to educate myself on, pertaing to voltage, SNR,frequency, etc. All seem to be within accepted range

Physically checked all connections and cabeling (no splitters, just a union between street and modem, all secure, home run to modem!

Hardware, software, OS related:

Disabled or uninsatalled all unused programs that  can access the internet in  the background without notification: youcam, one drive, skype, even deleted an old email server that I no longer use!

Checked windows update history for failed attemps, only two over the last three months and they eventually were succesfull.

Checked background processes' for any program downloading or uploading data out of the ordinary. None found.

Checked that all hardware "firmware" is up to date, ( modem, roku, playstation, printer,  ect.)

Have kept all devices using wifi when not in use unplugged.

Completely shut down modem at night (no power, no data transfer)

 Ran every security scan available ( norton, windows defender,ect)

I'm sure there could be lots of things I overlooked, lke I said, limited knowledge.

If anyone has any ideas, please, please, post tem, I will be forever gratefull. I'm about at my wits end!

@Bugg2

I don’t know exactly what kind of stuff you have but a few "generic" things I would try: See if the devices themselves have any metering (Win 10 does) and, if so, see if anything looks odd.  (BTW In Win 10, where you check the data, you can also set a limit.)  I just noticed something "tricky".  When I checked the usage of this Windows 10 machine (wired/desktop), it says 4.5GB in the last 30 days.  Then I grabbed the laptop that I do most of my Skype and Zooming on and it said a ridiculously low 1.15GB.  But I noticed that it was only showing wi-fi use and I always plug in for reliable conferencing.  Once the ethernet was plugged in, it showed the vastly higher amount that had been used on that connection.  Just something to watch out for.  I think I've seen similar confusion on Android about getting wi-fi vs cellular data.

Make sure no foreign devices are connected – especially via MoCA – that one seems to have bitten quite a few people lately. (MoCA is LAN over cable and if that's enabled and there is no MoCA filter on your drop or your neighbors, your MoCA LAN may automagically merge with a neighbor's.)  I see you checked Windows Update history – that's great.  I assume you also checked your update settings – one Win 10 update setting that has bitten some people is update sharing.  If you share with everyone, it can use Terabytes!

After that I would try some elimination.  You've mentioned turning stuff off at night, but since the Xfinity meter may not include data used in the last 24 hours, that may not provide meaningful results.  (Not that it's totally useless – for example if you have a streaming gizmo that just keeps streaming – it would at least take streaming at night out of your total.)

If I had a device (or two) that I suspected, I would start with that/them, otherwise I might just go to a binary search, meaning I would choose half the devices.  In either case, the plan would be: shut down the device(s), wait at LEAST 24 hours, record a baseline data usage, then wait at LEAST 24 more hours (if I was serious – maybe more like a week) and then check the data again and see how it compares to "normal" (ie all devices on.)

If you shutdown one suspect device, then you use the data to confirm your suspicion (or not.) If it's a binary search, then you should get an idea of whether the "offending device(s)" are in the half that were shutdown or not.  Then, whichever half is now suspect, you do the same test with half of those shut down, until you get down to one device.

Alternatively, flatlander3's way would get results faster, if you happen to understand the stuff he's talking about and have some spare equipment.  If not, it may take longer to learn.

I would be suspicious of the playstation only because I have heard of 100GB game updates. 

Unfortunately, in addition to the above, there are still certain mysterious (to me at least) cases where someone said they got a different modem/gateway and their extreme data use went away.  I don’t know how that happens or how to detect it other than replacing the device!

@strega7 

Hey, thanks for the speedy reply.

My network is pretty extensive consisiting of: Desktop (Windows 10),  Laptop: (Windows 8.1), 

Android tablet,  2 Roku devices,  Xfinity Flex box, Smart Blu Ray, network printer, PS4, and 2 Android phones. I know that sounds like a lot of junk but about half of it is hardly being used and stays mostly shut down.

I have checked data use through Windows 10 and it reads as follows:  Ethernet 63MB over the last thirty days(not a typo,MB not GB) Wireless <1MB last thirty days. The majority of use would be through the wireless connection which implies that this function does not record wireless data since this computer running Windows ten is on an ethernet connection. Don't know, kinda confusing. Incidently sharing is turned off so no chance of torrent bit sharing of updates.

I have already done, basically, what you described as binary. When I noticed data suddenly spiking this month I systematically began shutting down devices for well over a week now. I monitor the netwok map multiple times a day to check for devices connecting to the network and have alerts set to notify me when they do, so i definately know what is connected when.  The playstaion in question has internet connection turned off since majority of gaming is done off-line, plus it is blocked from connecting to the internet for downloads without permission. No background downloads! So recently, Flex box , smart blu ray, network printer, and playstaion have no access to wifi and no changes in data use have been revealed which tells me if there is a problem on my end it's not on any of these devices.  Next step will be the android phones which is not a big issue since we can use the data off our plan, although we "cheaped out" on our data plan since we are home most of the time and can use wifi. After that it's on to the roku's. If that don't work I guess we'll have to drop back and "punt".

Thnaks again for all your help.

@alanwaterman

"What won't count is any kind of attack that can pass into the modem that doesn't make it to the UDM via the WAN port. "

Yeah.  That's exactly one of the situtations that I'm trying to trip up with a gateway/modem that connects directly to a firewall port.  I have a seperate box with 2 ethernet ports.  External firewall network interface goes directly to the gateway/modem (the hive of scum and villainy that is the comcast network).  The firewall box -- which is just a BSD OS running the ole BSD version of pf(4).  And an internal network interface on that box connects to everything else.  Put your wan port on your Unifi gear on that.

So pretty much, I figure the firewall on the gateway/modem is junk.  It's going to get a remote exploit at some point.  It may not handle traffic correctly to a port I can't do anything with.  Maybe it doesn't deal with a portscan well and they can exhaust its resources remotely and "tip it over" by some means.  Dunno.  The little counter page on the gateway/modem - I know that's not accurate.  Transfer some known size traffic at it on yours and see if yours counts it correctly, you may be surprised.

Theory goes like this:  So say the gateway/modem may be actually masking traffic for whatever reason.  It's not counting right.  And I'm 26% off.  The traffic is real, it's not friendly, I can't see it.    How do you get the remote script to dump, stop banging on my gateway/modem and move on to someone else?  

Welp.  Most of the scum and villiany I see are script kids running megasploit or some script they found on the web.  How do you mess em up.....erm.....legally?  Open a couple of ports from the gateway and forward them to the firewall.  Pick a low one, and a high one.  Doesn't matter what you pick.

Now install Snort.  You can make dynamic rule sets yourself with pf(4) or ipchains...etc, but it's a lot of work.  Snort creates them for you, and can automatically remove rules after a set time period (if you want), and you pick what kind of traffic is allowed or not. 

Great.  Let's say you got port 83 and 8050 open on your gateway.  On a portscan -- which is the most common, they hit the port, it goes to the firewall and immediately sticks since you have no running service.  The remote script hangs when that happens.  Sometimes for a really long time if they don't have error handling.  That's fine.  Eventually their script times out and you're done with min traffic exchanged -- they were looking for a port, didn't find the droids they were looking for.  It's done.

As a bonus, Snort clobbered the IP address the attack came from, so if anything on your network did take traffic from the remote location, it's cut off -- forever if you want.  An hour.  Days.  Whatever.  You don't want to talk to them if they hit your honeypot ports.

Say for example, there is a known exploit on your gateway/modem and they tipped it over.  They did on an old DSL Zytel I had on CenturyLink.  What do they do next?  Open up all the ports looking for stuff.  You don't care though, all other ports just hit your firewall and get dynamically blocked by Snort.  Neat.

Does it work?  Yeah.  My traffic went down by 1/4 immediately.  With some traffic shaping, I can cut that to 1/3 of what I was using before.  I also found some oddities on my gear too.  Foscam cameras like to spew UDP looking for myfoscam.com for example.  Other things I don't like -- now I can block.  No, Microsoft doesn't need to know if I have a printer every 2 minutes.....gahd.....

Dunno.  Works for me.  Advice is cheap.

16.75 days into billing cycle. 562 GB reported by Xfinity.

WAN port going straight to modem 44.2 GB up / 411.2 GB down = 455.4 GB = 23.4 % inflated so it's narrowed a little bit. 

@Bugg2 

If you're looking at the data usage through the Network & Internet Setings box, then you may have to change the connection when looking at the data usage settings.

My desktop can handle both wires and wireless.

I started to use wireless via Hotspot when I was close to 1.2TB and noticed that I had to manually select the wireless interface to get its data usage.

Windows 10 was definitely recording wireless data usage for me and it was different than what was recorded for the wired ethernet jack.

Does anyone know of a way to limit bandwidth to individual wifi devices other than Qos. I checked my router and it doesn't offer that function. Preferably a cheap, or better yet free, program that works with Windows 10 connected via ethernet. Since talking to Comcast agent the other day it's sent me into "full blown" seek and destroy mode. I'm freaking determined they're gonna prove me wrong or I'm gonna prove them wrong. This "hangin' in limbo junk is for the birds.

Thanks!