Comcast experienced a city-wide outage this morning—affecting probably hundreds of thousands of people. Outages like this can be confusing to the tech-savvy and regular users alike, so I thought it'd be useful to share how I came to understand the problem so that I might help others.
Something's up (or down?)
I first started seeing issues this morning while I was making coffee breakfast. I tried to start WBEZ on our kitchen Sonos, but my station wasn't playing. Troubleshooting started as most early-morning fixes do... I just mashed the play button a bunch of times thinking it might work.
It didn't. If one feed didn't work, maybe another would? I searched and the Sonos app listed a number of sources, so I tried another, and another. Finally, one worked... and then cut out again a minute later. Maybe a different station? Nope.
So... it looked like the speaker and my phone were fine, but something was going on upstream.
Expanding the search
Many folks' assumption, in this case, is "it's just me"—whether that means "my phone," "my house," or something else. Troubleshooting often makes sense going smallest to largest.
I next checked my home internet... I'm a little old-school and still using Apple AirPort Extremes for my home network. I opened the AirPort Utility app and it showed a big orange dot next to "Internet" but green on my access points. Cool, my internal network is ok but the world is not.
Next up is the connection between my house and the world—my modem. Whether you have an Xfinity-provided modem or bring your own (like I do), their app (if you can reach it) will tell you if there's a detectable issue with your modem. Since we recently switched to T-Mobile from Verizon and now actually have a cell signal in our home, I flipped my Wi-Fi off and pulled up the Xfinity app.
The app showed me two things:
- There's no connection between my modem and the internet.
- (More importantly) There's a big problem.
Without #2 I could have tried unplugging and replugging the modem, but since there was a wider outage there was no need.
Rounding out the picture
Xfinity reported an outage, and I was able to get bits and pieces of information from their site as it came and went. A quick Twitter search for "Xfinity Chicago" reported many others with similar issues within the past 30 minutes—a sure sign that "it's not just me". I also corroborated on DownDetector which shows a map of user-reported outages.
With the issue confirmed, I emailed my neighbors from my phone with news of the outage and a link for them to track the status of the issue. Shortly after, Block Club Chicago reported similarly.
After all that, I put my devices down and... went outside.
To recap
Understanding the incredibly complex systems that keep us all connected these days is impossible, even for the most tech-savvy of us. In any complex system, here are a few things I try to remember when working through an issue:
- start small, then widen your view.
- don't bang your head on one specific issue when you haven't ruled out others.
- know where to go for help.
- when you've come to a solution, share it broadly.
One quick email, or even a tweet, can help countless others by saving time, reducing facepalms, and helping us get back to our lives.