​​ The Networks Team: Keeping DePaul Connected

The Networks Team works behind the scenes to keep our networks running and secure

The goal of the Networks team at DePaul isn’t exactly intuitive: they believe they’re most successful when the university community doesn’t notice their work. This group of dedicated staff members within Information Services ensures that the wired and wireless networks throughout Lincoln Park, Loop, Cinespace, and other shared DePaul campuses and spaces are working seamlessly. They plan for DePaul’s network needs, resiliency, redundancy, and future data demands so the rest of us can rely on a network that works whenever we need it.

The physical equipment that comprises our wired and wireless networks needs to reflect a complex puzzle that includes data capacity, installation costs, and anticipated future needs.

They think about issues that most people who use the network may never imagine:

• What is the extra burden on the WiFi network in between classes when thousands of devices are roaming and reauthenticating to new WiFi access points all at the same time?
• How can the university reduce its dependence on older Cisco office phones and the thousands of physical phone jacks that need to be maintained?
• How can students get reliable WiFi access on the Quad?
• How can Cinespace’s network be connected to the rest of DePaul to create a seamless campus experience?
• What happens to our networks when the power goes out?
• What’s the optimal physical cabling needs that we should be deploying – physical infrastructure is relied upon for an extended period of time and is considered a long-term asset of our facilities.
• What happens to the network when one of the critical data cables is cut with a concrete saw?

The team learned the answer to the last question the hard way.

Unfortunately, sometimes accidents happen. In 2017, some routine utility work led to a major network outage. In order to understand this event, it’s important to explain that DePaul’s extensive network includes cables that run between different locations on campus called clusters. These core network clusters are central routing stations for all the data that travels over the DePaul network and beyond. The university has multiple clusters on campus, and these clusters are connected by fiber optic cables that run underground.

Standards for how deep these cables should be buried can depend on the location and when they were installed. As you can imagine, the layers of utility lines, old infrastructure, and obstacles underneath the streets and properties of Chicago can be a tangled web of complexity that has accumulated over many years and different modes of transportation and utility usage. 

At one spot under Fullerton Avenue, a data cable connected one of DePaul’s distribution network clusters to the core network. Although it was buried below the required depth, an old streetcar track prevented it from being placed as deep as most university lines.

This wasn’t a problem—until a winter day in 2017, when a phone utility company began cutting into the street to install new lines. The crew accidentally sliced through one of DePaul’s critical cables, knocking out network service to 990 Fullerton and Seton Hall. That meant WiFi, wired networking, and phone service immediately became unavailable to those buildings.

Although the Networks team quickly discovered the issue, they didn’t immediately know why the network was down in part of the campus. They quickly ran some diagnostic testing and suspected the data line had been cut. They contacted Public Safety, checked out some security cameras, and saw the image below. 

The Networks team reached out to one of our utility service partners who quickly arrived at the location of the suspected cable cut and assessed the damage. The data lines had been completely severed and due to some previous damage as well, it would take three labor-intensive days to repair the run, pull new cable, and get the cluster back up and running. 

Three days was too long for both buildings to be without service, so they came up with a workaround. While a service company installed new lines, the team rerouted connections to the rooftops of two nearby buildings on either side of the damaged cable. They set up laser media converters (pictured below) on each roof, aiming them at one another. These devices converted the fiber data into light and radio signals, sent it across the gap, and then converted it back into fiber on the other side. This setup allowed network traffic between the two Lincoln Park buildings to continue—effectively sending data by laser over Sheffield Avenue.

From the time the data line was cut to the time the laser converters were transmitting data once again took a little over 24 hours. People in Seton Hall and 990 Fullerton probably didn’t know how the network came back online. They just noticed that it was working again. But within a single day, the Networks team had improvised a plan to transmit data through a laser while the service company scrambled to rerun the data line below.

Eventually, new data lines were connected, the network came back online, and the laser media converters were switched off. Most people notice the network downtime, but few see the work that happens after a network catastrophe to recover as soon as possible.

A few years later, DePaul embarked on an important fiber modernization project that replaced cables, added faster WiFi access points, expanded capacity, and improved network redundancy. Although the cut data line under Fullerton was disruptive, it helped to inform a new era of network redundancy. The modernization project created extra connections between network clusters. These improvements dramatically reduced the chances of another outage like the one in 2017.

DePaul’s Networks team tries to plan for these potential issues before they happen. In an era of rapidly increasing dependence on the WiFi network for all kinds of cloud services, connected devices, and connected apps, they work to anticipate these demands so that DePaul’s community can learn, teach, and work without interruption.

Accidents do happen—but they’re rare. Most days, the network just works, and that reliability is easy to take for granted. It sometimes takes a major disruption, like a big diamond bladed concrete saw cutting through the middle of Fullerton, to remind us how dependable it usually is. Behind that consistency is constant planning and effort to support tens of thousands of users and devices every day.

The Networks team knows demand will only grow, with more devices, more data, and more reliance on cloud and streaming services. They’re always planning for what’s next, and they believe if they do their job right, the rest of us may not notice much at all—just a steady, reliable network.

More images from the network outage: