Recently, the Fermilab Information Technology Division’s Network Services group members Tim Niemiec and Alan Varghese, along with Bob Sieloff, IT liaison to DUNE, made a weeklong trip to the Sanford Underground Research Facility, the future home of the DUNE far detector. Network Services worked on several upgrade projects to expand and improve the Fermilab network at SURF in Lead, South Dakota. The Fermilab network at SURF, which includes underground tunnels, is treated as and behaves like an extension of the local Batavia Fermilab network.
SURF’s network infrastructure must be put in place before scientists arrive to build the detectors and conduct their research, but this infrastructure must also be available to excavation crews and others currently working at the site for safety and communication. As the IT liaison, Sieloff serves as the point of contact between SURF and the Fermilab Information Technology Division.
“We have had several weeklong trips to SURF since 2019 to build out the general network infrastructure there and to make a working environment similar to the Batavia site,” said Andrey Bobyshev, Network Services group leader. “Last October, Tim [Niemiec] and Alan [Varghese] worked on expanding the seismic sensor network for excavation work, upgraded wireless network, installed required updates to software on routers and switches, and isolated a fiber fault in the nearly mile-long shaft leading to the tunnels. The trip’s success was due to six months of planning and remote support by staff back in Batavia.”
Among the trip’s biggest accomplishments was the deployment of the new, centrally managed Wi-Fi infrastructure in the Ross Dry building and the underground tunnels. Network Services members Adam West and Greg Stonehocker pre-staged the new system before the trip and then worked remotely with Niemiec and Varghese to deploy it in production at SURF. As a result, the wireless service has been extended underground for added convenience of the construction and operations crews and now allows for video-conferencing and other common uses of wireless capabilities deep underground.
Working on a project of this scale is not without its challenges.
“Working a mile underground takes some getting used to,” said Niemiec. “In addition to the usual equipment we need, there are respirators and other special safety gear that allow us to work in very dusty and near-blackout conditions.”
There are also challenges with deploying devices so far underground.
“Before deploying devices, we had to test them to make sure they worked properly,” said Varghese. “Underground, if the device doesn’t work, there’s no way you can fix it. You have to come up, and you don’t know when you’ll get the chance to go down again. Everything needs to be checked very carefully and tested as much as possible.”
But for now, being a part of laying the groundwork for DUNE is very exciting for all involved. In addition to this work, the team is also expanding wide-area network connectivity between Fermilab and SURF. Led by Phil Demar and Eric Smith from Fermilab and ESnet respectively, the team is working with the national fiber providers to bring two diverse 100G fibers between the two locations in order to support wide-area networking for DUNE and the Long-Baseline Neutrino Facility. In the meantime, a new 10G End-to-End circuit provisioned in January 2022 across several R&D wide-area networks currently serves as the primary path from Fermilab to SURF.
Original source can be found here.
