Sometimes in a large organization that has offices all over the world and only a small IT security staff, it takes a significant event to reveal security failures in remote offices. This is exactly what happened this week. Until now, it has been fairly peaceful around the office. Other than the regular projects and ongoing issues, there haven't been any fires to put out. But this week, a new wormlike virus took us by surprise.
Normally the IT desktop department handles viruses, but this one involved so many people and so many man-hours that my group ended up getting involved. The worm, Bat.Mumu.A.Worm, or Mumu for short, hasn't taken the spotlight in the same manner as worms such as Melissa or Code Red, but our IT staff had to spend hundreds of man-hours dealing with it. We were taken by surprise because we were focusing on taking preventive measures to avoid being hit with three other viruses: SoBig, Bugbear and Lovgate.
We decided that these viruses had caused enough problems for other organizations that we wanted to be proactive. We spent so much time doing discovery work on what signatures to watch, and looking for updated virus definition files and getting them out to the workforce that we never saw Mumu coming until it had spread.
The three viruses we were originally watching for are similar in that they propagate by using e-mail distribution lists or a Trojan horse-like technique in which the worm attacks servers by scanning for vulnerable workstations. They differ in the messages and names of services, programs and registry keys they create or modify, but all increase network traffic, fill up e-mail in-boxes and prevent legitimate mail from being delivered.
By contrast, Mumu attaches itself and copies its payload to drive shares on remote computers, which in our case have weak administrative passwords. The worm contains a set of batch files, some utility programs and a Trojan horse program that spreads to other computers. It copies a set of files to the vulnerable systems and remotely executes a script or batch file on that system, which sends the Trojan horse to yet more systems. Mumu scans for IP addresses similar to the IP address of the victim system, attempts to access a share via a default password and, if successful, copies over the various files and runs itself again.
Once we knew that some machines were infected, we accessed our Snort intrusion-detection system sensors on the infected network segments and began monitoring traffic.
By monitoring network traffic to several workstations known to be infected, we found common indicators, such as an increase of NetBIOS packets originating from those workstations. There's usually some NetBIOS traffic, but not in excessive amounts, especially on Port 445. Normally, most of it flows through Port 139. Our network engineers responded by limiting the amount of NetBIOS traffic generated, cutting back bandwidth utilization.
After more research, we discovered that almost all the infected workstations were in development departments in remote offices in India and southern California. But since all desktop systems are on the admin network, the worm was also able to propagate to other locations, including corporate headquarters.
Once we determined how to rid ourselves of the worm, we sent instructions to all desktop users. This was a big mistake, because the IT help desk received hundreds of calls from workers who thought their desktops were infected. Of course, the IT security department was copied on every one of these messages.
We also found that all of the desktops at our Indian site either had no virus protection software at all or used an outdated version. And in other remote locations, many desktops didn't have updated virus protection software or users had disabled it in the belief that the antivirus software slowed down their machines.
Eventually, we did rid ourselves of Mumu, but it took both manual steps and a virus definition update, which was released by Symantec Corp. a day after the attack began. (The company also sent a worm removal tool, which has been helpful.)
Now that we have eradicated Mumu, we face the much larger problem of enforcing the installation and proper use of our corporate antivirus software. That will be difficult because many remote locations, especially those overseas, provide their own IT support with no oversight from corporate IT. This needs to change. The Mumu episode attracted so much attention from executive managers that they have arranged a meeting to discuss it. Coming up with a way to enforce a common desktop configuration across the enterprise will be high on my agenda.
Initially, we'll conduct some manual audits by empowering individuals at various locations to review workstations to ensure that the most recent version of our antivirus software is installed and running on every desktop. We'll also have to ensure that the security policy on each desktop doesn't give individuals the ability to stop the antivirus service.
The long-term solution will be to get our hands on some robust, enterprise-class configuration and change-management software and get that infrastructure in place to mitigate future virus problems.