It takes a lot to shock Chris Goggans; he's been a pen (penetration) tester since 1991, getting paid to break into a wide variety of networks. But he says nothing was as egregious as security lapses in both infrastructure design and patch management at a civilian government agency -- holes that let him hack his way through to a major FBI crime database within a mere six hours.

Goggans, currently senior security consultant at security firm PatchAdvisor, says his adventure started when, during a routine network scan, he discovered a series of unpatched vulnerabilities in the civilian government agency's Web server, as well as other parts of the enterprise.

Goggans used a hole in the Web server to pull down usernames and passwords that were reused on a host of enterprise systems. In those systems, he found further account details that allowed him to get Windows domain administrator privileges -- a classic escalation-of-privileges attack.

Using this privileged access, he was able to gain full control of almost all Windows-based systems in the enterprise, including workstations used by the on-site police force. He noticed that several police workstations had a second networking card installed that used the SNA protocol to directly talk to an IBM mainframe.

By covertly installing remote control software on those workstations, he found programs on their desktops that automatically connected the workstations to the FBI's NCIC database. "That software, coupled with a keystroke capture program, would allow an attacker to grab the credentials needed to log into the FBI's National Crime Information Center database," he says.

Like most vulnerabilities he's found over his years of paid ethical hacking, this one could have easily been eliminated with some basic security strategies, he says. For instance, the police network should have been firewalled off from the main enterprise network, and the investigators' workstations kept out of the larger domain.

Also, he says the agency should not have allowed those workstations both NCIC and general enterprise network access, since they were connected to something with such obvious national security implications. Finally, the system administrators should have monitored and blocked the common reuse of passwords.