Security researcher Joanna Rutkowska has demonstrated several methods that sophisticated rootkits can use to hide from even the most reliable detection method currently available -- hardware-based products that read a system's RAM.

Rutkowska is a researcher with security firm Coseinc Advanced Malware Labs. She recently outlined several ways of getting around the User Account Control (UAC) feature introduced in Windows Vista. Several researchers have identified problems with UAC.

The demonstration, given at the Black Hat security conference in Arlington, Virginia, indicates that if a rootkit is advanced enough, there currently is no way it can be reliably detected, Rutkowska said. Rootkits are designed to hide some activity from observers, and have recently been used to conceal the presence of Trojans and hacker backdoors -- not to mention Sony BMG's copy-protection software.

Several hardware-based systems exist for acquiring an image of a computer's RAM, the most reliable way to detect the presence of certain kinds of rootkits, Rutkowska said. Those include Tribble, Komoku's CoPilot and RAM Capture Tool from BBN Technologies, but she said none are particularly easy to find.

Rutkowska's findings mean system designers may need to come up with a system architecture better suited to forensic analysis, such as an interface dedicated to memory acquisition.

"We live in the 21st century, but apparently can't reliably read the memory of our computers," she said at the presentation. "Maybe we should rethink the design of our computer systems, so that they were somehow verifiable."

Rutkowska's presentation outlined three types of attacks, one that crashes the machine when RAM acquisition is attempted, a "covering attack" that allows the acquisition tool to see only garbage when it inspects certain parts of physical memory, and a "full replacing attack" that allows the tool to see false information when it scans parts of physical memory.

The denial-of-service attack could leave investigators legally liable for crashing the system, Rutkowska said.

The "covering attack" might allow investigators to see the malware's "hooks" in some cases, but would make it impossible for investigators to analyze the malware. And the "full replacing attack" could simply mean that the malware is never detected at all, she said.