Instrumentation

"I don't think we expose enough stuff to sophisticated programmers to tell them what's going on in the kernel," Morton says. But of the three big holes, this is probably the one with the most work going on. Morton listed quite a few high spots in instrumentation, including per-task I/O accounting and per-process memory footprint monitoring.

He also mentioned Matt Mackall's "PSS" and "USS" metrics for memory used by a process as a good step forward. Current memory-use stats don't break out how many processes share a page of memory, so a user can't get an accurate measurement to answer a simple question, such as "How much memory will I free up if I kill this process?" Mackall's work makes it easier to assign memory-hogging blame to programs.

Currently, IA-64 Linux has access to that platform's hardware-performance counters via perfmon, and Morton says, "We'll get there eventually" for other platforms.

Torvalds doesn't see any hurry.

The problem is that, yes, we could get better data, but quite often, the gap between "data" and "information" then ends up being so big that it's not necessarily always even worth getting the data -- because turning it into information won't happen.

For example, we can get a _lot_ of profiling events on pretty much all modern CPUs, and you can do things like cache miss profiling, etc. In the end, most of the real performance problems end up being at a higher level, and in fact, simple cycle profiles tend to be quite sufficient to get enough low-level performance to know what the low-level problem is: and getting to the *high-level* problem is usually not about getting more low-level data.

So as far as I'm concerned, this is more of a "we could do better in gathering some data," but at the same time, I personally suspect that people already aren't even taking full advantage of the data that we *do* offer, and that sometimes the "I don't have information about detail XYZ" is used as an excuse for not doing the things you can already do.

Perhaps what Linux needs at this point is some good old-fashioned IT industry Fear, Uncertainty and Doubt. When Linux on Apache lost out to a Microsoft OS and Web server in tests run by Mindcraft in 1999, the competition made a PR splash of it.

It was a short-lived one. "Two months after the benchmark, the TCP stack was fully SMP threaded; five months after that the entire networking subsystem was fully parallelized. The result: last month's Specweb results, which blow away everything else," Corbet wrote a year later.