King of the NOS Hill: Windows 2000 Reigns Supreme

It all boils down to what you're looking for in a network operating system (NOS).

Do you want it lean and flexible so you can install it any way you please?

Perhaps administration bells and management whistles are what you need so you can deploy several hundred servers. Or maybe you want an operating system that's robust enough so that you sleep like a baby at night?

The good news is that there is a NOS waiting just for you. After the rash of recent software revisions, we took an in-depth look at four of the major NOSes on the market: Microsoft's Windows 2000 Advanced Server, Novell's NetWare 5.1, Red Hat Software's Linux 6.1 and The Santa Cruz Operation's (SCO) UnixWare 7.1.1. Sun declined our invitation to submit Solaris because the company says it's working on a new version.

Microsoft's Windows 2000 edges out NetWare for the Network World Blue Ribbon Award. Windows 2000 tops the field with its management interface, server monitoring tools, storage management facilities and security measures.

However, if it's performance you're after, no product came close to Novell's NetWare 5.1's numbers in our exhaustive file service and network benchmarks.

With its lightning-fast engine and Novell's directory-based administration, NetWare offers a great base for an enterprise network.

We found the latest release of Red Hat's commercial Linux bundle led the list for flexibility because its modular design lets you pare down the operating system to suit the task at hand. Additionally, you can create scripts out of multiple Linux commands to automate tasks across a distributed environment.

While SCO's UnixWare seemed to lag behind the pack in terms of file service performance and NOS-based administration features, its scalability features make it a strong candidate for running enterprise applications.

Regardless of the job you saddle your server with, it has to perform well at reading and writing files and sending them across the network. We designed two benchmark suites to measure each NOS in these two categories. To reflect the real world, our benchmark tests consider a wide range of server conditions.

NetWare was the hands-down leader in our performance benchmarking, taking first place in two-thirds of the file tests and earning top billing in the network tests.

Red Hat Linux followed NetWare in file performance overall and even outpaced the leader in file tests where the read/write loads were small. However, Linux did not perform well handling large loads - those tests in which there were more than 100 users. Under heavier user loads, Linux had a tendency to stop servicing file requests for a short period and then start up again.

Windows 2000 demonstrated poor write performance across all our file tests. In fact, we found that its write performance was about 10 percent of its read performance. After consulting with both Microsoft and Client/Server Solutions, the author of the Benchmark Factory testing tool we used, we determined that the poor write performance could be due to two factors. One, which we were unable to verify, might be a possible performance problem with the SCSI driver for the hardware we used.

More significant, though, was an issue with our test software. Benchmark Factory sends a write-through flag in each of its write requests that is supposed to cause the server to update cache, if appropriate, and then force a write to disk. When the write to disk occurs, the write call is released and the next request can be sent.

At first glance, it appeared as if Windows 2000 was the only operating system to honor this write-through flag because its write performance was so poor.

Therefore, we ran a second round of write tests with the flag turned off.

With the flag turned off, NetWare's write performance increased by 30 percent.

This test proved that Novell does indeed honor the write through flag and will write to disk for each write request when that flag is set. But when the write through flag is disabled, NetWare writes to disk in a more efficient manner by batching together contiguous blocks of data on the cache and writing all those blocks to disk at once.

Likewise, Red Hat Linux's performance increased by 10 percent to 15 percent when the write through flag was turned off. When we examined the Samba file system code, we found that it too honors the write through flag. The Samba code then finds an optimum time during the read/write sequence to write to disk.

This second round of file testing proves that Windows 2000 is dependent on its file system cache to optimize write performance. The results of the testing with the write through flag off were much higher - as much as 20 times faster.

However, Windows 2000 still fell behind both NetWare and RedHat Linux in the file write tests when the write through flag was off.

SCO honors the write through flag by default, since its journaling file system is constructed to maximize data integrity by writing to disk for all write requests. The results in the write tests with the write through flag on were very similar to the test results with the write through flag turned off.

For the network benchmark, we developed two tests. Our long TCP transaction test measured the bandwidth each server can sustain, while our short TCP transaction test measured each server's ability to handle large numbers of network sessions with small file transactions.

Despite a poor showing in the file benchmark, Windows 2000 came out on top in the long TCP transaction test. Windows 2000 is the only NOS with a multithreaded IP stack, which allows it to handle network requests with multiple processors. Novell and Red Hat say they are working on integrating this capability into their products.

NetWare and Linux also registered strong long TCP test results, coming in second and third, respectively.

In the short TCP transaction test, NetWare came out the clear winner. Linux earned second place in spite of its lack of support for abortive TCP closes, a method by which an operating system can quickly tear down TCP connections. Our testing software, Ganymede Software's Chariot, uses abortive closes in its TCP tests.

As enterprise networks grow to require more servers and support more end users, NOS management tools become crucial elements in keeping networks under control.

We looked at the management interfaces of each product and drilled down into how each handled server monitoring, client administration, file and print management, and storage management.

We found Windows 2000 and NetWare provide equally useful management interfaces.

Microsoft Management Console (MMC) is the glue that holds most of the Windows 2000 management functionality together. This configurable graphical user interface (GUI) lets you snap in Microsoft and third-party applets that customize its functionality. It's a two-paned interface, much like Windows Explorer, with a nested list on the left and selection details on the right.

The console is easy to use and lets you configure many local server elements, including users, disks, and system settings such as time and date.

MMC also lets you implement management policies for groups of users and computers using Active Directory, Microsoft's new directory service. From the Active Directory management tool inside MMC, you can configure users and change policies.

The network configuration tools are found in a separate application that opens when you click on the Network Places icon on the desktop. Each network interface is listed inside this window. You can add and change protocols and configure, enable and disable interfaces from here without rebooting.

NetWare offers several interfaces for server configuration and management.

These tools offer duplicate functionality, but each is useful depending from where you are trying to manage the system. The System Console offers a number of tools for server configuration. One of the most useful is NWConfig, which lets you change start-up files, install system modules and configure the storage subsystem. NWConfig is simple, intuitive and predictable.

ConsoleOne is a Java-based interface with a few graphical tools for managing and configuring NetWare. Third-party administration tools can plug into ConsoleOne and let you manage multiple services. We think ConsoleOne's interface is a bit unsophisticated, but it works well enough for those who must have a Windows- based manager.

Novell also offers a Web-accessible management application called NetWare Management Portal, which lets you manage NetWare servers remotely from a browser, and NWAdmin32, a relatively simple client-side tool for administering Novell Directory Services (NDS) from a Windows 95, 98 or NT client.

Red Hat's overall systems management interface is called LinuxConf and can run as a graphical or text-based application. The graphical interface, which resembles that of MMC, works well but has some layout issues that make it difficult to use at times. For example, when you run a setup application that takes up a lot of the screen, the system resizes the application larger than the desktop size.

Still, you can manage pretty much anything on the server from LinuxConf, and you can use it locally or remotely over the Web or via telnet. You can configure system parameters such as network addresses; file system settings and user accounts; and set up add-on services such as Samba - which is a service that lets Windows clients get to files residing on a Linux server - and FTP and Web servers. You can apply changes without rebooting the system.

Overall, Red Hat's interface is useful and the underlying tools are powerful and flexible, but LinuxConf lacks the polish of the other vendors' tools.

SCO Admin is a GUI-based front end for about 50 SCO UnixWare configuration and management tools in one window. When you click on a tool, it brings up the application to manage that item in a separate window.

Some of SCO's tools are GUI-based while others are text-based. The server required a reboot to apply many of the changes. On the plus side, you can manage multiple UnixWare servers from SCOAdmin.

SCO also offers a useful Java-based remote administration tool called WebTop that works from your browser.

One important administration task is monitoring the server itself. Microsoft leads the pack in how well you can keep an eye on your server's internals.

The Windows 2000 System Monitor lets you view a real-time, running graph of system operations, such as CPU and network utilization, and memory and disk usage. We used these tools extensively to determine the effect of our benchmark tests on the operating system. Another tool called Network Monitor has a basic network packet analyzer that lets you see the types of packets coming into the server. Together, these Microsoft utilities can be used to compare performance and capacity across multiple Windows 2000 servers.

NetWare's Monitor utility displays processor utilization, memory usage and buffer utilization on a local server. If you know what to look for, it can be a powerful tool for diagnosing bottlenecks in the system. Learning the meaning of each of the monitored parameters is a bit of a challenge, though.

If you want to look at performance statistics across multiple servers, you can tap into Novell's Web Management Portal.

Red Hat offers the standard Linux command-line tools for monitoring the server, such as iostat and vmstat. It has no graphical monitoring tools.

As with any Unix operating system, you can write scripts to automate these tools across Linux servers. However, these tools are typically cryptic and require a high level of proficiency to use effectively. A suite of graphical monitoring tools would be a great addition to Red Hat's Linux distribution.

UnixWare also offers a number of monitoring tools. System Monitor is UnixWare's simple but limited GUI for monitoring processor and memory utilization. The sar and rtpm command-line tools together list real-time system utilization of buffer, CPUs and disks. Together, these tools give you a good overall idea of the load on the server.

Along with managing the server, you must manage its users. It's no surprise that the two NOSes that ship with an integrated directory service topped the field in client administration tools.

We were able to configure user permissions via Microsoft's Active Directory and the directory administration tool in MMC. You can group users and computers into organizational units and apply policies to them.

You can manage Novell's NDS and NetWare clients with ConsoleOne, NWAdmin or NetWare Management Portal. Each can create users, manage file space, and set permissions and rights. Additionally, NetWare ships with a five-user version of Novell's ZENworks tool, which offers desktop administration services such as hardware and software inventory, software distribution and remote control services.

Red Hat Linux doesn't offer much in the way of client administration features.

You must control local users through Unix permission configuration mechanisms.

UnixWare is similar to Red Hat Linux in terms of client administration, but SCO provides some Windows binaries on the server to remotely set file and directory permissions from a Windows client, as well as create and change users and their settings. SCO and Red Hat offer support for the Unix-based Network Information Service (NIS). NIS is a store for network information like logon names, passwords and home directories. This integration helps with client administration.

A NOS is nothing without the ability to share file storage and printers. Novell and Microsoft collected top honors in these areas.

You can easily add and maintain printers in Windows 2000 using the print administration wizard, and you can add file shares using Active Directory management tools. Windows 2000 also offers Distributed File Services, which let you combine files on more than one server into a single share.

Novell Distributed Print Services (NDPS) let you quickly incorporate printers into the network. When NDPS senses a new printer on the network, it defines a Printer Agent that runs on the printer and communicates with NDS. You then use NDS to define the policies for the new printer.

You define NetWare file services by creating and then mounting a disk volume, which also manages volume policies.

Red Hat includes Linux's print tool utility for setting up server-connected and networks printers. You can also use this GUI to create printcap entries to define printer access.

Linux has a set of command-line file system configuration tools for mounting and unmounting partitions. Samba ships with the product and provides some integration for Windows clients. You can configure Samba only through a cryptic configuration ASCII file - a serious drawback.

UnixWare provides a flexible GUI-based printer setup tool called Printer SetUp Manager. For file and volume management, SCO offers a tool called VisionFS for interoperability with Windows clients. We used VisionFS to allow our NT clients to access the UnixWare server. This service was easy to configure and use.

Windows 2000 provides the best tools for storage management. Its graphical Manage Disks tool for local disk configuration includes software RAID management; you can dynamically add disks to a volume set without having to reboot the system. Additionally, a signature is written to each of the disks in an array so that they can be moved to another 2000 server without having to configure the volume on the new server. The new server recognizes the drives as members of a RAID set and adds the volume to the file system dynamically.

NetWare's volume management tool, NWConfig, is easy to use, but it can be a little confusing to set up a RAID volume. Once we knew what we were doing, we had no problems formatting drives and creating a RAID volume. The tool looks a little primitive, but we give it high marks for functionality and ease of use.

Red Hat Linux offers no graphical RAID configuration tools, but its command line tools made RAID configuration easy.

To configure disks on the UnixWare server, we used the Veritas Volume Manager graphical disk and volume administration tool that ships with UnixWare. We had some problems initially getting the tool to recognize the drives so they could be formatted. We managed to work around the disk configuration problem using an assortment of command line tools, after which Volume Manager worked well.

While we did not probe these NOSes extensively to expose any security weaknesses, we did look at what they offered in security features.

Microsoft has made significant strides with Windows 2000 security. Windows 2000 supports Kerberos public key certificates as its primary authentication mechanism within a domain, and allows additional authentication with smart cards. Microsoft provides a Security Configuration Tool that integrates with MMC for easy management of security objects in the Active Directory Services system, and a new Encrypting File System that lets you designate volumes on which files are automatically stored using encryption.

Novell added support for a public-key infrastructure into NetWare 5 using a public certificate schema developed by RSA Security that lets you tap into NDS to generate certificates.

Red Hat offers a basic Kerberos authentication mechanism. With Red Hat Linux, as with most Unix operating systems, the network services can be individually controlled to increase security. Red Hat offers Pluggable Authentication Modules as a way of allowing you to set authentication policies across programs running on the server. Passwords are protected with a shadow file. Red Hat also bundles firewall and VPN services.

UnixWare has a set of security tools called Security Manager that lets you set up varying degrees of intrusion protection across your network services, from no restriction to turning all network services off. It's a good management time saver, though you could manually modify the services to achieve the same result.

The most feature-rich NOS is of little value if it can't keep a server up and running. Windows 2000 offers software RAID 0, 1 and 5 configurations to provide fault tolerance for onboard disk drives, and has a built-in network load-balancing feature that allows a group of servers to look like one server and share the same network name and IP address. The group decides which server will service each request. This not only distributes the network load across several servers, it also provides fault tolerance in case a server goes down.

On a lesser scale, you can use Microsoft's Failover Clustering to provide basic failover services between two servers.

As with NT 4.0, Windows 2000 provides memory protection, which means that each process runs in its own segment.

There are also backup and restore capabilities bundled with Windows 2000.

Novell has an add-on product for NetWare called Novell Cluster Services that allows you to cluster as many as eight servers, all managed from one location using ConsoleOne, NetWare Management Portal or NWAdmin32. But Novell presently offers no clustering products to provide load balancing for applications or file services. NetWare has an elaborate memory protection scheme to segregate the memory used for the kernel and applications, and a Storage Management Services module to provide a highly flexible backup and restore facility.

Backups can be all-inclusive, cover parts of a volume or store a differential snapshot.

Red Hat provides a load-balancing product called piranha with its Linux. This package provides TCP load balancing between servers in a cluster. There is no hard limit to the number of servers you can configure in a cluster. Red Hat Linux also provides software RAID support through command line tools, has memory protection capabilities and provides a rudimentary backup facility.

SCO provides an optional feature to cluster several servers in a load-balancing environment with Non-Stop Clustering for a high level of fault-tolerance.

Currently, Non-Stop Clustering supports six servers in a cluster. UnixWare provides software RAID support that is managed using SCO's On-Line Data Manager feature. All the standard RAID levels are supported. Computer Associates' bundled ArcServeIT 6.6 provides backup and restore capabilities. UnixWare has memory protection capabilities.

Because our testing was conducted before Windows 2000's general availability ship date, we were not able to evaluate its hard-copy documentation. The online documentation provided on a CD is extensive, useful and well-organized, although a Web interface would be much easier to use if it gave more than a couple of sentences at a time for a particular help topic.

NetWare 5 comes with two manuals: a detailed manual for installing and configuring the NOS with good explanations of concepts and features along with an overview of how to configure them, and a small spiral-bound booklet of quick start cards. Novell's online documentation is very helpful.

Red Hat Linux comes with three manuals - an installation guide, a getting started guide and a reference manual - all of which are easy to follow.

Despite being the most difficult product to install, UnixWare offers the best documentation. It comes with two manuals: a system handbook and a getting started guide. The system handbook is a reference for conducting the installation of the operating system. It does a good job of reflecting this painful experience. The getting started guide is well-written and well-organized. It covers many of the tools needed to configure and maintain the operating system. SCO's online documentation looks nice and is easy to follow.

The bottom line is that these NOSes offer a wide range of characteristics and provide enterprise customers with a great deal of choice regarding how each can be used in any given corporate network.

If you want a good, general purpose NOS that can deliver enterprise-class services with all the bells and whistles imaginable, then Windows 2000 is the strongest contender. However, for high performance, enterprise file and print services, our tests show that Novell leads the pack. If you're willing to pay a higher price for scalability and reliability, SCO UnixWare would be a safe bet.

But if you need an inexpensive alternative that will give you bare-bones network services with decent performance, Red Hat Linux can certainly fit the bill.

The choice is yours.

Bass is the technical director and Robinson is a senior technical staff member at Centennial Networking Labs (CNL) at North Carolina State University in Raleigh. CNL focuses on performance, capacity and features of networking and server technologies and equipment. They can be reached at john_bass@ncsu.edu and james_robinson@ncsu.edu.

How we did it

We tested each NOS on Compaq ProLiant 1600 servers with dual 650MHz Pentium III CPUs, 512K-byte L2 cache and 640M bytes of RAM. The data partition consisted of 14 9.1G-byte drives loaded in a Compaq RAID Array 4214 drive array connected to an on-board Ultra2 SCSI controller. The NOSes were loaded on a 9.1G-byte drive connected to a second on-board Ultra2 SCSI controller. This configuration increased the available bandwidth of the drive subsystem and alleviated the bandwidth bottleneck to the drives.The client hardware consisted of four ProLiant 1600 machines with dual 600MHz Pentium III CPUs and 640M bytes of RAM.

Two additional ProLiant 1600 machines had dual 400MHz Pentium II chips with 256M bytes of RAM. Each client system ran Windows NT Server 4.0.Servers and clients were connected to the network with Intel Pro 100+ network interface cards (NIC). A Cisco Catalyst 2900 switch with 24 10/100M bit/sec Ethernet ports completed the network configuration. All the NICs and switch ports were configured for 100M bit/sec full-duplex operation. We used an additional Compaq 1600 with four Ethernet NICs as the control machine.For our NOS benchmarking, we focused on file service and network performance.

To test file service performance, we used Client/Server Solution's Benchmark Factory tool, which let us create tests that would stress each operating system's file subsystem. We configured the server to provide a Windows network file share for all clients using Windows System Message Block protocol over IP.

We installed a benchmark agent on each client and modified the clients' LMHOSTS files to evenly distribute file transaction requests to the server. We divided the tests into two categories - small and large file transfers.For the small file transfer tests, we used a 3-D test matrix of transfer direction (read/write), block size (1K and 8K bytes) and transaction type (random/ sequential), which resulted in eight individual tests. The small file transfer tests used a mix of 80 percent 1K-byte files, 10 percent 10K-byte files and 10 percent 50K-byte files. All of these write transaction tests were conducted with a write through flag set in the Benchmark Factory software. This flag is set to simulate an application forcing a write through each operating system's cache to disk.Because many applications do not force a write to disk, we asked Benchmark Factory to recompile its code with the write through flag turned off, and we reran the test with the new benchmark software build.

For the large file transfer tests, we combined reads and writes together in the same tests to emulate the behavior of large file service operations. We used a mix of 90 percent reads and 10 percent writes. We then created a set of four tests to cover all combinations of transfer type (random/sequential) and block size (1K and 8K bytes). The large file transfer tests used a mix of 80 percent 500K-byte files and 20 percent 1M-byte files. All of these write transaction tests were conducted with the write through flag set in the Benchmark Factory software. We reran the tests with the write through flag turned off.The benchmarking agent created the files each virtual user needed at the beginning of each test. We ran five iterations of each test with an increasing load of virtual users starting at one and increasing to 200 by 50-user increments for the majority of our tests. However, for our sequential read/write tests we started at one and increased to 40 in 10-user increments.

We did preliminary testing to establish the test parameters, then ran those parameters against each NOS.We graphed the results of each file test on a curve with five data points. The curves have a knee followed by a plateau. We averaged the data points in the plateau to yield the score for the test. We normalized the raw scores for each of the 20 file tests and then factored those normalized scores together to obtain a file benchmark score.To test the network performance of each NOS, we used Ganymede Software's Chariot software, which differs from the Benchmark Factory software in that all file transactions occur in memory. The disk subsystem is not utilized. We used Chariot to compare the efficiency of the NIC drivers and TCP/ IP stacks as measured by the number of operations each NOS could perform before the processor was bottlenecked and to compare the baseline throughput of the servers.For the TCP stack test, we only used two NICs on the server and disabled the other two. We set the IP subnet mask on all the machines to 255.255.252.0, which put 100.0.1.x and 100.0.2.x in the same subnet.We set up several bidirectional streams of short TCP file transfers from each of the clients to the server. A TCP session was built and torn down for each 3K-byte file transferred. This put a heavy load on the processors. Because the processors are the bottleneck, this test indicates the efficiency of the TCP stack and NIC driver for each NOS. We ran this test for 10 minutes and recorded the aggregate throughput value for all the streams.We ran a second Ganymede test to get an idea of the average aggregate throughput of all four NICs on the server.

The bidirectional streams between the Chariot endpoints were configured as a long TCP session with a large file size of 10M bytes. The tests opened a TCP session once when they began and then sent files for the duration of the test.

The session was not closed until the end of the test. We ran this test for 10 minutes to get an average aggregate throughput measurement. We averaged the short and long TCP file transaction results to get one number measured in megabits per second. This number was normalized to obtain the score for the network test.We also took a qualitative look at each NOS's management tools, security measures, stability and fault-tolerance features, installation process and documentation.We evaluated the usability of the overall management interface and how each product handled server monitoring, client administration, and file, print and storage management. We evaluated the scalability of each NOS based on its symmetric multiprocessor ability, failover clustering support and load-balancing clustering ability. For our security evaluation, we examined password file encryption, password and user ID encryption over the network, and any advanced security features offered. For stability and fault tolerance, we looked at each product's software RAID capabilities, backup and restore utilities, and memory protection.

Join the newsletter!

Error: Please check your email address.

More about CA TechnologiesCompaqEndPointsGanymede SoftwareIntelMicrosoftNDSNICNovellRed HatRSA, The Security Division of EMCSECVeritas

Show Comments

Market Place