When you've lived with VOIP for more than seven years, as Mike Shisko has, you learn a thing or two about what works and what doesn't. Inadequate prioritization techniques can derail a deployment, for example. So can choosing the wrong security gear. Despite some setbacks, the director of IT for Hitachi Consulting wouldn't do voice any other way.

"I cannot imagine going back to an old TDM world," says Shisko, who talked about his VOIP experiences at the recent Network World IT Roadmap Conference & Expo in Dallas. "VOIP makes things so much easier to manage and to use, and is so much more functional for the end users."

Hitachi Consulting's VOIP network offers its 1,500 employees spread across 18 US and four European offices not only seamless voice, but also state-of-the-art presence, mobility, conferencing and collaboration capabilities. Plus its long-distance calling bill -- for the entire global operation -- is less than US$2,000 a month.

But things weren't always so rosy for Shisko and his team. When the services arm of electronics giant Hitachi first implemented ShoreTel's VOIP technology in 2000, Hitachi Consulting had just 14 offices, all in the United States, and 500 employees, all linked by a WAN with Ethernet links to each building.

"We weren't pushing the network to the limit by any means," says Shisko, who at the time was the lone IT person supporting the Dallas-based organization. "We hadn't done any testing. We just put the voice on the network, prioritized it and it worked. And it worked because it was a good, fast network and it was utilized to a level that made it available for voice."

VPN speed bump

That all changed a few years later, when Hitachi Consulting decided to handle its VPN services in-house. The firm's Ethernet provider had gone out of business, prompting Shisko to move to a private T-1 network from Qwest Communications.

"Qwest had a managed service where they built our VPN at their core, and it worked very well," Shisko explains. "But we moved away from that because we thought that for what we're paying them to manage it, we can do it ourselves better and less expensively. And that was the beginning of our network problems."

After examining the options, Hitachi's decision came down to two VPN vendors: Cisco and Netscreen. The company went with Cisco, primarily because it was the least expensive option. It installed new Cisco routers, complete with hardware VPN acceleration, and had Cisco engineers come on-site to help with the deployment.

"It was a disaster from Day 1," Shisko says. "Their engineers tweaked and prodded the config, and it got marginally better. But it was still pretty much unusable."

Three months of struggling later, Hitachi decided to do a trial with Netscreen. "We put in the Netscreen and it really was like somebody had turned on a switch -- it went from unusable voice over the network to perfect voice over the network," he says. "We didn't replace the Cisco routers -- we just turned them into the most expensive CSU/DSUs anywhere in the country."

With the VPN problem solved, Hitachi's VOIP network hummed along for another three years -- until the company began experiencing huge growth, in part from acquisitions. "We went from offices that were designed to seat 50 and maybe had five or six people, to having 20 or 30 people in them on a regular basis. And the traffic on the network started to change."

Because Hitachi was handling the VPN on-premises, it could only control voice prioritization inbound, not outbound. "The voice started to break up -- one way, the call would be great, but the other way, you could start to hear the robotics and static. It was a problem."

At that point, the company decided to implement an MPLS network between its sites. "MPLS lets us prioritize voice in both directions on the local loop, which is really where the bulk of the contention comes in, on that T-1 that goes between us and the core," Shisko says. "And life is once again good on the VOIP network."