At a Glance CDW Customer: MARS Advertising Group Need: Rebuilt infrastructure in three days after a devastating fire destroyed their office building. Tech Service Provided: CDW worked through the weekend to place rush orders, track deliveries and provide technical expertise, ensuring mission-critical systems and hardware were in place and running by Monday morning. Benefit: CDW’s Account Managers build close relationships with their customers to gain understanding of their needs and utilize the valuable assistance of our Technology Specialists to ensure timely delivery of necessary solutions. Key Learning: CDW responds to your technology needs with a sense of urgency. Return to Customized Services CASE STUDY MARS Advertising Group Works with CDW after Disaster Strikes   Friday, Feb. 20, 2004, was going to be a busy day for Bill Davidson, Senior Vice President of Information Systems of MARS Advertising Group. His team had about 45 projects on its plate, including a Voice over Internet Protocol (VoIP) deployment, Mac testing and upgrades. But as he commuted to his Southfield, Mich., office at 7 a.m. that morning, a call on his cell phone forever changed his definition of “busy.” “I was on the expressway, and I got a call from our CFO saying MARS had burned down,” he recalls. “I couldn’t believe it.” Ten minutes later, he watched firefighters hose down the charred shell of the building. Since the fire started well before work hours, no one was hurt, but it was obvious that the office and everything in it was destroyed. As employees arrived, he saw the fear on their faces, as they wondered if they’d have jobs on Monday. “You know people are counting on you, and it drives you,” he explains. Davidson called MARS CEO Ken Barnett, who was on a rare vacation, to break the news. Barnett asked if they’d be able to get back up and running, and Davidson didn’t hesitate. “Like you wouldn’t believe,” he recalls telling his boss. By Monday morning, Davidson’s promise proved true. MARS had set up shop at an office down the street, complete with new desktops, servers, phones, Internet and e-mail. He attributes his success to four key factors: a strong offsite backup and documentation program; a solid disaster recovery plan that his whole team was involved in developing; the camaraderie and teamwork of the entire MARS staff; and his technology vendor, CDW, who worked through the weekend to place rush orders, track deliveries and provide technical expertise so that mission-critical systems were in place and humming by Monday morning. When asked what made him even try to rebuild an entire infrastructure in one weekend, his response is simple: “Because we could. We were really buttoned up.” Meeting Of Minds One of the first calls Davidson made that morning was to Chad Lohrentz, his account manager at CDW. Lohrentz was in a meeting at the time, but called Davidson back the second he heard his urgent message. With CDW’s 170 systems engineers and certified product specialists who advise business customers on a range of technology issues and recommend multibranded solutions for a range of technology requirements, Lohrentz was able to tap the precise expertise to respond to MARS’ call. He pulled Brad Bellew, CDW storage systems engineer, onto the case to help “make sure all the pieces would play nice with each other,” Lohrentz explains. The CDW and MARS teams then devised plans to get the office running again by Monday and brought those plans together to make sure they covered all the bases. “Time was the biggest challenge,” Lohrentz says. “It was a tense situation for everyone. Fortunately, CDW has the systems in place to respond quickly.” Once they agreed on a plan, Bellew and Lohrentz checked to see which products CDW had in stock and set up appointments to install the equipment. They came to the consensus that CDW would ship a new server room to MARS overnight for Saturday delivery, Lohrentz says. “We were certainly busy the whole weekend,” he recalls. “We were tracking orders, shipping packages, getting airline flight numbers. We made sure they got their equipment on time so they could properly allocate resources.” MARS had to replace 250 PCs, 20 servers (plus deploy new virtual servers) and 59 Macs. Most notebooks were spared because employees took them home that night, but Davidson’s team replaced some of the older models. MARS also installed VMware’s virtualization technology and upgraded desktop software to 2003 versions. “It wasn’t a question of if or when we were going to do it, it was just a question of how and what resources we would need to do it,” Lohrentz says. During a final conference call that afternoon, Lohrentz, Bellew and Davidson went through the list of products being shipped over the weekend to make sure their plans were all on course. Davidson included a technology specialist from MARS’ insurance firm in on the call to make sure they could get approvals for all the product replacements. A Trusted Partner For Davidson, having Lohrentz involved in the process fed his confidence throughout the weekend. Lohrentz knows technology well and he understood MARS’ sense of urgency. “He treated it like it was his problem, and he was pleasant to work with,” Davidson adds. “He just takes it in stride and delivers and doesn’t give any attitude.” If Lohrentz doesn’t have an immediate answer, he gets it from a specialist or from the manufacturer. After the fire, he was on the phone placing orders, tracking deliveries, helping MARS with product research and providing feedback about Davidson’s plans. That reality check was comforting considering he was functioning on little to no sleep, Davidson says. “I didn’t really have to worry about it,” he says. “I could focus on rebuilding. Chad [Lohrentz] always adds value wherever he can.” Bellew and Lohrentz gave Davidson their cell phone numbers, and the trio spoke numerous times throughout the weekend. Lohrentz provided regular updates to the MARS team about the status of shipments so they knew what to expect and when, and Bellew gave advice on product installations. “We were getting calls once or twice every hour,” says Lohrentz. “This was of utmost importance to us. Everyone had to give 100 percent. It’s very rare that a company relies on a single vendor to bring it back from disaster.” Preparation Pays Off Since customer data is a foundation of MARS Advertising’s business, Davidson had turned to Lohrentz and Bellew about a month before the fire for help in upgrading his backup system. MARS had used tape libraries for years, but Davidson wanted a faster way to backup data. Lohrentz and Bellew worked with Sony to get MARS a demo of its brand-new Super AIT Tape Drive, which has higher transfer speeds and simplifies data backup because there’s just one tape, instead of a whole library. The demo tape was charred in the fire, but the data was recoverable. Since it was all on that one tape, Davidson was able to restore it much faster than he would have been able to with a traditional tape library. “One thing that became apparent is that disaster recovery is very important in terms of tape storage and offsite storage,” Lohrentz says. Since the data was easily recovered, the disaster recovery plan was solidly in place and everyone was jumping into action. Davidson knew he could get the work done. “We had great documentation, backups and network diagrams,” he says. “It was a matter of re-implementing our documented network and servers. For a company our size, it makes a lot of sense. “The only thing that wasn’t documented was where we’d go,” he adds. Fortunately, MARS executives rented space in the church next door from its day care center, so they were able to work through some details there on Friday following the fire. They quickly found an empty office building just down the street for the long-term. Not only was MARS back in business Monday, but its infrastructure was also leaner because Davidson seized the opportunity to upgrade his systems. In those first moments, as he watched the smoke pour from the windows, Davidson and his team decided to deploy VMware virtualization technology, something they had contemplated in the past but found too daunting to integrate with the existing infrastructure. With no legacy infrastructure, that obstacle disappeared. They also upgraded from Active Directory and Exchange 2000 to the 2003 versions and cut the company’s 40 servers to 20, saving the firm $250,000, Davidson estimates. “No matter how buttoned up you are, things will Frankenstein over time,” Davidson explains. “When you get to wipe the slate clean and do it from scratch, it’s phenomenal.” Immediately after the fire, MARS’ 300 employees pulled together. People offered to work through the night unloading boxes. When Davidson noticed that sense of camaraderie, he embraced it, and the age-old IT challenge of introducing new technology to change-resistant end-users disappeared. Instead, they were cooperative and eager to learn, and they caught onto the new technology surprisingly fast. “Everyone was your best friend,” Davidson says. “During the fire, everyone just kind of pulled it together. It made it really easy to implement change.” He says he’s still humbled by the work done by MARS’ staffers such as Laura Willson, an IT manager who handled much of the VoIP and network details, and Phil Mulloy, his network administrator, who helped him build the majority of the servers and desktops. “There were days when we easily worked 36 hours straight,” Davidson says. “We got punchy. But it was the most fun I’ve ever had in IT.” Lessons Learned Flexibility went a long way during the recovery, Davidson says. “Honestly, so many things were learned along the way that if we had stuck to a formal plan, I think we would have been down longer,” he adds. One of those lessons learned was to get a new uninterruptible power supply (UPS) in place as soon as possible. About a week after the fire, the local power company blew a transformer and the new office space lost power. Davidson hadn’t yet finished installing his UPS products, and some data was lost from a server. Davidson also warns colleagues to expect a few returns and exchanges after rebuilding from the ground up on such a fast schedule. During the first few weeks, as he fine-tuned his setup, he needed to return five servers. Working with a vendor like CDW, which accepted the returns without hassle, made that much easier. “The fact that we as an organization stand today is due to our entire IT team and their quality technology partners,” says MARS CEO Barnett. While the MARS fire was extreme, Lohrentz says he’s used to dealing with emergencies. Since technology changes so fast it can be hard for IT managers to keep pace, and they often come to him with emergency requests. CDW, he adds, is configured in a way to facilitate such requests. “I know who to rely on and who to get involved,” Lohrentz says. In fact, it was a downed server that first showed Davidson how valuable CDW can be. He called the manufacturer, which said it would ship a new server in a few weeks. He called CDW, and they had a new HP server shipped overnight. From that day on, Davidson has been a loyal customer of both CDW and HP. “All emergencies are relative,” Lohrentz says. “A boss may be breathing down your neck because something’s needed the next day. We are often under tight deadlines. My job is to help meet them.”