“Today, with everything being paperless, you could literally lose your whole business in the snap of a finger if you don’t have the right backups.”

Those words weren’t spoken by someone trying to market a high-tech solution to manage veterinary practices’ digital data. They came from Mike Krajewski, practice manager at Dr. Nina’s Animal Hospital in Sarasota, Fla., who came out on the losing side of a battle with a malicious Russian hacker.

Krajewski, who acknowledges that he knows “quite a bit about IT,” had a seemingly solid plan in place to secure and store the animal hospital’s data. It worked well, until the international hacker launched a sneak attack that could easily threaten any similarly vulnerable hospital.

Although Krajewski basically had to start from scratch after the hack, he learned some valuable lessons that he shared to help other hospitals protect themselves.

So what happened, exactly?

It all started on a Monday in July when Krajewski returned to the hospital after a weekend away.

At the mercy of a hacker

“As soon as we opened on Monday, we noticed our emails weren’t coming through. Our entire internal network was down, so we immediately knew there was something going on,” Krajewski said.

Whenever he double-clicked on a file, he encountered a pop-up saying the computer had been locked down with 256-bit encryption, and that he needed to wire transfer $5,000 in order to unlock the file.

Krajewski called his IT team for help and they determined that a hacker had deployed a virus that double-encrypted files by adding an extension to each one. Accessing any file now required two 50-character, alphanumeric, case-sensitive passwords, which Krajewski called “nearly impossible to hack.”

Prior to the attack, his hospital - like many others - used remote desktop to log in to their workspace while away from the office. According to Krajewski, the hacker exploited a vulnerability in the remote desktop application to access the hospital’s servers and unleash the potent virus.

Just like that, the hospital had lost access to five years and $30,000-$40,000 worth of work, including marketing pieces he had designed, newsletters, purchased images, and QuickBooks records.

One of the few silver linings was that Krajewski’s accountant had his QuickBooks records from previous years, so Krajewski “only had to redo every single transaction for the last 12 months; I didn’t actually have to go back for the last five years.”

Another bright side was that the hospital relied on a second server to run their internal practice software and store their medical records. That server was not hacked, which prevented the loss of five years of medical records.

Trying to salvage the situation

Krajewski contacted the Sarasota police to file a report, and he eventually spoke to members of the Sarasota County Sheriff’s Office’s Cyber Unit. They told him the FBI was aware of the virus specifically attacking Windows Server 2003 R2 and was trying to shut the hackers down, but that it currently did not have control over the situation.

After fruitless attempts to regain access to the files, Krajewski even attempted to contact the hacker through the Gmail address shown on the virus-related pop-ups because “at that point, all of the sudden the five grand wasn’t that expensive anymore.”

In the end, after much frustration and $6,000 spent on IT services aimed at recovering the files, Krajewski was forced to wipe the server, rebuild it from scratch, reinstall Windows, and move forward.

Building a better backup system

Krajewski experimented for a month with backing up files in the cloud but he found it to be costly and slow, which could mean a lengthy delay in restoring business operations after data loss.

He evaluated other solutions costing $150-$300 per month that seemed like they would provide fairly rapid backup and recovery, but ultimately his preferred course of action was to manage his data using a combination of:

• Two one-terabyte external hard drives - Krajewski backs up data on one hard drive and takes it home with him, then replaces it with the other hard drive so that there is always one hard drive outside of the office and one directly plugged into the server.
• Symantec Backup Exec - This software gives Krajewski the ability to schedule backups every four hours, and perform one full weekly backup. He can also restore backup data to similar hardware instead of the original hardware in case the original equipment is stolen or damaged in a fire.
• LogMeIn - Krajewski’s servers are connected to the servers of LogMeIn, a third-party company. Now, he logs into LogMeIn’s servers instead of his own, and those servers are protected by stronger security protocols such as locking down after three wrong password attempts.

“The two hard drives I bought were $100 each, so while it’s less expensive (than paid monthly services) and a little more work on my side, I have my drives with me so the recoup time could be as quick as four hours,” Krajewski said.

Lessons learned the hard way

Whether hospitals are managing their own data security and backup or paying a hired gun, Krajewski says they can benefit from the lessons he learned from his ordeal. They include:

• Vet the hired help - Don’t trust just any company with managing vital data. Krajewski had been paying an IT company $500 per month for five years to manage the hospital’s network, including backing up data. When catastrophe struck and he needed the backed-up data, Krajewski discovered that the last backup performed by the company was in 2008.
  
  This letdown led to Krajewski’s biggest lesson learned: When hiring an outside company or contractor for any service, verify their credentials at the beginning and take the extra steps to make sure they’re doing the job correctly.

“What I didn’t do was sit them down and have them show me physically that what they were doing actually was the right thing,” Krajewski said. “I didn’t say, ‘Hey, you have a backup - great. Let’s put it on a different server, let’s restore it, and show me how long it’s going to take.’ While that takes a bit of effort, that would have saved me in the end.”

• Always have a backup plan - “Backup, backup, backup. I can’t stress that enough,” Krajewski said.
  
  In previous decades, hospitals could use paper records with only a small amount of disorganization, missed charges, and occasional lost files. Even if a hospital lost one full medical file, “that was one client, that wasn’t your entire business,” Krajewski said.

But as he pointed out earlier, the digital age has heightened the potential for major mistakes, attacks, and other business catastrophes. Hospitals need digital copies stored securely in case of unexpected problems.

• Read the fine print carefully - Despite having an insurance policy with all the bells and whistles, Krajewski was surprised to learn that it could do nothing for him in the event of hacking.

“What I learned was that my insurance policy, even though I have all these provisions for loss of revenue and work stoppage and all these different things, one of the few exclusions is hacking,” Krajewski said.

Even the insurance agent who sold him the policy didn’t know that it didn’t cover any part of his claim because all damages incurred stemmed from a hacking event.

“That was another thing I learned - read the fine print even in insurance policies which seem to be 50 pages long,” he said. “At the very least, if you don’t read the whole thing, specifically go to the things that are excluded. Know what you’re not covered for, because that was a surprise to me.”