IE 11 Not Supported

For optimal browsing, we recommend Chrome, Firefox or Safari browsers.

Cybersecurity Team Lessons from Football Game Defeats

Underestimating, or not properly preparing for, adversaries can lead to big trouble — in both football and cybersecurity. So what can cyber teams learn from “The Game”?  

Players during the Michigan versus OSU game in 2021.
Michigan Football/Roger Hart
“The Game” was about to be played — again. Just as in 2019, almost everyone in the sports world predicted another uneventful Ohio State University (OSU) victory over University of Michigan, or TTUN — “The Team Up North.” This would be just another box checked toward winning another Big 10 Championship before playing more worthy competition, like a southern university from Georgia or Alabama.

There were many reasons for this (not so quiet) confidence on OSU’s part, including a decade since their last defeat against this northern foe, a trio of wide-receiver superstars, superior recruiting, better talent (for the battle that was expected), a higher ranking and several Heisman Trophy hopefuls on the Ohio State team.

Perhaps the greatest reason for OSU fan optimism (and even hubris) were the previous results on the field. The University of Michigan lost a close game to in-state rival Michigan State University (MSU) earlier in the 2021 season. Meanwhile, OSU had just completely annihilated MSU by a score of 56-7 the previous Saturday. Adding fuel to that fire, OSU led 49-0 against MSU at halftime and pulled out many of their starting players in the third quarter.

Of course, every OSU player and coach said all the right words leading up to the Saturday Big Noon Kickoff on Fox. The press conferences, pregame speeches, respect for every Big 10 team and more seemed to check all the right boxes.

And no, this Michigan versus OSU game, played two days after Thanksgiving in 2021, did not sneak up on anyone. In fact, everyone who pays attention to college football was watching, with ESPN College Gameday and Fox Big Noon Kickoff both attracting millions of TV viewers for the pregame analysis. The consensus from the experts: Michigan would lose. Perhaps a close game, or maybe a blowout, but Michigan could not keep up with the OSU offense.

But, in reality, a perfect storm of unexpected trouble was awaiting the OSU Buckeyes when they entered “The Big House.” Cold, snowy weather, an especially loud crowd, an opponent that was beyond passionate and other factors seemed to shock the visitors.

A little less than four hours later, the once mighty OSU team walked off the field after a humbling loss to Michigan by a score of 42 to 27. Gone were the dreams of a Big 10 Championship, or even a National Championship.

Social media comments poured in, listing dozens of reasons why the defeat happened. Some claimed the OSU team was built mainly for dry conditions and warm(er) temperatures. Were they not physical enough, focusing on the “skill positions” and not the battle up front on the line of scrimmage? Others pointed to not playing “The Game” the year before because of COVID-19. Still others pointed to motivation and psychology on both teams.

“These guys have been disrespecting us, stepping on our jerseys, talking about hanging 100 on us, doing all the rah-rah, doing all the talk,” Michigan defensive end Aidan Hutchinson said. “But we were about it today.”

One fan wrote this: “Beating MSU the way we did, worst thing that could have happened to our team going into the GAME. TTUN, it was evident based on the game and footage of them walking into the locker room at half time, they were the bullies, tougher and had more of an EDGE. They wanted it MORE than us and it showed in many aspects of the GAME. We need to develop or find some DUDES on the defense side of the ball. Lets turn the page, and get ready for 2022.”

HOW IS FOOTBALL LIKE CYBERSECURITY?


So what does this football story have to do with cybersecurity? Quite a bit in my experience and opinion.

First, many experts have brought cyber teams and football teams together in past blogs and articles. Here are a few, including several from this blog:

SecurityInfoWatch.com — “How football helps explain critical infrastructure cybersecurity”

LiveAbout — “Five Life Lessons Learned from Football”

Government Technology — “Perspectives after the Nebraska Cyber Security Conference” 

RSA Conference website — “What the Super Bowl Teaches About Cyber Security”

Government Technology“How Football Can Help Explain Data Breaches,” “How to Improve Cyberstrategy by Learning NFL Defensive Tactics” and “Seven Career Lessons from Kirk Cousins”

CSO: “Blind spots: How cyber defense is like stopping Tim Tebow”

But in this blog, in light of what just happened in Ann Arbor between Michigan and OSU, I want to repost some of the words from this 2017 blog entitled “On Data Breaches: Beware of Professional IT Pride Leading to a Fall.” Before I do, I will say that preparation does not simply mean the actions taken during the week before a big football game. Some experts believe OSU could not beat Michigan on a very cold, snowy day in Ann Arbor in late November, but they would have won in a domed stadium or on warm, dry grass in Miami.

Regardless of whether this is true or not, preparation, for both sports teams and cyber teams, includes thinking through various (likely and unlikely) scenarios, playbooks, unexpected injections and more. OSU did not do that in 2021 when they had to play in enemy territory and the adversary didn’t follow their typical script during the game. They were dominated in many ways that were not anticipated.

So here you go with the ways that pride comes before a fall — with football teams and cyber teams.

From “On Data Breaches: Beware of Professional IT Pride Leading to a Fall”:

We’ve heard it hundreds of times: Pride comes before a fall.

I like this clip from the movie The Patriot, in which Mel Gibson dramatically illustrates the point that pride can be a weakness, even in war:
But what does any of this have to do with data breaches or recent enterprise security incidents?

The conventional wisdom says the opposite is true. Overconfidence should be the last thing on the minds of any cybersecurity pros in the world right now. Companies are being hacked daily, so why even mention hubris (or excessive pride) and cyber in the same sentence?

There are many great articles showing how cybersecurity is in trouble because we are out-gunned online. The common storyline is that the bad guy hackers are too good — right? We are facing nation-state experts who can go beyond anything we can possibly stop.

Typical chief information security officer (CISO) answers include needing more dollars for cyberprograms, more talent, more cybersecurity wake-up calls, better technology, more centers of excellence and accelerated public attention on all cybertopics at home and work.

Further, there are an untold number of lists of lessons learned after Equifax for companies and individuals which seem to point in lots of other directions. I really like this blog from Forrester outlining the conventional wisdom after the “B2B Breach Trifecta: Equifax, SEC, and Deloitte.”

Yes — I agree with most of the advice on these lists. No — I have not changed my mind on the global state of our cyberchallenges. Many black hat hackers and nation-state actors are very good at using zero-day malware or sophisticated techniques to get around the best defenses.

But I believe we also need to look at other people and culture issues, because I don’t think these lists adequately answer some basic security questions for enterprises.

Questions like: Why did Equifax not patch a well-known vulnerability that led to the massive breach? Or, why did Equifax use the word "admin" for the login and the password of an important database with sensitive information? Or, why Deloitte's enterprise email administrators did not use 2-factor authentication?

Were these just unlucky one-off mistakes by a select few staff?

These situations led to significant data breaches, and there has been plenty of media ridicule and online name-calling as a result of these incidents. Many are questioning the qualifications of the specific people doing the work (or their leadership) or mocking their experts for not doing what they have told others to do.

Yes — huge mistakes were made, but I’m not going to pick on specific individuals or their resumes. I have seen similar mistakes made by governments and private companies all over the world — albeit often with much less at stake.

But going back to the list of questions above, do we truly believe that Equifax, or the SEC or Deloitte (or for that matter OPM or Target or Yahoo or a long list of other top global companies and governments) did not have (or could not get) adequate resources to address their cyberproblems before these incidents occurred?

I don’t.

I have worked with experts at companies such as Deloitte and other top consulting firms, and I know they have smart, well-qualified consultants who know what to do to prevent these and other types of data breaches. They make billions of dollars in profits every year, so I don’t believe that they were lacking in global corporate resources for email.

Now whether management thought that more attention to detail was necessary, or deployed the right people, processes and technology at the right places is another matter — and goes to the heart of my comments on IT culture below.

While resource and cybertalent concerns certainly exist for many smaller companies and governments, these top organizations are supposed to be the best and the brightest, the standard for excellence. They understand the risks, and (in other parts of the organization) even teach cyber best practices to others. So why could they not prevent these straightforward issues that led to the data breaches?

No doubt, it’s sometimes an easy out to say: “the bad guys are just too good.” I think the real answer is sometimes a culture of IT pride and individual practices that are prevalent in many top-tier organizations — and yes I am referring to the top consulting firms, tech companies, and three-letter government agencies in Washington, D.C.

I am not talking about striving for excellence, pride of skill or craft or profession, being “proud of a job well done” or the great feeling of being proud of your child for bringing home straight A's on their report card.

No, this is a blind spot type of pride that plays out as overconfidence and/or a lack of preparation and/or a not “bringing your A-game” into a situation in many of the same respects as good sports teams get beat by lesser opponents.

But before I provide a list of specific ways that I believe that this issue can play out and what we can do about it, I want to say that this topic is by no means new. I have personally seen this challenge come up plenty of times during my career — in both vendor partners and in award-winning government security teams.

Back in 2010, I wrote about some of the people-oriented problems that cause security pros (and teams) to fail, along with aspects of this particular problem in the CSO blogs titled: “Not enough humble pie” and “Are you an insider threat?”

Nor is this topic limited to security. Similar issues can occur in almost any professional IT role, and I’ve seen professional overconfidence lead to network and email outages, backups going bad, poor code being written and many other technology issues and concerns.

How overconfidence can impact organizational security and cause data breaches
Here are some of the ways that executive management and technology and security pros fail under the banner of pride or overconfidence — possibly even leading to negligence:
  • Not putting the right person or the right team(s) on the right task(s). Or doing the proper things initially but pulling them off and bringing in the B-team or C-team. Or using college interns to run things over the weekend or during vacations to save money. Note: Top tech firms and consultants want to place their top players where they can be billed for top dollars. Oftentimes, that is not the system administrator for email or internal staff who patch security vulnerabilities.
  • Not fully implanting tools, processes or procedures. Or not enforcing policies — (such as allowing enterprise email administrators to forego 2-factor authentication.) Not training as you should. Note: This challenge can flow from professional pride because some think that they “wrote the book” and already know this stuff and can break their own rules. Or as Morpheus said in the movie The Matrix: “There’s a difference between knowing the path and walking the path.”
  • Underestimating your adversary, while overestimating a technology tool’s ability to stop incidents with junior staff. Therefore, not preparing properly to implement new projects. Professional negligence.
  • Qualified staff not bringing their “A-game” for any personal or professional reason. The old “Been there done that, got the T-shirt” mentality. Or, “I know the risk, but it’s fine.” Really? Are you sure?
  • Executive management assuming that everything is being done right — because millions of dollars are being spent on cybersecurity. Management thinks: “We’ve got this covered. We are the best! It won’t happen to us.” Or management not paying for awareness or technical training or new activity because they don’t understand the risks and mitigation steps being taken.
  • Assumption that the outsourced function (by support vendor or their team) is taking care of things properly (overconfidence in vendor’s ability) — without understanding that you can’t outsource the responsibility.
  • Staff not wanting to ask the questions that need to be asked out of fear of reprisal and/or telling their management things that could get them disciplined. Note: Overconfident management can also ignore warnings that they have heard from staff before because staff seem to be “crying wolf.”
  • Not willing to change with the times regarding security tools and techniques when cyberattacks change. Maintaining the old saying: “We’ve always done it that way.”
  • Burned-out teams, but management doesn’t see it. Executives believe you’ve done incident response miracles in the past, so you’ll somehow do it again. But now the team is worn-out and ill-quipped to keep performing at a top level. Good management understands that cyberincident response teams can only go for so long, just as when emergency management teams respond to hurricanes Harvey, Irma and now Maria recoveries.
  • Lack of understanding what talent you really have left. Yes, you were the best a few years back, but perhaps your top cybertalent left. Some managers don’t want the executives to know that they are in trouble. The executives are still proud of the trophies won a few years back, but they are about to get a rude awakening.  

FINAL THOUGHTS


I agree that there are times when top cyber teams bring their A-game and still get beat by Russia or China or some other hacker A-team. The best players and tools and processes and overall cyber defense can certainly be overmatched despite an organization’s best efforts. But that is not what happens in a large percentage of cyber incidents.

Moving forward, executives must collectively look in the mirror and recognize that we can do better. With cyber best practices being followed — such as good patching, proper cyber hygiene, the basic cyber blocking and tackling tasks, updated security awareness training for all staff, and implementing the cyber framework checklists from NIST and others — many more data breaches can be avoided.

And my message to cybersecurity pros is this: Stay humble and vigilant. You never know what or who is around the next corner — in cyber or football.
Daniel J. Lohrmann is an internationally recognized cybersecurity leader, technologist, keynote speaker and author.