Clickability tracking pixel

Big data analytics versus privacy: How can they both win?

Can these hot trends mix, or are they like oil and water? Can we find middle ground on data? Or, will these two trains collide? Do you feel the tension building yet?

by / March 16, 2014

Big data in elections

Photo Credit: AP/Gus Ruelas

Public profile or private behavior? Online spying or digital marketing?

Crowd surveillance or profiling individuals? Finding terrorists to protect homeland security or snooping on personal emails?

Helpful Internet shopping tip or unwanted spam? Thoughtful sales agent or weird stalker? World-class customer service or prying eyes? 

Mining big data using advanced analytics to solve difficult problems or leaving me alone for personal privacy reasons?

Can these hot 'big data' and 'personal privacy' trends mix, or are they like oil and water? Can we find middle ground? Or, will these two trains eventually collide?

Do you feel any tension building yet?

Headlines are full of big data and privacy stories

No matter where you look, the media is full of stories regarding the power of mining big data and the potential implications regarding our 21st century privacy.

In just the past few weeks, we’ve seen Edward Snowden defending his actions at an international conference on privacy grounds, Mark Zuckerberg calling President Obama to complain about surveillance, the National Security Agency (NSA) rejecting spying claims and calling for more transparency, Google asking a judge to black out portions of a court transcript that includes information on how it mines data from personal e-mails and President Obama discussing big data with privacy groups.

Back in January, John Podesta wrote a significant blog on big data and the future of privacy. Here’s an excerpt:

“We are undergoing a revolution in the way that information about our purchases, our conversations, our social networks, our movements, and even our physical identities are collected, stored, analyzed and used. The immense volume, diversity and potential value of data will have profound implications for privacy, the economy, and public policy….

When we complete our work, we expect to deliver to the President a report that anticipates future technological trends and frames the key questions that the collection, availability, and use of “big data” raise – both for our government, and the nation as a whole. It will help identify technological changes to watch, whether those technological changes are addressed by the U.S.’s current policy framework and highlight where further government action, funding, research and consideration may be required….”

The White House even co-hosted a big data privacy workshop at MIT at the beginning of this month. You can watch videos of those keynote speeches from links on the event agenda. (Side note: I highly recommend taking a look at this workshop material. I found the discussion during this MIT event to be fascinating - from big data privacy case studies to explanations of how re-identification can occur with anonymized data.)

We all know that one person’s cold, miserable, snowy winter is someone else’s great skiing season. But just in case you think you’ve heard enough about this 'big data privacy' topic, as the song goes: “You ain’t seen nothing yet.”

Make no mistake, only the potential war in Ukraine and the missing Malaysia Airlines 777 are getting more national attention than this important series of developments. (OK – you can add March Madness to the list.)

Even after Edward Snowden’s revelations last year, the extensive commentary on Snowden’s actions and hundreds of Google Glass articles, the intensity of the debate keeps coming back in new and different ways.

Data privacy is compatible with good information security

One aspect of this topic which is often confused comes from the simple question: "Does tight (data) security threaten privacy?" My answer was 'no' when asked by Hilton Collins for this video shot at RSA 2014.

I view privacy and security as partners, when it comes to protecting sensitive citizen data that is held by governments and private sector enterprises. Enterprise security policies, enforcement of security controls, encryption, logging of audit files and other security tools are needed to protect individuals from identity theft as well as enable privacy. Little conflict there.

Of course, this is a very different situation than using personal data collected (often without the knowledge of end users who don’t read the fine print) from networks, emails or shopping websites to target users with marketing ads or to look for suspicious activities. The lines get gray when personal data in disparate databases is used in the name of “homeland security” or to “provide a better customer experience.”

So the battle that often pits security against privacy may actually be better characterized as a struggle between mining big data using advanced analytics versus personal privacy, but that distinction may be even more confusing. No doubt, most young people in society are fine with corporations or governments mining their data, if there is a perceived benefit. But others are not so eager.

At the core, there are a few basic questions, such as:

- Who can see what data, when and why?
- What are the benefits for the end user?
- Do we trust the data keepers?
- What are the data collector’s capabilities and intentions?
- Can people opt-out or opt-in to various monitoring programs?
- How long is data held?
- Are there any lines that can’t be crossed? Who makes these decisions?
- Under what conditions can the privacy policy be changed?
- When does surveillance for protection become ‘big brother’ or creepy?
- Who is policing the marketers to ensure appropriate practices are maintained and due diligence and policies are followed.

The many benefits of big data

I try to report these technology issues as a pragmatist, looking to find middle ground on this important topic. The big data analytics boat has already left the dock, and there are numerous benefits to a more robust use of our big data. One example of this is the National Oceanic and Atmospheric Administration (NOAA) plan to release more big data to the public for helpful purposes.

Numerous companies are finding new uses for big data and products like Google Glass to potentially stop crime before it happens or help soldiers on battlefields. There are several states that use big data to stop fraud in government programs.  

In addition the new nominee to lead the National Security Agency (NSA) recently defended the collection of bulk data. Vice Admiral Michael Rogers said,

"I believe that we need to maintain an ability to make queries of phone records in a way that is agile and provides results in a timely fashion. Being able to quickly review phone connections associated with terrorists to assess whether a network exists is critical."

But on the other side, new apps like NameTag can allow Google Glass users to identify you just by looking at you. Some people view that use of the Internet big data as just too creepy. Another recent article explains the privacy implications of police wearing cameras.


At a recent Oakland University cybersecurity forum, some people in the audience and on the speakers panel argued that privacy has forever changed – and there is no going back. We will just live more open lives in the future – so get used to it. There were many persuasive arguments regarding the impact of social media and the open sharing of our lives on Facebook, Twitter and more.

But I countered that I was not so sure – at least not yet. These trends in society have a way of swinging back like a pendulum in the same way that 9/11/01 changed the way we defend our homeland.

What is clear is that more use of big data is hot right now, and thousands of new applications are coming with mobile interfaces to use this data in more ways.

What are the privacy implications? How far can app developers and the government go to mine big data? Is data truly anonymized? Is there middle ground that the majority in society will agree on regarding the privacy implications? We shall see.

As we travel this new cyber road and seek to find the middle ground, I call to mind the famous quote from Benjamin Franklin as one guide:

“People willing to trade their freedom for temporary security deserve neither and will lose both.”


Looking for the latest gov tech news as it happens? Subscribe to GT newsletters.

Dan Lohrmann Chief Security Officer & Chief Strategist at Security Mentor Inc.

Daniel J. Lohrmann is an internationally recognized cybersecurity leader, technologist, keynote speaker and author.

During his distinguished career, he has served global organizations in the public and private sectors in a variety of executive leadership capacities, receiving numerous national awards including: CSO of the Year, Public Official of the Year and Computerworld Premier 100 IT Leader.
Lohrmann led Michigan government’s cybersecurity and technology infrastructure teams from May 2002 to August 2014, including enterprisewide Chief Security Officer (CSO), Chief Technology Officer (CTO) and Chief Information Security Officer (CISO) roles in Michigan.

He currently serves as the Chief Security Officer (CSO) and Chief Strategist for Security Mentor Inc. He is leading the development and implementation of Security Mentor’s industry-leading cyber training, consulting and workshops for end users, managers and executives in the public and private sectors. He has advised senior leaders at the White House, National Governors Association (NGA), National Association of State CIOs (NASCIO), U.S. Department of Homeland Security (DHS), federal, state and local government agencies, Fortune 500 companies, small businesses and nonprofit institutions.

He has more than 30 years of experience in the computer industry, beginning his career with the National Security Agency. He worked for three years in England as a senior network engineer for Lockheed Martin (formerly Loral Aerospace) and for four years as a technical director for ManTech International in a US/UK military facility.

Lohrmann is the author of two books: Virtual Integrity: Faithfully Navigating the Brave New Web and BYOD for You: The Guide to Bring Your Own Device to Work. He has been a keynote speaker at global security and technology conferences from South Africa to Dubai and from Washington, D.C., to Moscow.

He holds a master's degree in computer science (CS) from Johns Hopkins University in Baltimore, and a bachelor's degree in CS from Valparaiso University in Indiana.

Follow Lohrmann on Twitter at: @govcso

E.REPUBLIC Platforms & Programs