SANTA CLARA, Calif. — Big data aficionados arrived in numbers at the Santa Clara Convention center again Thursday for the fourth and final day of the O’Reilly Strata Conference, an extravaganza for all things data.
As with previous years, this year’s event called in a multiplicity of data companies, engineers, government representatives and startup entrepreneurs eager to chalk out big picture ideas and innovations in big data analytics, architecture, tools and oncoming tech trends. And to be expected, there were the necessary dollops of fanfare and motivational “out-of-the-box” thinking typical of a tech convention.
Contrastingly, and more predominantly, the conference was also highly attuned to real world challenges companies and municipalities are struggling with. Sessions focused on issues as common as why spreadsheets are often unreadable, in addition to the many human struggles of big data, such as friendly interfaces and opening up data through face-to-face relationships.
Here are some of the highlight from day four of the conference:
San Francisco to Announce Chief Data Officer
San Francisco’s Deputy Director of Innovation Shannon Spanhake announced that the city, after roughly a year of waiting, has appointed a new chief data officer that will start shortly. While a formal announcement is coming, she said the woman chosen for the role will be mandated to make the city’s data more legible and more accessible. What this means for the position, the first of its kind in San Francisco, is the task of creating a new city standard for data formatting across departments and the implementation of data coordinators inside each department. Spanhake said under the guidance of the chief data officer, the coordinators would monitor data efforts and processes in departments. Spanhake did not specify whether the data coordinator role would be a new position or if it would be filled by current department staff members.
Big Innovations Come from Public-Private Partnerships
Speaking alongside Spanhake in the session called Government Data on Both Sides of the Bridge, Eddie Tejada, the co-founder of Code for America brigade OpenOakland and vice chair of Oakland’s Public Ethics Commission, said many of the big projects happening in the city are stemming from citizen support and public-private partnerships. Despite the hype around innovative tech in government, Tejada and Spanhake said city budgets are limited or non-existent when it comes to IT innovation projects. More often, they said, solutions are a result of expert volunteerism or public-private partnerships that are mutually beneficial. San Francisco was a prime example of the low or no-budget project success. Spanhake said the city's more than 100 recently created apps were developed at no cost to the city through the help of volunteers. When the city wanted to put its open data in the app world, Spanhake said, a partnership with Yelp aided the city in putting public health data up and the city received similar support when publishing their housing data through Trulia, a realstate app. The partnerships cost the city a minimal amount to jumpstart, Spanhake said.
Why Spreadsheets Stink as Data
If it’s ever been wondered why Excel spreadsheets are difficult to input as big data, Strata clarified this by way of Felienne Hermans, an assistant professor at Delft University of Technology. In her presentation,
Spreadsheets: The Dark Matter of Big Data, Hermans said her research studies showed that despite 95 percent of U.S. firms using spreadsheets for business and 50 percent of those firms using spreadsheet for decision-making, only 33 percent of all spreadsheets have a manual for understanding the data embedded inside them. Hermans said spreadsheets should not be seen as data but as code because, just like source code and software, spreadsheets have they’re own language for interpretation and suffer from many of the same coding problems of software. Hermans is currently working on software to interpret and format spreadsheets in mass quantities.
Appify Big Data
Ryan Cunningham, the product vice president at ClipCard, a company that puts big data into mobile-friendly formats, argued that the next wave in big data will be a push to make data user-friendly through simpler, not more complex, user experiences. Taking the app trend as an example, Cunningham said more than 90 percent of incoming big data is not analyzed immediately, and data productivity is not expanding at the same rate as it used to be. He attributed these observations to the fact that big data, and the enterprise software around it, has made it too difficult for the average user to work with. Cunningham said that even though mobile dashboards are ubiquitous in the consumer world, in enterprise solutions, he and Clipcard anticipate mobile being the next giant step forward.
