Worldwide, 83% of companies expect data analytics to have a decisive influence on decision-making processes in the next five years. In Germany, this figure is expected to rise from 52% to 90%.

© PwC

"Almost two out of three companies are now able to achieve concrete benefits through data analysis. In the previous year, this only applied to just under half of companies. The industrial sectors of mechanical and plant engineering and automotive engineering in particular are succeeding in achieving concrete benefits from data analysis," according to Bitkom's Big Data Report 2016. According to the report, more than two thirds of companies were able to reduce their business risks through the use of data analysis. More than half were able to tailor products, services and marketing more individually to the customer. According to the survey, reservations about data analysis also tend to be less widespread than in the previous year. However, data protection concerns and a lack of resources remain a relevant hurdle for every second company.

The results of the Bitkom Big Data Report 2016 show: The majority of companies have Big Data firmly in their sights.

© Bitkom

According to the big data report, simple, descriptive data analyses tend to dominate in companies, but the future belongs to more complex, future-oriented analyses. According to the report, 39% of companies are planning or discussing the use of predictive analyses in the future. However, only around a third of all companies have developed a big data strategy to date. Small and medium-sized companies in particular do not yet have a corresponding concept. "From today's perspective, a low level of maturity in the development and implementation of big data strategies can be identified," says the Big Data Report. 87.5% of the companies surveyed stated that they had only been dealing with big data for less than three years.

According to the IBM study 'Analytics: The real-world use of big data', almost two thirds of SMEs worldwide expect competitive advantages from data analysis of mass data, compared to just 36% two years previously. "This means that the use of big data has clearly arrived in the everyday lives of SMEs," says Prof. Dr. Christof Weinhardt, Head of Smart Data Accompanying Research and Director at the FZI Research Center for Information Technology.

Data silos and missing concepts

"More than 70% of the companies surveyed are shifting the focus of their analytics projects: Instead of customer-related processes, the focus is now more on operational functions." According to the Capgemini study 'Going Big', the potential of big data lies not only in a purely technical expansion of reporting, but also in the change of processes and the development of the organization towards a data and analysis-driven company. However, focusing solely on the provision of a new technological platform will not achieve the goal. The reporting culture must change from rigid, standardized forms of traditional reporting to forms of analysis that are open to methods and results.

"Unfortunately, it is not enough to simply shift the focus. Companies must finally get a grip on their data silos, establish functioning governance and create operating models for analytics that can be scaled more quickly," says Ingo Finck, Head of the Big Data & Analytics team at Capgemini Consulting.

Big data trends: NoSQL and Hadoopc

In many cases, companies are still using technologies that are less suitable for big data to analyze large volumes of data. This is the result of the 'Smart Data Business' study by Barc and Voice. According to the study, standard technologies from the field of 'relational databases' (78%), which have been in use since the 1970s and are not sufficiently scalable for large amounts of data, are used most frequently, reports the accompanying smart data research. According to the study, new technologies such as NoSQL or Hadoop are only used by just under a quarter of the companies surveyed, while standard tools in the areas of business intelligence (61%) and data integration (55%) are widely used. "In addition to data mining and predictive analytics solutions (40%), which are intended to enable better predictions, the Hadoop ecosystem (38%) as well as explorative analysis methods and analytical databases (37% each) are at the top of the planning list."

"In 2016, we will see a continued increase in systems that support non-relational or unstructured data and massive amounts of data," predicts Tableau. The company lists NoSQL as one of the top big data trends for 2016: "NoSQL databases are now clearly becoming the centerpiece of enterprise IT environments." 'NoSQL' stands for 'not only SQL' and describes database technologies that work with a wide variety of data schemas and have a wide range of approaches and functionalities for structuring data. NoSQL providers such as MongoDB, DataStax, Redis Labs, MarkLogic and Amazon Web Services (with DynamoDB) now also dominate over traditional database providers in the Gartner Magic Quadrant for operational database management systems.

From Tableau's point of view, Hadoop projects have reached the maturity phase. Existing users are using Hadoop even more and almost half of existing non-users are planning to use Hadoop in the future. Big data is also becoming faster, in line with user experiences with traditional data warehouses. The OLAP cube is being reactivated here, further blurring the boundaries between 'traditional' BI concepts and the world of 'big data'. The open-source Apache Spark is also evolving from a Hadoop component to the preferred big data platform because it processes data considerably faster than Hadoop.

Other trends according to Tableau: "Self-service data preparation tools are becoming increasingly popular. Business users also want to reduce the time and complexity of preparing data for analysis. According to analysts, 90% of organizations that have already adopted Hadoop will retain their data warehouses. With the new cloud offerings, these customers can dynamically scale up and down storage and compute resources in the data warehouse depending on the amount of data stored in the Hadoop data lake." In addition, IoT, cloud and big data will merge: "Data from devices in the Internet of Things will be one of the major applications for the cloud and one of the causes of the data explosion into the petabyte range. Leading cloud and data providers such as Google, Amazon Web Services and Microsoft will therefore develop services for the Internet of Things that allow data to be moved seamlessly into their cloud-based analytics engines."

Criticism of big data

Big data applications use machine learning methods to automate decisions from historical data. According to a ZEIT article at the re:publica conference, Microsoft and MIT researcher Kate Crawford named the weaknesses of 'artificial intelligence' (AI), learning machines and human-made algorithms: "Google's facial recognition, for example, once mistook black people for gorillas. Nikon's, on the other hand, mistook an Asian woman for someone blinking and suggested taking another photo without the supposed blemish. Obviously, says Crawford, both systems were predominantly trained with photos of Caucasians and non-Asians respectively." According to Crawford, if these or similar self-learning systems determine whether you get health insurance, a loan or are allowed to board an airplane, then the data basis should be more diverse. That's why we need ethics for algorithms and big data applications.

Author:
Martin Ortgies is a freelance journalist from Hanover.