Mr. Löhnert, many companies are already using BI systems to evaluate data. But why are conventional systems no longer sufficient in the big data age?
Löhnert: Companies should use their entire data ecosystem. And this is exactly where big data technologies can help - but they should complement BI systems, not replace them. Traditional BI systems have proven their worth and provide valuable insights generated by data analyses that have been optimized over the years. Rebuilding these proven analyses with a big data solution in order to obtain the same results is cost-intensive and in many cases uneconomical, as no added value is generated compared to the BI solution, at least at present.

What distinguishes BI technology from big data?
Löhnert: BI systems offer analyses for structured, consistent databases. Big data systems, on the other hand, are often used to analyze new, previously unused data that cannot be processed by traditional BI systems because it is too large, too unstructured or available in near real time. It is therefore often a question of tapping into new data sources, even if big data systems are not limited to this.

"In my opinion, the complementary use of traditional BI systems and new big data technologies is a key to the future success of a company."

© Consol

What options are there for analyzing big data?
Löhnert: The data hub approach aims to provide a central, highly available and scalable system that stores all of the ecosystem's data. A data lake is the central location for data storage, access and analysis. The data hub also offers additional functions such as data management, security and traceability. It is the original approach in the big data sector, but it is far from outdated. Data hub providers have been improving and optimizing their solutions for years and adding many company-relevant features that provide a stable foundation for a long-term data strategy.

Where is a data hub located?
Löhnert: Many companies rely on a solution in their own data center. However, many providers of data hub solutions now have tools that ensure simple distribution to the cloud and enable dynamic scaling. It is also possible to move from one provider to another or to your own environment. Alternatively, hybrid approaches can be pursued in which particularly sensitive data is still hosted on-premise, while less sensitive data migrates to the cloud data hub.

On the other hand, it should be noted that cloud providers have also jumped on the big data bandwagon and offer alternatives to the data hub. For example, they have a portfolio of services that can be used to perform batch and streaming analytics and forward the data to cloud storage or a visualization tool. The aim is to provide a flexible architecture on which companies can assemble data pipelines according to the modular principle without having to deal extensively with the infrastructure. Data pipelines or individual cloud services only run on demand and scale with the company and its changing requirements.

How can a combined BI and big data environment be set up?
Löhnert: BI environments are already in place in most companies. In such a case, the BI system can be connected to a data hub using transfer technologies such as Apache Sqoop in order to exchange data. Before implementation, however, a company must first clarify some fundamental questions. The service and operating model must be defined depending on the company's specific requirements and objectives: on-premise, cloud, data hub, services-based or hybrid.
Which approach a company pursues ultimately depends on its specific needs and use cases; a case-by-case approach is essential here. Depending on compliance requirements, security and data governance guidelines must also be defined and implemented. Last but not least, a company must determine the interactive access options to data for both the data scientist and the business analyst. Finally, a combined BI and big data environment should be set up gradually, i.e. additional data sources and additional analyses should be added step by step.

And what are the benefits of the combination?
Löhnert: The combination makes it possible to utilize the strengths of both systems. The solution combination thus offers the possibility of enriching BI-based analyses with new aggregated results from the big data systems and thus optimizing and improving them.

What opportunities does a BI/big data environment offer for production, for example?
Löhnert: IoT scenarios provide a typical example. By using sensor data from machines, data from other relevant points in production such as temperatures or secondary influencing factors such as weather data, production processes, machines and products can be monitored and optimized. All the information collected can also be correlated with existing key business figures, for example, in order to increase overall company efficiency.