Business Intelligence Solution for Unification and Historical Construction Projects Analysis

Client

The client is a world-known data center management company. The company needed a robust big data solution to enable unified analytics on all construction projects (the ongoing and historical ones). Functionality presumed real time analysis of data with instant visualization of detailed KPIs and charts to support the management with multi dimensional visibility on all construction projects, pricing history and ultimately help to take data-driven decisions.

Challenge: data in unstructured format was siloed across the organization thus making holistic real-time analysis not possible

The problem was that the client’s existing storage as well as the approach to process unstructured siloed data was inefficient. Digital transformation was required to build a business intelligence solution based on data lake serverless architecture for fast data collection and real-time processing. Project scope assumed the following data types: documents from all departments on every project, historical documents from Google Workspaces (word, complex layout xlsx, pdf and SAP).

Solution: a data lake serverless based on Azure platform for real-time business analytics and intelligent reporting

After analyzing the client’s business requirements, our big data engineering team delivered the project in 2 phases: proof on concept and production deployment:

• Deployed a centralized data lake on Azure serverless architecture
• Ingested data from Google Workspace and SAP
• Performed ETL of unstructured data using Azure Databricks
• Set up business dashboards in PowerBI – to enable holistic view on pricing of past and upcoming projects, schedule, vendor data, etc.

Data pipeline processes and updates 100s of files per data center per day.

The smart business intelligence solution development took 2 weeks for PoC, 2 months for live production.

Take a closer look at the solution architecture developed to understand how the data was processed:

During the work, we ingested, stored, processed, and visualized data from various sources.

Result: real-time reporting with BI

During the project, our team of data scientists and engineers performed Big Data development.

As a result, we’ve developed a custom business intelligence solution to unify and analyze historical construction projects data using the client’s company siloed data. Now, using our tech solution, the client can make data-driven business decisions supported by the analytics of masses of data aggregated from various sources into one up-to-date report based on relevant data.