Investment Data Management Solution with AWS Cloud Environment

Client

The client is a US-based group of investment specialists managing a diversified real estate portfolio. Like many firms in the sector, they relied on a mix of spreadsheets and manual reporting processes that struggled to keep pace with their growing number of investment vehicles and the complexity of fund performance tracking.

As the portfolio scaled, the lack of a centralized data infrastructure created bottlenecks: fund managers spent significant time reconciling data from Excel files and PDF reports instead of focusing on investment decisions. They approached InData Labs with a request for a data solution for their real estate investment platform.

The development of a complete data pipeline on the cloud required an end-to-end integration with the web application to process users’ inputs and enable all the investment management features that would allow general partners to thrive on fruitful investment opportunities by crafting a successful investment strategy.

Challenge: investment data management solution using unstructured data and web app development from scratch

Investment firms handling multiple funds and asset classes face a common challenge: data arrives in inconsistent formats from different sources, making it nearly impossible to get a unified, real-time view of portfolio performance without heavy manual effort. Our client was no exception.

Developing a solution from scratch is always a challenging task, as it requires constant alignment and balance between technical requirements and the customer’s vision of success.

We were challenged to develop a data solution from scratch, considering different data input formats, such as Excel files and PDFs, and their correct processing, enrichment, and integration to enable all the features planned in the web application which was being developed in parallel.

Our main objective was to develop two platforms separately and together simultaneously, ensuring that in the end, our team had a complete, integrated solution enabling functionalities, to provide users with a pleasant experience and a vision of comprehensive assessment of performance and business strategies.

Before the solution was built, fund managers spent up to 30% of their working time on manual data reconciliation across Excel and PDF reports. Errors in investment KPIs led to delayed decisions and increased operational risk — challenges typical for firms managing multiple investment vehicles without a centralized data infrastructure.

Solution: data entry templates & pipeline on AWS cloud environment for integration with a web app

The InData Labs team started with analyzing the customer’s business to get complete understanding of its needs and requirements. The project was phased to deliver a minimum viable product and allow the customer to test their product in real scenarios.

• Data and performance reports from a group of potential users were gathered and analyzed in detail to map and understand the data to be treated, input formats, and indicators used, among other particularities.
• Input templates were created to enable a standardized, efficient way to feed the application with investment performance data and reports. Microsoft Excel was used to ease the process for users that are used to this format.
• Examples of structured and unstructured input file formats were gathered and processed for data extraction and integration with the cloud environment using Python, Lambda Functions, and other AWS Big data processing resources.
• The integrated data was enriched and modelled in a structured way to provide calculated KPIs, performance indicators, and strategic information. Data visualization was also developed using a Flask application to deliver instant access to investments’ metrics.
• The data solution was integrated with the web application using API to receive inputs and feed the application with investment data and metrics in a safe way.
• CI/CD was implemented using Github Actions to enable a fully integrated infrastructure.

We have completed an MVP development in 6 months by a team of 5 professionals: a Project Manager, an AWS Data Engineer, a Data Analyst, a Business Consultant, and a Technical Supervisor.

By phasing the delivery around an MVP, the client was able to start validating the product with real users within 6 months — significantly faster than a traditional full-build approach. The modular AWS architecture also means the solution scales as the portfolio grows, without rebuilding the core infrastructure.

Solution Architecture

Result: streamlined processes and simplified investment management

Results at a glance:

Our team has built a reliable cloud infrastructure based on AWS that makes it possible to extract and process users’ investment data to simplify and add value to investment management.

As a result, we’ve provided the client with a managed data service solution for investment management that meets the client’s business  needs through optimizations such as decreased data crunching time, reduction of error and security risks, instant access to investment metrics; standardization and ease of processes for fund managers.

Fund managers went from spending hours reconciling data across spreadsheets to accessing real-time KPI dashboards in a single click. The automated pipeline processes incoming Excel and PDF reports in minutes rather than days, giving the investment team a consistent, reliable view of portfolio performance.

The InData Labs team connected the data solution with the client’s web application through API and provided data visualization through a Flask Application. The client can now ingest and consume data in a safe, practical, and efficient way, to craft a successful investment strategy.