As the main data solutions partner for one of the leading financial institutions in the Spanish market, we participated with their architecture area on an innovative Business Intelligence as a Service project aimed at offering users of different areas of the bank the possibility of exploiting data using any market exploitation tool, by passing the cost of use to the area and deploying the BI tool on Public Cloud infrastructure while keeping the data governed in the Private Cloud.

For this purpose, the front end was adapted to include the service as a new resource in Sandbox Manager, performing back-end jobs for authentication of the new service with the current systems, integration with current platform monitoring and subsequent integration with AWS for provision of the Billing & monitoring service.

Encompassed within the collaboration framework with the technology area of one of the largest Spanish financial institutions is the reviewing and improvement of its Big Data processes, considering both the redesigning of the developments and reviewing of the execution environment.

To this end, a review of the production environment was carried out by performing a realignment of the parametric configuration used for the execution of the various Spark Jobs, as well as making a proposal for a redistribution of resources based on the analysis performed for the various Jobs. A review of the critical path review was performed, identifying the most critical processes in the system with the goal of optimising overall times by means of configuration, and redesigning the Big Data processes using best practices to improve their efficiency.

The project is based on Scala/Spark technology under Informatica’s Big Data Management suite, knowledge of which is a priority in making the various specific parametric configurations and modifications.

Global Batch Monitoring and Control Project for the Corporate & Investment Banking area of a Multinational Entity, the objective of which is to develop an application for real-time monitoring of the various systems…

We developed the project based on a Kafka consumer for reading from a topic and generation of all the transformations necessary and storage in Elasticsearch. Design, maintenance and updating of the indexes are carried out in Elastic; with design of the dashboards in Kibana (mainly those particular to Accounting and Murex).

In addition to Kafka and Elastic technologies, the project is based primarily on process development with Apache NIFIs: the design process for reading Rest API (Accounting), the process for reading Syslog, as well as the process for dumping to Kafka after the necessary transformations.