Optimisation of the On-line Centre and Customer Profitability Module

Encompassed within the collaboration framework with the technology area of one of the largest Spanish financial institutions is the reviewing and improvement of its Big Data processes, considering both the redesigning of the developments and reviewing of the execution environment.

To this end, a review of the production environment was carried out by performing a realignment of the parametric configuration used for the execution of the various Spark Jobs, as well as making a proposal for a redistribution of resources based on the analysis performed for the various Jobs. A review of the critical path review was performed, identifying the most critical processes in the system with the goal of optimising overall times by means of configuration, and redesigning the Big Data processes using best practices to improve their efficiency.

The project is based on Scala/Spark technology under Informatica’s Big Data Management suite, knowledge of which is a priority in making the various specific parametric configurations and modifications.