CASE STUDY: Migration of data to a new Data Lake environment
CASE STUDY | AUTOMOTIVE
About the Client
Volkswagen Group Polska has been the leader of the new car market in Poland since its inception. Being an importer of seven brands: Volkswagen, Volkswagen Commercial Vehicles, SKODA, SEAT, CUPRA, Audi and Porsche, the company recorded a share of 25% in the Polish passenger car market in 2021. The Group is heavily involved in the development of electric mobility: it offers the latest generation of electric cars, creates public awareness in this area and supports the development of charging infrastructure in Poland. At the same time, the company is active in the distribution of spare parts and accessories.
About the project
The aim of the project was to build a Data Lake environment based on the Cloudera 6 technology stack, along with subsequent maintenance and implementation of changes.
Tomasz Mirowski
We started our cooperation with Volkswagen Group Polska by having a joint conversation about the Client’s needs. We listened to the requirements and presented possible options for data migration to the new Data Lake environment. Our priority was to be as flexible as possible both about the Client’s needs and processes.
Solution
The project was divided into phases.
The first one was to migrate the existing environment to a new architecture designed for this purpose, built on the Cloudera 6 technology stack.
The next task was to build and maintain data flows to the target, already existing Data Platform.
Wdrożenie i rozwój
We began our cooperation with Volkswagen Group Polska by analyzing source data and file formats to create optimal data transformation processes. Our experts acted with full flexibility and in accordance with the Client’s internal processes.
Then, based on Apache NiFi and Spark components, we prepared data flows from various sources, e.g.: csv, xml, json flat files, excel formats and Orcale and MSSQL databases, to the existing Data Platform.
Full on-site implementation of the project – launching the Platform based on the new technology stack and preparing the initial data loading processes took 6 months.
At the current stage, we provide support for the architecture built on the Cloudera 6 technology stack in accordance with the adopted SLA and implement the necessary changes reported by the Client.
Michal Lazarowicz, acting Reporting Systems and BI Manager, VGP:
We were looking for a company experienced in building Data Lake environments. Already during the initial talks, 3Soft’s representatives offered possible solutions and recommendations for the migration, and then implemented the project as agreed. It was very important to us that the experts from 3Soft approached our processes in a fully flexible manner, and the involvement of people on our side in the migration process was minimal – it mainly came down to making key decisions and clarifying any doubts.
Migration of data in Volkswagen Group Polska: