XCaliber specializes in providing systems for online casino operators, sportsbooks and lotteries. The goal of the project was to decrease the load of the current system used for analytical purposes, as well as to deliver a personalized configuration of the services in the cloud, allowing to launch a containerized environment corresponding with the client’s reporting needs on demand.
As a main part of the project, a dedicated Apache Hadoop (based on the Hortonworks distribution) cluster config was designed. The platform has been implemented on the Amazon Web Services cloud infrastructure in the PaaS model. The architecture of the cluster, implemented with Apache Hadoop and Apache Spark, has been extended with additional open source components dedicated for data flow modeling and data analysis such as Superset and Airflow. Moreover, the environment has been integrated with the cloud auto-scaling mechanisms and containerized using Kubernetes, Helm, Træfik and Docker, to name a few.
- design of the architecture and launch of the Hadoop cluster meeting customer’s requirements
- selection of appropriate services within Amazon Elastic Compute Cloud (EC2) available in the AWS cloud
- optimal placement of HDP cluster services on cloud servers
- commissioning and configuration of the Hadoop cluster located in the existing client’s cloud infrastructure
- creation of Data Lake using Kafka Connect and integration with existing IT systems of the client
- assortment of the components to be used in the project during the architectural workshops conducted in cooperation between 3Soft and XCaliber teams, which delivered a concept of a high-scale, containerized environment operating in cooperation with the Hortownworks Data Platform
- preparation of architectural documentation, being the foundations to start implementation activities
- launching the platform based on a distributed infrastructure of analytical components supporting data processing and visualization
- integration of the components within a single consistent containerized system
- implementation of the platform’s autoscaling mechanisms to ensure high availability of the services
Thanks to the extension of the Apache Hadoop ecosystem architecture with the mechanisms of autoscaling in the cloud, we have enabled our clients to optimize their business through data analysis and visualization.Wojciech Sznapka
The business advantage
The joint work of 3Soft and XCaliber teams resulted in a solution that enables XCaliber customers to optimize their business model by analyzing the activity of thousands of players and millions of transactions per day. The project addressed the advanced reporting needs, while guaranteeing the separation of individual clients’ data. The dedicated configuration of the Hadoop ecosystem, extended by the mechanisms of autoscaling in the cloud, enables operators to run on demand containerized analytical and reporting environments, tailored to the needs of XCaliber and its customers. Thanks to the expert knowledge and close cooperation between XCaliber and 3Soft teams, it was possible to implement project within its requirements and deliver the product in all consecutive phases. The leverage of the cloud technologies enabled the flexible allocation of the resources, which allowed to reduce the costs associated with the processing and storage of large data sets.
Bilateral transfer of knowledge through working in a joint, interdisciplinary team
Reduction of the project costs through the use of open source technology
Ensuring high availability and scalability of the platform thanks to component containerization
Ensuring the high flexibility of the solution and shortening the implementation time through the use of cloud mechanisms