Scalable machine learning with Apache Ignite, Python, and Julia: from prototype to production
Abstract
Apache Ignite incorporates a scalable, efficient machine learning (ML) framework that enables data transformation, ML model training, and inference on Apache Ignite nodes (without data leaving the cluster). However, sometimes scalability in regard to data size and data-transfer cost is not as important as scalability in regard to the number of parallel requests and the ability to use the external, more sophisticated models that are implemented in Python and Julia. In this talk, Peter Gagarinov shares his experience with integrating Apache Ignite with external machine frameworks. Using an ML-based, automated issue-management system (Alliedium) as an example, Peter shows how the business task of building a distributed and scalable service can be efficiently performed by relying on Apache Ignite and ML frameworks from Python and Julia ecosystems. In the second part of his talk, Peter presents a lightweight Apache Ignite data migration tool that was developed in-house for the needs of Alliedium: https://github.com/Alliedium/ignite-migration-tool
Peter Gagarinov
CTO | PhD | Head of Development, Leadership in AI
With over two decades of immersive experience in the IT sphere, particularly in AI/ML, DevOps, and FinTech, I am thrilled at the prospect of channeling my accumulated expertise and passion into innovative endeavors.
As CTO at Potential Energy LLC, I lead the creation and launch of a cutting-edge, cloud-based SaaS platform, richly embedded with AI and ML capabilities. This venture didn’t just reinforce my technical prowess in AI/ML; it significantly bolstered my leadership skills, enabling me to guide my teams toward the fruition of ambitious, technologically advanced objectives.
In my role as Head of DevOps and Backend Development at All of Us Financial, I was instrumental in integrating the latest technologies into our backend systems, significantly enhancing our financial platforms’ efficiency and reliability. My strategic focus on these technological advancements played a pivotal role in positioning the company as a valuable acquisition by PayPal.
My subsequent role at PayPal as Head of DevOps/Cloud Architect allowed me to leverage my previous experiences, integrating modern technologies into much larger systems. This position not only broadened my technical knowledge base but also enriched my understanding of applying cloud and ML technologies to FinTech industry, fostering innovation and adding value on a worldwide scale.
In my professional journey, which includes numerous successful projects at Allied Testing LLC, I has had a comprehensive exposure to AI/ML technologies. From hands-on development to strategic oversight, these experiences have sharpened my ability to lead teams towards the conceptualization and subsequent execution of impactful AI/ML solutions.