DevOps at ING Analytics: combining data engineering with data operations
ING was one of the early adopters of the DevOps movement. Currently, there is a lot of expertise in the organization: way of working, tools, and HR are all catered for DevOps. In the Analytics area, these best practices were the basis of a modern and stable architecture where data engineers, operations, and data scientists work together with business people on daily basis. The technology stack includes Hadoop, Spark, Flink, Kafka, Cassandra, and several IBM tools. In the talk I’m going to share tools evolution, skills and processes in place. Touching in the second part two use-cases.
Giuseppe is Chapter Lead and Senior Data Engineer at ING, working in the Analytics department. After completing a master in Computer Engineering, he joined ING Group in 2014, where he worked on several international projects in data analytics and security. With his academic background in Artificial Intelligence, Pattern Recognition and Software Development and his big passion for data analytics, the Fast Data field has been a perfect match. He now leads the Fast Data Chapter focusing on streaming applications that make the customer communication extremely personalized and relevant.
Taco Bakker studied Computer Science at the University of Amsterdam. After his graduation he started as an IT engineer, but quickly became a Project Manager for large IT projects. He discovered firsthand the challenges of the traditional IT approaches such as Prince2. He joined ING to become a team manager for both development and operations departments. In this role he spent a lot of time closing the gap between OPS and DEV. About 8 years ago he became a LEAN six sigma black belt and started to improve the traditional IT way of working. He quickly adopted Agile Scrum, helped to implement Dev