Welcome to Hapag-Lloyd, a leading global logistics company. As the fifth largest container liner shipping company in the world, we are here to make sure that the flow of goods never stops. We are an international team of 12,800 employees working across 400 offices in 128 countries. This year we are growing even more and opening up Hapag-Lloyd’s very first Knowledge Center – in Tricity, Poland.
The Center, located in Gdańsk, will function as a hub for innovation and develop state-of-the-art business and technology solutions to help us navigate the future. And we want to do that together with you.
Our Mission - Your Chance
We are on the Mission of building a world class AI Team capable of supporting a world class shipping company like Hapag Lloyd to stay best in class with intelligent customer centric services.
You are passionate about Big Data, AI and Machine Learning? Then come on board, cause we have tons of real business cases and data waiting for you to be brought to life.
For our location in Gdansk we are looking for a
Senior Data Engineer (m/f/d)Responsibilities and Tasks:
- Data Extraction
- Design or evaluate adaptive interface client generators that allow zero code data extraction or streaming from systems via exposed APIs.
- Develop intelligent data augmentation and labeling automation solutions
- Develop intelligent data preparation and evaluation pipelines providing high quality data ready to use for ML analysis and model development.
- Training Data
- Design or evaluate ai supported data labeling systems to enable fast and accurate labeling projects on big amount of unlabeled data.
- Design and conduct, internal and crowd based data labeling projects to provide high quality training data to be used when creating new or enhancing HL ML Services.
- Continuously search, evaluate and collect relevant Datasets for Training or Model Performance evaluations.
- You are curiously evaluating and testing new AI data preparation and Big Data related Papers, Libraries, Third Party Solutions and vividly participate in AI/Interface/Big Data Communities like Meetups, Conferences, Hackathons or Kaggle competitions, always looking for new arising opportunities with high innovation potential for Hapag Lloyd.
AI Product development
- You are a key player when new business case specific AI Modules are developed.
- You acquire the necessary understanding of the business case, the related processes and the available data structures.
- Consult and coordinate the training data extraction with the product teams
- If required design, coordinate and conduct data labeling sub-projects considering budget limits, quality requirements and schedule deadlines.
- Design and implement required data structures for the AI module specific training data
- Integrate the training data in the managed training data repository as required for the development of AI models enabling reproducible experiments.
- You are responsible for the alignment with the Data Lake Team and clarify the best approach to store & manage the training & operative data considering the training and operative requirements of the AI module designed together with the Data Scientists and DevOps team colleagues.
- Support the Data Scientists on ETL tasks during the explorative Model analysis and design phase.
- Implement the ETL pipeline as required by the Model design
- Lead the system/data integration development of the API endpoints of the new AI Module in cooperation with the Product Teams and other source/target-System owners.
- Conduct the operationalization of the ETL pipeline
- Design and implement the learning loop API endpoints in cooperation with the Product Teams and other source/target-System owners.
- participate in design and implementation of the Module performance monitoring
- Support product teams to handle AI Module related incident situations in a fast and solution oriented manner.
AI Platform Development
- You’ll be a key player in building the generic, standardized and highly reusable platform of the Hapag Lloyd Data Science Development and Analysis Stack composed of Tools, Services and Modules, to enable the AI Team as well as Business- and System- Analysts to continuously improve time to market and cost efficiency of AI Solutions.
- You’ll be a key player in establishing and managing the AI Data Platform with a clear and understandable architecture and process concept, ready for vast amounts of data, supporting AI Module Configuration Versioning as well as repeatable model development experiments, ready for any kind of structured and unstructured data (relational, text, images, videos,..) and ready for operational, scalable and high performances OLTP as well as OLAP use.
- You are organizing trainings and information sessions for IT and Business Departments as well as for Public Community Events on various AI Data Management and Quality Topics to spread the knowledge and awareness about the possibilities, limits and future of AI in Logistic and IT.
Requirements and Qualifications:
A bachelor’s or master’s degree in computer science, business administration, mathematics, physics or other scientific area is preferred, but not required. Much more important is your experience, your attitude and your hunger for state of the art AI development.
Two to three years of relevant experience in enterprise level IT that equipped you to communicate effectively with the diverse stakeholders at corporate level is a good starting point.
- You’ll need 3 - 4 years of development hands on experience, with significant backend, batch and API involvement. Experiences with Test-driven development is a plus.
- SQL, Python, JEE and JS and related Development Stack elements including Git, Jenkins, common IDEs and ML frameworks are important assets for your endeavor. C++ would be helpful but initially optional.
- Hands on experience with at two or more relational DBMS like DB2, SQLite, PostgreSQL is as well as NoSQL DBS is important. Distributed DBS is initially optional.
- You should be confident working with various data formats from tabular data like CSV to some markup formats like HTML and common transfer formats like xml and json. Hands on experience with at least some MS-Office, image, audio or video formats is also important.
- You should bring at least some hands one experience with Cloud Services and related concepts and protocols of distributed computing, most preferable AWS. More is better.
Data engineering experiences
- You should have basic understanding of Linear Algebra to work with Vectors and Matrices. And you should be able to come up with numeric stable algorithms.
- You’ll need relevant hands on experience in building ETL pipelines.
- So you should have experience in extracting data from databases and from other Systemns via Web service APIs. Being able to consume data streams would be also interesting asset.
- For ETL you’ll need to have extensive hands on experience on data cleaning, evaluating basic data statistics, discretize, impute, encode categorical data, Randomize, Normalize, detect outlier and other transformation and evaluation methods.
- You should have hands on experience in at least one of the major data science topics like supervised vs unsupervised learning, NLP, CNNs, RNNs, GANs … and related ML frameworks like sklearn, Keras, PyTorch or Tensorflow.
- Being able to present and explain data with proper diagrams as well as experiences with at least one of the current data science platforms like Anaconda, Dataiku, Rapidminer… is a must.
- You’ll need to be able to understand, explain and discuss complex topics in fluent English. German and any other language is a plus in context of Multilingual – Models.
- You enjoy sharing your expert knowledge with others and thus generate new knowledge.
- As a senior data engineer you feel responsible to support and enable your team colleagues on a professional and personal level to ensure a relaxed and collaborative atmosphere and a continuously improving team performance.
- At least 1 year experience of working in a SCRUM team is a must. But classical project management topics like task breakdown, requirements engineering, Make or Buy analysis , Gant Diagrams … will also be necessary.
- Your good analytical understanding of complex interrelationships and a confident handling of pre-processing and evaluation of large amounts of data will support you in dealing with exciting questions.
- Challenging problems want to be solved by you. You show high commitment and want to make a difference.
- Furthermore, you can present results to your team, our business stakeholders in a simple and understandable way. It makes no difference to you whether in Polish, English or German.
- Strong troubleshooting and problem solving skills.
- Thrive in a fast-paced, innovative environment.
- You enjoy working in an international team and are passionate about new technologies and software.
HR Management • Mrs Mateusz Grabarski
Ballindamm 25 • 20095 Hamburg