Data Engineer

Tasks

We are seeking a skilled Data Engineer with expertise in Azure technologies to design, implement, and maintain robust data pipelines and solutions. 

• Design, build, and maintain scalable ETL/ELT data pipelines.
• Ingest and transform structured and unstructured data from APIs, databases, files and external systems.
• Design and maintain data models and curated datasets for analytics and AI applications.
• Implement data quality, validation, monitoring and error-handling processes.
• Optimize pipelines for performance, scalability, reliability and cost.
• Build batch and, where required, real-time/streaming data processing workflows.
• Work with Azure data services and data lake/storage solutions.
• Prepare and maintain datasets used by LLM, RAG, evaluation and agentic applications.
• Implement data lineage, metadata and appropriate governance practices.
• Develop AI services and APIs using Python 3.12+ and FastAPI.