Qualification – B.E / Master of Engineering

Education – Computer Science

Experience – 6 – 14 years

Key Responsibilities:

• Lead the design and implementation of highly scalable data engineering solutions on Azure, leveraging services like Azure Data Factory, Azure Databricks, Azure Data Lake, and Azure Synapse Analytics.
• Architect and build data pipelines for optimal extraction, transformation, and loading of data from a wide variety of data sources using Azure 'big data' technologies.
• Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.
• Implement data flows to connect operational systems, data for analytics and BI systems.
• Ensure data integrity and compliance with data governance and security policies across the Azure data landscape.
• Work closely with data scientists to operationalize and scale machine learning models and algorithms.
• Provide thought leadership in the constant improvement of data reliability, efficiency, and quality.

Qualifications:

• 6+ years of experience in a Data Engineering role, with a strong emphasis on Azure cloud services.
• Expertise in SQL and experience with scripting languages such as Python or Scala for data processing.
• Strong knowledge of Azure cloud services related to data engineering such as Azure Data Lake, Azure Data Factory, Azure Stream Analytics, and Azure Synapse Analytics.
• Develop scalable and efficient data processing and transformation solutions using PySpark within the Azure cloud environment.
• Employ Python scripting to automate data flows, integrate data from multiple sources, and facilitate operational and analytical reporting.
• Demonstrated experience with big data tools like Hadoop, Spark, and Kafka.
• Proven ability in managing and communicating data warehouse plans to internal clients.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field.