Job description 

What you’ll do:

Develop and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity.

Created backend functions that help automate some of the complex data related processes.

Build and configure web services, pub-sub queues, Event Driven Architecture components.

Follow best practices and industry standards while implementing Data Engineering and Mastering functions.

Analyze business requirements and lead the effort in solutioning the product design including creating architecture that is scalable.

What you will need:

5+ years of working experience.

Experience in OOPS concept, Data structure and design (architecture, design patterns and scaling)

Strong SQL skills, ability to create Functions/Stored Procedures and effective querying involving multiple tables and subqueries.

Experience creating detailed architecture diagrams and solutioning for data-based products.

Experience working with Bigdata using libraries like PySpark, Hadoop.

Strong Python programming experience and knowledge of Python Frameworks like Pandas, Numpy and Python data processing libraries.

Experience with Data Pipelining, Data Transformation, ETL and creating custom Python data pipelines.

Good experience working with AWS Cloud and it’s services related to Data engineering like Athena, AWS batch jobs etc.

Proven ability to use data, analytics, and business knowledge to solve complex business problems.

Experience of data modeling and working with data warehouses.

 Good to have:

Data visualization experience with tools like PowerBI, Tableau.

Experience working with PostgreSQL database.

Familiarity with HBase, MapReduce, and other suitable platforms.