Data Engineer III
Walmart
Chennai, Tamil Nadu, India
Position Summary... Demonstrates up-to-date expertise and applies this to the development, execution, and improvement of action plans by providing expert advice and guidance to others in the application of information and best practices; supporting and aligning efforts to meet customer and business needs; and building commitment for perspectives and rationales. Provides and supports the implementation of business solutions by building relationships and partnerships with key stakeholders; identifying business needs; determining and carrying out necessary processes and practices; monitoring progress and results; recognizing and capitalizing on improvement opportunities; and adapting to competing demands, organizational changes, and new responsibilities. Models compliance with company policies and procedures and supports company mission, values, and standards of ethics and integrity by incorporating these into the development and implementation of business plans; using the Open Door Policy; and demonstrating and assisting others with how to apply these in executing business processes and practices. What you'll do... About Team: Our organization focuses on managing and delivering world-class data assets, including creating and maintaining data standards, driving policy compliance, creating partnerships, and developing pipelines and self-service tools. We empower our business to leverage data to fuel growth, driving revenue in our core and building new business model opportunities. At Walmart, we help people save money, so they can live better. This mission serves as the foundation for every decision we make and drives us to create the future of retail. We can’t do that without the best talent – talent that is innovative, curious, and driven to create exceptional experiences for our customers. Do you have boundless energy and passion for engineering data used to solve dynamic problems that will shape the future of retail? With the sheer scale of Walmart’s environment comes the biggest of big data sets. As a Walmart Data Engineer, you will dig into our mammoth scale of data to help unleash the power of retail data science by imagining, developing, and maintaining data pipelines that our Data Scientists and Analysts can rely on. You will be responsible for contributing to an orchestration layer of complex data transformations, refining raw data from source into targeted, valuable data assets for consumption in a governed way. You will partner with Data Scientists, Analysts, other engineers and business stakeholders to solve complex and exciting challenges so that we can build out capabilities that evolve the retail business model while making a positive impact on our customers’ lives. What you'll do: Data Strategy: Understands, articulates and applies principles of the defined strategy to routine business problems that involve a single function. Data Transformation and Integration: Extracts data from identified databases. Creates data pipelines and transform data to a structure that is relevant to the problem by selecting appropriate techniques. Develops knowledge of current analytics trends. Data Source Identification: Supports the understanding of the priority order of requirements and service level agreements. Helps identify the most suitable source for data that is fit for purpose. Performs initial data quality checks on extracted data. Data Modelling: Analyses complex data elements, systems, data flows, dependencies, and relationships to contribute to conceptual, physical and logical data models. Develops the Logical Data Model and Physical Data Models including data warehouse and data mart designs. Defines relational tables, primary and foreign keys and stored procedures to create a data model structure. Evaluates existing data models and physical databases for variances and discrepancies. Develops efficient data flows. Analyses data-related system integration challenges and proposes appropriate solutions. Code Development and Testing: Writes code to develop the required solution and application features by determining the appropriate programming language and leveraging business, technical and data requirements. Creates test cases to review and validate the proposed solution design. Creates proofs of concept. Tests the code using the appropriate testing approach. Deploys software to production servers. Contributes code documentation, maintains playbooks, and provides timely progress updates. Problem Formulation: Translates business problems within one's discipline to data related or mathematical solutions. Identifies what methods (for example, analytics, big data analytics, automation) would provide a solution for the problem. Shares use cases and gives examples to demonstrate how the method would solve the business problem. Applied Business Acumen: Provides recommendations to business stakeholders to solve complex business issues. Develops business cases for projects with a projected return on investment or cost savings. Translates business requirements into projects, activities, and tasks and aligns to overall business strategy. Serves as an interpreter and conduit to connect business needs with tangible solutions and results. Recommends new processes and ways of working. Data Governance: Establishes, modifies, and documents data governance projects and recommendations. Implements data governance practices in partnership with business stakeholders and peers. Interprets company and regulatory policies on data. Educates others on data governance processes, practices, policies, and guidelines. Provides recommendations on needed updates or inputs into data governance policies, practices, or guidelines. Demonstrates up-to-date expertise and applies this to the development, execution, and improvement of action plans by providing expert advice and guidance to others. Supporting and aligning efforts to meet customer and business needs and building commitment for perspectives and rationales. What you'll bring: 3+ years of experience in Data Engineering. · You have consistently high standards, your passion for quality is inherent in everything Well versed with Hadoop, Hive, Spark using Scala, Kubernetes, Cloud, API and Data Lake concepts . · You evangelize an extremely high standard of code quality, system reliability, and performance · You have a proven track record coding with at least one programming language (e.g., Java, Python) You’re experienced in computing platforms (e.g., GCP, Azure) You’re skilled in data modelling & data migration protocols Experience with Kafka connect, Druid, Big Query and Looker is added advantage. Experience with the integration tools like Automic, Airflow
