Senior Data Engineer
Myntra Data Platform - Analytics
Myntra is a one stop shop for all your fashion and lifestyle needs. Being India's largest online store for fashion and lifestyle products, Myntra.com aims at providing a hassle free and enjoyable shopping experience to shoppers across the country with the widest range of brands and products on offer. The brand is making a conscious effort to bring the power of fashion to shoppers with an array of the latest and trendiest products available in the country.
Myntra's cloud based big data platform is highly scalable and processes over 7 billion events per day. We are on a journey to modernize our data platform and offer multiple self-serve offerings for Myntra's data consumers. We use the best-of-breed open source components and SaaS solutions as starting points to build out these capabilities along with maintaining critical core data assets of the organization.
If you are interested in the fast growing field of big data and analytics, and want to work on big data engineering at scale, building data products and building analytical models (Insights) to power business decisions, then this is the team for you.
Responsibilities:
Data has become the new currency in the modern day businesses and we are seeing the emergence of new businesses solely built on data. Organizations world over are realizing the potential that their in-house data holds and are increasingly using it in quite creative ways to unlock hidden opportunities. "Data drives decisions" is one of Myntra's core value and is a reflection of how dearly we value data in taking decisions across the organization.
Why is data challenging and how would I contribute?
- There are two fundamental challenges that need to be dealt with. Quantity being the primary issues wherein more and more data is getting created on a daily basis and quality being the other. Data engineers come handy here helping out in achieving complex data transformations, merging structured data with unstructured data, discovering patterns, creating analyst friendly views, exposing data as services and much more.
- Myntra Data Platform has a multi-petabyte data lakehouse. To unify analytics on to a single platform, we are moving towards a Data Lakehouse first architecture with the migration of existing core aggregates from Data warehouse to Data Lakehouse.
What makes this opportunity unique?
- We are a motivated and small team discovering patterns in data and applying to fashion e-commerce to significantly enhance the user experience. Myntra is growing very fast and we need to build scalable and best of the breed engineering systems. Some qualities we are looking for - quick learner, able to work with ambiguous problem statements, ability to come up with clean software designs, code and solutions.
Data Intelligence team Responsibilities:
- Build and integrate robust data processing pipelines for enterprise-level business analytics.
- Help architect data processing pipelines on Big Data systems. Have an ability to identify backend source pipe issues, and collaborate with Tech Engineering to fix these issues.
- Strong engineering mindset - build automated monitoring, alerting, self healing (restartability/graceful failures) features while building the consumption pipelines
- Understand the business domain to create reusable cross functional data assets
- Own the creation, accuracy and quality of consumption pipelines.
- Work closely with Tech Engineering and Product teams to integrate BI solutions and algorithms into the Big Data Platform.
- Help resolve data issues, troubleshoot system problems and as needed, assist other staff with reporting, debugging data accuracy issues and other related functions.
- Build front-end visualization using available BI tools in line with business requirements.
Desired Skills and Experience:
- Hands on experience with building end to end, complex & robust Big Data Engineering pipelines on Spark (Databricks) and advanced PL/SQL
- Expertise in data modeling principles/methods including conceptual, logical & physical Data Models
- Proven data analytics expertise with ability to be Data Domain / Subject matter expert and leverage opportunities to maximize data potential
- Understanding of Architecture and Design of Data Engineering products
- Excellent skills in collaborating with business users (tech and non-tech) and designing Data Engineering solutions on Data Lakhouse & Data Warehouse
- Ability to utilize Business Intelligence tools (Power BI) to represent insights
- Proficiency in performance tuning - spark & pl/sql
- Hands on experience working with MPP Databases like SQL Server, Azure Synapse
- Exposure to programming languages like unix and python is good to have
