Startup Jobs
Post a Job
Blend360

Senior Data Engineer - Azure Databricks

Hyderabad , India
full-time

AI overview

Design and optimize data pipelines on Azure using Databricks, collaborating with Data Science teams and ensuring data quality and governance.

We are looking for an experienced Senior Azure Databricks Engineer with strong hands-on expertise in Python, SQL, and Apache Spark to design, build, and optimize scalable data pipelines and analytics solutions on the Azure cloud platform. The ideal candidate should have experience working with large datasets, distributed data processing, analytics use cases, and modern data engineering practices.

Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure Databricks
  • Implement ETL/ELT workflows using PySpark, Spark SQL, and Python
  • Implement pipelines for data ingestion using Azure Data Factory
  • Optimize Spark jobs for performance, cost, and scalability
  • Work with structured and semi-structured data (Parquet, Delta, JSON, CSV)
  • Build and manage Delta Lake tables (ACID, time travel, schema evolution)
  • Integrate Databricks with Azure Data Lake Storage (ADLS Gen2)
  • Develop complex queries and transformations using SQL
  • Collaborate with Data Science teams to prepare data for modelling use cases,

ensuring appropriate transformations, feature generation, and storage.

  • Follow best practices for security, access control, and governance in Azure
  • Ensure data quality, validation, and monitoring using testing tools
  • Deployment of solutions to Production environments
  • 4+ years of experience in Data Engineering, ideally supporting POS and SKU datasets.
  • Handling high volume transactional datasets
  • Strong hands-on experience with Azure Databricks
  • Understanding of the Medallion Architecture and implementing it within Databricks
  • Good understanding of data modelling techniques
  • Proficiency in Python for data processing
  • Strong knowledge of SQL (joins, window functions, performance tuning)
  • Hands-on experience with Apache Spark / PySpark
  • Experience working with Delta Lake
  • Knowledge of Azure Data Lake Storage (ADLS Gen2)
  • Understanding of distributed computing concepts
  • Experience with Git version control
  • Understanding of ML use cases and data considerations for model development
 Blend360

 BLEND360 is an award-winning, new breed Data Science Consultancy focused on powering exceptional results for our Fortune 500/1000 clients and other major organizations. We are a growing company—born at the intersection of advanced analytics, data, and technology.Who we are:People are everything here at BLEND360.  We are inspired by advancing our Client’s most critical initiatives, products and projects by matching our clients with the right talent. BLEND360 has been among the Inc. 5000 fastest growing companies 8 years in a row, and we’re very proud of our World Class NPS score. Our success is a direct result of our passion for advancing the careers of the talented people we work with every day. When you work at BLEND360, you will:Collaborate with a smart, passionate group of people who are invested in your success.Partner with an impressive list of clients, who value Blend360’s services and the world class experience we deliver with every engagement. Thrive with a company and leadership team who are committed to growth.

View all jobs
Ace your job interview

Understand the required skills and qualifications, anticipate the questions you may be asked, and study well-prepared answers using our sample responses.

Senior Data Engineer Q&A's
Report this job
Apply for this job