Big Data Lead

What will you do?

  • To help us go to the next level we are looking to onboard a hands-on SME in leveraging big data tech to solve the most complex data issues. You will spend almost half of time with hands-on coding.
  • It involves large scale text data processing, event driven data pipelines, in-memory computations, optimization considering CPU core to network IO to disk IO.
  • You will be using cloud native services in AWS and GCP.

Who Are You? 

  • Solid grounding in computer engineering, Unix, data structures and algorithms would enable you to meet this challenge.
  • Designed and built multiple big data modules and data pipelines to process large volume. 
  • Genuinely excited about technology and worked on projects from scratch. 

Must have:

  • 7+ years  of hands-on experience in Software Development with a focus on big data and large data pipelines.
  • Minimum 3 years of experience to build services and pipelines using Python.
  • Expertise with a variety of data processing systems, including streaming, event, and batch (Spark, Hadoop/MapReduce)
  • Understanding of at least one NoSQL stores like MongoDB, Elasticsearch, HBase
  • Understanding of how data models, sharding and data location strategies for distributed data stores in large scale high-throughput and high-availability environments and their effect in non-structured text data processing
  • Experience with running scalable & high available systems with AWS or GCP.

Good to have:

  • Experience with Docker / Kubernetes
  • Exposure with CI/CD
  • Knowledge of Crawling/Scraping

 

  • Entire Work From Home
  • Birthday Leave
  • Remote Work 

Careers at DemandMatrix. Find Great Talent with Career Pages. | powered by SmartRecruiters | Find Great Talent with a Career Page.

View all jobs
Get hired quicker

Be the first to apply. Receive an email whenever similar jobs are posted.

Report this job
Apply for this job