Data Engineer

The Sr. Data Engineer (Hadoop Development) within the Sales Analytics Delivery team will be responsible for designing, developing and implementing Data integration and Analytics solutions on Hadoop and using BI tools, Data mining and/or Machine learning provide insights on the data. This position requires close collaboration with business partners to understand their business goals, understand requirements, design, develop and implement Data Analytics solutions following SDLC and agile methodologies.

Ideal candidate for this position will have a minimum of five years of experience in development and delivery of Analytics projects and three years development experience on Hadoop.

Responsibilities:

• Participate in technology project delivery activities such as gathering Business requirements, conceptual approach, design, development, test case preparation, unit/integration test execution, support process documentation

• Develop the source to target mapping documents and ETL code to load data from source to target systems

• Develop workflows and ETL using Hive scripts, Spark, Airflow, NiFi, Sqoop, Oozie and other utilities on Hadoop

• Build analytical applications using Hadoop eco system using packaged or open source technologies

• Partner with IT groups such as CIT, Engineering, Product, Security, and Infrastructure on project delivery activities

• Provide insights from data and present to IT and non-technical users to improve operations and productivity

• Unit test data loads and write scripts for data validation

• Support QA, UAT and performance testing phases of development cycle and implement DevOps principles from development to deployment to production

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

• Bachelor’s Degree in Computer Science or related discipline
• Proven experience in Hadoop development and data integration on Hadoop is required, preferably Hive, Spark ,Presto, Airflow, Sqoop, NiFi etc.
• Two Years of experience in BI/ Analytics Tools – Power BI or Tableau or any BI tools
• Experience in RDBMS (Oracle and SQLServer) is required and knowledge of emerging technologies (NoSQL/Graph Database(neo4j)/Time Series) nice to have
• Experience in writing complex SQL and Hive QL queries is required
• Experience in Python, Shell/ Perl scripting, scheduling and version control, CI/CD technologies is required
• Experience in real time data ingestion using Kafka/Spark Streaming
• Experience in Advanced Analytics, Machine learning and data mining is nice to have.
• Experience in at least one ETL technology is required – Pentaho, Talend, Informatica
• Strong follow-through, problem identification, analysis and problem solving skills
• Demonstrated ability to learn new Information technologies and apply quickly
• Exceptional communication and customer facing skills, able to interact effectively with diverse groups of global stakeholders, both technical and business users
• Experience in Payment Processing, CRM (Salesforce/Microsoft Dynamics), Genesys Call Center domain are nice to have

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.