Data Engineer

This role is based in our Menara Skymind office. 

We are looking for a Data Engineer to join our Data Automation team in Penang. 

Purpose of role:

Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support the organization’s data and analytics needs. They play a critical role in the end-to-end data lifecycle by ensuring that data is reliably collected, efficiently processed, and made accessible for analysis across various business functions. This position requires strong technical proficiency in data engineering tools, cloud platforms, and integration frameworks, as well as a deep understanding of data modelling, data quality, and automation best practices. In addition, they are expected to troubleshoot issues, manage risks, and continuously optimize data workflows for performance and reliability.

The Data Engineer also collaborates closely with key stakeholders across the Intelligence & Advisory (I&A) division, including Data Analysts, Research teams, and Technology partners, to translate business requirements into robust technical solutions. They contribute to shaping data strategy, reviewing integration and pipeline designs, and mentoring junior team members to uplift engineering standards across the organization.

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion from diverse sources (APIs, web scraping, cloud systems, files).
• Implement data quality checks, monitoring, and validation processes to ensure accuracy and reliability.
• Collaborate with stakeholders to understand data requirements and translate them into efficient technical solutions.
• Manage and optimize data storage across cloud and on-premises environments (e.g., AWS S3, databases, and warehouses).
• Automate manual processes (e.g., Excel, VBA workflows) into centralized and scalable solutions.
• Work with orchestration tools to schedule and monitor data workflows.
• Maintain documentation for data pipelines, transformations, and system integrations.
• Ensure data security, compliance, and governance standards are applied consistently.

Ethics & Transparency:

• Ensure strict compliance with global data regulations (such as GDPR, CCPA) and internal governance standards to safeguard personal and sensitive data across all data pipelines and storage systems.
• Maintain transparency in data engineering processes by clearly documenting data sources, transformation logic, data quality rules, and pipeline behavior so stakeholders understand how data is processed and delivered.
• Take ownership of assigned engineering tasks and proactively engage with stakeholders, including analysts, researchers, and technology teams to ensure clear, timely, and accurate communication throughout the data lifecycle.
• Demonstrate a strong understanding and commitment to Data Engineering and I&A goals, values, and best practices, adhering to all relevant policies, security guidelines, and ethical data-handling frameworks.

• Minimum of 3 years of experience in Data Engineering or Data Analytics.
• Strong experience with SQL (Postgres/ MySQL or similiar), schema design, indexing, and performance tuning.
• Proven ability to build ETL pipelines in Python (pandas, SQL Alchemy, etc.) or any similar language.
• Comfortable working with AWS core services (S3, RDS, ECS/Lambda).
• Familiarity with orchestration tools (e.g., Airflow, Lambda, Alteryx).
• Understanding data modeling, warehousing concepts, and schema design.
• Experience with customs/trade datasets (HS codes, import/export flows).
• Hands-on with Streamlit (or similar tools like Dash/Gradio) for interactive data apps.
• Exposure to forecasting/ML workflows in Python (scikit-learn, statsmodels, Prophet).).
• Ability to work independently in a contractor capacity, with clear deliverables.
• Strong documentation and communication skills.

We believe that great things happen when people connect face-to-face. That's why we work in-person with each other, or with customers and partners, three days a week or more. When you’re not spending time together in one of our offices or other workplaces – like at an Informa event – you get the flexibility and support to work from home or remotely.

TechTarget, Inc., doing business as Informa TechTarget, including its subsidiaries is an equal opportunity employer and complies with all applicable federal, state, and local fair employment practices laws. We strictly prohibit and do not tolerate discrimination against employees, applicants, or any other covered persons because of race, color, sex (including pregnancy), age, national origin or ancestry, ethnicity, religion, creed, sexual orientation, gender identity or expression, status as a veteran, and basis of disability or any other federal, state or local protected class. This policy applies to all terms and conditions of employment, including, but not limited to, hiring, training, promotion, discipline, compensation, benefits, and termination of employment. If you would like to request reasonable adjustments or accommodations to assist your participation in the hiring process and, or in the advertised position, please inform the appropriate Talent Acquisition Partner for the role once they have been in touch. Your request will be reviewed and considered in confidence. 

Informa TechTarget complies with the Americans with Disabilities Act (ADA), as amended by the ADA Amendments Act, and all applicable federal, state or local law.

Our benefits include:

• Great community: a welcoming culture with in-person and online social events, our fantastic Walk the World charity day and active diversity and inclusion networks
• Broader impact: take up to four days per year to volunteer, with charity match funding available too
• Career opportunity: the opportunity to develop your career with bespoke training and learning, mentoring platforms and on-demand access to thousands of courses on LinkedIn Learning. When it’s time for the next step, we encourage and support internal job moves
• Time out: annual leave, birthday leave and the chance to work from (almost!) anywhere for up to four weeks a year
• A flexible range of personal benefits to choose from, plus company funded private medical cover
• A ShareMatch scheme that allows you to become an Informa shareholder with free matching shares
• Strong wellbeing support through EAP assistance, mental health first aiders, a healthy living subsidy, access to health apps and more
• Recognition for great work, with global awards and kudos programmes
• As an international company, the chance to collaborate with teams around the world