Job Purpose *
We are seeking a Data Engineer to design, develop, and support reliable data pipelines and platforms that enable efficient data ingestion, processing, and availability for business and analytical needs.
The role collaborates with cross-functional teams to ensure data solutions align with business requirements while maintaining high standards of data quality, system performance, and operational stability.
It also supports continuous improvement through technology assessments, monitoring, and adherence to security and governance practices.
Job Description *
Team Collaboration
• Collaborate with developers, application support teams, and cross-functional stakeholders to design, develop, and maintain data pipelines and platforms.
• Assist team members in troubleshooting pipeline issues and resolving production incidents.
• Stay updated with emerging data technologies and share knowledge, best practices, and development standards to support continuous improvement.
Data Pipeline Design and Development
• Design, develop, and optimize data ingestion pipelines from multiple source systems into target data platforms.
• Work closely with business analysts to understand data requirements and ensure solutions align with business objectives.
• Perform end-to-end testing across extraction, transformation, and loading stages to ensure data accuracy, reliability, and readiness for consumption.
• Implement best practices for pipeline development, including error handling, logging, and performance optimization.
• Prepare deployment documentation and present changes to the Change Management Committee to obtain approval.
Production Support & Operations
• Provide operational support for data pipelines and platforms, including troubleshooting failures, performance issues, and data quality incidents.
• Monitor pipeline health, system performance, and data availability.
• Implement logging, monitoring, and alerting mechanisms to ensure stable and reliable operations.
• Coordinate regular maintenance activities such as patches, upgrades, and backups to maintain system security and performance.
Technology Assessment and Continuous Improvement
• Conduct regular assessments of server and application performance to identify optimization opportunities and ensure compliance with technology and security policies.
• Analyze server utilization before and after deployments or upgrades to evaluate their impact on Service Level Agreements (SLAs).
• Develop and implement remediation plans based on findings from Vulnerability Assessment and Penetration Testing (VAPT).
Job Specification *
Qualifications
(Basic Degree/Diploma etc) Bachelor’s Degree in Computer Science / Information Technology or equivalent.
Relevant Work Experience • Strong academic foundation or internship experience in data engineering, data platforms, or data pipeline development.
• Familiarity with data lakehouse architectures, relational databases (e.g., PostgreSQL, SQL Server, Oracle), and basic data warehousing concepts.
• Exposure to big data tools or platforms such as Huawei DWS, MRS, or DataArts Studio (through coursework, training, or projects).
• Basic hands-on experience working in Linux/Unix environments.
• Knowledge of Shell scripting, Python, SQL, and introductory understanding of Spark or similar data processing frameworks.
• Strong willingness to learn, problem-solving mindset, and ability to work collaboratively in a team environment
Key Dimension of Impact *
-
Required Competencies and Skills *
Competencies/Skills
(Essential to succeed in this job)
• Familiar with various project management tools (e.g. Microsoft Project).
• Posses analytical and strategic with a track record of success in delivering results within a large complex organization.
• Must have strong leadership and people management skills; ability to influence, organizational ability and time management.
• Embody the organisation culture and aligns and support of organisation vision and mission.
• Excellent interpersonal, written and communication skills.
• Highly result oriented and can work independently.
• Demonstrates ability to build relationships and work well with peers from both business and technology stakeholders and interact effectively with internal and external parties.
• Ability to plan and prioritise work / projects in the team.
• Display good leadership qualities, high commitment and drive to complete the assignment.