About the job
We are seeking an experienced Data Engineer to join our dynamic team. The ideal candidate will be responsible for managing large-scale databases, optimizing data pipelines, and ensuring the smooth operation of data infrastructure. The role involves working with cloud-based databases, specifically Alibaba Cloud or AWS, and optimising big data SQL queries. You will play a crucial role in maintaining data quality, improving performance and scaling data systems.
Key Responsibilities
Database Administration:
-Manage and operate cloud databases on Alibaba Cloud (ApsaraDB, AnalyticDB, RDS) or AWS (Redshift, RDS, DynamoDB).
-Perform routine database monitoring, performance tuning, backup, and disaster recovery procedures.
-Automate database tasks such as backups, monitoring, and scaling to ensure high availability and reliability.
Big Data SQL Writing & Optimization
-Write, optimize, and troubleshoot complex SQL queries for high-performance data retrieval in big data environments.
-Work with large-scale distributed databases and data processing platforms like Hadoop, Spark, or equivalent.
-Implement query tuning strategies and indexing to improve the performance of data pipelines.
Data Pipeline Management:
-Design, develop, and maintain ETL/ELT processes for data ingestion, transformation, and loading.
-Build and optimize data pipelines for both batch and real-time data processing.
-Collaborate with data scientists, analysts, and stakeholders to ensure data needs are met.
Cloud Data Infrastructure & Data Security
-Manage cloud data infrastructure costs by optimizing resource usage.
-Ensure data security, access control, and privacy compliance across cloud services.
-Implement data governance practices, including auditing and monitoring data usage.
Collaboration & Documentation:
-Work closely with software engineering, data science and business intelligence teams to integrate and optimize data flow.
-Provide technical documentation, best practices and training on data tools and workflows.
Required Qualifications
-Bachelor's or Master’s degree in Computer Science, Information Technology or related field.
-2+ years of experience in data engineering, database operations and administration.
-Hands-on experience with cloud databases and data warehouses on Alibaba Cloud (ApsaraDB, AnalyticDB, MongoDB etc.) or AWS (Redshift, RDS, DynamoDB, etc.).
-Proficiency in SQL and experience with big data technologies like Hadoop, Spark, or similar platforms.
-Experience with ETL tools and data pipeline frameworks (e.g. Apache NiFi, Airflow, AWS Glue).
-Strong problem-solving and analytical skills with experience in query performance tuning and optimisation.
-Knowledge of database security, backup and recovery strategies.
-Familiarity with Python or other programming languages for data manipulation.
Preferred Qualifications
-Certifications in AWS or Alibaba Cloud.
-Experience with data governance and privacy regulations (GDPR, CCPA, etc.).
-Familiarity with NoSQL databases like MongoDB, Cassandra.
Disclaimer : By clicking the button below, you consent for CareerFirst and partners to use automated technology, including pre-recorded messages, cell phones and texts, and email to contact you at the number and email address provided. This includes if the number is currently on any Do Not Call Lists. This consent is not required to make a purchase. We are redirecting you to the employer's career page. Please note that we are not sending your CV to the employer on your behalf. Privacy Policy.
RELATED JOBS