1 month ago
Job title : Senior Data Engineer (Remote)
Job Location : Lagos
Deadline : September 07, 2024
Quick Recommended Links
Responsibilities
- Build a centralized data lake on GCP Data services by integrating diverse data sources throughout the enterprise.
- Develop, maintain, and optimize SPARK-powered batch and streaming data processing pipelines. Leverage GCP data services for complex data engineering tasks and ensure smooth integration with other platform components
- Design and implement data validation and quality checks to ensure data’s accuracy, completeness, and consistency as it flows through the pipelines.
- Work with the Data Science and Machine Learning teams to engage in advanced analytics.
- Collaborate with cross-functional teams, including data analysts, business users, operational and marketing teams, to extract insights and value from data.
- Collaborate with the product team to design, implement, and maintain the data models for analytical use cases.
- Design, develop, and upkeep data dashboards for various teams using Looker Studio.
- Engage in technology explorations, research and development, POC’s and conduct deep investigations and troubleshooting.
- Design and manage ETL/ELT processes, ensuring data integrity, availability, and performance.
- Troubleshoot data issues and conduct root cause analysis when reporting data is in question.
Requirements
Required Technical Skills:
- PySpark
- GCP – Big Query, Dataproc, Dataflow, Dataplex, Pub-Sub and Cloud Storage
- Advance SQL knowledge
- NoSQL (Preferably MongoDB)
- Programming languages – Scala/Python
- Great Expectation – similar DQ framework
- Familiarity with workflow management tools like Airflow, Prefect or Luigi.
- Understanding of Data Governance, DWH and Data Modelling.
Good to have skills:
- Infrastructure as Code – Terraform
- Docker and Kubernetes
- Looker Studio
- AI and ML engineering knowledge.
How to Apply for this Offer
Interested and Qualified candidates should Click here to Apply Now
- Research / Data Analysis jobs
Email Me Jobs Like These
Showing 1–0 of 0 jobs