Principal Data Engineer (GCP) - Arbisoft

Our client Arbisoft is hiring a Principal Data Engineer (GCP) in Lahore. Experience Required: 5+ years We are seeking a highly skilled Principal Data Engineer to design, build, and maintain scalable, reliable, and secure data platforms on Google Cloud Platform (GCP). The ideal candidate will have deep expertise in modern data engineering practices, cloud-native architectures, data warehousing, orchestration, CI/CD automation, and large-scale data processing. This role will work closely with Analytics Engineers, Data Scientists, Software Engineers, and Business Stakeholders to deliver robust data solutions that power analytics, reporting, and machine learning initiatives. The most important criteria for success in this role is to be proactive rather than reactive and taking ownership. Job Responsibilities: Data Platform Engineering ● Design, develop, and maintain scalable data pipelines and ingestion frameworks. ● Build and optimize ELT/ETL workflows using cloud-native technologies. ● Design and implement data models, data marts, and curated datasets in BigQuery. ● Ensure data quality, lineage, governance, and observability across the data ● Proficient in writing the queries. ● Optimize data processing performance, cost, and reliability. Cloud Data Architecture ● Architect data solutions leveraging GCP services including BigQuery, Cloud Storage, Dataflow, and Cloud Composer (Airflow). ● Implement data lake and data warehouse solutions following best practices. ● Design scalable batch and streaming data processing architectures. ● Develop reusable frameworks and standards for data engineering teams. Workflow Orchestration ● Develop and maintain Apache Airflow DAGs for complex data workflows. ● Implement monitoring, alerting, retry mechanisms, and SLA tracking. ● Troubleshoot and optimize workflow execution and dependencies. CI/CD & DevOps ● Build and maintain CI/CD pipelines using GitLab CI/CD. ● Promote Infrastructure-as-Code practices. ● Implement automated code quality checks and deployment controls. Data Transformation & Analytics Engineering ● Develop and maintain Dataform projects for data transformations. ● Implement testing, documentation, and data quality checks. ● Collaborate with analytics teams to create trusted datasets. Reliability & Operations ● Monitor production systems and resolve data incidents. (Production support) ● Participate in on-call support and operational excellence initiatives. ● Perform root cause analysis and implement preventive measures. ● Improve platform observability through logging, monitoring, and alerting. Technical Leadership ● Mentor junior and mid-level data engineers. ● Conduct code reviews and establish engineering best practices. ● Drive architectural discussions and technical decision-making Technical Skills Required: ● Cloud: GCP ● Data Warehouse: BigQuery ● Orchestration: Apache Airflow (Cloud Composer) ● Storage: GCS ● Processing: Dataflow / Apache Beam ● Transformation: Dataform ● CI/CD: GitLab CI/CD ● Languages: Python, SQL ● Monitoring: Cloud Monitoring, Cloud Logging ● Source Control: GitLab Good To have: ● Event-driven architectures. ● Infrastructure as Code: Terraform. ● CDC (Change Data Capture) pipelines. ● Streaming platforms such as Kafka.