Data Engineer - GCP

Core AI Consulting Inc · Phoenix, AZ
Pay · estimate~$120k – $160kMarket rate · not set by employer
PostedNov 20, 2026
LevelMid level
In short We are seeking a highly skilled GCP Data Engineer with 5+ years of experience designing and building scalable, production-grade data platforms.
SQLPYTHONGCPSNOWFLAKESPARK

📋 About the job

We are seeking a highly skilled GCP Data Engineer with 5+ years of experience designing and building scalable, production-grade data platforms. This role requires deep expertise in Google Cloud Platform, Python, PySpark, SQL, and distributed data processing across the complete data lifecycle—including ingestion, transformation, orchestration, storage, quality, and serving.

The ideal candidate is a hands-on engineer with strong experience building batch and real-time streaming pipelines, developing reusable Python data frameworks, optimizing large-scale workloads, and designing high-performance data models in BigQuery. The engineer will help establish reliable, secure, observable, and cost-efficient data solutions using modern GCP architecture and engineering practices.

Key Responsibilities

  • Design, develop, and maintain enterprise-scale batch and streaming data pipelines using Python, PySpark, Dataflow, Dataproc, Pub/Sub, BigQuery, and Cloud Composer
  • Build modular, reusable, and testable Python frameworks for data ingestion, transformation, validation, and pipeline automation
  • Develop high-performance data-processing solutions using PySpark, Spark SQL, and Apache Beam
  • Integrate data from relational and NoSQL databases, REST APIs, flat files, cloud storage, third-party platforms, and event streams
  • Build real-time streaming pipelines using Pub/Sub and Dataflow , including windowing, triggers, watermarking, late-arriving data, deduplication, and delivery semantics
  • Design event-driven architectures supporting real-time analytics, operational reporting, and downstream data products
  • Create and maintain workflow orchestration using Cloud Composer and Apache Airflow , including reusable operators, dependency management, retries, alerting, and recovery
  • Design scalable BigQuery schemas using dimensional, normalized, denormalized, and lakehouse modeling patterns
  • Optimize BigQuery workloads through effective partitioning, clustering, materialized views, query tuning, and cost-management strategies
  • Tune Spark workloads by resolving data skew, excessive shuffling, inefficient joins, memory constraints, partitioning issues, and small-file problems
  • Develop incremental loading, change data capture, and high-volume data-processing solutions
  • Implement automated data-quality checks, reconciliation, schema validation, lineage, logging, monitoring, and alerting
  • Develop Python-based data services and APIs using frameworks such as FastAPI…
📄 Read the full description → The rest of this listing is on the hiring partner’s site.
Interested in this role?Apply in one click — no JobQ account needed.
Apply now →
Data Engineer - GCPCore AI Consulting Inc
Apply now