Senior Data Engineer
SparkCognition
Who We Are & Why Join Us
Avathon is revolutionizing industrial AI with a powerful platform that enables businesses to harness the full potential of their operational data. Our technology seamlessly integrates and contextualizes siloed datasets, providing a 360-degree operational view that enhances decision-making and efficiency. With advanced capabilities like digital twins, natural language processing, normal behavior modeling, and machine vision, we create real-time virtual replicas of physical assets, enabling predictive maintenance, performance simulation, and operational optimization. Our AI-driven models empower companies with scalable solutions for anomaly detection, performance forecasting, and asset lifetime extension—all tailored to the complexities of industrial environments.
Cutting-Edge AI Innovation – Join a team at the forefront of AI, developing groundbreaking solutions that shape the future.
High-Growth Environment – Thrive in a fast-scaling startup where agility, collaboration, and rapid professional growth are the norm.
Meaningful Impact – Work on AI-driven projects that drive real change across industries and improve lives.
Learn more at: Avathon
As a Senior Data Engineer in the Engineering team:
Join us in:
- Building and optimizing data architectures to handle large-scale data from various sources, including cloud and on-premise systems.
- Architecting scalable ETL and data workflows to support advanced analytics and machine learning models.
- Collaborating with cross-functional teams to deliver end-to-end data solutions that drive business insights.
- Owning and leading projects that focus on data platform modernization, ensuring high performance, security, and scalability.
- Working on complex data engineering challenges that push the boundaries of cloud data infrastructure.
You'll have:
- A Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.
- 5+ years of experience designing and implementing large-scale data platforms, with at least 2 years of experience leading or mentoring data engineers.
- Hands-on experience with Python (Pandas, Polars) and modern big data tools such as Databricks (Spark), Flink, or Kafka.
- Strong understanding of data pipeline development, ETL frameworks, and data lakes/warehouses.
- Extensive experience with cloud data platforms such as AWS (S3, Redshift, Lambda, Glue), GCP (BigQuery, Dataflow, Dataproc), or Snowflake.
- Familiarity with workflow orchestration tools like Airflow or Prefect.
- Proven ability to lead data engineering teams and mentor junior engineers.
It would be great if you have:
- Experience working with IoT or real-time streaming data.
- Knowledge of data governance frameworks and best practices.
- Exposure to MLOps workflows and integrating data with machine learning models.
- Experience in the energy or manufacturing sector or familiarity with time-series data.
- A strong understanding of data warehousing concepts, including Kimball model design.
- A deep understanding of data security and compliance requirements, particularly in cloud environments.
**A PLUS if you provide the link to your GitHub Website
Avathon is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, pregnancy, genetic information, disability, status as a protected veteran, or any other protected category under applicable federal, state, and local laws.
Avathon is committed to providing reasonable accommodations throughout the recruiting process. If you need a reasonable accommodation, please contact us to discuss how we can assist you.