Big Data Engineer

ClearVUE.Business is an environmental and energy consultancy and technology provider. We believe that data underpins productive net zero strategies. We work with companies across all sectors to define and execute their carbon reduction and energy efficiency strategies within their wider ESG initiatives. Through comprehensive energy and carbon assessments and strategy building and implementation services – underpinned by industry-leading energy and carbon management technology – we are creating revolutionary solutions for businesses to eradicate their impact on the climate one step at a time.

The opportunity

We are seeking a skilled Big Data Engineer to join our team and play a crucial role in driving our data platforms forward. You will have the opportunity to design and implement data pipelines that support our data analytics platform and provide insights into our data. With your expertise, you will be instrumental in enabling our teams to rapidly deliver innovative data-driven solutions that deliver real value to our customers.

What is the job about?

As a Data Engineer, you will be responsible for the design, implementation, and maintenance of data streaming solutions that support our software platforms. You will collaborate with Software Developers, DevOps Engineers, and Security Analysts to ensure that data processing is secure, scalable, and reliable. Your proficiency in event sourcing, stream analytics, and streaming ETL processes will be crucial to further evolve existing pipelines. You will continuously evaluate and implement new technologies and tools to improve data engineering processes and infrastructure.

What exactly will you do?

Responsibilities:

• Work closely with software developers in Malta and Chennai to improve, extend, and revolutionise our IoT ingestion processes as required.
• Take ownership of our data pipelines and ensure data access and processing is high-performing and secure.
• Collaborate with DevOps to automate and maintain a scalable and fault-tolerant data pipeline infrastructure.
• Ensure data integrity and build tools to detect and reduce occurrences of errors.
• Evaluate and propose enhancements to our operations.
• Provide technical expertise and support for data-related projects and initiatives.

What we’re looking for

Requirements:

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience as a software developer working on data projects, data warehouse developer, or big data engineer.
• Familiarity with cloud computing platforms, such as Google Cloud or AWS.
• Good coding and scripting skills on scripting language such as Python, Golang, or JavaScript, ideally js.
• Experience developing and querying databases such as ClickHouse, Redis, or other Big Data stores.
• Experience building real-time streaming ETL processes using NiFi or other Apache projects.
• Experience designing data flows, publishing, and consuming data over Google PubSub, RabbitMQ, Kafka, Pulsar, or other message delivery systems.

What’s in it for you?

• Industry-leading compensation package: Negotiable base salary of €50,000 – 60,000 (DoE).
• Learn from industry-leading experts: We have cultivated some of best talent in the industry, allowing you to supercharge your career progression and learn from the best minds out there.
• Work-life balance: No late finishes – keep your plans with friends and family.
• Unrivalled perks (including but not limited to):
  • 24 days annual leave + public holidays.
  • Full month of December off (fully paid).