Azure Databricks Engineer

Adastra is a global IT company, offering premier class services and solutions in the area of Information Management. We work in international teams and on projects for leading global financial, automotive, telecommunications and insurance pioneers. We pride ourselves in accepting the challenge of mastering new technologies with each new task and in empowering our clients in achieving results. What gives us satisfaction is the opportunity to apply innovation in our jobs daily, inclusive of new areas such as Big Data, Artificial Intelligence, Internet of Things, Blockchain, Cloud, Data Monetization and Robotic Process Automation.

We are seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable data solutions on Microsoft Azure. The role focuses on building reliable, high-performance batch and streaming data pipelines using Azure Databricks, PySpark, and Delta Lake to support analytics, reporting, and AI initiatives. The successful candidate will be skilled in data ingestion, transformation, governance (including Unity Catalog), and pipeline automation in Azure environments.

Responsibilities:

• Design, develop, test, and optimize batch and streaming data pipelines using Azure Databricks, PySpark, Delta Lake, and SQL.

• Build scalable ETL/ELT processes and reusable transformation and validation frameworks.

• Implement data ingestion frameworks from various sources (APIs, databases, flat files, cloud storage) and integrate with Azure Data Factory.

• Configure and maintain Databricks clusters, jobs, workflows, and notebooks; tune Spark jobs for performance and cost efficiency.

• Integrate Databricks with Azure Data Lake Storage Gen2, Azure Synapse Analytics, and other Azure services as required.

• Implement and operate CI/CD processes for data engineering deployments, including notebooks, jobs, and infrastructure-as-code.

• Implement data quality, lineage, and governance standards; manage access controls and security policies using Unity Catalog and other governance tools.

• Monitor and troubleshoot production workloads and pipelines; participate in incident response, root-cause analysis, and continuous improvement.

• Collaborate with Data Architects, Data Scientists, BI Developers, and Business Analysts to translate business requirements into technical solutions and participate in architecture and design reviews.

• Maintain technical documentation, runbooks, and operational playbooks for pipelines and systems owned.

Requirements:

• Over 3.5 years of professional experience as a Data Engineer or in a closely related role working on cloud-based data platforms.

• Over 3 years of hands-on experience with Microsoft Azure services, including practical experience with Azure Databricks.

• Proven hands-on experience developing data processing jobs with Apache Spark / PySpark and applying Spark performance tuning techniques.

• Strong proficiency in Python and SQL for data transformation, validation, and analytics queries.

• Experience with Delta Lake and designing ETL/ELT patterns for reliable, repeatable pipelines.

• Experience integrating and operating Azure Data Factory and ADLS Gen2 within data pipelines.

• Familiarity with Databricks workspace concepts, cluster configuration, job orchestration, and day-to-day control-plane/compute-plane operations.

• Experience implementing data governance and access controls; specific experience with Unity Catalog for governance and fine-grained access control.

• Practical experience with CI/CD for data pipelines and basic infrastructure-as-code concepts (e.g., Git, Azure DevOps, GitHub Actions, Terraform, ARM/Bicep).

• Solid understanding of data modeling and data warehousing concepts and practices.

• Strong analytical, communication, and collaboration skills; ability to work with technical and non-technical stakeholders.

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience.

What Adastra can offer you:

• Diverse career growth opportunities

• Extensive portfolio of technical trainings

• Access to certification programs

• Leadership and soft skills trainings

• Exposure to challenging projects in an international environment

• Opportunity to work with the latest technologies in the Data field

• Multicultural, dynamic, and learning-focused working environment

• Extended Additional Health insurance package

• Multisport cards

• Additional annual paid leave days

• Opportunity to work from home

• Recharge program for a better balance between work and personal commitments

• Referral bonus program

• Corporate social and charity initiatives and Events

• Regular team buildings and office parties

• Three office locations in Bulgaria with a relax and gaming area

Are you ready to join a dynamic team of Information Management professionals committed to deliver best-in class solutions and services to the world’s leading organizations? Apply now or send your resume in English. We thank all interested applicants and will only contact a short list of them.