Full-Time Senior Devops Engineer

Devops IDI DevOps Engineer – IDI (Nearshore)

Tech Stack: Azure DevOps, Databricks CLI, Terraform, Python, CI/CD Pipelines, Azure Monitor, KeyVault

Key Interfaces: Developers, QA, Deployment Engineer, Onshore DevOps Lead, Technical Delivery Manager

1. Build and maintain deployment pipelines for data ingestion and analytics. Build and maintain CI/CD pipelines for ADF, Databricks notebooks, Spark jobs, and Power BI using Azure Pipelines.
2. Configure monitoring and alerting using Azure Monitor and Log Analytics for pipeline health and cost tracking.
3. Automate provisioning of compute clusters and environments for Spark jobs
4. Integrate security, logging, and monitoring tools for data platforms. Enforce security: manage RBAC, Key Vaults, data encryption (Voltage), and secure deployment practices.
5. Support rollout of Databricks notebooks, workflows, and ADF pipelines.
6. Integrate and automate Event Hubs, ADF, Databricks, and Spark job orchestration
7. Partner with QA and Data Engineers for test environment readiness

Required Experience:

• 5+ years in DevOps with strong data pipeline experience
• Familiarity with Databricks deployment and infrastructure-as-code
• Strong Azure DevOps, Terraform, and scripting (Python) experience
• Experience supporting large-scale ETL jobs and analytics platforms

Technical Skills (Must-Have):

• Azure DevOps / Azure Pipelines (YAML-based pipelines, service connections)
• Azure Data Factory (ARM or template-based deployment, JSON configs)
• Azure Databricks (notebook versioning, dbutils, CI/CD integration)
• Python & Scala (for Spark jobs and deployment scripts)
• Azure Monitor, Log Analytics, Application Insights
• Azure Storage (ADLS Gen2), Event Hubs
• Power BI (workspace deployment, REST API usage)
• RBAC, Key Vault, Azure Security Center
• Git (repo strategy, branch mgmt, PR automation)

Nice-to-Have:

• Azure CLI, PowerShell scripting for ad hoc resource mgmt
• Purview integration (lineage tracking, data classification policies)
• Cost Management tools in Azure + 3rd-party (e.g., CloudHealth, FinOps practices)
• Experience with Voltage SecureData or similar (tokenization, masking frameworks)
• Experience with secrets management at scale (Key Vault + pipeline secrets)

What is the last date for applying to the job?

Which countries are accepted for this remote job?