Full-Time Databricks Architect - US Based Remote

We are seeking a talented and experienced Databricks Architect to join our growing analytics practice as a full-time team member. This position is US Based remote.

In this role, you will design and implement innovative data solutions, leveraging Databricks and the Medallion Architecture to address complex business challenges for our clients. This is a long-term position offering the opportunity to lead cutting-edge data initiatives while contributing to the strategic growth of our practice.

As a key leader in our team, you will work closely with both clients and internal stakeholders to deliver robust, scalable, and automated data architectures that support advanced analytics and reporting needs. This role requires a blend of technical expertise, hands-on development, and collaboration skills to achieve impactful outcomes.

Key Responsibilities include:

• Design and implement scalable, high-performance data architectures using Databricks and the Medallion Architecture.
• Lead requirement-gathering sessions with clients and internal teams to understand business needs and define best practices for solutions including data workflows.
• Collaborate with cloud platform teams to optimize data storage and retrieval in environments like AWS S3, Azure Data Lake, and Delta Lake.
• Translate complex data processes, such as those in Alteryx and Tableau, into optimized Databricks workflows using PySpark and SQL.
• Develop reusable automation scripts to streamline workflow migrations and improve operational efficiency.
• Provide hands-on development and troubleshooting support to ensure smooth implementation and optimal performance.
• Partner with cross-functional teams to establish data governance frameworks, best practices, and standardized reporting processes.
• Deliver training, documentation, and ongoing support to empower users and enhance organizational data literacy.
• Stay ahead of industry trends, identifying opportunities to integrate new tools and methods that enhance the practice's capabilities.

Qualifications

Technical Expertise

• 7+ years’ experience in data engineering or software development, with a strong focus on data architecture
• Extensive experience (3-5+ years) with Databricks and the Medallion Architecture.
• Expertise in Python, including developing production-grade data pipelines, reusable automation scripts, and leveraging libraries such as Pandas, and NumPy for advanced data manipulation and analytics.
• Proficiency in PySpark and SQL with in-depth experience integrating a wide variety of data sources with Databricks.
• Expertise in cloud platform services (AWS, Azure, GCP), including their data storage solutions (e.g., S3, Azure Data Lake Storage, Google Cloud Storage), with hands-on experience in designing, optimizing, and integrating scalable data pipelines and architectures across one or more of these platforms.
• In-depth experience with Delta Lake, including schema enforcement, time travel, and optimization for ACID transactions.
• Strong understanding of data pipelines, transformation workflows, and modeling for analytics and reporting, including scheduling, monitoring, and error handling in distributed data processing environments.
• Experience in building automation tools or reusable scripts to accelerate data engineering processes.
• Knowledge of Alteryx, Tableau, and Tableau Server integration is a plus.

Leadership and Collaboration

• Proven ability to lead technical workshops, gather complex requirements, and translate them into actionable solutions.
• Strong communication skills to collaborate with clients, internal teams, and stakeholders at all levels.
• Experience mentoring team members or supporting the professional growth of peers.

What is the last date for applying to the job?

Which countries are accepted for this remote job?