Providing innovative, comprehensive, and end-to-end information technology and workforce management solutions that empower businesses to achieve digital transformation, enhance operational efficiency, and drive sustainable growth

Job Description

We are seeking a Senior Databricks Platform Architect to lead the design, deployment, security, and governance of enterprise-scale Databricks platforms on AWS. This is a highly technical, customer-facing role requiring deep expertise in Databricks Platform Architecture, Terraform Infrastructure as Code (IaC), and AWS cloud infrastructure.

The ideal candidate will have extensive hands-on experience designing, configuring, deploying, administering, and securing Databricks Classic and Serverless Workspaces in production environments. The architect will work closely with cloud, security, networking, and platform engineering teams to build secure, scalable, automated, and compliant Databricks platforms while leading architecture workshops, technical enablement sessions, and design reviews with enterprise customers.

Key Responsibilties

• Design, configure, deploy, and administer Databricks Classic and Serverless Workspaces on AWS for enterprise-scale production environments.
• Architect secure, scalable, and highly available Databricks platform deployments aligned with enterprise security and governance standards.
• Develop and maintain reusable Terraform modules to provision and manage AWS and Databricks resources using Infrastructure as Code (IaC).
• Automate infrastructure provisioning, configuration, and deployment through CI/CD pipelines and DevOps best practices.
• Design and implement Databricks platform security including Workspace isolation, Identity and SSO integration, SCIM provisioning, Cluster policies, Cluster access controls, Secret Scopes, Credential management, Network security architecture.
• Configure and manage Databricks platform administration, workspace settings, access controls, and production deployments.
• Design AWS infrastructure supporting Databricks, including Amazon S3, IAM Roles and Policies, VPC architecture, Security Groups, PrivateLink, Route Tables, NAT Gateways, Load Balancing, DNS and networking connectivity
• Collaborate with cloud infrastructure, networking, security, DevOps, and platform engineering teams to validate architecture decisions and resolve deployment challenges.
• Lead customer-facing architecture workshops, platform enablement sessions, technical reviews, and solution design discussions.
• Guide enterprise customers through platform security, compliance, governance, and accreditation processes.
• Produce architecture documentation including High-Level Design (HLD), Low-Level Design (LLD), Network diagrams, Security architecture, Platform runbooks, Operational documentation
• Establish architecture standards, deployment best practices, and operational procedures for enterprise Databricks platforms.
• Troubleshoot complex platform, networking, infrastructure, identity, and deployment issues across AWS and Databricks environments.
• Mentor engineering teams on Databricks platform architecture, Infrastructure as Code, security, and operational excellence.

Required Qualifications

• Bachelor's degree in computer science, Information Technology, Engineering, or a related technical discipline.
• 10+ years of experience in Cloud Infrastructure, Platform Engineering, Solution Architecture, or Cloud Engineering.
• 3–5+ years of hands-on experience designing, configuring, deploying, administering, and securing Databricks on AWS.
• Extensive experience deploying and managing both Databricks Classic and Serverless Workspaces.
• Strong expertise in Databricks Platform Administration, including Workspace architecture, Workspace configuration, Platform administration, Production deployments, Workspace lifecycle management
• Proven experience implementing Databricks security including Identity Federation, SSO integration, SCIM provisioning, Cluster Policies, Cluster Access Controls, Secret Scopes, Workspace Isolation, Secure platform configuration
• Strong hands-on experience with Terraform and Infrastructure as Code (IaC) for provisioning and managing AWS and Databricks resources.
• Experience implementing CI/CD pipelines, infrastructure automation, deployment administration, and DevOps best practices.
• Strong working knowledge of AWS cloud infrastructure, including Amazon S3, IAM, VPC, PrivateLink, Security Groups, Route Tables, Networking, Load Balancers, Identity services
• Experience designing secure enterprise cloud platforms following security and governance best practices.
• Demonstrated ability to independently lead Architecture workshops, technical enablement sessions, Design reviews, Customer technical discussions, Executive-level architecture presentations
• Experience creating architecture documentation, implementation guides, platform diagrams, operational procedures, and runbooks.
• Excellent analytical, troubleshooting, problem-solving, verbal, and written communication skills.
• Ability to collaborate effectively with cloud, networking, security, DevOps, and platform engineering teams.

Preferred Qualifications

• AWS Certified Solutions Architect (Associate or Professional).
• Databricks Certified Data Engineer, Platform Administrator, or Solution Architect certification.
• Experience implementing enterprise cloud governance and security frameworks.
• Experience working within Banking, Financial Services, Insurance (BFSI), Healthcare, or other highly regulated industries.
• Experience designing and deploying multi-account AWS environments using AWS Organizations and Landing Zones.
• Knowledge of enterprise compliance frameworks including SOC 2, PCI-DSS, HIPAA, ISO 27001, and NIST.
• Experience with Unity Catalog, Databricks governance, and enterprise data security.
• Experience integrating Databricks with enterprise identity providers such as Azure AD (Microsoft Entra ID), Okta, or Ping Identity.
• Experience with GitOps methodologies and modern DevOps tooling.
• Experience mentoring engineering teams, defining platform standards, and driving architecture best practices across enterprise environments.