Job Description

Databricks Architect

Location: Boston, MA Hybrid (client wants the associate to be in Boston).

Job Summary: Join our dynamic team as an Architect, where youll leverage your 10 years of experience to design and implement cutting-edge data solutions. Youll work with technologies like Azure Synapse, AWS, and Databricks to drive innovation and efficiency. This hybrid role offers the opportunity to make a significant impact on our data strategy and contribute to our companys success.

Experience Required: 10+ years.

Required Skills

• Python
• Grafana
• Datadog
• Snowflake
• Azure Databricks
• PySpark
• Apache Airflow
• AWS
• Apache Hudi
• Data Build Tool (DBT)
• DataLake
• Databricks CLI
• Azure Synapse Data Warehouse
• Snowflake SQL
• Snowflake Tasks
• Databricks Unity Catalog Admin
• Databricks Delta Lake
• Apache Iceberg
• Fivetran
• Multicloud (AWS / GCP / Azure)
• Databricks Assistant

Responsibilities

• Design and implement scalable data architectures using Azure Synapse and DataLake to optimize data processing and storage solutions.
• Collaborate with crossfunctional teams to integrate AWS, GCP, and Azure Cloud services, ensuring seamless data flow and accessibility.
• Develop and maintain data pipelines using PySpark and Python, enhancing data processing efficiency and reliability.
• Utilize Grafana and Datadog for monitoring and performance tuning, ensuring optimal system performance and availability.
• Implement and manage Apache Airflow for orchestrating complex data workflows, improving automation and process efficiency.
• Leverage Apache Hudi and Apache Iceberg for data versioning and timetravel capabilities, enhancing data management practices.
• Utilize Data Build Tool and Fivetran for data transformation and integration, streamlining data workflows.
• Optimize Snowflake Tasks and Snowflake SQL for efficient data querying and analysis, driving datadriven decisionmaking.
• Administer Databricks Unity Catalog and CLI for data governance and management, ensuring data security and compliance.
• Implement Databricks Delta Lake for reliable data lakes, improving data consistency and reliability.
• Utilize Databricks Assistant and Azure Databricks for collaborative data engineering, enhancing team productivity.
• Ensure data solutions align with business objectives, contributing to the companys strategic goals and societal impact.
• Stay updated with industry trends and emerging technologies, driving continuous improvement and innovation.

Qualifications

• Extensive experience with Azure Synapse and DataLake, demonstrating expertise in data architecture design.
• Strong background in AWS, GCP, and Azure Cloud services, showcasing proficiency in cloud integration.
• Proficiency in PySpark and Python, highlighting skills in data processing and automation.
• Experience with Grafana and Datadog, emphasizing capabilities in system monitoring and performance tuning.
• Proficiency in Apache Airflow, showcasing skills in workflow orchestration and process automation.
• Familiarity with Apache Hudi and Apache Iceberg, demonstrating knowledge in data versioning and management.
• Experience with Data Build Tool and Fivetran, highlighting skills in data transformation and integration.
• Proficiency in Snowflake SQL and Snowflake Tasks, showcasing expertise in data querying and analysis.
• Experience with Databricks Unity Catalog and CLI, emphasizing skills in data governance and management.
• Familiarity with Databricks Delta Lake, demonstrating knowledge in data lake optimization.
• Experience with Azure Databricks, showcasing skills in collaborative data engineering.

Job Location: Primary USMA BOSA02 Boston, MA 745 Atlantic Ave COG. Alternate location declared but not specified.

Job Type: 45CW01 Business Associate BC.

Demand: No travel required.

Certifications Required: Azure Solutions Architect Expert, AWS Certified Solutions Architect, Databricks Certified Data Engineer Associate.

Work Model: Hybrid. SubContractor allowed.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

Job Tags

Similar Jobs