California Licensed Databricks Engineer

Job Summary

A company is looking for a Databricks Data Engineer to work remotely in California for a 12-month duration.

Key Responsibilities

• Manage and optimize the Databricks platform, including workspace management and job scheduling
• Develop and maintain ETL pipelines using Azure Data Factory and ensure data quality and governance
• Utilize Spark and Python for data manipulation, automation, and advanced querying

Required Qualifications

• Expertise in the Databricks platform and Spark (PySpark/Scala)
• Experience with SQL optimization and data warehousing concepts, including Delta Lake
• Proficiency in Python scripting and familiarity with Azure Cloud services
• Knowledge of data governance, security, and compliance standards
• Experience with BI/reporting tools such as Power BI or Tableau