Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

VitqbjJHbjJCQ0VoSG5WcERURjBkdE5BTmc9PQ==
  • Purple Drive
  • Boston, MA

Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

Providence Health & Services

Social Worker MSW - Palliative Care Job at Providence Health & Services

 ...services to terminally-ill patients, and/or palliative services to patients and their families....  ...culture of patient-focused, whole-person care built on understanding, commitment, and...  ...pharmacy services, home medical equipment, hospice and palliative care, Program of All-... 

10X Health System

Compliance & Risk Management Specialist Job at 10X Health System

 ...~ Prolonged periods of sitting at a desk COMMITMENT TO DIVERSITY As an equal opportunity employer committed to meeting the needs...  ..., sexual orientation, or disability, and actively foster inclusion in all forms both within our company and across interactions with... 

Liberty Healthcare Corporation

Assessor - developmental disabilities (hybrid) Job at Liberty Healthcare Corporation

If you gain joy and fulfilment through serving, supporting and caring for people who have developmental disabilities and autism - you are encouraged to consider an outstanding career opportunity with Liberty Healthcare. Liberty Healthcare helps people who have disabilities...

American Airlines

Manager, Airport Customer Operations Job at American Airlines

 ...the Hubs and Gateways Team, within the Customer Experience Division.+ Responsible for...  ...expenses+ Oversees day to day customer service operations+ Solicits ideas and/or feedback...  ...airline supervisory experience3 years' airport operations experience**Preferred Qualifications... 

Siebert Williams Shank

Infrastructure & Public Finance Investment Banking Analyst (Chicago) Job at Siebert Williams Shank

 ...experienced Investment Banking Analyst to join its Infrastructure and Public team and work hybrid (4 days in-office) from its Chicago, IL office. At SWS, all bankers are an integral part of the financing team, assume a high level of responsibility, and actively...