Databricks Architect Job at Inizio Partners Corp, Jersey City, NJ

NkJNams4bERLMnUwekZiWUM3anFENU9GRFE9PQ==
  • Inizio Partners Corp
  • Jersey City, NJ

Job Description

Job Description

Job Description

  • Title: Databricks Architect
  • Key Requirements: Hands on experience in Databricks, SQL, PySpark. Must have Insurance domain experience.
  • Location: Jersey City, NJ
  • Job Type: Hybrid (2 days work from office)
  • Duration: 12+ Months
  • Start Date: Immediate
  • Number of roles: 1

Role & Responsibilities Overview:

  • Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.)
  • Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for monitoring
  • Continuously improve systems through performance enhancements and cost reductions in compute and storage
  • Data Processing and API Integration: Utilize Spark Structured Streaming for real-time data processing and integrate data outputs with REST APIs
  • Lead Data Engineering Projects to manage and implement data-driven communication systems
  • Experienced with Scrum and Agile Methodologies to coordinate global delivery teams, run scrum ceremonies, manage backlog items, and handle escalations
  • Integrate data across different systems and platforms
  • Strong verbal and written communication skills to manage client discussions

Candidate Profile:

  • 8+ years experience in developing and implementing ETL pipelines from various data sources using Databricks on cloud
  • Experience in insurance domain/ data is must
  • Programming Languages SQL, Python
  • Technologies - IaaS (AWS or Azure or GCP), Databricks platform, Delta Lake storage, Spark (PySpark, Spark SQL).
  • Good to have - Airflow, Splunk, Kubernetes, Power BI, Git, Azure DevOps
  • Project Management using Agile, Scrum
  • B.S. Degree in a data-centric field (Mathematics, Economics, Computer Science, Engineering or other science field), Information Systems, Information Processing or engineering.
  • Excellent communication & leadership skills, with the ability to lead and motivate team members
  • Ability to work independently with some level of ambiguity and juggle multiple demands

Job Tags

Immediate start,

Similar Jobs

Idaho National Laboratory

Facility Manager Job at Idaho National Laboratory

 ...Job Description Idaho National Laboratory is hiring a Facility Manager to work in our Facility Management department. Our team works an onsite position with 4x10 schedule located out of our Central Facilities Area with every Friday off. You will be responsible for... 

Mercy

RN- Charge, Med/Surg - Nights Job at Mercy

 ...Sign on Bonus for Experienced Nurses - up to $20,000$3,000 Relocation Assistance Available The Medical Surgical unit, also known as Med-Surg, is a specialty unit that makes up the foundation of nursing. The Medical Surgical unit at Mercy South is a unit that manages... 

BridgeBio Pharma

Associate Director/Director, Computational Biologist San Francisco, California, United States Job at BridgeBio Pharma

 ...Raleigh, with small satellites in other parts of the country. We also have international offices in Montreal, Canada, and Zurich, Switzerland, and are expanding across Europe. The Computational Genetics team is seeking a full-time Computational Biologist who will... 

PF Changs Careers

Server Job at PF Changs Careers

 ...and part-time /hourly Prep Cooks, Pastry Cooks or Cooks. Hiring immediately for full-time or part-time hourly restaurant dine-in food and beverage Servers, Waitresses or Waiters. Benefits We Offer: Vacation pay* Benefit plan options for full-time team members... 

Nachhilfeunterricht

Work from home as an private online English Tutor - Part Time Job at Nachhilfeunterricht

At Nachhilfeunterricht we are actively seeking tutors to provide private lessons in English. We are looking for individuals passionate about teaching eager to play a crucial role in the academic development of our students. No prior experience is required to apply for...