Databricks Architect Job at Inizio Partners Corp, Jersey City, NJ

NkJNams4bERLMnUwekZiWUM3anFENU9GRFE9PQ==
  • Inizio Partners Corp
  • Jersey City, NJ

Job Description

Job Description

Job Description

  • Title: Databricks Architect
  • Key Requirements: Hands on experience in Databricks, SQL, PySpark. Must have Insurance domain experience.
  • Location: Jersey City, NJ
  • Job Type: Hybrid (2 days work from office)
  • Duration: 12+ Months
  • Start Date: Immediate
  • Number of roles: 1

Role & Responsibilities Overview:

  • Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.)
  • Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for monitoring
  • Continuously improve systems through performance enhancements and cost reductions in compute and storage
  • Data Processing and API Integration: Utilize Spark Structured Streaming for real-time data processing and integrate data outputs with REST APIs
  • Lead Data Engineering Projects to manage and implement data-driven communication systems
  • Experienced with Scrum and Agile Methodologies to coordinate global delivery teams, run scrum ceremonies, manage backlog items, and handle escalations
  • Integrate data across different systems and platforms
  • Strong verbal and written communication skills to manage client discussions

Candidate Profile:

  • 8+ years experience in developing and implementing ETL pipelines from various data sources using Databricks on cloud
  • Experience in insurance domain/ data is must
  • Programming Languages SQL, Python
  • Technologies - IaaS (AWS or Azure or GCP), Databricks platform, Delta Lake storage, Spark (PySpark, Spark SQL).
  • Good to have - Airflow, Splunk, Kubernetes, Power BI, Git, Azure DevOps
  • Project Management using Agile, Scrum
  • B.S. Degree in a data-centric field (Mathematics, Economics, Computer Science, Engineering or other science field), Information Systems, Information Processing or engineering.
  • Excellent communication & leadership skills, with the ability to lead and motivate team members
  • Ability to work independently with some level of ambiguity and juggle multiple demands

Job Tags

Immediate start,

Similar Jobs

Glean.info

Business Development Representative Job at Glean.info

 ...more humane. We believe that AI will fundamentally transform how people work. In the future, everyone will work in tandem with expert AI...  ...synthesize information, and execute work. These assistants will free people up to focus on the higher-level, creative aspects of... 

MeetFrank

Staff Backend Engineer, Channels Foundation Reddit 4 (1) Hybrid-remote San Francisco Job at MeetFrank

This job opening is already off the market.Get the app or see similar offers.About the Role Reddit is a community of communities. Its built on shared interests, passion, and trust and is home to the most open and authentic conversations on the internet. Every day, Reddit... 

Seven Law Group, APC

Senior Litigation Attorney - Plaintiff's Lemon Law Firm Job at Seven Law Group, APC

 ...Experienced LitigationAttorney (Century City) Our office exclusively handles Lemon Law Claims against all major manufactures such as, General Motors, Jeep,Ford, KIA, Mercedes-Benz, McLaren, Bentley, and more. Our office has hundreds of active cases. The ideal candidate... 

Jackson and Coker

Podiatric Medicine Physician Job at Jackson and Coker

Podiatrist Locums Job in California Details/Requirement: Podiatrist needed for clinic Possible inpatient work too Monday-Friday, 8am-5pm Monday-Thursday option too Three month need starting ASAP Jackson and Coker offers: + Weekly Direct Deposit +...

Southport Contracting Inc

Licensed Plumber Job at Southport Contracting Inc

We are looking for a Licensed Journeyman Plumber. You must have P-2 or S-2 License . Commercial work with opportunities to join the local union and you must be able to work alone or with an ; Hourly wage ($ ) + benefits (Medical, and pension). Most jobs are locat...