Senior Data bricks Data Engineer/Architect
Bloomfield, CT- Onsite
12+ months contract
Relevant Exp- 10+ years
Only GC & USC apply
Experience Required
• Minimum 10+ years of experience in Design, Development, Architect, implementation, and analytics solutions
• Minimum 5+ Years of Experience on Data Bricks especially on Architecture side.
• Experience in creating Databricks environment, access pattern, workflows, Schema set up etc.
• Design scalable and efficient data architectures using Databricks.
• Hands on - Design and Develop applications using Databricks.
• Candidates must have hands on in Python core programming and Spark.
o Good experience/knowledge on AWS
• Setup and manage ETL (Extract, Transform, Load) pipelines using Databricks to process and transform large volumes of data.
• From an AWS perspective, candidates need to be very well versed with Terraform, Glue, Lambda, S3 and their usage and practical issues faced when dealing with them.
• At least 5+ Experience on AWS (Glue, S3, lambda at the minimum)
• Ability to design and demonstrate system Architecture with different environments.
• Optimize the performance of Databricks clusters, queries, and workloads.
• Hands on experience on Data Warehousing and SQL
• Experience on Pandas
• Strong expertise in SQL is required.
• Solid experience with git software version control and good
understanding of code branching strategies and organization for code reuse
• Very Strong Data Bricks Architecture/Development knowledge
• Experience with DevOps & CI/CD tools – Jenkins, Cloudbees, Please Build, etc.
• Proficiency working with large data stores and data sets
• Health care knowledge is preferred.
Agile methodology (Jira Preferable)
Roles & Responsibilities
• Must have capability to understand the current Architecture
• Lead the team and individual contribution as well.
• Design, Coding, testing and implementation of cloud-based Data bricks apps
• Support existing code thru enhancing, trouble shooting and maintaining per requirements.
• Implement security best practices, including data encryption, access control, and secure data sharing.
• Support migration work.
• Support to understand current ETL process Python or Scala
• maintain consistent practices for source code management in github
• comply with established CICD practices, and participate in the creation of new CICD methods
• attend daily scrum calls, and actively participate in updating/creating user stories and tasks in Jira
• Work closely with business analysts.
• Lead innovation by exploring, investigating, recommending, benchmarking, and implementing data centric technologies for the platform.
• Collaborating within a project team to solve complex problems.
Lead by example coaching and mentoring those of your peers and mentoring less experienced team members. Operate as part of the extended management team.
Nitesh Jaiswal | Tekgence Inc
6655 Deseo Dr, Suite 104,Irving, TX , 75039 • www.tekgence.com
