Sr Data Engineer Location : MN (remote until covid pandemic) Please share linkedin link along with resume Dairy Foods is seeking a Sr Data Engineer to join our Dairy Foods Data & Analytics Product Team. This role is critical to the success of leveraging data to optimize business outcomes. This role will be an integral team member of our scrum team, leading our transition to the snowflake modern data platform from several historical platforms (Hadoop’s Enterprise Data Lake, Oracle Business Inteligence.) Required Education and Experience - Bachelor's degree in Computer Science, MIS, or related field
- 5+ years of experience building data integration & warehousing/data lake solutions using tools like Informatica, Talend, Mulesoft, Qlik, etc.
- 2+ years of experience working with cloud-native data solutions on Microsoft Azure, AWS, or Google Cloud platform
- Strong experience leading full lifecycle, large, complex reporting, or data engineering efforts
- Strong experience in working with heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using various data integration technologies (ETL/ELT, data replication/CDC, message-oriented data movement, API design, etc.)
- Experience with DevOps, CI/CD pipelines and automated testing
Required Technical Competencies/skills - Implement data structures using standards and best practices in data modeling, ETL/ELT processes, SQL, database, and other technologies
- Strong working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of database
- Thorough experience building and optimizing data pipelines and data sets
- Deep knowledge of the data vault method, model, and architecture
- Ability to manage the overall data landscape: meta data, processing patterns, and data quality
- A successful history of manipulating, processing and extracting value from large datasets.
- Strong working knowledge of message queuing, stream processing, and highly scalable data stores
Required Soft Skill Competencies - Ability to obtain a clear understanding of business needs and value, developing a detailed vision for the initiative, mapping out the solution, and guiding its implementation
- Demonstrated ability to collaborate across all levels (Engineers, Management, Architects, etc.) & across all skill sets (Data scientists, Data visualization developers, Salesforce developers etc.) particularly in a Product-oriented culture
- Excellent communication and documentation skills, including the ability to distill highly technical concepts and communicate them effectively to non-technical stakeholders
- Ability to function as a technical lead, working closely with developers and data analysts, as well as hands-on implementation
- Capable of using agile methodology and implementing Continuous Integration/Continuous Delivery (CI/CD) pipelines
- Ability to proactively communicate and manage technology and projects risks
- Troubleshooting skills, ability to determine impacts, ability to resolve complex issues, and ability to exercise sound judgment and initiative in challenging situations.
Preferred Education and Experience - Bachelor’s or Master’s degree in Business Analytics, Computer Science, Mathematics, Statistics, or a related discipline; or an equivalent combination of demonstrated skills and experience
- Experience working with Databricks (or Spark) & Qlik technologies (Replicate and Compose)
Experience with any scripting languages, preferably Python | |
