Data Engineer
HireTalent | Newark | www.resume-library.com |
High level requirements for the role:
Data ingestion from various on-prem and cloud sources
Ability to build ETL pipelines to transform raw data into format suitable for analysis (hands on experience working with ETL tools like AWS Glue, Informatica etc.) and orchestrate transformation (using Airflow, other DAG tools)
Experience working with big data technologies and data formats (e.g., Parquet, Delta, Iceberg)
Expertise in Python, Spark and SQL programming languages
Strong experience working on cloud platforms (AWS, Azure)
Proven track record of enterprise data modeling including building schemas for data warehousing
Knowledge of real-time data streaming and real-time analytics
Knowledge of Microsoft suite of tools is desired (Power BI, Synapse suite of analytics tools, Azure Data Factory)
Your Impact:
Data ingestion from various on-prem and cloud sources, optimize data for large data processing
Build ETL pipelines to collect, blend, transform raw data into suitable formats for analysis (hands on experience working with ETL tools like AWS Glue, Informatica etc.) and orchestrate transformation (using Airflow, other DAG tools)
Create enterprise data models and define standards for data warehouse management adhering to data model definitions
Interact with data analytics and reporting teams to understand how data needs to be structured for consumption.
Configure and maintains MDM tools and system processes to ensure proper function of MDM mechanisms.
Define master data quality validation criteria and implements automatic error detection schemes.
Implement data lineage capability to understand data movement across the enterprise
Implement data governance and standards including data definitions, data quality rules, data lineage, business glossary functions
Support domain owners and data stewards to analyze quality issue root causes and remediation efforts.
Monitor cross-BU data provisioning to ensure provisioning is done via official sources.
Your Skills and Experience:
Bachelors degree is required in a relevant field.
6+ years of experience working in a data engineering role supporting an on-prem Data Lake; experience within institutional asset management is a plus.
4+ years of strong experience with working on cloud platforms (AWS, Azure) (CloudFormation Templates, IAM, Aurora, EventBridge, Lambda, CloudWatch);
2+ years of experience with Snowflake (SnowSQL, SnowPark)
5+ years of experience with any relational database (SQL, Store procedures, functions)
5+ years of experience with Python, Spark and SQL programming languages .
Extensive hands-on experience in modern data lake architecture, database development, and data modeling.
Knowledge of real-time data streaming and real-time analytics
Knowledge of Microsoft suite of tools is desired (Power BI, Synapse suite of analytics tools, Azure Data Factory)
Strong implementation skills and working knowledge of data structures, algorithms and big data tools (Spark, Hadoop, Python, SQL, NoSQL, Hive); Must have hands-on experience using Spark.
Solid Linux OS and Shell Scripting experience.
Experience working with big data technologies and data formats (e.g., Parquet, Delta, Iceberg)
Extensive experience with at least one of the following RDMS: Oracle, SQL Server, Postgres or MySQL.
Strong communication skills: able to partner with technical and business stakeholders to drive innovative solutions.
Prior experience using data governance technology tools.
Prior experience in a lead role is desired but not required.
Experience working in an agile environment embracing collaboration within and across teams.
Excellent written and verbal communication skills.
Detail oriented; Analytical with strong problem-solving abilities.
Professional and energetic self-starter. Comfortable with ambiguity, able to effectively take the conceptual to the pragmatic.
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender, identity, national origin, disability, or protected veteran status.
Data ingestion from various on-prem and cloud sources
Ability to build ETL pipelines to transform raw data into format suitable for analysis (hands on experience working with ETL tools like AWS Glue, Informatica etc.) and orchestrate transformation (using Airflow, other DAG tools)
Experience working with big data technologies and data formats (e.g., Parquet, Delta, Iceberg)
Expertise in Python, Spark and SQL programming languages
Strong experience working on cloud platforms (AWS, Azure)
Proven track record of enterprise data modeling including building schemas for data warehousing
Knowledge of real-time data streaming and real-time analytics
Knowledge of Microsoft suite of tools is desired (Power BI, Synapse suite of analytics tools, Azure Data Factory)
Your Impact:
Data ingestion from various on-prem and cloud sources, optimize data for large data processing
Build ETL pipelines to collect, blend, transform raw data into suitable formats for analysis (hands on experience working with ETL tools like AWS Glue, Informatica etc.) and orchestrate transformation (using Airflow, other DAG tools)
Create enterprise data models and define standards for data warehouse management adhering to data model definitions
Interact with data analytics and reporting teams to understand how data needs to be structured for consumption.
Configure and maintains MDM tools and system processes to ensure proper function of MDM mechanisms.
Define master data quality validation criteria and implements automatic error detection schemes.
Implement data lineage capability to understand data movement across the enterprise
Implement data governance and standards including data definitions, data quality rules, data lineage, business glossary functions
Support domain owners and data stewards to analyze quality issue root causes and remediation efforts.
Monitor cross-BU data provisioning to ensure provisioning is done via official sources.
Your Skills and Experience:
Bachelors degree is required in a relevant field.
6+ years of experience working in a data engineering role supporting an on-prem Data Lake; experience within institutional asset management is a plus.
4+ years of strong experience with working on cloud platforms (AWS, Azure) (CloudFormation Templates, IAM, Aurora, EventBridge, Lambda, CloudWatch);
2+ years of experience with Snowflake (SnowSQL, SnowPark)
5+ years of experience with any relational database (SQL, Store procedures, functions)
5+ years of experience with Python, Spark and SQL programming languages .
Extensive hands-on experience in modern data lake architecture, database development, and data modeling.
Knowledge of real-time data streaming and real-time analytics
Knowledge of Microsoft suite of tools is desired (Power BI, Synapse suite of analytics tools, Azure Data Factory)
Strong implementation skills and working knowledge of data structures, algorithms and big data tools (Spark, Hadoop, Python, SQL, NoSQL, Hive); Must have hands-on experience using Spark.
Solid Linux OS and Shell Scripting experience.
Experience working with big data technologies and data formats (e.g., Parquet, Delta, Iceberg)
Extensive experience with at least one of the following RDMS: Oracle, SQL Server, Postgres or MySQL.
Strong communication skills: able to partner with technical and business stakeholders to drive innovative solutions.
Prior experience using data governance technology tools.
Prior experience in a lead role is desired but not required.
Experience working in an agile environment embracing collaboration within and across teams.
Excellent written and verbal communication skills.
Detail oriented; Analytical with strong problem-solving abilities.
Professional and energetic self-starter. Comfortable with ambiguity, able to effectively take the conceptual to the pragmatic.
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender, identity, national origin, disability, or protected veteran status.
Don’t miss out on new job openings!
Create a job alert for: Asset Management, Newark
It's free, and you can cancel email updates at any time