Senior Data Engineer (Data Lake) in Woodbridge Township, NJ at ConsultNet

Date Posted: 5/29/2021

Job Snapshot

Job Description

Title:  Senior Data Engineer (Data Lake)

Location:  Iselin, NJ

Duration: 6 months contract with a probable extension

JOB DESCRIPTION:

Requirements

Under the leadership of the Data Management and Analytics organization, the Data Engineer is responsible for delivering data lake, data warehouse, analytics and business intelligence solutions.  The main function of this role is serving as senior technical person for coding, debugging, testing, documenting, and implementing AWS data environments, and maintenance/enhancement to existing data environments.

 

Responsibilities:

  • Design, develop, and test enhancements to Data Lake technology management using the Zaloni Data Platform – Arena.
  • Experience using Lake Formation or other third party tools for expediting creations of data pipelines
  • Significant experience in Hue/Hive SQL
  • Work with complex Data modeling and design patterns for BI/Analytics reporting requirements.
  • Serve as senior technical person for data solutions and architectures in relation to storage, governance, quality, cleansing and distribution.
  • Design the infrastructure and reusable processes for optimal execution of data transformation.
  • Facilitate the evaluation and selection of data toolsets required to achieve business objectives.
  • Provide design decisions around data warehouse solutions that involve multiple disparate business systems.
  • Ensure quality level of documentation is produced and maintained in regards to data solution architecture.
Education/ Experience:
  • Bachelor's Degree in Computer Science or related field.
  • 7 to 10 years' experience in a data engineering environment, report automation, batch and real time processing
  • Experience working in an Agile/Scrum environment
  • Background in banking and financial services industry
Strong data skills required:
  • SQL, ETL/ELT and MDM
  • Python, Spark, PySpark, Jupyter/Zepplin
  • Strong hands on experience in Hive and HiveQL, SQL server
  • AWS Data Lakes related skills Hadoop, HDFS, S3, EMR, Hive, Athena, Redshift, Glue, QuickSight, Apache Ranger
  • Familiarity with Linux Shell Scripting
  • C# is a plus
  • Kafka streaming a big plus
  • AWS Certified is a plus

Be a part of the ConsultNet difference. As a leading national provider of IT staffing and solutions, ConsultNet delivers exceptional services to startup, midmarket and Fortune 1000 companies across North America. Since 1996, we've partnered with clients to create rewarding opportunities for our consultants, successfully building teams that have surefire results.

In the past two years alone, we have placed more than 1,500 consultants in contract, contract-to-hire, or direct placement opportunities. We understand communication is key to finding the right job that matches your skills and career goals. For us, it's not just the work that we do; it's how we do the work. Our breadth of offerings extends to multiple IT positions in major markets throughout the country, see more at - www.consultnet.com