Req ID: 134411
NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.
We are currently seeking a Senior Data Engineer to join our team in Pittsburgh, Pennsylvania (US-PA), United States (US).
- Build data streams to ingest, load, transform, group, logically join and assemble data ready for data analysis/analytics/ reporting, build data streams to ingest, load, transform, group, logically join and assemble data ready for data analysis/analytics/ reporting.
- Pipeline data using Cloud: Databricks, AWS Big Data Services, etc.
- Responsible to write Pyspark code using Databricks to connect databases, AWS services to transform data.
- Design/Implement QA framework within the Data-Lake. Design and implement test strategies, write test cases, design/implement test automation
- Responsible for maintaining integrity between Data Lake, databases.
- Work on Data Lake/Delta Lake Data Pipeline to take data across from the source all the way to the consumption layer.
- Maintain knowledge and proficiency of current and upcoming hardware/software technologies. Mentor junior staff in ramping up analytical and technical skills.
- Must have 5+ years of Data Systems/Warehouse/Lakes/Equivalent system experience with multiple OS platforms
- Must have 3 + years of Python and Pyspark.
- Must have 3 + years of AWS or alternative cloud systems.
- 1+ year of Matallion/IBM Data Stage or equivalent ETL tool.
- Strong knowledge of Databricks Data Lake/Delta Lake developments
- Strong knowledge of AWS data-related services (DMS, Glue, EMR, S3, Athena, Lambda, Redshift, DynamoDB, KMS).
- Strong knowledge of Python and Pyspark (Hive).
- Strong database/relational/non-relational concepts required.
- Strong analytical and problem-solving skills.
- Databricks, oracle, SQL, Hadoop, Kafka, Spark, Scala (preferred).
- Must possess or develop business knowledge of how customer transactions reflect the business logic that drives the existing or future code.
- Must possess or develop the ability to converse with the business, development, operations, carriers, vendors, etc.
- Strong experience of different architectural components comprising the middleware is required.
- Ability to communicate (oral/written) effectively to exchange information with our clients.
- The shift timings can be changed as per client requirements. Additionally, resources may have to do overtime and work on weekend’s basis business requirements.
- The position will be temporarily remote but will revert to on-site with the client in Pittsburgh.
About NTT DATA Services
NTT DATA Services is a global business and IT services provider specializing in digital, cloud and automation across a comprehensive portfolio of consulting, applications, infrastructure and business process services. We are part of the NTT family of companies, a partner to 85 % of the Fortune 100.
NTT DATA Services is an equal opportunity employer and will consider all qualified applicants for employment without regard to race, gender, disability, age, veteran-status, sexual orientation, gender identity, or any other class protected by law.