Job Description

About Wipro

Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) is a leading technology services and consulting company focused on building innovative solutions that address clients’ most complex digital transformation needs.

We leverage our holistic portfolio of capabilities in consulting, design, engineering, operations, and emerging technologies to help clients realize their boldest ambitions and build future-ready, sustainable businesses.

A company recognized globally for its comprehensive portfolio of services, strong commitment to sustainability and good corporate citizenship, we have over 250,000 dedicated employees serving clients across 66 countries.

We deliver on the promise of helping our customers, colleagues, and communities thrive in an ever-changing world.

· A PROUD HISTORY OF OVER 75 YEARS

· FY22 REVENUE 10.4 BN USD

· WE’RE PRESENT IN 66 COUNTRIES

· OVER 1,400 ACTIVE GLOBAL CLIENTS

Position: DataHub Developer

Location: Austin, TX

Mode: Full-Time or Contract

Job Description: DataHub Developer with Committer Experience

Position Overview

We are looking for an experienced DataHub Developer with Committer Experience to join our team and contribute to the design, development, and optimization of enterprise metadata management and data lineage solutions. The ideal candidate will have strong expertise in data cataloging , data lineage , data governance , and hands-on experience with DataHub , Spark-based frameworks , and machine learning for anomaly detection. This role demands a mix of open-source contribution, technical problem-solving, and metadata management expertise.

Key Responsibilities

1. DataHub Development and Integration

• Lead projects involving metadata cataloging using the DataHub open-source framework.
• Design and develop custom APIs to integrate ETL pipelines and enable real-time metadata ingestion.
• Ingest metadata from multiple systems, including data lakes, upstream, and downstream systems, to provide a holistic metadata ecosystem.
• Customize and extend DataHub to enrich impact analysis by identifying pipelines reading/writing to data assets.

1. Data Lineage and Governance Implementation

• Provide end-to-end data lineage solutions for PII identification, governance, and compliance reporting.
• Develop and implement processes to enhance impact analysis and ensure seamless data governance practices.

1. Spark-Based Framework Development

• Design, develop, and maintain Spark-based custom frameworks for config-as-code mechanisms to facilitate data enrichment and transfer.
• Improve the performance and scalability of Spark applications to ensure seamless data processing.
• Provide recommendations and guidance on the design and development of ETL pipelines using Spark.

1. Machine Learning Integration for Anomaly Detection

• Collaborate with ML engineers to create features from profiled batch data.
• Develop and integrate machine learning models for anomaly detection in data patterns.

1. AWS Cost Optimization and Platform Efficiency

• Lead AWS cost optimization initiatives to enhance platform-wide efficiency.
• Successfully support Spark version upgrades and ensure the platform's scalability and performance.

1. Community Engagement and Contributions

• Act as a committer to the DataHub open-source community by contributing new features, fixing issues, and enhancing documentation.
• Participate in open-source discussions, propose architectural improvements, and represent the organization in community events.

Required Qualifications

• Experience:
• 5+ years in metadata management, data lineage, or data governance roles.
• Proven track record as a committer or active contributor to the DataHub open-source project.
• Technical Skills:
• Proficiency in Java , Python , and REST API development.
• Strong experience with Apache Spark for ETL pipeline design and custom framework development.
• Expertise in metadata ingestion from systems like data lakes, databases, and ETL tools.
• Hands-on experience with AWS services and cost optimization strategies.
• Familiarity with machine learning techniques for anomaly detection.
• Other Skills:
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.

Preferred Qualifications

• Knowledge of data governance regulations like GDPR , CCPA , or HIPAA .
• Experience with infrastructure-as-code tools such as Terraform or Helm .
• Familiarity with other metadata management tools like Amundsen , Collibra , or Alation .
• Understanding of version control, CI/CD pipelines, and open-source development practices.

Wipro is an Equal Employment Opportunity employer and makes all employment and employment-related decisions without regard to a person's race, sex, national origin, ancestry, disability, sexual orientation, or any other status protected by applicable law. Any complaints or concerns regarding unethical/unfair hiring practices should be directed to our Ombuds Group.

The potential compensation for this role is based on labor costs in local markets, as well as the job-related skills, knowledge and experience of the candidate. Based on the position, the role is also eligible for Wipro’s standard benefits and additional compensation offerings, including a full range of medical and dental benefits options, disability insurance, paid time off (inclusive of sick leave), other paid and unpaid leave options as well as potential incentive or variable compensation.

Job Tags

Similar Jobs