Big Data Engineer
Pittsburgh, PA 15203
To build highly resilient, fault tolerant, batch and real time data pipelines in Google Cloud Platform.
- Develop high performing, Scalable ETL Pipelines and applications in a fast paced and Agile environment.
- Apply principles of SDLC and methodologies like Scrum, CI/CD, Software and Product Security, Scalability, Documentation Practices, Refactoring and Testing Techniques.
- knowledgeable with open source tools and technologies and can use and extend them where appropriate to develop solutions.
- 6+ years of technical experience developing scalable enterprise application using Java or Python.
- Experience designing and building Big Data ETL Pipelines.
- Proficiency in Cloud and Distributed Computing.
- Experience with AWS or Google Cloud Platform.
- Experience using Docker, Kubernetes, CI/CD pipelines and build automation tools like Gradle, Ant or Maven
- Must have strong ANSI SQL skills and experience.
- Experience with cloud-based big data platforms such as DataProc, BigQuery, BigTable and Redshift a plus.
- Experience with orchestration and scheduling platforms like Apache Airflow or Luigi