Sr. Data Engineer
8306 Landing Lane Pittsburgh, PA 15108 | Direct Hire
The Senior Data Engineer is a key member of a product team that contributes to software design, development and overall product lifecycle for a product that delights our users and adds value to the organization. The engineering process is highly collaborative. The Senior Data Engineer is expected to pair on a daily basis as they work through user stories and support products as they evolve. In addition, the Senior Data Engineer may be involved in product configuration, performance tuning and testing as well as production monitoring. As a Senior Data Engineer, you will be part of a team with more experienced engineers to help build and grow your skills while you create, support, and deploy production applications.
Identify, design and implement internal process improvements (automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc).
Build the infrastructure required for optimal extraction, transformation and loading of data from a variety of data sources using SQL, SQOOP, Python, Google Big Query and other Big Data technologies
Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.
Create data tools for analytics and data scientist team members that assist them in building and optimizing our data systems.
Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
Experience with NoSQL database technologies (MongoDB, Cassandra, etc)
Experience with Hadoop and other Big Data technologies such as Spark, PySpark and Kafka
Experience with message queuing, stream processing, and highly scalable ‘ big data’ data stores.
Experience with relational database design methodologies and authoring complex SQL queries
Experience with Agile Development and Agile Deployment tools and versioning using Git or similar tools
Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
Experience building data pipelines utilizing Google Cloud platform preferred but not required.