Senior Data Engineer

Company: Comcast Technology & Product
Location: Colorado Springs
Posted on: February 15, 2020

Job Description:

Comcast's Technology & Product organization works at the intersection of media and technology. Our innovative teams are continually developing and delivering products that transform the customer experience. From creating apps like TVGo to new features such as the Talking Guide on the X1 platform, we work every day to make a positive impact through innovation in the pursuit of building amazing products that are enjoyable, easy to use and accessible across all platforms. The team also develops and supports our evolving network architecture, including next-generation consumer systems and technologies, infrastructure and engineering, network integration and management tools, and technical standards. Comcast's Technology & Product organization works at the intersection of media and technology. Our innovative teams are continually developing and delivering products that transform the customer experience. From creating apps like TVGo to new features such as the Talking Guide on the X1 platform, we work every day to make a positive impact through innovation in the pursuit of building amazing products that are enjoyable, easy to use and accessible across all platforms. The team also develops and supports our evolving network architecture, including next-generation consumer systems and technologies, infrastructure and engineering, network integration and management tools, and technical standards. Core Responsibilities: -Work directly with the project SCRUM teams to define the solution strategy, performing an integral role in overall project strategy and facilitating effective teamwork and collaboration. -Collaborates with project stakeholders to identify product and technical requirements. Conducts analysis to determine integration needs. -Lead multiple projects at the same time and act as a technical software architect. -Responsible for Planning, Architecture and Design for journey towards Spark, Hadoop Big Data and cloud solutions like AWS, Databricks, etc. -Oversee and contribute to the creation and maintenance of relevant data artifacts (data lineages, source to target mappings, high level designs, interface agreements, etc.). -Socialize architecture and design to the project team consisting of Data Architects, Developers, the QA test team, and the User Acceptance test team. -Ensure that developer responsibilities are being met by mentoring, reviewing code and test plans, verifying that design best practices as well as coding and architectural guidelines, standards, and frameworks are adhered to by offering guidance, communicating risk, and addressing roadblocks as they arise. -Monitors current and future trends, technology and information that will positively affect organizational projects; applies and integrates emerging technological trends to new and existing systems architecture. Mentors team members in relevant technologies and implementation architecture. -Acts as architectural lead on project. -Applies new and innovative ideas to old or new problems. Fosters environments that encourage innovation. -Oversee the project life cycle from intake through delivery. -Develop and test ETL components to high standards of data quality and performance as a hands-on development lead. -Assist with planning and execute releases of ETL components into production. -Develop proofs of concept for new technologies as needed. -Lead in providing operational support for the analytics infrastructure as part of the DevOps model. -Research, identify and recommend technical and operational improvements resulting in improved reliability and/or efficiencies in maintaining and/or developing the application. -Displays in-depth knowledge of engineering methodologies, concepts, skills, and their application in the area of specified engineering specialty. -Displays in-depth knowledge of, and ability to apply, process design and redesign skills. Presents and defends architectural, design, and technical choices to internal audiences. -Evaluate technical aspects of open work requests in the product backlog with the project team. -Participate on interdepartmental teams to support organizational goals -Responsible to deliver Solution design. -Responsible to deliver Best Practices and Standards. -Regular, consistent and punctual attendance. Must be able to work nights and weekends, variable schedule(s) and overtime as necessary. -Other duties and responsibilities as assigned Education Level: Bachelor's Degree in Computer Science, Information & Communication Systems or other related field or equivalent work experience. Required Skills/Experience: -8-10 years of overall IT & software engineering work experience with relevant experience in Big data/Hadoop, AWS, Databricks ecosystems & Business intelligence systems -Work experience as data integration, data solution architect, ETL architect or similar role required. -8-10 years of experience with ETL Architecture, Design and Development. -Hands-on experience in Apache Spark (Scala or Java or Python), Databricks, and Linux/UNIX ecosystems. -Profound experience in designing and developing software applications in Hadoop Ecosystem i.e. Sqoop, Hive, HDFS, MapReduce, Oozie, Zookeeper, Distcp, Pig, Accumulo, Kafka, HBase and Storm etc. -Hands-on experience in implementing applications in AWS Ecosystem i.e. S3, EC2, EMR, Kinesis, Lambda, Glue, Athena, Redshift, DynamoDB, CloudWatch etc. -Extensive experience with one or more ETL Administration platforms & tools (Informatica/Pentaho/Databricks). -Experience in data warehouse concepts/architecture with high volume, real-time streaming transactions and batch ETL jobs. -Working knowledge of Relational Database Management Systems (RDBMS) like Oracle, Teradata, SQLserver etc. and data warehouse query tools like PRESTO etc. -Extensive knowledge of data warehouse and data mart concepts. -Experience with full SDLC and Agile development methodology. -Proficient ability to write complex code/scripts in Spark, Scala or Python, Shell, Java, SQL and C etc. -Experience in Containerizing applications using Kubernetes and Docker etc. -Experience with CI/CD tools like Jenkins, GoCD etc. is preferred. -Experience with information security and access control protocols. -Experience with managed service and onshore/offshore team collaboration is must. -Experience in Debugging, QA Testing and performance tuning. Desired Skills/Experience: -Apache Spark Certified. -Informatica Certified. -Telecommunications experience Knowledge of Telecommunication/Cable billing, customer care systems, marketing data, customer data and privacy data Comcast is an EOE/Veterans/Disabled/LGBT employer

Keywords: Comcast Technology & Product, Colorado Springs , Senior Data Engineer, Engineering , Colorado Springs, Colorado