DevOps Engineer required by client. You will engage in discussion with the technical design lead and other technical staff recommending options for implementing technical infrastructure and data products in a way that meets security and privacy responsibilities.
You will work as part of multidisciplinary teams continually seeking to innovate better practices to delivery ensuring that services can continue to deliver value to energy consumers on a sustainable basis.
Key responsibilities:
: Collaborating with teams who are provisioning infrastructure in a DevOps environment
: Ensure principles of privacy, security and resilience are assessed and designed into solutions
: Responsible for implementation and on:going administration of Hadoop infrastructure
: Building and supporting the technical stack
: Evaluating technology for use in the stack
: Integrating technology to provide an end to end architecture
: Aligning with the systems engineering team to propose and deploy new hardware and software environments required for Hadoop and to expand existing environments
: Working with delivery teams to setup new Hadoop users, including setting up Linux users, Kerberos principles and testing HDFS and Hive access for new users
: Monitor and maintain ecosystem connectivity and security to guarantee confidentiality, integrity and availability
: Working with the infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability
: Key outputs and deliverable
: Support and maintain the service infrastructure with appropriate security and privacy controls and meets users needs
: Business solutions comply with privacy and security requirements
: Work in an agile way, collaborating with colleagues
: Provide advice and assurance about the work of junior colleagues
: Deliver new functionality to service platforms, with an emphasis on platform independence, automation and cost:benefit optimisation
: Provide support and documentation to ensure knowledge transfer and retention
: Ensure that governance rules, particularly security standards are always upheld
: Key stakeholder relationships
: The service technical leads and infrastructure engineers
: The Data privacy and information security teams
: The service operational teams
: The service business leads
: The wider DevOps community
: Specific skills and capabilities
Essential:
: Agile software engineering practices such as CI/CT/CD Eagerness to prioritise collaboration and partnering within the team, experience defining and implementing effective data engineering working practices in agile teams and across teams
: Experience with architecture/engineering of cloud based distribution systems (such as Azure)
: Experience of Microservices Architecture and APIs
: Strong knowledge of Azure and its offerings and the Hadoo ecosystem using Hortonworks (preferred) or Cloudera
: Cloudbreak, Hive, Ambari, Sqoop, Oozie, SPARK, Atlas, Ranger, HBase, HDFS, YARN and ELK
: Hands on experience connecting cloud infrastructure to on:prem networks
: VPNs, tunnelling, AD connectors, routing
: Knowledge of network protocols such as TCP, UDP, HTTP/HTTPS, SSL/TLS, and APIs
: Authentication technologies. i.e. LDAP, OAuth, 2FA, SAML, and SSO via Kerberos
: Bash, Python, Java or Ruby
: Experienced with security controls
: VPNs, encryption and key/certificate management, endpoint protection, virtual firewalls/ACLs/NSGs, setting up bastion nodes, etc
: Background in distributed systems
: A deep understanding of distributed system design and dependency management
: Operational expertise
: A strong grasp of protective monitoring tools, approach and implementation
: Linux system administration (configuration, installs, automation, and monitoring)
: Experience automating infrastructure and software delivery to production o Familiarity with open source configuration management and deployment tools such as Ansible, Puppet and Ter
