Cloud and DevOps Engineer

Position Description: The Cloud and DevOps Engineer will be responsible initially for managing 2 Cloudera clusters, one in a GovCloud/AWS environment, and on on premises cluster. This includes managing and maintaining the data ingest and ETL scripts currently used to process terabytes of data from an S3 bucket. They will also be responsible for maintaining data storage and compute resources across competing processes (Hive, Impala, Spark). A key component of this job will be the ability to assist in planning for future architecture changes with Hbase, Kafka, and Apex in pursuit of a new data ingest pipeline coupled with data enrichment.


Desired Skills:
  • Strong Unix sys admin skills
    • RedHat / Centos
    • Puppet, Ansible
    • LDAP and certificate management (e.g. SSL)
    • Security hardening
    • Continuous monitoring and maintenance
    • Familiar with government security accreditation processes
  • Strong understanding/administration of Hadoop ecosystem
    • Cloudera, Hortonworks stacks
    • HDFS, Yarn
    • Hive, Impala, HBase, Kafka
    • Spark
  • Experience administering in cloud (AWS/GovCloud) environment
  • Good development skills
    • Java, Python, Bash
  • Experience ingesting and processing large amounts of data
    • Scaling/distributing ingest processes
    • Familiarity with a variety of data formats (json, xml, binary mapping)
    • Data validation and gotchas (e.g., character encodings, de-duplication)
  • Able to get a Secret level clearance (or currently possess one)

PUNCH Cyber Analytics Group

Phone: 703-594-7266