Principal Data Engineer, ElasticSearch - Secureworks - 100% US Remote

Key Responsibilities

• Design, build, launch, and maintain infrastructure to support large-scale data routing, streaming and historical processing, data warehousing, and container orchestration.
• Coordinate Database Operations (Elasticsearch & Scylla) including deploying clusters to production, performing post-deployment logistics (restart/backup), and monitoring the health of clusters.
• Employ a variety of languages and tools to marry big data systems together and hunt down opportunities to acquire new data from other systems.
• Recommend and implement ways to improve data reliability, efficiency, and quality.
• Collaborate with data scientists and security researchers to productize new methods of extracting information from structured and unstructured data.
• Work effectively on a geographically distributed team to deliver high quality software against aggressive schedules.

Minimum Requirements

• Minimum of 6 years of experience in data engineering, data modeling, ETL development, or data warehousing.
• Minimum of 2 years of experience with Elasticsearch/Lucene and/or Scylla/Cassandra
• Minimum of 2 years of experience with AWS cloud services.
• Minimum of 2 years of software engineering in Scala and scripting languages like Python, Ruby, Perl, Bash, etc.
• Minimum of 3 years of experience with big data platforms (e.g. Hadoop, Spark, Kafka, HBase, etc)

Preferred Experience

• Experience working with containers and container orchestration solutions such as Docker, Kubernetes, Mesos, etc.
• Experience with stream-processing systems, such as NiFi/StreamSets, Kafka, Storm/Heron, Spark, or Flink.