Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related technical discipline with 4-7 years of relevant experience. [Required]
- Experience supporting DISA, DISN, or other DoD network environments.
- Experience developing and deploying software applications that meet DoD security standards, including applicable STIGs.
- Knowledge of network security, encryption, and access controls within classified environments.
- Experience with big data technologies and platforms such as Kafka, Spark, NiFi, Hadoop, Elasticsearch, Logstash, Databricks, and ELK Stack.
- Experience applying big data technologies to text mining, summarization, search, and entity extraction.
Skills
Nice to have
Full job description
Clearance: Top Secret/SCI
Job Type: Full-time
Target Salary Range*: $175,000 - $200,000
- This represents the potential salary range for this position depending on education level, years of experience and/or certifications in addition to other position specific requirements which may impact salary
The Data Engineering Software Developer Mid develops solutions using Confluent Kafka, Elastic Stack, AWS, Red Hat OpenShift/Kubernetes, data warehouses, and data lakes. The position also supports automation, CI/CD, platform optimization, visualization capabilities, and collaboration across software, data, analytics, and network operations teams.
Key ResponsibilitiesData Pipeline Development and Integration
- Design, build, and maintain scalable and reliable data systems and pipelines to ingest, process, and transform large, complex, and disparate data sources for analytics and reporting.
- Unify and integrate data from multiple network operations systems into a consistent and accessible platform.
- Develop and maintain ETL/ELT processes supporting data ingestion, transformation, validation, and cleansing.
- Create and maintain Kafka, Elastic, and Logstash pipelines supporting enterprise data processing requirements.
- Automate data-related processing and monitoring activities.
- Implement data solutions using Confluent Kafka, Elastic Stack, AWS, and Red Hat OpenShift/Kubernetes.
- Develop and manage data warehouses and data lakes supporting analytics, reporting, and operational dashboards.
- Develop data-engineering capabilities using Python, Java, SQL, shell scripting, and related technologies.
- Support Kibana visualizations and dashboards using React, JavaScript, and HTML.
- Apply software engineering and automation practices to improve the maintainability and scalability of data solutions.
- Analyze and optimize data-ingestion throughput, query performance, indexing strategies, and storage efficiency across Elastic and Kafka platforms.
- Optimize infrastructure for performance, scalability, reliability, and cost effectiveness.
- Troubleshoot and resolve issues related to data ingestion, processing, storage, memory utilization, partitioning, and cluster performance.
- Identify and resolve performance bottlenecks affecting data availability and downstream analytics.
- Ensure data quality, accuracy, security, and compliance with applicable DoD and DISA requirements.
- Apply appropriate data security and access-control practices.
- Support CI/CD pipelines for automated build, test, and deployment activities.
- Apply version control and DevOps practices to data engineering and software development workflows.
- Collaborate with data analysts, visualization engineers, software developers, and other stakeholders to deliver integrated and actionable data solutions.
- Document data architecture, integration patterns, configurations, and operational procedures.
- Develop and maintain technical documentation and runbooks supporting data platforms, pipelines, and applications.
- Bachelor’s degree in Computer Science, Engineering, or a related technical discipline with 4-7 years of relevant experience. [Required]
- Hands-on experience designing, building, and maintaining ETL/ELT data pipelines and integrating multiple disparate data sources. [Required]
- Experience with data warehouse and/or data lake technologies such as AWS Redshift, Amazon S3, Hadoop, Snowflake, or similar platforms. [Required]
- Experience with Elastic Stack, including Elasticsearch, Logstash, and Kibana, and/or Kafka for data ingestion and processing. [Required]
- Experience with Linux/UNIX system administration and automation. [Required]
- Experience with CI/CD pipelines, containerized pipelines, version control technologies such as Git and Bitbucket, and DevOps practices. [Required]
- Familiarity with network operations data, including logs, metrics, events, ticketing, CMDB data, and related data models.
- Proficiency with data engineering and software development technologies and languages such as Python, Java, SQL, and shell scripting. [Required]
- Knowledge of data security, access controls, and compliance frameworks, including DoD, DISA, RMF, and STIG requirements. [Required]
- Strong communication and documentation skills supporting collaboration and knowledge sharing. [Required]
- Current Security+ certification or higher/equivalent certification. [Required]
- Active TS/SCI security clearance. [Required]
- Willingness and ability to complete and maintain additional customer suitability screenings throughout employment. [Required]
- Experience supporting DISA, DISN, or other DoD network environments.
- Experience developing and deploying software applications that meet DoD security standards, including applicable STIGs.
- Knowledge of network security, encryption, and access controls within classified environments.
- Experience with big data technologies and platforms such as Kafka, Spark, NiFi, Hadoop, Elasticsearch, Logstash, Databricks, and ELK Stack.
- Experience applying big data technologies to text mining, summarization, search, and entity extraction.
- Experience with cloud-based data platforms, including AWS, Azure, GCP, or AWS GovCloud.
- Experience with cloud-integrated platforms and Infrastructure as Code, including networking and security policies.
- Familiarity with Kubernetes deployment, platform upgrades, and patching.
- Experience with configuration management technologies such as Ansible, Puppet, or Chef.
- Experience with monitoring, alerting, and observability technologies such as Prometheus, Grafana, and Elastic.
- Experience with business intelligence and visualization tools such as Kibana.
- Experience developing and maintaining technical documentation and operational runbooks.
- Strong understanding and practical experience with Agile methodologies, including Scrum and SAFe.
- Experience with Atlassian Jira and Confluence.
- Confluent Developer certification.
- Elastic Certified Engineer certification.
- Experience working remotely with geographically dispersed teams and within matrixed organizations.
- Experience combining software development, integration, and data engineering practices.
- Experience applying knowledge of system architecture, networking, and centralized logging using ELK to support data transformation initiatives.
Opens The Amatriot Group’s listing on LinkedIn in a new tab.