Overview:
We are looking for a Data
Engineer to join our fast-growing team. At C4ADS, we are pushing the bounds of what is possible with open data and technology for national-security and
threat-finance research. As our Data Engineer, you will be responsible for managing, expanding, and
optimizing our data and data pipeline architecture, as well as optimizing data
flow and collection for cross functional teams. The ideal candidate is an
experienced data pipeline builder and data wrangler who enjoys optimizing data
systems and building them from the ground up. The Data Engineer will support
our analysts in leveraging open data sets across languages, jurisdictions, and
mediums of dissemination. They must be self-directed and comfortable supporting the
data needs of multiple teams, systems, and products. You will quickly gain
practical skills through our in-depth training and will have the opportunity to
expand your professional network—all in addition to a fun work environment with
Nerf guns, generous benefits, and flexible hours. Read more about the role and
requirements below.
Responsibilities
for Data Engineer:
- Create and maintain optimal data pipelines
- Assemble large, complex data sets that meet functional
business requirements
- Identify, design, and implement internal process
improvements like automating manual processes, optimizing data delivery,
re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for optimal extraction,
transformation, and load (ETL) of data from a wide variety of data sources
using an Amazon Web Services (AWS) platform
- Work with stakeholders, including the management and
analytical teams, to assist with data-related technical issues and support
their data infrastructure needs
- Build analytics tools that utilize the data pipeline to
provide actionable insights into customer acquisition, operational efficiency,
and other key business performance metrics
- Keep our data separated and secure across company and
client requirements
- Work with data and analytics experts to strive for
greater functionality in our data systems
Qualifications for
Data Engineer:
- 3+ years of experience in a
Data Engineer role and has attained a degree in Computer Science, Statistics,
Informatics, Information Systems, or another quantitative field. They should
also have experience using the following software/tools:
- Experience with big data
tools: Hadoop, Spark, Kafka, etc.
- Experience with relational
SQL and NoSQL databases, including Postgres and Cassandra
- Experience with AWS cloud
services: EC2, EMR, RDS, Redshift
- Experience with
stream-processing systems: Storm, Spark-Streaming, etc.
- Experience with
object-oriented/object function scripting languages: Python, R, Java, C++,
Scala, Golang, etc.
- Advanced SQL knowledge and
experience working with relational databases, query authoring (SQL), and a working
familiarity with a variety of databases
- Experience building and
optimizing ‘big data’ data pipelines, architectures, and data sets
- Built processes supporting
data transformation, data structures, metadata, dependency and workload
management
- A successful history of
manipulating, processing, and extracting value from large disconnected and
unstructured datasets
- Working knowledge of
message queuing, stream processing, and highly scalable ‘big data’ data stores
- Strong project management
and organizational skills
- Experience supporting and
working with cross-functional teams in a dynamic environment
- Dedication to our values, and
passion for our mission
- Self-motivated, strategic thinker with the initiative and
drive to seize opportunities, but the attention to detail to connect the
analytical dots
How We'll Reward You:
- Competitive pay
- Medical, Dental, and Vision
Insurance
- Voluntary 401 k program
- Generous vacation and flex time
that focuses on the work, not hours
- Professional development programs
that allow you to customize your own courses and build your own
development path
- A
passionate team that values hard work and is serious about building and
maintaining a great culture
