Software Engineer of Data Acquisition

Quid builds software that augments human intelligence. We're defining a new market and pioneering technology that helps strategic thinkers make decisions in a fraction of the time. Quid algorithms reveal patterns in large, unstructured datasets and then generate beautiful, actionable visualizations.

As a member of the Data Acquisition and Frameworks team, you will help us build out and maintain our real-time data pipeline and supporting infrastructure. You will write software that acts on streams of data and tackle technological and operational challenges to make sure our pipeline is scalable, robust and real-time. You will be leveraging open source data processing technologies and frameworks such as Spark, Kafka and elastic. Additionally, you will research new frameworks, technologies, tools and standards to increase our efficiency, reliability, and ability to innovate as individuals, as a team, and as a company.

• Design and implement high-performance, scalable, and reliable streaming and batch data acquisition pipelines
• Work closely with product and other engineering teams to identify requirements and iterate on solutions
• Own Quid’s data acquisition and processing pipeline features, from design and proof-of-concept through development into robust, scalable, reliable solutions deployed in production
• Own Quid’s frameworks, practices and standards for distributed systems, streaming pipelines, batching jobs and other data processing needs
• Research and advocate for new technologies to add to Quid’s technology stacks
• Evangelize and facilitate best practices, code reviews, architecture and design reviews, tested code and a great engineering culture

• At least 3 years of professional experience in building large software systems
• Expert level knowledge in Python, we also use Java, Scala, C++ as needed (any combination of the above is ok)
• Solid computer science foundation; good understanding of data-structures, memory and runtime complexity
• Experience with building large-scale distributed systems
• BS in Computer Science or equivalent
• Experience with Spark, Kafka or elastic is a plus