Data Pipeline Engineer
Mapzen
(San Francisco, California)Mapzen is an open and accessible mapping platform. We're focused on the core components of geo platforms, including search, rendering, navigation, and data. We take a radical approach to working on these components—we build them all entirely in the open.
As a data pipeline engineer, you will join Mapzen’s mobility team to work on the Open Traffic project. Open Traffic is an open-source project working to create world-wide traffic data for planning and routing applications. Open Traffic provides the platform and tools to share traffic statistics from connected vehicles and mobile services. Mapzen’s role in this project is to scale the service to support multiple traffic data providers with the goal of providing worldwide traffic information. Privacy and security of traffic provider’s data is paramount.
You will work closely with software developers and architects to develop a robust and scalable data ingestion pipeline. You will also work with routing engineers to design and develop interfaces with Mapzen’s open-source routing solution (Valhalla) and with mapping and data engineers to develop methods to visualize traffic data. While this role will focus on Open Traffic, there may be opportunities in the future to also contribute to the Valhalla routing engine and to the Transitland open transit data platform.
We are looking for a talented senior-level developer to work with a diverse team. Excellent communication skills are key to the success of our open-source project, so it’s critical that team members can express themselves in a way that is courteous, clear, and collaborative.
- at least 5 years experience with software development or research
- proficiency with Python and/or C++ 11
- experience with distributed processing for both batch and real-time data-pipeline operations (e.g., Hadoop, Spark, Storm, Flink, Kinesis, or bespoke approaches)
- experience with distributed data-stores (e.g., RedShift, Hadoop formats, S3, Postgres, SQL, or alternatives)
- experience running such services at scale using a collaborative “dev ops” approach (along with tooling such as containers, Chef, OpsWorks)
- passion for open-source technologies, open-data, and collaborative software development
- experience with Amazon Web Services and their managed services
- experience with geographic data
- experience with exploratory data analysis and data visualization
Questions
There are no answered questions, sign up or login to ask a question
Want to see jobs that are matched to you?
DreamHire recommends you jobs that fit your
skills, experiences, career goals, and more.