Senior Data Engineer

Shopkick is the indispensable app that makes everyday shopping more personal and rewarding. Millions of people use shopkick every day to discover new products, plan shopping trips and earn rewards in stores everywhere. To create the best shopping experience for our users, we partner with major retailers and brands, including American Eagle Outfitters, Best Buy, Hallmark, Kellogg's, Macy's, Marshalls, P&G, Starbucks, TJ Maxx, and many more.

We collect tons of unique data on in store shopping and consumer behavior, and we need your help to build a data and insights platform to help us optimize it’s use internally as well as a service for our brand and retail partners.

• Design and implement a system for capturing, validating and documenting all data in shopkick
• Build a Big Data platform from the ground up using the latest technologies
• Review designs and architecture for detecting suspicious user activity in real time
• Investigate the root causes of performance issues in a real-time Spark pipeline
• Mentor junior engineers on defensive coding and designing decoupled, reusable modules
• Work on a highly collaborative team of 3-5 engineers

• Excellent communication skills in English, both oral and written
• 7+ years of experience designing, building and implementing server software
• Strong coding ability in an object oriented language (Java, C#, C++, Python etc.)
• Experience with Big Data technologies is a definite plus (Hadoop, Spark, Kafka, Druid etc.)
• Experience with NoSQL datastores is also a plus (Cassandra, HBase, MongoDB etc.)
• Enthusiastic about collaborative problem solving
• Eagerness to solve challenging problems
• Experience designing for horizontal scalability and distributed computation
• Ability to examine and evaluate server architectures
• Solid understanding of sources of performance bottlenecks in distributed architectures
• Deep understanding of the uses and characteristics of common infrastructure technologies (such as Zookeeper, Memcached, Redis, Hadoop and MySQL)