apache-spark-sql

Data Engineers at Flow Traders are responsible for maximizing the value of our data. You will develop and maintain our data infrastructure and data intensive applications. Working closely with Quantitative Traders and Fundamental Analysts, you will make data available, performant and accessible, choosing the right technology for the task.

A day in the life of a Data Engineer

A typical day for a Data Engineer is a blend of short cycle maintenance and ongoing development work. It starts with checking the data monitoring systems for alerts and deciding what needs immediate attention, or can picked up in the next development cycle. From there you will attend a stand-up meeting with your team. This is an opportunity to catch up on progress of development, raise problems and discuss plans for the rest of the day. After stand-up, you go back to your desk to program, review code on pull requests submitted by colleagues, and discuss functionality with traders. Sometimes you get a request to debug a workload with a Data Analyst and help with performance tuning. At lunch time, you join your team for lunch in the canteen and enjoy a company-provided lunch.

During the day, if there are no pressing issues or you feel that you need a break, you can unwind at the pool table or table tennis, or do a workout at the gym. After the day is done you can join your colleagues to celebrate a successful release or just socialize at our bar.

What will you work on

• Develop and maintain data infrastructure and data intensive applications.
• Create insights and make data available, accessible and performant for business.
• Work closely with Business, helping Quantitative Traders and Fundamental Analysts.
• Support operations with deployment and configuration of data infrastructure.

What you need to succeed:

• Strong background in Computer Science or Software Engineering;
• SQL, Spark, Python, Java;
• Experience with Hadoop (Cloudera is a plus);
• Experience with streaming data processing;
• Experience working closely with Data Scientists and Data Analysts;
• Linux expertise;
• Maven, Git;
• Atlassian stack:
  • Bitbucket for code review. We do all work in pull requests, which are reviewed by other developers;
  • Bamboo for continuous integration;
  • JIRA, Confluence.