Note: this post was originally published here on Monzo’s blog.

We are currently hiring for a Machine Learning Scientist in my team. This is always an exciting time - I get to meet and talk to a ton of interesting candidates from around the world. 🙌

I often receive questions about the role, team, or company via email, LinkedIn messages (you can find me here), or during calls with candidates. I’ve been collecting them for some time, and so decided to put this blog post together as an FAQ, so that my answers can be transparent and available to everyone, rather than buried in private messages.

I apologise in advance if I email you a link to this page. 😅

⚠️ A word of warning before I begin. This post touches on a lot of different technologies, data types, and problem spaces that we have. If you are reading this post because you are interested in the role, please don’t feel that you need to know everything already in order to qualify for it!

How is Data Science structured at Monzo?

Machine Learning is part of the Data Science discipline at Monzo.

The broader discipline currently has more than 30 people. Everyone in Data Science falls into one of three “areas.” The following is my categorisation, not an official one –I cannot do full justice to the enormous range of work from all my colleagues.

The bird’s eye view of these areas are:

1. Analytics. These folks are our powerhouse of analysis and insight. They spend their time doing things like analysing our customer behaviour, monitoring the entire spectrum of customer operations, launching and evaluating product experiments, and partnering with stakeholders across the business to make sure that they are all enabled to make great data-driven decisions.
2. Core banking. There are many parts of building a bank that are very specific to banking. For example: building lending products (overdrafts & loans) and credit scoring models, analytics of financial crime, payment schemes, and more. This “area” is actually multiple areas that I’ve lumped together for the sake of simplicity, and contains Data Scientists with deep domain expertise.
3. Data Platform & (us!) Machine Learning. This final group is currently two squads; if you imagine Data Science spanning analytics to system building (the Type A / Type B categorisation), then these two squads mostly sit on the “build” end of the spectrum. The Data Platform Squad owns, manages, and improves all of the core systems and tooling that the broader discipline uses; finally, the Machine Learning Squad builds and owns or co-owns multiple systems that use or enable machine learning.

As of October 2020, we currently have three people (myself included) who have the words “Machine Learning” in their job title. While we remain an official squad in the company roster, we iterate between embedding ourselves into other Engineering Squads, or inviting others to embed with us to work on projects. In Q1 this year, we were joined by another backend Engineer, and this quarter, we are working with one of our Data Scientists from the Operations team. I both lead and line manage the squad, and report into our VP of Data.

Here’s our core team - including the empty slot that we’re hiring for:

As an aside, being hired into the “analytics” or “core banking” areas I listed above does not mean you are excluded from using machine learning; for example, we have a dedicated team of Credit Decisioning Scientists who use machine learning for credit scoring, and we have folks who have built forecasting models, with applications in customer contact rate and growth.

Working as a Machine Learning Scientist does mean you will be using ML more frequently, but also means you will be spending time building systems (prior experience of system building is not something that we specifically look for when hiring; this is a skill that we teach and onboard new joiners into).

What sort of problems do you work on?

Just like there is no single product feature that defines Monzo (it’s a bank!), there is no single machine learning problem that captures all of the value we can get from applying this technological approach. There is scope to use machine learning to improve many things and build unique data-driven product features.

To date, we have mostly spent our time tackling problems where we are introducing machine learning for the first time, rather than squeezing more out of existing systems. I foresee this changing as we grow; we balance this out by assessing where we believe we’ll have the most impact. One way that I like to characterise the different flavours of problems that we tackle is based on the types of data that are the input to that system:

1. Text Data. Monzo runs majority of its customer support via in-app chat. This opens up a huge range of opportunities and NLP applications that I have talked about before. For example, we built the system which customers use to search for help articles in the app, and text classifiers are one of the pieces of the system that helps route customer queries to the right agent.
2. Tabular Data. Bank card transactions are one of the many examples of raw tabular data that we have. Recently, we’ve been working with tabular data to supercharge the detection of different financial crime typologies–to protect customers from common scams. As a smaller example, we used transactional data for a one-off hack day that resulted in our Trending Spending blog posts.

These two areas are just the tip of the iceberg. We collect log data about how our customers use the app and how our customer support agents resolve customer queries, all of our rule engines log data about their decisions, and we collect image and video data when, for example, customers complete our identity verification process.

How do you decide what to work on?

At a “macro” level, we set quarterly goals, where each quarter has a specific area of focus. We tend to change gears across different quarters, to balance between exploration and building up our ability to quickly tackle different families of problems vs iterating on specific systems.

So far, this year, we spent time on our NLP and task routing systems (Q1/Q2), deploying fraud-detection models (Q2), and building a feature store and experimenting with a new classification problem (Q3). We are wrapping up the year with a deeper dive into our transactional data, shipping a new classifier system, and starting a new collaboration with other financial crime teams.

At a “micro” level, we use two types of documents to discuss ideas. Everyone in the team writes these:

• Proposals when we are scoping out a new problem and deciding both whether and how we want to work on it. We use these to gather feedback and break our work down into chunks.
• Analyses when we have finished exploring data or running any kind of (online or offline) experiment; we use these to decide what to do next in a specific area.

Our impact as a team is measured in terms of business metrics, not in terms of offline metrics, so we put a huge focus on solving the right problems and on getting to a viable model quickly.

What sort of tech do you use?

The Monzo banking infrastructure lives in AWS, and is primarily Go microservices that use Cassandra as a data store. Our analytics infrastructure is in Google Cloud, where we are heavy BigQuery users and, therefore, mainly write SQL, using data build tool, and orchestrate our queries with Airflow. We surface metrics and insights to the rest of the business via Looker dashboards. You can read about our AWS and GCP customer case studies for more details. I also gave a talk at the Looker 2019 Join Conference about the principles behind our analytics stack.

Machine learning lives at the intersection of our analytics and backend infrastructure stacks: we train models and run batch inference in our analytics infrastructure, where we write Python jobs that we submit to the GCP AI Platform, and we ship Python microservices in AWS for live inference. Everyone in my team has ended up learning at least some Go, which has helped us to be more autonomous and move faster. We benefit from having many brilliant Engineers all around us and we have learned a ton from all of them.

We keep a fairly agnostic view about machine learning libraries: we have shipped a variety of them in the past (scikit-learn, PyTorch, gensim, transformers), and do explore new ones when they pop up. For example, we’ve recently been experimenting with the autogluon library. For deep learning, we’ve mostly used PyTorch; this lead us to being early adopters of the transformers library.

We do have a fairly opininated way of developing our systems; I have written before about how we try to optimise for speed in our work. We try to stay true to the common systems that are used across Monzo, and only sway from them when we have a use case that they simply cannot solve. For example, we monitor our ML models using Grafana and Looker, which are common tools across Engineering and Analytics. We have a feature store which, unsurprisingly, bridges between BigQuery and Cassandra under the hood.

Beyond those examples: we use notebooks/Colab, but limit this to exploratory work (which is super important!). By design, we tend to avoid writing pipelines that start with raw data and end with a trained model, because this leads us towards reinventing the wheel when tackling similar problems. Instead, as our understanding of a data source grows, we treat data processing pipelines as a common, reusable tool. For example, we now have a create-text-dataset pipeline which does all the munging we need to create a new dataset for supervised text classification. We then write separate pipelines to train models.

What are the take-aways?

We’re a small, but very impactful team 😎. We’re continuously learning new things. We do our utmost to avoid “throwing problems over the wall,” and so we build systems, iterate on machine learning models, and collaborate very closely with each other and with many folks across the business.

Hopefully, all of that paints a picture of a team that aims to bring real and valuable machine learning systems to life. We do not spend time trying to advance the state-of-the-art in machine learning or tweak models to absolute perfection.

The goal of this post was to give you an inside view of my team: if you still have questions, feel free to reach out! Did I mention we’re hiring? 😇

Update: we have currently paused the job ad because we have a huge number of applicants. Note that we also have a number of other Data Science roles that are currently open - particularly in Product Analytics and Operations.