The Architecture of Modle

A few months ago, while on a walk through Riverside Park in New York, I had an idea for a simple side project: a guessing game. In it, you would be shown a decision boundary and asked to figure out which machine learning algorithm generated it. Modle, as I named it, came out a while ago. You can check it out here, and I’d suggest you do. For one, I think it’s a pretty cool little project if I do say so myself. Secondly, this article is all about its architecture, and it might be a more interesting read if you’ve actually played some Modle.

Quick! What’s this?

In case you didn’t want to try the project out, here’s a quick summary of what you see from the user’s side. When the page comes up you’ll see a page with a little text blurb, a dropdown, and a lot of white space with a loading overlay all over it. After a second, the overlay disappears and you’ll see a graph like the one above. You can then use the dropdown to select an algorithm: for example, Logistic Regression, or non-linear Neural Network. When you find one and submit your answer, you’ll find out if you guessed right, and if not what the correct answer was. Then the process repeats. The question that this article is going to answer is “how does all of that happen?”

The thousand-foot view of Modle is that it’s a TypeScript-React-Flask app with a Postgres database. It uses Pandas and Scikit-Learn to handle the data generation and machine learning models on the backend, Redux to handle state on the frontend, and (indirectly) D3.js to display the chart. Postgres comes in as a way to store the details of each individual game without letting the user know any of them.

Let’s start with the frontend. When the user hits the page, a React hook is triggered in the main React component. This hook has two purposes. The first is to ask the backend for a list of all of the possible algorithms. This gets stored in Redux permanently, and displayed in that dropdown I mentioned before. The second purpose is to ask the backend for the first problem. We’ll get to how that happens on the backend later. For now, suffice to say that when we ask for the problem, we get two things in the response: a problem ID, and a massive JSON object. Both are stored in Redux, but the JSON object is also rendered into the UI, as it how the decision boundary chart is defined!

You see, I’m able to get a D3.js chart from a Python backend with use of a neat little library I found called mpld3. It includes both a Python library for turning matplotlib visualizations into D3, and some JavaScript code to help turn the JSON it generates into the chart you see on the page. The JavaScript code gets brought in through a script tag in my root HTML document, and the actual chart is rendered with a React hook in the chart component which looks like this:

import React, { useEffect } from 'react'

// adapted from https://stackoverflow.com/questions/72632853/rendering-mpld3-json-chart-in-react

const Graph = ({ graphJson }) => {
    const fig_name = (graphJson && graphJson.id);

    useEffect(() => {
        if (typeof mpld3 !== 'undefined' && fig_name) {
            mpld3.remove_figure(fig_name); // eslint-disable-line no-undef
            mpld3.draw_figure(fig_name, graphJson); // eslint-disable-line no-undef
        }
    }, [graphJson, fig_name]);

    return (
        <div id='graph-container'>
            <div id={fig_name || 'empty'}></div>
        </div>
    )
}
export default Graph;

Basically, it listens for changes to the chart definition in Redux and renders the D3 chart when it sees one.

When the user selects an algorithm and submits it, the frontend sends off an HTTP request with their selection and the problem ID. The response contains a Boolean for whether or not they guessed right, and a string to tell them what the correct answer was if they got it wrong. When that appears in the reducer, a piece of text appears above the chart which tells the user whether or not they got it right. Additionally, the dropdown and submit buttons get swapped out for a button that says “Another?” Clicking that clears everything about the previous problem out of Redux: the ID, the chart, the correct answer, all of it; and requests a new problem. That process consists of getting a new ID and chart JSON. Once that is done, the app goes back to the state is was in right after the first problem was loaded, and the whole process can repeat until the user gets frustrated and leaves.

Before I get to the backend, there’s one more feature I want to talk about on the frontend. There is a scoring system where the user can see how many problems they’ve done and how many they got right. Right now it’s all done in Redux, and as such isn’t persistent. It’s just one counter for how many problems the user has done this session, and another for how many they’ve gotten right.

The backend is a bit more complicated. Like I mentioned, it’s all done in Python. It’s all built around Flask, and also uses pandas, scikit-learn, and matplotlib for the data work. What this boils down to is two files: app.py which exposes a bunch of API endpoints, and problem-gen.py which contains an object which creates one of these classification problems — aptly called Problem.

Going through the use flow from the backend, the first thing that happens is that that the / endpoint gets hit and serves up the HTML, CSS, and JavaScript which constitute the frontend. Next, a GET request lands on the /answers endpoint, which calls a class method of Problem to get all of the possible algorithms that we can use, e.g. Logistic Regression, SVM, etc.

Immediately after that, another GET lands at the /problem endpoint. This endpoint instantiates a Problem object. During that instantiation, a couple of decisions have to be made. First is the number of classes. Second is the choice of algorithm. Third are the hyperparameters of that algorithm. Finally, the actual method of data generation (chosen from scikit-learn’s dataset generation methods).

I want to zoom into the first two steps in that process: choosing the number of classes, and choosing an algorithm. The number of classes is always two, three, or four. This step has no bearing on which algorithm is chosen, which has been a common point of contention as I’ve gotten feedback on this project. Specifically, many people have told me that I’m doing something wrong because Logistic Regression and Support Vector Machine are possible answers for problems with more than two classes.

I get where those comments are coming from. If you open a machine learning textbook, you will initially learn about the two-class cases for both of those algorithms. However, several methods exist to handle multiclass problems with those algorithms: multinomial logistic regression, one-versus-one, and one-versus-all to name a few. To keep things simple, I went with the same method for both algorithms in Modle: one-versus-rest.

Back to Modle’s operation, once all of those factors have been decided, we train the selected model type on the generated data (which always has two features), and then generate a matplotlib chart with the (test) data and the decision boundary. This is translated into a D3-chart-defining JSON string which can be sent to the frontend and displayed. With all of that done, a few things have to happen. First, we save the algorithm that was used in the Postgres database as a string along with a unique ID number. Then, the ID and JSON object are returned to the frontend as an HTTP response.

As soon as the chart renders, the page is fully loaded and the game is ready to play. When the user selects an answer from the dropdown and submits it, the frontend sends a POST to the /check endpoint. The body of this request has two things in it: the problem ID, and the user’s guess. When this hits the backend, it queries the database for the given game ID, and compares the user’s guess to the algorithm stored in the DB. If they are the same, the response will look something like this:

{
  "correct": true,
  "answer": "Logistic Regression"
}

If not, that correct field will show false and the answer field will be used to tell the user what the correct answer was. Then, as soon as the user hits “Another?” the process starts again, sending a brand new GET to the /problem endpoint.

All in all, it’s a very simple app that does what it set out to do pretty well. It was a fun excuse to dip my toes into frontend tech again and to experiment with TypeScript a little, and it let me do it in a way that also integrates a lot of what I’ve learned recently about machine learning. Like any good software tinkerer, I also have a couple of plans for things I might add in the future.

At the top of that list is some persistence. It’d be nice for users to be able to see what their success rate looks like from the time they started playing, even if they close the tab. Because of the app’s simplicity and the fact that it’s on a short-lived, free database a account creation/login flow would be way too much. Instead, I think using a cookie would be the better option. That means no persistence if the user clears their cookies. But it also means no making a new account every time Render clears out my free database. All in all, I think the former is better UX.

As an extension for persistence, I’ve had quite a few people who used Modle request the ability to look back through guesses they’ve made to try and improve their skills in the future. That would be a lot of work on the frontend, but might be in the cards. That would require a User ID associated with each saved problem, such that getting the history would just be a query for all of the saved problems. Of course, this would also mean saving the chart JSON as well, which could be a problem because those objects are massive. That would also mean a second page on the frontend.

Beyond that, my plans get a bit more nebulous. I can always add more algorithms. I’ve also played around with the idea of a second game mode where the user has to make guesses for regression problems instead of classification. We’ll see what happens.

If you want to check out Modle, you can it at modle.site. If you want to take a look at the code for it, you can do that at Modle’s GitHub repo. If you have comments or questions about Modle, feel free to drop a comment here, raise an issue on GitHub, or get in touch with me on LinkedIn. Finally, if you want to check out what else I’ve worked on, take a look at my website.