Machine learning in schools #1

Ryan has to prepare for a math test. The test is going to be on a theorem taught in class some days ago. The teacher explained the theorem on the board and gave examples for the same. She then gave her students some problems to try in class. These problems were supposed to train her students for the upcoming test. These problems form what in machine learning is known as the training set. The training set allows machine learning models to find and learn patterns in our data.

Since Ryan is weak at math, he has joined a private tutor for the subject. The tutor explains the theorem to him again. He then gives him a test. The test, he believes, will help prepare Ryan for the actual one. The problems in this test are known as the validation set. The validation set is part of the data from our training set that we’ve held back. We use the validation set to see how well our model has trained. It gives us an estimate of our model’s abilities while we are still training it. We then use the results from our validation set to tune our model into giving better results. Thus, Ryan’s tutor evaluates his test results, works with him on his mistakes and gives him some more practice until he sees an improvement in his performance.

Ryan now feels ready. He has been trained on the training as well as the validation set. The test set (i.e. the actual exam) should be a walk in the park. Unfortunately, it isn’t. There are problems in the test set that Ryan has never seen before. He struggles with them and fails the test. What just happened?

The test set, just like the validation set is part of the data that is held back. However, it is shown to the model only once. The test set is also well curated. It spans all types of problems that can be asked in an exam. Hence, it is very important for the validation set to be an accurate representation of the test set. When it isn’t, our model will fail. And that is exactly what happened to Ryan.

What happens now? Learn more about Ryan and his classmates in my next article.

~happy learning