This short post will discuss the first principle constraints in software-based Machine Learning(ML) approaches. ML algorithms are broadly classified into supervised ML, unsupervised ML, and reinforcement ML. Let’s first talk about the first two categories, which mainly deal with “extracting meaningful science from a set of abstract data points”. There are two ways to increase the accuracy of the results; a) Increase the number of data points, or b) Build a better neural network, where the financial resources limit the former, and the latter is limited by the (wo)man power you have in your team. Given these two constraints, our job is “how to extract more science by using fewer data points than the existing framework”. Let’s apply limits on these two constraints. Financial constraints are about the money you can spend on this project, which can be all money available in the world (around $100 Trillion), and (wo)man power is limited by the number of people available in this world (around 7.7 Billion), in a hypothetical world where you can use all money and every living person is expert ML-engineer/mathematician.
Yes, it is almost impossible to make every living person (from infant to old age) literate in those skills but let’s take the most optimistic case scenarios by assuming the most positive limits on the given two constraints. This is the universe from which you have to pick up the combination of the values of these constraints for any given ML problem in the world. If you have limited financial constraints that can buy you only 10 data points at maximum, then your job is to find a better ML/mathematician who can help you extract a little more science from the existing 10 data points. If you cannot find a better ML/mathematician, then find ways to get more money to generate 11th, 12th … data points until you get the desired results. As far as the third ML type (reinforcement ML) is concerned, it is all about second constrained, i.e., the number of people in the world, and less about financial constraints. However, some hybrid techniques can also use a set of data points, but that will be a secondary part of that framework. For this approach, we are limited by the brain power of the most expert person in the world in the given reinforcement ML domain, and/because we can not generate a new human being for the purpose of building better ML code.
I hope you enjoyed reading this, and thank you for stopping by. Have a good day/night.