Your instinct? Stop reading.<\/p>\n Don\u2019t.<\/p>\n That\u2019s exactly what I told myself when I started reading Probabilistic Machine Learning \u2013 An Introduction<\/em> by Kevin P. Murphy.<\/p>\n And it was absolutely worth it.<\/p>\n It changed how I think about machine learning.<\/p>\n Sure, some formulas might look complicated at first glance.<\/p>\n But let\u2019s look at the formula to see that what it describes is simple.<\/p>\n When a machine learning model makes a prediction (for example, a classification), what is it really doing?<\/p>\n It\u2019s distributing probabilities across all possible outcomes \/ classes.<\/p>\n And those probabilities must always add up to 100 % \u2014 or 1.<\/p>\n Let\u2019s take a look at an example: Imagine we show the model an image of an animal and ask: \u201cWhat animal is this?\u201d<\/p>\n The model might respond:<\/p>\n Add them up? This means the model believes it\u2019s most likely a cat \u2014 but it\u2019s also leaving a small chance for dog or fox.<\/p>\n This simple formula reminds us that machine learning models can not only give us an answer (=It\u2019s a cat!), but also reveal how confident they are in their prediction.<\/p>\n And we can use this uncertainty to make better decisions.<\/p>\n Table of Contents<\/strong> Tom Mitchell<\/a>, an American computer scientist, defines machine learning as follows:<\/p>\n > A computer program is said to learn from experience E with respect to some class of tasks T, and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.<\/em><\/p>\n<\/blockquote>\n Let\u2019s break this down:<\/strong><\/p>\n In classical machine learning, a value is often simply predicted:<\/p>\n > \u201cThe house price is 317k CHF.\u201d<\/em><\/p>\n<\/blockquote>\n The probabilistic view, however, focuses on learning probability distributions.<\/p>\n Instead of generating fixed predictions, we are interested in how likely which different outcomes (in this example prices) are.<\/p>\n Everything that is uncertain \u2014 outputs, parameters, predictions \u2014 is treated as a random variable.\u00a0<\/p>\n In the case of a house price, there might still be negotiation opportunities or risks that are mitigated through mechanisms like insurance. <\/p>\n But let\u2019s now look at an example where it is really crucial for good decisions that the uncertainty is explicitly modelled:<\/p>\n Imagine an energy supplier who needs to decide today how much electricity to buy.<\/p>\n The uncertainty lies in the fact that energy demand depends on many factors: temperature, weather, the economic situation, industrial production, self-production through photovoltaic systems and so on. All of which are uncertain variables.<\/p>\n If we rely only on a single best estimate, we risk either:<\/p>\n With a probability calculation, on the other hand, we can plan that there is a 95% probability that demand will remain below 850 MWh, for example. And this, in turn, allows us to calculate the safety buffer correctly \u2014 not based on a single point prediction, but on the entire range of possible outcomes.<\/p>\n If we have to make an optimal decision under uncertainty, this is only possible if we explicitly model the uncertainty.<\/p>\n To understand why the probabilistic view is so powerful, we need to look at how machines actually learn (Supervised Learning, Unsupervised Learning or Reinforcement Learning<\/a>). So, this is next.<\/p>\n In simple terms, Supervised Learning<\/a> means that we have examples \u2014 and for each example, we know what it means.<\/p>\n For instance:<\/p>\n > If you see this picture (input x), then the flower is called Setosa (output y).<\/em><\/p>\n<\/blockquote>\n The aim is to find a rule that makes good predictions for new, unseen inputs. Typical examples of supervised learning tasks are classification or regression.<\/p>\n The probabilistic view reminds us that there is no absolute certainty in the real world.<\/p>\n In the real world, nothing is perfectly predictable.<\/strong><\/p>\n Therefore, instead of working with a single \u2018fixed answer\u2019, probabilistic models work with probabilities:<\/p>\n > \u201cThe model is 95% certain to be a Setosa.\u201d<\/em><\/p>\n<\/blockquote>\n This way, the model does not just guess, but also expresses how confident it is.<\/p>\n In machine learning, there is no single \u201cbest method\u201d that works for every problem.<\/p>\n The No Free Lunch Theorem tells us:<\/p>\n > If an algorithm performs particularly well on a certain type of task, it will perform worse on other types of tasks.<\/em><\/p>\n<\/blockquote>\n Why is that?<\/p>\n Because every algorithm makes assumptions about the world. These assumptions help in some situations \u2014 and hurt in others.<\/p>\n Or as George Box<\/a> famously said:<\/p>\n > All models are wrong, but some models are useful.<\/em><\/p>\n<\/blockquote>\n J. Pearl<\/a> describes supervised learning as \u2018glorified curve fitting\u2019.<\/p>\n What he meant is that supervised learning is, at its core, about connecting known points (x, y) as smoothly as possible \u2014 like drawing a clever curve through data.<\/p>\n In contrast, Unsupervised Learning<\/a> is about making sense of the data without any labels \u2014 trying to understand the underlying structure without a predetermined target.<\/p>\n Unsupervised learning means that the model receives data \u2014 but no explanations or labels.<\/p>\n For example:<\/p>\n When the model sees an image (input x), it is not told whether it is a Setosa, Versicolor or Virgnica.<\/p>\n The model has to find out for itself whether there are groups, patterns or structures in the data. A typical example of unsupervised learning is clustering.<\/p>\n The aim is therefore not to learn a fixed rule, but to better understand the hidden structure of the world.<\/p>\n We are not trying to say:<\/p>\n > \u201cThis picture is definitely a Setosa.\u201d<\/em><\/p>\n<\/blockquote>\n but rather:<\/p>\n > \u201cWhat structures or patterns are probably hidden in the data?\u201d<\/em><\/p>\n<\/blockquote>\n Probabilistic thinking allows us to capture uncertainty and diversity in possible explanations. Instead of forcing a hard classification, we model possibilities.<\/p>\n Sometimes there are no labels for the data \u2014 or they would be very expensive or difficult to collect (e.g. medical diagnoses).<\/p>\n Sometimes the categories are not clearly defined (for example, when exactly an action starts and when it is finished).<\/p>\n Or sometimes the task of the model is to discover patterns that we do not yet recognise ourselves.<\/p>\n Let\u2019s look at an example:<\/p>\n Imagine we have a collection of animal images \u2014 but we don\u2019t tell the model which animal is shown.<\/p>\n The task is: The model should group similar animals together. Purely based on patterns it can detect.<\/p>\n Reinforcement learning means that a system learns from experience by acting and receiving feedback about whether its actions were good or bad.<\/p>\n In other words:<\/p>\n In simple words, it\u2019s actually similar to how we train a dog.<\/p>\n Let\u2019s take a look at an example:<\/p>\n A robot is trying to learn how to walk. It tries out various movements. If the roboter falls over, it learns, that action was bad. If the robot manages a few steps, it gets a positive reward.<\/p>\n Behind the scenes, the robot builds a strategy or a rule called a policy \u03c0(x):<\/p>\n > \u201cIn situation x, choose action a.\u201d<\/em><\/p>\n<\/blockquote>\n Initially, these rules are purely random or very bad. The robot is in the exploration phase to find out what works and what does not. Through each experience (e.g. falling or walking), the robot receives feedback (rewards) such as +1 point for standing upright, -10 points for falling over.\u00a0<\/p>\n Over time, the robot adjusts its policy to prefer actions that lead to higher cumulative rewards. It changes its rule \u03c0(x) to make more out of good experiences and avoid bad experiences.<\/p>\n What is the robot\u2019s goal?<\/p>\n The robot wants to find actions that bring the highest reward over time (e.g. staying upright, moving forwards).<\/p>\n Mathematically, the robot tries to maximise its expected future reward value.<\/p>\n The system (in this example the robot) often does not know exactly which of its many actions has led to the reward. This means that it has to learn under uncertainty which strategies (policies) are good.<\/p>\n In reinforcement learning, we are therefore trying to learn a policy:<\/p>\n \u03c0(x)<\/em><\/p>\n This policy defines, which action should the system perform in which situation to maximize rewards over time.<\/p>\n Reinforcement learning mirrors the way humans and animals learn.<\/p>\n It is perfect for tasks where there are no clear examples, but where improvement comes through experience.<\/p>\n The film AlphaGo and the breakthrough are based on reinforcement learning.<\/p>\n When we talk about a model in machine learning, we mean more than just a function in the probabilistic view.<\/p>\n A model is a distributional assumption about the world.<\/p>\n Let\u2019s take a look at the classical view:<\/strong><\/p>\n A model is a function f(x)=y that translates an input into an output.<\/p>\n Let\u2019s now take a look at the probabilistic view:<\/strong><\/p>\n A model explicitly describes uncertainty \u2014 for example in f(x)=p(y\u2223x).<\/p>\n It is not about providing one \u201cbest answer\u201d, but about modelling how likely different answers are.<\/p>\n On my <\/em>Substack<\/em><\/a>, I regularly write summaries about the published articles in the fields of Tech, Python, Data Science, Machine Learning and AI. If you\u2019re interested, take a look or subscribe.<\/em><\/p>\n In the real world, almost nothing is truly certain.<\/p>\n Uncertainty, incomplete information and randomness characterise every decision we make.<\/p>\n Probabilistic machine learning helps us to deal with exactly that.<\/p>\n Instead of just trying to be \u201cmore accurate\u201d, a probabilistic approach becomes:<\/p>\n These advantages make probabilistic models more transparent, trustworthy and interpretable systems (instead of black box algorithms).<\/p>\n The post Beyond Glorified Curve Fitting: Exploring the Probabilistic Foundations of Machine Learning<\/a> appeared first on Towards Data Science<\/a>.<\/p>\n<\/div>\n![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/Probabilistic-View-on-Machine-Learning.png?ssl=1\")
<\/figure>\n\n
Exactly 100%.<\/p>\n![\"\"](\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/Introducing-the-Probabilistic-View-of-Machine-Learning.png?ssl=1\")
1\u00a0What does machine learning from a probabilistic view mean?<\/a>
2 So, what is supervised learning?<\/a>
3\u00a0So, what is unsupervised learning?<\/a>
4\u00a0So, and what is reinforcement learning?<\/a>
5 From a mathematical perspective: What are we actually learning?<\/a>
Final Thought \u2014 What\u2019s the point of understanding the probabilistic view anyway?<\/a>
Where Can You Continue Learning?<\/a><\/em><\/p>\nWhat does machine learning from a probabilistic view mean?<\/h2>\n
\n
\n
Where does the probabilistic view come in?<\/h3>\n
\n
And where does probability help us now?<\/h3>\n
\n
Why is this important?<\/h3>\n
\n
If our model understands uncertainty, we can better weigh risks. For example, in credit scoring, a customer labelled as an \u2018unsafe customer\u2019 could trigger additional verification steps.<\/li>\n
For us humans, probabilities are more tangible than rigid point predictions. Probabilistic outputs help stakeholders understand not only what a model predicts, but also how confident it is in its predictions.<\/li>\n<\/ul>\n
\nMany machine learning models are deterministic \u2014 but the world is uncertain:<\/h2>\n
So, what is supervised learning?<\/h3>\n
\n
What does the probabilistic view add?<\/h3>\n
\n
\n
And what about the No Free Lunch Theorem?<\/h3>\n
\n
\n
Supervised learning as \u201cglorified curve fitting\u201d<\/h3>\n
So, what is unsupervised learning?<\/h3>\n
How does the probabilistic view help us here?<\/h3>\n
\n
\n
Why do we need unsupervised learning?<\/h3>\n
So, and what is reinforcement learning?<\/h2>\n
\n
\n
How does the probabilistic view help us?<\/h3>\n
Why is reinforcement learning so fascinating?<\/h3>\n
From a mathematical perspective: What are we actually learning?<\/h2>\n
\n
The probability of a label y, given an input x.
We ask: \u201cWhat is the correct answer to this input?\u201d
Formula: f(x)=p(y\u2223x)<\/li>\n
The probability of the data itself, without explicit target values.
We ask, \u2018How probable is this data itself?\u2019.
Formula: f(x)=p(x)<\/li>\n
A rule that suggests an action a for every possible state x, which brings as much reward as possible in the long term.
We ask: \u2018Which action should be carried out now so that the system receives the best reward in the long term?
Formula: a=\u03c0(x)<\/li>\n<\/ul>\n
\n
\nFinal Thought \u2014 What\u2019s the point of understanding the probabilistic view, anyway?<\/h2>\n
\n
For example, in a medical diagnostic system, we want a model that indicates its uncertainty (\u2018it is 60 % certain that it is cancer\u2019) instead of making a fixed diagnosis. In this way, additional tests can be carried out if there is a high degree of uncertainty.<\/li>\n
For example, a model that models weather data probabilistically can react more easily to new climate conditions because it learns about uncertainties.<\/li>\n
For example, in a credit scoring system, we can show stakeholders that the model is 90% certain that a customer is creditworthy. The remaining 10% uncertainty is explicitly communicated \u2014 this helps with transparent decisions and risk assessments.<\/li>\n<\/ol>\nWhere Can You Continue Learning?<\/h2>\n
\n