Gaussian Naive Bayes: 2D Decision Regions

What is Gaussian Naive Bayes?

Gaussian Naive Bayes is a probabilistic classifier based on applying Bayes' theorem with strong (naive) independence assumptions between the features. It assumes that each class has an underlying distribution of data points, which in this case are modeled as multivariate Gaussians.

In 2D space, Gaussian Naive Bayes models the probability density function for each class using a two-dimensional Gaussian distribution centered at the mean and with a covariance matrix representing the spread and orientation.

How Does It Work?

The classifier calculates the posterior probability of belonging to each class given the input features. The decision rule is then based on which class has the highest posterior probability, leading to a clear separation between classes in the decision regions.

For example, if we have two classes with different Gaussian distributions, the decision boundary will be determined by where these distributions overlap and where one distribution's density exceeds that of another.

live demo · related simulation● LIVE

Why It Matters

Gaussian Naive Bayes is particularly useful for its simplicity and efficiency. Despite making strong assumptions, it often performs well in practice, especially with high-dimensional data.

It's widely used in applications such as spam filtering, recommendation systems, and medical diagnosis where quick and interpretable models are needed.

Real-World Applications

In natural language processing, Gaussian Naive Bayes can be used to classify emails into spam or not-spam based on the frequency of certain words.

In medical diagnosis, it can help predict the likelihood of a patient having a disease given their symptoms and test results.

Frequently asked questions

What is Bayes' theorem in this context?

Bayes' theorem provides a way to calculate posterior probabilities from prior probabilities and evidence. In Gaussian Naive Bayes, it helps determine the probability of each class given the input features.

Why are the assumptions called 'naive'?

The term 'naive' refers to the assumption that all features are independent of each other, which is often not true in real-world data. However, this simplification can still lead to effective models.

How does the classifier handle different class separations?

The decision regions and boundaries are adjusted based on the spread and overlap of the Gaussian distributions for each class. Closer classes result in more complex boundaries, while well-separated classes have simpler boundaries.

What is the role of the covariance matrix in this model?

The covariance matrix captures how features vary together. In 2D, it determines not only the spread but also the orientation and correlation between dimensions, influencing the shape of the Gaussian distribution.

Try it live

Everything above runs in your browser — open Gaussian Naive Bayes: 2D Decision Regions and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.

▶ Open Gaussian Naive Bayes: 2D Decision Regions simulation