Get Help And Discuss STEM Concepts From Math To Data Science & Financial Literacy
STEM Gender Equality | Join us on ZOOM | Spreading Mathematical Happiness

Help The Education Support Forum through MathsGee serve learners across Africa with a DONATION

1 like 0 dislike
I am struggling to find a multivariable generalization for the KNN regression algorithm.
in Data Science by | 21 views

1 Answer

0 like 0 dislike
Best answer

KNN  regression algorithm is a non-parametric (meaning there is no assumed underlying distribution in the data) that is used for estimating continuous variales.

As an input your have a multivariate feature space which happen to be vectors that each have has a class label.  This implies that this is a supervised learning algorithm.

KNN for example uses a weighted average of the k nearest neighbours, weighted by the inverse of their distance. The most commonly used distance is Euclidean that is given by the general formula 

$$d = \sqrt{(x-x_{0})^2+(y-y_{0})^2}$$

but sometimes the Mahalanobis distance is preferred.


The algorithm generally works is the following way:

  1. Compute the Euclideam or Mahalanobis distance from the query example to the labeled examples
  2. Order the labeled examples by increasing distance (ranking)
  3. Find a hueristically optimal number $k$ of nearest neighbours, base on the Root Mean Square Error (RMSE). This is achieved through cross-validation.
  4. Calculate an inverse distance weighted average with the $k$-nearest multivariate neighbours.
by Diamond (49,546 points)
selected by

Related questions

0 like 0 dislike
0 answers
0 like 0 dislike
1 answer

Welcome to MathsGee Q&A Bank, Africa’s largest personalized STEM and Financial Literacy education network that helps people find answers to problems, connect with others and take action to improve their outcomes.

MathsGee Q&A is the STEM and Financial Literacy knowledge-sharing community where students and experts put their heads together to crack their toughest questions.

Help The Education Support Forum through MathsGee serve learners across Africa with a DONATION

Enter your email address:

11,239 questions
9,393 answers
10,283 users