This set of Machine Learning MCQs tests the classic classification algorithms. It includes hand calculations of entropy, Gini impurity and information gain for decision trees, k-nearest neighbour voting, Bayes' rule in Naive Bayes, and SVM margins, support vectors, the C parameter and kernels. These questions are useful for GATE DA, university semester exams, UGC NET Computer Science and machine learning interview rounds where algorithm intuition is examined.
How to practise: read each question and tap the option you think is correct. You will see right away whether you got it, along with a short explanation, and your score is tracked at the bottom of the screen. This set has 18 questions.
Topics covered in this set
- Entropy, Gini impurity and information gain
- Decision tree splitting, gain ratio and pruning
- k-nearest neighbours and choice of k
- Naive Bayes and Laplace smoothing
- SVM margin, support vectors and the C parameter
- Kernel trick and RBF kernel
Machine Learning MCQs with answers and explanations
Q1. A node in a decision tree contains 9 positive and 5 negative examples. What is its entropy (base 2), rounded to three decimals?
H = -sum p_i log2(p_i)
a) 1.000
b) 0.643
c) 0.357
d) 0.940
Q2. A node contains 6 examples of class A and 4 of class B. What is its Gini impurity?
Gini = 1 - sum p_i^2
a) 0.50
b) 0.48
c) 0.97
d) 0.24
Q3. A parent node has 8 positive and 8 negative examples. A split produces a left child with 6 positive and 2 negative, and a right child with 2 positive and 6 negative. What is the information gain (base 2), approximately?
a) 0.189
b) 0.811
c) 0.500
d) 1.000
Q4. What is the maximum possible Gini impurity of a node in a two-class problem?
a) 1.0
b) 0.25
c) 0.5
d) 0.693
Q5. Plain information gain tends to favour attributes with many distinct values, such as a customer ID. Which criterion, used in C4.5, corrects this bias?
a) Mean squared error
b) Chi-square distance
c) Gini index
d) Gain ratio
Q6. In k-nearest neighbours, what is the typical effect of choosing a very small k such as k = 1?
a) High variance and a jagged decision boundary sensitive to noise
b) High bias and an overly smooth boundary
c) The algorithm always predicts the majority class
d) Training time increases sharply
Q7. Training points (with class) are: (1,0) Red, (-1,-1) Red, (0,2) Blue, (2,2) Blue, (3,0) Blue. Using Euclidean distance and majority voting, what does k-NN predict for the query point (0,0) with k = 3 and with k = 5?
a) Blue for k = 3 and red for k = 5
b) Red for k = 3 and blue for k = 5
c) Red for both
d) Blue for both
Q8. In a spam filter, P(spam) = 0.4, P(word 'offer' | spam) = 0.6 and P('offer' | not spam) = 0.1. Given that an email contains 'offer', what is P(spam | 'offer')?
a) 0.24
b) 0.60
c) 0.80
d) 0.86
Q9. What is the 'naive' assumption in the Naive Bayes classifier?
a) Features are conditionally independent of each other given the class
b) All classes have equal prior probability
c) Features follow a uniform distribution
d) The decision boundary is always linear
Q10. In a multinomial Naive Bayes text classifier, a word never appears in the training documents of one class. Why is Laplace (add-one) smoothing applied?
a) To normalise document lengths
b) To remove stop words automatically
c) To make the features independent
d) To prevent a zero likelihood from making the whole class probability zero
Q11. A linear hard-margin SVM has weight vector w = (3, 4). What is the width of the margin between the two supporting hyperplanes?
margin = 2 / ||w||
a) 0.2
b) 0.4
c) 2.5
d) 5.0
Q12. Which training points determine the decision boundary of a trained SVM?
a) All training points equally
b) Only points far from the boundary
c) Only the support vectors lying on or inside the margin
d) Only the class centroids
Q13. What does the kernel trick allow an SVM to do?
a) Compute inner products in a high-dimensional feature space without explicitly mapping the data
b) Reduce the number of training examples
c) Convert regression into classification
d) Avoid solving any optimisation problem
Q14. In a soft-margin SVM, what happens when the regularisation parameter C is made very large?
a) Misclassifications are ignored and the margin becomes wider
b) Margin violations are heavily penalised, giving a narrower margin and risk of overfitting
c) The kernel changes from linear to RBF
d) All points become support vectors
Q15. An SVM with an RBF kernel shows near-perfect training accuracy but poor test accuracy. Which change is most likely to help?
K(x, z) = exp(-gamma * ||x - z||^2)
a) Increase gamma
b) Increase C
c) Switch off feature scaling
d) Decrease gamma
Q16. What is the main purpose of pruning a decision tree?
a) To increase the depth of the tree
b) To speed up computing entropy
c) To reduce overfitting by removing branches that add little predictive value
d) To convert the tree into a linear model
Q17. Why is k-NN called a lazy learner?
a) It defers computation to prediction time, storing the training data instead of building a model
b) It uses only a small fraction of the training data
c) It converges slowly during training
d) It ignores feature values that are missing
Q18. Which of these classifiers is least affected by whether features are standardised or left on their original scales?
a) k-nearest neighbours
b) Decision tree
c) SVM with RBF kernel
d) Logistic regression trained with gradient descent
Quick revision notes
- Entropy H = -sum p_i log2 p_i; binary max = 1 at p = 0.5.
- Gini = 1 - sum p_i^2; binary max = 0.5.
- Information gain = H(parent) - weighted average H(children); ID3 uses IG, C4.5 uses gain ratio, CART uses Gini.
- k-NN: small k = high variance, large k = high bias; scale features before computing distances.
- Naive Bayes: P(C | x) proportional to P(C) * product of P(x_i | C); use Laplace smoothing for zero counts.
- SVM margin = 2/||w||; only support vectors define the boundary.
- Large C = narrow margin, fewer violations; large RBF gamma = more complex boundary.
- Kernel trick: K(x, z) = phi(x) . phi(z) without computing phi explicitly.
Frequently asked questions
What is the difference between entropy and Gini impurity in decision trees?
Both measure how mixed the classes in a node are and are zero for a pure node. Entropy uses logarithms and peaks at 1 for two balanced classes, while Gini is cheaper to compute and peaks at 0.5; in practice they usually produce similar trees.
How do I choose the value of k in k-NN?
Use cross-validation to try several values and pick the one with the best validation score. Odd values help avoid ties in binary classification, and features should be scaled first.
Why does Naive Bayes work well even though its independence assumption is often false?
Classification only needs the correct class to have the highest score, not accurate probabilities. Naive Bayes often ranks classes correctly despite dependent features, and its few parameters make it robust with small datasets such as text.
Comments
Post a Comment