🚨 Quiz Instructions:
Attempt all questions first.
✔️ Click SUBMIT at the end to unlock VIEW ANSWER buttons.
Quiz Mode:

Advanced Word2Vec MCQs with Answers (Skip-gram, SGNS & Softmax)

This page provides 20 advanced multiple-choice questions (MCQs) on Word2Vec covering Skip-gram, CBOW, Negative Sampling (SGNS), Full Softmax, subsampling, PMI matrix factorization, cosine similarity, and embedding theory. These questions are designed for postgraduate students, research scholars, competitive exams, and machine learning interviews.

Topics Covered in These Word2Vec MCQs

  • Skip-gram vs CBOW differences
  • Full Softmax computational complexity O(|V|)
  • Negative Sampling and the 3/4 distribution smoothing
  • Subsampling of frequent words
  • Shifted PMI matrix factorization interpretation of SGNS
  • Cosine similarity and embedding geometry
  • Static embedding limitations (polysemy problem)
  • Effect of window size and dimensionality

Who Should Practice These Questions?

These advanced Word2Vec MCQs are suitable for learners preparing for NLP exams, machine learning viva, university theory exams, research interviews, and technical placements. The explanations emphasize conceptual understanding rather than memorization.



1.
In Skip-gram with full softmax, what is the primary computational bottleneck when vocabulary size is extremely large (e.g., 1 million words)?






Correct Answer: C

Explanation:

The denominator of the softmax requires summing over all vocabulary words. If vocabulary size is 1 million, 1 million dot products must be computed for every update, making training computationally expensive.

Full softmax requires computing the denominator over the entire vocabulary: Σ exp(vwT vwc) Time complexity = O(|V|) per training example.
2.
In negative sampling, if a negative word vector is orthogonal to the center word vector, what happens to its gradient update?






Correct Answer: C

Explanation:

If vectors are orthogonal, their dot product is zero. Since sigmoid(0) = 0.5, the gradient is small but not zero. The model still updates the vectors to push negative samples away.

3.
Given the subsampling probability formula P(w) = 1 - √(t / f(w)), what happens when word frequency f(w) is much larger than t?






Correct Answer: B

Explanation:

When frequency is very high, t/f(w) becomes very small, making the discard probability approach 1. Thus very frequent words like "the" are removed most of the time.

4.
Why does Word2Vec learn two embedding matrices (W and W') but typically use only W after training?






Correct Answer: C

Explanation:

The input matrix W represents center-word embeddings and captures semantic structure. The output matrix W' represents context embeddings and is usually discarded after training.

5.
If two words have nearly identical context distributions in a corpus, their Word2Vec embeddings will most likely:






Correct Answer: C

Explanation:

According to the distributional hypothesis, words appearing in similar contexts obtain similar embeddings, resulting in high cosine similarity.

6.
Which scenario particularly favors Skip-gram over CBOW?






Correct Answer: C

Explanation:

Skip-gram generates more training signals per word and performs better for rare words, while CBOW is generally faster and smoother for frequent words.

7.
Why is the negative sampling distribution raised to the power of 3/4?






Correct Answer: C

Explanation:

Raising frequencies to the power 3/4 reduces dominance of very frequent words and increases medium-frequency sampling, improving embedding quality.

8.
Why does the analogy king - man + woman ≈ queen work in Word2Vec?






Correct Answer: C

Explanation:

Word2Vec embeddings capture linear semantic relationships, allowing vector arithmetic to represent analogies like gender direction in embedding space.

9.
If neither negative sampling nor hierarchical softmax is used, training Word2Vec with full softmax becomes:






Correct Answer: C

Explanation:

Full softmax requires computation across entire vocabulary for each update, making complexity proportional to vocabulary size and thus very slow.

10.
Which limitation is fundamentally unavoidable in static Word2Vec embeddings trained without contextualization?






Correct Answer: B

Explanation:

Classic Word2Vec learns a single static vector representation for each word type, regardless of context. Therefore, polysemous words like “bank” (river bank vs financial bank) receive only one embedding and cannot represent different meanings based on context.

11.
Skip-gram with Negative Sampling (SGNS) has been theoretically shown to approximate factorization of which matrix?






Correct Answer: C

Explanation:

Research shows that Skip-gram with Negative Sampling implicitly factorizes a shifted PMI matrix. This explains why semantic similarity emerges geometrically in Word2Vec embeddings.

12.
Increasing the context window size in Word2Vec primarily encourages the model to capture more:






Correct Answer: C

Explanation:

Small window sizes focus on syntactic relationships, while larger window sizes capture broader topical and semantic relationships across sentences.

13.
In trained Word2Vec embeddings, very frequent words often tend to have:






Correct Answer: B

Explanation:

Frequent words receive many gradient updates during training, which often leads to larger embedding magnitudes compared to rare words.

14.
If the number of negative samples (k) is significantly increased in Skip-gram with Negative Sampling, what is the most likely effect?






Correct Answer: B

Explanation:

Increasing k improves approximation to full softmax and may enhance embedding quality, but training time increases linearly with k.

15.
Is cosine similarity between two Word2Vec embeddings symmetric?






Correct Answer: A

Explanation:

Cosine similarity is mathematically symmetric: cos(a, b) equals cos(b, a). This property is independent of the training architecture.

16.
Why does Word2Vec use the dot product between word vectors during training?






Correct Answer: B

Explanation:

The dot product measures alignment between vectors. Higher dot product increases predicted probability that two words co-occur.

17.
Very rare words in Word2Vec training tend to have:






Correct Answer: B

Explanation:

Rare words receive very few updates, so their embeddings are often poorly trained and unstable compared to frequent words.

18.
If subsampling of frequent words is completely removed, what is the most likely outcome?






Correct Answer: C

Explanation:

Without subsampling, high-frequency words appear in nearly every context and dominate gradient updates, harming semantic representation learning.

19.
If embedding dimensionality increases significantly (e.g., from 100 to 1000), what is the most likely effect?






Correct Answer: C

Explanation:

Higher dimensional embeddings increase representational capacity but also computational cost and risk of overfitting, especially with limited data.

20.
In Word2Vec, a word’s embedding primarily reflects:






Correct Answer: B

Explanation:

Word2Vec learns embeddings based on global co-occurrence patterns throughout the corpus, not on individual sentence position.