Please visit, subscribe and share 10 Minutes Lectures in Computer Science

Sunday, 13 June 2021

Find the average precision of a retrieval system - Information retrieval exercise

Find average precision of a retrieval system, information retrieval system evaluation, how to find the average precision of a search engine, information retrieval systems solved exercises

Question:

An information retrieval system give us the following result (10 documents) where the green colored are relevant ones and the red colored are non-relevant. The numbers indicate the rank of the document. Find the average precision of the given system.

 1

 2

 3

 4

 5

 6

 7

 8

 9

 10

 

Solution:

Precision = Fraction of retrieved documents that are relevant.

Precision = No. of relevant items retrieved/No. of retrieved items

Precision@k = Fraction of retrieved documents that are relevant in the top k documents.

Precision@k = No. of relevant docs in the top 5 results / k

Precision@5 = 3/5 = 0.6

Precision values @k is calculated and shown in the table below for all documents.

Rank

1

 2

 3

 4

 5

 6

 7

 8

 9

 10

Precision

1.0

0.5

0.33

0.5

0.6

0.5

0.43

0.5

0.44

0.5

 

We need to compute Precision@k for each of the k values in the ranked document list. Then we can calculate the average precision as follows;

Average precision = average of precision@k


Here, rel(k) is 1 if the document is relevant, otherwise 0. Hence, we can add the relevant documents precision@k and calculate the average precision as follows;

Average precision = (1.0 + 0.5 + 0.6 + 0.5 + 0.5)/5 = 0.62

Average precision for the retrieved documents = 0.62

 

************

 

How to calculate the average precision of a search result?

What is average precision?

What is precision@k?

How to calculate precision@k? 

How to evaluate an information retrieval system?

Featured Content

Multiple choice questions in Natural Language Processing Home

MCQ in Natural Language Processing, Quiz questions with answers in NLP, Top interview questions in NLP with answers Multiple Choice Que...

All time most popular contents