Singular Value Decomposition

Singular Value Decomposition - term used to describe the process of breaking down a large database to find the document vector (relevance) for various items by comparing them to other items and documents.

Three big steps are

• stemming - taking in account for various forms of a word on a page
• local weighting - increasing the relevance of a given document based on the frequency a term appears in the document
• global weighting - increasing the relevance of terms which appear in a small number of pages as they are more likely to be on topic than words that appear in most all documents.
• normalization - penalizing long copy and rewarding short copy to allow them fair distribution in results. a good way of looking at this is like standardizing things to a scale of 100.

Multi dementional scaling is more efficient than singular value decomposition because it requires exceptionally less computation. When combined with other ranking factors only a rough approximation of relevance is necissary.

for more information see http://javelina.cet.middlebury.edu/lsa/out/tdm.htm

mathematical formulas http://www.acm.org/sigmm/MM98/electronic_proceedings/huang/node4.html