Rishabh Mittal
← All projects

Private experiment

Text Similarity Analyzer

A text-comparison pipeline using weighted frequency vectors rather than surface-level string matching.

Java · TF-IDF · Cosine similarity

  • TF-IDF vectors
  • Cosine similarity

01 / Problem

The constraint

Compare texts while reducing the influence of common words and document length.

02 / Structure

Architecture at a glance

  1. Text + stop-word filtering
  2. Hash-table counts → TF-IDF vectors
  3. Pairwise cosine similarity

03 / Implementation

Engineering decisions

  1. Token counts are accumulated in hash tables and filtered with a stop-word list.
  2. TF-IDF weights emphasize distinguishing terms.
  3. Cosine similarity compares document vectors independently of magnitude.

04 / Evidence

Measurement context

Author-supplied notes describe 500+ stop-word filtering and 85%+ accuracy on a 20-pair authorship test set. That small evaluation does not establish general authorship-classification accuracy.

05 / Trade-offs

Limits worth stating

  • Tokenization and corpus selection strongly affect the result.
  • A 20-pair evaluation is too small for broad accuracy claims.