NLP KNN LSA & Chatbot — Ninditya Salma Nur Aini

01 · Problem / Business Need

Students kept asking the same questions. The department had no scalable answer.

The Informatics Department delivered information through circulars and manual replies — limited by office hours, answering the same questions repeatedly. A classification chatbot was the right solution.

Student Question

raw Indonesian text

↓

NLP Classification

LSA + KNN pipeline

↓

Topic Prediction

intent label

↓

Automated Reply

template answer from DB

Challenge

Repetitive student questions answered manually
Limited to office hours — no 24/7 self-service
Standard KNN fails on high-dimensional sparse vectors
Indonesian morphology breaks English NLP tools

Solution

NLP-powered FAQ chatbot for the department
Indonesian preprocessing (Nazief-Andriani stemmer)
Semantic classification using LSA + KNN

Impact

Accuracy ↑ 1.1 pp (92.1% → 93.2%)
Recall ↑ 3.0 pp
Inference time ↓ 35% (27.49s → 17.97s)

02 · Architecture

From raw text to semantic classification.

The pipeline solves two problems simultaneously: Indonesian morphology fragmentation, and the curse of dimensionality in KNN. LSA via SVD handles both.

—

Pertanyaan Mahasiswa

raw Indonesian text input

01

Normalisasi

case fold · hapus tanda baca & angka

clean text

02

Tokenisasi

split per kata (whitespace tokeniser)

token[ ]

03

Stopword Removal

hapus kata fungsi bahasa Indonesia

filter

04

Stemming — Sastrawi

Nazief-Andriani · kembalikan ke kata dasar

kata dasar

05

TF-IDF Vectorisation

sparse, high-dimensional term weights

sparse vec

06

SVD → LSA Projection

k=661 · dimensionality ↓ · semantic space

dense vec

07

KNN Classification

Manhattan distance · k-nearest neighbors

intent label

—

Jawaban Chatbot

template answer from database

# Full classification pipeline
raw_question
  → case_fold() + remove_punct() + remove_num()   # normalise
  → tokenise()                                      # split to words
  → remove_stopwords(id_stoplist)                   # filter noise
  → stem(nazief_andriani)                           # Indonesian stemmer
  → tfidf_vectorise()                               # sparse, high-dim
  → svd_reduce(k=661)                               # dimensionality ↓
  → lsa_project()                                   # semantic space
  → knn_classify(manhattan_distance)                # → intent label

Why LSA over raw TF-IDF?

KNN on raw TF-IDF has two structural problems: sparse high-dimensional vectors make distance computation slow and noisy, and lexical matching fails when students phrase the same question differently.

LSA fixes both: SVD reduces dimensionality (cutting noise and compute time), and the latent semantic space groups words that appear in similar contexts — so "daftar ulang" and "her" map to the same intent even if they never overlap in training data.

Indonesian NLP specifics

Bahasa Indonesia uses prefix, suffix, and circumfix morphology — the root "ajar" can appear as "mengajar", "pelajaran", "diajarkan", "pembelajaran". Treating each as a separate token fractures the vocabulary.

The Nazief-Andriani stemmer strips affixes according to Indonesian morphological rules, collapsing all surface forms back to their root. Without this, TF-IDF produces severely fragmented features.

03 · Experiment Design

KNN vs LSA + KNN. Why this experiment?

Every design choice has a reason. Here's the experimental setup and the logic behind each decision.

Dataset

1,410 Labelled Questions

collected via structured form · 20 intent categories

↓

Validation Method

10-Fold Cross Validation

90% train / 10% test per fold

↓

Baseline

KNN

TF-IDF sparse vectors
Manhattan distance
No dimensionality reduction

92.1% accuracy · 27.49s

Experiment

LSA + KNN

TF-IDF → SVD → LSA dense vectors
Manhattan distance (same classifier)
k=661 latent dimensions

93.2% accuracy · 17.97s ↑

Why KNN?

Simple, interpretable, requires no training iteration — appropriate for a labelled corpus where similarity in vector space maps directly to category membership. A strong baseline before adding LSA.

Why TF-IDF?

TF-IDF assigns higher weight to terms that are distinctive to a question but rare across the corpus — the right prior for intent classification where rare keywords carry the most signal.

Why LSA?

KNN on raw TF-IDF suffers from the curse of dimensionality. LSA via SVD compresses features into a dense semantic space, reducing noise and enabling synonym grouping — so the classifier doesn't need an exact vocabulary match.

Why 10-fold CV?

With 1,410 samples, 10-fold CV maximises the use of data for both training and evaluation while keeping each fold large enough for reliable metric estimates. Standard choice for this scale.

04 · Results

LSA improves every metric and cuts compute time by 35%.

Results across 10-fold cross-validation. The key finding: LSA does not just improve accuracy — it improves precision, recall, and AUC simultaneously, while making the model meaningfully faster.

KNN baseline LSA + KNN

Accuracy

92.1%

93.2%

Precision

76.7%

78.8%

Recall

74.9%

77.9%

AUC

85%

87%

Compute time

27.49s

17.97s

Metric	KNN (baseline)	LSA + KNN	Change
Accuracy (avg)	92.1%	93.2%	+1.1pp
Precision (avg)	76.7%	78.8%	+2.1pp
Recall (avg)	74.9%	77.9%	+3.0pp
AUC (avg)	85%	87%	+2pp
Compute time	27.49s	17.97s	−35%

"LSA doesn't just improve accuracy — it makes the model faster by giving KNN less distance to compute over a denser, more meaningful space." — Thesis conclusion, Aug 2021

05 · Product Delivery

Not a notebook. A working department chatbot.

Web application

PHPMySQLBootstrap 3jQuery

The web interface exposes the full pipeline interactively — users can see the raw input, the TF-IDF matrix, the SVD reduction, the LSA projection, and the final KNN classification result step by step.

Telegram bot

Telegram APIPHPWebhook

The same classification backend powers a Telegram bot, giving students a conversational interface to ask questions and get categorised answers at any time — without waiting for a staff reply.

Web app chatbot — antarmuka tanya-jawab mahasiswa dengan tampilan step-by-step pipeline TF-IDF, SVD, dan KNN

Web app · pipeline classification step-by-step visible

Telegram bot chatbot — mahasiswa tanya lewat Telegram dan mendapat jawaban otomatis dari sistem KNN LSA

Telegram bot · same backend, conversational interface

NLP stack

Rubix MLSastrawiNumPHP

Sastrawi provides the Nazief-Andriani Indonesian stemmer. Rubix ML handles the KNN classifier and TF-IDF vectorisation. SVD is implemented using custom PHP matrix operations — the 59 MB precomputed SVD matrix is stored as JSON for fast retrieval.

Model artifacts

JSONCSV.model files

Serialised models: knn.model (8.7 MB) and knn-lsa.model (18.5 MB). The precomputed SVD matrix (svd.json, 59 MB) is loaded at request time to avoid re-computing decomposition on each query.

06 · Technical Deep Dive

What I actually built, end to end.

→ Expand 8-step implementation walkthrough

This was not just a model experiment. I went from collecting data via a form to a working chatbot deployed on the web and Telegram — implementing KNN and LSA from the mathematical level up in PHP.

Step 01 · Data Collection

Collect real questions from the department.

I gathered student questions from the Informatics Department of UPN "Veteran" Yogyakarta using a structured form. Each question was manually labelled by topic category.

1,410 questions across multiple intent categories
Labels: per topic (academic, registration, facilities, etc.)

Step 02 · Text Preprocessing

Clean and normalise Indonesian text.

Raw questions go through a 6-stage pipeline before any ML is applied. Indonesian has complex morphology — stemming here requires a purpose-built Indonesian algorithm.

Case folding → remove punctuation → remove numbers
Tokenising → stopword removal (custom ID list)
Stemming: Nazief-Andriani algorithm for Bahasa Indonesia

Step 03 · TF-IDF Feature Extraction

Turn words into weighted vectors.

Each preprocessed question is encoded as a TF-IDF vector over the full vocabulary. This creates a high-dimensional, sparse feature matrix — the problem KNN struggles with.

Term Frequency × Inverse Document Frequency weighting
Output: sparse document-term matrix (high dimensionality)

Step 04 · SVD Decomposition

Reduce dimensions via Singular Value Decomposition.

The TF-IDF matrix is decomposed using SVD. I implemented the SVD matrix operations directly in PHP using custom numerical computation. The top-k singular values are retained.

SVD: A = U × Σ × Vᵀ
Precomputed SVD matrix: 59MB stored as JSON

Step 05 · LSA Feature Selection

Map questions into a dense semantic space.

LSA uses the reduced SVD representations to project questions into a latent semantic space, capturing relationships between words that co-occur in similar contexts.

Cosine similarity in reduced space (k=661 dimensions)
Connects semantically similar questions even with different words

Step 06 · KNN Training

Train the classifier on LSA-reduced features.

K-Nearest Neighbors is trained on the LSA-projected feature vectors. Distance is computed using Manhattan distance. The model is serialised to disk for serving.

Distance metric: Manhattan
Models: knn.model (8.7 MB), knn-lsa.model (18.5 MB)

Step 07 · K-Fold Evaluation

Evaluate rigorously with 10-fold cross-validation.

Both KNN and LSA+KNN are evaluated on all 10 folds. Accuracy, precision, recall, and AUC are compared — not just accuracy.

10-fold CV: 90% train / 10% test per fold
Metrics: accuracy, precision, recall, F-beta, AUC

Step 08 · Web App + Telegram Bot

Deploy as a working product.

The trained model is integrated into a PHP web application that visualises the full pipeline step by step, and a Telegram bot for real conversational access.

Web: PHP + MySQL + Bootstrap 3
Telegram bot: connected to same classification backend

Steps 1–2 of 8

07 · Lessons Learned

Three things I still carry from this thesis.

01 · Feature Engineering

Feature engineering > model complexity

The classifier didn't change — KNN is KNN. What changed was the feature space. LSA gave KNN better geometry to work with: denser, lower-dimensional, semantically coherent. Fix your representation before tuning hyperparameters.

02 · Language

Language matters

Applying an English pipeline to Bahasa Indonesia would have produced a fractured vocabulary and degraded every downstream metric. Indonesian stemming significantly reduced vocabulary fragmentation. Language-specific preprocessing is non-negotiable.

03 · Deployment

Research ≠ product

The thesis passed. But wiring the model to a MySQL database and exposing it through a Telegram bot taught me things no experiment could: serialisation, request handling, and what "accuracy" means when a real user gets a wrong answer.

"Accuracy improved 1.1 pp. Compute time dropped 35%. But the real result was learning to take a research paper all the way to a deployed chatbot." — Ninditya Salma Nur Aini, 2021

08 · References

Grounding the technical choices.

Deerwester et al. (1990) — LSA

Original paper introducing Latent Semantic Analysis via SVD on term-document matrices.

Tala (2003) — Indonesian Stemming

University of Amsterdam thesis evaluating Indonesian stemming algorithms including Nazief-Andriani. Primary reference for Indonesian morphological normalisation.

Rubix ML

PHP machine learning library used for KNN classifier and TF-IDF vectorisation.

Sastrawi

PHP library providing the Nazief-Andriani Indonesian stemmer used in preprocessing.

Cover & Hart (1967) — Nearest Neighbor

Seminal paper proving KNN asymptotically achieves at most twice the Bayes error rate. Theoretical foundation for using KNN as the classifier.

Kohavi (1995) — Cross-Validation

IJCAI paper establishing k-fold cross-validation as the standard for model evaluation. Grounds the 10-fold CV methodology.

Classical NLP, from research to working chatbot.