NLP · Deep Learning
Indonesian Twitter · 11,000 Tweets · Binar Academy 2023 → 2026
ML vs GenAI
Sentiment Analysis
Same dataset. Three approaches. One systematic question: when does BiLSTM beat Gemini Flash? And what does it actually cost to find out?
Dataset 11,000 Indonesian tweets · 3 classes
Models TF-IDF+MLP · BiLSTM · Gemini Flash 2.0
Evaluation Accuracy · F1 · Cost · Latency
Context Binar Academy final project, revisited 2026
Same Dataset
Indonesian Twitter · 11,000 Tweets
Approach 1
TF-IDF + MLP
89.7%
Acc · F1 0.8930
Approach 2 · Winner
BiLSTM
93.0%
Acc · F1 0.9328
Approach 3
Gemini Flash Zero-shot
85.3%
Acc · F1 0.8404
Accuracy
F1 Score
Annual Cost
Latency
01 · Business Question
If you're building Indonesian sentiment analysis in 2026, which approach do you actually choose?
GenAI is fast to prototype, but nobody has directly measured it against trained LSTM on the same Indonesian dataset with the same evaluation — including cost and latency.
This project started as a 2023 Binar Academy final project (TF-IDF + MLP, 89.67%). In 2026 I revisited it with two questions: can a BiLSTM close the gap, and is GenAI zero-shot a practical alternative for production use?
The evaluation covers four dimensions: accuracy, cost, latency, and prompt engineering effort. The answer isn't obvious until you look at all four together.
Indonesian NLPTF-IDF
BiLSTMGemini Flash 2.0
Cost AnalysisREST API
02 · Final Result
BiLSTM wins — not just on accuracy, but on every dimension that matters for production.
93.0% accuracy, F1 0.9328, 0.25ms inference, $0/year. GenAI zero-shot sits 7.7 points lower in accuracy and costs $62/year for the same volume.
🏆 Winner
BiLSTM
93.0%
F1 Score0.9328
Latency0.25ms
Annual cost$0
#2
TF-IDF + MLP
89.7%
F1 Score0.8930
Latency0.0008ms
Annual cost$0
#3
Gemini Flash Zero-shot
85.3%
F1 Score0.8404
Latency~195ms
Annual cost$62+
Decision matrix
| Criterion |
TF-IDF + MLP |
BiLSTM |
Gemini Zero-shot |
| Accuracy | 🟡 | 🟢 | 🔴 |
| Macro F1 | 🟡 | 🟢 | 🔴 |
| Latency | 🟢 | 🟡 | 🔴 |
| Annual cost | 🟢 | 🟢 | 🔴 |
| Training needed | 🔴 | 🔴 | 🟢 |
| Labeled data needed | 🔴 | 🔴 | 🟢 |
| Neutral class F1 | 🔴 | 🟢 | 🔴 |
When you have labeled data: BiLSTM is the clear choice — higher accuracy, zero cost, sub-millisecond inference. GenAI zero-shot earns its place only when you need to classify without any training data or when you're building a quick prototype before the labeled dataset exists.
03 · Architecture
Three pipelines, same evaluation framework.
Each approach was evaluated under identical conditions: same 11,000-tweet dataset, same 80/20 train-test split, same metrics. Only the pipeline changed.
Raw tweet text
↓
Preprocessing (lower, regex, stopwords)
↓
TF-IDF vectorization (max_features=5000)
↓
Dense MLP (512→256→128→3)
↓
Softmax → class label
Raw tweet text
↓
Preprocessing (lower, regex, stopwords)
↓
Tokenizer + pad_sequences (maxlen=100)
↓
Embedding layer (dim=128)
↓
SpatialDropout1D + Bidirectional LSTM (128)
↓
Dense(64, relu) + L2 + Dropout
↓
Softmax → class label
Raw tweet text
↓
Prompt construction (ZS / FS / CoT)
↓
Gemini Flash 2.0 API call
↓
JSON response parsing
↓
Class label extraction
04 · Dataset & EDA
11,000 Indonesian tweets. Class imbalance baked in.
The dataset is heavily skewed toward positive sentiment — 58% positive, 31% negative, just 10% neutral. This imbalance shapes every evaluation decision, especially for the neutral class.
The neutral class (10%) proved hardest to classify across all three approaches. BiLSTM achieved the highest neutral F1 at 0.96 — significantly better than NN (0.77) or GenAI zero-shot. The imbalance is why macro-averaged F1 matters more than accuracy alone.
Per-class performance (BiLSTM — best model)
| Class |
Precision |
Recall |
F1 |
Support |
| Negative | 0.83 | 0.86 | 0.84 | 997 |
| Neutral | 0.96 | 0.95 | 0.96 | 1,035 |
| Positive | 0.93 | 0.92 | 0.92 | 1,930 |
05 · Training Journey
Four iterations to get to 93%. Each one taught something different.
The baseline LSTM overfit badly. Each iteration added one intervention and measured the delta. The best result came at iteration 3 — adding L2 in iteration 4 slightly hurt.
Baseline
Unidirectional LSTM, no regularization
89.9%
Starting point
+ Early Stopping
patience=5, restore best weights
90.2%
+0.28%
+ BiDir + SpatialDropout
Bidirectional LSTM, SpatialDropout1D(0.2)
90.9%
+0.76%
+ L2 regularization → Final
L2(0.01) on Dense layer → best validation
93.0%
Best overall
Training history
06 · GenAI Experiments
Six prompt strategies tested. More examples ≠ better results.
The GenAI track tested zero-shot, 3/5/10-shot few-shot, chain-of-thought, and multilingual prompting. Adding more examples increased cost but did not reliably improve accuracy.
Temperature sweep — accuracy vs randomness
Temperature 0.0 (deterministic) produced the highest accuracy at 86.0%. Higher temperatures introduced variance without improving results.
Prompt strategy comparison
07 · Benchmark
Accuracy, F1, and latency side by side.
Accuracy & F1
Latency comparison (log scale)
GenAI is 772× slower than the neural network. On log scale, this 24ms → 195ms range dwarfs both trained models. Bar width represents log₁₀ position.
Log₁₀ scale: NN=−3.10, LSTM=−0.60, GenAI=2.29. Bar widths normalized to log range. Actual difference: 772× NN→GenAI.
Benchmark charts
08 · Cost Analysis
More prompting strategy = more cost, not more accuracy.
Assuming 1,000 classifications/day at 2025 Gemini Flash pricing. Zero-shot costs $62/year. Chain-of-thought costs $304/year — and doesn't improve accuracy over zero-shot.
Trained ML (BiLSTM)
$0
/year · after training
Acc 93.0% · F1 0.9328
Zero-shot
$62
/year · 1K req/day
Acc 85.3% · F1 0.8404
3-shot few-shot
$120
/year
~same accuracy as ZS
5-shot few-shot
$175
/year
Marginal improvement
10-shot few-shot
$268
/year
Diminishing returns
Chain-of-Thought
$304
/year
No accuracy gain vs ZS
"The Pareto-optimal point is zero-shot at $62/year — but even that is dominated by BiLSTM at $0. The entire GenAI cost curve sits below 89% accuracy."
