Sorting my inbox with a model I trained on it
My personal Gmail had a few thousand unsorted emails. I didn't want to hand all of them to someone else's servers, so I sorted them on my own PC.
Three attempts
A small classifier, untrained. I tried Laya, a small classification model, straight off the shelf. It marked 29 of the first 50 emails as spam.
The big local model. Qwythos 9B, the uncensored model on my graphics card, read each email and picked a category. It was much better, and slow. I ran it over 1,000 emails without applying anything, then built a plain review page and corrected every label by hand.
The small classifier, trained. I used those 1,000 reviewed emails to fine-tune Laya. On emails held back from training it went from 26.3% correct to 87.4%. Training needed 7.8 GB on the graphics card, so the big model had to be unloaded first.
The order it runs in now
- Simple rules for senders I already know.
- Gmail's own tabs.
- The small model, if it's at least 80% sure.
- The big local model for whatever is left.
- The small model again at 50%, as a last resort.
About 2,600 emails are labelled so far.
What I learned
- The reviewed examples mattered more than the model. The same small model was useless before them and good after.
- Rare categories don't get learned. The training set had five personal emails, five spam, three about jobs and one social. The small model still can't recognise any of those.
- The slow part isn't the AI. Gmail limits how fast you can download your own mail, which works out to about nine minutes per 1,000 emails. Applying the labels takes about one.