If NN (Neural Network) beats baseline, present the NN solution.

If baseline beats NN, present NN as the baseline, and say you have an algorithm even better than NN.

(Joke only.)

I mean...you can always appeal to “old school” AI. Just dig in to the old papers and use their words. Latent semantic analysis (LSA) is an example of a hard to beat baseline model for text:

“By inducing global knowledge indirectly from co-occurrence data in a large body of representative text, LSA acquired knowledge about the full vocabulary of English at a comparable rate to schoolchildren.” (http://www.stat.cmu.edu/~cshalizi/350/2008/readings/Landauer...)

It's not always easy with MBA types.

I once had a mentor with clout on a 9 figure investment committee tell me that maximum likelihood estimation is "the dumbest idea" he'd ever heard.

Words like "Cramer-Rao bound" didn't get through. What worked was saying "deep learning is usually just MLE with a mysteriously effective function approximation".

Modern methods for deriving word embeddings easily beat LSA.

Hard to beat in terms of effort vs. quality of the outcome is more accurately what I meant (it’s two lines of code in scikit-learn [CountVectorizer() + TruncatedSVD()] to go from raw text to document/word embeddings, and the result is often “good enough” depending on what you’re trying to do). See the results on pg. 6 (note LSI==LSA): http://proceedings.mlr.press/v37/kusnerb15.pdf

Also, at least based on papers I’ve read recently, BERT doesn’t work that well for producing word embeddings compared to word2vec and GloVe (which can be formulated as matrix factorization methods, like LSA). See table on pg. 6: https://papers.nips.cc/paper/9031-spherical-text-embedding.p...

Point being: mastering the old models gives you a solid foundation to build from.

Agree, but I bet LSA is still a good baseline due to it's simplicity.