How to test NLP models from the lens of software engineering?

机器学习

🔗

linamagr 2022-5-16 04:36:12 | 只看该作者 |倒序浏览

本楼：

👍 0

👎

全局：

0 0

注册一亩三分地论坛，查看更多干货！

您需要登录才可以下载或查看附件。没有帐号？注册账号

x

Welcome to the MLnotes Newsletter. Here we discuss topics in learning and having a career in ML, as well as interesting industry applications.

Today we would like to get some tips on what to think about before we launch a NLP model to production.

Once we built our new NLP model, beat some benchmarks, what should we do next? When should we pull the trigger to use it in a real production setting?

Microsoft Researcher and Assistant Professor at the University of Washington, Marco Tulio Ribeiro, talked about how to think about testing NLP models in preparation for a successful launch, similar to how we are used to thinking about testing softwares using unit test, regression test, and integration test.

Principal #1: Testing small units (unit test)[1]
In case of a NLP model: Go beyond looking at the accuracies of a test dataset, and test specific "linguistic capabilities", such as the following:

- Vocabulary/POS
- Named Entities
- Negation

Read more here:
https://mlnotes.substack.com/p/h ... &utm_medium=web

Happy practicing!