求解释一下MLP

Google有这样一个介绍：
https://developers.google.com/ma ... assification/step-4

里面有一部分介绍 Build n-gram model【Option A】：
We refer to models that process the tokens independently (not taking into account word order) as n-gram models. Simple multi-layer perceptrons (including logistic regression), gradient boosting machines and support vector machines models all fall under this category; they cannot leverage any information about text ordering.

We compared the performance of some of the n-gram models mentioned above and observed that multi-layer perceptrons (MLPs) typically perform better than other options. MLPs are simple to define and understand, provide good accuracy, and require relatively little computation.

The following code defines a two-layer MLP model in tf.keras, adding a couple of Dropout layers for regularization (to prevent overfitting to training samples).

接下来有一段code，我的问题是这个code怎么反映出来ngram的？我怎么知道这段code是unigram，bigram，trigram还是其他的gram。。。

1. from tensorflow.python.keras import models
   
2. from tensorflow.python.keras.layers import Dense
   
3. from tensorflow.python.keras.layers import Dropout
   
4. 
   
5. def mlp_model(layers, units, dropout_rate, input_shape, num_classes):
   
6.     """Creates an instance of a multi-layer perceptron model.
   
7. 
   
8.     # Arguments
   
9.         layers: int, number of `Dense` layers in the model.
   
10.         units: int, output dimension of the layers.
    
11.         dropout_rate: float, percentage of input to drop at Dropout layers.
    
12.         input_shape: tuple, shape of input to the model.
    
13.         num_classes: int, number of output classes.
    
14. 
    
15.     # Returns
    
16.         An MLP model instance.
    
17.     """
    
18.     op_units, op_activation = _get_last_layer_units_and_activation(num_classes)
    
19.     model = models.Sequential()
    
20.     model.add(Dropout(rate=dropout_rate, input_shape=input_shape))
    
21. 
    
22.     for _ in range(layers-1):
    
23.         model.add(Dense(units=units, activation='relu'))
    
24.         model.add(Dropout(rate=dropout_rate))
    
25. 
    
26.     model.add(Dense(units=op_units, activation=op_activation))
    
27.     return model

复制代码

谢谢

mlp接收vector/matrix作为输入，所以你需要把text转成vector，可以用ngram，具体n是多少由自己定义，mlp接收转换成vector的data作为输入。
ngram里没有word embedding，只有每个gram vocab的count。比如unigram你就count vocab里的每个单词数量，bigram就count vocab里的 两词短语的数量。vocab也是自己定义的，根据你关注哪些gram

我猜是input_shape决定的，每次把n 个 word embedding 同时输入进去。

(在地里甚至可以讨论学术)

这边说的不是NLP的ngram，而是ngram model，相对于文章底下的sequence model而言的（也所以跟unigram/bigram没有关系）。这词感觉是G自己发明的。。。它就是指这类model的input feature之间没有顺序关系：你的所有training data从(f0, f1, f2)变成(f2, f1, f0)的话，MLP的性能不变，相比之下sequence model的性能会变。

我找到答案了，大家感兴趣的可以去看看step 2，里面提到了具体的implementation

补充内容 (2019-10-3 08:48):
应该是step 3

pagpires 发表于 2019-10-3 04:18
这边说的不是NLP的ngram，而是ngram model，相对于文章底下的sequence model而言的（也所以跟unigram/bigra ...

谢谢，确实是跟顺序没关系。但是这个词就是nlp的n-gram，不是G家发明的，你也可以看看step 2：）