注册一亩三分地论坛,查看更多干货!
您需要 登录 才可以下载或查看附件。没有帐号?注册账号 
x
之前放出了自己的一些面试准备笔记,大家都很喜欢,现在放出第二弹。第一弹在这里:https://www.1point3acres.com/bbs ... &extra=page%3D2 首先道个歉,由于笔记是随着学习一点一点准备的,历时几个月,所有会有很多重复内容,请大家谅解。如果有整理错误的地方,请指正。
楼主只放纯干货,觉得有用的话,请大家加米,谢谢!
Machine learning books: Machine learning videos: 1. Linear regression residue: normally distributed. Otherwise, the data need to be shuffled or the model need to be changed to a higher order model. 用来判断模型是否足够复杂! 2. Evaluation linear regression 可以使用MSE,或者使用R squared(coeffiencient of detemination)。选择logistics regression的阈值,可以用F1 score或者ROC选出最优
3. Linear regression error evaluation, MSE cost function 可以保证convex。MAE不可导,不可做cost function避免local minima:
4. False positive = Type 1 error: reject H0, but H0is true; False negative = Type 2 error: accept H0, but H0is false 5. KNN: Almost no training, only need to store the training samples. Prediction cost is very high; Performance is poor for high dimensional data; Multi-class don’t work well 6. Random forest: Randomly choose feature. 这样防止有一个特别强的特征,在bagging一直起到主导作用,使bagging (bootstrap aggregating)失去意义 9. 使用momentum可以加速stochastic gradient descent。可用可变alpha学习速率加速学习 10. Sample in 0.1 – 0.00001, 使用log域,可以保证概率一致。不然大部分数字将会集中在10-1级 11. Vanishing and exploding gradient,对于多层神经网络,使用sigmoid函数时,由于初始化不当,前面几层的参数过小,导致后面偏差变为0,是vanishing。Exploding相反,输出饱和。解决方法是用ReLu代替sigmoid 12. Reinforcement learning: 输入数据本身没有标签,让机器自由输出。通过设定奖励规则,给机器的输出标记标签,达到学习目的。有model-free RL: Q learning, Policy Gradients和model-based RL等。Policy-based是基于概率的选择,可以选择连续动作,比如下棋等。valus-based为基于价值的选择。 13. Reinforcement learning: 对于更新动作,有Monte-Carlo update (Policy Gradients,Monte-Carlo Learning), 将所有动作完成后,最终评价结果。还有Temporal-Difference update(Q Learning,Sarsa),对于每一步都进行打分和评价,运行更有效率。 14. Reinforcement learning: On-policy:机器自己玩,自己评估。Off-policy:让机器看人玩,获得输入数据 .
To get Power: a. Bigger sample b. Look for large change
c. Make alpha larger (make null hypothesis smaller, but will increase type I error) . 1point3acres.com
22. Chi-Square test: a. Useful for estimating statistical parameters where data are non-normal b. Have unknown statistical properties c. Lack a standard calculation
24. 决策树C4.5 vs ID3 训练法对比: a. Handling both continuous and discrete attributes - In order to handle continuous attributes, C4.5 creates a threshold and then splits the list into those whose attribute value is above the threshold and those that are less than or equal to it. b. Handling training data with missing attribute values - C4.5 allows attribute values to be marked as ? for missing. Missing attribute values are simply not used in gain and entropy calculations.
c. Pruning trees after creation - C4.5 goes back through the tree once it's been created and attempts to remove branches that do not help by replacing them with leaf nodes . ----
26. Regression 使用Categorical data: a. One-Hot encoding for categorical data b. Arbitrary numbers for ordinal data c. Use something like group means for categorical data (e. g. mean prices for city districts). 如果直接使用数值表示categorical data有时可以用,有时会出现较大问题:例如,logistic regression, 分类数据1,2,3对应结果为true,false,true时,经过训练,如果1的数据较少,3数据较多,则1会失去预测功能,产生较大残差 27. 梯度消失和梯度爆炸:使用ReLu可以改善 梯度消失:梯度消失是back-propagation导致。具体现象是,由于back-propagation,靠近输出的神经层参数更新比较快,靠近输入得参数,更新很慢,多次迭代仍然离初始值不远。原因是使用sigmoid函数或者tanh函数,其导数值小于1,使梯度接近于零,如下图 梯度爆炸: 初始化时选值过大,浅层比深层参数变化更快,从而发散 .google и
28. Gradient descent:回归模型初始化时,如果初始化值正好是cost function的最优值时,gradient descent的微分就会变成0,参数不会被更新,会一直保持不变。如果使用恒定步长时alpha,当迭代趋近最小值是,参数的导数会趋于平缓,更新会自然的变得越来越慢。 29. Back propagation (Andrew Ng 课9.1和9.2): 从输出把error反传回每个节点,用来求每个节点的参数对cost function的偏导数,这样才能使用gradient descent
.
30. Gradient checking: 在参数上加上和减去一个很小的数值,验证cost function的微分值应约等于线性机计算的数值。此方法用来验证获得的偏导数是正确的
31. 神经网络初始化,避免使用全0初始化。使用全0,back propagation之后会导致一些节点无法更新,节点参数一直卡在0的状态。在一个小的范围内,使用随机初始化所有参数。 32. 神将网络做classification的时候,输出也应该使用one hot encoding,避免使用数值代替category 33. 选择模型阶数:要把数据分成三份,training,CV和test。从最低阶试起,开始训练。先使用training和CV数据找到误差合适的阶数最低的模型,然后再用training和test数据,用test数据控制iteration停止,防止overfitting。同样,选择regularization参数l的时候,也要另用一组数据。每次选择一个hyper-parameter,都要用一组新数据。
34. Collaborative filtering: 通过用户互动,确定电影的分类信息x1, x2值。之后可以通过计算两个电影feature之间的距离,找到相似的电影 35. MapReduce on machine learning: 只要涉及到training sample的sum operation,都可以分配成多机器 36. 除了gradient descent,还有lbfgs算法求极小值,或者直接对参数求导数,令一阶导数为零,二阶导数为负数。Logistic regression可以用牛顿法: 优缺点: 牛顿法用二次曲面逼近所求曲面,通过导数迭代求极值。 要求Hessian矩阵必须有逆;cost function必须二阶可导;迭代只是局部的,初值值要足够接近终止,否则发散;不能保证每次迭代都逼近最优解 37.Machine learning system design: 了解Machine learning pipeline
38. Collinearity feature理论上不会影响模型预测,但是会影响regularization项,使训练出问题,iso-surface上,会影响最小值点的位置。同样SVM的kernel也会受到影响。只要是feature之间非线性的操作都会受到影响。可以使用PCA减少collinearity。
39. NLP:bag of words, TF-IDF(可以消除it,the等高频词)。 使用NLTK包tokenizer: 使用stop words, Tokenizer feature vector使用binary,word出现为1,不出现为0,不计出现次数。 使用bag of word,使用raw count的缺点:长文章形成的向量模长会很长,短文章向量很短。需要normalize count Latent sematic analysis:减少word feature,组合近义词,例如,z = 0.7*computer + 0.2*PC + 0.1*laptop。LSA也是一种dimensionality reduction手段。 40. Continuous feature转成categorical feature:把连续特征放入bin中,然后用bin的index作为categorical feature White noise is stationary 42.What is R squared, diff between R squared: Variations represented by the model in percentage. 1 – R2means the variations we are missing, either missing feature or missing model complexity 43. P, Q, D in Arima model. P is the number of points used in autoregressive model; Q is size of moving average window, moving average is applied on residual error term, usually white noise, not on Ytterm; D is the degree of differencing (the number of times the data have had past values subtracted), Yt– Yt-1, 可以理解为向前的D阶微分, differencing 可以使time series变stable,没有offset或者trending。 44. Fourier transformation, Gabor transformation 47. Multilayer perceptron 就是feedforward的ANN
|