Stanford Machine Learning Week 3: Regularization

我正在上Stanford Machine Learning这门课，第三周的编程作业刚刚交过了，但是关于Regularization的这两道题试了两次都没过，我也重新看了视频，觉得没有过多说这两道题里面的内容，有明白的童鞋能否帮忙讲解一下，非常感谢帮忙。

=====================================================

这一道我选的是：BC

You are training a classification model with logistic regression. Which of the following statements are true? Check all that apply.

A：Adding many new features to the model helps prevent overfitting on the training set.

B：Introducing regularization to the model always results in equal or better performance on the training set.

C：Adding a new feature to the model always results in equal or better performance on the training set.

D：Introducing regularization to the model always results in equal or better performance on examples not in the training set.

=======================================================================================

这一道我选的是：BD

Which of the following statements about regularization are true? Check all that apply.

A：Because logistic regression outputs values 0≤hθ(x)≤1, it's range of output values can only be "shrunk" slightly by regularization anyway, so regularization is generally not helpful for it.

B：Using a very large value of λ cannot hurt the performance of your hypothesis; the only reason we do not set λ to be too large is to avoid numerical problems.

C：Using too large a value of λ can cause your hypothesis to overfit the data; this can be avoided by reducing λ.

D：Consider a classification problem. Adding regularization may cause your classifier to incorrectly classify some training examples (which it had correctly classified when not using regularization, i.e. when λ=0).

5. You are training a classification model with logistic regression. Which of the following statements are true? Check all that apply.【D】 A. Introducing regularization to the model always results in equal or better performance on the training set.Introducing regularization to the model always results in equal or better performance on the training set.  【解析】If we introduce too much regularization, we can underfit the training set and have worse performance on the training set. B.Adding many new features to the model helps prevent overfitting on the training set. 【解析】Adding many new features gives us more expressive models which are able to better fit our training set. If too many new features are added, this can lead to overfitting of the training set. C. Adding a new feature to the model always results in equal or better performance on examples not in the training set. 【解析】Adding more features might result in a model that overfits the training set, and thus can lead to worse performs for examples which are not in the training set. D.Adding a new feature to the model always results in equal or better performance on the training set. 【解析】By adding a new feature, our model must be more (or just as) expressive, thus allowing it learn more complex hypotheses to fit the training set.

看到楼主回复我的帖子了。quiz刚刚做完。这两题正好我刚做过。
第一题只有C对。B是错的，因为在介绍overfitting时给了定义如下：If we have too many features, the learned hypothesis may fit the training set very well, but fail to generalize to new examples. 而regularization是为了解决overfitting的，所以regularization不可能在TRAINING SET上比原先已经“overfit”的情况做得更好。

第二题确实只有D是对的。你再重新做一次之后提交试试？B错的原因是，lamda过大会导致theta1，theta2，……，thetan很小，最终只有一个theta0其作用，那么decision boundary会变成接近一条平行于x轴的直线。这一部分andrew在lecture有提到，你可以再看一下。

hope my answer helps.

stanslug 发表于 2016-3-28 23:46
第二题只选D提交了以后还是提示错误。

我还是觉得你发出来的第二题只选D是没错的。你再提交的时候仔细看看选项，他的多选题每次重新做的时候选项的顺序和叙述都是不一样的，所以答案也不是固定的。

第二题的A肯定错的，regularization直接shrunk的是theta，对控制logistic regression overfit是有用的。
BC错，lambda不能太大是怕underfit (极端些如果lambda趋向正无穷，theta都会趋向0) lambda太小才有可能overfit

时间久了，有点记不清了。仅供参考
第一题
CD.
正则化是为了更好的扩展适用于非训练集，防止过拟合训练集。 more features 是更好的fit 训练集。

第二题
BC都错误， 过大的lambda应该会严重影响算法吧。会很严重的欠拟合，我记得是变成一条横线。
看起来只能选D了。

faelivrin 发表于 2016-3-28 10:21
时间久了，有点记不清了。仅供参考
第一题
CD.

谢谢帮忙解释。

我刚刚提交了，不过还是错的。

第一题只有C，第二题只有D，吧

BabeLucia 发表于 2016-3-28 13:21
第一题只有C，第二题只有D，吧

第二题只选D提交了以后还是提示错误。

膜拜一记大神……

请问下这一个quiz的这一题是什么意思。感觉半天没看懂。

Suppose you ran logistic regression twice, once with λ=0, and once with λ=1. One of the times, you got

parameters θ=[74.8145.05], and the other time you got

θ=[1.370.51]. However, you forgot which value of

λ corresponds to which value of θ. Which one do you

think corresponds to λ=1?

提前谢谢