注册一亩三分地论坛，查看更多干货！
您需要 登录 才可以下载或查看附件。没有帐号？注册账号
x
关于linear regression 和 logistic regression面试问题的总结，欢迎大家指正错误和补充, 也请XDJM们高抬贵手给加点大米。谢谢！
1. Bias-Variancetrade-off In statistical modeling (supervised machinelearning), the prediction error is made of two parts, bias and variance. Bias is the difference between the averageprediction of our model and the actual value we are trying to predict. Modelwith high bias pays very little attention to the training data andoversimplifies the model (underfitting, model too simple, data too less). Italways leads to high error on training and test data. Variance is the variability of theestimated model when estimated/trained using different training data set. Highvariance model pays too much attention to the noise in the training data, ratherthan the trend of the data (overfitting, model too complex, e.g. Decision tree).High variance models perform very well on training data but have high errorrates on test data. If our model is too simple and has very fewparameters, then it may have high bias and low variance. On the hand if ourmodel is too complex and has large number of parameters then it’s going to havehigh variance and low bias. To build a good model, we need to find a goodbalance between bias and variance such that it minimized the total error.
2. linearregression performance measure: typically use RMSE (root mean square error,l2 norm), if there are many outliers consider using MAE (mean absolute error,l1 norm). assumption: 1). Linearrelationship: the dependent variable has a roughly linear relationship witheach of the independent variable Check: (a)scatter plot between each independent variable and dependent variable or (b)Pearson correlation between each independent variable and dependent variable,df.corr(dependent.var) (c) the scatterplot of the actual outcome variableagainst the predicted outcome includetransformed versions of the predictors in the model, for instance, to include aquadratic term (X**2) 2). Noperfect multicollinearity: There is no perfect linear relationship betweenexplanatory variables. Check:examine a correlation matrix. Ifcorrelations are above .80 then you may have a problem The Variance inflation factor (VIF) andtolerance statistic can tell you whether a given explanatory variable has astrong relationship with the other explanatory variables. a VIF value thatexceeds 5 or 10 indicates a problematic amount of collinearity. rectify: Either iterativelyremove the X var with the highest VIF or, See correlation between allvariables and keep only one of all highly correlated pairs. 3). Homoscedasticity: he variance of theresiduals is constant at all levels of the predicted outcome from the fullmodel. Check: plottingthe residuals against the values predicted by the regression model. In caseswhere the assumption of homoscedasticity is not met it may be possible to transformthe outcome measure (dependent variable) 4). Residualsshould be independently distributed (error terms are not correlated) When errorterms are correlated, estimated standard errors will tend to underestimate thetrue standard errors. Confidence interval of estimated coefficients isnarrower, P value smaller, etc. We areover confident about the model. Check: Suchcorrelations frequently occur in the context of time series data, we can plotthe residuals from our model as a function of time to see if there is nodiscernible pattern. 5). Normallydistributed residuals: (note: neither independent nor dependent variablesare required to be normal, only the residual)The residuals shouldbe normally distributed A histogram ofthe residuals (errors) inour model can be used to check that they are normally distributed.However it is often hard to tell if the distribution is normal from just ahistogram so additionally you should use a P-P plot ofstandardised regression residual, if we can see the expected and observedcumulative probabilities matches, it suggests that the residuals areapproximately normally distributed If residualis not normal, · Sometimesnon-normality of residuals indicates presence of outliers. If this is the case,handle the outliers first. · Maybe using some transformations solve the purpose. · Additionally, to deal with multi-colinearity, you canrefer · Theremay be overfitting the response with many independent variables. · yourmodel doesn't correctly/fully explain your data. Check different kind ofmodels. · Yourdata may not contain enough covariates (independent variables) to explain theresponse (outcome) Outliers: weconsider a data point to be an outlier only if it is extreme with respect tothe other y values, not the x values. (generally, not influential to fittedregression line, predicted value, estimated coefficients, hypothesis testresults) Check: studentizedresiduals. Studentizedresidual are computed by dividing each residual ei by its estimated standarderror. Observations whose studentized residuals are greater than 3 in absolutevalue are possible outliers. If we believethat an outlier has occurred due to an error in data collection or recording,then one solution is to simply remove the observation. However, care should betaken, since an outlier may instead indicate a deficiency with the model, suchas a missing predictor. Highleverage point: have an unusual value for xi. Check: leverage statistic The leverage statistic hi is always between 1/nand 1, and the average leverage for all the observations is always equal to(p+1)/n. So if a given observation has a leverage statistic that greatlyexceeds (p+1)/n, then we may suspect that the corresponding point has highleverage. If a pointhas an extreme x value, but the data point does follow the general trend of therest of the data. then it is notdeemed an outlier here. But it does have high leverage. This data point is notinfluential. Influentialpoints: A data point which is an outlier as well as a high leverageobservation. This is a particularly dangerous combination! Check: (1)scatter plot of studentized residual VS leverage (2) Cook’s distance Cook's Di depends on both the residual, ei, and the leverage, hii. That is, both the x value and the y value of the data point play a role in thecalculation of Cook's distance. not everyoutlier or high leverage data point strongly influences the regressionanalysis. The mean ofresiduals is zero: Check: mean(mod$residuals) Variance inall predictors: Explanatory variables may be continuous, ordinal or nominal but each musthave at least a small range of values. Check:histogram of each independent variable Samplesize: In orderfor your regression model to be reliable and capable of detecting certaineffects and relationships you will need an appropriate sample size. There is ageneral rule of thumb for this: For each explanatory variable in the model 15cases of data are required. A goodsample size depends on the strength of the effect or relationship that you'retrying to detect - the smaller the effect you’re looking for the larger the sampleyou will need to detect it! Givenpractically possible, is the bigger the sample size the better? NO, cautionis required when interpreting data from large samples. A dataset this large isvery likely to produce results which are 'statistically significant'. This isbecause the sheer size of the sample overwhelms the random effects of sampling.'statistically significant' finding can have the effect of causing researchersto over emphasize their findings. A p-value does not tell the researcher how large aneffect is and it may be that the effect is statistically significant but sosmall that it is not important. For this reason it is important to look at the effect size (the strength) of an effect orrelationship as well as whether or not it is statistically likely to haveoccurred by chance in a sample. Of course it is also important toconsider who is in yoursample. Does it represent the population you want it to? There isalso software that will allow you to estimate quite precisely the sample sizeyou need to detect a difference of a given size with a given level ofconfidence. One example is the dramatically named ‘GPower’. python code: from pandas import Series, DataFrame [color=rgba(0, 0, 0, 0.84)]importpandas as pd [color=rgba(0, 0, 0, 0.84)]importnumpy as np [color=rgba(0, 0, 0, 0.84)]importos [color=rgba(0, 0, 0, 0.84)]importmatplotlib.pylab as plt [color=rgba(0, 0, 0, 0.84)]fromsklearn.model_selection import train_test_split [color=rgba(0, 0, 0, 0.84)]fromsklearn.linear_model import LinearRegression [color=rgba(0, 0, 0, 0.84)]importsklearn.metrics [color=rgba(0, 0, 0, 0.84)]raw_data= pd.read_csv(“history.csv”) [color=rgba(0, 0, 0, 0.84)]raw_data.dtypes raw_data.head() cleaned_data = raw_data.drop(“CUST_ID”,axis=1) [color=rgba(0, 0, 0, 0.84)]cleaned_data .corr()[‘CLV’] predictors = cleaned_data.drop(“CLV”,axis=1) [color=rgba(0, 0, 0, 0.84)]targets= cleaned_data.CLV [color=rgba(0, 0, 0, 0.84)]pred_train,pred_test, tar_train, tar_test = train_test_split(predictors, targets,test_size=.1) [color=rgba(0, 0, 0, 0.84)]print(“Predictor — Training : “, pred_train.shape, “Predictor — Testing : “,pred_test.shape) #Build model on training data [color=rgba(0, 0, 0, 0.84)]model= LinearRegression() [color=rgba(0, 0, 0, 0.84)]model.fit(pred_train,tar_train) [color=rgba(0, 0, 0, 0.84)]print(“Coefficients:\n”, model.coef_) [color=rgba(0, 0, 0, 0.84)]print(“Intercept:”,model.intercept_) [color=rgba(0, 0, 0, 0.84)]#Teston testing data [color=rgba(0, 0, 0, 0.84)]predictions= model.predict(pred_test) [color=rgba(0, 0, 0, 0.84)]predictions [color=rgba(0, 0, 0, 0.84)]sklearn.metrics.r2_score(tar_test,predictions) new_data = np.array([100,0,50,0,0,0]).reshape(1,-1) [color=rgba(0, 0, 0, 0.84)]new_pred=model.predict(new_data) [color=rgba(0, 0, 0, 0.84)]print(“TheCLV for the new customer is : $”,new_pred[0])
3. logisticalregression . check 1point3acres for more.
performance measure: maximum likelihood Assumptions: First, logistic regression does not require alinear relationship between the dependent and independent variables. Second, the error terms (residuals) do not need to be normallydistributed. Third, homoscedasticity is not required. Finally, thedependent variable in logistic regression is not measured on an interval orratio scale.
However, some other assumptions still apply. ..
1). First, binary logistic regression requiresthe dependent variable to be binary and ordinal logistic regression requiresthe dependent variable to be ordinal. 2). Second, logistic regression requiresthe observations to be independent of each other. In other words, theobservations should not come from repeated measurements or matched data. 3).Third, logistic regression requires there tobe little or no multicollinearity among the independent variables. Thismeans that the independent variables should not be too highly correlated witheach other. 4). Fourth, logistic regression assumeslinearity of independent variables and log odds. although this analysisdoes not require the dependent and independent variables to be relatedlinearly, it requires that the independent variables are linearly related tothe log odds. 5). Finally, logistic regression typicallyrequires a large sample size. A general guideline is that you need atminimum of 10 cases with the least frequent outcome for each independentvariable in your model. For example, if you have 5 independent variables andthe expected probability of your least frequent outcome is .10, then you wouldneed a minimum sample size of 500 (10*5 / .10). related questions: python codefor logistic regression: from sklearn.linear_modelimport LogisticRegression log_reg =LogisticRegression() log_reg.fit(X,y) X_new =np.linspace(0,3,1000).reshape(-1,1) Y_proba =log_reg.predict_proba(X_new)
Q1: What is a logisticfunction? What is the range of values of a logistic function? f(z) = 1/(1+e -z ) The valuesof a logistic function will range from 0 to 1. The values of Z will vary from-infinity to +infinity. Q2: How can the probabilityof a logistic regression model be expressed as conditional probability? P(Discrete value of Target variable | X1, X2, X3….Xk). It is the probability of the target variable to take up a discrete value(either 0 or 1 in case of binary classification problems) when the values ofindependent variables are given. Q3: What are odds? What is odds ratio? It is the ratio of the probability of an event occurring tothe probability of the event not occurring. Odds ratio is the ratio of oddsbetween two groups. If odds ratio = 1, then there is no differencebetween the Q4: How to interpret the results of alogistic regression model? Or, what are the meanings of beta_0 and beta_i in alogistic regression model?Beta_0 is the baseline in a logisticregression model. It is the log odds for an instance when all the attributes(X1, X2,………….Xk) are zero. In practical scenarios, the probability of all theattributes being zero is very low. In another interpretation, Alpha is the logodds for an instance when none of the attributes is taken into consideration.Beta_i is the value by which the log oddschange by a unit change in a particular attribute by keeping all other attributesfixed or unchanged (control variables). Q5: Is the decision boundary linear ornonlinear in the case of a logistic regression model?For logistic regression model, thedecision boundary is a straight line.Logistic regression model formula =α+1X1+2X2+….+kXk. This clearly represents a straight line. Logistic regressionis only suitable in such cases where a straight line is able to separate thedifferent classes. If a straight line is not able to do it, then nonlinearalgorithms should be used to achieve better results. Q6: What is the Maximum LikelihoodEstimator (MLE)?The likelihood function gives theprobability of observing the results using unknown parameters. The MLEchooses those sets of unknown parameters (estimator) that maximize thelikelihood function. The method to find the MLE is to use calculus and settingthe derivative of the logistic function with respect to an unknown parameter tozero and solving it will give the MLE.MLE is a statistical approach to estimating theparameters of a mathematical model. MLE and ordinary square estimation give thesame results for linear regression if the dependent variable is assumed to benormally distributed. MLE does not assume anything about independent variables. Q7: What is the output of a standard MLEprogram? The output of a standard MLE program is asfollows: Maximized likelihood value: This is the numerical valueobtained by replacing the unknown parameter values in the likelihood functionwith the MLE parameter estimator. Estimated variance-covariance matrix: The diagonal of this matrixconsists of estimated variances of the ML estimates. The off-diagonal consistsof the covariances of the pairs of the ML estimates. Q8: what is Cross-entropy? Cross-entropy loss, or log loss, measures theperformance of a classification model whose output is a probability valuebetween 0 and 1. Cross-entropy loss increases as the predicted probabilitydiverges from the actual label. Code def CrossEntropy(yHat, y): if y == 1: return -log(yHat) else: return -log(1 - yHat) Q9: confusion matrix: A confusion matrix is a summary of prediction results on aclassification problem.The number of correct and incorrect predictions aresummarized with count values and broken down by each class. Python code: From sklearn.metrics import confusion_matrix Confusion_matrix(y_train, y_train_pred) Q10: Why is accuracy not a good measure for classificationproblems? Accuracy is the number of correct predictions out of allpredictions made. Accuracy = (TP+TN)/(The total number of Predictions) Accuracy isnot a good measure for classification problems because it gives equalimportance to both false positives and false negatives. However, this may notbe the case in most business problems. For example, in case of cancerprediction, declaring cancer as benign is more serious than wrongly informingthe patient that he is suffering from cancer. Accuracy gives equal importanceto both cases and cannot differentiate between them. For imbalanced cases like credit card fraud rate is 0.3%,predicting no fraud will give a high accuracy, but the prediction is useless. Q11: What is TPR (true positive rate), and what is FPR (falsepositive rate) TPR = TP/TP+FN, Sensitivity,or True Positive Rate, also known as recall, 1-type II error, power, is theproportion of correctly identified positive out of all actual positives. type IIerror, fails to reject the null hypothesis when in fact the alternativehypothesis is true. FPR =FP/TN+FP Falsepositive rate, type I error, P value, 1- specificity, FPR=false positives/total number of actual negative events. False positive rate is the probabilityof falsely rejecting the null hypothesis for a particular test. Precision:true positive/all predicted as positive. Q12: Explainwhat precision and recall are. How do they relate to the ROC curve? Precision:is the proportion of true positive out of all positive identifications (precision= TP / (TP +FP)) Recall: isthe proportion of correctly identified positive out of all actual positives (recall =TP/ (TP + FN)) Precisionand recall: a tug of wall. Improve precision typically reduce recall and viceversa. Increaseclassification threshold, the number of false positive decreases, but falsenegative increases, as a result, precision increased, while recall decreases. Decreaseclassification threshold, false positives increase, and false negativesdecrease, as a result precision decreases and recall increases. Python code: From sklearn.metrics importprecision_score, recall_score Precision_score(y_train,y_train_pred) recall_score(y_train,y_train_pred) Q13: Explainthe use of ROC curves and the AUC of an ROC Curve. An ROC (Receiver OperatingCharacteristic) curve illustrates the performance of a binary classificationmodel. (The curves of differentmodels can be compared directly in general or for different thresholds.) It is basically a TPR versus FPR (true positive rate versusfalse positive rate) curve for all the threshold values ranging from 0 to 1. Inan ROC curve, each point in the ROC space will be associated with a differentconfusion matrix. A diagonal line from the bottom-left to the top-right on theROC graph represents random guessing. The Area Under the Curve (AUC) signifieshow good the classifier model is. If the value for AUC is high (near 1), thenthe model is working satisfactorily, whereas if the value is low (around 0.5),then the model is not working properly and just guessing randomly. Python code: Import matplotlib.pyplot as plt From sklearn.model_selection import cross_val_predict probs = cross_val_predict(log_reg, X_train,y_train, cv=3, method = “decision_function”) from sklearn.metrics import roc_curve fpr,tpr, thresholds = roc_curve(y_train, probs) def plot_roc_curve(fpr, tpr, label=None): plt.plot(fpr,tpr,linewidth=2, label=label) plt.plot([0,1],[0,1], ‘k--’) plt.axis([0,1,0,1]) plt.xlabel(‘False Positive Rate’) plt.ylabel(‘True Positive Rate’) plot_roc_curve(fpr, tpr) plt.show() from sklearn.metrics import roc_auc_score roc_auc_score(y_train, probs) A complete example of calculating the ROC curve and AUC fora logistic regression model on a small test problem is listed below # roc curve and auc from sklearn.datasets importmake_classification from sklearn.neighbors importKNeighborsClassifier from sklearn.model_selection importtrain_test_split from sklearn.metrics import roc_curve from sklearn.metrics import roc_auc_score from matplotlib import pyplot # generate 2 class dataset X, y = make_classification(n_samples=1000, n_classes=2,weights=[1,1], random_state=1) # split into train/test sets trainX, testX, trainy, testy = train_test_split(X,y, test_size=0.5, random_state=2) # fit a model model = KNeighborsClassifier(n_neighbors=3) model.fit(trainX, trainy) # predict probabilities probs = model.predict_proba(testX) # keep probabilities for the positive outcomeonly probs = probs[:, 1] # calculate AUC auc = roc_auc_score(testy, probs) print('AUC: %.3f' % auc) # calculate roc curve fpr, tpr, thresholds = roc_curve(testy, probs) # plot no skill pyplot.plot([0, 1], [0, 1], linestyle='--') # plot the roc curve for the model pyplot.plot(fpr, tpr, marker='.') # show the plot pyplot.show() Q14: Reviewingboth precision and recall is useful in cases where there is an imbalance in theobservations between the two classes. Specifically, there are many examples ofno event (class 0) and only a few Python code: # precision-recall curve and f1 from sklearn.datasets importmake_classification from sklearn.neighbors importKNeighborsClassifier from sklearn.model_selection importtrain_test_split from sklearn.metrics importprecision_recall_curve from sklearn.metrics import f1_score from sklearn.metrics import auc from sklearn.metrics importaverage_precision_score from matplotlib import pyplot # generate 2 class dataset X, y = make_classification(n_samples=1000, n_classes=2,weights=[1,1], random_state=1) # split into train/test sets trainX, testX, trainy, testy = train_test_split(X,y, test_size=0.5, random_state=2) # fit a model model = KNeighborsClassifier(n_neighbors=3) model.fit(trainX, trainy) # predict probabilities probs = model.predict_proba(testX) # keep probabilities for the positive outcomeonly probs = probs[:, 1] # predict class values yhat = model.predict(testX) # calculate precision-recall curve precision, recall, thresholds = precision_recall_curve(testy,probs) # calculate F1 score f1 = f1_score(testy, yhat) # calculate precision-recall AUC auc = auc(recall, precision) # calculate average precision score ap = average_precision_score(testy, probs) print('f1=%.3f auc=%.3f ap=%.3f' % (f1, auc, ap)) # plot no skill pyplot.plot([0, 1], [0.5, 0.5], linestyle='--') # plot the roc curve for the model pyplot.plot(recall, precision, marker='.') # show the plot pyplot.show()
Q15: Whatis F-measure? It is the harmonic mean of precision andrecall. In some cases, there will be a trade-off between the precision and therecall. In such cases, the F-measure will drop. It will be high when both theprecision and the recall are high. Depending on the business case at hand andthe goal of data analytics, an appropriate metric should be selected. F-measure = 2 X (Precision X Recall) /(Precision+Recall) Python code: From sklearn.metricsimport f1_score f1_score(y_train,y_train_pred)
Q16. Which algorithm is better at handlingoutliers logistic regression or SVM?Logisticregression will find a linear boundary if it exists to accommodate theoutliers. Logistic regression will shift the linear boundary in order toaccommodate the outliers. SVM is insensitive to individual samples. There willnot be a major shift in the linear boundary to accommodate an outlier. SVMcomes with inbuilt complexity controls, which take care of overfitting. This isnot true in case of logistic regression. Q17. How will you deal with the multiclassclassification problem using logistic regression?The mostfamous method of dealing with multiclass classification using logisticregression is using the one-vs-all approach. Under this approach, a number ofmodels are trained, which is equal to the number of classes. The models work ina specific way. For example, the first model classifies the datapoint dependingon whether it belongs to class 1 or some other class; the second modelclassifies the datapoint into class 2 or some other class. This way, each datapoint can be checked over all the classes. Q18. How can you use the concept ofROC in a multiclass classification?The conceptof ROC curves can easily be used for multiclass classification by using theone-vs-all approach. For example, let’s say that we have three classes ‘a’,’b’, and ‘c’. Then, the first class comprises class ‘a’ (true class) and thesecond class comprises both class ‘b’ and class ‘c’ together (false class).Thus, the ROC curve is plotted. Similarly, for all the three classes, we willplot three ROC curves and perform our analysis of AUC.
| 
狗仔卡
收藏
淘帖
好苗
杂草
提升卡
置顶卡
关贴卡
开贴卡
变色卡
千斤顶
