小白转DS，面试常见问题总结，新人求大米

马上要到找工作季了，想要转ds，吐血总结一些面试中会问道的问题。新人上路，求大米~
1. 1,P-value: the p-value is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is correct. A very small p-value means that such an extreme observed outcome would be very unlikely under the null hypothesis. (The p value is the evidence against a null hypothesis. The smaller the p-value, the stronger the evidence that you should reject the null hypothesis. P values are expressed as decimals although it may be easier to understand what they are if you convert them to a percentage. For example, a p value of 0.0254 is 2.54%.) .
A p-value is the probability that the results from your sample data occurred by chance. P-values are from 0% to 100%. They are usually written as a decimal. For example, a p value of 5% is 0.05. Low p-values are good; They indicate your data did not occur by chance

2. normal distribution: About 68% of values drawn from a normal distribution are within one standard deviation σ away from the mean; about 95% of the values lie within two standard deviations; and about 99.7% are within three standard deviations.

3. VIF: A variance inflation factor(VIF) detects multicollinearity in regression analysis. Multicollinearity is when there’s correlation between predictors (i.e. independent variables) in a model; it’s presence can adversely affect your regression results. The VIF estimates how much the variance of a regression coefficient is inflated due to multicollinearity in the model.

4.  R-squared is a statistical measure of how close the data are to the fitted regression line. It is also known as the coefficient of determination, or the coefficient of multiple determination for multiple regression.
The definition of R-squared is fairly straight-forward; it is the percentage of the response variable variation that is explained by a linear model. Or:
R-squared = Explained variation / Total variation
The adjusted R-squared compares the descriptive power of regression models that include diverse numbers of predictors. Every predictor added to a model increases R-squared and never decreases it. Thus, a model with more terms may seem to have a better fit just for the fact that it has more terms, while the adjusted R-squared compensates for the addition of variables and only increases if the new term enhances the model above what would be obtained by probability and decreases when a predictor enhances the model less than what is predicted by chance.

5. batch size is the data size used in the stochastic gradient descent process used in training the neural network.

6. Dropout
A single model can be used to simulate having a large number of different network architectures by randomly dropping out nodes during training. This is called dropout and offers a very computationally cheap and remarkably effective regularization method to reduce overfitting and improve generalization error in deep neural networks of all kinds.

7. AUC (area under the curve) is a matric used to measure the accuracy of classification model. This curve plots two parameters:
•        True Positive Rate
•        False Positive Rate

如果大家觉得有用的话，会继续更新一些自己整理的常考知识点~真诚求米，感激~

感谢大米，再来更新下：
7,
Hyperparameter is usually tuned by cross-validation method.

8, kernel function
I think Kernel function is a dot product of two vectors that represent map to other higher dimensional spaces. Using it saves the effort to transform the data to a higher-dimensional space and transform it back. It is equivalent to that process.
Kernel is a way of computing the dot product of two vectors xx and yy in some (possibly very high dimensional) feature space. Kernels give a way to compute dot products in some feature space without even knowing what this space is and what is φ.
https://stats.stackexchange.com/ ... in-what-a-kernel-is

9，
Precision and recall: all care about the true positive.
Accuracy care all the predictions

What is the confusion matrix?
a table of predicted true, predicted false vs actual true,  actual false.

F1 Score is the Harmonic Mean between precision and recall
ROC Curve
The receiver operating characteristic curve is the plot that shows the performance of a binary classifier as a function of its cut-off threshold
AUC (area under the curve) is an aggregated measure of the performance of a binary classifier on all possible threshold values.

9, Truist 公司面试问过的：Data types
(1), Categorical, two subtypes: Nominal, Ordinal
(2) Numerical, two subtypes: Interval, Ratio

Nominal, Ordinal, Interval, and Ratio are defined as the four fundamental levels of measurement scales that are used to capture data in the form of surveys and questionnaires, each being a multiple choice question
Nominal values represent discrete units and are used to label variables, that have no quantitative value. Just think of them as „labels“. Note that nominal data that has no order. Therefore if you would change the order of its values, the meaning would not change
Ordinal values represent discrete and ordered units. It is therefore nearly the same as nominal data, except that it’s ordering matters. You can see an example below:

Interval values represent ordered units that have the same difference. interval values data is that they don’t have a „true zero. With interval data, we can add and subtract, but we cannot multiply, divide or calculate ratios. An example would be a feature that contains temperature of a given place
Ratio values are also ordered units that have the same difference. Ratio values are the same as interval values, with the difference that they do have an absolute zero. Good examples are height, weight, length

Nominal scale is a naming scale, where variables are simply “named” or labeled, with no specific order. Ordinal scale has all its variables in a specific order, beyond just naming them. Interval scale offers labels, order, as well as, a specific interval between each of its variable options.  Ratio scale bears all the characteristics of an interval scale, in addition to that, it can also accommodate the value of “zero” on any of its variables.


10,
Selection bias is a phenomenon that involves the selection of individual or grouped data in a way that is not considered to be random

11,
Type one error: rejecting a true null hypothesis
Type two error : accepting a false null hypothesis


12， what is a T test?
A t-test is a type of inferential statistic used to determine if there is a significant difference between the means of two groups, which may be related in certain features.
The t score is a ratio between the difference between two groups and the difference within the groups. The larger the t score, the more difference there is between groups.
https://en.wikipedia.org/wiki/Student%27s_t-test
An Independent two Sample t-test compares the means for two groups.
Python code: scipy.stats.ttest_ind
https://docs.scipy.org/doc/scipy ... tats.ttest_ind.html
https://www.reneshbedre.com/blog/ttest.html
A Paired sample t-test compares means from the same group at different times (say, one year apart).
A One sample t-test tests the mean of a single group against a known mean. is used to compare the sample mean (a random sample from a population) with the specific value (hypothesized or known mean of the population).

Python code: scipy.stats.ttest_1samp
https://docs.scipy.org/doc/scipy ... ts.ttest_1samp.html

感觉ds是个很迷的工种 写代码就是sde 做分析就是statistician/da 面向用户就是pm

补充内容 (2022-12-20 00:37 +08:00):
然后考得也是大杂烩, 不过感觉第一学期写完就很偏统计