注册一亩三分地论坛,查看更多干货!
您需要 登录 才可以下载或查看附件。没有帐号?注册账号 
x
人肉整理的 求加大米!. 1point 3 acres
Feature Engineering
Categorical Encoding:
. 1point 3acres
. check 1point3acres for more.
count encoding: .google и
use count of the categorical variable to encode the variable
target encoding: .--
For example, given the country value "CA", you'd calculate the average outcome for all the rows with country == 'CA', around 0.28.. 1point 3acres
This technique uses the targets to create new features. So including the validation or test data in the target encodings would be a form of target leakage. Instead, you should learn the target encodings from the training dataset only and apply it to the other datasets.
The category_encoders package provides TargetEncoder for target encoding. The implementation is similar to CountEncoder.
CatBoost encoding
Finally, we'll look at CatBoost encoding. This is similar to target encoding in that it's based on the target probablity for a given value. However with CatBoost, for each row, the target probability is calculated only from the rows before it.
. ----
Feature Generation
Interaction feature -> label encode
• if one record has the country "CA" and category "Music", you can create a new value "CA_Music"
This is a new categorical feature that can provide information about correlations between categorical variables.
.1point3acres
• label encode the interaction feature and add it to our data..
Rolling 7 day count
. check 1point3acres for more.
rolling to select time periods as the window. For example launched.rolling('7d')
Time since the last project in the same category
. 1point3acresdef time_since_last_project(series):. 1point3acres
# Return the time in hours
return series.diff().dt.total_seconds() / 3600.
df = ks[['category', 'launched']].sort_values('launched')
timedeltas = df.groupby('category').transform(time_since_last_project)-baidu 1point3acres
timedeltas.head(20). ----
Transforming numerical features
Eg. Long tail. Some models work better when the features are normally distributed
Common choices for this are the square root and natural logarithm -> help constrain outliers.
plt.hist(np.sqrt(ks.goal), range=(0, 400), bins=50);
plt.hist(np.log(ks.goal), range=(0, 25), bins=50);
The log transformation won't help our model since tree-based models are scale invariant. However, this should help if we had a linear model or neural network.
.
-baidu 1point3acres
Feature Selection
Univariate Feature Selection: consider only one feature at a time when making a selection decision.
For each feature, measure how strongly the target depends on the feature using a statistical test like χ2 or ANOVA.
From the scikit-learn feature selection module, feature_selection.SelectKBest returns the K best features given some scoring function. For our classification problem, the module provides three different scoring functions: χ2 , ANOVA F-value, and the mutual information score. The F-value measures the linear dependency between the feature variable and the target. This means the score might underestimate the relation between a feature and the target if the relationship is nonlinear. The mutual information score is nonparametric and so can capture nonlinear relationships.
select features using only a training set
L1 regularization
perform feature selection by adjusting the regularization parameter.
. Waral dи,
, feature selection with L1 regularization is more powerful the univariate tests, but it can also be very slow when you have a lot of data and a lot of features
Intro to ML
Underfitting vs Overfitting:. .и
Decision Tree:
max_leaf_nodes argument provides a very sensible way to control overfitting vs underfitting
. Χ
Missing value impute:. 1point3acres
1) A Simple Option: Drop Columns with Missing Values¶
. ----2) A Better Option: Imputation: SimpleImputer to replace missing values with the mean value along each column. From 1point 3acres bbs
3. Mean Imputation+missing flag
categorical variable:
1. Drop
2) Label Encoding:
Label encoding assigns each unique value to a different integer.
.1point3acres
This approach assumes an ordering of the categories: "Never" (0) < "Rarely" (1) < "Most days" (2) < "Every day" (3).
This assumption makes sense in this example, because there is an indisputable ranking to the categories. Not all categorical variables have a clear ordering in the values, but we refer to those that do as ordinal variables. For tree-based models (like decision trees and random forests), you can expect label encoding to work well with ordinal variables.
3) One-Hot Encoding: does not assume an ordering of the categories
We refer to categorical variables without an intrinsic ranking as nominal variables.
OneHotEncoder class. 1point3acres
. 1point3acres.com
One-hot encoding generally does not perform well if the categorical variable takes on a large number of values (i.e., you generally won't use it for variables taking more than 15 different values). ..
.1point3acres
Pipeline class
to define a pipeline that bundles the preprocessing and modeling steps
CV:
For small datasets, where extra computational burden isn't a big deal, you should run cross-validation.
For larger datasets, a single validation set is sufficient. Your code will run faster, and you may have enough data that there's little need to re-use some of it for holdout.
XG Boost
goes through cycles to iteratively add models into an ensemble.
1. initializing the ensemble with a single model
2. Start cycle: use the current ensemble to generate predictions for each observation in the dataset.
3. These predictions are used to calculate a loss function
4. use the loss function to fit a new model that will be added to the ensemble
Parameter tuning:. 1point3acres.com
• n_estimators: how many times to go through the modeling cycle
• early_stopping_rounds: offers a way to automatically find the ideal value for n_estimators
• learning_rate: Instead of getting predictions by simply adding up the predictions from each component model, we can multiply the predictions from each model by a small number (known as the learning rate) before adding them in
• n_jobs: equal to the number of cores on your machine
.1point3acres
Data leakage:
Good performance on training and test set, but poor in production. Χ
1. Target Leakage:
any variable updated (or created) after the target value is realized should be excluded.. ----
.google и2. Train-Test Contamination
Eg. For example, imagine you run preprocessing (like fitting an imputer for missing values) before calling train_test_split().
if your validation is based on a simple train-test split, exclude the validation data from any type of fitting, including the fitting of preprocessing steps..
Data Cleaning
Missing Value
. ----
. .и
!!! Is this value missing because it wasn't recorded or because it doesn't exist? -- Data intuition
.--
.--
Approach:
• Drop
. 1point3acres
• Fill
This makes a lot of sense for datasets where the observations have some sort of logical order to them.. .и
. 1point3acres.com
Scaling vs. Normalization
• in scaling, you're changing the range of your data, while
• in normalization, you're changing the shape of the distribution of your data
Scaling
. .и
transforming your data so that it fits within a specific scale, like 0-100 or 0-1.
You want to scale data when you're using methods based on measures of how far apart data points are, like support vector machines (SVM) or k-nearest neighbors (KNN). With these algorithms, a change of "1" in any numeric feature is given the same importance.
Normalization
. Χ
Normal distribution: Also known as the "bell curve", this is a specific statistical distribution where a roughly equal observations fall above and below the mean, the mean and the median are the same, and there are more observations closer to the mean. The normal distribution is also known as the Gaussian distribution.
. 1point 3 acres
. Χ
Parsing Dates
. 1point 3 acres
1/1/2019: Python doesn’t know it's date, O means Object
.
Convert object to datetime (Parsing dates). 1point3acres
For 1/1/2019
Now, type is datetime64
. Χ
multiple date formats:
landslides['date_parsed'] = pd.to_datetime(landslides['Date'], infer_datetime_format=True)
Select the day of the month:
-baidu 1point3acres
day_of_month_landslides = landslides['date_parsed'].dt.day
Convert to datetime then dt.day
Inconsistent Data Entry
The first thing I'm going to do is make everything lower case (I can change it back at the end if I like) and remove any white spaces at the beginning and end of cells
Fuzzy matching logic
|