在职个人战拖打卡DataCamp

06/07 打卡完成Supervised Learning with Scikit Learn～
1）preprocessing data
categorical variables: OneHotCoding / get_dummies
missing data:
• replace with 0: .replace(replacing value, value to be replaced, inplace = True)
• impute with an educated guess: Imputer(missing_values = '', strategy = '', axis=0)
centering and scaling:
• scale
• StandardScaler

06/08 完成PROJECT: PREDICTING CREDIT CARD APPROVALS～
1）replace missing values:
my_data = my_data.replace(replacing_value, np.NaN)

2) imputing missing values:
mode: df[col].mode()[0]

3) convert non-numerical to numerical data (We do this because not only it results in a faster computation but also many machine learning models (like XGBoost) (and especially the ones developed using scikit-learn) require the data to be in a strictly numeric format.)
from sklearn.preprocessing import LabelEncoder
https://scikit-learn.org/stable/ ... g.LabelEncoder.html
instantiate --> fit_transform on the columns

4) train test split
convert the DataFrame to a NumPy array using .values

5) scale the feature values to a uniform range
# Import MinMaxScaler
from sklearn.preprocessing import MinMaxScaler
# Instantiate MinMaxScaler and use it to rescale X_train and X_test
scaler = MinMaxScaler(feature_range=(0, 1))
rescaledX_train = scaler.fit_transform(X_train)

6) fit model

7) evaluate the performance
• confusion matrix: from sklearn.metrics import confusion_matrix; confusion_matrix(y_test, y_pred)
• accuracy score: logreg.score(rescaledX_test, y_test)

8) perform a grid search of model parameters to improve model
• define the grid of values for each parameter: []
• create a dictionary of parameter grid
d = dict([
    (<key>, <value>),
    (<key>, <value),
      .
      .
      .
    (<key>, <value>)
])
• fit grid model with rescaledX, y
# Instantiate GridSearchCV with the required parameters
grid_model = GridSearchCV(estimator=logreg, param_grid=param_grid, cv=5)

# Use scaler to rescale X and assign it to rescaledX
rescaledX = scaler.fit_transform(X)

# Fit data to grid_model
grid_model_result = grid_model.fit(rescaledX, y)

# Summarize results
best_score, best_params = grid_model_result.best_score_, grid_model_result.best_params_
print("Best: %f using %s" % (best_score, best_params))
• Output: "Best: 0.852174 using {'max_iter': 100, 'tol': 0.01}"

06/09 打卡Unsupervised Learning in Python前两章～
Unsupervised learning的用处
• Find patterns (w/o a specific predication task in mind)
• Reduce dimensions

Measure clustering performance:
• compare predicted labels and real labels via crosstab:
pd.crosstab(df['labels'], df['species'])
df = pd.DataFrame({'labels': labels, 'species': species})
fit_predict: fit model and obtain cluster labels, labels = model.fit_predict(samples)
• inertia: measures how spread out the clusters are
after fit(), model.inertia_; choose an elbow in the inertia plot
plt.plot(ks, inertias, '-o') --> plot the inertia with '-o' marker

Improve model parameters
• StandardScaler: in K Means model, feature variance = feature influence
• make_pipeline(): directly pass in the instantiated objects, e.g. pipeline = make_pipeline(scaler, kmeans)

Hierarchy Clustering
• Scipy 无法fit into sklearn pipelines, 所以使用normalize而非Normalizer进行standardize
• 使用linkage来创造clusters：
methods：1）complete - max distance; 2）single - min distance
• 使用dendrogram来绘制clusters
• Extract labels: fcluster(linkage, max_height, criterion = )

t-SNE
• from sklearn.manifold import TSNE
• instantiate TSNE with learning_rate
• fit_transform, xs = [:,0], ys = [:,1]
• visualize

06/10 打卡完成Unsupervised Learning～

PCA
• To identify intrinsic dimensions and reduce dimensions: intrinsic dimension = number of PCA features with significant variance
• instantiate --> fit --> transform
• PCA(n_components = 2)
• principal components: model.components_; first principal component: model.components_[0,:]
• mean of the sample: model.mean_
• extract the number of features: range(model.n_components_)
• variance: model.explained_variance_
• xs = transformed[:,0], ys = transformed[:,1]

TruncatedSVD
• equivalent of PCA for text documents
• use csr_matrix instead of numpy array

Non-negative Matrix Factorization
• more interpretable than PCA
• all sample features must be non-negative
• sample reconstruction: Multiply components by feature values, and add up

Recommender System
1. word frequency array
2. apply NMF
3. calculate cosine similarity
from sklearn.preprocessing import normalize
(normalized features.dot(current article))

Other basic stuff
• slicing: cannot apply iloc or loc to numpy arrays
• showing bitmap:
def show_as_image(sample):
    bitmap = sample.reshape((13, 8))
    plt.figure()
    plt.imshow(bitmap, cmap='gray', interpolation='nearest')
    plt.colorbar()
    plt.show()

06/13 打卡Machine Learning with Tree-Based Models in Python前两章～
Classification Trees
• Information Gain (Feature, Split Point)  = Information (Parent) - % of samples on the left * Information (left) - % of samples on the right * Information (right)
• Maximize IG at each internal node, and if IG = 0 then make it a leaf
• Preprocessing, train_test_split --> Instantiate (specify criterion, max_depth, random_state) --> Fit --> Predict --> Accuracy Score

Regression Trees
• Impurity (node) = MSE (node)

Generalization Error
• error terms ^ 2 + variance from different training sets + irreducible error
• tradeoff between low bias (accuracy) and low variance (precision)
• estimate generalization error: use the test set error; use cross validations --> K-fold CV: error = avg(CV error)
• high variance: overfitting, CV error > training set error --> reduce model complexity
• high bias: underfitting, CV error ~= training set error >> desired error --> increase model complexity, gather more relevant features
# Compute the array containing the 10-folds CV MSEs
MSE_CV_scores = - cross_val_score(dt, X_train, y_train, cv=10,
                       scoring='neg_mean_squared_error',
                       n_jobs=-1)

06/14 打卡完成了Machine Learning with Tree-Based Models in Python～
Bagging
• Training models with the same algorithm, based on different training subsets

OOB
• Train models on bootstrap samples, and evaluate based on OOB samples
• set oob_score = True in BaggingClassifier

Random Forest
• Different from bagging: use only trees as base estimators; more randomization for the # of features used
• Feature importance: the weight of the feature used in training and prediction
• Visualization of feature importance: horizontal barplot of the feature importance

AdaBoost
• Combine weak learners in ensemble learning
• Each predictor pays more attention to wrongly predicted points from its predecessor predictions (by adjusting alpha)
• Tradeoff between learning rate and number of predictors
• Instantiate base estimator --> Instantiate AdaBooster --> fit, predict_proba, evaluate with auc scores

Gradient Boosting
• Use CART as estimators, does not tweak weights of training instances
• Starting from predictor 2, use residual errors from predecessor predictors as labels (instead of dataset labels y)
• Instantiate GradientBoostingRegressor --> fit, predict, evaluate with RMSE

Stochastic Boosting
• To avoid the same split points and features being used in individual CARTs for Gradient Boosting
• Specify subsample (% of training data) and max_features (% of features) used to find the best split

Hyperparameter Tuning
• Scores: accuracy score (classifiers) & R^2 (regressors)
• Approaches: GridSearchCV, RandomSearch, Baysian Optimization, Genetic Algorithms

06/15 完成Case Study - School Budgeting前两章～
Workflow
Identify problem --> EDA --> Data Types --> Measure Success --> Create a simple model to set the baseline (numerical data only/limited features, get to predictions quickly) --> sample dataset --> instantiate, train --> predict to get the baseline performance --> Predict with holdout dataset

Thinking Process/Goals
• The probability of this label being xx is ??, and if it is not xx, then it's most likely yy.
• Predictions will be probabilities for each label

EDA
• Plot historgram: drop null values

Predicting with holdout datasets
• pd.DataFrame(columns = pd.get_dummies(label_columns, prefix_sep='_').columns, index = holdout.index, data = predicted_data)

06/16 完成Case Study - School Budgeting
Improving the model
Pipeline pre-processing: FunctionTransformer, FeatureUnion; separate pipelines for numeric/text data and then union the features

Tricks
• Text processing: stop words, bi-grams
• Statistical methods
• Computational efficiency

06/17 完成Cluster Analysis in Python前两章～
说实在话和第20节 Unsupervised Learning in Python内容很重复。。。就当复习啦！倒数一天w