专注下来，好好刷题

May 13 (150 min)

Section 2 Using Hadoop's Core: HDFS and MapReduce

Lesson 14,15

Section 3 Programming Hadoop with Pig

Lesson 16, 17, 18, 19, 20

补充内容 (2018-5-14 02:32):
Lesson 21,22

May 12

Section 4 Programming Hadoop with Spark

23. Why Spark

24. The Resilient Distributed Dataset (RDD)

25. Find the movie with the lowest average rating -- with RDD

26. Data-set and Spark 2.0

27. Find the movie with the lowest average rating - with DataFrames

28. Movie recommendations with MLLib

29. Filter the lowest-rated movies by number of ratings

30. Check your results against mine

补充内容 (2018-5-14 21:29):
写错了，是May 14。为什么现在一亩三分地连编辑都不允许了，只能补充啊，太不方便了

从别的楼摸过来：）我感觉大部分phd都是不缺执行力的人 lz一定会有好结果哒

阿钟 发表于 2018-5-15 03:24
从别的楼摸过来：）我感觉大部分phd都是不缺执行力的人 lz一定会有好结果哒

谢谢，你也加油哈

May14

CS 109

Week 1
Lecture 1: Course Overview
HW1

好久没来，竟然没被加入回收站。楼主去做了激光治疗近视，一周多没工作，现在充满了学习的热情（虽然看东西还有点重影。。。

今天计划：
CS 109 完成lab3

data science in business那本书最少看15页

希望眼睛的状态一直良好，加油加油～

补充内容 (2018-5-25 23:48):
完成lab2以及lab3

NewRita 发表于 2018-5-25 07:44
好久没来，竟然没被加入回收站。楼主去做了激光治疗近视，一周多没工作，现在充满了学习的热情（虽然看东西 ...

加油加油！ 楼主又变美又变聪明

vtiaocao 发表于 2018-5-26 04:06
加油加油！ 楼主又变美又变聪明

哈哈，谢谢！你也是，更帅更聪明更有钱

May 31 机器算法基石

# Week 2 Learning to Answer Yes/No

    1. Perceptron Hyothesis Set      
   
        Credit Approval Problem: A Simple Hypothesis Set: "The Perceptron"

        线性分类器
   
    2. Perceptron Learning Algorithm (PLA)
   
        选一条最合适的线性分类器

        want g = f on known dataset

        W(t+1) = W(t) + y*x   
   
        利用向量计算,判断正确的perceptron
        
        如果不会停下来??
        
        在一定情况下,这个算法一定会停下来
   
    3. Guarantee of PLA
   
        If PLA halts, D allows some w to make no mistake; call such D linear seperable
        
        Linearly seperable

June 1 2018

    4. Non-seperable Data
   
        PLA: simple to implement, fast, works in any dimension
        
        Assumes linear seperable D to halt; not fully sure how long halting takes (不知道是否线性可分:可能不会停下; 不知道什么时候才会停下)
        
        Learning with Noisy Data : Line with Noise Tolerance
        
        Pocket Algorithm: Modify PLA by keeping best weights in pocket
        
        Since we do no know whether D is linear seperable in advance, we may decide to just go with pocket instead of PLA.
        
        When D is linear seperable:
        
            pocket on D is slower than PLA for two reasons:
            
                1. pocket要花时间存起来weights
               
                2. 对于每一条线,pocket需要检查所有的点的误差                     

Week 3 Types of Learning

    1. Learning with different Output Space
   
        是非题（二元分类）：
              
            1 binary classification
            2 multiclass classification
            3 regression
            4 structured learning

    2. Learning with different data label
   
        Supervised: Coin Recognition Revisited
        
        Unsupervised: Coin Recognition without yn        

        unsupervised learning: diverse, with possibly very different performance goals
        
        Semi-supervised: Coin Recognition with Some y n
        
        Other Semi-supervised Learning Problems
            • face images with a few labeled ⇒ face identifier (Facebook)
            • medicine data with a few labeled ⇒ medicine effect predictor
            
        semi-supervised learning: leverage unlabeled data to avoid ‘expensive’ labeling
        
        Reinforcement Learning
        
            Other Reinforcement Learning Problems Using (x, ỹ , goodness)
                • (customer, ad choice, ad click earning) ⇒ ad system
                • (cards, strategy, winning amount) ⇒ black jack agent
                                
        Learning with Different Data Label y n
            • supervised: all y n
            • unsupervised: no y n
            • semi-supervised: some y n
            • reinforcement: implicit y n by goodness( ỹ n )
            • . . . and more!!
   
    3. learning with different protocol
   
        Batch Learning: Coin Recognition Revisited
        
        batch supervised multiclass classification: learn from all known data
        
            • batch of (email, spam?) ⇒ spam filter
            • batch of (patient, cancer) ⇒ cancer classifier
            • batch of patient data ⇒ group of patients
        
        batch learning: a very common protocol
        
        • batch spam filter: learn with known (email, spam?) pairs, and predict with fixed g
        • online spam filter, which sequentially:        
            1 observe an email x t
            2 predict spam status with current g t (x t )
            3 receive ‘desired label’ y t from user, and then update g t with (x t , y t )
            
        Learning with Different Protocol f ⇒ (x n , y n )
            • batch: all known data unknown target function
            • online: sequential (passive) data
            • active: strategically-observed data
            • . . . and more!!
            
    4. learning with different input space
   
        More on Concrete Features
        
            • (size, mass) for coin classification
            • customer info for credit approval
            • patient info for cancer diagnosis
            • often including ‘human intelligence’ on the learning task
        
        concrete features: the ‘easy’ ones for ML
        
        Raw Features: Digit Recognition Problem
        
            • image pixels, speech signal, etc.
        
        raw features: often need human or machines to convert to concrete ones
        
        Abstract Features: Rating Prediction Problem
        
        Rating Prediction Problem (KDDCup 2011)
            • given previous (userid, itemid, rating) tuples, predict the rating that some userid would give to itemid?
            • a regression problem with Y ⊆ R as rating and X ⊆ N × N as (userid, itemid)
            • ‘no physical meaning’; thus even more difficult for ML Other Problems with Abstract Features
            • student ID in online tutoring system (KDDCup 2010)
            • advertisement ID in online ad system