简单python爬虫的source code分享

自己写的一个简单python爬虫，粘贴即用。opttimeline可以来看整体情况。这个程序可以帮你查具体自己周围case的具体情况。

1. from bs4 import BeautifulSoup
   
2. import csv
   
3. import requests
   
4. import time
   
5. 
   
6. candidates = set(['Card Was Delivered To Me By The Post Office', 'New Card Is Being Produced', 'Card Was Mailed To Me', 'Card Was Picked Up By The United States Postal Service', 'Case Was Approved'])
   
7. other = set(['', ' ', 'Case Rejected Because I Sent An Incorrect Fee', 'Fees Were Waived', 'Request for Initial Evidence Was Mailed', 'Fingerprint Fee Was Received'])
   
8. pending ='Case Was Received'
   
9. my_case_num = 1990232000
   
10. url_prefix = 'https://egov.uscis.gov/casestatus/mycasestatus.do?appReceiptNum=YSC'
    
11. count_total = 0
    
12. count_valid = 0
    
13. count_other = 0
    
14. count_pending = 0
    
15. step = 0
    
16. for i in range(my_case_num - 50, my_case_num + 50):
    
17.     url = url_prefix + str(i)
    
18.     response = requests.get(url)
    
19.     bs = BeautifulSoup(response.text, 'html.parser')
    
20.     temp = bs.find(class_='col-lg-12 appointment-sec center')
    
21.     if not temp:
    
22.         print("error")
    
23.         time.sleep(0.2)
    
24.         continue
    
25.     content = temp.find('p').get_text()
    
26.     status = temp.find('h1').get_text()
    
27. 
    
28.     if status in candidates:
    
29.         count_valid += 1
    
30.     elif status == pending:
    
31.         count_pending += 1
    
32.     else:
    
33.         count_other += 1
    
34.     time.sleep(0.2)
    
35.     step += 1
    
36.     print(i, status)
    
37. print('total:', step)
    
38. print('total I-765:', count_total)
    
39. print('finished:', count_valid)
    
40. print('pending:', count_pending)
    
41. print('other:', count_other)

复制代码

祝大家好运！

谢谢分享

感谢! So convenient

之前写了个C#版本的，没爬几次，我的IP就被 USCIS给封了....


IT WAS REPORTED TO US THAT YOUR IP ADDRESS OR INTERNET GATEWAY HAS BEEN LOCKED OUT FOR A SELECT PERIOD OF TIME. THIS IS DUE TO AN UNUSUALLY HIGH RATE OF USE. IN ORDER TO AVOID THIS ISSUE, PLEASE CREATE A CUSTOMER ACCOUNT (SINGLE APPLICANT) OR A REPRESENTATIVE ACCOUNT (REPRESENTING MANY INDIVIDUALS).

多谢！！非常好用！！

q107770540 发表于 2019-7-25 05:02
之前写了个C#版本的，没爬几次，我的IP就被 USCIS给封了....

封了以后隔一天会恢复的。我试过一个ip差不多能很快速度的爬3000个号左右。

怕被封号的话试试 rotating proxy

试了下，很有用，感谢楼主！

友情提示 pip3 install bs4

感谢分享

很有用，如果没看到我就差不多快自己写了
随便你忘了更新i797 那个变量
最后打印出来是没有i797 的数目的