Preface

For one net The development of this crawler has never been written before . This is the time to start learning python Reptiles , This weekend, I wrote a piece of code to crawl the pictures of renting houses in Shanghai , In fact, it's very brief to use the third-party library of crawler Requests And BeautifulSoup.python edition :python3.6 ,IDE :pycharm. Just a few lines of code , But I hope that people who have no development foundation can understand it all at once , So God, please detour .

Third party Library

First installation

I am using pycharm So I'm not going to introduce the script installation for you .

 

As shown in the figure above, open the default setting selection Project Interprecter, double-click pip Or click the plus sign , Search for third party libraries to install . If you build more projects, remember Project Interprecter Select the correct installation location, otherwise it cannot be imported .

Requests library

requests The official definition of Library :Requests The only non GMO Python HTTP library , People can enjoy it safely . In fact, he is asking the Internet to get web data .

 

give the result as follows :

 

among Request Headers The parameters of are as follows :

 

 

BeautifulSoup library

BeautifulSoup It's easy to parse Requests The page requested by the library , And the page source code is parsed as Soup file , While filtering and extracting data . This is a bs4.2 Documents .

Beautiful Soup Support Python In the standard library HTML Parser , It also supports some third-party parsers , If we don't install it , be Python Will use Python Default parser , among lxml It's said to be relatively strong, and my next hint is python Standard library .

 

Selectors select

 

BeautifulSoup Knowledge points that the library needs to learn

 

Case study : Take a picture of renting in Shanghai

 

It's just a function , As for the code, the result is as follows :

 

python Get more articles about the pictures of a station renting in Shanghai

  1. python Introduction to reptiles ---- Take a picture of a station renting in Shanghai

    Preface For one net The development of this crawler has never been written before . This is the time to start learning python Reptiles , This weekend, I wrote a piece of code to crawl the pictures of renting houses in Shanghai , In fact, it's very brief to use the third-party library of crawler Requests And BeautifulSou ...

  2. Mengxin learning Python Crawling B Stand in the barrage +R Language participle demo explain

    The code address is as follows :http://www.demodashi.com/demo/11578.html One . Write it at the front I saw it on the front page of the book Python Introduction to reptiles , So I wanted to crawl B Stand on the barrage and draw the word cloud , So there's this ...

  3. use Python Crawling B standing 、 Tencent video 、 Iqiyi and mango TV Video barrage !

    as everyone knows , bullet chat , Comment subtitles pop up when watching videos on the Internet . I don't know if you'll click the bullet screen when you watch the video , In my case , Bullet screen is a good supplement to video content , It's a well-organized review sequence . By analyzing the barrage , We can get a quick insight into the audience's view of the video ...

  4. Use python Crawling P Station pictures

    At the beginning of school, there was a time when weekends were free , So often in P The special edition of the station contains pictures , however P The speed of loading pictures is very moving , I feel like I'm a computer major , How can you make a picture and slowly go down like this low What happened , And it's too inefficient , So I wanted to write a program to help me ...

  5. python Crawling a news station , And analysis of recent news keywords

    When crawling a station and doing a simple analysis , I would like to share with you the following questions , Avoid making mistakes : One, the website path by /info/1013/13930.htm , among 13930 For different news ID value , But this number is in ascending order , But no one ...

  6. Python Climb the fried egg net girl picture

    #!/usr/bin/env python # -*- coding: utf-8 -*- # @Date : 2017-08-24 10:17:28 # @Author : EnderZhou (z ...

  7. use python Crawling B The number of online users of the station

    I'm learning by myself recently Python Reptiles , So I want to practice , use python To crawl B The number of people online , It should be able to take a little analysis Design thinking First look at the web source code , Find the appropriate html, And then use all kinds of tools (BeautifulSoup or ...

  8. Python Crawling B Stop the rat tail juice 、 Not talking about the source of martial arts video barrage

    The text and pictures of this article come from the Internet , Just for learning . Communication use , Not for any commercial purpose , If you have any questions, please contact us in time for handling . Preface What's the meaning of a rat feeding juice ? Maybe a lot of people don't know , This stem is from Ma Baoguo , People who often surf the Internet may have heard of this ...

  9. Python Crawling B Station video information

    The content of this article is invalid , Now it's realized scrapy+scrapy-splash To crawl the website video and user information , because B The anti climbing seal of the station IP, And free agents on the Internet IP Most of them fail , It's impossible to achieve a reliable IP Agent pool , Free agent website is a variety of ...

Random recommendation

  1. iOS Development —— advanced technology & Address book service

    Address book service AddressBook iOS With a individual Contacts Applications to manage contacts , But sometimes we want our application to be able to access or modify this information , This is where it comes in AddressBook.frame ...

  2. Eclipse Report errors :java.lang.ClassNotFoundException: ContextLoaderListener

    Eclipse in tomcat An error is reported after the deployment project starts : serious : Error configuring application listener of class org.springframework.web ...

  3. Android IOS WebRTC Summary of audio and video development ( 32 )-- WebRTC Project development proposals

    This paper mainly introduces WEBRTC Some phenomena in the development process , The article comes from the blog Garden RTC.Blacker, Support the original , Welcome to WeChat official account. blacker, More on www.rtc.help With the rapid development of mobile Internet and intelligent hardware , The sound ...

  4. PAT (Basic Level) Practise ( chinese )1027. Print hourglass (20)

    1027. Print hourglass (20) The time limit 200 ms Memory limit 32000 kB Code length limit 8000 B The procedure of judging questions Standard author CHEN, Yue This problem requires you to write a program to print the given symbol into ...

  5. recvmsg and sendmsg function

    These two functions are the most common I/O function . In fact, we put all read.readv.recv and recvfrom Replace the call with recvmsg call . Similarly , Various output function calls can also be replaced by sendmsg call #include ...

  6. Share a crawl HUST( Harbin Institute of Technology ) The students' grades are Python Program (OCR Automatic identification verification code )

    Python edition :3.5.2 date :2018/1/21 __Author__ = "Lance#" # -*- coding = utf-8 -*- from urllib imp ...

  7. pycharm How to display the toolbar

    1. The effect without a toolbar is as follows : 2. stay view Find Toolbar Tick to show : 3. The toolbar is set successfully, and the display effect is as follows : 3. How to display the file where a class or method is located , And all the methods under this file , You can quickly navigate to the line

  8. Hive & SparkSQL Compare

    Hive stay   Hadoop All data on the cluster is accessed through Java Compiling MapReduce Homework , These let Java There's no problem with programmers doing it . But yes. SQL For the programmer , Write MapRedu ...

  9. Angular 2 Medium ViewChild and ViewChildren

    https://segmentfault.com/a/1190000008695459 ViewChild ViewChild It's a property decorator , Used to get matching elements from the template view . View query in ngAfter ...

  10. springcloud( 3、 ... and )-Eureka

    Eureka yes Netflix An open source product that provides services for registration and discovery , It provides the complete Service Registry and Service Discovery Realization . It's also springcloud The most important and core group in the system ...