Web Scraping using Python

What is web scraping?

Web scraping is the process of using bots to extract content and data from a website. Using web scarping we can store unstructured website data to structured website data.

Scrap data from website using python.

To scrap data from website using python we will use three libraries.

requests : The request library allow to send HTTP request using python. Using this library you can send request to URL for use that data.

BeautifulSoup : BeautifulSoup collect data from HTML and XML documents.

Pandas : Pandas is Python package that is for data analysis and manipulation. It will collect data and store that in proper format.

Steps to scrape data from websites

1. Find the URL that you want to scrape

In this blog we are going to scrape Flipkart website for product info.

https://www.flipkart.com/search?q=phone&otracker=search&otracker1=search&marketplace=FLIPKART&as-show=on&as=off

We will scrap data from above URL.

2. Inspecting the Page and find data that you want

Right click on website and click on inspect

find tag of data that you want to scrape

3. Write the code

import libraries

importlibraries.py

Store webpage content using requests and BeautifulSoup.

storecontents.py

Separate specific data from content using respective class-names and respective tags. and store that data into list.

sepratedata.py

We can print or store data for further use

storecontents.py

output data

For whole code, check my GitHub.

Thank You.