PyTutorial

Web Scraper

#Using BeautifulSoup

An example of how to scrape a website using the requests and bs4 libraries.

python
Open IDE
import requests
from bs4 import BeautifulSoup

URL = "https://realpython.github.io/fake-jobs/"
page = requests.get(URL)

soup = BeautifulSoup(page.content, "html.parser")
results = soup.find(id="ResultsContainer")
print(results.prettify())