Beautiful Soup with ProxyPanel

Beautiful Soup with ProxyPanel

Beautiful Soup is used for parsing HTML. Combine it with requests and ProxyPanel for web scraping.

Installation

pip install beautifulsoup4 requests[socks]

Example

import requests
from bs4 import BeautifulSoup

proxies = {
    "http": "socks5h://api-key:api-key@proxy-host:socks5-port",
    "https": "socks5h://api-key:api-key@proxy-host:socks5-port",
}

response = requests.get("https://example.com", proxies=proxies)
soup = BeautifulSoup(response.text, "html.parser")

print(soup.title.string)

Beautiful Soup doesn't handle HTTP requests itself — it parses the HTML returned by requests (or another HTTP client).