SIGN IN SIGN UP

community[minor]: add user agent for web scraping loaders (#22480)

**Description:** This PR adds a `USER_AGENT` env variable that is to be
used for web scraping. It creates a util to get that user agent and uses
it in the classes used for scraping in [this piece of
doc](https://python.langchain.com/v0.1/docs/use_cases/web_scraping/).
Identifying your scraper is considered a good politeness practice, this
PR aims at easing it.
**Issue:** `None`
**Dependencies:** `None`
**Twitter handle:** `None`
E
Emilien Chauvet committed
c3d4126eb198e54e316f84f961452aa454f89a8d
Parent: 8250c17
Committed by GitHub <[email protected]> on 6/5/2024, 3:20:34 PM