community[minor]: add user agent for web scraping loaders (#22480)
**Description:** This PR adds a `USER_AGENT` env variable that is to be used for web scraping. It creates a util to get that user agent and uses it in the classes used for scraping in [this piece of doc](https://python.langchain.com/v0.1/docs/use_cases/web_scraping/). Identifying your scraper is considered a good politeness practice, this PR aims at easing it. **Issue:** `None` **Dependencies:** `None` **Twitter handle:** `None`
E
Emilien Chauvet committed
c3d4126eb198e54e316f84f961452aa454f89a8d
Parent: 8250c17
Committed by GitHub <[email protected]>
on 6/5/2024, 3:20:34 PM