

0 / 2 embers
0 / 3000 xp
click for more info
Complete a lesson to start your streak
click for more info
Still calibrating
click for more info
Not enough gems
Cost: 6 gems
1: Welcome
incomplete
2: Python Setup
incomplete
3: Normalize URLs
incomplete
4: Extract Page Content
incomplete
5: Extract Links and Images
incomplete
6: Structure Page Data
incomplete
This lesson's interactive features are locked, please to keep using them
We're going to use a virtual environment for our project.
Virtual environments are Python's way to keep dependencies (e.g. the web scraping and http client libraries we're going to use) separate from other projects on our machine.
uv init --no-package your-project-name
cd your-project-name
uv venv
source .venv/bin/activate
You should see (your-project-name) at the beginning of your terminal prompt, for example, mine is:
(webcrawler) wagslane@MacBook-Pro-2 webcrawler %
Always make sure that your virtual environment is activated when running the code or using the Boot.dev CLI.
uv add beautifulsoup4==4.13.4
uv add aiohttp==3.12.12
uv add requests==2.32.4
This tells Python that this project requires these dependencies:
beautifulsoup4 version 4.13.4 (an easy-to-use webscraper library)aiohttp version 3.12.12 (an async http client library)requests version 2.32.4 (a simple http library)To run the project using the uv virtual environment, you use:
uv run main.py
In your terminal, you should see Hello from YOUR PROJECT NAME
Submit the CLI tests while in your virtual environment. There's no penalty on failure for this lesson.