This lesson's interactive features are locked, please to keep using them

JSON Report

We're almost done! Our web crawler now extracts rich data from every page and stores it efficiently in a dictionary. Let's export it to JSON so it's easy to read and share.

For example, one page record in that dictionary might look like this:

page_data = {
    "learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001": {
        "url": "https://learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001/",
        "heading": "Ashenfang Longsword",
        "first_paragraph": "A balanced battlefield blade with a smoldering fuller and leather-wrapped grip.",
        "outgoing_links": [
            "https://learnwebscraping.dev/practice/ecommerce/",
            "https://learnwebscraping.dev/practice/ecommerce/categories/",
            "https://learnwebscraping.dev/practice/ecommerce/categories/longswords/",
        ],
        "image_urls": ["https://learnwebscraping.dev/images/catalog/longswords.svg"],
    }
}

We want to create a report.json file containing a list of all page records.

Assignment

Create a new file called json_report.py.
Import Python's built-in json module:

import json

Create a new function called write_json_report. Here's the signature:

def write_json_report(page_data, filename="report.json"):

page_data is the dictionary returned by your crawler (keys are normalized URLs, values are page data dictionaries)
filename is the JSON file to create (defaults to "report.json")

Implement the write_json_report function. It should:
- Convert page_data.values() to a sorted list (sort by "url")
- Write the list to a JSON file using json.dump with indent=2

Here are some tips:

Sort the pages: pages = sorted(page_data.values(), key=lambda p: p["url"])
Open the file with open(filename, "w", encoding="utf-8")
Write with json.dump(pages, f, indent=2)

Update your main.py to use the JSON report:
- Import your write_json_report function
- After crawling completes, call write_json_report(page_data)
- Remove any console printing of individual page details
Test your implementation:
- Verify report.json is created after running the crawler
- Check that it contains a valid JSON array of page objects

Run and submit the CLI tests.