

0 / 2 embers
0 / 3000 xp
click for more info
Complete a lesson to start your streak
click for more info
Difficulty: 5
click for more info
Not enough gems
Cost: 6 gems
1: JSON Report
incomplete
2: Submit the Link to Your Repository!
incomplete
This lesson's interactive features are locked, please to keep using them
We're almost done! Our web crawler now extracts rich data from every page and stores it efficiently in a dictionary. Let's export it to JSON so it's easy to read and share.
For example, one page record in that dictionary might look like this:
page_data = {
"learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001": {
"url": "https://learnwebscraping.dev/practice/ecommerce/products/ashenfang-longsword-fan-1001/",
"heading": "Ashenfang Longsword",
"first_paragraph": "A balanced battlefield blade with a smoldering fuller and leather-wrapped grip.",
"outgoing_links": [
"https://learnwebscraping.dev/practice/ecommerce/",
"https://learnwebscraping.dev/practice/ecommerce/categories/",
"https://learnwebscraping.dev/practice/ecommerce/categories/longswords/",
],
"image_urls": ["https://learnwebscraping.dev/images/catalog/longswords.svg"],
}
}We want to create a report.json file containing a list of all page records.
Import Python's built-in json module:
import jsondef write_json_report(page_data, filename="report.json"):page_data is the dictionary returned by your crawler (keys are normalized URLs, values are page data dictionaries)
filename is the JSON file to create (defaults to "report.json")
Convert page_data.values() to a sorted list (sort by "url")
Write the list to a JSON file using json.dump with indent=2
Here are some tips:
Sort the pages: pages = sorted(page_data.values(), key=lambda p: p["url"])
Open the file with open(filename, "w", encoding="utf-8")
Write with json.dump(pages, f, indent=2)
Import your write_json_report function
After crawling completes, call write_json_report(page_data)
Remove any console printing of individual page details
Verify report.json is created after running the crawler
Check that it contains a valid JSON array of page objects
Run and submit the CLI tests.