Web URL Crawler at Scale

You are given helper APIs: fetchpage(url) -> str returns page content parseurls(content) -> list[str] returns outgoing URLs from that page Design traversal and failure handling for crawling problems.