Back
42/65
Lesson 42
5 min
MDWB: Web to Markdown
Convert web pages to clean Markdown for AI consumption
Ready to level up?
Mark complete to track your learning progress.
Convert web pages to clean Markdown for AI consumption
Mark complete to track your learning progress.
Convert web pages into clean Markdown that AI agents can read and reason about.
MDWB (Markdown Web Browser) fetches web pages and converts them into clean Markdown, stripping out ads, navigation, and scripts. The result is perfect for feeding into LLMs as context.
When AI agents need information from the web, raw HTML is noisy and token-expensive. MDWB extracts just the content, preserving headings, code blocks, lists, and links in a format LLMs handle efficiently.
Works with most websites
No ads, scripts, or clutter
Preserves code formatting
Built in Rust for speed
Web-to-Markdown Conversion Studio
Watch HTML get fetched, noise stripped, and content converted to clean Markdown
Submit a URL as a capture job, then read the Markdown snapshot back by job id.
# Capture a web page (streams progress, prints the job id)mdwb fetch "https://docs.example.com/api/reference" --watch# Read the Markdown snapshot for that jobmdwb show <job-id># Save it to a filemdwb show <job-id> > guide.md# Pipe directly to an AI agentmdwb show <job-id> | claude "summarize this API"
MDWB shines when you need web content in LLM-friendly format.
# Research a library's docs before using itmdwb fetch "https://docs.rs/tokio/latest" --watchmdwb show <job-id> > tokio-docs.md# Get error context from StackOverflowmdwb fetch "https://stackoverflow.com/questions/12345" --watchmdwb show <job-id> > context.md# Crawl a docs site one level deepmdwb crawl "https://blog.example.com/architecture-decisions"# Capture several pages as one tagged batch, then search across themmdwb batch --url "https://docs.example.com/a" --url "https://docs.example.com/b" --tag researchmdwb search "architecture" --tag research | claude "analyze these documents"
MDWB connects web knowledge to the agent workflow.
Fetch docs, then compress for maximum context
Index downloaded pages for local semantic search
Archive both web pages and AI conversations
Feed research papers into Brenner corpus