Web Node

What it does

Automatically visits websites you specify and extracts content. When you paste a URL, the Web node can auto-execute without manual triggers for instant results.

Extract content from websites and web pages with instant auto-execution capabilities.

Content

• A URL entered in this node or passed from connected parent nodes. If multiple URLs are present upstream, the first valid URL is used.

Output

• Extracted page content in the selected format (Markdown, Text, or HTML). For social video URLs, returns the transcript text.

Settings

• Scraper type:
  • Markdown Scraper (webcrawlerapi-scrape-v1) — 2 credits
  • Text Scraper (webcrawlerapi-scrape-v1-cleaned) — 2 credits
  • HTML Scraper (cleaned) (webcrawlerapi-scrape-v1-html-cleaned) — 2 credits
  • HTML Scraper (raw) (webcrawlerapi-scrape-v1-html) — 2 credits
  • YouTube & Shorts (yt-scrape-transcript) — 10 credits
  • TikTok (tiktok-transcript) — 10 credits
  • Instagram Reels (instagram-transcript) — 20 credits
  • Twitter/X Videos (twitter-transcript) — 20 credits

Use cases

Research & Analysis

• Extract articles, blogs, and news for analysis
• Monitor competitors and market trends

Content Processing

• Video transcription and summarization
• Data collection and dataset building

Automation

• Website monitoring and change tracking
• Content aggregation across multiple sites

Credits cost

Charged per URL after a successful extraction. Standard site scrapers cost 2 credits. Social video scrapers cost 10 credits for YouTube and TikTok, and 20 credits for Instagram and Twitter.