Skip to main content

Query any Webpage or API

Access, crawl, enrich, and clean data from anywhere to anywhere.

Lights, camera, 'n IT! ✨

|| find comments
|| filter "(nodeName == 'A') and (parentElement.attributes.class == 'subline')"
|| open $attributes.href$
|| filter "(attributes.class == 'comment')"

Dreamy Webpage and API data feeds. 🌜

|| freeze --store "kafka-prod" --kafka.topic "hn_comments"

Yeah, can do a whole lot... 🎨

Push incremental changes

Compare results from previous results and only retain what is new.

Learn more ↗

Schedule queries

Schedule queries to be run on a set interval.

Learn more ↗

Send data to 30+ stores

Send your data to Amazon S3, Kafka, Splunk and a whole lot more.

Learn more ↗

Client authentication for OAuth

Authenticate with multiple OAuth providers in order to access protected API data.

Learn more ↗

Credentials vault for secrets

Store custom credentials for use in queries.

Learn more ↗

Expand queries

Expand API/web links and aggregate results.

Learn more ↗

Domain throttling

Control the interval of accessing a domain based on custom prescribed rate limit policies.

Learn more ↗


Take webpage screenshots

Take full size and/or element screenshots from a webpage.

Learn more ↗


Run queries with your favorite language

Access the query results with auth keys and the REST API.

Learn more ↗

The web is yours.