Skip to main content

Basics

Ok, you've read the introduction, you've tried the quickstart, maybe an example or two, what next?

This getting started section aims to get you going fast and learn what crul can do, and which commands are valuable to start enjoying the power of crul.

What can I do with crul​

Transform a webpage into a filterable table​

With crul, you can use the open command with a provided url to transform a webpage into a CSV table of elements. Your next step will probably be to filter for certain elements, say headlines, links, etc. which can be accomplished with data filtering commands.

Transform API results into a filterable table​

Crul can also be used for transforming API responses into a table using the api command. Note: many APIs will return a nested structure where the elements are embedded in an array. See the normalize command for expanding nested API response arrays into rows per array element.

Ok, you've opened one link at a time, either through the open command, or the api command. What makes crul extra cool is the ability to make multiple requests in one query stage based on the rows of the previous stage, essentially making a request for each row. This expanding stage feature, in combination with tokenization, allows you to use the same open command or api command to make many requests in a single stage.

Export results locally or to a 3rd party service​

Once you have interesting set of results, you may want to download them locally to a CSV or JSON file, or push them to a 3rd party analytics, visualization, ticketing, etc. service. This can be accomplished in crul also by configuring a store and using data import/export commands.

Schedule queries to run and export results to a 3rd party service​

Many crul queries interact with web and API content that changes over time. Scheduling queries and pushing result sets (possibly diffed to avoid duplicates) on an interval is great way to get data into another system for further analysis, etc.

Note that scheduling is enterprise feature.

Import CSV/JSON/NDJSON files for processing​

The crul language is a great way to quickly filter and transform a data set, even one that might not be pulled live from an API or Webpage. You can upload your own CSV, JSON, or NDJSON files to the cellar, and thaw them to use the crul language to process.