Commands | crul

📄️ addcolumn

Adds a new column using the provided column name with each row containing the provided value. The value can be a template that includes tokens from the results of the previous stage. The addcolumn command can also be the first command of a query.

📄️ api

Constructs and sends an http request.

📄️ auth

Browser authenticate with a supported provider (okta|splunk|basic). The returned

📄️ average

Returns the average value for the provided column.

📄️ branch

Starts a new branch, all stages following the branch command will be run on the root of the first branch, until another branch command or a mergebranches command.

📄️ contains

Filter columns containing the provided string

📄️ countOf

Counts instances of a value for the provided column

📄️ curl

Curl stands for client URL, it is a free command-line tool for transferring files with URL syntax. The curl implementation in crul is a small subset of the most common HTTP tasks performed with curl. See the api for a full featured HTTP client.

📄️ devices

Devices and associated viewport properties that can be emulated.

📄️ diff

Diffs the incoming results against a previously frozen set of results in the cellar (if the file does not exist in the cellar, a new one will be created). This command can use flags to control overwriting the cellar comparison file to include new entries, and/or remove old entries.

📄️ duplicate

Duplicates a result set.

📄️ echo

Echos back input.

📄️ encode

Encodes a column to the provided encoding

📄️ endswith

Filter columns ending with the provided string

📄️ evaluate

Evaluate a JavaScript expression and return results.

📄️ excludes

Excludes rows containing the provided string in the provided column

📄️ extract

Extracts keys and values from the provided column containing stringified JSON

📄️ fail

Just fails.

📄️ fillEmpty

Fills empty column values

📄️ filter

Filters results based on the provided expression

📄️ find

Finds any of the provided strings

📄️ flatten

Adds a sequence number to each row. Useful for preserving order after grouping/joining/etc. operations.

📄️ fnmatch

Find a string in provided column using a provided fnmatch pattern

📄️ form

Populate a form based on selector/value pairs and invoke a click

📄️ freeze

Freezes results for retrieval using the thaw command

📄️ graphql

Runs a query against a GraphQL API.

📄️ groupBy

Groups rows based on the provided column

📄️ hash

Hashes the provided columns

📄️ head

Returns the top [count] results from subsequent results.

📄️ html

Extract the matching html elements from the provided column. Defaults to a selector '*'.

📄️ identify

Identifies matching known patterns (Credit Card Numbers, Social Security Numbers, Crypto Addresses, Email Addresses, Phone Numbers). Can optionally provide columns to only check provided columns rather than scanning the entire row.

📄️ join

Joins rows from a previously labeled stage using the provided column as a join key

📄️ keyContains

Useful for working with nested data sets with unique key values. Returns the value of a key matching the provided regex. Additional keys from that level can be provided. For example: keyContains "body$" id would return any key matching the "body$" regex (ex. data.nested.body, data.nested.nested.body), as well as the key id at that level (ex. data.nested.id, data.nested.nested.id).

📄️ logs

Returns system and (non query) worker logs.

📄️ lowercase

Transforms the provided columns to lowercase strings

📄️ math

Runs the mathematical expression provided. The equal sign must be on the left side of the expression (ex. math "sum = 10 + column") and columns used in the expression should have type number as row values (5 - ok vs. "5" - fail). See the tonumber command for casting/converting string values to numbers.

📄️ max

Find the max value of a column

📄️ melt

Extracts keys and values from the provided column containing stringified JSON (if no column is provided, the row is considered to be an array)

📄️ merge

Merges rows based on the provided column

📄️ mergebranches

Ends a branching operation by merging all previous branches into a single results set.

📄️ mergecolumn

Merges all values in the provided column(s) from ALL rows, split by a delimiter.

📄️ mergecolumns

Merges all values for the provided columns in EACH row, split by a delimiter.

📄️ min

Returns the minimum value for the provided column

📄️ navigate

Capture Cookie, localStorage and sessionStorage from page navigate lifecyle (load page, do something, load new page).

📄️ normalize

Extracts keys and values from the provided column containing stringified JSON (if no column is provided, the row is considered to be an array)

📄️ oauth

Client authentication for the OAuth 2.0 authorization framework.

📄️ open

Opens a url in a web browser and returns the matching HTML elements.

📄️ parseTimestamp

Parses the provided timestamp containing column into a date.

📄️ prompt

Returns the response to a prompt sent to a GPT.

📄️ vectorize

Creates an embedding vector for each row of the previous set of results.

📄️ vectorload

flags:

📄️ vectorquery

arguments:

📄️ parseHTMLTable

Extracts tables from a provided html column and converts them to JSON rows of objects.

📄️ parseArticle

Extracts an article from raw HTML.

📄️ random

Creates a column (named "name") of random values between "start" and "end"

📄️ range

Creates a column (named "name") of values from "start" to "end"

📄️ regex

Returns rows from the previous stage results containing the provided regex pattern.

📄️ rename

Renames a column

📄️ replace

Replaces the "replaced" string with the "replacer" string in each result of the provided column

📄️ requests

A summary of all external requests made from a url. NOTE: This is a slower operation as all requests are processed.

📄️ runTemplate

Runs a saved template from the local template store or library

📄️ scrape

The general swiss-army knife for execution of arbitrary JavaScript in an <html> document.

📄️ screenshot

Take a screenshot of a specified element from an <html> document.

📄️ seed

Seeds following commands with user submitted data

📄️ selector

Opens a url in a web browser and returns the matching HTML elements.

📄️ sequence

Flattens a row so all columns are strings.

📄️ sh

Runs the provided shell command.

📄️ sleep

Just sleeps.

📄️ slice

Returns the column value from the provided starting index to the provided ending index.

📄️ soap

Runs a query against a soap API.

📄️ sort

Sorts values in provided column

📄️ split

Splits column by delimiter into values

📄️ startswith

Filter columns starting with the provided string

📄️ sum

Sums up all the row values of one or more columns

📄️ synthesize

Synthesizes results from a provided set of columns which can be grouped together, can also take a prompt flag to provide a GPT prompt to describe the desired synthesized data set.