Skip to main content

open

open [url]

Opens a url in a web browser and returns the matching HTML elements.

arguments:

url

The url for the request. (type: string)

flags:

--appendStage

Used to append the results from a previous stage to the current stage. (provide a label, stage index, or boolean true to append the previous results)

--args

array of JSON encoded values to pass to expression as input

--cache

A boolean value of true/false that determines whether or not to use the cache. Generally most commands will default to true.

--checkpoint

Format: "{CHECKPOINT NAME}:{COLUMN}" Used to store the value of the provided column (in the first row of results) in the provided name for use as a checkpoint in scheduled queries or other stages. Not encrypted. Can be accessed using $CHECKPOINTS.{CHECKPOINT NAME}$

--cookieStorage

JSON serialized Cookie values to set pre-request

--device

device to emulate, see devices command

--dimension

extract dimension calculations from matching elements. NOTE: Running certain selectors like '*' can with dimension enabled can have performance implications due to DOM thrashing.

--engine

css selector engine; "dom" or "jquery"

--enrich

Enriches each result row with the previous row. The previous columns will be appended with a _previous.

--expression

javascript expression to evaluate after completion ES6 fat-arrow format with an argument of data passed (example "(data)=>data.map(item=>item.innerText)")

--filter

A filter to run on the command results before completing the command. If not provided, no filter is run on the results.

--format

screenshot format type (jpeg or png)

--fresh

Starts the stage as if it was a fresh query, so will not use any previous result.

--guid

Adds a populated random guid column.

--hashtml

extract innerHTML and outerHTML from matching elements and hash the values using the sha1 algorithm. NOTE: Running certain selectors like '*' with --hashtml enabled can have performance implications.

--headers

JSON header values to set pre-request

--html

extract innerHTML and outerHTML from matching elements. NOTE: Running certain selectors like '*' with --html enabled can have performance implications.

--jquery

inject jquery library in last object argument passed to expression (example "(data, { $ })=>$('a').html()

--labelStage

Used to label a stage with a user provided label.

--localStorage

JSON sessionStorage values to set pre-request

--lodash

inject lodash library in last object argument passed to expression (example "(data, { })=>)

--mathjs

inject mathjs library in last object argument passed to expression (example "(data, { mathjs })=>mathjs)

--maxConcurrent

Override for the system max concurrent workers for a stage.

--pagination.argIndex

The index of the argument to store the pagination context.

--pagination.argKey

The key to store the pagination context in the argument (assumes key value is JSON).

--pagination.done

The value of pagination.next thats indicates that pagination is complete.

--pagination.filter

Filter the results using a filter expression.

--pagination.flag

The flag to store the pagination context.

--pagination.flagKey

The key to store the pagination context in the flag (assumes key value is JSON).

--pagination.max

Maximum number of pages visited.

--pagination.next

The location within the results set describing the location of the next set of results. If not a complete URL, use the --pagination.url to construct the url using a token

--pagination.url

The url of the next set of results, which can include a token to substitute in the value selected by --pagination.next. If blank, the url is assumed to be contained in the value described by --pagination.next

--parseHTMLTable

Parse all HTML table elements into results

--proxy

The URI for a http, https, socks(v5), socks5, socks4, or pac proxy

--quality

screenshot compression/quality for jpeg only

--randomizeHash

Randomizes the stage hash, even if args and flags are the same.

--retry

number attempts to sucessfully run the following task

--screenshot

take a screenshot of page

--selector

CSS selector expression

--session

capture session information; cookieStorage, localStorage and sessionStorage

--sessionStorage

JSON serialized sessionStorage values to set pre-request

--stats

Controls if a stats calculation is run on a stage after it completes.

--table

A comma separated list of columns to include in the command results. If not provided, all columns will be included.

--type

Each command has a default type, either "mapping" or "reducing". Some commands can operate as either, when "reducing" they will operate on all rows at once, when "mapping", they will operate on one row at a time.

--variable

Format: "{VARIABLE NAME}:{COLUMN}" Used to store the value of the provided column (in the first row of results) in the provided name for use as a variable in other stages. Can be accessed using $VARIABLES.{VARIABLE NAME}$. Stored as an encrypted secret. Not stored across queries.

--wait

Wait criteria before resolving page load. Think of it as when to take a snapshot of browser data. window.onload: open http://crul.com --wait 'winload'

DOMContentLoaded: open http://crul.com --wait 'domload'

No more than 0 network connections for at least 500 ms. open http://crul.com --wait 'idle'

No more than 2 network connections for at least 500 ms. open http://crul.com --wait 'net2'

Element with matching selector exists: open http://crul.com --wait '.enjoying-it'

Wait for 30 seconds: open http://crul.com --wait 3000

Run the following fat-arrow ()=> until returns truthy: open http://google.com --wait '()=>true'

'winload' - consider navigation to be finished when the window load event is fired. 'domload' - consider navigation to be finished when the DOMContentLoaded event is fired. 'idle' - consider navigation to be finished when there are no more than 0 network connections for at least 500 ms. 'net2' - consider navigation to be finished when there are no more than 2 network connections for at least 500 ms. <integer> - the number of milliseconds to wait for. <()=>expression> - the function to be evaluated in browser context. <cssselector> A css selector of an element to wait for.

--while

Will rerun the stage until the provided expression is valid for the first line of results.

support

AMI_ENTERPRISE AMI_FREE AMI_PRO BINARY_ENTERPRISE BINARY_FREE BINARY_PRO DESKTOP_ENTERPRISE DESKTOP_FREE DESKTOP_PRO DOCKER_ENTERPRISE DOCKER_FREE DOCKER_PRO