Skip to main content

Changelog

Release 1.9.0​

Date​

2024-06-09

Breaking Changes​

The vectorload command by default now sets the metadata key to text instead of input resulting in any subsequent vectorquery command results having metadata in the metadata.text column instead of the metadata.input column.

Major Features and Improvements​

  • postman command (new) to run and test postman collections using the newman command-line runner.
  • fnmatch command full wildcard support (*, .js) and negation ('!a/.js', '!(b).js').
  • api command support for controlling body data payload encoding --data.encoding flag.
  • curl command fix for direct passthru of content using the --data and -d flags and default setting of method to POST. NOTE: Use --json flag for setting correct content-type and serialization of json if required.
  • vectorload --pinecone.metadata.namespace flag (new) to specify the key name used for storing metadata with a vector insertion, previous default name of input changed to text for compatibility reasons with LangChain.
  • vectorload --input flag (new) to overide the default column to extract metadata from, defaults to input.

Release 1.8.1​

Date​

2024-05-06

Breaking Changes​

  • CRUL_WEB_PORT .env is replaced with CRUL_WEB value for full scheme, host and port of Web UI.

Major Features and Improvements​

  • Web UI available on same port as REST service layer :1968 with /ui prefix, for backwards compatibility :2001 Space Odyssey remains available to access the web ui.
  • Hard UI refresh (CMD+R) for deeply nested URLS no longer responds with a 404.

Release 1.8.0​

Date​

2024-04-29

Breaking Changes​

None.

Major Features and Improvements​

  • addcolumn command supports adding multiple columns in one call through the use of multiple argument pairs (e.g. addcolumn foo1 bar1 foo2 bar2)
  • api command supports reading content-type application/octet-stream response content.
  • api command supports deflating and reading content-type application/gzip and application/x-gzip response content.
  • api command supports reading parquet response content.
  • api command --verifySSL flag fix for broken logic.
  • api command auth support for recoreded future via --credentials.recordedfuture flag.
  • api command aws4 signing fix for HTTP methods other than GET via --credentials.aws4 flag.
  • api command new --pagination.next.encode flag to specify one of "raw" (default) or "urlencode" to apply to pagination.next token.
  • extract command now works on streams vs loading entire buffer into memory.
  • fillEmpty command now optionally supports a target column via the --column flag.
  • freeze command now optionally supports override of store path for s3 streaming stores.
  • freeze command now optionally supports override of parquet compression via the --parquet.compression flag.
  • join command now allows to specify column key to join from the other table (useful if column name is different in both tables) via the --key flag.
  • jsonvalidate command (new) allows for the validation and optional type coercion of each row against a JSON schema either defined as an argument or as an external cellar reference.
  • logparser command (new) parses a csv formatted log file.
  • parseTimestamp command is fail safe, invalid timestamp cell row values are ignored vs. breaking the halting the query pipeline.
  • parsepath command (new) extracts the root, directory, base, ext, and name from a file path.
  • rename command supports multiple column renames in one call through the use of multiple argument pairs (e.g. rename foo1 bar1 foo2 bar2).
  • replace command supports multiple replaces in one call through the use of multiple argument pairs (e.g. replace col1 matchval1 replaceval1 col1 matchval2 replaceval2).
  • tonumber command supports multiple number converstion in one call through the use of multiple argument pairs (e.g., tonumber col1 col2)
  • Recorded Future added to credentials setup UI.
  • Fix for math grammar in PEG.
  • flatten documentation fix.
  • sequence documentation fix.

Release 1.7.0​

Date​

2023-11-03

Breaking Changes​

None.

Major Features and Improvements​

  • Parquet data file format support for AWS S3 Streaming store.
  • Okta for Custom Authorization Servers oauth support.
  • Microsoft 365 oauth support.
  • Zoom oauth support.
  • Admin tables have zebra striping.
  • Templates support description Markdown - rich text content including images and hyper links.
  • Fix for RELTIO icon in credentials page.
  • Addition of Amazon S3 Streaming and Observe stores.
  • New flatten command to turn nested rows into flat columns.
  • New encode command to encode columns to hex or base64.
  • Fix to the sum command.

Release 1.6.0​

Date​

2023-09-31

Breaking Changes​

Pagination flags have been changed to allow finer control over which arg/flag contains pagination content. Specifically --pagination.data ... is no longer supported and can instead be replicated through --pagination.flag data --pagination.flagValue ...

Major Features and Improvements​

  • Domain throttling can be turned on or off globally.
  • Cache can be turned on or off globally.
  • Variables can be set in a query for re-use in other stages using the --variable flag.
  • Multiple checkpoints can be set per stage
  • curl command -d flag supports multiple values.
  • Pagination is now finer controlled through --paigination.argIndex, --paigination.argValue, --paigination.flag, --paigination.flagValue which allows for pagination control in args and arbitrary flags.
  • --pagination support in the --open command
  • Fix for users page list and edit.
  • timestamp command supports --offset flag to control the time offset of a generated timestamp.
  • timestamp command supports custom --name flag to change the name prefix.
  • soap command supports pagination.
  • Workday oauth credentials support.
  • Salesforce oauth credentials support.
  • Reltio oauth credentials support.
  • Mulesoft oauth credentials support.
  • Zoom oauth credentials support.
  • A credential can be downloaded from the UI for backup.
  • oauth command --options.credentialsEncodingMode flag to control the encoding of client id and client secret in header.
  • Overrides of query processor configuration variables now possible through storage/configs/{service}/.env files

Release 1.5.2​

Date​

2023-08-23

Breaking Changes​

None.

Major Features and Improvements​

  • Added a --headers flag to the scrape command.

Release 1.5.1​

Date​

2023-08-22

Breaking Changes​

None.

Major Features and Improvements​

  • Added a --wait flag to the scrape command.
  • Added a --json flag to the prompt command.

Release 1.5.0​

Date​

2023-08-13

Breaking Changes​

The fillEmpty command has finer grained control through flags, but no longer uses the first arg as the filler value.

Major Features and Improvements​

  • Added a soap command that allows for simplified interaction with SOAP APIs.
  • Added a untable command that removes the provided columns.
  • Added a melt command that melts data.
  • Added a disabled sh command that provides sh access for exeuction of scripts and shell commands.
  • Added support for XML files to the cellar.

Release 1.4.0​

Date​

2023-06-30

Breaking Changes​

Updated the evaluate command to operate as a reducing command by default.

Major Features and Improvements​

  • Updated the branch command with a --branchFilter flag.
  • Added a --url flag to the curl command.
  • Support for tokens in array flags.
  • Added a --retry flag and a --source flag to the synthesize command.
  • Added a --curl flag to the prompt command to generate curl commands.
  • The replace command better supports regex.
  • Added a crul syntax highlighter to the results table.
  • Javascript and JSON syntax highlighting support in the query bar through the javascript/json syntax.
  • Description made optional for query templates.
  • Fixed a bug in the table command for wildcards mixed with specific columns.

Release 1.3.1​

Date​

2023-06-19

Breaking Changes​

None.

Major Features and Improvements​

  • Fix for homepage getting started query syntax highlighting.

Release 1.3.0​

Date​

2023-06-07

Breaking Changes​

API keys are now REQUIRED by default for API requests.

Major Features and Improvements​

  • Added a graphql command that allows for simplified interaction with GraphQL APIs.
  • Added a curl command with a subset of most commonly used flags.
  • Support for regex in filter expressions.
  • API keys created from the UI will be configured with the provided API key value (pre-generated by default).
  • Free Tier scheduled query limit increased from 0 to 1.
  • In-product activation using Google Credentials for Mac OSX Desktop application fix.
  • Only check for product security updates on first UI session.
  • Query Grammar now supports supports duplicate flags required by curl command.
  • Diagnostic error download fix.
  • The api command supports a user name and password to use for server authentication split up on the first colon using the --user flag.
  • The api command supports RFC2068 encoded headers (e.g. header: value\n) using the --headers.format flag.
  • The api command supports redirect control using the --redirect flag.
  • The api command supports a data serialization bypass using --serializer none flag.

Release 1.2.2​

Date​

2023-05-26

Breaking Changes​

None.

Major Features and Improvements​

  • Support for in product license activation for Free/Pro/Enterprise tiers.
  • Added a vectorize command that can transform results into embeddding vectors.
  • Added a vectorload command that loads vectors into a pinecone.io vector database.
  • Added a vectorquery command that queries a pinecone.io vector database.
  • Added a --randomizeHash to override default cache strategy for commands, while retaining stage level caching.
  • User uploaded licenses stored in packages/storage/licenses. Default shipped license remains in packages/licorice/licenses.
  • UI footer and license page shows tier branding.

Release 1.2.1​

Date​

2023-05-08

Breaking Changes​

None.

Major Features and Improvements​

  • The filter command now supports wildcards for != operations. This allows for excluding rows based on a wildcard (*) in the filter.
  • Added a --parseHTMLTable flag to the open command. If this flag is true, all html tables in the results will be parsed into results.
  • 30% startup initialization speedup.

Release 1.2.0​

Date​

2023-05-03

Breaking Changes​

  • Enabled license based limits for Pro and Enterprise features. This has introduced limits to the number of credentials (2) and stores (1) allowed in crul Free. If this affects you, upgrade to a free crul Pro trial license.
  • parseHTMLTable hash, and sequence prefixed with "_" to avoid collisions

Major Features and Improvements​

  • Added a prompt command that allows for arbitrary prompts to be run against the OpenAI API. Requires a configured OpenAI credential.
  • Added a prompts store to persist reusuable prompts for use in queries. Accessible in queries through the $PROMPTS.{prompt name}$ token syntax.
  • Added a synthesize command that can synthesize data sets from existing data and enrich with columns and values described by a natural language prompts. See the Synthetic Data Generation docs for examples.
  • Added OpenAI credential type.
  • 20% startup initialization speedup.
  • Query dispatch performance improvement.
  • Filters added from table cell menus introspect type (user request).
  • The seed command accepts strings and objects for seeding single rows.
  • [Shift/Option] + [Enter] query bar bug which occasionally added a newline to the query has been fixed.
  • The api command now displays improved content-type parsing error debugging including request and response meta data.
  • Improved query error introspection in UI.
  • Query internal tokens are now displayed in UI.
  • Improved query editor performance and usability in UI.
  • Improved api command credential failure introspection.

Release 1.1.1​

Date​

2023-04-18

Breaking Changes​

None.

Major Features and Improvements​

  • Query in UI that errors out displays detailed error info on status tab.
  • open command missing --device "Desktop" (user request).
  • api displays debugging information on payload parsing errors (user request).
  • UI query assistant performance fix for trailing ||.
  • api no longer sends empty bearer header when not set.

Release 1.1.0​

Date​

2023-03-31

Breaking Changes​

The internal version of Chromium has been updated. The open, requests, scrape, evaluate, screenshot, and form browser commands could have changes in performance/behavior due to internal browser changes not covered by internal tests.

Major Features and Improvements​

  • The evaluate command supports the injection of the faker javascript library for synthetic data generation.
  • The evaluate command handles no leading row generator (can be used as a leading command) and can support both array and object returned values for generation of rows.
  • The mergecolumn command now converts the provided columns row values into either an array (new) or a delimited string.
  • UI desert theme found under the mega menu.
  • Free disabled features are now part of upcoming crul pro
  • UI displays warnings for throttled outbound queries.
  • Browser commands can now operate in stealth mode to thwart headless chromium detection via the .env STEALTH_PLUGIN_ENABLED variable.
  • parseTimestamp command ms fix
  • Updates to headless browser clustering.
  • Non-sink entries can be downloaded from cellar UI.
  • api command zip archive entry scanning and extraction.
  • Update headless browser Chromium version to 111.0.5555.0.
  • UI query bar command help includes link to docs examples section.
  • Added a parseHTMLTable command to parse HTML tables into datasets.
  • The api command supports optional proxy via --proxy flag.
  • The open and browser commands supports optional proxy via --proxy flag.
  • The open command with --selector having no match will no longer error.
  • In-product linking to new discord channel.
  • Docker container no longer prints info logs and displays UI host/port/scheme.
  • Added a parseArticle command to parse HTML page content into article content based on the Firefox Reader View.
  • Intermittent upgrade bug
  • Remove third-party contrib warnings.
  • api command 'csv' serialization of text/csv response content-type.