Create Custom cnquery Providers

While cnquery includes providers for many common platforms, you can create custom providers to query systems or services that aren't yet supported. Custom providers work with any MQL runtime, including cnquery and cnspec. Providers can be written in any language or framework as long as they adhere to the plugin structure and API.

Provider structure

Provider plugins are located by default in either the system or user location. They consist of a folder containing a set of required files. For example, for a provider mypro:

./mypro/                    Folder containing the provider and additional files
├── mypro                   Executable used to start the provider plugin
├── mypro.json              Provider metadata
└── mypro.resources.json    Resources and fields schema

• Provider binary: The executable spawned when a new provider instance is created. It must adhere to the plugin behavior defined in go-plugin, communicating via gRPC with the caller and implementing the provider plugin proto API. Due to these specifications, providers can be created in any language or stack.

• Provider metadata: Contains information about the provider like its name, UID, version, connections, and connectors. It is also used to build the CLI interface. The structure is defined in the Provider struct.

• Resources and fields schema: Contains all resources and fields offered by the provider. This includes version constraints for compatibility, field types, and basic documentation. Defined in the Schema message in the resources proto. Provider schemas can be auto-generated (see scaffolding below).

Providers are distributed as tar.xz files containing the above structure. Install them using:

cnquery providers install -f provider.tar.xz

You can also install providers manually by creating the above structure in the user system provider location.

Provider scaffolding

cnquery includes a scaffolding utility to help create providers. The scaffolding tool is designed for Go and requires Go 1.21 or later.

To install it, clone the cnquery repository and install the tool:

git clone https://github.com/mondoohq/cnquery.git
cd cnquery
go install ./apps/provider-scaffold

To create a new provider (in this example, mypro):

provider-scaffold --path mypro --provider-id mypro --provider-name "My Provider" --go-package github.com/myuser/mql-provider-mypro/mypro

The scaffolding tool currently generates Go-based providers. Contributions to extend it to other languages are welcome.

Builtin providers

You can build Go-based providers directly into any MQL runtime binary instead of having separate provider binaries that are spawned. This means no additional files are installed or updated, and no additional processes are spawned.

To configure builtin providers, modify the list of builtinProviders. Once added, you can see it in the list of builtin providers:

> cnquery providers

→ builtin (found 3 providers)

  core 11.0.98
  mock 9.0.0 with connectors: mock, upstream
  mypro 0.0.1 with connectors: mytarget

...

Builtin providers cannot be updated without creating a new build, and they increase the file size of the runtime. However, this approach gives you an overall smaller footprint and tighter security profile.