Git Repositories¶

Some text-based content is more conveniently stored in a separate Git repository rather than internally in the Nautobot database. Such a repository may currently include any or all of the following for Nautobot to consume:

Job source files and associated data files,
Configuration context data
Export templates
Additional data types as registered by any installed Apps

Important

Nautobot's Git integration depends on the availability of the git program. If git is not installed, Nautobot will be unable to pull data from Git repositories.

Changed in version 1.6.2

To proactively avoid conflicts in data, it is no longer possible to configure multiple Git repository entries that both have the same remote_url and also provide the same data type(s). Configuration of multiple entries with the same remote_url is still permitted if they are configured to provide entirely distinct types of data.

Repository Configuration¶

When defining a Git repository for Nautobot to consume, the name, remote URL, and branch parameters are mandatory - the name acts as a unique identifier, and the remote URL and branch are needed for Nautobot to be able to locate and access the specified repository. Additionally, if the repository is private you may specify a secrets group that can be used to gain access to the repository.

Note

Nautobot currently only supports repositories that can be cloned using the standard git command line, git clone. This means App-style integrations like GitHub Apps are not currently supported, as their workflow of managing files leverages a REST API.

Removed in version 2.0.0

In Nautobot 1.x it was possible to configure the secrets (username and/or token) for a private Git Repository directly in Nautobot's database. Due to security concerns and maintainability challenges, this option has been removed. To access a private Git repository you now must use Secrets Groups.

The implementation of private repository access can vary from Git provider to Git provider. The following providers have been confirmed to work; in theory, additional providers using the same pattern will work, but there is currently no specific support for all providers.

GitHub's token does not require a username.
GitLab's token requires a username, conventions are to use the username "oauth2". In addition, GitLab's deploy tokens are also supported.
For Bitbucket, there are two options: personal access tokens or OAuth2 depending on the product.

Note

When defining a secrets group for a Git repository, the group must contain assigned secret(s) with an access type of HTTP(S) and secret type(s) of Token (and Username, if required by the provider).

Whenever a Git repository record is created, updated, or deleted, Nautobot automatically enqueues a background task that will asynchronously execute to clone, fetch, or delete a local copy of the Git repository on the filesystem (located under GIT_ROOT) and then create, update, and/or delete any database records managed by this repository. The progress and eventual outcome of this background task are recorded as a JobResult record that may be viewed from the Git repository user interface.

Important

The repository branch must exist and have a commit against it. At this time, Nautobot will not initialize an empty repository.

Note

If you are using a self-signed Git repository, you will need to set the environment variable GIT_SSL_NO_VERIFY="1" in order for the repository to sync.

Repository Structure¶

Jobs¶

Jobs can be defined in Python files located in a /jobs/ directory or jobs.py at the root of a Git repository. Any job classes defined in these files that have been registered during import will be discovered by Nautobot and made available to be run as a job. See the section on Job registration for more information.

Note

There must be an __init__.py file in the /jobs/ directory.

Changed in version 2.0.0

Jobs provided by a Git repository are loaded as real Python modules and now support inter-module relative Python imports (i.e., you can package Python "libraries" into a Git repository and then import them from Jobs in that repository). As a result, the top-level directory of Git repositories that provide jobs must now contain an __init__.py file.

When syncing or re-syncing a Git repository, the Nautobot database records corresponding to any provided jobs will automatically be refreshed. If a job is removed as a result of the sync, the corresponding database record will not be automatically deleted, but will be marked as installed = False and will no longer be runnable. A user with appropriate access permissions can delete leftover Job database records if desired, but note that this will result in any existing JobResult records no longer having a direct reference back to the Job that they originated from.

Configuration Contexts¶

Config contexts may be provided as JSON or YAML files located in /config_contexts/. There are three different types of config context scopes; explicit, implicit, and local.

Explicit: Defined as JSON or YAML files at the root of the /config_contexts/ folder. Multiple config contexts can be specified within the each file. The metadata regarding the naming and scoping of the config context is determined by the _metadata key for each list element.
Implicit: They're defined using a specific folder and file structure to apply the config context to a specific scope.
Local: Defined at the device/virtual machine level and only being applied to the specific device/virtual machine.

Added in version 1.5.6

Config contexts provided in a Git repository can now be filtered by locations in addition to all previously supported filters.

Metadata¶

The metadata used to create the config context has the following options and is specified by the _metadata key.

Key	Required	Default	Description
name	True	N/A	The name that will be assigned to the Config Context
weight	False	1000	The weight that will be assigned to the Config Context that determines precedence
description	False	N/A	The description applied to the Config Context
is_active	False	True	Whether or not the Config Context is active
config_context_schema	False	N/A	Config Context Schema that it should be validated against

Changed in version 2.0.0

The key for specifying Config Context Schemas was renamed from schema to config_context_schema.

There are several other keys that can be defined that match the scope of what the Config Context will be assigned to.

Here is an example _metadata key defined:

{
    "_metadata": {
        "name": "Location NYC servers",
        "weight": 1000,
        "description": "NTP and Syslog servers for location NYC",
        "is_active": true,
        "locations": [{"name": "NYC"}],
        "config_context_schema": "Config Context Schema 1"
    },
    "acl": {
        "definitions": {
            "named ": {
                "PERMIT_ROUTES": [
                  "10 permit ip any any"
                ]
            }
        }
    },
    "route-maps": {
        "PERMIT_CONN_ROUTES": {
            "seq": 10,
            "statements": [
                "match ip address PERMIT_ROUTES"
            ],
            "type": "permit"
        }
    }
}

Important

The only config context scope that does not require any metadata defined is the local configuration context

Explicit Config Contexts¶

As stated above, these explicit files live at the root of /config_contexts. These files will be imported as described below, with no special meaning attributed to their filenames (the name of the constructed config context will be taken from the _metadata key within the file, not the filename). To provide a visual, the context_1.json and context_2.yml are explicit config context scopes.

config_contexts/
  context_1.json   # JSON data will be imported as-is, with scoping explicit from its contents
  context_2.yaml   # YAML data will be imported as-is, with scoping explicit from its contents

For files in the root of the /config_contexts/ directory, a single file may define a single config context as above, or alternatively it may contain a list of config context data definitions, as in the following example:

---
- _metadata:
    name: "Router hostname pattern"
    roles:
      - name: "Router"
  hostname_pattern_string: "rtr-.+"
- _metadata:
    name: "Console Server hostname pattern"
    roles:
      - name: "Console Server"
  hostname_pattern_string: "cs-.+"
- _metadata:
    name: "Switches hostname pattern"
    roles:
      - name: "Aggregation Switch"
      - name: "Services Switch"
  hostname_pattern_string: "switch-.+"
- _metadata:
    name: "Appliance hostname pattern"
    roles:
      - name: "Security Appliance"
  hostname_pattern_string: "fw-.+"
...

The _metadata key will map to the attributes required when creating a config context via the UI or API such as name and the scope of the config context. If we take a look at the first element, the name assigned to the config context will be "Router hostname pattern" and be scoped to roles with a name of Router.

Any key/value pair defined at the same level as _metadata will be converted to the config context data. Keeping with the first element, it will have a key set as hostname_pattern_string with a value of rtr-.+.

Implicit Config Contexts¶

Implicit config context files will have the following folder/file structure /config_contexts/<filter>/<name>.[json|yaml], in which case their path and filename will be taken as an implicit scope for the context. For example:

config_contexts/
  locations/
    NYC.yaml       # YAML data, with implicit scoping to the Location with name "NYC"
    NYC 01.json    # JSON data, with implicit scoping to the Location with name "NYC 01"

Changed in version 2.0.0

In Nautobot 1.x, the filenames were interpreted as slug strings for the related objects. In Nautobot 2.0 and later, the filenames are based on the name (or for device-types files, the model) of the related object instead.

The implicit config contexts will be defined using dictionaries for both _metadata and any context data for the config context.

JSON¶

{
    "_metadata": {
        "name": "Region NYC servers",
        "weight": 1000,
        "description": "NTP and Syslog servers for region NYC",
        "is_active": true,
        "config_context_schema": "Config Context Schema 1"
    },
    "ntp-servers": [
        "172.16.10.22",
        "172.16.10.33"
    ],
    "syslog-servers": [
        "172.16.9.100",
        "172.16.9.101"
    ]
}

YAML¶

_metadata":
  name: "Region NYC servers"
  weight: 1000
  description: "NTP and Syslog servers for region NYC"
  is_active: true
  config_context_schema: "Config Context Schema 1"

ntp-servers:
  - 172.16.10.22
  - 172.16.10.33
syslog-servers:
  - 172.16.9.100
  - 172.16.9.101

This will create a config context with two keys: ntp-servers and syslog-servers.

Local Configuration Contexts¶

Files in a config_contexts/devices/ and/or config_contexts/virtual_machines/ directory will be used to populate "local" config context data for individual devices or virtual machines. For these files, the device/VM name will always be taken from the filename, and the data in the file will be used precisely as-is (there is no need, or support, for a _metadata key in these files).

config_contexts/
  devices/
    rtr-01.yaml    # YAML data, local to the Device named "rtr-01"
  virtual_machines/
    vm001.json     # JSON data, local to the VirtualMachine named "vm001"

Note

While virtual machines are always uniquely identified by their name, it is possible for devices associated with different locations and/or tenants to share an identical name. Currently, Nautobot is unable to automatically apply local config context via Git to devices that have a non-globally-unique name (or no name at all).

Configuration Context Schemas¶

Config context schemas may be provided as JSON or YAML files located in /config_context_schemas/.

Files in the root of the /config_context_schemas/ directory will be imported as described below, with no special meaning attributed to their filenames (the name of the constructed config context schema will be taken from the _metadata within the file, not the filename). Similar to config context definitions, a single file may define a single config context schema or a list of such schemas - see examples below.

config_context_schemas/
  context_schema_1.json
  context_schema_2.yaml

When loading the schema, the key _metadata will be extracted from the loaded data and used to define the config context schema's metadata, while the actual config context data schema will be based on the key data_schema.

JSON single example:

{
  "_metadata": {
    "name": "Config Context Schema 1",
    "description": "Schema for defining first names."
  },
  "data_schema": {
    "title": "Person",
    "properties": {
      "firstName": {
        "type": "string",
        "description": "The person's first name."
      }
    }
  }
}

JSON list example:

[
  {
    "_metadata": {
      "name": "Config Context Schema 1",
      "description": "Schema for defining first names."
    },
    "data_schema": {
      "title": "Person",
      "properties": {
        "firstName": {
          "type": "string",
          "description": "The person's first name."
        },
      }
    }
  },
  {
    "_metadata": {
      "name": "Config Context Schema 2",
      "description": "Schema for defining last names."
    },
    "data_schema": {
      "title": "Person",
      "properties": {
        "lastName": {
          "type": "string",
          "description": "The person's last name."
        },
      }
    }
  },
]

YAML single example:

---
_metadata:
  name: "Config Context Schema 1"
  description: "Schema for defining first names."
data_schema:
  title: "Person"
  properties:
    firstName:
      type: "string"
      description: "The person's first name"

YAML list example:

---
- _metadata:
    name: "Config Context Schema 1"
    description: "Schema for defining first names."
  data_schema:
    title: "Person"
    properties:
      firstName:
        type: "string"
        description: "The person's first name"
- _metadata:
    name: "Config Context Schema 2"
    description: "Schema for defining last names."
  data_schema:
    title: "Person"
    properties:
      lastName:
        type: "string"
        description: "The person's last name"

Export Templates¶

Export templates may be provided as files located in /export_templates/<grouping>/<model>/<template_file>; for example, a JSON export template for Device records might be /export_templates/dcim/device/mytemplate.json.

The name of a discovered export template will be presented in Nautobot as <repository name>: <filename>.
The MIME type of a file rendered from a discovered export template will try to match the extension to IANA's list. If not detected, it will default to text/plain.
The file extension of a file rendered from a discovered export template will match that of the template itself (so, in the above example, the extension would be .json)