NSO Resources

More from Cisco DevNet

NSO Developer Studio provides an integrated framework for developing NSO services using Visual Studio (VS) Code extensions. The extensions come with a core feature set to help you create services and connect to running CDB instances from within the VS Code environment. The following extensions are available as part of the NSO Developer Studio:

• NSO Developer Studio - Developer: Used for creating NSO services. Also referred to as NSO Developer extension in this guide.

• NSO Developer Studio - Explorer: Used for connecting to and inspecting NSO instance. Also referred to as NSO Explorer extension in this guide.

NSO Developer Studio - Developer Extension

This section describes the installation and functionality of the NSO Developer extension.

The purpose of the NSO Developer extension is to provide a base framework for developers to create their own NSO services. The focus of this guide is to manifest the creation of a simple NSO service package using the NSO Developer extension. At this time, reactive FastMAP and Nano services are not supported with this extension.

In terms of an NSO package, the extension supports YANG, XML, and Python to bring together various elements required to create a simple service.

After the installation, you can use the extension to create services and perform additional functions described below.

System Requirements

To get started with development using the NSO Developer extension, ensure that the following prerequisites are met on your system. The prerequisites are not a requirement to install the NSO Developer extension, but for NSO development after the extension is installed.

• Visual Studio Code.

• Java JDK 11 or higher.

• Python 3.9 or higher (recommended).

Install the Extension

Installation of the NSO Developer extension is done via the VS Code marketplace.

To install the NSO Developer extension in your VS Code environment:

1. Open VS Code and click the Extensions icon on the Activity Bar.

2. Search for the extension using the keywords "nso developer studio" in the Search Extensions in Marketplace field.

3. In the search results, locate the extension (NSO Developer Studio - Developer) and click Install.

4. Wait while the installation completes. A notification at the bottom-right corner indicates that the installation has finished. After the installation, an NSO icon is added to the Activity Bar.

Make a New Service Package (Python only)

Use the Make Package command in VS Code to create a new Python package. The purpose of this command is to provide functionality similar to the ncs-make-package CLI command, that is, to create a basic structure for you to start developing a new Python service package. The ncs-make-package command, however, comes with several additional options to create a package.

To make a new Python service package:

1. In the VS Code menu, go to View, and choose Command Palette.

2. In the Command Palette, type or pick the command NSO: Make Package. This brings up the Make Package dialog where you can configure package details.

3. In the Make Package dialog, specify the following package details:

   • Package Name: Name of the package.

   • Package Location: Destination folder where the package is to be created.

   • Namespace: Namespace of the YANG module, e.g. http://www.cisco.com/myModule.

   • Prefix: The prefix to be given to the YANG module, e.g. msp.

   • Yang Version: The YANG version that this module follows.

4. Click Create Package. This creates the required package and opens up a new instance of VS Code with the newly created NSO package.

5. If the Workspace Trust dialog is shown, click Yes, I Trust the Authors.

Open an Existing Package

Use the Open Existing Package command to open an already existing package.

To open an existing package:

1. In the VS Code menu, go to View, then choose Command Palette.

2. In the Command Palette, type or pick the command NSO: Open Existing Package.

3. Browse for the package on your local disk and open it. This brings up a new instance of VS Code and opens the package in it.

Edit YANG files

Opening a YANG file for edit results in VS Code detecting syntax errors in the YANG file. The errors show up due to missing path to YANG files and can be resolved using the following procedure.

Add YANG models for Yangster

For YANG support, a third-party extension called Yangster is used. Yangster is able to resolve imports for core NSO models but requires additional configuration.

To add YANG models for Yangster:

1. Create a new file named yang.settings by right-clicking in the blank area of the Explorer view and choosing New File from the pop-up.

2. Locate the NSO source YANG files on your local disk and copy the path.

3. In the file yang.settings, enter the path in the JSON format: { "yangPath": "<path to Yang files >" }, for example, { "yangPath": /home/my-user-name/nso-6.0/src/ncs/yang}. On Microsoft Windows, make sure that the backslash (\) is escaped, e.g., "C:\\user\\folder\\src\\yang".

4. Save the file.

5. Wait while the Yangster extension indexes and parses the YANG file to resolve NSO imports. After the parsing is finished, errors in the YANG file will disappear.

View YANG Diagram

YANG diagram is a feature provided by the Yangster extension.

To view the YANG diagram:

1. Update the YANG file. (Pressing Ctrl+space brings up auto-completion where applicable.)

2. Right-click anywhere in the VS Code Editor area and select Open in Diagram in the pop-up.

Add a New YANG Module

To add a new YANG module:

1. In the Explorer view, navigate to the yang folder and select it.

2. Right-click on the yang folder and select NSO: Add Yang Module from the pop-up menu. This brings up the Create Yang Module dialog where you can configure module details.

3. In the Create Yang Module dialog, fill in the following details:

   • Module Name: Name of the module.

   • Namespace: Namespace of the module, e.g., http://www.cisco.com/myModule.

   • Prefix: Prefix for the YANG module.

   • Yang Version: Version of YANG for this module.

4. Click Finish. This creates and opens up the newly created module.

Add a Service Point

Often while working on a package, there is a requirement to create a new service. This usually involves adding a service point. Adding a service point also requires other parts of the files to be updated, for example, Python.

Service points are usually added to lists.

To add a service point:

1. Update your YANG model as required. The extension automatically detects the list elements and displays a CodeLens called Add Service Point. An example is shown below.

   Copy

     container users {
       list user {
         key "name";
         description
           "This is a list of users in the system.";
       leaf name {
         type string;
         }
       leaf type {
         type string;
         }
       leaf full-name {
         type string;
         }
       }
     }

2. Click the Add Service Point CodeLens. This brings up the Add Service Point dialog.

3. Fill in the Service Point ID that is used to identify the service point, for example, mySimpleService.

4. Next, in the Python Details section, select using the Python Module field if you want to create a new Python module or use an existing one.

   • If you opt to create a new Python file, relevant sections are automatically updated in package-meta-data.xml.

   • If you select an existing Python module from the list, it is assumed that you are selecting the correct module and that, it has been created correctly, i.e., the package-meta-data.xml file is updated with the component definition.

5. Enter the Service CB Class, for example, SimpleServiceCB.

6. Finish creating the service by clicking Add Service Point.

Register an Action Point

All action points in a YANG model must be registered in NSO. Registering an action point also requires other parts of the files to be updated, for example, Python (register_action), and update package-meta-data if needed.

Action points are usually defined to lists or containers.

To register an action point:

1. Update your YANG model as required. The extension automatically detects the action point elements in YANG and displays a CodeLens called Add Action Point. An example is shown below.

   Copy

     ...
     container server {
         tailf:action ping {
         tailf:actionpoint pingaction;
           input {
             leaf destination {
               type inet:ip-address;
             }
           }
           output {
             leaf packet-loss {
               type uint8;
             }
           }
         }
       }

   Note that it is mandatory to specify tailf:actionpoint <actionpointname> under tailf:action <actionname>. This is a known limitation.

   The action point CodeLens at this time only works for the tailf:action statement, and not for the YANG rpc or YANG 1.1 action statements.

2. Click the Add Action Point CodeLens. This brings up the Register Action Point dialog.

3. Next, in the Python Details section, select using the Python Module field if you want to create a new Python module or use an existing one.

   • If you opt to create a new Python file, relevant sections are automatically updated in package-meta-data.xml.

   • If you select an existing Python module from the list, it is assumed that you are selecting the correct module, and that it has been created correctly, i.e., the package-meta-data.xml file is updated with the component definition.

4. Enter the action class name in the Main Class name used as entry point field, for example, MyAction.

5. Finish by clicking Register Action Point.

Edit Python Files

Opening a Python file uses the Microsoft Pylance extension. This extension provides syntax highlighting and other features such as code completion.

Add a New Python Module

To add a new Python module:

1. In the Primary Sidebar, Explorer view, right-click on the python folder.

2. Select NSO: Add Python Module from the pop-up. This brings up the Create Python Module dialog.

3. In the Create Python Module dialog, fill in the following details:

   • Module Name: Name of the module, for example, MyServicePackage.service.

   • Component Name: Name of the component that will be used to identify this module, for example, service.

   • Class Name: Name of the class to be invoked, for example, Main.

4. Click Finish.

Use Python Code Completion Snippets

Pre-defined snippets in VS Code allow for NSO Python code completion.

To use a Python code completion snippet:

1. Open a Python file for editing.

2. Type in one of the following pre-defined texts to display snippet options:

   • maapi: to view options for creating a maapi write transaction.

   • ncs: to view options for snippet for ncs template and variables.

3. Select a snippet from the pop-up to insert its code. This also highlights config items that can be changed. Press the Tab key to cycle through each value.

Edit XML Template Files

The final part of a typical service development is creating and editing the XML configuration template.

Add a New XML Template

To add a new XML template:

1. In the Primary Sidebar, Explorer view, right-click on the templates folder.

2. Select NSO: Add XML Template from the pop-up. This brings up the Add XML Template dialog.

3. In the Add XML Template dialog, fill in the XML Template name, for example, mspSimpleService.

4. Click Finish.

Use XML Code Completion Snippets

Pre-defined snippets in VS Code allow for NSO XML code completion of processing instructions and variables.

To use an XML code completion snippet:

1. Open an XML file for editing.

2. Type in one of the following pre-defined texts to display snippet options:

   • For processing instructions: <? followed by a character, for example <?i to view snippets for an if statement. All supported processing instructions are available as snippets.

   • For variables: $ followed by a character(s) matching the variable name, for example, $VA to view the variable snippet. Variables defined in the XML template via the <?set processing instruction or defined in Python code are displayed.

     Note: Auto-completion can also be triggered by pressing the Ctrl+Space keys.

3. Select an option from the pop-up to insert the relevant XML processing instruction or variable. Items that require further configuration are highlighted. Press the Tab key to cycle through the items.

XML Code Validation

The NSO Developer extension also performs code validation wherever possible. The following warning and error messages are shown if the extension is unable to validate the code:

• A warning is shown if a user enters a variable in an XML template that is not detected by the NSO Developer extension.

• An error message is shown if the ending tags in a processing instruction do not match.

Limitations

The extension provides help on a best-effort basis by showing error messages and warnings wherever possible. Still, in certain situations, code validation is not possible. An example of such a limitation is when the extension is not able to detect a template variable that is defined elsewhere and passed indirectly (i.e., the variable is not directly called).

Consider the following code for example, where the extension will successfully detect that a template variable IP_ADDRESS has been set.

vars.add('IP_ADDRESS','192.168.0.1')

Now consider the following code. While it serves the same purpose, it will fail to be detected.

ip_add_var_name = 'IP_ADDRESS' vars.add(ip_add_var_name, '192.168.0.1')

NSO Developer Studio - Explorer Extension

This section describes the installation and functionality of the NSO Explorer extension.

The purpose of the NSO Explorer extension is to allow the user to connect to a running instance of NSO and navigate the CDB from within VS Code.

System Requirements

To get started with the NSO Explorer extension, ensure that the following prerequisites are met on your system. The prerequisites are not a requirement to install the NSO Explorer extension, but for NSO development after the extension is installed.

• Visual Studio Code.

• Java JDK 11 or higher.

• Python 3.9 or higher (recommended).

Install the Extension

Installation of the NSO Explorer extension is done via the VS Code marketplace.

To install the NSO Explorer extension in your VS Code environment:

1. Open VS Code and click the Extensions icon on the Activity Bar.

2. Search for the extension using the keywords "nso developer studio" in the Search Extensions in Marketplace field.

3. In the search results, locate the extension (NSO Developer Studio - Explorer) and click Install.

4. Wait while the installation completes. A notification at the bottom-right corner indicates that the installation has finished. After the installation, an NSO icon is added to the Activity Bar.

Connect to NSO Instance

The NSO Explorer extension allows you to connect to and inspect a live NSO instance from within the VS Code. This procedure assumes that you have not previously connected to an NSO instance.

To connect to an NSO instance:

1. In the Activity Bar, click the NSO icon to open NSO Explorer.

2. If no NSO instance is already configured, a welcome screen is displayed with an option to add a new NSO instance.

3. Click the Add NSO Instance button to open the Settings editor.

4. In the Settings editor, click the link Edit in settings.json. This opens the settings.jsonfile for editing.

5. Next, edit the settings.json file as shown below:

   Copy

     "NSO.Instance": [
       {
         "host": "<hostname/ip>",
         "port": "<port>",
         "scheme": "http|https",
         "username": "<username>",
         "password": "<password>"
       }
     ]

6. Save the file when done.

   If settings have been configured correctly, NSO Explorer will attempt to connect to the running NSO instance and display the NSO configuration.

Inspect the CDB Tree

Once the NSO Explorer extension is configured, the user can inspect the CDB tree.

To inspect the CDB tree, use the following functions:

• Get Element Info: Click the i (info) icon on the Explorer bar, or alternatively inline next to an element in the Explorer view.

• Copy KeyPath: Click the {KP} icon to copy the keypath for the selected node.

• Copy XPath: Click the {XP} icon to copy the XPath for the selected node.

• Get XML Config: Click the XML icon to retrieve the XML configuration for the selected node and copy it to the clipboard.

If data has changed in NSO, click the refresh button at the top of the Explorer pane to fetch it.

Visit the link below to learn more.

Phased Provisioning is a Cisco NSO add-on package for scheduling provisioning tasks. Initially designed for gradual service rollout, it leverages NSO actions to give you more fine-grained control over how and when changes are introduced into the network.

A common way of using NSO is by an operator performing an action through the NSO CLI, which takes place immediately. However, when you perform a large number of changes or other actions, you likely have additional requirements, such as:

• You want to limit how many changes or actions can run at the same time.

• You want to schedule changes or actions to run outside of business hours.

• One or two actions failing is fine, but if several of them fail, you want to stop provisioning and investigate.

Phased Provisioning allows you to do all of that and more. As the framework invokes standard NSO actions to do the actual work, you can use it not just for services provisioning but for NED migrations and other operations too.

Installation

The NSO Phased Provisioning binaries are available from and contain the phased-provisioning package. Add it to NSO in a manner suitable for your installation. This usually entails copying the package file to the appropriate packages/ folder and performing a package reload. If in doubt, please refer to the NSO product documentation on package management.

To verify the status of the package on your NSO instance, run the show packages package phased-provisioning command.

If you later wish to uninstall, simply remove the package from NSO, which will also remove all Phased-Provisioning-specific configuration and data. It is highly recommended that you make a backup before removing the package, in case you need to restore or reference the data later.

Quickstart

After adding the package, Phased Provisioning does not require any special configuration and you can start using it right away. All you need is an NSO action that you want to use it with. In this Quickstart, that will be the device NED migrate action, which is built into NSO.

The goal is to migrate a number of devices from router-nc-1.0 NED to router-nc-1.1. One way of doing this is with the /devices/migrate action all at once, or by manually invoking the /devices/device/migrate action on each device with the new-ned-id parameter as:

admin@ncs# devices device <some-device> migrate new-ned-id router-nc-1.1

Create a Task

However, considering you want to achieve a phased (staggered) rollout, create a Phased Provisioning task to instruct the framework of the actions that you want to perform:

admin@ncs# config
admin@ncs(config)# phased-provisioning task run_ned_migrate
admin@ncs(config-task-run_ned_migrate)# target /devices/device
admin@ncs(config-task-run_ned_migrate)# action action-name migrate
admin@ncs(config-task-run_ned_migrate)# action variable new-ned-id value router-nc-1.1
admin@ncs(config-variable-new-ned-id)# show configuration
phased-provisioning task run_ned_migrate
 target /devices/device
 action action-name migrate
 action variable new-ned-id
  value router-nc-1.1
 !
!

This configuration defines a task named run_ned_migrate. It also defines a target value (that is an instance identifier) to select the nodes on which you want to run the action.

You provide the action name with the action/action-name value and set any parameters that the action requires. The name of the parameter can be set through variable/name and the value of the parameter can be set through any one of the below:

• variable/value for the string value of the parameter.

• variable/expr for XPath expression (value is determined through XPath calculation with respect to nodes filtered by target and filter or the target-nodes defined while running the task).

Here, the single argument is new-ned-id with the value of router-nc-1.1.

If the action has an input empty leaf, then you can only set variable/name without defining any value, for example, device sync-from action with no-wait-for-lock flag.

In the current configuration, the action will run on all the devices. This is likely not what you want and you can further limit the nodes using an XPath expression through a filter value, for example, to only devices that currently use the router-nc-1.0 NED:

admin@ncs(config-task-run_ned_migrate)# filter device-type/netconf/ned-id='router-nc-1.0:router-nc-1.0'

If you want to run an action on heterogeneous nodes which may not be determined from a single target and filter, then you can define a task without target and filter values. But, while running the task, you must dynamically set the nodes in target-nodes of run action, described later in this document.

Note: Please check the description for /phased-provisioning/task/action/action-name regarding the conditions to determine action execution status.

Create a Policy for the Task

In addition to what the task will do, you also need to specify how and when it will run. You do this with a Phased Provisioning policy:

admin@ncs(config)# phased-provisioning policies policy one_by_one
admin@ncs(config-policy-one_by_one)# batch size 1
admin@ncs(config-policy-one_by_one)# error-budget 1
admin@ncs(config-policy-one_by_one)# schedule immediately
admin@ncs(config-policy-one_by_one)# show configuration
phased-provisioning policies policy one_by_one
 schedule immediately
 batch size 1
 error-budget 1
!

The "one_by_one" policy, as it is named in this example, will run one migration at a time (batch/size), with an error-budget of 1, meaning the task will stop as soon as more than one migration fails. The value for schedule is immediately, which means as soon as possible after you submit this task for processing. Instead, you could also schedule it for a particular time in the future, such as Saturday at 1 a.m.

Finally, configure the task to use this policy:

admin@ncs(config)# phased-provisioning task run_ned_migrate
admin@ncs(config-task-run_ned_migrate)# policy one_by_one
admin@ncs(config-task-run_ned_migrate)# commit
admin@ncs(config-task-run_ned_migrate)# end

Run the Task

Having committed the task, you must also submit it to the scheduler if you want it to run. Use the /phased-provisioning/task/run action to do so:

admin@ncs# phased-provisioning task run_ned_migrate run

If the task does not already have a target set, you must pass dynamic nodes in target-nodes, for example:

admin@ncs# phased-provisioning task upgrade run target-nodes [ /devices/device{cisco-8201} /devices/device{ncs-5500} /custom-service{nexus} ]

Note: The selected target-nodes must support invoking the selected action or self-test action with the provided parameters, as defined in the task.

View the Task Status

You can observe the status of the task with the show phased-provisioning task-status command, such as:

admin@ncs# show phased-provisioning task-status run_ned_migrate
phased-provisioning task-status run_ned_migrate
 state                  completed
 reason                 "All scheduled requests are processed."
 current-error-budget   1
 allocated-error-budget 1
 completed-nodes /ncs:devices/device{ex0}
 completed-nodes /ncs:devices/device{ex1}
 completed-nodes /ncs:devices/device{ex2}

Brief View of Task Status

With many items (nodes) in the task, the output could be huge and you might want to use the brief action instead (note that there is no show in the command now):

admin@ncs# phased-provisioning task-status run_ned_migrate brief

Resume a Suspended task

In case enough actions fail, the error budget runs out and the execution stops:

phased-provisioning task-status run_ned_migrate
 state                  suspended
 reason                 "Phased provisioning has exceeded the maximum number of errors allowed."
 current-error-budget   -1
 allocated-error-budget 1
 pending-nodes /ncs:devices/device{ex2}
 failed-nodes /ncs:devices/device{ex0}
  failure-reason "external error (19): Trying to migrate to the NED identity already configured"
 failed-nodes /ncs:devices/device{ex1}
  failure-reason "external error (19): Trying to migrate to the NED identity already configured"

To restart processing, use the /phased-provisioning/task/resume action, allowing more errors to accumulate (if you reset the error budget) or not:

admin@ncs# phased-provisioning task run_ned_migrate resume reset-error-budget true

Pause a Task

You can temporarily pause an in-progress task, such as when you observe a problem and want to intervene to avoid additional failures.

Use the /phased-provisioning/task/pause action for pausing a task. This will suspend the task with an appropriate reason. You can later restart the task by executing the /phased-provisioning/task/resume action.

admin@ncs# phased-provisioning task run_ned_migrate pause

The task will be suspended with a reason as observed in task-status.

phased-provisioning task-status run_ned_migrate
 state                  suspended
 reason                 "Task is paused by user."
 current-error-budget   0
 allocated-error-budget 1
 pending-nodes /ncs:devices/device{ex2}
 completed-nodes /ncs:devices/device{ex0}
 failed-nodes /ncs:devices/device{ex1}
  failure-reason "external error (19): Trying to migrate to the NED identity already configured"

Retry Failed Nodes

If you want to re-try running the task for the failed nodes, use the/phased-provisioning/task/retry-failures action. This will move the failed nodes back to pending, so that, the nodes can be re-executed again. You can also re-execute specific failed nodes by specifying these in failed-nodes input of retry-failures action. This action does not change the error-budget.

To retry all failed nodes:

admin@ncs# phased-provisioning task run_ned_migrate retry-failures

To retry specific failed nodes:

admin@ncs# phased-provisioning task run_ned_migrate retry-failures failed-nodes [ /devices/device{ex1} ]

If the task has already completed, then after executing this action, the task will be marked suspended with appropriate reason. Then you can resume the task again to retry the failed nodes.

phased-provisioning task-status run_ned_migrate
 state                  suspended
 reason                 "Failed nodes are moved back to pending. Resume task to retry."
 current-error-budget   0
 allocated-error-budget 1
 pending-nodes /ncs:devices/device{ex1}
 completed-nodes /ncs:devices/device{ex0}
 failed-nodes /ncs:devices/device{ex2}
  failure-reason "external error (19): Trying to migrate to the NED identity already configured"

Phased Service Provisioning

While great for running actions, you can also use this functionality to provision (or de-provision) services in a staged/phased manner. There are two steps to achieving this:

• First, configure service instances as you would normally, but commit the changes with the commit no-deploy command.

• Second, configure a Phased Provisioning task to invoke the reactive-re-deploy action for these services, taking advantage of all the Phased Provisioning features.

Here is an example of a trivial static-dns service.

admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# static-dns ex0 dns 10.0.0.1
admin@ncs(config-static-dns-ex0)# static-dns ex1 dns 10.1.0.1
admin@ncs(config-static-dns-ex1)# static-dns ex2 dns 10.2.0.1
admin@ncs(config-static-dns-ex2)# top
admin@ncs(config)# commit no-deploy
Commit complete.
admin@ncs(config)# exit

You can verify that using the commit no-deploy did not result in any device configuration yet:

admin@ncs# show static-dns * modified
                         LSA
NAME  DEVICES  SERVICES  SERVICES
-----------------------------------
ex0   -        -         -
ex1   -        -         -
ex2   -        -         -

admin@ncs#

Then, create a task for phased provisioning, using the one_by_one policy from Quickstart:

admin@ncs(config)# phased-provisioning task deploy-dns
admin@ncs(config-task-deploy-dns)# target /static-dns
admin@ncs(config-task-deploy-dns)# filter starts-with(name,'ex')
admin@ncs(config-task-deploy-dns)# action action-name reactive-re-deploy
admin@ncs(config-task-deploy-dns)# policy one_by_one
admin@ncs(config-task-deploy-dns)# show configuration
phased-provisioning task deploy-dns
 target /static-dns
 filter starts-with(name,'ex')
 action action-name reactive-re-deploy
 policy one_by_one
!
admin@ncs(config-task-deploy-dns)# commit
admin@ncs(config-task-deploy-dns)# end

Finally, start the task:

admin@ncs# phased-provisioning task deploy-dns run

You can follow the task's progress with the following show command:

admin@ncs# show phased-provisioning task-status deploy-dns | repeat 1

Note: This command will refresh the output every second, stop it by pressing Ctrl+c.

Custom Tests for Provisioning Validation

For simple services, such as the preceding static-dns, successfully updating device configuration may be a sufficient indicator that the service was deployed without problems. For more complex services, you typically want to run additional tests to ensure everything went according to plan. Such services will often have a self-test action that performs this additional validation.

Phased Provisioning allows you to run custom verification, whether you are deploying services or doing some other type of provisioning. You can configure this under self-test container in the task configuration.

Please check the description for /phased-provisioning/task/self-test/action-name regarding the restrictions applied for action validation.

For example, the following commands will configure the service self-test action for validation.

admin@ncs(config)# phased-provisioning task deploy-dns
admin@ncs(config-task-deploy-dns)# self-test action-name self-test

Alternatively, you can use self-test/test-expr with an XPath expression, which must evaluate to a true value.

Setting Start Time

In addition to an immediately scheduled policy, you can opt for a policy with future scheduling. This allows you to set a (possibly recurring) time when provisioning takes place.

You can set two separate parameters:

• time: Configures at what time to start, in the Vixie-style cron format (further described below).

• window: Configures for how long after the start time new items can start processing.

Using both of these parameters enables you to limit the execution of a task to a particular time of day, such as when you have a service window. If there are still items in the task after the current window has passed, the system will wait for the next occurrence to process the remaining items.

The format for the time parameter is as follows:

---------- minute (0 - 59)
| ---------- hour (0 - 23)
| | ---------- day of month (1 - 31)
| | | ---------- month (1 - 12) | (jan - dec)
| | | | ---------- day of week (0 - 6) | (sun - sat)
| | | | |
* * * * *

Each of the asterisks (*) represents a field, which can take one of the following values:

• A number, such as 5.

• A number range, such as 5-10.

• An asterisk *, meaning any. For example, 0-59 and * are equivalent for the first (minute) field.

Each of these values can further be followed by a slash (/) and a number, denoting a step. For example, if used in the first field, */3 means every third minute instead of every minute (* only).

A number, range, and step can also be combined together with a comma (,) for each of these values. For example, if used in the first field, 5,10-13,20,25-28,*/15 means at minute 5, every minute from 10 through 13, at minute 20, every minute from 25 through 28, and every 15th minute.

Updating policy

You can update a policy used in a task irrespective of the task's running status (init, in-progress, completed, or suspended).

• Updating a completed task's policy will not impact anything.

• If an init task's policy schedule is updated to immediately, then the task will start executing batches immediately. Change to error-budget will also be reflected immediately. Change to batch-size or schedule/future/time or schedule/future/window will only reflect when the task starts as per the new schedule time.

• If a suspended task's policy is updated, then the changes will be reflected upon resuming the task.

• For an in-progress task,

  • If the policy schedule updated from immediately to schedule/future/time or schedule/future/time changed to a new time, then after the completion of the current batch, the next batch execution will be stopped and scheduled as per the new schedule time.

  • If the policy schedule updated from schedule/future/time to immediately, the task will continue to run till it completes.

  • Update to batch-size or schedule/future/window will be reflected upon the next batch execution after the current batch completion.

  • Update to error-budget will be reflected immediately to allocated-error-budget whereas the current-error-budget is adjusted depending on previously failed nodes.

Security Considerations

Phased Provisioning tasks perform no access checks for the configured actions. When a user is given access to the Phased Provisioning feature through NACM, they can implicitly invoke any action in NSO. That is, even if a user can't access an action directly, they can configure a task that invokes this action.

To amend this behavior, you can wrap Phased Provisioning functionality with custom actions or services and in this way limit available actions.

Tasks with future-scheduled policies make use of the NSO built-in scheduler functionality, which runs the task as the user that submitted it for scheduling (the user that invoked the run action on the task). If external authentication or PAM supplies the user groups for this user or you explicitly set groups using the ncs_cli -g command when connecting, the scheduling may fail.

This happens if the admin user is not mapped to a group with sufficient NACM permissions in NSO, such as in the default system-install configuration.

To address this issue, add the "admin" user to the correct group, using the /nacm/groups/group/user-name configuration. Instead of "admin", you can choose a different user with the /phased-provisioning/local-user setting. In any case, this user must have permission to invoke actions on the /cisco-pdp:phased-provisioning/task/ node. For example:

admin@ncs(config)# nacm groups group ncsadmin user-name admin

As a significantly less secure alternative, you can change the default for a user without a matching group by using the /nacm/exec-default setting.

Further Reading

• The phased-provisioning data model in phased-provisioning/src/yang/cisco-phased-provisioning.yang.

Visit the link below to learn more.

Deploying Cisco NSO on Kubernetes offers numerous advantages, including consistent deployments, self-healing capabilities, and better version control. This document outlines best practices for deploying NSO on Kubernetes to ensure optimal performance, security, and maintainability.

Prerequisites

Kubernetes Cluster

• Version Compatibility: Ensure that your Kubernetes cluster is within the three most recent minor releases to maintain official support.

• Persistent Storage: Install a Container Storage Interface (CSI) if not using a managed Kubernetes service. Managed services like EKS on AWS or GKE on GCP handle this automatically.

• Networking: Install a Container Network Interface (CNI) such as Cilium, Calico, Flannel, or Weave. Additionally, configure an ingress controller or load balancer as needed to expose services.

• TLS Certificates: Use TLS certificates for HTTPS access and to secure communication between different NSO instances. This is crucial for securing data transmission.

Deployment Architecture

Namespace Design

• Isolation: Run NSO in its own namespace to isolate its resources (pods, services, secrets, and so on.) from other applications and services in the cluster. This logical separation helps manage resources and apply specific RBAC policies.

Pod Design

• Stateful Pods: Use StatefulSets for production deployments to ensure that each NSO pod retains its data across restarts by mounting the same PersistentVolume. StatefulSets also provide a stable network identity for each pod.

• Data Persistence: Attach persistent volumes to NSO pods to ensure data persistence. Avoid using hostPath volumes in production due to security risks.

Service Design

• Service Types:

  • ClusterIP: Use for internal communications between NSO instances or other Kubernetes resources.

  • NodePort: Use for testing purposes only, as it exposes pods over the address of a Kubernetes node.

  • LoadBalancer: Use for external access, such as exposing SSH/NETCONF ports.

• Ingress Controllers: Use Ingress for managing external access to HTTP or HTTPS traffic. For more advanced routing capabilities, consider using the Gateway API.

Storage Design

Volume Management

• Persistent Volumes: Use PersistentVolumeClaims to manage storage and ensure that critical directories like NSO running directory, packages directory, and logs directory persist through restarts.

• NSO Directories: Mount necessary directories, such as the NSO running directory, packages directory, and logs directory to persistent volumes.

• Avoid HostPath: Refrain from using hostPath volumes in production environments, as they expose NSO data to the host system and add maintenance overhead.

Deployment Strategies

YAML Manifests

• Version Control: Define Kubernetes objects using YAML manifests and manage them via version control. This ensures consistent deployments and easier rollback capabilities.

• ConfigMaps and Secrets: Use ConfigMaps for non-sensitive configuration files and Secrets for sensitive data like Docker registry credentials. ConfigMaps are used to manage NSO configuration files, while Secrets can store sensitive information such as passwords and API keys. In NSO, the sensitive data that should go into Secrets is, for example, encryption keys for the CDB.

Helm Charts

• Simplified Deployment: Use Helm charts for packaging YAML manifests, simplifying the deployment process. Manage deployment parameters through a values.yaml file.

• Custom Configuration: Expose runtime parameters, service ports, URLs, and other configurations via Helm templates. Helm charts allow for more dynamic and reusable configurations.

Security Considerations

Running as Non-Root

• SecurityContext: Limit the Linux capabilities that are allowed for the NSO container and avoid running containers as the root user. This can be done by defining a SecurityContext in the Pod specification.

• Custom Dockerfile: Create a Dockerfile to add a non-root user and adjust folder permissions, ensuring NSO runs as a dedicated user. This can help in adhering to the principle of least privilege.

Network Policies

• Ingress and Egress Control: Implement network policies to restrict access to NSO instances and managed devices. Limit the communication to trusted IP ranges and namespaces.

• Service Accounts: Create dedicated service accounts for NSO pods to minimize permissions and reduce security risks. This ensures that each service account only has the permissions it needs for its tasks.

Monitoring & Logging

Observability Exporter

• Setup: Transform Docker Compose files to Kubernetes manifests using tools like Kompose. Deploy the observability exporter to export data in industry-standard formats such as OpenTelemetry.

• Container Probes: Implement readiness probes to monitor the health and readiness of NSO containers. Use HTTP checks to ensure that the NSO API is operational. Probes can help in ensuring that the application is functioning correctly and can handle traffic.

Scaling & Performance Optimization

Resource Requests & Limits

• Resource Management: Define resource requests and limits for NSO pods to ensure appropriate CPU and memory allocation. This helps maintain cluster stability and performance by preventing any single pod from using excessive resources.

Affinity & Anti-Affinity

• Pod Distribution: Use affinity and anti-affinity rules to ensure optimal distribution of NSO pods across worker nodes. This helps in achieving high availability and resilience by ensuring that pods are evenly distributed across nodes.

High Availability & Resiliency

Raft HA

• Setup: Configure a three-node Raft cluster for high availability. Ensure that each node has a unique pod and network identity, as well as its own PersistentVolume and PersistentVolumeClaim.

• Annotations: Use annotations to direct requests to the primary NSO instance. Implement sidecar containers to periodically check and update the Raft HA status. This ensures that the primary instance is always up and running.

Backup & Disaster Recovery

NSO Backup

• Automated Backups: Use Kubernetes CronJobs to automate regular NSO backups. Store the backups securely and periodically verify them.

• Disaster Recovery: Ensure that NSO backups are stored in a secure location and can be restored in case of cluster failure. Use temporary container instances to restore backups without running NSO.

Upgrade & Maintenance

Upgrading NSO

• Persistent Storage: Ensure that the NSO running directory uses persistent storage to maintain data integrity during upgrades.

• Testing: Test upgrades on a dummy instance before applying them to production. Clone the existing PVC and spin up a new NSO instance for testing.

• Rolling Upgrades: Update the container image version in YAML manifests or Helm charts. Delete the old NSO pods to allow Kubernetes to deploy the new ones. This minimizes downtime and ensures a smooth transition to the new version.

Cluster Maintenance

• Rolling Upgrades: Perform rolling node upgrades to minimize downtime and ensure high availability. Ensure the compatibility with Kubernetes API and resource definitions before upgrading.

• Node Draining: Drain and cordon nodes to safely migrate NSO instances during maintenance. This helps in ensuring that the cluster remains functional during maintenance activities.

Conclusion

By adhering to these best practices, you can ensure a robust, secure, and efficient deployment of Cisco NSO on Kubernetes. These guidelines help maintain operational stability, improve performance, and enhance the overall manageability of your Kubernetes deployments. Implementing these practices will help in achieving a reliable and scalable Kubernetes environment for NSO.

The official covers:

• Product Information

  • At-a-glance

  • Datasheets

  • End-of-life and end-of-sale notices

• Security Notices

  • Bulletins

  • Security advisories

  • Field notices

• Troubleshooting and Support Information

• Product Literature

  • Case studies

  • Solution overviews

  • White papers

API Updates

To stay updated with the latest releases, refer to the file available on .

Community

If you have generic questions, chances are big that someone in the NSO Community can help you out.

Technical Assistance

The Cisco TAC team can support you with questions regarding NSO. The first file below will instruct you on how to open a service request. The second file includes more information about what needs to be included and explains more about why that information is needed. The third file is overall information from Cisco TAC on how to get support.

Opening an NSO Service Request with Cisco TAC

Detailed Process for NSO and NED Support

Cisco TAC Support Guidelines

Postman Collections are a way to group API requests together to explore the APIs and automate common tasks.

Click the following Run in Postman button to launch the NSO API Postman collection and try out the APIs in Postman.

Note: You must have a Postman account or create one to use the "Run in Postman" feature. You can use the web-based Postman app if you do not prefer to install it locally.

We have created a public organization on GitHub to ensure we have one common place to share code for all of us working with NSO.

Since the NSO Developer is a public GitHub everyone can fork and read the uploaded code. By using a pull request everyone can also contribute to existing projects.

If you want to contribute to a new project, please read the instructions below.

Licenses

The license tells you what rights you have that are provided by the copyright holder. It is important that the contributor fully understands what rights they are licensing and agrees to them. Sometimes the copyright holder isn't the contributor, such as when the contributor is doing work on behalf of a company.

To ensure that the criteria in the license are met, there is a need for a Developer Certificate of Origin (DCO) sign-off on all contributions. More information about that can be found below.

Contributing a Project on the NSO Developer GitHub

Getting Started

1. Create an account on github.com.

2. Create your own repository on github.com.

3. Make sure that your project fulfills all the criteria listed below (under “Requirements on your project”).

That’s it! When the move is done, your repository is now part of the NSO developer. Keep hacking on your project, you will still have owner privileges, and as such you can decide to give others write access for example.

Users of your repository can use Issues to report bugs and suggest new features, and Pull Requests to contribute code.

Expectations

When using packages from the library you can expect the following:

• The code you find is provided “as is” and when you use it you get to keep it. If it breaks, you get to keep all the pieces.

• If you extend a project or fix bugs, please contribute back in the form of pull requests.

• When contributing you can expect:

• The code you contribute is made available to everyone in source form “as is”.

• Your code might be used to: teach others about NSO, build new products, and provide a starting point for customer projects.

• At the very minimum, your contribution should have a title and a short README.md explaining what it does, see the full list of requirements here

• You are not required to support your contributed code, but please consider pull requests and try to answer questions in the project wiki

• Only contribute code for which you own the IPR

• Do not include explicit references to customers (be it customer names, network configuration/templates, or otherwise)

Requirements for your Project

Before your project can be accepted as a repository of the NSO Developer it needs to fulfill the following criteria.

Developer Certificate of Origin

Signed-off

When using the NSO-develop-hub every commit needs to be signed off (git commit –s) by the contributor that he or she has the right to submit it. The -s flag adds a line with the text 'Signed-off-by' followed by the same email address as the contributor. E.g.:

My commit message

Signed-off-by Aron Aronsson <aron.aronsson@example.com>

It is important that the git config user.name and user.email is configured correctly.

What is Signed-off

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Signing Pull Requests

When creating a pull request every commit in that pull request will be checked for DCO, if you haven’t signed all commits the checks will fail and the pull request will be denied.

Therefore it is a good idea to sign all commits before doing the pull request.

It should be NSO-related

Sure, it could be a cool YANG plugin too - but it should at least be relevant to NSO development.

Naming

Choose a name. A good name. A catchy name, a nerdy name, a happy name - you decide. This is especially important if your contribution is a tool or a reusable library. In that case, it doesn’t even have to be descriptive. Better YxT than “YANG Extension Tool”. Don’t pick “generic-tool”, “misc-template”…

If your contribution is more of a demo or example, then a more descriptive name could be in order, perhaps even pre- or postfixed with a demo or example. For example: l3-vpn-demo or example-stacked-service.

README.md

Add a README.md. Your README must include:

• A brief explanation of what your project does.

• List dependencies (build and runtime, for example, compilers, libraries, operating system).

• Instructions on how to build it.

• If your project contains any copies of code derived from open source you need to explicitly list which projects.

It Must be Open

The whole point of the NSO Developer space is to share code to the NSO ecosystem, as such we don’t want to make it “private”. However, that means that anyone can access the NSO Developer repositories, which requires us to approve the open access and ensure that no private information is included.

The information in the README.md file will be displayed on the Cisco NSO DevNet site.

Recommendations

• Add test cases and instructions on how to run them. Why not use Lux to automate your tests? NSO uses it!

• Packaging makes one repository for every stand-alone project. But don’t make a lot of small repositories of things that actually belong together, it just makes the space cluttered and it will be harder to find your project.

• The naming convention for YANG modules. For a demo or example the module name and namespace does not matter that much (you can use example.com/... as a namespace). But if your project is a reusable piece, then consider using the URL of the project the namespace (as in: github.com/NSO-developer/PACKAGE-NAME/MODULE-NAME).

• If you make some kind of release, consider tagging the releases and using a Changes file / Release Notes document.

The NSO Resource Manager package contains both an API for generic resource pool handling called the resource allocator, and the two applications ( and) utilizing the API. The applications are explained separately in the following sections below:

Background

NSO is often used to provision services in the networking layer. It is not unusual that these services require network-level information that is not (or cannot be) part of the instance data provided by the northbound system, so it needs to be fetched from, and eventually released back to a separate system. A common example of this is IP addresses used for layer-3 VPN services. The orchestrator tool is not aware of the blocks of IP addresses assigned to the network but relies on lower layers to fulfill this need.

Some customers have software systems to manage these types of temporary assets. E.g., for IP-addresses, they are usually known as IP Address Management (IPAM) systems. There is a whole industry of solutions for such systems, ranging from simple open-source solutions to entire suites integrated with DNS management. See for more on this.

There are customers that either don't have an IPAM system for services that are planned for NSO or are not planning to get one for this single purpose. They usually don't want the operational overhead of another system and/or don't see the need for a separate investment. These customers are looking for NSO to provide basic resource allocation and lifecycle management for the assets required for services managed by NSO. They appreciate the fact that NSO is not an appropriate platform to provide more advanced features from the IPAM world like capacity planning nor to integrate with DNS and DHCP platforms. This means that the NSO Resource Manager does not compete with full-blown systems but is rather a complementary feature.

Overview

The NSO Resource Manager interface, the resource allocator, provides a generic resource allocation mechanism that works well with services and in a high availability (HA) configuration. Expected is implementations of specific resource allocators implemented as separate NSO packages. A service will then have the possibility to use allocator implementations dedicated to different resources.

The YANG model of the resource allocator (resource-allocator.yang) can be augmented with different resource pools, as is the case for the two applications id-allocator and ipaddress-allocator. Each pool has an allocation list where services are expected to create instances to signal that they request an allocation. Request parameters are stored in the request container and the allocation response is written in the response container.

Since the allocation request may fail the response container contains a choice where one case is for error and one for success.

Each allocation list entry also contains an allocating-service leaf-list. These are instance identifiers that point to the services that requested the resource. These are the services that will be redeployed when the resource has been allocated.

The resource allocation packages should subscribe to several points in this resource-pool tree. First, they must detect when a new resource pool is created or deleted. Secondly, they must detect when an allocation request is created or deleted. A package may also augment the pool definition with additional parameters, for example, an IP address allocator may wish to add configuration parameters for defining the available subnets to allocate from, in which case it must also subscribe to changes to these settings.

Installation

Data Model for Resource Allocator

The API of the resource allocator is defined in this YANG data model:

  grouping resource-pool-grouping {
    leaf name {
      tailf:info "Unique name for the pool";
      type string;
    }

    list allocation {
      key id;

      leaf id {
        type string;
      }

      leaf username {
        description
          "Authenticated user for invoking the service";
        type string;
        mandatory true;
      }

      leaf-list allocating-service {
        tailf:info "Instance identifiers of service that own resource";
        type instance-identifier;
      }

      container request {
        description
          "When creating a request for a resource the
           implementing package augments here.";
      }

      container response {
        config false;
        tailf:cdb-oper {
          tailf:persistent true;
        }
        choice response-choice {
          case error {
            leaf error {
              type string;
            }
          }
          case ok {
            // The implementing package augments here
          }
        }
      }
      }

HA Considerations

Looking at High Availability, there are two things we need to consider - the allocator state needs to be replicated, and the allocation needs only to be performed on one node.

The easiest way to replicate the state is to write it into CDB-oper and let CDB perform the replication. This is what we do in the ipaddress-allocator.

We only want the allocator to allocate addresses on the primary node. Since the allocations are written into CDB they will be visible on both primary and secondary nodes, and the CDB subscriber will be notified on both nodes. In this case, we only want the allocator on the primary node to perform the allocation.

We therefore read the HA mode leaf from CDB to determine which HA mode the current subscriber is running in; if HA mode is not enabled, or if HA mode is enabled and the current node is primary we proceed with the allocation.

Synchronous Allocation

This synchronized allocation API request uses a reactive fastmap, so the user can allocate resources and still keep a synchronous interface. It allocates resources in the create callback, at that moment everything we modify in the database is part of the service intent and fast map. We need to guarantee that we have used a stable resource and communicate to other services, which resources we have used. So, during the create callback, we store what we have allocated. Other services that are evaluated within the same transaction which runs subsequent to ours will see allocations, when our service is redeployed, it will not have to create the allocations again.

When an allocation raises an exception in case the pool is exhausted, or if the referenced pool does not exist in the CDB, commit will get aborted. Synchronous allocation doesn't require service re-deploy to read allocation. The same transaction can read allocation, commit dry-run or get-modification should show up the allocation details as output.

If the HA mode is not set to primary and the synchronization RM API is enabled, the restriction will be enforced, preventing IP or ID allocation and resulting in an exception being thrown to the user.

NSO ID Allocator Deployment

This section explores deployment information and procedures for the NSO ID Allocator (id-allocator). The NSO Resource ID Allocator is an extension of the generic resource allocation mechanism called the NSO Manager. It can allocate integers which can serve for instance as VLAN identifiers.

Overview

The ID Allocator can host any number of ID pools. Each pool contains a certain number of IDs that can be allocated. They are specified by a range, and potentially broken into several ranges by a list of excluded ranges.

The ID allocator YANG models are divided into a configuration data-specific model (idallocator.yang), and an operational data-specific model (id-allocator-oper.yang). Users of this package will request allocations in the configuration tree. The operational tree serves as an internal data structure of the package.

An ID request can allocate either the lowest possible ID in a pool or a specified (by the user) value, such as 5 or 1000.

Allocation requests can be synchronized between pools. This synchronization is based on the ID of the allocation request itself (such as for instance allocation1), the result is that the allocations will have the same allocated value across pools.

Examples

This section presents some simple use cases of the NSO presented using Cisco-style CLI.

admin@ncs# resource-pools id-pool pool1 range start 100 end 1000
admin@ncs# commit

admin@ncs# resource-pools id-pool pool1 allocation a1 user myuser
admin@ncs# commit

| NAME  | START | END | START | END | START | END  |  ID  |
|-------|-------|-----|-------|-----|-------|------|------|
| pool1 |   -   |  -  |       |     |  101  | 1000 | 100  |

admin@ncs# resource-pools id-pool pool1 allocation a1 allocating-service \
           /services/vl:loop[name='myservice1'] user myuser
admin@ncs# resource-pools id-pool pool1 allocation a1 allocating-service \
           /services/vl:loop[name='myservice2'] user myuser
admin@ncs# commit

admin@ncs# resource-pools id-pool pool2 range start 100 end 1000
admin@ncs# resource-pools id-pool pool1 allocation b user myuser request sync true
admin@ncs# resource-pools id-pool pool2 allocation b user myuser request sync true
admin@ncs# commit

| NAME  | START | END | START | END | START | END |  ID  |
|-------|-------|-----|-------|-----|-------|-----|------|
| pool1 |   -   |  -  |       |     |  101  | 999 | 100  |
|       |   -   |  -  |       |     |       |     | 1000 |
| pool2 |   -   |  -  |       |     |  101  | 999 | 1000 |

admin@ncs# set resource-pools id-pool testPool allocation testAlloc username admin
admin@ncs# commit
admin@ncs# set resource-pools id-pool testPool allocation testAlloc request id 150
admin@ncs# commit
admin@ncs# set resource-pools id-pool testPool allocation testAlloc request id 180
admin@ncs# commit

admin@ncs# set resource-pools id-pool methodRangeFirst allocation a username \
           admin request method firstfree

admin@ncs# set resource-pools id-pool methodRoundRobin allocation a username \
           admin request method roundrobin

Security

The NSO ID Allocator requires a username to be configured by the service application when creating an allocation request. This username will be used to redeploy the service application once a resource has been allocated. Default NACM rules deny all standard users access to the /ralloc:resource-pools list. These default settings are provided in the (initial_data/aaa_init.xml) file of the resource-manager package.

It is up to the administrator to add a rule that allows the user to perform the service re-deploy.

Alarms

There are two alarms associated with the ID Allocator:

• Empty Alarm: This alarm is raised when the pool is empty, and there are no available IDs for further allocation.

• Low threshold Reached Alarm: This alarm is raised when the pool is nearing empty, e.g., there is only 10% or fewer left in the pool.

CDB Upgrade from Package version below 4.0.0

Since the Resource Manager's version 4.0.0, the operational data model is not compatible with the previous version. In version 4.0.0 Yang model, there is a new element called allocationId added for /Id-allocator/pool/allocation to support sync ID allocation. The system will run the upgrade script automatically (when the Resource Manager of the new version is loaded) if there is a Yang model change in the new version. Users can also run the script manually for Resource Manager from 3.5.6 (or any version below 4.0.0) to version 4.0.0 or above; the script will add the missing allocationId element in the CDB operational data path /id-allocator/pool/allocation. The upgrade Python script is located in the Resource Manager package: python/resource_manager/rm_upgrade_nso.py.

The id-allocator-tool Action

A set of debug and data tools (contained in rm-action/id-allocator-tool action) is available to help admin or support to operate on RM data. Two parameters in the id-allocator-tool action can be provided: operation, pool. All the process info and results will be logged in ncs-java-vm.log, and the action itself just returns the result. Here is a list of the valid operation values for the id-allocator-tool action:

• check_missing_report: Scan the current resource pool and ID pool in the system, and identify and report the missing element for each id-allocator entry without fixing.

• fix_missing_allocation_id: Add the missing allocation ID for each ID allocator entry.

• fix_missing_owner: Add the missing owner info for each ID allocator entry.

• fix_missing_allocation: Create the missing allocation entry in the ID allocator for each ID pool allocation response/id.

• fix_response_id: Scan the ID pool and check if the allocation contains an invalid allocation request ID, and release the allocation from the ID pool if found. It happens for sync allocation when the device configuration fails after a successful ID allocation and then causes a service transaction fail. This leaves the ID pool containing successfully allocated ID while the allocation request response doesn't exist

• persistAll: Manually sync from ID pool in memory to ID allocator in CDB.

• printIdPool: Print the current ID pool data in ncs-java-vm.log for debugging purposes.

Action Usage Example

Note that when a pool parameter is provided, the operation will be on this specific ID pool, and if no pool is provided, the operation will be running on all ID pools in the system.

admin@ncs> unhide debug
admin@ncs> request rm-action id-allocator-tool operation fix_missing_allocation
admin@ncs> request rm-action id-allocator-tool operation printIdPool pool multiService

NSO IP Address Allocator Deployment

This section contains deployment information and procedures for the Tail-f NSO IP Adress Allocator (ipaddress-allocator) application.

Overview

The NSO IP Address Allocator application contains an IP address allocator that use the Resource Manager API to provide IP address allocation. It uses a RAM-based allocation algorithm that stores its state in CDB as oper data.

The file resource-manager/src/java/src/com/tailf/pkg/ipaddressallocator/IPAddressAllocator.java contains the part that deals with the resource manager APIs whereas the RAM-based IP address allocator resides under resource-manager/src/java/src/com/tailf/ pkg/ipam.

The IPAddressAllocator class subscribes to five points in the DB:

• /ralloc:resource-pools/ip-address-pool: To be notified when new pools are created/deleted. It needs to create/delete instances of the IPAddressPool class. Each instance of the IPAddressPool handles one pool.

• /ralloc:resource-pools/ip-address-pool/subnet: To be notified when subnets are added/removed from an existing address pool. When a new subnet is added, it needs to invoke the addToAvailable method of the right IPAddressPool instance. When a pool is removed, it needs to reset all existing allocations from the pool, create new allocations, and re-deploy the services that had the allocations.

• /ralloc:resource-pols/ip-address-pool/exclude: To detect when new exclusions are added and when old exclusions are removed.

• /ralloc:resource-pols/ip-address-pool/range: To be notified when ranges are added to or removed from an address pool.

• /ralloc:resource-pols/ip-address-pool/allocation: To detect when new allocation requests are added and when old allocations are released. When a new request is added, the right size of the subnet is allocated from the IPAddressPool instance and the result is written to the response/subnet leaf, and finally, the service is redeployed.

Examples

This section presents some simple use cases of the NSO IP Address Allocator. It uses the C-style CLI.

admin@ncs# resource-pools ip-address-pool pool1 subnet 10.0.0.0 24
admin@ncs# resource-pools ip-address-pool pool1 range 192.168.0.0 192.168.255.255

admin@ncs# resource-pools ip-address-pool pool1 allocation a1 username \
myuser request subnet-size 30

admin@ncs# resource-pools ip-address-pool pool1 allocation a1 allocating-service \
/services/vl:loop[name='myservice1'] user myuser request subnet-size 30
admin@ncs# resource-pools ip-address-pool pool1 allocation a1 allocating-service \
/services/vl:loop[name='myservice2'] user myuser request subnet-size 30
admin@ncs# commit

admin@ncs# resource-pools ip-address-pool pool1 allocation a2 username \
myuser request subnet-start-ip 10.0.0.36 subnet-size 32

admin@ncs# show resouce-pools

| NAME    | ID   | ERROR | SUBNET        | FROM          |
| ------- | ---- | ----- | ------------- | ------------- |
| `pool1` | `a1` | -     | `10.0.0.0/30` | `10.0.0.0/24` |

admin@ncs# resource-pools ip-address-pool pool1 allocation a1 allocating-service \
/services/vl:loop[name='myservice'] username myuser request subnet-size 30

Security

The NSO IP Address Allocator requires a username to be configured by the service applications when creating an allocation request. This username will be used to redeploy the service applications once a resource has been allocated. The default NACM rules deny all standard users access to the / ralloc:resource-pools list. These default settings are provided in the (initial_data/ aaa_init.xml) file of the Resource Manager package.

Alarms

There are two alarms associated with the IP Address Allocator:

• Empty Alarm: This alarm is raised when the pool is empty, and there are no available IPs that can be allocated.

• Low Threshold Reached Alarm: This alarm is raised when the pool is nearing empty, e.g., there are only 10% or fewer separate IPs left in the pool.

The ip-allocator-tool Action

A set of debug and data tools contained in the rm-action/ip-allocator-tool action is available to help the admin or support personnel to operate on the RM data. Two parameters in the ip-allocator-tool action can be provided: operation, pool. All the process info and the results will be logged in ncs-java-vm.log, and the action itself just returns the result. Here is a list of the valid operation values for the ip-allocator-tool action.

• fix_response_ip: Scan the IP pool to check if the allocation contains an invalid allocation request ID, and release the allocation from the IP pool, if found. It happens for sync allocation when the device configuration fails after a successful IP allocation and then causes a service transaction to fail. This leaves the IP pool to contain successfully allocated IP while the allocation request response doesn't exist.

• printIpPool: Print the current IP pool data in the ncs-java-vm.log for debugging purposes.

• fix_missing_allocation: Create the missing allocation entry in the IP allocator for each IP pool allocation response/IP.

• persistAll: Manually sync from IP pool in memory to IP allocator in CDB.

Action Usage Example

Note that when a pool parameter is provided, the operation will be on this specific IP pool.

admin@ncs> unhide debug
admin@ncs> request rm-action ip-allocator-tool operation fix_response_ip pool multiService
admin@ncs> request rm-action ip-allocator-tool operation printIpPool pool multiService
admin@ncs> request rm-action ip-allocator-tool operation fix_missing_allocation pool multiService
admin@ncs> request rm-action ip-allocator-tool operation persistAll pool multiService

NSO Resource Manager Data Models

This section covers the NSO Resource Manager data models.

module resource - allocator {
    namespace "http://tail-f.com/pkg/resource-allocator";
    prefix "ralloc";

    import tailf - common {
        prefix tailf;
    }
    import ietf - inet - types {
        prefix inet;
    }
    organization "Tail-f Systems";
    description
        "This is an API for resource allocators.
    An allocation request is signaled by creating an entry in the
    allocation list.
    The response is signaled by writing a value in the response
    leave(s).The responder is responsible
    for re - deploying the
    allocating owners after writing the result in the response
    leaf.

    We expect a specific allocator package to do the following:
        1. Subscribe to changes in the allocation list and look
    for create operations.
        2. Perform the allocation and respond by writing the result
        into the response leaf, and then invoke the re - deploy
        action of the services pointed to by the owners leaf - list.

    Most allocator packages will want to annotate this model with
    additional pool definition data.
    ";

    revision 2022 - 03 - 11 {
        description
            "support multi-service and synchronous allocation request.";
    }

    revision 2020 - 07 - 29 {
        description
            "1.1
        Enhancements:
            -Add 'redeploy-type'
        option
        for service redeploy action.
        If not provided, the 'default'
        is assumed, where the
        redeploy type is chosen based on NSO version.
        ";
    }
    revision 2015 - 10 - 20 {
        description
            "Initial revision.";
    }
    grouping resource - pool - grouping {
        leaf name {
            type string;
            description
                "The name of the pool";
            tailf: info "Unique name for the pool";
        }
        leaf sync - dryrun {
            tailf: hidden debug;
            config false;
            type empty;
        }
        list allocation {
            key id;
            tailf: info "contains all the details of a resource request made from user";
            leaf id {
                type string;
                description "allocation id";
                tailf: info "allocation id.";
            }
            leaf username {
                description
                    "Authenticated user for invoking the service";
                type string;
                mandatory true;
            }
            leaf - list allocating - service {
                type instance - identifier {
                    require - instance false;
                }
                description
                    "Points to the services that own the resource.";
                tailf: info "Instance identifiers of services that own resource";
            }
            leaf sync - alloc {
                tailf: hidden debug;
                tailf: info "process allocation in synchronous flow";
                type empty;
            }
            leaf redeploy - type {
                description "Service redeploy type:
                default, touch, reactive - re - deploy, re - deploy.
                ";
                type enumeration {
                    enum "default";
                    enum "touch";
                    enum "reactive-re-deploy";
                    enum "re-deploy";
                    enum "no-redeploy";
                }
                default "default";
            }
            container request {
                description
                    "When creating a request for a resource the
                implementing package augments here.
                ";
            }
            container response {
                config false;
                tailf: cdb - oper {
                    tailf: persistent true;
                }
                choice response - choice {
                    case error {
                        leaf error {
                            type string;
                            description
                                "Text describing why the allocation request failed";
                        }
                    }
                    case ok {}
                }
                description
                    "The response to the allocation request.";
            }
        }
    }
    container resource - pools {}
    container rm - action {
        tailf: action sync - alloc {
            tailf: hidden debug;
            tailf: actionpoint sync - alloc - action;
            input {
                leaf pool {
                    type string;
                }
                leaf allocid {
                    type string;
                }
                leaf user {
                    type string;
                }
                leaf cidrmask {
                    type uint8 {
                        range "1..128";
                    }
                }
                leaf invertcidr {
                    type boolean;
                }
                leaf owner {
                    type string;
                }
                leaf subnetstartip {
                    type string;
                }
                leaf dryrun {
                    type boolean;
                    default false;
                }
            }
            output {
                leaf allocated {
                    type string;
                    mandatory true;
                }
                leaf subnet {
                    type string;
                }
            }
        }
        tailf: action sync - alloc - id {
            tailf: actionpoint sync - alloc - id - action;
            input {
                leaf pool {
                    type string;
                }
                leaf allocid {
                    type string;
                }
                leaf user {
                    type string;
                }
                leaf owner {
                    type string;
                }
                leaf requestedId {
                    type int32;
                }
                leaf method {
                    type string;
                    default "firstfree";
                }
                leaf sync {
                    type boolean;
                    default false;
                }
                leaf dryrun {
                    type boolean;
                    default false;
                }
            }
            output {
                leaf allocatedId {
                    type string;
                    mandatory true;
                }
            }
        }
    }
}

module id - allocator {
    namespace "http://tail-f.com/pkg/id-allocator";
    prefix idalloc;
    import tailf - common {
        prefix tailf;
    }
    import resource - allocator {
        prefix ralloc;
    }
    include id - allocator - alarms {
        revision - date "2017-02-09";
    }
    organization "Tail-f Systems";
    description
        "This module contains a description of an id allocator for defining pools of id: s.This can
        for instance be used when allocating VLAN ids.
        This module contains configuration schema of the id allocator.For the
        operational schema, please see the id - allocator - oper module.
    ";
    revision 2023 - 11 - 16 {
        description
            "Add action id-allocator-tool.";
    }
    revision 2022 - 03 - 11 {
        description
            "support multi-service and synchronous allocation request.";
    }
    revision 2017 - 08 - 14 {
        description
            "2.2
        Enhancements:
            Removed 'disable', add 'enable'
        for alarms.
        This means that
        if you want alarms you need to enable this explicitly
        now.
        ";
    }
    revision 2017 - 02 - 09 {
        description
            "2.1
        Enhancements:
            Added support
        for alarms
            ";
    }
    revision 2015 - 12 - 28 {
        description "2nd revision. Added support for allocation methods.";
    }
    revision 2015 - 10 - 20 {
        description "Initial revision.";
    }
    grouping range - grouping {
        leaf start {
            type uint32;
            mandatory true;
        }
        leaf end {
            type uint32;
            mandatory true;
            must ". >= ../start" {
                error - message "range end must be greater or equal to range start";
                tailf: dependency "../start";
            }
        }
    }
    // This is the interface
    augment "/ralloc:resource-pools" {
        list id - pool {
            key "name";
            container range {
                description "The range the resource-pool should contain";
                uses range - grouping;
            }
            list exclude {
                tailf: info "list of id resource not available for allocation ";
                key "start end";
                leaf stop - allocation {
                    type boolean;
                    default "false";
                }
                uses range - grouping;
                tailf: cli - suppress - mode;
            }
            uses ralloc: resource - pool - grouping {
                augment "allocation/response/response-choice/ok" {
                    leaf id {
                        type uint32;
                        description "id from pool";
                        tailf: info "id from pool.";
                    }
                }
            }
            container alarms {
                leaf enabled {
                    type empty;
                    description "Set this leaf to enable alarms";
                }
                leaf low - threshold - alarm {
                    type uint8 {
                        range "0 .. 100";
                    }
                    default 10;
                    description "Change the value for when the low threshold alarm is
                    raised.The value describes the percentage IDs left in
                        the pool.The
                    default is to raise the alarm when there
                    are ten(10) percent IDs left in the pool.
                    ";
                }
            }
            description "The state of the id-pool.";
            tailf: info "Id pool";
        }
    }
    //augmenting the request/responses form resource-manager
    augment "/ralloc:resource-pools/id-pool/allocation/request" {
        leaf sync {
            type boolean;
            default "false";
            description "Synchronize allocation with all other allocation
            with same allocation id in other pools ";
            tailf: info "Synchronize allocation id with other pools";
        }
        leaf id {
            type uint32;
            description "The specific id to sync with";
            tailf: info "Request a specific id";
        }
        container method {
            choice method {
                default firstfree;
                case firstfree {
                    leaf firstfree {
                        type empty;
                        description "The default method to allocating a new id
                        is using the first free method.Using this
                        allocation method might mean that an id is reused
                        quickly which might not be what one wants nor is
                        supported in lower layers.
                        ";
                        tailf: info "Default method used to request a new id.";
                    }
                }
                case roundrobin {
                    leaf roundrobin {
                        type empty;
                        description "Pick the next available id using a round
                        robin approach.Earlier used id: s will not be
                        reused until the range is exhausted and allocation
                        restarts from the start of the range again.
                        Note that sync will override round robin.
                        ";
                        tailf: info "Round robin method used to request a new id.";
                    }
                }
            }
        }
    }
    augment "/ralloc:rm-action" {
        tailf: action id - allocator - tool {
            tailf: hidden debug;
            tailf: actionpoint id - allocator - tool - action;
            input {
                leaf pool {
                    type leafref {
                        path "/ralloc:resource-pools/idalloc:id-pool/name";
                    }
                }
                leaf operation {
                    type enumeration {
                        enum printIdPool;
                        enum check_missing_report;
                        enum fix_missing_allocation_id;
                        enum fix_missing_owner;
                        enum fix_missing_allocation;
                        enum fix_response_id;
                        enum persistAll;
                    }
                    mandatory true;
                }
            }
            output {
                leaf result {
                    type string;
                }
            }
        }
    }
}

module ipaddress - allocator {
    namespace "http://tail-f.com/pkg/ipaddress-allocator";
    prefix ipalloc;
    import tailf - common {
        prefix tailf;
    }
    import ietf - inet - types {
        prefix inet;
    }
    import resource - allocator {
        prefix ralloc;
    }
    include ipaddress - allocator - alarms {
        revision - date "2017-02-09";
    }
    organization "Tail-f Systems";
    description
        "This module contains a description of an IP address allocator for defining
        pools of IPs and allocating addresses from these.
        This module contains configuration schema of the ip allocator.For the
        operational schema, please see the ip - allocator - oper module.
        ";
    revision 2022 - 03 - 11 {
        description
            "support multi-service and synchronous allocation request.";
    }
    revision 2018 - 02 - 27 {
        description
            "Introduce the 'invert' field in the request container that enables
            one to allocate the same size network regardless of the network
            type(IPv4 / IPv6) in a pool by using the inverted cidr.
            ";
    }
    revision 2017 - 08 - 14 {
        description
            "2.2
        Enhancements:
            Removed 'disable', add 'enable'
        for alarms.
        This means that
        if you want alarms you need to enable this explicitly
        now.
        ";
    }
    revision 2017 - 02 - 09 {
        description
            "1.2
        Enhancements:
            Added support
        for alarms
            ";
    }
    revision 2016 - 01 - 29 {
        description
            "1.1
        Enhancements:
            Added support
        for defining pools using IP address ranges.
        ";
    }
    revision 2015 - 10 - 20 {
        description "Initial revision.";
    }
    // This is the interface
    augment "/ralloc:resource-pools" {
        list ip - address - pool {
            tailf: info "IP Address pools";
            key name;
            uses ralloc: resource - pool - grouping {
                augment "allocation/request" {
                    leaf subnet - size {
                        tailf: info "Size of the subnet to be allocated.";
                        type uint8 {
                            range "1..128";
                        }
                        mandatory true;
                    }
                    leaf subnet - start - ip {
                        description
                            "Optional parameter used to request for a particular IP for
                        the allocation(instead of the first available subnet matching the subnet - size).The subnet - start - ip has to be the first IP in the range defined by subnet - size, otherwise the allocation will
                        fail.Ex: subnet - start - ip 10.0 .0 .36 subnet - size 30 gives the
                        equivalent range 10.0 .0 .36 - 10.0 .0 .39, and it 's a valid value.
                        subnet - start - ip 10.0 .0 .36 subnet - size 29 gives the range
                        10.0 .0 .32 - 10.0 .0 .39 and it 's NOT a valid option.";
                        type inet: ip - address;
                    }
                    leaf invert - subnet - size {
                        description
                            "By default subnet-size is considered equal to the cidr, but by
                        setting this leaf the subnet - size will be the\ "inverted\" cidr.
                        I.e: If one sets subnet - size to 8 with this leaf unset 2 ^ 24
                        addresses will be allocated
                        for a IPv4 pool and in a IPv6 pool
                        2 ^ 120 addresses will be allocated.By setting this leaf only
                        2 ^ 8 addresses will be allocated in either a IPv4 or a IPv6
                        pool.
                        ";
                        type empty;
                    }
                }
                augment "allocation/response/response-choice/ok" {
                    leaf subnet {
                        type inet: ip - prefix;
                    }
                    leaf from {
                        type inet: ip - prefix;
                    }
                }
            }
            leaf auto - redeploy {
                tailf: info "Automatically re-deploy services when an IP address is " +
                    "re-allocated";
                type boolean;
                default "true";
            }
            list subnet {
                key "address cidrmask";
                tailf: cli - suppress - mode;
                description
                    "List of subnets belonging to this pool. Subnets may not overlap.";
                must "(contains(address, '.') and cidrmask <= 32) or
                    (contains(address, ':') and cidrmask <= 128)
                " {
                error - message "cidrmask is too long";
            }
            tailf: validate ipa_validate {
                tailf: dependency ".";
            }
            leaf address {
                type inet: ip - address;
            }
            leaf cidrmask {
                type uint8 {
                    range "1..128";
                }
            }
        }
        list exclude {
            key "address cidrmask";
            tailf: cli - suppress - mode;
            description "List of subnets to exclude from this pool. May only " +
                "contains elements that are subsets of elements in the list of " +
                "subnets.";
            tailf: info "List of subnets to exclude from this pool. May only " +
                "contains elements that are subsets of elements in the list of " +
                "subnets.";
            must "(contains(address, '.') and cidrmask <= 32) or
                (contains(address, ':') and cidrmask <= 128)
            " {
            error - message "cidrmask is too long";
        }
        tailf: validate ipa_validate {
            tailf: dependency ".";
        }
        leaf address {
            type inet: ip - address;
        }
        leaf cidrmask {
            type uint8 {
                range "1..128";
            }
        }
    }
    list range {
        key "from to";
        tailf: cli - suppress - mode;
        description
            "List of IP ranges belonging to this pool, inclusive. If your " +
            "pool of IP addresses does not conform to a convenient set of " +
            "subnets it may be easier to describe it as a range. " +
            "Note that the exclude list does not apply to ranges, but of " +
            "course a range may not overlap a subnet entry.";
        tailf: validate ipa_validate {
            tailf: dependency ".";
        }
        leaf from {
            type inet: ip - address - no - zone;
        }
        leaf to {
            type inet: ip - address - no - zone;
        }
        must "(contains(from, '.') and contains(to, '.')) or
            (contains(from, ':') and contains(to, ':'))
        " {
        error - message "IP addresses defining a range must agree on IP version.";
    }
}
container alarms {
    leaf enabled {
        type empty;
        description "Set this leaf to enable alarms";
    }
    leaf low - threshold - alarm {
        type uint8 {
            range "0 .. 100";
        }
        default 10;
        description "Change the value for when the low threshold alarm is
        raised.The value describes the percentage IPs left in
            the pool.The
        default is to raise the alarm when there
        are ten(10) percent IPs left in the pool.
        ";
    }
}
}
}
augment "/ralloc:rm-action" {
    tailf: action ip - allocator - tool {
        tailf: hidden debug;
        tailf: actionpoint ip - allocator - tool - action;
        input {
            leaf pool {
                type leafref {
                    path "/ralloc:resource-pools/ipalloc:ip-address-pool/name";
                }
            }
            leaf operation {
                type enumeration {
                    enum printIpPool;
                    enum fix_response_ip;
                    enum fix_missing_allocation;
                    enum persistAll;
                }
                mandatory true;
            }
        }
        output {
            leaf result {
                type string;
            }
        }
    }
}
}

Further Reading

The NSO Observability Exporter (OE) package allows Cisco NSO to export observability-related data using software-industry-standard formats and protocols, such as the OpenTelemetry protocol (OTLP). It supports the export of progress traces using OTLP, as well as the export of transaction metrics based on the progress trace data into an InfluxDB database.

Observability Data Types

To provide insight into the state and working of a system, operators make use of different types of data:

• Logs: Information about events taking place in the system, usually for humans to interpret.

• Traces: Detailed information about the requests as they traverse the system.

• Metrics: Measures of quantifiable aspects of the system for statistical analysis, such as the amount of successful and failed requests.

Each of the data types serves a different purpose. Metrics allow you to get a high-level view of whether the system behaves in an expected manner, for example, no or few failed requests. Metrics also help identify the load on the system (i.e. CPU usage, number of concurrent requests, etc.) but they do not inform you what is happening with a particular request or transaction, for example, the one that is failing.

Tracing, on the other hand, shows the path and the time that the request took in different parts of the overall system. Perhaps the request failed because one of the subsystems took too long to provide the necessary data. That's the kind of information a trace gives you.

However, to understand what took a specific subsystem a long time to respond, you need to consult the relevant logs.

As these are different types of data, different software solutions exist to process, store, and examine them.

For tracing, the package exports progress trace data using the standard OTLP format. Each trace carries a trace-id that uniquely identifies it and can be supplied as part of the request (see the Progress Trace section in the NSO Development Guide for details), allowing you to find the relevant data in a busy system. Tools such as Jaeger or Grafana (with Grafana Tempo) can then ingest the OTLP data and present it in a graphical way for further analysis.

The Observability Exporter package also performs additional processing of the tracing data and exports the calculated metrics to an InfluxDB time-series database. Using Grafana or a similar tool, you can extract and accumulate the relevant values to produce customized dashboards, for example, showing the average transaction length for each type of service in NSO.

The package exports four different types of metrics, called measurements, to InfluxDB:

• span: Data for individual parts of the transaction, also called spans.

• span-count: Number of concurrent spans, for example, how many transactions are in the prepare phase (prepare span) at the same time.

• transaction: Sum of span durations per transaction, for example, the cumulative time spent in creating code when a transaction configures multiple services.

• transaction-lock: Details about transaction lock, such as queue length when acquiring or releasing the lock.

Installation

To install the Observability Exporter add-on, follow the steps below:

1. Install the prerequisite Python packages: parsedatetime, opentelemetry-exporter-otlp, and influxdb. To install the packages, run the command pip install -r src/requirements.txt from the package folder.

2. Add the Observability Exporter package in a manner suitable for your NSO installation. This usually entails copying the package file to the appropriate packages/ folder and performing a package reload. For more information, refer to the NSO product documentation on package management.

Configure Data Export

Observability Exporter configuration resides under the progress export container in NSO. All export functions can be enabled or disabled through the top-level enabled leaf.

To configure exporting tracing data, use the otlp container. This is a presence container that controls whether export is enabled or not. In the container, you can define the target host and port for sending data, as well as the transport used. Unless configured otherwise, the data is exported to the localhost using the default OTLP port, so there is minimal configuration required if you run the collector locally, for example, on the same system or as a sidecar in a container deployment.

The InfluxDB export is configured and enabled using the influxdb presence container, where you set the host to export metrics to. You can also customize the port number, username, password, and database name used for the connection.

Under the progress export you can also configure extra-tags, the additional tag name-value pairs that the system adds to the measurements. These are currently only used for InfluxDB.

The following is a sample configuration snippet using different syntax styles:

progress export enabled
progress export influxdb host localhost
progress export influxdb username nso
progress export influxdb password ...
progress export influxdb database nso
progress export otlp host localhost
progress export otlp transport http

progress {
    export {
        enabled;
        influxdb {
            host     localhost;
            username nso;
            password ...;
            database nso;
        }
        otlp {
            host      localhost;
            transport http;
        }
    }
}

Using InfluxDB 2.x

Note that the current version of the Observability Exporter uses InfluxDB v1 API. If you run an InfluxDB 2.x database instance, you need to enable v1 API client access with the influx v1 auth create command or a similar mechanism. Refer to Influxdata docs for more information.

Minimal Tracing Example with Jaeger

This example shows how to use the Jaeger software (https://www.jaegertracing.io) to visualize the progress traces. It requires you to install Jaeger on the same system as NSO and is therefore only suitable for demo or development purposes.

1. First, make sure that you have a running NSO instance and that you have successfully added the Observability Exporter package. To verify, run the show packages package observability-exporter command from the NSO CLI.

2. Download and run a recent Jaeger all-in-one binary from the Jaeger website, using the --collector.otlp.enabled switch:

   Copy

   $ jaeger-all-in-one --collector.otlp.enabled

3. Keep Jaeger running, and from another terminal, enter the NSO CLI to enable OTLP data export:

   Copy

   admin@ncs# unhide debug
   admin@ncs# config
   admin@ncs(config)# progress export otlp
   admin@ncs(config)# commit

   Jaeger should now be receiving the transaction traces. However, if you have no running transactions in the system, there will be no data. So, make sure that you have some traces by performing a trivial configuration change:

   Copy

   admin@ncs(config)# session idle-timeout 100001
   admin@ncs(config)# commit

4. Now you can connect to the Jaeger UI at http://localhost:16686 to explore the data. In the Search pane, select "NSO" service and click Find Traces.

   Clicking on one of the traces will bring you to the trace view, such as the following one.

Minimal Metrics Example with InfluxDB

This example shows you how to store and do basic processing and visualization of data in InfluxDB. It requires you to install InfluxDB on the same system as NSO and is therefore only suitable for demo or development purposes.

1. First, ensure you have a running NSO instance and have successfully added the Observability Exporter package. To verify, run the show packages package observability-exporter command from the NSO CLI.

2. Next, set up an InfluxDB instance. Download and install the InfluxDB 2 binaries and the corresponding influx CLI appropriate for your NSO system. See Influxdata docs for details, e.g. brew install influxdb influxdb-cli on a macOS system.

3. Make sure that you have started the instance, then complete the initial configuration of InfluxDB. During the configuration, create an organization named my-org and a bucket named nso. Do not forget to perform the Influx CLI setup. To verify that everything works in the end, run:

   Copy

   $ influx bucket list --org my-org
   ID                      Name            Retention       Shard group duration    Organization ID         Schema Type
   bc98fe2ae322d349        _monitoring     168h0m0s        24h0m0s                 b3e7d8ac9213a8fe        implicit
   dd10e45d802dda29        _tasks          72h0m0s         24h0m0s                 b3e7d8ac9213a8fe        implicit
   5d744e55fb178310        nso             infinite        168h0m0s                b3e7d8ac9213a8fe        implicit

   In the output, find the ID of the NSO bucket that you have created. For example, here it is 5d744e55fb178310 but yours will be different.

4. Create a username/password pair for v1 API access:

   Copy

   $ influx v1 auth create --org my-org --username nso --password nso123nso --write-bucket BUCKET_ID

   Use the BUCKET_ID that you have found in the output of the previous command.

5. Now connect to the NSO CLI and configure the InfluxDB exporter to use this instance:

   Copy

   admin@ncs# unhide debug
   admin@ncs# config
   admin@ncs(config)# progress export influxdb
   admin@ncs(config)# progress export influxdb host localhost
   admin@ncs(config)# progress export influxdb username nso
   admin@ncs(config)# progress export influxdb password nso123nso
   admin@ncs(config)# commit

   The username and password should match those created with the previous command, while the database name (using the default of nso here) should match the bucket name. Make sure that you have some data for export by performing a trivial configuration change:

   Copy

   admin@ncs(config)# session idle-timeout 100002
   admin@ncs(config)# commit

6. Open the InfluxDB UI at http://localhost:8086 and log in, then select the Data Explorer from the left-hand menu. Using the query builder, you can explore and visualize the data.

   For example, select the nso bucket, span measurement, and duration as a field filter. Keeping other settings at their default values, it will graph the average (mean) times that various parts of the transaction take. If you wish, you can further configure another filter for name, to only show the values for the selected part.

   Note that the above image shows data for multiple transactions over a span of time. If there is only a single transaction, the graph will look empty and will instead show a single data point when you hover over it.

Observability Exporter Integration with Grafana

This example shows integrating the Observability Exporter with Grafana to monitor NSO application performance.

1. First, ensure you have a running NSO instance and have successfully added the Observability Exporter package. To verify, run the show packages package observability-exporter command from the NSO CLI.

2. Next, set up an InfluxDB instance. Follow steps 2 to 4 from the Minimal Metrics Example with InfluxDB.

3. Next, set up a Grafana instance. Refer to Grafana Docs for installing Grafana on your system. A MacOS example:

   1. Install Grafana.

      Copy

      $ brew install grafana

   2. Start the Grafana instance.

      Copy

      $ sudo brew services start grafana

4. Configure the Grafana Organization name.

   Copy

   $ curl -i "http://admin:admin@127.0.0.1:3000/api/orgs/1" \
          -m 5 -X PUT --noproxy '*' \
          -H 'Content-Type: application/json;charset=UTF-8' \
          --data-binary "{\"name\":\"NSO\"}"

5. Add InfluxDB as a Data Source in Grafana. Download the file influxdb-data-source.json and replace "my-token" with the actual token from the InfluxDB instance in the file and run the below command.

   Copy

   $ curl -i "http://admin:admin@127.0.0.1:3000/api/datasources" \
          -m 5 -X POST --noproxy '*' \
          -H 'Content-Type: application/json;charset=UTF-8' \
          --data @influxdb-data-source.json

6. Set up the NSO example Dashboard. This step requires the JQ command-line tool to be installed first on the system.

   Copy

   $ brew install jq

   Download the sample NSO dashboard JSON file dashboard-nso-local.json and run the below command. Replace the "value" field's value with the actual Jaeger UI URL where "name" is INPUT_JAEGER_BASE_URL under "inputs".

   Copy

   $ curl -i "http://admin:admin@127.0.0.1:3000/api/dashboards/import" \
          -m 5 -X POST -H "Accept: application/json" --noproxy '*' \
          -H 'Content-Type: application/json;charset=UTF-8' \
          --data-binary "$(jq '{"dashboard": . , "overwrite": true, "inputs":[{"name":"DS_INFLUXDB","type":"datasource", "pluginId":"influxdb","value":"InfluxDB"},{"name":"INPUT_JAEGER_BASE_URL","type":"constant","value":"http://127.0.0.1:49987/"}]}' dashboard-nso-local.json)"

7. (Optional) Set the NSO dashboard as a default dashboard in Grafana.

   Copy

   $ curl -i 'http://admin:admin@127.0.0.1:3000/api/org/preferences' \
          -m 5 -X PUT --noproxy '*' \
          -H 'X-Grafana-Org-Id: 1' \
          -H 'Content-Type: application/json;charset=UTF-8' \
          --data-binary "{\"homeDashboardId\":`curl -m 5 --noproxy '*' 'http://admin:admin@127.0.0.1:3000/api/dashboards/uid/nso' 2>/dev/null | jq .dashboard.id`}"

8. Connect to the NSO CLI and configure the InfluxDB exporter:

   Copy

   admin@ncs# unhide debug
   admin@ncs# config
   admin@ncs(config)# progress export influxdb
   admin@ncs(config)# progress export influxdb host 127.0.0.1
   admin@ncs(config)# progress export influxdb port 8086
   admin@ncs(config)# progress export influxdb username nso
   admin@ncs(config)# progress export influxdb password nso123nso
   admin@ncs(config)# commit

9. To perform a few trivial configuration changes, open the Grafana UI at http://localhost:3000/ and log in with username admin and password admin. Setting the NSO dashboard as a default dashboard will show different charts and graphs showing NSO metrics.

   Below are the panels showing metrics related to the transactions, such as transaction throughput, longest transactions, transaction locks held, and queue length.

   Below are the panels showing metrics related to the services, such as mean/max duration for create service, mean duration for run service, and the service's longest spans.

   Below are the panels showing metrics related to the devices, such as device locks held, longest device connection, longest device sync-from, and concurrent device operations.

Observability Exporter Docker multi-container setup example

All previously mentioned databases and virtualization software can also be brought up in a Docker environment with Docker volumes, making it possible to persist the metric data in the data stores after shutting down the Docker containers.

To facilitate bringing up the containers and the interconnectivity of databases and virtualization containers, a setup bash script called setup.sh is provided together with a compose.yaml file that describes all Docker containers to create and start, as well as configuration files to configure each container.

This diagram shows an overview of the containers that Compose creates and starts and how they are connected.

To create the Docker environment described above, follow these steps:

1. Make sure Docker and Docker Compose are installed on your machine. Refer to the Docker documentation on installing Docker for your respective OS. You can verify that Docker and Compose are installed by executing the following commands in a terminal and getting a version number as output:

   • docker

   Copy

   $ docker version

   • docker compose

   Copy

   $ docker compose version

2. Download the NSO Observability Exporter package from CCO, untar it, and cd into the setup folder:

   Copy

   $ sh ncs-6.2-observability-exporter-1.2.0.signed.bin
   $ tar -xzf ncs-6.2-observability-exporter-1.2.0.tar.gz
   $ cd observability-exporter/setup

3. Make the setup.sh script executable:

   Copy

   $ chmod u+x setup.sh

4. Run the setup.sh script without arguments to use the default ports for containers and the default username and password for InfluxDB or supply arguments to set a specific port for each container:

   • Use the default values defined in the script.

   Copy

   $ ./setup.sh

   • Provide port values and InfluxDB configuration.

   Copy

   $ ./setup.sh --otelcol-grpc 12344 --otelcol-http 12345 --jaeger 12346 --influxdb 12347 --influxdb-user admin --influxdb-password admin123 --influxdb-token my-token --prometheus 12348 --grafana 12349

   • To run secure protocol configuration, whether it's HTTP or gRPC, utilize the provided setup script with the appropriate security settings. Ensure the necessary security certificates and keys are available. For HTTPS and gRPC Secure, a TLS certificate and private key files are necessary. For instructions on creating self-signed certificates, refer to Creating Self-Signed Certificate.

   Copy

   $./setup.sh --otelcol-cert-path /path/to/certificate.crt --otelcol-key-path /path/to/privatekey.key

   • The script will output NSO configuration to configure the Observability Exporter and URLs to visit the dashboards of some of the containers.

   Copy

   NSO configuration:
   <config xmlns="http://tail-f.com/ns/config/1.0">
     <progress xmlns="http://tail-f.com/ns/progress">
       <export xmlns="http://tail-f.com/ns/observability-exporter">
         <enabled>true</enabled>
         <influxdb>
           <host>localhost</host>
           <port>12347</port>
           <username>admin</username>
           <password>admin123</password>
         </influxdb>
         <otlp>
           <port>12345</port>
           <transport>http</transport>
           <metrics>
           <port>12345</port>
           </metrics>
          </otlp>
       </export>
     </progress>
   </config>
   
   Visit the following URLs in your web browser to reach respective systems:
   Jaeger      : http://127.0.0.1:12346
   Grafana     : http://127.0.0.1:12349
   Prometheus  : http://127.0.0.1:12348

   • You can run the setup.sh script with the --help flag to print help information about the script and see the default values used for each flag.

   Copy

   $ ./setup.sh --help

   • Enable HTTPS to enable OTLP through HTTPS, root certificate authority (CA) certificate file in PEM format needs to be specified in NSO configuration for both traces and metrics.

   Copy

   <config xmlns="http://tail-f.com/ns/config/1.0">
       <progress xmlns="http://tail-f.com/ns/progress">
       <export xmlns="http://tail-f.com/ns/observability-exporter">
           <otlp>
           <endpoint><endpoint></endpoint>
           <server-certificate-path></path/to/rootCA.pem></server-certificate-pathh>
           <service_name>nso</service_name>
           <transport>https</transport>
           <metrics>
               <endpoint><endpoint></endpoint>
               <server-certificate-path></path/to/rootCA.pem></server-certificate-path>
           </metrics>
           </otlp>
       </export>
       </progress>
   </config>

5. After configuring the Observability Exporter with the NSO configuration printed by the setup.sh script, e.g., using the CLI load command or the ncs_load tool, traces, and metric data should be seen in Jaeger, InfluxDB, and Grafana as shown in the previous setup.

6. The setup can be brought down with the following commands:

   • Bring down containers only.

   Copy

   $ ./setup.sh --down

   • Bring down containers and remove volumes.

   Copy

   $ ./setup.sh --down --remove-volumes

Creating Self-Signed Certificate

Prerequisites: OpenSSL: Ensure that OpenSSL is installed on your system. Most Unix-like systems come with OpenSSL pre-installed.

Generate a Private Key: First, generate a private key using OpenSSL. Run the following command in your terminal or command prompt:

1. Install OpenSSL:

$sudo apt-get install openssl

1. Create a Root CA (Certificate Authority):

$openssl genrsa -out rootCA.key 2048
$openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 3650 -out rootCA.pem

1. Generate SSL Certificates Signed by the Root CA:

$openssl genrsa -out server.key 2048
$openssl req -new -key server.key -out server.csr
$openssl x509 -req -in server.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out server.crt -days 365 -sha256

  - server.key: Private key for the server.
  - server.csr: Certificate Signing Request (CSR) for the server.
  - server.crt: SSL certificate for the server, signed by the root CA.

1. Use the Certificates:

Now, server.key and server.crt can be used in the server configuration. 
Ensure that rootCA.pem is added to the trust store of clients that need to verify the server's certificate.

Export NSO Traces and Metrics to Splunk Observability Cloud

In the previous test environment setup, we exported traces to Jaeger and metrics to Prometheus but progress-trace and metrics can also be sent to Splunk Observability Cloud.

In order to send traces and metrics to Splunk Observability Cloud, either the OpenTelemetry Collector Contrib or Splunk OpenTelemetry Collector can be used.

Here is an example config that can be used with the OpenTelemetry Collector Contrib to send traces and metrics:

exporters:
  sapm:
    access_token: <ACCESS_TOKEN>
    access_token_passthrough: true
    endpoint: https://ingest.<SIGNALFX_REALM>.signalfx.com/v2/trace
    max_connections: 10
    num_workers: 5
  signalfx:
    access_token: <ACCESS_TOKEN>
    access_token_passthrough: true
    realm: <SIGNALFX_REALM>
    timeout: 5s
    max_idle_conns: 10

service:
  pipelines:
    traces:
      exporters: [sapm]
    metrics:
      exporters: [signalfx]

An access token and the endpoint of your Splunk Observability Cloud instance are needed to start exporting traces and metrics. The access token can be found under the settings -> Access Tokens menu in your Splunk Observability Cloud dashboard. The endpoint can be constructed by looking at your Splunk Observability Cloud URL and replacing <SIGNALFX_REALM> with the one you see in the URL. e.g. https://ingest.us1.signalfx.com/v2/trace.

Traces can be accessed at https://app.us1.signalfx.com/#/apm/traces and Metrics are available when accessing or creating a dashboard at https://app.us1.signalfx.com/#/dashboards.

More options for the sapm and signalfx exporters can be found at https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/sapmexporter/README.md and https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/signalfxexporter/README.md respectively.

In the current Observability Exporter version, metrics from spans, that is metrics that are currently sent directly to InfluxDB, cannot be sent to Splunk.

Export NSO Traces and Metrics to Splunk Enterprise

1. Download Splunk Enterprise. Visit the Splunk Enterprise. Select the appropriate version for your operating system (Linux, Windows, macOS). Download the installer package.

2. Install Splunk Enterprise.

• On Linux:

  • Transfer the downloaded .rpm or .deb file to your Linux server.

  • Install the package:

    • For RPM-based distributions (RedHat/CentOS):

      Copy

      sudo rpm -i splunk-<version>-linux-2.6-x86_64.rpm

    • For DEB-based distributions (Debian/Ubuntu):

      Copy

      sudo dpkg -i splunk-<version>-linux-2.6-amd64.deb

• On Windows:

  • Run the downloaded .msi installer.

  • Follow the prompts to complete the installation.

1. Start Splunk.

• On Linux:

sudo /opt/splunk/bin/splunk start --accept-license

• On Windows:

  • Open the Splunk Enterprise application from the Start Menu.

1. Access Splunk Web Interface.

   Navigate to http://:8000. Log in with the default credentials (admin/changeme).

2. Create an Index via the Splunk Web Interface:

   • Click on Settings in the top-right corner.

   • Under the Data section, click on Indexes.

   • Create a New Index:

   • Click on the New Index button.

   • Fill in the required details:

     • Index Name: Enter a name for your index (e.g., nso_traces, nso_metrics).

     • Index Data Type: Select the type of data (e.g., Events or Metrics).

     • Home Path, Cold Path, and Thawed Path: Leave these as default unless you have specific requirements.

   • Click on the Save button.

3. Enable HTTP Event Collector (HEC) on Splunk Enterprise. Before you can use Event Collector to receive events through HTTP, you must enable it. For Splunk Enterprise, enable HEC through the Global Settings dialog box.

   • Click Settings > Data Inputs.

   • Click HTTP Event Collector.

   • Click Global Settings.

   • In the All Tokens toggle button, select Enabled.

   • Choose a nso_traces/nso_metrics for their respective HEC tokens.

   • Click Save.

4. Create an Event Collector token on Splunk Enterprise. To use HEC, you must configure at least one token.

   • Click Settings > Add Data.

   • Click monitor.

   • Click HTTP Event Collector.

   • In the Name field, enter a name for the token.

   • Click Next.

   • Click Review.

   • Confirm that all settings for the endpoint are correct, click Submit. Otherwise, click < to make changes.

5. Configure the OpenTelemetry Protocol (OTLP) Collector:

   • Create or edit the otelcol.yaml file to include the HEC configuration. Example configuration:

   Copy

   exporters:
     splunk_hec/traces:
       token: "<YOUR_HEC_TOKEN_FOR_TRACES>"
       endpoint: "http://<your_splunk_server_ip>:8088/services/collector"
       index: "nso_traces"
       tls:
         insecure_skip_verify: true
     splunk_hec/metrics:
       token: "<YOUR_HEC_TOKEN_FOR_METRICS>"
       endpoint: "http://<your_splunk_server_ip>:8088/services/collector"
       index: "nso_metrics"
       tls:
         insecure_skip_verify: true
   
   service:
     pipelines:
       traces:
         exporters: [splunk_hec/traces]
       metrics:
         exporters: [splunk_hec/metrics]

6. Save the configuration file.

Support

For additional support questions, refer to Cisco Support.

About this Guide

This NSO Resource Manager (RM) API Guide describes the APIs exposed by the Resource Manager package that you can use to allocate IPs from the IP resource pool and to allocate the IDs from ID resource pools.

Intended Audience

This guide is intended for Cisco advanced services developers, network engineers, and system engineers to install the RM package inside NSO and then utilize the APIs exposed by the RM package to allocate and manage IP subnets and IDs as required by other CFPs installed alongside this RM package inside NSO.

Additional Documentation

This documentation requires the reader to have a good understanding of NSO and its usage as described in the following NSO documentation:

• NSO Installation

• NSO Operation and Usage Guide

Resource Manager IP/ID Allocation APIs

The APIs exposed by the Resource Manager package are used to allocate IP subnets and IDs from the IP and ID resource pools respectively by the applications requesting the resources. The APIs help to allocate, update, or deallocate the resources. You can make API calls to the resource pools as long as the pool is not exhausted of the resources. If the pool is exhausted of resources or if the referenced pool does not exist in the database when there is a request, the allocation raises an exception.

When a service makes multiple resource allocations from a single pool, the optional ‘name’ parameter allows the service to distinguish the different allocations. By default, the parameter value is an empty string.

Resource allocation can be synchronous or asynchronous.

The synchronized allocation API request uses a Reactive-Fast-Map to allocate resources and still manages the interface to look synchronous. This means that as you create any allocation request from Northbound, you can see the allocation results, such as the requested IP subnet/ID in the same transaction. If a NB is making an allocation request, and in the same transaction a configuration is being applied to a specific device, the commit dry run receives the request response, and the response is processed by the RM and the configurations are pushed to the device in the same transaction. Thus, the NB user can see the get modification in the commit dry run.

During a resource request, the resource is allocated and stored in the create callback. This allocation is visible to other services that are run in the same or subsequent transactions and therefore avoids the recreation of resource when the service is redeployed. Synchronous allocation does not require service re-deploy to read allocation. The same transaction can read allocation. Commit dry-run or get-modification displays the allocation details as output.

Example

The following is an example for a Northbound service callback passed with required API parameters for both synchronous and asynchronous IPv4 allocations. The example uses pool-example package as a reference. The request describes the details it uses, such as the pool, device. Each allocation has an allocation ID. In the following example, the allocating service pulls one IPv4 address from the IPv4 resource pool. The requesting service then uses this allocated IP address to set the interface address on the device southbound to NSO.

class AllocateCallbacks(Service):
    @Service.create
    def cb_create(self, tctx, root, service, proplist):
        self.log.info('AllocateCallbacks create(service=', service._path, ')')
        self.log.info('requested allocation {} from {}'.format(service.device, service.pool))
        
        service_xpath = (
            "/allocating-service-async:allocating-service-async[name='{}']"
        )
        
        propslist = ip_allocator.net_request(
            service,
            service_xpath.format(service.name),
            "admin",
            service.pool,
            service.ipv4,
            service.subnet_size,
            False,
            "default",
            True,
            proplist,
            self.log
        )
        
        # Check
        net = ip_allocator.net_read("admin", root, service.pool, service.ipv4)
        self.log.info('Check n/w create(IP=', net, ')')
        
        if net:
            self.log.info(
                'received device {} ip-address value from {} is ready'.format(
                    service.device, service.pool
                )
            )
            
            template = ncs.template.Template(service)
            vars = ncs.template.Variables()
            vars.add("SERVICE", str(service.ipv4))
            vars.add("DEVICE_NAME", str(service.device))
            vars.add("IP", str(net))
            
            template.apply('device', vars)
        
        return propslist

class AllocateCallbacksAsync(Service):
    @Service.create
    def cb_create(self, tctx, root, service, proplist):
        self.log.info('AllocateCallbacksAsync create(service=', service._path,
        ')')
        self.log.info('requested allocation {} from {}'.format(service.device,
        service.pool))
        async[name='{}']")
        service_xpath = ("/allocating-service-async:allocating-service-
        ip_allocator.net_request(service,
        service_xpath.format(service.name),
        tctx.username,
        service.pool,
        service.ipv4,
        service.subnet_size)
        # Check
        net=ip_allocator.net_read(tctx.username, root, service.pool,
        service.ipv4)
        self.log.info('Check n/w create(IP=', net, ')')
        if net:
            self.log.info('received device {} ip-address value from {} is
            ready'.format(service.device, service.pool))
            template = ncs.template.Template(service)
            vars = ncs.template.Variables()
            vars.add("SERVICE", str(service.ipv4))
            vars.add("DEVICE_NAME", str(service.device))
            vars.add("IP", str(net))
            template.apply('device', vars

The payloads below demonstrate the Northbound service allocation request using the Resource Manager synchronous and asynchronous flows. The API pulls one IP address from the IPv4 resource pool and sets the returned IP address on the interface IOS1 device. The payloads demonstrate both synchronous and asynchronous flows.

admin@ncs% load merge alloc-sync.xml
[ok]
admin@ncs% commit dry-run
cli {
    local-node {
        data devices {
            device ios1 {
                config {
                    ip {
                        prefix-list {
                            + prefixes sample {
                                + permit 11.1.0.0/32;
                            + }
                            + prefixes sample1 {
                                + permit 11.1.0.1/32;
                            + }
                            + prefixes sample3 {
                                + permit 11.1.0.2/32;
                            + }
                            + prefixes sample4 {
                                + permit 11.1.0.3/32;
                            + }
                        }
                    }
                }
            }
        }
        +allocating-service sync-test-1 {
            + device ios1;
            + pool IPv4;
            + subnet-size 32;
            + ipv4 sample;
        +}
        +allocating-service sync-test-2 {
            + device ios1;
            + pool IPv4;
            + subnet-size 32;
            + ipv4 sample1;
        +}
        +allocating-service sync-test-3 {
            + device ios1;
            + pool IPv4;
            + subnet-size 32;
            + ipv4 sample3;
        +}
        +allocating-service sync-test-4 {
            + device ios1;
            + pool IPv4;
            + subnet-size 32;
            + ipv4 sample4;
        +}
    }
}

admin@ncs% load merge alloc-async.xml
[ok]
admin@ncs% commit dry-run
cli {
    local-node {
        data resource-pools {
            + ip-address-pool IPv4 {
                + allocation sample {
                    + username admin;
                    + allocating-service /allocating-service-
                      async[name='async-test'];
                    + redeploy-type default;
                    + request {
                        + subnet-size 32;
                    + }
                + }
            + }
        }
        + allocating-service-async async-test {
            + device ios1;
            + pool IPv4;
            + subnet-size 32;
            + ipv4 sample;
        + }
    }
}

IPv4 and IPv6 have separate IP pool types; there is no mixed IP pool. You can specify a prefixlen parameter for IP pools to allocate a net of a given size. The default value is the maximum prefix length of 32 and 128 for IPv4 and IPv6, respectively.

The following APIs are used in IPv4 and IPv6 allocations.

IP Allocations

Resource Manager exposes the API calls to request IPv4 and IPv6 subnet allocations from the resource pool. These requests can be synchronous or asynchronous. This topic discusses the APIs for these flows.

The NSO Resource Manager interface and the resource allocator provide a generic resource allocation mechanism that works well with services. Each pool has an allocation list where services are expected to create instances to signal that they request an allocation. The request parameters are stored in the request container, and the allocation response is written in the response container.

The APIs exposed by RM are implemented in Python as well as Java, so the NB user can configure the service to be a Java package or a Python package and call the allocator API as per the implementation. The NB user can also use NSO CLI to make an allocation request to the IP allocator RM package.

Using Java APIs for IP Allocations

This section covers the Java APIs exposed by the RM package to the NB user to make IP subnet allocation requests.

Creating Asynchronous IP Subnet Allocation Requests

The asynchronous subnet allocation requests can be created for a requesting service with:

• The redeploy type set to default type or set to redeployType.

• The CIDR mask length can be set to invert the subnet mask length for Boolean

  operations with IP addresses or set not to be able to invert the subnet mask length.

• Pass the starting IP address of the subnet to the requesting service redeploy type

  (default/redeployType).

The following are the Java APIs for asynchronous IP allocation requests.

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service,
        String poolName,
        String username,
        int cidrmask,
        String id)

| Parameter   | Type       | Description                                                     |
|-------------|------------|-----------------------------------------------------------------|
| service     | NavuNode   | NavuNode referencing the requesting service node.               |
| poolName    | String     | Name of the resource pool to request the subnet IP address from.|
| Username    | String     | Name of the user to use when redeploying the requesting service.|
| cidrmask    | Int        | CIDR mask length of the requested subnet.                       |
| id          | String     | Unique allocation ID.                                           |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, poolName, userName, cidrMask,
id);

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service,
        String poolName,
        String username,
        int cidrmask,
        String id,
        boolean invertCidr)

| Parameter        | Type       | Description                                                        |
|------------------|------------|--------------------------------------------------------------------|
| Service          | NavuNode   | NavuNode referencing the requesting service node.                  |
| poolName         | String     | Name of the resource pool to request the subnet IP address from.   |
| Username         | String     | Name of the user to use when redeploying the requesting service.   |
| cidrmask         | Int        | CIDR mask length of requested subnet.                              |
| id               | String     | Unique allocation ID.                                              |
| invertCidr       | Boolean    | If boolean value is true, the subnet mask length is inverted.      |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, poolName, userName, cidrMask,
id, invertCidr.booleanValue());

@ServiceCallback(servicePoint = "ipaddress-allocator-test-servicepoint",
    callType = ServiceCBType.CREATE)
public Properties create(ServiceContext context,
                         NavuNode service,
                         NavuNode ncsRoot,
                         Properties opaque)
        throws DpCallbackException {
    LOGGER.info("IPAddressAllocatorTest Servicepoint is triggered");
    try {
        String servicePath = service.getKeyPath();

        CdbSession sess = cdb.startSession(CdbDBType.CDB_OPERATIONAL);
        try {
            String dPath = servicePath + "/deploys";
            int deploys = 1;

            if (sess.exists(dPath)) {
                deploys = (int) ((ConfUInt32) sess.getElem(dPath)).longValue();
            }

            if (sess.exists(servicePath)) { // Will not exist the first time
                sess.setElem(new ConfUInt32(deploys + 1), dPath);
            }

            NavuLeaf size = service.leaf("subnet-size");
            if (!size.exists()) {
                return opaque;
            }

            int subnetSize = (int) ((ConfUInt8) service.leaf("subnet-size").value()).longValue();
            String redeployOption = null;

            if (sess.exists(servicePath + "/redeploy-option")) {
                redeployOption = ConfValue.getStringByValue(
                        servicePath + "/redeploy-option",
                        service.leaf("redeploy-option").value()
                );
            }

            System.out.println("IPAddressAllocatorTest redeployOption: " + redeployOption);

            if (redeployOption == null) {
                IPAddressAllocator.subnetRequest(service, "mypool", "admin", subnetSize, "test");
            } else {
                RedeployType redeployType = RedeployType.from(redeployOption);
                System.out.println("IPAddressAllocatorTest redeployType: " + redeployType);

                IPAddressAllocator.subnetRequest(
                        service, redeployType, "mypool", "admin", subnetSize, "test", false
                );
            }

            boolean error = false;
            boolean allocated = sess.exists(servicePath + "/allocated");
            boolean ready = IPAddressAllocator.responseReady(service.context(), cdb, "mypool", "test");

            if (ready) {
                try {
                    IPAddressAllocator.fromRead(cdb, "mypool", "test");
                } catch (ResourceErrorException e) {
                    LOGGER.info("The allocation has failed");
                    error = true;
                }
            }

            if (ready && !error) {
                if (!allocated) {
                    sess.create(servicePath + "/allocated");
                }
            } else {
                if (allocated) {
                    sess.delete(servicePath + "/allocated");
                }
            }
        } finally {
            sess.endSession();
        }
    } catch (Exception e) {
        throw new DpCallbackException("Cannot create service", e);
    }
    return opaque;
}

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service,
        RedeployType redeployType,
        String poolName,
        service node
        String username,
        int cidrmask,
        subnet IP address from
        String id,
        boolean invertCidr)

| Parameter    | Type        | Description                                                       |
|--------------|-------------|-------------------------------------------------------------------|
| service      | NavuNode    | NavuNode referencing the requesting service node.                 |
| poolName     | String      | Name of the resource pool to request the subnet IP address from.  |
| username     | String      | Name of the user to use when redeploying the requesting service.  |
| cidrmask     | Int         | CIDR mask length of the requested subnet.                         |
| id           | String      | Unique allocation ID.                                             |
| invertCidr   | Boolean     | If boolean value is true, the subnet mask length is inverted.     |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, redeployType, poolName,
userName, cidrMask, id, invertCidr.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service,
        String poolName,
        String username,
        String startIp,
        int cidrmask,
        String id,
        boolean invertCidr)

| Parameter     | Type        | Description                                                        |
|---------------|-------------|--------------------------------------------------------------------|
| service       | NavuNode    | NavuNode referencing the requesting service node.                  |
| poolName      | String      | Name of the resource pool to request the subnet IP address from.   |
| username      | String      | Name of the user to use when redeploying the requesting service.   |
| startIP       | String      | Starting IP address for the requested subnet.                      |
| cidrmask      | Int         | CIDR mask length of the requested subnet.                          |
| id            | String      | Unique allocation ID.                                              |
| invertCidr    | Boolean     | If boolean value is true, the subnet mask length is inverted.      |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, poolName, userName, startIp,
cidrMask, id, invertCidr.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service,
        RedeployType redeployType,
        String poolName,
        String username,
        String startIp,
        int cidrmask,
        String id,
        boolean invertCidr)

| Parameter   | Type       | Description                                                       |
|-------------|------------|-------------------------------------------------------------------|
| service     | NavuNode   | NavuNode referencing the requesting service node.                 |
| poolName    | String     | Name of the resource pool to request the subnet IP address from.  |
| username    | string     | Name of the user to use when redeploying the requesting service.  |
| startIP     | String     | Starting IP address for the requested subnet.                     |
| cidrmask    | Int        | CIDR mask length of the requested subnet.                         |
| id          | String     | Unique allocation ID.                                             |
| invertCidr  | Boolean    | If boolean value is true, the subnet mask length is inverted.     |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, redeployType, poolName,
userName, startIp, cidrMask, id, invertCidr.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(ServiceContext context,
        NavuNode service,
        String poolName,
        String username,
        int cidrmask,
        String id)

| Parameter   | Type       | Description                                                      |
|-------------|------------|------------------------------------------------------------------|
| service     | NavuNode   | NavuNode referencing the requesting service node.                |
| poolName    | String     | Name of the resource pool to request the subnet IP address from. |
| username    | string     | Name of the user to use when redeploying the requesting service. |
| cidrmask    | Int        | CIDR mask length of the requested subnet.                        |
| id          | String     | Unique allocation ID.                                            |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(context, service, poolName, userName,
cidrMask, id);

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(ServiceContext context,
        NavuNode service,
        RedeployType redeployType,
        String poolName,
        String username,
        int cidrmask,
        String id,
        boolean invertCidr)

| Parameter   | Type           | Description                                                                   |
|-------------|----------------|-------------------------------------------------------------------------------|
| Context     | ServiceContext | ServiceContext referencing the requesting context the service was invoked in. |
| service     | NavuNode       | NavuNode referencing the requesting service node.                             |
| poolName    | String         | Name of the resource pool to request the subnet IP address from.              |
| username    | String         | Name of the user to use when redeploying the requesting service.              |
| cidrmask    | Int            | CIDR mask length of the requested subnet.                                     |
| id          | String         | Unique allocation ID.                                                         |
| invertCidr  | Boolean        | If the boolean value is true, the subnet mask length is inverted.             |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(context, service, redeployType,
poolName, userName, cidrMask, id, invertCidr.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(ServiceContext context,
        NavuNode service,
        String poolName,
        String username,
        String startIp,
        int cidrmask,
        String id,
        boolean invertCidr)

| Parameter   | Type           | Description                                                                   |
|-------------|----------------|-------------------------------------------------------------------------------|
| Context     | ServiceContext | ServiceContext referencing the requesting context the service was invoked in. |
| service     | NavuNode       | NavuNode referencing the requesting service node.                             |
| poolName    | String         | Name of the resource pool to request the subnet IP address from.              |
| username    | String         | Name of the user to use when redeploying the requesting service.              |
| startIP     | String         | Starting IP address for the requested subnet.                                 |
| cidrmask    | Int            | CIDR mask length of the requested subnet.                                     |
| id          | String         | Unique allocation ID.                                                         |
| invertCidr  | Boolean        | If boolean value is true, the subnet mask length is inverted.                 |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(context, service, poolName, userName,
startIp, cidrMask, id, invertCidr.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(ServiceContext context,
        NavuNode service,
        RedeployType redeployType,
        String poolName,
        String username,
        String startIp,
        int cidrmask,
        String id,
        boolean invertCidr)

| Parameter   | Type           | Description                                                                   |
|-------------|----------------|-------------------------------------------------------------------------------|
| Context     | ServiceContext | ServiceContext referencing the requesting context the service was invoked in. |
| service     | NavuNode       | NavuNode referencing the requesting service node.                             |
| poolName    | String         | Name of the resource pool to request the subnet IP address from.              |
| username    | String         | Name of the user to use when redeploying the requesting service.              |
| startIP     | String         | Starting IP address for the requested subnet.                                 |
| cidrmask    | Int            | CIDR mask length of the requested subnet.                                     |
| id          | String         | Unique allocation ID.                                                         |
| invertCidr  | Boolean        | If boolean value is true, the subnet mask length is inverted.                 |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(context, service, redeployType,
poolName, userName, startIp, cidrMask, id, invertCidr.booleanValue());

Creating Synchronous or Asynchronous IP Subnet Allocation Requests

The sync_alloc parameter in the API determines if the allocation request is for a synchronous or asynchronous mode. Set the sync_alloc parameter to true for synchronous flow.

The subnet allocation requests can be created for a requesting service with:

• The redeploy type set to default type or redeployType type.

• The CIDR mask length can be set to invert the subnet mask length for Boolean operations with IP addresses or set to not be able to invert the subnet mask length.

• Pass the starting IP address of the subnet to the requesting service redeploy type (default/redeploytype).

The following are the Java APIs for synchronous or asynchronous IP allocation requests.

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service,
        String poolName,
        String username,
        int cidrmask,
        String id,
        boolean invertCidr,
        boolean sync_alloc)

| Parameter    | Type     | Description                                                       |
|--------------|----------|-------------------------------------------------------------------|
| service      | NavuNode | NavuNode referencing the requesting service node.                 |
| poolName     | String   | Name of the resource pool to request the subnet IP address from.  |
| username     | String   | Name of the user to use when redeploying the requesting service.  |
| cidrmask     | Int      | CIDR mask length of the requested subnet.                         |
| id           | String   | Unique allocation ID.                                             |
| invertCidr   | Boolean  | Set value to true to invert the subnet mask length.               |
| sync_alloc   | Boolean  | Set value to true to make a synchronous allocation request.       |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, poolName, userName, cidrMask,
id, invertCidr.booleanValue(), testSync.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service, 
        RedeployType redeployType, 
        String poolName,
        String username,
        int cidrmask,
        String id,
        boolean invertCidr,
        boolean sync_alloc)

| Parameter   | Type     | Description                                                        |
|-------------|----------|--------------------------------------------------------------------|
| service     | NavuNode | NavuNode referencing the requesting service node.                  |
| poolName    | String   | Name of the resource pool to request the subnet IP address from.   |
| username    | String   | Name of the user to use when redeploying the requesting service.   |
| cidrmask    | Int      | CIDR mask length of the requested subnet.                          |
| id          | String   | Unique allocation ID.                                              |
| invertCidr  | Boolean  | Set value to true to invert the subnet mask length.                |
| sync_alloc  | Boolean  | Set value to true to make a synchronous allocation request.        |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, redeployType, poolName,
userName, cidrMask, id, invertCidr.booleanValue(),
testSync.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service,
    String poolName,
    String username,
    String startIp,
    int cidrmask,
    String id,
    boolean invertCidr,
    boolean sync_alloc)

| Parameter     | Type       | Description                                                      |
|---------------|------------|------------------------------------------------------------------|
| service       | NavuNode   | NavuNode referencing the requesting service node.                |
| poolName      | String     | Name of the resource pool to request the subnet IP address from. |
| username      | string     | Name of the user to use when redeploying the requesting service. |
| cidrmask      | Int        | CIDR mask length of the requested subnet.                        |
| id            | String     | Unique allocation ID.                                            |
| invertCidr    | Boolean    | Set value to true to invert the subnet mask length.              |
| sync_alloc    | Boolean    | Set value to true to make a synchronous allocation request.      |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, poolName, userName, startIp,
cidrMask, id, invertCidr.booleanValue(), testSync.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(NavuNode service,
        RedeployType redeployType,
        String poolName,
        String username,
        String startIp,
        int cidrmask,
        String id,
        boolean invertCidr,
        boolean sync_alloc)

| Parameter     | Type       | Description                                                      |
|---------------|------------|------------------------------------------------------------------|
| service       | NavuNode   | NavuNode referencing the requesting service node.                |
| poolName      | String     | Name of the resource pool to request the subnet IP address from. |
| username      | String     | Name of the user to use when redeploying the requesting service. |
| startIP       | String     | Starting IP address for the subnet allocation request.           |
| cidrmask      | Int        | CIDR mask length of the requested subnet.                        |
| id            | String     | Unique allocation ID.                                            |
| invertCidr    | Boolean    | Set value to true to invert the subnet mask length.              |
| sync_alloc    | Boolean    | Set value to true to make a synchronous allocation request.      |

import com.tailf.pkg.ipaddressallocator.IPAddressAllocator;

IPAddressAllocator.subnetRequest(service, redeployType, poolName,
userName, startIp, cidrMask, id, invertCidr.booleanValue(),
testSync.booleanValue());

void com.tailf.pkg.ipaddressallocator.IPAddressAllocator.
    subnetRequest(ServiceContext context,
        NavuNode service,
        String poolName,
        String username,
        int cidrmask,
        String id,
        boolean invertCidr,
        boolean sync_alloc)

| Parameter      | Type           | Description                                                                    |
|----------------|----------------|--------------------------------------------------------------------------------|
| Context        | ServiceContext | ServiceContext referencing the requesting context the service was invoked in.  |
| service        | NavuNode       | NavuNode referencing the requesting service node.                              |
| poolName       | String         | Name of the resource pool to request the subnet IP address from.               |
| username       | String         | Name of the user to use when redeploying the requesting service.               |
| startIP        | String         | Starting IP address for the subnet allocation request.                         |
| cidrmask       | Int            | CIDR mask length of the requested subnet.                                      |
| id             | String         | Unique allocation ID.                                                          |
| invertCidr     | Boolean        | Set value to true to invert the subnet mask length.                            |
| sync_alloc     | Boolean        | Set value to true to make a synchronous allocation request.                    |