README v5.56.1 2025-03-14
Table of contents
1. General
This document describes the cisco-staros NED.
Additional README files bundled with this NED package
Common NED Features
Custom NED Features
Verified target systems
1.1 Extract the NED package
It is assumed the NED package ncs-<NSO version>-cisco-staros-<NED version>.signed.bin has already
been downloaded from software.cisco.com.
In this instruction the following example settings will be used:
NSO version: 6.0
NED version: 1.0.1
NED package downloaded to: /tmp/ned-package-store
Extract the NED package and verify its signature:
In case the signature can not be verified (for instance if no internet connection), do as below instead:
The result of the extraction shall be a tar.gz file with the same name as the .bin file:
1.2 Install the NED package
There are two alternative ways to install this NED package. Which one to use depends on how NSO itself is setup.
In the instructions below the following example settings will be used:
NSO version: 6.0
NED version: 1.0.1
NED download directory: /tmp/ned-package-store
NSO run time directory: ~/nso-lab-rundir
A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory:
1.2.1 Local install
This section describes how to install a NED package on a locally installed NSO (see "NSO Local Install" in the NSO Installation guide).
It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1.
Untar the tar.gz file. This creates a new sub-directory named:
cisco-staros-<NED major digit>.<NED minor digit>:Install the NED into NSO, using the ncs-setup tool:
Open a NSO CLI session and load the new NED package like below:
Alternatively the tar.gz file can be installed directly into NSO. Then skip steps 1 and 2 and do like below instead:
Set the environment variable NED_ROOT_DIR to point at the NSO NED package:
1.2.2 System install
This section describes how to install a NED package on a system installed NSO (see "NSO System Install" in the NSO Installation Guide).
It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1.
Do a NSO backup before installing the new NED package:
Start a NSO CLI session and fetch the NED package:
Install the NED package (add the argument replace-existing if a previous version has been loaded):
Load the NED package
1.3 Configure the NED in NSO
This section describes the steps for configuring a device instance using the newly installed NED package.
Start a NSO CLI session:
Enter configuration mode:
Configure a new authentication group (my-group) to be used for this device:
Configure a new device instance (example: dev-1):
If configured protocol is ssh, do fetch the host keys now:
Finally commit the configuration
Verify configuration, using a sync-from.
If the sync-from was not successful, check the NED configuration again.
2. Optional debug and trace setup
It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting)
This can be achieved by configuring NSO to generate a trace file for the NED. A trace file contains information about all interactions with the device. Messages sent and received as well as debug printouts, depending on the log level configured.
NSO creates one separate trace file for each device instance with tracing enabled. Stored in the following location:
$NSO_RUNDIR/logs/ned-cisco-staros-cli-1.0-<device name>.trace
Do as follows to enable tracing in one specific device instance in NSO:
Start a NSO CLI session:
Enter configuration mode:
Enable trace raw:
Alternatively, tracing can be enabled globally affecting all configured device instances:
Configure the log level for printouts to the trace file:
Alternatively the log level can be set globally affecting all configured device instances using this NED package.
The log level 'info' is used by default and the 'debug' level is the most verbose.
IMPORTANT: Tracing shall be used with caution. This feature does increase the number of IPC messages sent between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should normally be disabled in production systems.
An alternative method for generating printouts from the NED is to enable the Java logging mechanism. This makes the NED print log messages to common NSO Java log file.
$NSO_RUNDIR/logs/ncs-java-vm.log
Do as follows to enable Java logging in the NED
Start a NSO CLI session:
Enter configuration mode:
Enable Java logging with level all from the NED package:
Configure the NED to log to the Java logger
Alternatively Java logging can be enabled globally affecting all configured device instances using this NED package.
IMPORTANT: Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not affected. However, all log printouts from all log enabled devices are saved in one single file. This means that the usability is limited. Typically single device use cases etc.
3. Dependencies
This NED has the following host environment dependencies:
Java 1.8 (NSO version < 6.2)
Java 17 (NSO version >= 6.2)
Gnu Sed
Dependencies for NED recompile:
Apache Ant
Bash
Gnu Sort
Gnu awk
Grep
Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob)
4. Sample device configuration
See what you are about to commit:
Commit new configuration in a transaction:
Verify that NCS is in-sync with the device:
Compare configuration between device and NCS:
Note: if no diff is shown, supported config is the same in NSO as on the device.
5. Built in live-status actions
Execute native device command:
The NED has support for a subset of native Cisco STAROS exec commands residing under device live-status. Presently, the following commands are supported:
To execute a command, run it in NCS exec mode like this:
NOTE: To excute show commands under context.
Use live-status " exec any" action to configure commands in device:
Note:
Its not possible to rollback if we get some error, when live-status actions used to configure commands.
User must give "\n" for newline/cr and also user must give all exit commands.
For example to configure below commands:
Execute device config commands as action:
Note:
Its not possible to rollback if we get some error, when this action is used to configure commands.
User must give "\n" for newline/cr and also user must give all exit commands.
Example-1:
Example-2:
6. Built in live-status show
NED supports following live-status show
Example:
7. Limitations
To configure
no neighbor <ip> activatecommand in/context/router/bgp/address-familyuser/service should configureneighbor <ip>command beforeno neighbor <ip> activate.
8. How to report NED issues and feature requests
Issues like bugs and errors shall always be reported to the Cisco NSO NED team through the Cisco Support channel:
The following information is required for the Cisco NSO NED team to be able to investigate an issue:
Do as follows to gather the necessary information needed for your device, here named 'dev-1':
Enable full debug logging in the NED
Configure the NSO to generate a raw trace file from the NED
If the NED already had trace enabled, clear it in order to submit only relevant information
Do as follows for NSO 6.4 or newer:
Do as follows for older NSO versions:
Run a compare-config to populate the trace with initial device config
Reproduce the found issue using ncs_cli or your NSO service. Write down each necessary step in a reproduction report.
In addition to this, it helps if you can show how it should work by manually logging into the device using SSH/TELNET and type the relevant commands showing a successful operation.
Gather the reproduction report and a copy of the raw trace file containing data recorded when the issue happened.
Contact the Cisco support and request to open a case. Provide the gathered files together with access details for a device that can be used by the Cisco NSO NED when investigating the issue.
Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when applicable. Such requests shall also go through the Cisco support channel.
The following information is required for feature requests and extensions:
Set the config on the real device including all existing dependent config and run sync-from to show it in the trace.
Run sync-from # devices device dev-1 sync-from
Attach the raw trace to the ticket
List the config you want implemented in the same syntax as shown on the device
SSH/TELNET access to a device that can be used by the Cisco NSO NED team for testing and verification of the new feature. This usually means that both read and write permissions are required. Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access through VPNs, jump servers etc as long as we can connect to the NED via SSH/TELNET.
9. How to rebuild a NED
To rebuild the NED do as follows:
When the NED has been successfully rebuilt, it is necessary to reload the package into NSO.
10. Configure the NED to use ssh multi factor authentication
This NED supports multi factor authentication (MFA) using the ssh authentication method 'keyboard-interactive'.
Some additional steps are required to enable the MFA support:
Verify that your NSO version supports MFA. This is configurable as additional settings in the authentication group used by the device instance.
Enter a NSO CLI and enter the following and do tab completion:
If 'mfa' is displayed in the output like above, NSO has MFA support enabled. In case MFA is not supported it is necessary to upgrade NSO before proceeding.
Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the ssh server and expects to get a proper response in return.
The executable can be a simple shell script or a program implemented in any programming language.
The required behaviour is like this:
read one line from stdin The line passed from the NED will be a semi colon separated string containing the following info:
The elements for device name, user, password and opaque corresponds to what has been configured in NSO. The ssh server name, instruction and prompt are given by the ssh server during the authentication step.
Each individual element in the semi colon separated list is Base64 encoded.
Extract the challenge based on the contents above.
Print a response matching the challenge to stdout and exit with code 0
In case a matching response can not be given do exit with code 2
Below is a simple example of an MFA authenticator implemented in Python3:
Configure the authentication group used by the device instance to enable MFA. There are two configurables available:
executable The path to the external multi factor authentication executable (mandatory).
opaque Opaque data that will passed as a cookie element to the executable (optional).
Try connecting to the device.
10.1 Trouble shooting
In case of connection problems the following steps can help for debugging:
Enable the NED trace in debug level:
Try connect again
Inspect the generated trace file.
Verify that the ssh client is using the external authenticator executable:
Verify that the executable is called with the challenges presented by the ssh server:
Check for any errors reported by the NED when calling the executable
11. NED Secrets - Securing your Secrets
11.1 General STAROS NED secret problems and solutions
Problems:
STAROS device encrypts passwords/secrets clear-text value.
STAROS device always returns new/randomly encrypted value every time "show config" is executed, which means this could possibly trigger compare-config diff.
Possible solutions in NED:
NED supports configuring password/url/secrets in clear-text which gets encrypted on the device. NED stores clear-text value in operDB and replaces device encrypted value with clear-text when sync-from/compare-config is performed.
Note: NED assumes that passwords/secrets are not changed outside NSO. There is no way to differentiate encrypted configs in NED/NSO as STAROS device returns new random encrypted value every time "show config" is executed.
There can be secret encrypted configs on STAROS device which are not configured via NED, these config could trigger compare-config diff even after multiple sync-from due above mentioned problem(2). One way to avoid compare-diff is to let NED store these encrypted secrets in operDB by doing sync-to operation. When performing sync-to NED stores all encrypted values in operDB and replaces device encrypted value with operDB when sync-from/compare-config is performed.
By default all secret/encrypted yang nodes are supported/configurable in NED.
Please use "cisco-staros/disable-encrypted-config" ned-settings to disable all encrypted/secret yang nodes. You can use this ned-settings, if you are not managing encrypted configs via NSO or if you do not want to sync/commit any encrypted configs in NSO.
Note: in order for the above settings to take effect, user must disconnect and do sync-from.
Filter config when reading from device. There may be few commands NSO not authorized to configure on device, or in any other scenario where user would like to filter certain configs in sync-from(e.g password/encrypted/secret configs).
Note: in order for the above settings to take effect, user must disconnect and connect again.
Please check README-ned-settings.md for more info regarding this ned-settings.
NOTE: It is not possible for NED/NSO to avoid compare-config diff if user do not have any of above option(s).
11.2 NSO encryption and NEDCOM_SECRET_TYPE
It is best practice to avoid storing secrets (e.g. passwords and urls) in clear-text in NSO. NSO in general has no way of encrypting/decrypting secrets config on the device. This means that if nothing is done about this, NED will become out of sync once we write secrets to the device.
In order to avoid becoming out of sync the NED stores clear-text value(s)
in a special secrets table in oper data. Later on, when config is read from the
device, the NED replaces all device encrypted values with their clear-text
values; effectively avoiding all config diffs in this area.
-- Handling auto-encryption
Let us say that we have password-encryption on and we want to write a new snmp user to our device:
this will be automatically encrypted by the device
But the secrets management will store this clear-text value in our secrets table:
NOTE: clear-text is hidden from below table.
which means that compare-config or sync-from will not show any changes and will not result in any updates to CDB. In fact, we can still see the unencrypted value in the NSO device tree:
-- Increasing security with NSO-side encryption
NED handles the device-side encryption, but passwords are still unencrypted in NSO. To deal with this NED supports NSO-encrypted strings instead of clear-text secrets in the NSO data model.
We have two alternatives, either we can manually encrypt our values using
one of the NSO-encrypted types (e.g tailf:aes-256-cfb-128-encrypted-string) and
set them to the tree, or we can recompile the NED to always encrypt secrets.
-- Setting encrypted value
Let us say we know that the NSO-encrypted string$9$IUbJUkP0ggBHEHDUNd8fwygAp8QLTHUBtA72VByxNqc= (admin123), we
can then set it in the device tree as normal
when committing this value, NED will decrypt it and the clear-text will be written to the device. Unlike the previous example the clear-text is not visible in the NSO device tree:
-- Auto-encrypting passwords in NSO
To avoid having to pre-encrypt your passwords you can rebuild your NED in your OS command shell specifying NSO encryption type in NEDCOM_SECRET_TYPE flag:
Or by adding the line NEDCOM_SECRET_TYPE=tailf:aes-cfb-128-encrypted-string
in top of the Makefile located in <cisco-staros-cli-x.y>/src directory.
When the NED has been successfully rebuilt, it is necessary to reload the package into NSO.
Doing this means that even if the input to a passwords a clear-text string, NSO will always encrypt it, and you will never see clear-text secrets in the NSO device tree.
If we reload our example with the new NEDCOM_SECRET_TYPE, all of the secrets are now NSO encrypted:
and if we create yet another user we get the desired result:
NOTE: Its not possible to support NEDCOM_SECRET_TYPE for "snmp community name" config. "snmp community name" is a list in yang model and having tailf:aes-cfb-128-encrypted-string for a list key, user can not delete or modify those instances using the clear text option, this is known NSO issue/side effect.
11.3 NSO key-rotation and standard NED secrets oper-data
From NED version 5.55 cisco-staros NED supports storing NED secrets oper-data under standard secrets path which is common in most NEDs i.e /devices/device/ned-settings/secrets/secret(standard). Before version 5.55 NED supports only /devices/device/ned-settings/cisco-staros-oper/secrets(legacy).
From NED version 5.55, NED will move ned-settings/staros-op:cisco-staros-oper/secrets (legacy-secrets) to standard path ned-settings/secrets:secrets/secret automatically when data found in legacy-secrets path. Legacy data checked and moved in SHOW/PREPARE operations. This is one time NED internal operation i.e once existing data moved to standard path, NED will continue to use standard secrets path for future operations.
If you see 'no entries', all good with NED, below information is not relevant.
For some reason, if legacy data is not moved automatically, user need to move oper secrets manually to standard secrets path. These are NED internal operational data caches that are usually not relevant to the user. However when new NSO key-rotation feature applied, NSO re-encrypts internal operational data caches only under /devices/device/ned-settings/secrets/secret. This means if a user wants to use the NSO key-rotation feature and there is data in the legacy secrets path, the legacy secrets cache must be moved to the standard path. This can be achieved by following an internal live-status exec action. Note: This is only relevant if the NED is recompiled with the NSO encryption type (e.g., tailf:aes-cfb-128-encrypted-string) and the NSO key-rotation feature is applied.
Last updated
Was this helpful?