1. General
1.1 Extract the NED package
1.2 Install the NED package
1.2.1 Local install
1.2.2 System install
1.3 Configure the NED in NSO
2. Optional debug and trace setup
3. Dependencies
4. Sample device configuration
5. Built in RPC actions
5.1. rpc add-filter-path
5.2. rpc clean-package
5.3. rpc clear-cached-capabilities
5.4. rpc clear-filter-paths
5.5. rpc compare-config
5.6. rpc compile-modules
5.7. rpc export-package
5.8. rpc get-modules
5.9. rpc import-filter-paths
5.10. rpc list-filter-paths
5.11. rpc list-module-sets
5.12. rpc list-modules
5.13. rpc list-profiles
5.14. rpc patch-modules
5.15. rpc rebuild-package
5.16. rpc remove-filter-path
5.17. rpc show-default-local-dir
5.18. rpc show-loaded-schema
5.19. rpc verify-get-config
6. Built in live-status show
7. Limitations
8. How to report NED issues and feature requests
9. How to rebuild a NED
10. Configure the NED to use ssh multi factor authentication
11. Run arbitrary commands on device
1. General
This document describes the nokia-sros_nc NED.
IMPORTANT:
This NED is delivered without any of the device YANG models bundled to the NED package.
It is required to download the YANG files separately and rebuild the NED package before the NED is
fully operational. See the README-rebuild.md for further information.
In summary, the below steps are needed to have a fully functioning NED:
1. Compile the empty package and load it into NSO
2a. Connect to device and fetch yang modules (if yang available on device or in git repository)
2b. Copy vendor yang directly into src/yang (if yang is available elsewhere)
3. Verify yang, potentially fixing any issues
4. Re-Compile package (i.e. in NSO), and do packages reload
Additional README files bundled with this NED package
+---------------------------+------------------------------------------------------------------------------+
| Name | Info |
+---------------------------+------------------------------------------------------------------------------+
| README-ned-settings.md | Information about all run time settings supported by this NED. |
| | |
| README-rebuild.md | Detailed instructions on how to download the device YANG models and |
| | rebuilding the NED with them. |
+---------------------------+------------------------------------------------------------------------------+
Common NED Features
+---------------------------+-----------+------------------------------------------------------------------+
| Feature | Supported | Info |
+---------------------------+-----------+------------------------------------------------------------------+
| netsim | yes | |
| | | |
| check-sync | yes | See the README-ned-settings.md, 'transaction trans-id-method' |
| | | for details. |
| | | |
| partial-sync-from | yes | |
| | | |
| live-status actions | yes | Can run CLI commands through 'exec any', see README.md |
| | | |
| live-status show | yes | All models are by default mounted under live-status |
| | | |
| load-native-config | no | Since config can be loaded in xml format in NSO, this can be |
| | | used instead. |
+---------------------------+-----------+------------------------------------------------------------------+
Verified target systems
+---------------------------+-----------------+--------+---------------------------------------------------+
| Model | Version | OS | Info |
+---------------------------+-----------------+--------+---------------------------------------------------+
| 7950 | 22 | SROS | |
| | | | |
| 7950 | 23 | SROS | |
+---------------------------+-----------------+--------+---------------------------------------------------+
The result of the extraction shall be a tar.gz file with the same name as the .bin file:
> ls *.tar.gz
ncs-6.0-nokia-sros_nc-1.0.1.tar.gz
1.2 Install the NED package
There are two alternative ways to install this NED package.
Which one to use depends on how NSO itself is setup.
In the instructions below the following example settings will be used:
NSO version: 6.0
NED version: 1.0.1
NED download directory: /tmp/ned-package-store
NSO run time directory: ~/nso-lab-rundir
A prerequisite is to set the environment variable NSO_RUNDIR to point at the NSO run time directory:
> export NSO_RUNDIR=~/nso-lab-rundir
IMPORTANT:
This NED is delivered as an “empty” package, i.e without any device YANG models bundled.
It must be rebuilt with the device YANG models to become operational.
The procedure to rebuild the empty NED (described in the README-rebuild.md) shall typically
be done in a lab environment. For this step a “local install” of the NED shall be used.
It is not suitable to use “system install” here since it is intended for production systems only.
Once this NED has been rebuilt with the device YANG and exported to one or many
separate tar.gz customized NED packages, a “system installation” can be used on them.
1.2.1 Local install
This section describes how to install a NED package on a locally installed NSO
(see "NSO Local Install" in the NSO Installation guide).
It is assumed the NED package has been been unpacked to a tar.gz file as described in 1.1.
Untar the tar.gz file. This creates a new sub-directory named:nokia-sros_nc-<NED major digit>.<NED minor digit>:
> tar xfz ncs-6.0-nokia-sros_nc-1.0.1.tar.gz
> ls -d */
nokia-sros_nc-gen-1.0
Install the NED into NSO, using the ncs-setup tool:
Configure a new authentication group (my-group) to be used for this device:
admin@ncs(config)# devices authgroup group my-group default-map remote-name <user name on device> \
remote-password <password on device>
Configure a new device instance (example: dev-1):
admin@ncs(config)# devices device dev-1 address <ip address to device>
admin@ncs(config)# devices device dev-1 port <port on device>
admin@ncs(config)# devices device dev-1 device-type generic ned-id nokia-sros_nc-gen-1.0
admin@ncs(config)# devices device dev-1 state admin-state unlocked
admin@ncs(config)# devices device dev-1 authgroup my-group
IMPORTANT:
The device-type shall always be set to generic when configuring a device instance
to use a 3PY NED. A common mistake is configuring it as netconf, which will cause
NSO to use its internal netconf client instead.
Finally commit the configuration
admin@ncs(config)# commit
Verify configuration, using a sync-from.
admin@ncs(config)# devices device dev-1 sync-from
result true
If the sync-from was not successful, check the NED configuration again.
2. Optional debug and trace setup
It is often desirable to see details from when and how the NED interacts with the device(Example: troubleshooting)
This can be achieved by configuring NSO to generate a trace file for the NED. A trace file
contains information about all interactions with the device. Messages sent and received as well
as debug printouts, depending on the log level configured.
NSO creates one separate trace file for each device instance with tracing enabled.
Stored in the following location:
The log level 'info' is used by default and the 'debug' level is the most verbose.
IMPORTANT:
Tracing shall be used with caution. This feature does increase the number of IPC messages sent
between the NED and NSO. In some cases this can affect the performance in NSO. Hence, tracing should
normally be disabled in production systems.
An alternative method for generating printouts from the NED is to enable the Java logging mechanism.
This makes the NED print log messages to common NSO Java log file.
IMPORTANT:
Java logging does not use any IPC messages sent to NSO. Consequently, NSO performance is not
affected. However, all log printouts from all log enabled devices are saved in one single file.
This means that the usability is limited. Typically single device use cases etc.
3. Dependencies
This NED has the following host environment dependencies:
Java 1.8 (NSO version < 6.2)
Java 17 (NSO version >= 6.2)
Gnu Sed
Dependencies for NED recompile:
Apache Ant
Bash
Gnu Sort
Gnu awk
Grep
Python3 (with packages: re, sys, getopt, subprocess, argparse, os, glob)
4. Sample device configuration
devices authgroups group nokia-auth-group
default-map remote-name <user>
default-map remote-password <password>
!
devices device dev-1
address <address>
port <port>
authgroup nokia-auth-group
device-type generic ned-id nokia-sros_nc-gen-1.0
trace raw
! Set get-config-no-filter true if openconfig models are being used
ned-settings nokia-sros_nc transaction get-config-no-filter true
ned-settings nokia-sros_nc transaction exclude-namespaces urn:ietf:params:xml:ns:yang:ietf-interfaces
ned-settings nokia-sros_nc live-status filter-invalid-values true
state admin-state unlocked
!
5. Built in RPC actions
5.1. rpc add-filter-path
Add a path to be filtered, possibly removing paths being made redundant.
Input arguments:
Either of:
- include <empty>
OR:
- exclude <empty>
- force <empty>
- path <string>
5.2. rpc clean-package
Cleans the NED package from all downloaded third party YANG files.
Input arguments:
- verbose <empty>
Print the full clean output also for successful executions (otherwise only printed on errors).
5.3. rpc clear-cached-capabilities
Clear all cached capabilities (module-set-id/content-id/yang-library/netconf-state).
No input arguments
5.4. rpc clear-filter-paths
Clear all filter-paths, except content from ned-setting 'filter-paths-file'.
No input arguments
5.5. rpc compare-config
Do a NED-internal compare-config, with data either from device or file, optionally disabling
filtering.
Input arguments:
- config-file <string>
Optional file to load config from instead of fetching from device (NOTE, should be content of
rpc-reply, i.e. config wrapped in data-tag).
- strict <empty>
Match defaults strict, according to capabilities.
- unfiltered <empty>
Don't apply filter-paths.
- outformat <enum> (default tree)
tree - Standard NSO diff tree.
compact - Compact diff showing key-paths.
xml - Show diff as netconf edit-config XML.
5.6. rpc compile-modules
Compile YANG modules, showing all non-fatal warnings found.
Input arguments:
- local-dir <string>
Path to the directory where the YANG files are found (defaults to src/yang in package).
- no-deviations <empty>
Set to disable deviations.
- ignore-errors <empty>
Ignore errors while compiling, i.e. which would normally cause compilation to abort.
5.7. rpc export-package
Export the customized and rebuilt NED. The exported archive file can then be used to install the
NED package in other NSO instances. The name of the file will have the following format ncs-<NSO
version>-<NED name>-<NED-version>-customized.tgz.
Input arguments:
- destination <string> (default /tmp)
Set destination directory for the exported archive file.
- suffix <string> (default -customized)
Configure a customized suffix to the name of the archive file.
5.8. rpc get-modules
Fetch the YANG modules from the device.
Input arguments:
- module-include-regex <string>
Regular expression matching all YANG models to be included in the download. Example:
'openconfig-.*'.
- module-exclude-regex <string>
Regular expression matching all YANG models to be excluded from the download. Example:
'tailf-.*'.
- namespace-include-regex <string>
Regular expression matching all namespaces to be included in the download. Example:
'tailf-.*'.
- namespace-exclude-regex <string>
Regular expression matching all namespaces to be excluded from the download. Example:
'tailf-.*'.
- module-set <string>
Only include modules from the given yang-library module-set (if device supports yang-library
1.1).
- only-present <empty>
- only-oper-filter <string>
- profile <union> (default sros-combined-from-device)
Use a download profile to match a predefined subset of matching YANG files.
- local-dir <string>
Path to the directory where the YANG files are to be copied (defaults to src/yang in package).
- ignore-errors <empty>
Ignore errors during download. For example missing files of failed revision checks.
Either of:
- remote device <empty>
The device itself.
OR:
- remote dir <string>
A directory on the local host holding all YANG files. For instance a local clone of a git
repository.
OR:
- remote archive <string>
A path to a zip/tgz archive file containing the YANG files.
OR:
- remote git repository <string>
The URL to the git repository. Example: https://github.com/YangModels/yang.git.
- remote git dir <string>
Path to a sub directory inside the git repo where the YANG files can be found. Example:
vendor/cisco/nx/10.1-2.
- remote git checkout <string>
Optionally, a name of a branch/tag in the git repo where the YANG files can be found.
Example: master.
- remote git include-dir <string>
Optional extra include paths to be used when searching for YANG files. Each include path
is relative to the git root directory.
5.9. rpc import-filter-paths
Import filter-paths from file, will be merged with currently loaded.
Input arguments:
- filter-file <string>
File containing filter-paths, one on each line: <include|exclude> <schema-path>.
5.10. rpc list-filter-paths
List currently loaded/generated filter-paths.
Input arguments:
- deviation-module <empty>
Generate a module which deviates all excluded paths as 'not-supported'.
- save-to-file <string>
Save result to given file. For deviation module, optionally just give name of module to
generate in src/yang.
5.11. rpc list-module-sets
List the yang-library module-sets advertised by the device, if device supports it.
No input arguments
5.12. rpc list-modules
List the YANG modules advertised by the device. Including revision tag.
Input arguments:
- module-include-regex <string>
Regular expression matching all YANG models to be included in the download. Example:
'openconfig-.*'.
- module-exclude-regex <string>
Regular expression matching all YANG models to be excluded from the download. Example:
'tailf-.*'.
- namespace-include-regex <string>
Regular expression matching all namespaces to be included in the download. Example:
'tailf-.*'.
- namespace-exclude-regex <string>
Regular expression matching all namespaces to be excluded from the download. Example:
'tailf-.*'.
- module-set <string>
Only include modules from the given yang-library module-set (if device supports yang-library
1.1).
- only-present <empty>
- only-oper-filter <string>
- profile <union> (default sros-combined-from-device)
Use a download profile to match a predefined subset of matching YANG files.
5.13. rpc list-profiles
List all predefined download profiles bundled with the NED. Including a short description of each.
No input arguments
5.14. rpc patch-modules
Patch YANG modules, to remove non-fatal warnings found.
Input arguments:
- local-dir <string>
Path to the directory where the YANG files are found (defaults to src/yang in package).
- no-deviations <empty>
Set to disable deviations.
- output-dir <string>
Path to the directory where the patched YANG files are written (defaults to src/yang in
package), existing files will be renamed to <name>.yang.orig.
5.15. rpc rebuild-package
Rebuild the NED package directly from within NSO. This invokes the gnu make internally.
Input arguments:
- verbose <empty>
Print the full build output also for successful builds (otherwise only printed on errors).
- profile <string>
Apply a certain build profile.
- filter scope dir <string>
Directory containing one or many xml file representing the wanted scope.
- filter trim-schema nodes <union>
List of nodes to trim. Use one of the pre-defined top node names. Alternatively, specify a
custom xpath to trim (prefix is mandatory on each element in the path).
- filter trim-schema file <string> (default /tmp/nedcom-trim-deviations.yang)
Name of auto generated deviation file with nodes to trim.
- filter auto-config dir <string>
Directory containing the files used for auto-config filtering. The following files must be
present: before.xml and after.xml.
- filter auto-config file <string> (default /tmp/nedcom-auto-deviations.yang)
Name of auto generated deviation file.
- ned-id major <string>
Set a custom major number in the generated ned-id.
- ned-id minor <string>
Set a custom minor number in the generated ned-id.
- ned-id suffix <string>
Set a custom suffix in the generated ned-id.
5.16. rpc remove-filter-path
Remove a path from filter-paths.
Input arguments:
Either of:
- include <empty>
OR:
- exclude <empty>
- path <string>
5.17. rpc show-default-local-dir
Show the path to the default directory where the YANG files are to be copied. I.e <path to current
NED package>/src/yang.
No input arguments
5.18. rpc show-loaded-schema
Display the schema currently built into the NED package. Each node will by default be listed with
a schema path.
Input arguments:
- scope <enum> (default all)
Select the scope for the nodes that will be listed.
all - Display all nodes in the schema. This is the default.
used - Display only the config nodes in use, i.e currently populated in CDB.
unused - Display only the config nodes that are not in use.
- count <empty>
Count the nodes and return the sum instead of the full list of nodes.
- root-paths <string>
Specify root paths for which nodes shall be listed or counted. Only nodes with a schema path
starting any of the specified roots will then be processed.
- details <empty>
Display schema details like must/when expression, leafrefs and leafref targets.
- config <true|false> (default true)
Set to false to display non config nodes in the schema. Note: scope will in this case be
'all'.
5.19. rpc verify-get-config
Verify XML contents of config, either from device or file, to validate
data and look for unmodeled structures (the yang-modules are compiled on
the fly making this a convenient way to verify yang-updates).
Input arguments:
Either of:
- local-dir <string>
Path to the directory where the YANG files are found (defaults to src/yang in package).
- no-deviations <empty>
Set to disable deviations.
- patch <empty>
Auto-patch yang when possible (e.g. missing leafref targets will expand referrer type).
- config-file <string>
Optional file to load config from instead of fetching from device (NOTE, should be content
of rpc-reply, i.e. config wrapped in data-tag).
OR:
- no-compile <empty>
- verbose <empty>
Show verbose output, like 'sync-from verbose'.
6. Built in live-status show
The nokia-sroc_nc NED has full support for fetching operational data via the NSO live-status API.
7. Limitations
The SROS device must be properly configured in model driven mode and with netconf enabled for this NED to function properly.
Any other mode such as 'mixed mode' makes SROS limited and will prevent the NED from using certain features. For instance
the fast transaction id calculation nethod which has been customized for SROS.
Some issues has been observed when running towards SROS devices. These are briefly described here, please note that these
are only what have been observed in specific use-cases, it might be related to device/SROS model/version, and also to specific
configuration used in these use-cases. Hence, these issues might not be relevant for your use-case, and of course, there might
be other issues which have not been observed.
As has been observed with many vendors yang-models, SROS seems to contain some leafrefs that are not enforced in the
device, which might lead to NSO reporting "illegal reference" when doing sync-from. See file src/customize-schema.schypp
for these, i.e. paths which have an 'inline_type' operation in this file. Also see the README-rebuild.md for more info on
mitigating "illegal reference".
In some older versions of SROS, support for yang-library version 1.1 is announced, however, the implementation has bugs in it
(this relates at least to SROS versions <= 20.10 which announce yang-library 1.1 support). In this case the NED needs to be
configured to make it look like the device actually implements yang-library 1.0. This can be done by setting the
below ned-settings:
NOTE: The "fake" module-set-id used here is not relevant, it can be anything
Another issue which has been observed is that while the device announces support for the yang-module ietf-interfaces, it
can not be used for example inside the filter of a get-config request. Hence this module needs to be filtered out, which is
done with the below ned-setting:
At least SROS version 22 has problems with properly returning valid data from the openconfig data tree when the NED is doing
a get-config request. The device seems to have problems with the default filter used by the NED. If the NED is built with
openconfig, this filter will contain namespaces of the openconfig models. This will result in the device returning an error
instead of the valid config.
This issue can be solved by configuring the following NED setting:
All current versions of SROS have a special non rfc compliant behaviour for settings related to the configuration groups 'bof' and 'li'.
These configuration trees are regarded as separate datastores by SROS. This means SROS does not allow editing standard config together
with 'bof' or 'li' settings in the same transaction. Furthermore editing 'bof' and 'li' settings does require Nokia specific non rfc
compliant extensions to the netconf messages.
Do as follows to enable support for managing 'bof' settings in the NED:
Make sure the Nokia 'bof' models are included when rebuilding the NED.
For example, use the profile 'sros-combined-with-bof-from-device' when downloading the models.
The following information is required for the Cisco NSO NED team to be able
to investigate an issue:
- A detailed recipe with steps to reproduce the issue.
- A raw trace file generated when the issue is reproduced.
- Access to a device where the issue can be reproduced by the Cisco NSO NED team.
This typically means both read and write permissions are required.
Pseudo access via tools like Webex, Zoom etc is not acceptable.
However, it is ok with device access through VPNs, jump servers etc though.
Do as follows to gather the necessary information needed for your device, here named 'dev-1':
Reproduce the found issue using ncs_cli or your NSO service.
Write down each necessary step in a reproduction report.
Gather the reproduction report and a copy of the raw trace file
containing data recorded when the issue happened.
Contact the Cisco support and request to open a case. Provide the gathered files
together with access details for a device that can be used by the
Cisco NSO NED when investigating the issue.
Requests for new features and extensions of the NED are handled by the Cisco NSO NED team when
applicable. Such requests shall also go through the Cisco support channel.
The following information is required for feature requests and extensions:
A detailed use case description, with details like:
Data of interest
The kind of operations to be used on the data. Like: 'read', 'create', 'update', 'delete'
and the order of the operation
Device APIs involved in the operations (For example: REST URLs and payloads)
Device documentation describing the operations involved
Run sync-from # devices device dev-1 sync-from (if relevant)
Attach the raw trace to the ticket (if relevant)
Access to a device that can be used by the Cisco NSO NED team for testing and verification
of the new feature. This usually means that both read and write permissions are required.
Pseudo access via tools like Webex, Zoom etc is not acceptable. However, it is ok with access
through VPNs, jump servers etc.
9. How to rebuild a NED
Check the README-rebuild.md file, chapter 1.3, for more information.
10. Configure the NED to use ssh multi factor authentication
This NED supports multi factor authentication (MFA) using the ssh authentication
method 'keyboard-interactive'.
Some additional steps are required to enable the MFA support:
Verify that your NSO version supports MFA. This is configurable as additional
settings in the authentication group used by the device instance.
Enter a NSO CLI and enter the following and do tab completion:
> ncs_cli -C -u admin
admin@ncs# show running-config devices authgroups group default default-map <tab>
Possible completions:
action-name The action to call when a notification is received.
callback-node Invoke a standalone action to retrieve login credentials for managed devices on the 'callback-node' instance.
mfa Settings for handling multi-factor authentication towards the device
public-key Use public-key authentication
remote-name Specify device user name
remote-password Specify the remote password
remote-secondary-password Second password for configuration
same-pass Use the local NCS password as the remote password
same-secondary-password Use the local NCS password as the remote secondary password
same-user Use the local NCS user name as the remote user name
If 'mfa' is displayed in the output like above, NSO has MFA support enabled.
In case MFA is not supported it is necessary to upgrade NSO before proceeding.
Implement the authenticator executable. The MFA feature relies on an external executable to take care of the client part
of the multi factor authentication. The NED will automatically call this executable for each challenge presented by the
ssh server and expects to get a proper response in return.
The executable can be a simple shell script or a program implemented in any programming language.
The required behaviour is like this:
read one line from stdin
The line passed from the NED will be a semi colon separated string containing the following info:
[<device name>;<user>;<password>;<opaque>;<ssh server name>;<ssh server instruction>;<ssh server prompt>;]
The elements for device name, user, password and opaque corresponds to what has been configured in NSO.
The ssh server name, instruction and prompt are given by the ssh server during the authentication step.
Each individual element in the semi colon separated list is Base64 encoded.
Extract the challenge based on the contents above.
Print a response matching the challenge to stdout and exit with code 0
In case a matching response can not be given do exit with code 2
Below is a simple example of an MFA authenticator implemented in Python3:
#!/usr/bin/env python3
import sys
import base64
# This is an example on how to implement an external multi factor authentication handler
# that will be called by the NED upon a ssh 'keyboard-interactive' authentication
# The handler is reading a line from stdin with the following expected format:
# [<device name>;<user>;<password>;<opaque>;<ssh server name>;<ssh server instruction>;<ssh server prompt>;]
# All elements are base64 encoded.
def decode(arg):
return str(base64.b64decode(arg))[2:-1]
if __name__ == '__main__':
query_challenges = {
"admin@localhost's password: ":'admin',
'Enter SMS passcode:':'secretSMScode',
'Press secret key: ':'2'
}
# read line from stdin and trim brackets
line = sys.stdin.readline().strip()[1:-1]
args = line.split(';')
prompt = decode(args[6])
if prompt in query_challenges.keys():
print(query_challenges[prompt])
exit(0)
else:
exit(2)
Configure the authentication group used by the device instance to enable MFA. There
are two configurables available:
executable The path to the external multi factor authentication executable (mandatory).
opaque Opaque data that will passed as a cookie element to the executable (optional).
> ncs_cli -C -u admin
admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# devices authgroups group <name> default-map mfa executable <path to the executable>
admin@ncs(config)# devices authgroups group <name> default-map mfa opaque <some opaque data>
admin@ncs(config)# commit
Try connecting to the device.
10.1 Trouble shooting
In case of connection problems the following steps can help for debugging:
Verify that the ssh client is using the external authenticator executable:
using ssh external mfa executable: <configured path to executable>
Verify that the executable is called with the challenges presented by the ssh server:
calling external mfa executable with ssh server given name: '<name>', instruction: '<instruction>', prompt '<challenge>'
Check for any errors reported by the NED when calling the executable
ERROR: external mfa executable failed <....>
11. Run arbitrary commands on device
Some commands that are available to a user logged in to an interactive CLI
session on the device might not be available through NETCONF. For situations
like this the NED provides the feature to run arbitrary commands on the device.
The NED feature to execute commands is connected in NSO to a live-status action
called 'exec any'.
The NED does by default use the 'md-cli-raw-command' feature to tunnel commands
to the device. This feature is supported by all newer versions of Nokia SROS.
It is also possible to configure the NED to use an interactive SSH login to
the device. This extra SSH session is then handled internally in the NED.
This option works also with older versions of SROS.
There are some ned-settings to control the behaviour of this feature, see the
section 'ned-settings nokia-sros_nc live-status exec-any-mode and cli' in
README-ned-settings.md for details on this.
admin@ncs# devices device dev-1 live-status exec any admin display-config
Note that when using ncs_cli, the command-line given might need to be quoted
if it contains characters that are interpreted by the ncs_cli itself.
