Device Programmability

The tools we currently use to configure and monitor our networks aren’t great for network automation.

The command-line interface (CLI) is great for us humans, but not so great for network automation. When you enter a command, there is no clear feedback. On Cisco IOS, you see an empty prompt when it accepts a command. When the command has an error, you see an error message. The output of the error message can be different depending on the command you used. Show and debug commands are also problematic because there is no uniform format. The layout and presentation of each show and debug command is different. They are easy to read for humans but difficult to parse with scripts and network automation software.

SNMP can be used to configure network devices, but in reality this isn’t used much. SNMP is widely in use to monitor networks.

For network automation we need:

• A programmatic interface for device configuration.
• Separation between configuration and operational data.
• Integrated error checking and recovery.

There are three network automation protocols that meet these requirements:

• NETCONF (RFC 4741) – Since 2006
• RESTCONF (RFC 8040) – Since 2017
• gRPC (RFC 6020) – Since 2015

NETCONF is the oldest protocol. gRPC is an open source protocol from Google. RESTCONF is the latest protocol from 2017. These protocols use YANG data models.

If you are new to network automation and want to start with a less programmatic way of configuring network devices with automation, check out our Introduction to Ansible first.

In this lesson, I’ll explain how these protocols work and you will learn how to use them to configure and monitor a router. Let’s get started!

NETCONF

NETCONF is a protocol developed by IETF to “install, manipulate, and delete the configuration of network devices”. The goal of NETCONF is to make network automation easier. It uses XML for data encoding and Remote Procedure Call (RPC) for messages. It runs over SSH.

Here is an overview of the different NETCONF layers:

• We use XML as the data format:
  • The items we want to configure or retrieve from a network device are encapsulated in the <data> tag.
  • We have different actions, for example <get-config> to retrieve the configuration or <edit-config> to configure the device.
  • We execute operations as remote procedure calls (RPCs). You can recognize this with the<rpc> and <rpc-reply> tags.
  • We run NETCONF over SSH.

Here is a full list of NETCONF operations:

Configuration

Let’s see NETCONF in action.

This is the topology I’ll use:

R1 is a CSR1000V router running IOS XE Software, Version 16.06.01.

Router

We need to enable NETCONF and create a privilege level 15 user on the router:

R1(config)#netconf-yang
R1(config)#username cisco privilege 15 secret cisco

NETCONF requires only a single command. It runs on TCP port 830 by default. Let’s SSH into that port:

$ ssh cisco@172.16.1.1 -p 830 netconf

The router responds with a “hello” message:

<?xml version="1.0" encoding="UTF-8"?>
<hello
  xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <capabilities>
    <capability>urn:ietf:params:netconf:base:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:writeable-running:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:startup:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:url:1.0</capability>
    <capability>urn:cisco:params:netconf:capability:pi-data-model:1.0</capability>
    <capability>urn:cisco:params:netconf:capability:notification:1.0</capability>
  </capabilities>
  <session-id>2035438880</session-id></hello>]]>]]>

This hello message contains the capabilities that the router supports. We need to reply to the router with a hello message of our own. This hello message contains the capabilities that we support:

<?xml version="1.0" encoding="UTF-8"?>
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
    <capabilities>
        <capability>urn:ietf:params:netconf:base:1.0</capability>
    </capabilities>
</hello>]]>]]>

The router won’t show anything to acknowledge our hello message but we now have an active NETCONF session. Let’s request information about the GigabitEthernet2 interface of the router:

<?xml version="1.0"?>
<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"xmlns:cpi="http://www.cisco.com/cpi_10/schema"
message-id="101">
   <get-config>
        <source>
             <running/>
         </source>
          <filter>
              <config-format-text-cmd>
               <text-filter-spec>
        interface GigabitEthernet2
                 </text-filter-spec>
              </config-format-text-cmd>
          </filter>
      </get-config>
</rpc>

Above, you can see the XML formatted message is embedded with the <rpc> </rpc> tags. I also embed our action with the <get-config> tag. The router replies with the following message:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply message-id="101" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><data><cli-c</cmd>data><cmd>!
</cmd>nterface GigabitEthernet2
</cmd>ip address dhcp
</cmd>negotiation auto
</cmd></cli-config-data></data>
</rpc-reply>]]>]]>

We receive the configuration of the GigabitEthernet2 interface in XML format, embedded with the <rpc-reply> tag. The configuration of the router is embedded within the <data> tag. You can also see that the message-id matches the ID that I added in my request. Once we are finished, we need to close the session with the router by sending the following message:

<?xml version="1.0" encoding="UTF-8"?>
 <rpc message-id="102" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
 <close-session />
 </rpc>
 ]]>]]>

The router lets us know that the session is closed:

<?xml version="1.0" encoding="UTF-8"?><rpc-reply message-id="102" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><ok /></rpc-reply>]]>]]>

Connection to 172.16.1.1 closed by remote host.

Interacting with NETCONF manually over SSH like this works but it’s a terrible method. It’s inconvenient and, it’s easy to make errors. Instead, we should use clients or code libraries that do most of the work for us. Ncclient is a popular python library we can use for NETCONF.

Let’s install it:

$ pip install ncclient

I created some NETCONF sample scripts that we can run against our router. Let’s retrieve the capabilities of the router:

$ python netconf-get-capabilities.py

The router replies with a list of all its capabilities:

http://cisco.com/ns/yang/Cisco-IOS-XE-features?module=Cisco-IOS-XE-features&revision=2017-02-07&features=virtual-template,punt-num,parameter-map,multilink,l2vpn,l2,ezpm,eth-evc,esmc,efp,crypto
http://openconfig.net/yang/vlan?module=openconfig-vlan&revision=2016-05-26&deviations=cisco-xe-openconfig-vlan-deviation
http://openconfig.net/yang/network-instance-l3?module=openconfig-network-instance-l3&revision=2017-01-13
urn:ietf:params:xml:ns:yang:smiv2:RMON2-MIB?module=RMON2-MIB&revision=1996-05-27
urn:ietf:params:xml:ns:yang:smiv2:CISCO-VOICE-DIAL-CONTROL-MIB?module=CISCO-VOICE-DIAL-CONTROL-MIB&revision=2012-05-15
urn:ietf:params:xml:ns:yang:smiv2:MPLS-TE-STD-MIB?module=MPLS-TE-STD-MIB&revision=2004-06-03
urn:ietf:params:xml:ns:yang:smiv2:SNMP-TARGET-MIB?module=SNMP-TARGET-MIB&revision=1998-08-04
urn:ietf:params:xml:ns:yang:smiv2:CISCO-PTP-MIB?module=CISCO-PTP-MIB&revision=2011-01-28
urn:ietf:params:xml:ns:yang:smiv2:CISCO-VLAN-MEMBERSHIP-MIB?module=CISCO-VLAN-MEMBERSHIP-MIB&revision=2007-12-14
http://cisco.com/ns/yang/Cisco-IOS-XE-card?module=Cisco-IOS-XE-card&revision=2017-02-07
urn:ietf:params:xml:ns:yang:smiv2:CISCO-IP-TAP-MIB?module=CISCO-IP-TAP-MIB&revision=2004-03-11

I omitted some output above since the router produces a big list with capabilities it supports. How about retrieving the running configuration of our router? Let’s try it:

$ python netconf-get-running-configuration.py

We receive the running configuration in XML output:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:e82715c5-271c-44c8-9385-e13c3370f51a">
  <data>
    <native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native">
      <version>16.6</version>
      <boot-start-marker />
      <boot-end-marker />
      <service>
        <timestamps>
          <debug>
            <datetime>
              <msec />
            </datetime>
          </debug>
          <log>
            <datetime>
              <msec />
            </datetime>
          </log>
        </timestamps>
      </service>
      <platform>
        <console xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-platform">
          <output>serial</output>
        </console>
      </platform>
      <hostname>R1</hostname>
      <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
        <interface>
          <name>GigabitEthernet1</name>
          <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
          <enabled>true</enabled>
          <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
            <address>
              <ip>10.255.1.209</ip>
              <netmask>255.255.0.0</netmask>
            </address>
          </ipv4>
          <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip" />
        </interface>
        <interface>
          <name>GigabitEthernet2</name>
          <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
          <enabled>true</enabled>
          <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
            <address>
              <ip>172.16.1.1</ip>
              <netmask>255.255.255.0</netmask>
            </address>
          </ipv4>
          <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip" />
        </interface>
      </interfaces>
    </native>
  </data>
</rpc-reply>

In the output above, we see the running configuration formatted in XML. I omitted some information since the XML output of the entire running configuration is quite long.

If I am only interested in a small part of the configuration then I can use a filter. For example, let’s only retrieve the interface configuration with another script:

$ python netconf-get-running-configuration-filter.py

We now receive the XML configuration of our interfaces:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:8af7b1b2-ed46-4caa-a23f-f7975cec67ec">
  <data>
    <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
      <interface>
        <name>GigabitEthernet1</name>
        <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
        <enabled>true</enabled>
        <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
          <address>
            <ip>10.255.1.209</ip>
            <netmask>255.255.0.0</netmask>
          </address>
        </ipv4>
        <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip" />
      </interface>
      <interface>
        <name>GigabitEthernet2</name>
        <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
        <enabled>true</enabled>
        <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
          <address>
            <ip>172.16.1.1</ip>
            <netmask>255.255.255.0</netmask>
          </address>
        </ipv4>
        <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip" />
      </interface>
    </interfaces>
  </data>
</rpc-reply>

Excellent. We can retrieve configurations from the router but how about adding something to the configuration? I can use the output above as a blueprint to create a template for a new loopback interface.

Let’s try another script to create a loopback interface on our router:

$ python netconf-edit-config-add-loopback.py

The router replies with an OK message:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:d8bd3e0e-da45-41e7-80b4-49722ed4f06f">
  <ok />
</rpc-reply>

We can verify whether the loopback is created with our previous script to fetch the running configuration or we can use the CLI:

R1#show running-config interface Loopback 1
Building configuration...

Current configuration : 63 bytes
!
interface Loopback1
 ip address 1.1.1.1 255.255.255.255
end

What about deleting something from the router? Let’s delete that loopback we just created with another script:

$ python netconf-delete-config-loopback.py

The router replies with another OK:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:e60f2c59-72de-483f-a934-e60de6451749">
  <ok />
</rpc-reply>

The interface is deleted. That’s it. I hope these examples have been useful to understand the basics of NETCONF.

RESTCONF

RESTCONF is protocol which works similar to a REST API. It maps a YANG specification to a RESTful interface and uses the HTTPS protocol for transport. You can use JSON or XML as data formats. RESTCONF is newer than NETCONF but not a replacement. It’s more of a lightweight option for engineers who are familiar with REST APIs.

Here is an overview of the RESTCONF layers:

In the picture above, you can see that RESTCONF uses HTTP actions (GET, POST, PUT, PATCH, DELETE) for its operations.

Below is a comparison of RESTCONF operations to NETCONF operations:

The structure of a RESTCONF URL looks like this:

https://address/root/data/yang-module:container/leaf?options

Let me explain the highlighted parts:

• address: This is the hostname or IP address of the RESTCONF agent (network device).
• root: The main entry point for RESTCONF requests.
• data: The RESTCONF API resource type for data.
• yang-module:container: The base YANG data model container we want to use. The container is optional.
• leaf: An individual element from within the container.
• ?options: Optional parameters that impact the returned results.

Let me give you an example of what a RESTCONF URL looks like. To create one, we need YANG data models. You can find these in the YANG git repository. Here’s the IETF interfaces YANG data model to retrieve interface information from a Cisco IOS XE router.

You can scroll through the YANG file but it’s difficult to read. There are better ways to view a YANG data model. One great open source tool is pyang. Pyang is a python YANG validator, transformer, and code generator. You can use it to validate YANG modules or convert them to other formats.

We can use pyang to look at the IETF interfaces data model in a tree format:

$ pyang -f tree ietf-interfaces.yang --tree-path interfaces
module: ietf-interfaces
  +--rw interfaces
     +--rw interface* [name]
        +--rw name                        string
        +--rw description?                string
        +--rw type                        identityref
        +--rw enabled?                    boolean
        +--rw link-up-down-trap-enable?   enumeration {if-mib}?

This is easier to read than a plain text YANG file. The items above give me the information I need for the URL that requests interface information: