Device Programmability

The tools we currently use to configure and monitor our networks aren’t great for network automation.

The command-line interface (CLI) is great for us humans, but not so great for network automation. When you enter a command, there is no clear feedback. On Cisco IOS, you see an empty prompt when it accepts a command. When the command has an error, you see an error message. The output of the error message can be different depending on the command you used. Show and debug commands are also problematic because there is no uniform format. The layout and presentation of each show and debug command is different. They are easy to read for humans but difficult to parse with scripts and network automation software.

SNMP can be used to configure network devices, but in reality this isn’t used much. SNMP is widely in use to monitor networks.

For network automation we need:

  • A programmatic interface for device configuration.
  • Separation between configuration and operational data.
  • Integrated error checking and recovery.

There are three network automation protocols that meet these requirements:

NETCONF is the oldest protocol. gRPC is an open source protocol from Google. RESTCONF is the latest protocol from 2017. These protocols use YANG data models.

If you are new to network automation and want to start with a less programmatic way of configuring network devices with automation, check out our Introduction to Ansible first.

In this lesson, I’ll explain how these protocols work and you will learn how to use them to configure and monitor a router. Let’s get started!



NETCONF

NETCONF is a protocol developed by IETF to “install, manipulate, and delete the configuration of network devices”. The goal of NETCONF is to make network automation easier. It uses XML for data encoding and Remote Procedure Call (RPC) for messages. It runs over SSH.

Here is an overview of the different NETCONF layers:

Netconf Protocol Layers

  • We use XML as the data format:
    • The items we want to configure or retrieve from a network device are encapsulated in the <data> tag.
    • We have different actions, for example <get-config> to retrieve the configuration or <edit-config> to configure the device.
    • We execute operations as remote procedure calls (RPCs). You can recognize this with the<rpc> and <rpc-reply> tags.
    • We run NETCONF over SSH.

Here is a full list of NETCONF operations:

Operation Description
<get> Retrieve running configuration and device state
information.
<get-config> Retrieve all or part of a specified configuration
datastore.
<edit-config> The <edit-config> operation loads all or part of a specified
configuration to the specified target configuration datastore.
<copy-config> Create or replace an entire configuration datastore
with the contents of another complete configuration datastore.
<delete-config> Delete a configuration datastore. The <running>
configuration datastore cannot be deleted.
<commit> The <commit> operation instructs the device to implement the
configuration data contained in the candidate configuration.
<lock> The <lock> operation allows the client to lock the
entire configuration datastore system of a device.
<unlock> The <unlock> operation is used to release a
configuration lock, previously obtained with the <lock> operation.
<close-session> Request graceful termination of a NETCONF session.
<kill-session> Force the termination of a NETCONF session.

 

Configuration

Let’s see NETCONF in action.




This is the topology I’ll use:

R1 H1

R1 is a CSR1000V router running IOS XE Software, Version 16.06.01.

Router

We need to enable NETCONF and create a privilege level 15 user on the router:

R1(config)#netconf-yang
R1(config)#username cisco privilege 15 secret cisco

NETCONF requires only a single command. It runs on TCP port 830 by default. Let’s SSH into that port:

$ ssh cisco@172.16.1.1 -p 830 netconf

The router responds with a “hello” message:

<?xml version="1.0" encoding="UTF-8"?>
<hello
  xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <capabilities>
    <capability>urn:ietf:params:netconf:base:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:writeable-running:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:startup:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:url:1.0</capability>
    <capability>urn:cisco:params:netconf:capability:pi-data-model:1.0</capability>
    <capability>urn:cisco:params:netconf:capability:notification:1.0</capability>
  </capabilities>
  <session-id>2035438880</session-id></hello>]]>]]>

This hello message contains the capabilities that the router supports. We need to reply to the router with a hello message of our own. This hello message contains the capabilities that we support:

<?xml version="1.0" encoding="UTF-8"?>
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
    <capabilities>
        <capability>urn:ietf:params:netconf:base:1.0</capability>
    </capabilities>
</hello>]]>]]>

The router won’t show anything to acknowledge our hello message but we now have an active NETCONF session. Let’s request information about the GigabitEthernet2 interface of the router:

<?xml version="1.0"?>
<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"xmlns:cpi="http://www.cisco.com/cpi_10/schema"
message-id="101">
   <get-config>
        <source>
             <running/>
         </source>
          <filter>
              <config-format-text-cmd>
               <text-filter-spec>
        interface GigabitEthernet2
                 </text-filter-spec>
              </config-format-text-cmd>
          </filter>
      </get-config>
</rpc>

Above, you can see the XML formatted message is embedded with the <rpc> </rpc> tags. I also embed our action with the <get-config> tag. The router replies with the following message:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply message-id="101" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><data><cli-c</cmd>data><cmd>!
</cmd>nterface GigabitEthernet2
</cmd>ip address dhcp
</cmd>negotiation auto
</cmd></cli-config-data></data>
</rpc-reply>]]>]]>

We receive the configuration of the GigabitEthernet2 interface in XML format, embedded with the <rpc-reply> tag. The configuration of the router is embedded within the <data> tag. You can also see that the message-id matches the ID that I added in my request. Once we are finished, we need to close the session with the router by sending the following message:

<?xml version="1.0" encoding="UTF-8"?>
 <rpc message-id="102" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
 <close-session />
 </rpc>
 ]]>]]>

The router lets us know that the session is closed:

<?xml version="1.0" encoding="UTF-8"?><rpc-reply message-id="102" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><ok /></rpc-reply>]]>]]>

Connection to 172.16.1.1 closed by remote host.

Interacting with NETCONF manually over SSH like this works but it’s a terrible method. It’s inconvenient and, it’s easy to make errors. Instead, we should use clients or code libraries that do most of the work for us. Ncclient is a popular python library we can use for NETCONF.

Let’s install it:

$ pip install ncclient

I created some NETCONF sample scripts that we can run against our router. Let’s retrieve the capabilities of the router:

$ python netconf-get-capabilities.py

The router replies with a list of all its capabilities:

http://cisco.com/ns/yang/Cisco-IOS-XE-features?module=Cisco-IOS-XE-features&revision=2017-02-07&features=virtual-template,punt-num,parameter-map,multilink,l2vpn,l2,ezpm,eth-evc,esmc,efp,crypto
http://openconfig.net/yang/vlan?module=openconfig-vlan&revision=2016-05-26&deviations=cisco-xe-openconfig-vlan-deviation
http://openconfig.net/yang/network-instance-l3?module=openconfig-network-instance-l3&revision=2017-01-13
urn:ietf:params:xml:ns:yang:smiv2:RMON2-MIB?module=RMON2-MIB&revision=1996-05-27
urn:ietf:params:xml:ns:yang:smiv2:CISCO-VOICE-DIAL-CONTROL-MIB?module=CISCO-VOICE-DIAL-CONTROL-MIB&revision=2012-05-15
urn:ietf:params:xml:ns:yang:smiv2:MPLS-TE-STD-MIB?module=MPLS-TE-STD-MIB&revision=2004-06-03
urn:ietf:params:xml:ns:yang:smiv2:SNMP-TARGET-MIB?module=SNMP-TARGET-MIB&revision=1998-08-04
urn:ietf:params:xml:ns:yang:smiv2:CISCO-PTP-MIB?module=CISCO-PTP-MIB&revision=2011-01-28
urn:ietf:params:xml:ns:yang:smiv2:CISCO-VLAN-MEMBERSHIP-MIB?module=CISCO-VLAN-MEMBERSHIP-MIB&revision=2007-12-14
http://cisco.com/ns/yang/Cisco-IOS-XE-card?module=Cisco-IOS-XE-card&revision=2017-02-07
urn:ietf:params:xml:ns:yang:smiv2:CISCO-IP-TAP-MIB?module=CISCO-IP-TAP-MIB&revision=2004-03-11

I omitted some output above since the router produces a big list with capabilities it supports. How about retrieving the running configuration of our router? Let’s try it:

$ python netconf-get-running-configuration.py

We receive the running configuration in XML output:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:e82715c5-271c-44c8-9385-e13c3370f51a">
  <data>
    <native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native">
      <version>16.6</version>
      <boot-start-marker />
      <boot-end-marker />
      <service>
        <timestamps>
          <debug>
            <datetime>
              <msec />
            </datetime>
          </debug>
          <log>
            <datetime>
              <msec />
            </datetime>
          </log>
        </timestamps>
      </service>
      <platform>
        <console xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-platform">
          <output>serial</output>
        </console>
      </platform>
      <hostname>R1</hostname>
      <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
        <interface>
          <name>GigabitEthernet1</name>
          <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
          <enabled>true</enabled>
          <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
            <address>
              <ip>10.255.1.209</ip>
              <netmask>255.255.0.0</netmask>
            </address>
          </ipv4>
          <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip" />
        </interface>
        <interface>
          <name>GigabitEthernet2</name>
          <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
          <enabled>true</enabled>
          <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
            <address>
              <ip>172.16.1.1</ip>
              <netmask>255.255.255.0</netmask>
            </address>
          </ipv4>
          <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip" />
        </interface>
      </interfaces>
    </native>
  </data>
</rpc-reply>

In the output above, we see the running configuration formatted in XML. I omitted some information since the XML output of the entire running configuration is quite long.

If I am only interested in a small part of the configuration then I can use a filter. For example, let’s only retrieve the interface configuration with another script:

$ python netconf-get-running-configuration-filter.py

We now receive the XML configuration of our interfaces:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:8af7b1b2-ed46-4caa-a23f-f7975cec67ec">
  <data>
    <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
      <interface>
        <name>GigabitEthernet1</name>
        <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
        <enabled>true</enabled>
        <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
          <address>
            <ip>10.255.1.209</ip>
            <netmask>255.255.0.0</netmask>
          </address>
        </ipv4>
        <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip" />
      </interface>
      <interface>
        <name>GigabitEthernet2</name>
        <type xmlns:ianaift="urn:ietf:params:xml:ns:yang:iana-if-type">ianaift:ethernetCsmacd</type>
        <enabled>true</enabled>
        <ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip">
          <address>
            <ip>172.16.1.1</ip>
            <netmask>255.255.255.0</netmask>
          </address>
        </ipv4>
        <ipv6 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip" />
      </interface>
    </interfaces>
  </data>
</rpc-reply>

Excellent. We can retrieve configurations from the router but how about adding something to the configuration? I can use the output above as a blueprint to create a template for a new loopback interface.

NETCONF is XML-based, so network vendors need to model their configuration structure for their network devices, which means that somebody (your network vendor) needs to model their configuration structure appropriately. YANG is often used for this. YANG data modules are for NETCONF what MIBs are for SNMP.

Let’s try another script to create a loopback interface on our router:

$ python netconf-edit-config-add-loopback.py

The router replies with an OK message:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:d8bd3e0e-da45-41e7-80b4-49722ed4f06f">
  <ok />
</rpc-reply>

We can verify whether the loopback is created with our previous script to fetch the running configuration or we can use the CLI:

R1#show running-config interface Loopback 1
Building configuration...

Current configuration : 63 bytes
!
interface Loopback1
 ip address 1.1.1.1 255.255.255.255
end

What about deleting something from the router? Let’s delete that loopback we just created with another script:

$ python netconf-delete-config-loopback.py

The router replies with another OK:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:e60f2c59-72de-483f-a934-e60de6451749">
  <ok />
</rpc-reply>

The interface is deleted. That’s it. I hope these examples have been useful to understand the basics of NETCONF.

RESTCONF

RESTCONF is protocol which works similar to a REST API. It maps a YANG specification to a RESTful interface and uses the HTTPS protocol for transport. You can use JSON or XML as data formats. RESTCONF is newer than NETCONF but not a replacement. It’s more of a lightweight option for engineers who are familiar with REST APIs.

Here is an overview of the RESTCONF layers:

Restconf Protocol Stack

In the picture above, you can see that RESTCONF uses HTTP actions (GET, POST, PUT, PATCH, DELETE) for its operations.

Below is a comparison of RESTCONF operations to NETCONF operations:

RESTCONF NETCONF
GET <get>, <get-config>
POST <edit-config> (operation=”create”)
PUT <edit-config> (operation=”create/replace”)
PATCH <edit-config> (operation=”merge”)
DELETE <edit-config> (operation=”delete”)

The structure of a RESTCONF URL looks like this:

https://address/root/data/yang-module:container/leaf?options

Let me explain the highlighted parts:

  • address: This is the hostname or IP address of the RESTCONF agent (network device).
  • root: The main entry point for RESTCONF requests.
  • data: The RESTCONF API resource type for data.
  • yang-module:container: The base YANG data model container we want to use. The container is optional.
  • leaf: An individual element from within the container.
  • ?options: Optional parameters that impact the returned results.

Let me give you an example of what a RESTCONF URL looks like. To create one, we need YANG data models. You can find these in the YANG git repository. Here’s the IETF interfaces YANG data model to retrieve interface information from a Cisco IOS XE router.

You can scroll through the YANG file but it’s difficult to read. There are better ways to view a YANG data model. One great open source tool is pyang. Pyang is a python YANG validator, transformer, and code generator. You can use it to validate YANG modules or convert them to other formats.

We can use pyang to look at the IETF interfaces data model in a tree format:

$ pyang -f tree ietf-interfaces.yang --tree-path interfaces
module: ietf-interfaces
  +--rw interfaces
     +--rw interface* [name]
        +--rw name                        string
        +--rw description?                string
        +--rw type                        identityref
        +--rw enabled?                    boolean
        +--rw link-up-down-trap-enable?   enumeration {if-mib}?

This is easier to read than a plain text YANG file. The items above give me the information I need for the URL that requests interface information:

We're Sorry, Full Content Access is for Members Only...

If you like to keep on reading, Become a Member Now!

  • Learn any CCNA, CCNP and CCIE R&S Topic. Explained As Simple As Possible.
  • Try for Just $1. The Best Dollar You’ve Ever Spent on Your Cisco Career!
  • Full Access to our 800 Lessons. More Lessons Added Every Week!
  • Content created by Rene Molenaar (CCIE #41726)
531 Sign Ups in the last 30 days
satisfaction-guaranteed

  • 100% Satisfaction Guaranteed!
  • You may cancel your monthly membership at any time.
  • No Questions Asked!

Tags:


Forum Replies

  1. Hello Rene,

    I have been watching videos about Device Programmability and Automation for last two months. It seems Netconf and Restconf more complicated compared to Netmiko or Ansible. Besides that I see a lot of companies asking for Restconf. I think we can do a lot of configurations easily by using Ansible. In presence of Ansible or other tools, why do companies still use Restconf and Netconf?

    Could you please help me understand this?

    Thanks

  2. Hi Sinasi,

    NETCONF/RESTCONF don’t have to exclude Ansible/Netmiko. In fact, Ansible is a great way to work with NETCONF/RESTCONF.

    With network automation, the CLI doesn’t work very well. It was created for humans, not scripts. You can use Ansible with regular CLI commands but you can also use it with NETCONF or RESTCONF.

    Even if you use NETCONF/RESTCONF, you still need tools to talk with a device somehow. You could use a python script or something like postman. That’s great for some quick examples but not the best way to work with it every day. You could use An

    ... Continue reading in our forum

  3. Hi Rene,

    I created a netmiko script to do sh run on IOS-XE router and it works with Visual Studio. Can I use the same script with Postman or do I have to convert it to json first?

    in other words, how can we get python scripts to work with Restconf.

    Thanks

    Hany

  4. Hi Hany,

    Netmiko is a library to interact with network devices through SSH. You can use it to automate things that you would normally do yourself with SSH.

    Postman is (simply said) an API client. If you want to interact with your router through postman, you’ll have to use something that both postman and the router support (like RESTCONF).

    If you want to use Python with RESTCONF, you can use the requests library.

    Cisco DevNet has some examples. Here’s one to get the hostname of a router:

    https://github.com/CiscoDevNet/restconf-examples/blob/master/restconf-102/g

    ... Continue reading in our forum

  5. Hi Rene,
    i have a very basic question,i know from github we can get scripts,and they need to run on a linux based server to change the config on network kit.My point of confusion is, how does the linux box know to reach the network kit,is it all down to routing /static route in a vmware environment or is there a setting i have missed which points the linux box to the router eg.

45 more replies! Ask a question or join the discussion by visiting our Community Forum