gRPC on Cisco IOS XR Packet Capture

This packet capture shows a Python script talking gRPC to a Cisco IOS XR router. It replaces part of the BGP configuration with a ReplaceConfig call, then reads the configuration back with two GetConfig calls. The session runs without TLS, which is exactly why it’s worth studying: you can see every layer of gRPC in clear text. TCP, HTTP/2 streams, gRPC framing, a protobuf message, and inside it the YANG-modeled configuration as JSON.

It also shows why you shouldn’t run gRPC without TLS in production: the username and password are right there in the HTTP/2 headers. More captures are in the Packet Capture Library.

Want to open this capture in Wireshark yourself? The pcap file is free with a NetworkLessons account. Your free account also unlocks all our free lessons and every capture in the Packet Capture Library.

Create a free account to download the pcap

Already have an account? Log in and the download button appears here.

Capture overview

File wireshark-grpc-config-replace-get.pcapng
Packets 35
Protocol gRPC over HTTP/2 (cleartext, h2c), protobuf, Cisco IOS XR gRPC service IOSXRExtensibleManagabilityService.gRPCConfigOper
Transport TCP: client 10.82.100.188 → router 172.16.1.1 port 57400
Devices Cisco IOS XR router, Python client (grpc-python 1.18.0)
Wireshark filter grpc or http2. If Wireshark shows only TCP, use Decode As: TCP port 57400 → HTTP2.

Packets in this capture

No. Time      Source         Destination    Info
1-3 0.000000  ...                           TCP handshake to port 57400
6   0.018793  10.82.100.188  172.16.1.1     Magic, SETTINGS[0], WINDOW_UPDATE[0], PING[0]
8   0.019106  172.16.1.1     10.82.100.188  SETTINGS[0]
10  0.019656  10.82.100.188  172.16.1.1     HEADERS[1]: POST .../ReplaceConfig
11  0.019673  10.82.100.188  172.16.1.1     DATA[1] (GRPC) (PROTOBUF)
15  0.106614  172.16.1.1     10.82.100.188  HEADERS[1]: 200 OK, DATA[1] (GRPC)
16  0.106696  172.16.1.1     10.82.100.188  HEADERS[1] (trailers: grpc-status 0)
18  4.170141  10.82.100.188  172.16.1.1     HEADERS[3]: POST .../GetConfig, DATA[3]
21  4.247249  172.16.1.1     10.82.100.188  HEADERS[3]: 200 OK
22  4.247290  172.16.1.1     10.82.100.188  DATA[3] (GRPC) (PROTOBUF)
26  4.270221  10.82.100.188  172.16.1.1     HEADERS[5]: POST .../GetConfig, DATA[5]
28  4.316297  172.16.1.1     10.82.100.188  HEADERS[5]: 200 OK
33  7.176358  10.82.100.188  172.16.1.1     TCP FIN, session closed

The number in brackets is the HTTP/2 stream ID. Every RPC gets its own stream (1, 3, 5) on the same TCP connection. Client-initiated streams always use odd numbers, and stream 0 is for connection-level frames like SETTINGS and PING.

Packet walkthrough

HTTP/2 connection preface (packets 6-9)

Right after the TCP handshake, the client sends the HTTP/2 connection preface: the fixed “Magic” string PRI * HTTP/2.0, followed by a SETTINGS frame. The router answers with its own SETTINGS. There’s no TLS handshake and no HTTP/1.1 upgrade: the client speaks HTTP/2 directly in clear text, which is called h2c. On IOS XR, this is what you get with grpc no-tls.

ReplaceConfig request (packets 10-11)

HyperText Transfer Protocol 2
    Stream: HEADERS, Stream ID: 1, Length 345, POST /IOSXRExtensibleManagabilityService.gRPCConfigOper/ReplaceConfig
        Type: HEADERS (1)
        Flags: 0x04, End Headers
        Header: :scheme: http
        Header: :method: POST
        Header: :authority: 172.16.1.1:57400
        Header: :path: /IOSXRExtensibleManagabilityService.gRPCConfigOper/ReplaceConfig
        Header: username: cisco
        Header: password: cisco
        Header: te: trailers
        Header: content-type: application/grpc
        Header: user-agent: grpc-python/1.18.0 grpc-c/7.0.0 (manylinux; chttp2; goose)
        Header: grpc-accept-encoding: identity,deflate,gzip
        Header: accept-encoding: identity,gzip
        Header: grpc-timeout: 9990m
  • Every gRPC call is an HTTP/2 POST. The :path is /package.Service/Method: here the IOS XR gRPCConfigOper service and its ReplaceConfig method. This is Cisco’s own IOS XR gRPC service, not gNMI. A gNMI client would call /gnmi.gNMI/Set and /gnmi.gNMI/Get instead.
  • username and password as metadata: gRPC metadata travels as HTTP/2 headers. IOS XR authenticates every call with these two headers. Without TLS, anyone who can capture the traffic sees the credentials. With TLS, the same headers are encrypted.
  • content-type: application/grpc: tells the server the body uses gRPC framing.
  • te: trailers: required by gRPC. The result of the call comes in trailing headers after the response body (see below).
  • grpc-timeout: 9990m: the client’s deadline for this call, 9990 milliseconds (“m” means milliseconds). The server can give up once it expires.

The DATA frame (packet 11) carries the request body. Each gRPC message starts with a 5-byte prefix: a compressed flag (0 = not compressed) and a 4-byte message length. Then comes the protobuf message. Wireshark can’t decode its fields without the IOS XR .proto file, so it shows PROTOBUF (<UNKNOWN>). But the bytes give it away: the payload is a protobuf string field that contains the new configuration as YANG JSON:

{
 "Cisco-IOS-XR-ipv4-bgp-cfg:bgp": {
  "instance": [
   {
    "instance-name": "default",
    "instance-as": [
     {
      "as": 0,
      "four-byte-as": [
       {
        "as": 65400,
        "bgp-running": ...
        ...
             "remote-as": {
              "as-xx": 0,
              "as-yy": 65450
             },
             "neighbor-afs": {
              "neighbor-af": [
               {
                "af-name": "ipv4-unicast",
                "activate": [ null ],
                "next-hop-self": true
               }
              ]
             }

That’s the Cisco-IOS-XR-ipv4-bgp-cfg YANG model: BGP AS 65400 with a neighbor in AS 65450 and next-hop-self for IPv4 unicast. gRPC is the transport, YANG is the data model, and JSON is the encoding.

Response and trailers (packets 15-16)

HEADERS[1]
    Header: :status: 200 OK
    Header: content-type: application/grpc
DATA[1] (GRPC)
    .... ...0 = Compressed Flag: Not Compressed (0)
    Message Length: 0
HEADERS[1]
    Header: grpc-status: 0
    Header: grpc-message:
  • :status 200 only means the HTTP/2 request was delivered. It doesn’t mean the RPC succeeded.
  • grpc-status: 0 in the trailers (the second HEADERS frame on stream 1) is the real result: 0 is OK. A failure would show another code, such as 16 (UNAUTHENTICATED) for wrong credentials, with an explanation in grpc-message.
  • An empty response (Message Length 0): ReplaceConfig returns no errors, so there’s nothing to say.

GetConfig (packets 18-30)

The client then calls GetConfig twice, on streams 3 and 5. These are server-streaming RPCs: the router answers with HEADERS, one or more DATA frames with the requested configuration (again YANG JSON inside protobuf), and trailers with the grpc-status. Both calls reuse the same TCP connection. That’s one of the advantages of HTTP/2: many RPCs share one connection, each in its own stream.