Tag: Ansible

We’ve collected our state data in Part 2.

Now let’s do something interesting with our data. I’ve switched out the role I will be using. So this will be a long blog post on how to set up and finally make something pretty with your data!

 

The Setup

What you’ll need –

• Python
• Ansible
• PyATS and Genie Modules
• Ansible PyATS  – You can git clone this into your roles directory
• Pandas
• Jupyter Notebook

The reason I’m using a different module for this, is that it will automatically grab a snapshot and format it into JSON. It has other features, like compare that I won’t get into. But for now, this will be good enough to start manipulating data.

The Play Book

---
- hosts: localhost
connection: local
gather_facts: no

- name: Interface Snapshot
hosts: switches
gather_facts: no
connection: network_cli
roles:
    - ansible-pyats

tasks:
  - name: Gather Snapshot
include_role:
name: ansible-pyats
tasks_from: snapshot_command

vars:
command: show interfaces
file: "snapshots/{{ inventory_hostname }}_interface_snapshot.json"

This play book is assigned to the switches in the inventory. It calls the role that we downloaded from GitHub with the task snapshot_command. I’m telling it to snapshot the show interfaces command. 

Step 1 – Grab the state of the interfaces using your playbook and output to JSON.

Step 2 – Import the data into Jupyter Notebook and convert it into useable information using Pandas.

Now you may be asking why Pandas and why Jupyter? I have a blog post here on why it’s easier to work with. On the note about Pandas, it’s a great tool for quickly putting information into a structure that easier to analyze. Need to do math based on Date time? Need to do some quick counting on cell values? Or maybe convert row data to columns? Very quick an easy to do in Pandas. So let’s get started.

Now this may seem a little denser than normal. But this is a direct export from Jupyter. This is using Python and the two modules to make the magic happen. If you want to try it out, you can copy the code below into Jupyter and use this JSON document. 

 

Now that you have backups running successfully. Let’s talk about abstracting data states!

The Why?

It’s pretty important to differentiate between state and config data. Configuration will only get you half the picture and in many cases less than half. How are relationships built on your network? Do you know who the primary neighbors for your devices are? That’s state information, many people keep it in their visio. But the critical piece here is you need that information to build programmability. That’s where state abstraction comes into play.

We will be using Ansible to gather this information and with the help of the PyATS library do some quick state abstraction and turn it into something useful.

State Abstraction –

What you’ll need –

• Python
• Ansible
• PyATS – Ansible Galaxy Module

The Play –

---
- name: pyATS testing
  hosts: cisco
  gather_facts: no
  connection: network_cli
  roles:
    - parse_genie

  tasks:
  - name: Run the show ip interface command
    ios_command:
      commands:
        - show ip interface brief
    register: interface_output

  - name: Set fact with Genie filter plugin
    set_fact:
      pyats_showipinterface: "{{ interface_output['stdout'][0] | parse_genie(command='show ip interface brief', os='ios') }}"

  - name: Debug interface parse
    debug:
      var: pyats_showipinterface

  - name: Convert parse data to YAML
    template:
      src: templates/show_ip_interface_brief.j2
      dest: "reports/{{ inventory_hostname }}"

This play book logs into each device and runs the “show ip interface brief” command. It then parses the output into a JSON structure using PyATS Genie. In the final step it calls a Jinja template and converts to a structured YAML format.

You’ll need to install Parse Genie. The directions are available on Ansible Galaxy. I did rename my folder for easier reference. Normally the role is “(user).(module)”.

Here’s the Jinja template –

---
interfaces:
{% for key1 in pyats_showipinterface %}
{% for nested_key in pyats_showipinterface[key1] %}
- name: {{ nested_key }}
- IP address: {{ pyats_showipinterface[key1][nested_key]["ip_address"] }}
- Status: {{ pyats_showipinterface[key1][nested_key]["status"] }}
{% endfor %}
{% endfor %}

Jinja is a Python templating engine. If you need anything done at speed, you’ll need to hand your data to Jinja. Could I simplify this and just call the nested key? Yes, but this also shows you how to parse through multiple nested keys. Very handy if say you have a Route Table -> VRFs -> Routes. You may need to parse multiple key value structures.

You can take data that’s human readable-

R1#sh ip int bri
Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet1       unassigned      YES NVRAM  up                    up      
GigabitEthernet1.13    155.1.13.1      YES NVRAM  up                    up      
GigabitEthernet1.100   169.254.100.1   YES NVRAM  up                    up      
GigabitEthernet1.146   155.1.146.1     YES NVRAM  up                    up      
GigabitEthernet2       unassigned      YES NVRAM  administratively down down    
GigabitEthernet3       192.168.3.61    YES NVRAM  up                    up      
Loopback0              150.1.1.1       YES NVRAM  up                    up      
Tunnel0                155.1.0.1       YES NVRAM  up                    up

And present it in a format that a machine can consume and is human readable –

---
interfaces:
- name: GigabitEthernet1
  - IP address: unassigned
  - Status: up
- name: GigabitEthernet1.13
  - IP address: 155.1.13.1
  - Status: up
- name: GigabitEthernet1.100
  - IP address: 169.254.100.1
  - Status: up
- name: GigabitEthernet1.146
  - IP address: 155.1.146.1
  - Status: up
- name: GigabitEthernet2
  - IP address: unassigned
  - Status: administratively down
- name: GigabitEthernet3
  - IP address: 192.168.3.61
  - Status: up
- name: Loopback0
  - IP address: 150.1.1.1
  - Status: up
- name: Tunnel0
  - IP address: 155.1.0.1
  - Status: up

Conclusion –

Ansible with the power of modules and Jinja allow you to take your current deployment and put it into a consumable format. This is just a fraction of what’s possible. I only picked interfaces since it’s a common and easy item to pull state info from. But the possibilities are endless. Take a look at the Genie parsers for yourself and consider what you can accomplish!

https://pubhub.devnetcloud.com/media/genie-feature-browser/docs/#/parsers

In this series I plan on taking a look at how we can begin improving the reliability and re-useability of infrastructure as code. 

Backups – 

The old way of doing backups is pretty useless. It’s often a flat text file in a flat directory. With little version control other than time stamps. And we often backup a config whether it’s changed or not. This can cause a bit of head ache when trying to track down a specific change or even just auditing over time.

Backups 2.0 – 

What you’ll need – 

• Python Virtual Environment
• Ansible(Tower is optional)
• Git
• Your favorite text editor(I use Sublime)
• Git Repository(I use Gogs)
• Test Environment(I have ESXi with CSR1000v’s Running)

I’ll show to perform backups without Tower. They require two different approaches since Tower works in a sandbox and after a playbook is run the working directory is deleted.

Local Backup – 

Step 1 – Clone Backups Repository

We will start off by cloning an empty repository. This will show how effective an initial backup commit and diff works in Git.

Before we run the playbook we check for the directory backups. There is none. This playbook connects to the git repository and clones the resources.

---
#This module runs to build git repository locally
- name: Clone Git Backups Repository
  hosts: 127.0.0.1
  connection: local
  gather_facts: no
  tasks:
    - name: “Clone Git Backups"
      git:
        repo: git@[your git repository here].git
        dest: ./backups
        update: yes
        version: master
      register: git
      ignore_errors: True
    - debug:
        var: git

Step 2 – Perform Device Backup

The next playbook to run is the device configuration backups. The repository initially is blank. All devices are in the state “changed” due to not having been run before. The playbook takes the backup and renames it according the device name and drops it in the backups folder. When navigating to backups you can see the conf files for the 10 routers and that the directory is a git repository.

---
#Device Backup
- name: Backup
  hosts: cisco
  connection: network_cli
  gather_facts: no  tasks:
    - name: Backup
      ios_config:
        backup: yes
        backup_options:
          dir_path: "./backups"
          filename: "{{inventory_hostname}}.config"
      register: config_output
    - debug:
        var: config_output

Step 3 – Git Add Commit Push

This playbook is more complicated as it’s stepping through all the git procedures to push to a repository. It adds all existing files in the directory. Then commits with ansible date time as the comment. Keep in mind to perform the date time function you will need get_facts enabled for the localhost.

---
#Push the config to the git add and push to repository
- name: Add devices to Git and Push
  hosts: 127.0.0.1
  connection: local
  gather_facts: yes
  tasks:
    - name: Add to Git
      shell: "git add ."
      args:
        chdir: ./backups
      register: gitadd
    - debug:
        var: gitadd
    - name: Commit
      shell: git commit -m "{{ ansible_date_time.iso8601 }}"
      args:
        chdir: ./backups
      register: gitcommit
    - debug:
        var: gitcommit.stdout
    - name: Push
      shell: git push
      args:
        chdir: ./backups
      register: gitpush
    - debug:
        var: gitpush

Step 4 – Ansible Idempotency at Play

At the start of the play I push an interface change to Router 1. I then run the config backup and git steps together. As the playbook runs through, you will notice that only R1 has changed. Buried within the wall of text there is also a summary of the amount of lines changed. This can be quite handy for report generation.

Looking at the Git repository, we can see that only R1 has been updated. Browsing the history of R1 Git will report the changes that have taken place. 

As a final note. I did use local SSH keys to push to git. If you plan on using Tower. It’s going to be a different process that I’ll detail in a future post. But requires special credentials and roles to successfully run this playbook.