Data Loader Plugins

Data plugins responsible for loading data to render from various formats such as YAML structured text or xlsx spreadsheets.

By default TTR uses file extension to choose plugin for loading data, alternatively TTR object data_plugin attribute can be used to specify default plugin to use.

For instance, to load data.csv file TTR will use csv plugin, to load data.yaml file, yaml plugin will be used, etc.

Data plugins load data in a list of dictionaries, where each item rendered using template as per template_name_key attribute.

XLSX Spreadsheets loader plugin

Plugin reference name: xlsx

This plugin supports loading data from multiple sheets, combining them for rendering.

Prerequisites:

Restrictions and guidelines

Spreadsheets must contain a column or multiple columns with headers starting with template_name_key argument string, default is template. Values of template(s) columns either names of the template to use for rendering or OS path string to template file relative to template_dir argument supplied to TTR object on instantiation.

In addition, table must contain column with result_name_key values, default is device, they used to combine results, i.e. rendering results for identical result_name_key combined in a single string. result_name_key used further by returners to return results.

Spreadsheet tabs with names starting with # are skipped, useful to comment out tabs that no need to render.

First row in the spreadsheet must contain headers, otherwise spreadsheet not loaded.

Note

empty cells loaded with value of None

Sample spreadsheet table that contains details for interfaces configuration:

device

interface

vid

vrf

ip

mask

template

rt1

Gi1/1.100

100

MGMT

10.0.0.1

24

ttr://simple/interface.cisco_ios.txt

rt1

Gi2/3

CUST

10.3.0.1

30

ttr://simple/interface.cisco_ios.txt

sw23

Vlan21

MGMT

10.0.0.2

24

ttr://simple/interface.cisco_ios.txt

where:

  • device column contains result_name_key values

  • template column contains template_name_key values

  • ttr://simple/interface.cisco_ios.txt - is a template included in TTR templates collection

Above table loaded into this list of dictionaries:

[{'device': 'rt1',
  'interface': 'Gi1/1.100',
  'ip': '10.0.0.1',
  'mask': 24,
  'template': 'ttr://simple/interface.cisco_ios.txt',
  'vid': 100,
  'vrf': 'MGMT'},
 {'device': 'rt1',
  'interface': 'Gi2/3',
  'ip': '10.3.0.1',
  'mask': 30,
  'template': 'ttr://simple/interface.cisco_ios.txt',
  'vid': None,
  'vrf': 'CUST'},
 {'device': 'sw23',
  'interface': 'Vlan21',
  'ip': '10.0.0.2',
  'mask': 24,
  'template': 'ttr://simple/interface.cisco_ios.txt',
  'vid': None,
  'vrf': 'MGMT'}]

Combined with ttr://simple/interface.cisco_ios.txt it will produce these results:

ttr -d /path_to_table.xlsx/ -p

# ---------------------------------------------------------------------------
# rt1 rendering results
# ---------------------------------------------------------------------------
interface Gi1/1.100
 encapsulation dot1q 100
 vrf forwarding  MGMT
 ip address 10.0.0.1 24
 exit
!
interface Gi2/3
 encapsulation dot1q None
 vrf forwarding  CUST
 ip address 10.3.0.1 30
 exit
!

# ---------------------------------------------------------------------------
# sw23 rendering results
# ---------------------------------------------------------------------------
interface Vlan21
 encapsulation dot1q None
 vrf forwarding  MGMT
 ip address 10.0.0.2 24
 exit
!

Multiple Templates suffix separation

Using multitemplate processor it is possible to define multiple template columns within same spreadsheet tab using suffixes. Columns with headers with same suffixes considered part of same datum and combined together. Headers without suffixes shared across all datums.

For example, this table uses :a and :b suffixes to denote relationship with certain templates:

device:a

interface:a

ip:a

mask

device:b

interface:b

ip:b

template:a

template:b

rt1

Gi1/1

10.0.0.1

30

rt2

Gi1

10.0.0.2

ttr://simple/interface.cisco_ios.txt

ttr://simple/interface.cisco_nxos.txt

rt3

Gi2/3

10.3.0.1

30

rt4

Gi3

10.3.0.2

ttr://simple/interface.cisco_ios.txt

ttr://simple/interface.cisco_nxos.txt

where:

  • device columns contains result_name_key values

  • template columns contains template_name_key values

  • ttr://simple/interface.cisco_ios.txt - is a template included in TTR templates collection

  • ttr://simple/interface.cisco_nxos.txt - is a template included in TTR templates collection

Above table data, after passing through multitemplate processor loaded into this list of dictionaries:

import pprint
from ttr import ttr

gen = ttr("./path/to/table.xlsx", processors=["multitemplate"])

pprint.pprint(gen.data_loaded)

# prints:
# [{'device': 'rt1',
#   'interface': 'Gi1/1',
#   'ip': '10.0.0.1',
#   'mask': 30,
#   'template': 'ttr://simple/interface.cisco_ios.txt'},
#  {'device': 'rt2',
#   'interface': 'Gi1',
#   'ip': '10.0.0.2',
#   'mask': 30,
#   'template': 'ttr://simple/interface.cisco_nxos.txt'},
#  {'device': 'rt3',
#   'interface': 'Gi2/3',
#   'ip': '10.3.0.1',
#   'mask': 30,
#   'template': 'ttr://simple/interface.cisco_ios.txt'},
#  {'device': 'rt4',
#   'interface': 'Gi3',
#   'ip': '10.3.0.2',
#   'mask': 30,
#   'template': 'ttr://simple/interface.cisco_nxos.txt'}]

That technique allows to simplify definition of “paired” configurations, e.g. device A and device B configurations or forward and rollback configurations etc.

ttr.plugins.data.xlsx_loader.load(data, templates_dict, template_name_key, **kwargs)

Function to load XLSX spreadsheet. Takes OS path to .xlsx file and returns list of dictionaries, where keys equal to headers of spreadsheets’ tabs.

Parameters

  • data – string, OS path to .xlsx file

  • templates_dict – dictionary to load templates from spreadsheet

  • template_name_key – string, templates column header prefix

CSV Spreadsheets loader plugin

Plugin Name: csv

Support loading data from CSV text file.

Spreadsheet must contain a column or multiple columns with headers starting with template_name_key argument string. Values of template(s) columns either names of the template to use for rendering or OS path string to template file relative to template_dir argument supplied to TTR object on instantiation.

In addition, table must contain column with result_name_key values, they used to combine results, i.e. rendering results for identical result_name_key combined in a single string. result_name_key used further by returners to return results.

ttr.plugins.data.csv_loader.load(data, templates_dict=None, template_name_key=None, **kwargs)

Function to load CSV spreadsheet.

Parameters

  • data – OS path to CSV text file

  • templates_dict – (dict) dictionary to load templates from spreadsheet, not supported by csv loader

  • template_name_key – (str) templates column header prefix, not supported by csv loader

  • kwargs – (dict) any additional arguments to pass on to csv.DictReader object instantiation

YAML loader

Plugin Name: yaml

Prerequisites:

  • Requires PyYAML library

Plugin to load data to render from YAML structured text.

ttr.plugins.data.yaml_loader.load(data, templates_dict=None, template_name_key=None, **kwargs)

Function to load YAML data from text file or from string. Text file should have .yml or .yaml extension to properly detect loader.

Parameters

  • data – string, OS path to text file or YAML structured text

  • templates_dict – (dict) dictionary to load templates from spreadsheet, not supported by yaml loader

  • template_name_key – (str) templates column header prefix, not supported by yaml loader

  • kwargs – (dict) any additional arguments are ignored