Automation & Go(lang): Real World Use Case

Over the past month-and-a-half, I've shared some articles with you about the go-junos library, mostly regarding how it works. With most things in this industry, I like to see them in action, and how others, as well as myself, might use them on a day-to-day basis.

This post is about just that. I'll show you an example of how I use this library to help simplify and save lots of time when deploying new SRX installations.

A Little Background

I work for a large retailer, with a lot of remote sites. Each year, we add roughly 10-12 new locations. We have this process down so it's pretty "cookie-cutter," but when it comes to deploying the networking gear, there still is a lot of manual configuration that has to be done initially.

A cluster of SRX240's is deployed at each location, and we have many other clusters throughout our company, ranging from the 240's up to the 3600's. To manage all of these, we rely heavily on Junos Space.

For these specific locations, the policy is pretty much the same for each one. Because of that, we use a lot of polymorphic (variable) objects when building our policy. This essentially lets us assign every SRX cluster to the same policy. It makes it very handy when pushing out changes (and troubleshooting)...since every site adheres to the same policy.

For more on variables, check out the following documentation.

We use Infoblox for all of our IP/DNS/DHCP managment.

When we know the new locations that will be built in the upcoming year, we create all of their network subnets in Infoblox. Each location uses a /21, and a co-worker of mine created an Excel spreadsheet to easily lay out the subnets for the new location.

Before Automation...

Once all the new location information is in Infoblox, I would create the SRX configurations (interfaces, zones, etc.) from the subnet information that was populated. I then would create the networks for each location within Junos Space, so when it came time to publish the policy, all the necessary information was already there.

I had some scripts to do a few tasks, but I still had to touch every system in order to either input the information or extract it. Doing this 10+ times, it took me the better part of a few days (weaving in other projects, workload, etc.).

Where go-junos, Infoblox API's Help

Now we'll get into the nuts and bolts of how I use this library to save time on deployments.

So let's recap the steps I need to take:

  • Get all the subnet information from within Infoblox regarding each location.
  • Build the configuration that I will need to upload/paste into each SRX cluster.
  • Create all the networks for each location within Junos Space to be applied to a policy.

Surely there has to be a better way, right? Right!?

So, with the go-junos library, and how well it works with Junos Space, I was able to write a script that takes a given store location ID (numerical value), and create everything from the bullet points I mentioned above.

Get Subnet Information From Infoblox

First I had to query Infoblox and get the subnet information. I did this by using their API and searching for the location ID in the extended attributes field of "subnet_org." I then take that information, and based on the subnet, I easily identify what interfaces need to be created, and in what zones they belong.

With this information I can easily use it later to build the config.

Here's a sample of the Infoblox query. We have a couple of structs defined which model the format in which we want to place our data in:

type storeNetworks struct {
	Number   string
	Networks []storeNetwork
}

type storeNetwork struct {
	Vlan        string
	InterfaceIP string
	Network     string
	Mask        string
	Zone        string
	Reth        string
}

And here's the function we use to query Infoblox:

func getStoreNetworks(s, user, password string) (*storeNetworks, error) {
	storeNumber := leadingZero(s)
	tr := &http.Transport{
		TLSClientConfig: &tls.Config{
			InsecureSkipVerify: true,
		},
	}
	url := fmt.Sprintf("https://infoblox.company.com/wapi/v1.0/network?*subnet_org~:=%s", storeNumber)
	client := &http.Client{Transport: tr}
	req, _ := http.NewRequest("GET", url, nil)
	req.SetBasicAuth(user, password)
	res, err := client.Do(req)
	defer res.Body.Close()

	if err != nil {
		return nil, err
	}

	data, _ := ioutil.ReadAll(res.Body)
	json.Unmarshal([]byte(data), &networks)

	// Regex to skip subnets we do not care about.
    // ** some values ommited **
	skipRegex := regexp.MustCompile(`^.*(Open|VoIP|WAN|Loopback|Reserved|MPLS).*$`)
	vlanRegex := regexp.MustCompile(`^.*Vlan(\d+).*[\r\n]*$`)
	subnets := []storeNetwork{}

	for _, network := range networks {
		if skipRegex.MatchString(strings.Trim(network.Comment, "\r\n")) {
			continue
		} else {
			var zone string
			vlans := vlanRegex.FindStringSubmatch(network.Comment)
			subnet := strings.Split(network.Network, "/")
			net := subnet[0]
			mask := subnet[1]
			iface := gateway(vlans[1], net)

			switch vlans[1] {
			case "1":
				zone = "management"
			case "914":
				zone = "corp"
			case "915", "916":
				zone = "servers"
			case "920":
				zone = "lan"
			case "980", "981":
				zone = "vendors"
			case "950", "951":
				zone = "wireless"
			}

			subnets = append(subnets, storeNetwork{
				Vlan:        vlans[1],
				InterfaceIP: iface,
				Network:     net,
				Mask:        mask,
				Zone:        zone,
				Reth:        fmt.Sprintf("reth0.%s", vlans[1]),
			})
		}
	}

	storeData := storeNetworks{Number: storeNumber, Networks: subnets}

	return &storeData, nil
}

Build The Base SRX Config

Given the information returned from querying Infoblox, I can build my SRX config. Most of this is just looping over the VLAN's/subnets to create the interfaces, security-zones, forwarding-options, routing-instances, etc.

I won't place the whole config here, as it's pretty redundant. All the commands are in "set" format and I just do variable replacement (interpolation) for most of it. Here is a sample of the interface creation code:

// the buildInterfaces function
func buildInterfaces(buf *bufio.Writer, vlan, iface, mask, reth, network string) {
	if vlan != "914" {
		fmt.Fprintf(buf, fmt.Sprintf("set interfaces %s vlan-id %s disable family inet address %s/%s\n", reth, vlan, iface, mask))
	} else {
		fmt.Fprintf(buf, fmt.Sprintf("set interfaces %s vlan-id %s family inet address %s/%s\n", reth, vlan, iface, mask))
	}
}

// txtBuf is a buffer (bufio.NewWriter()) I write everything to before writing it to the file.
fmt.Fprintf(txtBuf, "set interfaces reth0 vlan-tagging\n")
for _, i := range stores.Networks {
	buildInterfaces(txtBuf, i.Vlan, i.InterfaceIP, i.Mask, i.Reth, i.Network)
}

Create Hosts/Networks in Junos Space

We always have a few specific static hosts that we know will be in each location. Knowing the DNS names, I do a lookup to get the IP, and create the hosts in Junos Space using the AddAddress() function.

NOTE: I also create all the networks using this same function.

Here's a sample of creating the networks in Junos Space, as well as adding certain ones to (Address) groups we use. The first section is where I do an DNS query to get the IP address of the hosts I need to add.

server := fmt.Sprintf("server.location-%s.company.com", *store)
dns := fmt.Sprintf("dns-server.location-%s.company.com", stores.Number, *store)
windows := fmt.Sprintf("windows-server-%s.company.com", stores.Number)
serverIP, _ := net.LookupIP(server)
dnsIP, _ := net.LookupIP(dns)
windowsIP, _ := net.LookupIP(windows)
    
for _, s := range stores.Networks {
	space.AddAddress(fmt.Sprintf("net-location-%s-vlan%s", stores.Number, s.Vlan), fmt.Sprintf("%s/%s", s.Network, s.Mask), "")
}

space.AddAddress(server, fmt.Sprintf("%s", serverIP[0]), "")
space.AddAddress(dns-server, fmt.Sprintf("%s", dnsIP[0]), "")
space.AddAddress(windows-server, fmt.Sprintf("%s", windowsIP[0]), "")
space.ModifyObject(true, "add", "store-servers", fmt.Sprintf("net-st%s-vlan915", stores.Number))
space.ModifyObject(true, "add", "store-vendors", fmt.Sprintf("net-st%s-vlan980", stores.Number))

Modifying Variable (Polymorphic) Objects in Space

Once we have all of our networks and hosts added into Junos Space, we then use the following code to modify our variable definitions that exist within our policies.

firewall := fmt.Sprintf("firewall-%s", stores.Number)
v, err := space.ModifyVariable()
if err != nil {
	log.Fatal(err)
}
v.Add("store-vendors-lan", firewall, "net-location-%s-vlan980")
v.Add("store-vendors-wireless", firewall, "net-location-%s-vlan981")
-- more of the same --

Conclusion

Within a couple of keystrokes and a few seconds, I have all of my template configs generated, and all of my objects and networks published in Junos Space ready to be deployed when the site comes online. It may not seem like much, but we are able to cut our deployment time down by a lot, thus using that time to put towards other projects/resources.

I hope you enjoyed the article. If you would ever like to know more, please feel free to contact me.