Allied Telesis (AT) is a small and historical network equipment manufacturer, with headquarters in Japan and the United States. Its global turnover may be less than 1/100 of the turnover of the biggest global network equipment manufacturers.
But AT switches, for instance AT X230 line, are some of the cheapest switches compatible with SDN and OpenFlow. In fact, every managed AT switch is OpenFlow capable, as mentions on AT website. Buying a license is required to activate these functionalities. These capabilities are very interesting in various situations. You may imagine to test or deploy innovative functionalities, like filtering traffic or implementing new fail-over mechanism at the LAN level on this hardware.
AT switches programming interfaces
We met these switches for the first time in an infrastructure to test SDN mechanisms. As we always try to stick to the devops way of managing network, we looked for the programming interfaces we are familiar with, Ansible or Rest API, to manage these switches. Unfortunately, no Ansible module is published for these switches; and, as far as we know, they don't offer any REST API. Other AT equipments offer a REST API.
The only alternative to a manual CLI configuration - the CLI is very similar to other major vendors CLI - is to implement a program to ssh into the switch and send automatic commands to it this way. In this post, we'd like to share our first thoughts about this method.
Go programming language as an alternative to Python to write network management program
Python is a de-facto devops programming language standard. We decided to work with Go, for this project, because we know that some great devops teams have switch to Go for good reasons:
- Compiling is very fast and painless
- Binary code is smaller and faster
- Go offers a stronger safeguard to the developer, type-check, memory-check, variables management, ...
Compare to Python, Go has certainly many weakness as far as devops is concerned, but we think that compiled languages could soon gain strong momentum because compiling isn't a great fuss anymore and they still have an edge in performance.
To conclude this first experiment on AT switches management with go programming language, we provide you with a first (very incomplete and raw) tutorial to manage AT switches automatically.
Installing Go using an Ansible role
Go can be installed system-wide or in user space. In this experiment, we install it in user space.
A guide is published on golang official site: see here.
We always configure Linux stations using Ansible. We define a golang role as follows:
# roles/golang/tasks
# golang
- name: check if golang has already been downloaded
stat:
path: "{{working_directory}}/{{golang_archive_file}}"
register: golang
- name: download golang installation archive
get_url:
url: "https://dl.google.com/go/{{golang_archive_file}}"
dest: "{{working_directory}}"
sha256sum: "{{golang_sha256}}"
when: golang.stat.exists == False
- name: test if golang is already installed
stat:
path: "/home/{{homeuser}}/go"
register: golang_installed
- name: install go in home directory
unarchive:
src: "{{working_directory}}/{{golang_archive_file}}"
dest: "/home/{{homeuser}}"
owner: "{{homeuser}}"
group: "{{homeuser}}"
mode: '755'
when: golang_installed.stat.exists == False
- name: add golang to bashrc for {{homeuser}}
lineinfile:
path: "/home/{{homeuser}}/.bashrc"
regexp: 'go/bin'
line: "export PATH=\"/home/{{homeuser}}/go/bin:$PATH\""
state: present
insertafter: EOF
- name: set GOPATH environment variable in bashrc
lineinfile:
path: "/home/{{homeuser}}/.bashrc"
regexp: 'GOPATH'
line: "export GOPATH=\"/home/{{homeuser}}/go/bin\""
state: present
insertafter: EOFThe task depends on the following variables :
| Variable | Role |
| working_directory | A directory where you store package installed by ansible |
| homeuser | The name of the user (and of its home directory) where golang is installed |
| golang_archive_file | Name of the archive downloaded from google download site |
| golang_sha256 | sha256 of the archive file golang_archive_file |
We set the value of golang variables in a golang ansible role var default file:
# roles/golang/defaults/main.yml
golang_archive_file: go1.13.5.linux-amd64.tar.gz
golang_sha256: 512103d7ad296467814a6e3f635631bd35574cab3369a97a323c9a585ccaa569If you use ansible playbooks, you just have to create the ad-hoc repositories, copy-paste the file above and add the role to your ansible playbook.
You'll find the corresponding files in our GitHub repository, but you'll probably find or develop a more concise version of this quick and dirty script with little effort.
Adding go support to Visual Studio Code
We work with Microsoft Visual Studio Code. Visual Studio Code automatically detects .go files and adapts its configuration to ease your development experience as long as you set the GOPATH environment variable, as we did in the ansible task. In our environment, GOPATH is defined in .bashrc under the home of the user.
Connecting to an AT switch using SSH with Go
Go includes a package to manage SSH. The documentation is available here. There are many tutorials about how to use the library in a correct manner, but not so many that really work. We tried a lot. This one is the simpler and best one we found.
Managing and loading authentication keys on the client
The switches are configured to accept SSH key authentication. If you think that using login/password is simpler, you're wrong. Authenticating programmatically and securely with login/password is a complex task; you have:
- to store the login/password in a secured file
- to setup an environment variable to pass it to the program
At the end of the day, public key authentication is easier to manage than login/password. You can begin with very simple self-signed certificates and, as your skills improve, build a small crypto infrastructure management methodology, with scripts to generate root certificate and client certificates.
You need to know a few things to work with Go ssh library:
- If you use passphrases to protect private keys on the SSH clients, they are ciphered when stored on disk. You have to either produce the password at runtime or pre-load the keys (unciphered) in memory on boot or in a post-boot script.
- On linux distribution, keys loading is managed by ssh-agent
- If you work with Ubuntu distribution (and possibly with other distributions too), ssh-agent is replaced by keyrings. You don't need to install ssh-agent. Keyrings is, in general, already installed and publishes the same API as ss-agent.
- Latests versions of Go validate the certificate of the server. See here and here for further information. To test ssh connection, you may disable the certificate checking. On a production environment, you have to deploy the certificate of the server.
You have to check that the private key you authenticate against your switch is loaded in keyring before you launch the program.
// SSHAgent returns ssh keys registered in the user memory key ring
func SSHAgent() ssh.AuthMethod {
if sshAgent, err := net.Dial("unix", os.Getenv("SSH_AUTH_SOCK")); err == nil {
return ssh.PublicKeysCallback(agent.NewClient(sshAgent).Signers)
}
return nil
}Generating a self-signed server key on the switch
atswitch# crypto key generate rsa 4096Enabling public key authentication on the switch
You enable public key authentication on the switch:
atswitch> enable
atswitch# configure terminal
atswitch# ssh server v2only
atswitch# ssh server authentication publickey
atswitch# no ssh server authentication password
atswitch# ssh server allow-users <USERNAME>
atswitch# service ssh
atswitch# ssh server scpCopying the client public key from a TFTP server to the switch
You need to generate a public-private key pair for your client. You then store the private key file on a secured folder (check the permission). And you copy the public key to a TFTP server directory (docker tftp, EZtftp...). Afterward, you tftp-copy the public key to the switch and attach it to the user :
atswitch# copy tftp://<PATH TO THE KEY> flash:/lankey.pub
atswitch# conf t
atswitch# crypto key pubkey-chain userkey <USERNAME> lankey.pub
atswitch# exitDebugging ssh connections
In our example, when we test the connection, the client generates the following error message : Too many authentication failures for USERNAME.
To investigate this cas, we activate debug mode on the switch:
atswitch# en
atswitch# conf t
atswitch# debug ssh server brief
atswitch# exit
atswitch# terminal monitorWe then find the culprit in the log:
16:26:01 awplus sshd[11841]: Authentication refused: bad ownership or modes for directory /flash/.home/<USERNAME>/.sshThe way we uploaded the public key triggers a strange ownerhip problem on the switch underlying linux system. We opt for dropping and registering the key again as follows:
atswitch# en
atswitch# cd flash:
atswitch# cd .home/
atswitch# rmdir force <USERNAME>
atswitch# cd ..
atswitch# conf t
atswitch# crypto key pubkey-chain userkey <USERNAME> lankey.pub
atswitch# exitNew ssh connection trial : it works :-).
Connecting to the switch with the Go program
Connecting to a switch is similar to connecting to another ssh server.
The code for various steps can be found in the aforementioned tutorial:
- Set up and configuring the client
- Connect to the host
- Create a session
- Redirect stdin, stdout and stderr
- Start a remote shell
To test our approach, we modify the code to send commands to the switch :
commands := []string{
"enable",
"configure terminal",
"interface port1.0.1-1.0.10",
"description lan port",
"exit",
"no ip route 0.0.0.0/0 192.168.123.1",
"ip route 0.0.0.0/0 192.168.123.254",
"exit",
"write mem",
"exit",
}
for _, cmd := range commands2 {
_, err = fmt.Fprintf(stdin, "%s\n", cmd)
if err != nil {
log.Fatal(err)
}
}Final thoughts
It seems easy to manage network equipment with a few lines of Go code. But there are still a lot of work to do to get from this first test to a production-ready program:
- Managing the SSH connection
When configuring devices, especially network equipment, you sometimes have to reset network interfaces. The connection goes down and up. You have to deal with this point and reconnect to get the state of the equipment after the configuration is applied.
More broadly, TCP connection timeout are slow to trigger. The client detects that the connection is down in seconds, sometimes in minutes. You have to tune and monitor the connection parameters to improve network fault detection. - Committing configuration change
The example doesn't check that the changes are done and persistent :- Sending a command
- Checking that that no error araises
- And then, committing a group of commands by sending a write mem
- Idempotence
If you run the example code twice, you don't know what it will do. In a production environment, you'll want to enforce some kind of idempotence.
