As every IT administrator knows, the infrastructure (meaning storages, compute, VMware virtualisation stack) is just a fundaments to run various operating systems (OS) and finally (containerized) application. Therefore, installation of (let’s call it) infrastructure in the datacenter (SDDC), in that sense is just the beginning of the adventure. No wonder, that administrators needs tools, to properly prepare OS, install application etc in mass scale, where such environment is created from the scratch or migrated (for example to the VMware in the cloud).
Nowadays administrators has various tools available on the market like Ansible, Salt, Puppet, Chef, Terraform (major player in infrastructure as a code) and many others.

In this series of articles, we will discuss Salt. I hope, at the and you (dear reader) will understand why Salt has s big role to play, why it has been chosen by the VMware and that it is quite handful tool to use when you know the basic.

Introduction

Salt (or SaltStack) is an opensource software, mainly based on Python language to automate and run remote task execution in a very large scale. It is fantastic software, that can be scaled to a high number of workstation (measured in thousands) and use with high speed. Software was created by Thomas S Hatch in 2011 (wiki) and because of it many values, it has been acquired by VMware in 2020. Now it is of course branded by Broadcom but Salt is used in many other project like SuSE Manger (SUMA) which I hope I will introduce to you in future article.

What makes Salt stand out:

Components

Communications

Communication between Minions and Salt Master depends on the type of Minion. If it is agent (default) communication is always from Minion to the Server. Therefore there is no need to open communication from the Master to any of the server in the environment – this will keep security team extremely happy. On the other hand, for agentless communication, it is Master initiating the connection.
Requred ports:

• agentless: communication is over ssh (using sshfs), default port tcp/22
• agent type (Minion): tcp/4505 and tcp/4506

The main components:

• Salt master
• Minions
• Jobs (on minions, scheduled by the master)
• Salt state (declarative, how minion should be configured)
• Event– signals triggered by actions like completion of the job, system change
• Event bus – messaging system (tranport between master, salt, API)
• Salt API
• Target (specified by the greans)
• Beacons and reactors
• Pillar
• Runners
• ZeroMQ – default messaging library (asyc)
• Tornado – python library for asynchronous event-driven networking

The Salt master is the central service for managing minions. The master distributes commands and configurations and can collect data from the minions. Keep in mind that minions actively retrieve commands from the master, ensuring rapid execution, though not always within precisely the same timeframe.
Minions: agent (special package mostly written in python) installed on managed systems that execute commands from the Master. Sometimes by minions are called also those systems without the agent but managed by master (via ssh – agentless)
Jobs – tasks executed on the client, scheduled, managed by the master. Master should be always informed about the task execution status. Which will create Events. Events can also create system status change (or service/application change).
Event Bus it is communication channel (with ZeroMQ technology) between all the components. Salt also has API (Salt API) where interacting can happen by calling the RESTful interface (for reporting or proceeding any action).
Target is defined by the selection criteria. Not all Minions are always targeted (or rather I should say, that it is very rare scenario). Beacons is a functionality to trigger event on the master when some event happen on the Minion. Beacons usually come together with Reactors, where automate response to events is triggered. Mentioned Pillar will secure data and keep key-value data (stored on Master) for dynamic reference. Runners – modules executed directly on the master for administrative tasks or interacting with the external systems.

Two already mentioned background technology that plays main role in Salt are ZeroMQ and Tornado. ZeroMQ is asynchronous messaging library used by Salt for efficient communication. Tornado is a a web framework used by the Salt Api (basically for handling also asynchronous request).

Installation

Minion installation can be proceed in various way. Sometimes this process is called bootstrapping.

1. In many solution (like SUMA) scripts are provided and can be run on the minion. Also scripts directly from the project can be used (https://github.com/saltstack/salt-bootstrap). Basically the scripts should install necessary software and register the system to the master using the key. Master won’t be able to manage the Minion until the key is accepted.
2. Salt minion is installed manually, server is configured (this implies simple change in one file) and Salt minion is started. In such approach we can simply write script and run it remotely (via ssh) where salt package is installed with all the dependencies, server is defined and the salt minion service is started
3. Salt master is on the installation. This method is preferred when mass-scale installation is needed and is presented below.

For below example, I am using opensuse as a salt master distribution. As a client it is debian, opensuse or windows.

• Make sure that on salt master, salt-ssh package is installed:
  

• Create salt state file in similar way to below:

• create minion_config.jina in similar way as below. This is used in salt state file where the master name is updated (and propagated to the minions)

• below is just verification, that no salt-minion package is installed and no configuration files for minion are presented on the client

• restart salt-master and execute the salt state for the client “osuse-vmin1”

• below you can see, checking the salt logs, that package has been installed

• also on salt master the information is visible

Comment (or better remove) the lines in /etc/salt/roaster (on the Master) as it is not needed anymore and and restart master.
Check the salt-key status (on master). New client should be visible under unaccepted keys. Run command salt-key -A to accept all pending keys.

• Execute some simple commands on the client to verify if the execution is running as excepted.

That is all for today. More scenarios will be provided soon in the next article of that series.