Bacula Security Issues
- Security means being able to restore your files, so read the Critical Items Chapter of this manual.
- The Clients (bacula-fd) must run as root to be able to access all the system files.
- It is not necessary to run the Director as root.
- It is not necessary to run the Storage daemon as root, but you must ensure that it can open the tape drives, which are often restricted to root access by default. In addition, if you do not run the Storage daemon as root, it will not be able to automatically set your tape drive parameters on most OSes since these functions, unfortunately require root access.
- You should restrict access to the Bacula configuration files, so that the passwords are not world-readable. The Bacula daemons are password protected using CRAM-MD5 (i.e. the password is not sent across the network). This will ensure that not everyone can access the daemons. It is a reasonably good protection, but can be cracked by experts.
- If you are using the recommended ports 9101, 9102, and 9103, you will probably want to protect these ports from external access using a firewall and/or using tcp wrappers (etc/hosts.allow).
- By default, all data that is sent across the network is unencrypted. However, Bacula does support TLS and can encrypt transmitted data. Please read the TLS (SSL) Communications Encryption section of this manual.
- You should ensure that the Bacula working directories are readable and writable only by the Bacula daemons.
- If you are using MySQL it is not necessary for it to run with root permission.
- The default Bacula grant-mysql-permissions script grants all permissions to use the MySQL database without a password. If you want security, please tighten this up!
- Don't forget that Bacula is a network program, so anyone anywhere on the network with the console program and the Director's password can access Bacula and the backed up data.
- You can restrict what IP addresses Bacula will bind to by using the appropriate DirAddress, FDAddress, or SDAddress records in the respective daemon configuration files.
- Be aware that if you are backing up your database using the default script, if you have a password on your database, it will be passed as a command line option to that script, and any user will be able to see this information. If you want it to be secure, you will need to pass it by an environment variable or a secure file.
See also Backing Up Your Bacula Database -- Security Considerations for more information.
Backward Compatibility
One of the major goals of Bacula is to ensure that you can restore tapes (I'll use the word tape to include disk Volumes) that you wrote years ago. This means that each new version of Bacula should be able to read old format tapes. The first problem you will have is to ensure that the hardware is still working some years down the road, and the second problem will be to ensure that the media will still be good, then your OS must be able to interface to the device, and finally Bacula must be able to recognize old formats. All the problems except the last are ones that we cannot solve, but by careful planning you can.Since the very beginning of Bacula (January 2000) until today (December 2005), there have been two major Bacula tape formats. The second format was introduced in version 1.27 in November of 2002, and it has not changed since then. In principle, Bacula can still read the original format, but I haven't tried it lately so who knows ...
Though the tape format is fixed, the kinds of data that we can put on the tapes are extensible, and that is how we added new features such as ACLs, Win32 data, encrypted data, ... Obviously, an older version of Bacula would not know how to read these newer data streams, but each newer version of Bacula should know how to read all the older streams.
If you want to be 100% sure that you can read old tapes, you should:
- Try reading old tapes from time to time - e.g. at least once a year.
- Keep statically linked copies of every version of Bacula that you use in production then if for some reason, we botch up old tape compatibility, you can always pull out an old copy of Bacula ...
Configuring and Testing TCP Wrappers
TCP Wrappers are implemented if you turn them on when configuring (./configure -with-tcp-wrappers). With this code enabled, you may control who may access your daemons. This control is done by modifying the file: /etc/hosts.allow. The program name that Bacula uses when applying these access restrictions is the name you specify in the daemon configuration file (see below for examples). You must not use the twist option in your /etc/hosts.allow or it will terminate the Bacula daemon when a connection is refused.
The exact name of the package you need loaded to build with TCP wrappers depends on the system. For example, on SuSE, the TCP wrappers libraries needed to link Bacula are contained in the tcpd-devel package. On Red Hat, the package is named tcp_wrappers.
Dan Langille has provided the following information on configuring and testing TCP wrappers with Bacula.
If you read hosts_options(5), you will see an option called twist. This option replaces the current process by an instance of the specified shell command. Typically, something like this is used:
ALL : ALL \ : severity auth.info \ : twist /bin/echo "You are not welcome to use %d from %h."
The libwrap code tries to avoid twist if it runs in a resident process, but that test will not protect the first hosts_access() call. This will result in the process (e.g. bacula-fd, bacula-sd, bacula-dir) being terminated if the first connection to their port results in the twist option being invoked. The potential, and I stress potential, exists for an attacker to prevent the daemons from running. This situation is eliminated if your /etc/hosts.allow file contains an appropriate rule set. The following example is sufficient:
undef-fd : localhost : allow undef-sd : localhost : allow undef-dir : localhost : allow undef-fd : ALL : deny undef-sd : ALL : deny undef-dir : ALL : deny
You must adjust the names to be the same as the Name directives found in each of the daemon configuration files. They are, in general, not the same as the binary daemon names. It is not possible to use the daemon names because multiple daemons may be running on the same machine but with different configurations.
In these examples, the Director is undef-dir, the Storage Daemon is undef-sd, and the File Daemon is undef-fd. Adjust to suit your situation. The above example rules assume that the SD, FD, and DIR all reside on the same box. If you have a remote FD client, then the following rule set on the remote client will suffice:
undef-fd : director.example.org : allow undef-fd : ALL : deny
where director.example.org is the host which will be contacting the client (ie. the box on which the Bacula Director daemon runs). The use of “ALL : deny” ensures that the twist option (if present) is not invoked. To properly test your configuration, start the daemon(s), then attempt to connect from an IP address which should be able to connect. You should see something like this:
$ telnet undef 9103 Trying 192.168.0.56... Connected to undef.example.org. Escape character is '^]'. Connection closed by foreign host. $
This is the correct response. If you see this:
$ telnet undef 9103 Trying 192.168.0.56... Connected to undef.example.org. Escape character is '^]'. You are not welcome to use undef-sd from xeon.example.org. Connection closed by foreign host. $
then twist has been invoked and your configuration is not correct and you need to add the deny statement. It is important to note that your testing must include restarting the daemons after each connection attempt. You can also use btool(8) and tcpdmatch(8) to validate your /etc/hosts.allow rules. Here is a simple test using tcpdmatch:
$ tcpdmatch undef-dir xeon.example.org warning: undef-dir: no such process name in /etc/inetd.conf client: hostname xeon.example.org client: address 192.168.0.18 server: process undef-dir matched: /etc/hosts.allow line 40 option: allow access: granted
If you are running Bacula as a standalone daemon, the warning above can be safely ignored. Here is an example which indicates that your rules are missing a deny statement and the twist option has been invoked.
$ tcpdmatch undef-dir 10.0.0.1 warning: undef-dir: no such process name in /etc/inetd.conf client: address 10.0.0.1 server: process undef-dir matched: /etc/hosts.allow line 91 option: severity auth.info option: twist /bin/echo "You are not welcome to use undef-dir from 10.0.0.1." access: delegated
Running as non-root
Security advice from Dan Langille:
It is a good idea to run daemons with the lowest possible privileges. In other words, if you can, don't run applications as root which do not have to be root. The Storage Daemon and the Director Daemon do not need to be root. The File Daemon needs to be root in order to access all files on your system. In order to run as non-root, you need to create a user and a group. Choosing bacula as both the user name and the group name sounds like a good idea to me.
The FreeBSD port creates this user and group for you. Here is what those entries looked like on my FreeBSD laptop:
bacula:*:1002:1002::0:0:\bacula{} Daemon:/var/db/bacula:/sbin/nologin
I used vipw to create this entry. I selected a User ID and Group ID of 1002 as they were unused on my system.
I also created a group in /etc/group:
bacula:*:1002:
The bacula user (as opposed to the Bacula daemon) will have a home directory of /var/db/bacula which is the default location for the Bacula database.
Now that you have both a bacula user and a bacula group, you can secure the bacula home directory by issuing this command:
chown -R bacula:bacula /var/db/bacula/
This ensures that only the bacula user can access this directory. It also means that if we run the Director and the Storage daemon as bacula, those daemons also have restricted access. This would not be the case if they were running as root.
It is important to note that the storage daemon actually needs to be in the operator group for normal access to tape drives etc (at least on a FreeBSD system, that's how things are set up by default) Such devices are normally chown root:operator. It is easier and less error prone to make Bacula a member of that group than it is to play around with system permissions.
Starting the Bacula daemons
To start the Bacula daemons on a FreeBSD system, issue the following command:
/usr/local/etc/rc.d/bacula-dir start /usr/local/etc/rc.d/bacula-sd start /usr/local/etc/rc.d/bacula-fd start
To confirm they are all running:
$ ps auwx | grep bacula
root 63418 0.0 0.3 1856 1036 ?? Ss 4:09PM 0:00.00
/usr/local/sbin/bacula-fd -v -c /usr/local/etc/bacula-fd.conf
bacula 63416 0.0 0.3 2040 1172 ?? Ss 4:09PM 0:00.01
/usr/local/sbin/bacula-sd -v -c /usr/local/etc/bacula-sd.conf
bacula 63422 0.0 0.4 2360 1440 ?? Ss 4:09PM 0:00.00
/usr/local/sbin/bacula-dir -v -c /usr/local/etc/bacula-dir.conf
Storage Daemon Encryption
When using the Device directive Volume Encryption = yes, the blocks in the volumes are encrypted using the BLOCK_CIPHER_AES_128_XTS or BLOCK_CIPHER_AES_256_XTS cipher algorithm. These symmetrical ciphers are fast, and used by most applications that need to perform symmetrical encryption of blocks.
Every block is encrypted using a *key* that is unique for the volume and an IV (Initialization Vector) - the block number that is saved in the block header. The XTS ciphers are specifically designed to support an IV with a low entropy.
The first block of the volume that holds the Volume Label is not encrypted because certain fields such as the Volume Name are required to manage the volume and the encryption. A user has an option to obfuscate the fields that are not required which might hold critical information, e.g. hostname. These fields are replaced by the string "OBFUSCATED".
The header of the block is not encrypted. This 24-byte header does not hold user information. See the content of the header:
- 32bit header option bit field
- 32bit block length
- 32bit block number
- BB03 string
- 32bit volume session id
- 32bit volume session time.
The Volume session time is the time of the Storage Daemon at startup. The Volume session id is reset to zero when the daemon starts and is incremented at every backup by the Storage Daemon.
The 64bit XXH64 checksum is encrypted with the data. The block must be be decrypted to verify the checksum. If the checksum matches, Bacula uses the right encryption key and the block is not modified. It is currently not possible to verify the integrity of the block without the encryption key.
If data spooling is enabled, the data in the data spool is not encrypted.
Do not store the data spool on an unsafe storage.
Storage Daemon Directives
There is one new directive in the Storage resource of the Storage Daemon:
Encryption Command = <command> The command specifies an external program that must provide the keys used to encrypt the volume.
There is one directive that must be added to all the devices used to encrypt volumes:
Volume Encryption = <no|yes|strong> It allows you to enable the encryption for a given device. The encryption can be of 3 different types:
- no Default, the device don't do any encryption.
- yes The device encrypts the data, but all information in the volume label are left in clear text.
- strong The device encrypts the data, and obfuscate any information in the volume label except the one that are needed for the management of the volume. The fields that are obfuscate are: volume name.
Encryption Command
The Encryption Command is called by the Storage Daemon every time a new volume is initialized or mounted using a device with encryption enabled. We provide a very simple example script that can be used to manage volume encryption keys.
Example Encryption Command setting in the Storage Daemon configuration:
Encryption Command = "/opt/bacula/scripts/key-manager.py getkey"
The command is limited to 127 characters. The same variable substitutions as the Autochanger Command are available to be passed to the script.
The program can be an interface with your existing key management system or perform the key management on its own.
The sample script called key-manager.py may be installed via the bacula-storage-key-manager package. The install-key-manager.sh script is designed to help to setup a master key:
# sudo -u bacula /opt/bacula/scripts/install-key-manager.sh check # sudo -u bacula /opt/bacula/scripts/install-key-manager.sh install
Example interaction using the sample key-manager.py script::
$ OPERATION=LABEL VOLUME_NAME=Volume0001 ./key-manager.py getkey --cipher AES_128_XTS --key-dir /opt/bacula/keys cipher: AES_128_XTS cipher_key: G6HksAYDnNGr67AAx2Lb/vecTVjZoYAqSLZ7lGMyDVE= volume_name: Volume0001 $ OPERATION=READ VOLUME_NAME=Volume0001 ./key-manager.py getkey --cipher AES_128_XTS --key-dir /opt/bacula/keys cipher: AES_128_XTS cipher_key: G6HksAYDnNGr67AAx2Lb/vecTVjZoYAqSLZ7lGMyDVE= volume_name: Volume0001 $ cat /opt/bacula/keys/Volume0001 cipher: AES_128_XTS cipher_key: G6HksAYDnNGr67AAx2Lb/vecTVjZoYAqSLZ7lGMyDVE= volume_name: Volume0001 $ OPERATION=READ VOLUME_NAME=DontExist ./key-manager.py getkey --cipher AES_128_XTS --key-dir /opt/bacula/keys 2>/dev/null error: no key information for volume "DontExist" $ echo $? 0 $ OPERATION=BAD_CMD VOLUME_NAME=Volume0002 ./key-manager.py getkey --cipher AES_128_XTS --key-dir /opt/bacula/keys 2>/dev/null error: environment variable OPERATION invalid "BAD_CMD" for volume "Volume0002" $ echo $? 0
In the command above, notice that the keys are kept in one directory /opt/bacula/keys, and the arguments are passed using the environment variables.
Bacula passes the following variables via the environment:
- OPERATION
- This is LABEL when the volume is labeled. In this case the script should generate a new key or this can be READ when the volume already has a label and the Storage Daemon needs the already existing key to read or append data to the volume
- VOLUME_NAME
- This is the name of the volume.
Some variables already exist to support a *Master Key* in the future. This feature is not yet supported, but will come later:
- ENC_CIPHER_KEY
- This is a base64 encoded version of the key encrypted by the master key.
- MASTER_KEYID
- This is a base64 encoded version of the key ID of the master key that was used to encrypt the ENC_CIPHER_KEY above.
Bacula expects some values in return:
- volumename
- This is a repetition of the name of the volume that is given to the script. This field is optional and ignored by Bacula.
- cipher
- This is the cipher that Bacula must use. Bacula knows the following ciphers: AES_128_XTS and AES_256_XTS. Of course, the key lengths varies depending on the cipher used.
- cipher_key
- This is a symmetric key in base 64 format.
- comment
- This is a single line of text that is optional and ignored by Bacula.
- error
- This is a single line error message. This is optional, but when provided, Bacula considers that the script returned an error and displays this error in the job log.
Bacula expects an exit code of 0. If the script exits with a different error code, any output is ignored and Bacula displays a generic message with the exit code in the job log.
To return an error to Bacula, the script must use the *error* field and return an error code of 0.
Compatibility between Volumes Using the BB02 and BB03 Formats
Any volume initialized using Bacula version 13.0 or prior versions can be read and appended in version 15 and above. Volumes that have been initialized or simply appended using version 15.0 and after cannot be read with version 13.0 or prior versions.
Interoperability between Encrypted and Unencrypted Volumes
The volume has a flag in the header of the first block which indicated whether the volume is encrypted or not. The field VolEncrypted in the Catalog reflects this state. Bacula does not mix encrypted and unencrypted data inside a volume.
Volumes that have been labeled or re-labeled on a Device with the directive VolumeEncryption are encrypted volumes. Other volumes, including volumes which have been initialized with a BB02 format using version 13.0 or older, are seen as unencrypted volumes.
You can change the value of the directive VolumeEncryption to Yes or Strong at any time. Volumes that are not encrypted can be read on this device but cannot be appended until they are recycled. Switching between Yes and Strong is fine, because the difference is only in the unencrypted label and not in the blocks that come after the label.
Changing the value of the directive back to No will deactivate the encryption and make all encrypted volumes unreadable and useless on this device until they are recycled.
Using a Master Key
Bacula uses symmetric keys to encrypt the volumes. This means that the same key is used to encrypt and also to decrypt the volume content. To improve security, each volume is expected to be encrypted using a different key. The task of the key manager is to provide these keys to the Storage Daemon. When needed, these keys must be generated for a new volume or when the volume is recycled. This can result in a large amount of key files to handle, which must also be backed up or kept in a safe place.
The master key is implemented using a public/private key pair. The public key is kept on the Storage Daemon to generate the encrypted version of the symmetric keys while the private key may be kept somewhere else.
The master key is designed to provide two advantages:
- To be able to decrypt multiple volumes using one, or a set of master keys
- When the **stealth** mode is used, the symmetric key is never stored on disk, which makes any restore impossible without the private key. If your server is hacked or stolen, nobody is able to restore your data.
How the Master Key Is Implemented
The master key is implemented using a public/private key pair. The public key is stored on the Storage Daemon and is used to encrypt the automatically generated symmetrical keys. The private key does not need to be stored on the Storage Daemon to run backups.
The encrypted key and a unique identifier for the master key are stored into the label of the volume. At restore time, these two pieces of information, and the volume name are provided to the key manager that must provide the correct symmetrical key.
There are a few scenarios to consider:
- You don't use the stealth mode, and the key-manager can use the passphrase of the private key that you have provided in the configuration file to decrypt the symmetrical key coming from the volume *or use the clear version of the symmetrical key stored locally.
- You use the stealth mode, and the key-manager asks GnuPG to decrypt the symmetrical key coming from the volume. There are two possibilities here:
- The passphrase of the private key has been preset, or is still in the cache of the *gpg* agent and *gpg* can decrypt the symmetrical key
- Bacula waits until the user provides the passphrase
The key-manager provided with Bacula uses GnuPG to manage the public/private key.
Format of the key-manager.conf File
When using the master key feature, you can set up your master keys in the key-manager.conf file that is stored in /opt/bacula/etc. This file is automatically populated with one master-key at installation time. This file is not required if you don't use a master-key.
[default] gnupghome="/opt/bacula/etc/gnupg" [0378FB9C839FF9F207834D89DB856A1A513B7AB4] #volume_regex=Volume[0-9]+|TestVolume[0-9]+ uid=bacula@localhost passphrase=xm3ynBi7MfHTUovls4QV8OtBT5rfAnXwT8Wb7wjVRyCT stealth=off
The supported fields are:
- gnupghome
- This is where GnuPG stores your keys
- volume_regrex
- A regex that is used by the key manager to know if this master key must be used with one volume
- uid
- The uid helps you to identify the key
- passphrase
- This is the passphrase of the key
- stealth
- When stealth is used, the key manager does not store the symmetrical keys in the KEYDIR directory.
The default gnupghome directory is in /opt/bacula/etc/gnupg. When using GnuPG, you must use the -homedir option like: -homedir /opt/bacula/etc/gnupg, or set the GNUPGHOME environment variable::
export GNUPGHOME="/opt/bacula/etc/gnupg"
You must define a section for every master key you want to use. The name of the section is the Key-ID of your public/private key. You can get the Key-ID when listing your keys::
bacula $ GNUPGHOME=/opt/bacula/etc/gnupg gpg -k /opt/bacula/etc/gnupg/pubring.kbx ----------------------------------------------------- pub rsa3072 2023-01-11 [SC] 0378FB9C839FF9F207834D89DB856A1A513B7AB4 uid [ultimate] Bacula <bacula@localhost> sub rsa3072 2023-01-11 [E]
Notice that the public key has a sub key that allows encryption. This is the [E] in the last line. If your public/private key does not own such a key, you cannot use it with the key manager.
The volume_regex are checked sequentially. The first regex that matches the volume name submitted to the key manager will be used even if another volume_regex matches the name of the volume. By default the volume_regex is commented out to not activate the master key feature.
The uid is only informative, and is not used by the key manager.
When stealth is set to on the key manager does not keep the symmetrical keys in clear text in the KEYDIR directory. When on, you must also not specify a passphrase.
When the passphrase of the public/private key is not set, but required by the key manager, it relies on the gpg-agent to provide the key.
Backup Your Keys
In case of losing your symmetrical keys and/or your master key(s), your data is not recoverable. Therefore, it is important to backup your keys.
Your symmetrical keys are stored in the /opt/bacula/etc/keydir directory by default. This directory may be modified with the -key-dir option in the command line that is configured with the Encryption Command directive defined above. This directory must be backed up regularly.
Your master keys are stored in the /opt/bacula/etc/gnupg directory by default. This directory is defined in the key manager configuration file (by default /opt/bacula/etc/key-manager.conf) in the [default] section under the gnupghome directive as seen above. The key-manager.conf file can be relocated with the -config option in the command defined in Encryption Command directive defined above. The default passphrase is stored in the key-manager.conf. You just need to backup the key-manager.conf* file, and the /opt/bacula/etc/gnupg directory.
You can export your default private key using the command::
# gpg --homedir /opt/bacula/etc/gnupg --output private.pgp --armor --export-secret-key bacula@localhost
It asks you for the passphrase that is saved in you key-manager.conf file. This exports an ASCII armored version of your private key into the file private.pgp. You can print it and/or save it on USB drive or elsewhere.
What Is Encrypted and What Is Not
The main goal of encryption is to prevent outsiders without the volume key to read the volume's data. Bacula does this well, but encryption alone cannot protect against volume modifications.
The first block of the volume is the volume label and it is not encrypted. Some information is required for the management of the volume itself. The only data in the volume label coming from the user are: the hostname, volumename, poolname. The hostname can be obfuscated using the STRONG mode of encryption, the poolname and the volumename could be made useless to an attacker by using a generic name like PoolAlpha or Volume12345.
Data in your catalog database, for example the directories, filenames, and the JobLog are not encrypted.
An attacker can also make some undetected modifications to the volumes. The easiest way is to remove one block inside the volume. Other verifications inside Bacula could detect such a modification and the attacker must be meticulous, but it is a possibility.
The XXH64 checksum inside each volume is encrypted using the encryption key. This is not as strong as using a certified signature, but it provides substantial confidence that the block cannot be modified easily.
To summarize, with volume encryption enabled, you can be confident that:
- An attacker can not read any of your data: Very Strong.
- An attacker can not substitute the volume with another one: Strong.
- An attacker can not modify the contents of the volume: Good.