Next: Testing Your Tape Drive Up: Bacula User's Guide Previous: Bacula Frequently Asked Questions Contents Index
There are a number of example scripts for various things that can be found in the example subdirectory and its subdirectories of the Bacula source distribution.
For additional tips, please see the Bacula wiki.
The first thing to do before upgrading from one version to another is to ensure that you don't overwrite or delete your production (current) version of Bacula until you have tested that the new version works.
If you have installed Bacula into a single directory, this is simple: simply make a copy of your Bacula directory.
If you have done a more typical Unix installation where the binaries are placed in one directory and the configuration files are placed in another, then the simplest way is to configure your new Bacula to go into a single file. Alternatively, make copies of all your binaries and especially your conf files.
Whatever your situation may be (one of the two just described), you should probably start with the defaultconf script that can be found in the examples subdirectory. Copy this script to the main Bacula directory, modify it as necessary (there should not need to be many modifications), configure Bacula, build it, install it, then stop your production Bacula, copy all the *.conf files from your production Bacula directory to the test Bacula directory, start the test version, and run a few test backups. If all seems good, then you can proceed to install the new Bacula in place of or possibly over the old Bacula.
When installing a new Bacula you need not worry about losing the changes you made to your configuration files as the installation process will not overwrite them providing that you do not do a make uninstall.
If the new version of Bacula requires an upgrade to the database, you can upgrade it with the script update_bacula_tables, which will be installed in your scripts directory (default /etc/bacula), or alternatively, you can find it in the <bacula-source>/src/cats directory.
One of the first things you should do is to ensure that you are being properly notified of the status of each Job run by Bacula, or at a minimum of each Job that terminates with an error.
Until you are completely comfortable with Bacula, we recommend that you send an email to yourself for each Job that is run. This is most easily accomplished by adding an email notification address in the Messages resource of your Director's configuration file. An email is automatically configured in the default configuration files, but you must ensure that the default root address is replaced by your email address.
For additional examples of how to configure a Bacula, please take a look at the .conf files found in the examples sub-directory. We recommend the following configuration (where you change the paths and email address to correspond to your setup). Note, the mailcommand and operatorcommand should be on a single line. They were split here for presentation:
Messages {
Name = Standard
mailcommand = "/home/bacula/bin/bsmtp -h localhost
-f \"\(Bacula\) %r\"
-s \"Bacula: %t %e of %c %l\" %r"
operatorcommand = "/home/bacula/bin/bsmtp -h localhost
-f \"\(Bacula\) %r\"
-s \"Bacula: Intervention needed for %j\" %r"
Mail = your-email-address = all, !skipped, !terminate
append = "/home/bacula/bin/log" = all, !skipped, !terminate
operator = your-email-address = mount
console = all, !skipped, !saved
}
You will need to ensure that the /home/bacula/bin path on the mailcommand and the operatorcommand lines point to your Bacula binary directory where the bsmtp program will be installed. You will also want to ensure that the your-email-address is replaced by your email address, and finally, you will also need to ensure that the /home/bacula/bin/log points to the file where you want to log all messages.
With the above Messages resource, you will be notified by email of every Job that ran, all the output will be appended to the log file you specify, all output will be directed to the console program, and all mount messages will be emailed to you. Note, some messages will be sent to multiple destinations.
The form of the mailcommand is a bit complicated, but it allows you to distinguish whether the Job terminated in error or terminated normally. Please see the Mail Command section of the Messages Resource chapter of this manual for the details of the substitution characters used above.
Once you are totally comfortable with Bacula as I am, or if you have a large number of nightly Jobs as I do (eight), you will probably want to change the Mail command to Mail On Error which will generate an email message only if the Job terminates in error. If the Job terminates normally, no email message will be sent, but the output will still be appended to the log file as well as sent to the Console program.
The section above describes how to get email notification of job status. Occasionally, however, users have problems receiving any email at all. In that case, the things to check are the following:
director = director-name = all
mailcommand = "mail -s test your@domain.com"
mailcommand = "/home/bacula/bin/bsmtp -f \"root@localhost\" %r"
If like me, you have setup Bacula so that email is sent only when a Job has errors, as described in the previous section of this chapter, inevitably, one day, something will go wrong and Bacula can stall. This could be because Bacula crashes, which is vary rare, or more likely the network has caused Bacula to hang for some unknown reason.
To avoid this, you can use the RunAfterJob command in the Job resource to schedule a Job nightly, or weekly that simply emails you a message saying that Bacula is still running. For example, I have setup the following Job in my Director's configuration file:
Schedule {
Name = "Watchdog"
Run = Level=Full sun-sat at 6:05
}
Job {
Name = "Watchdog"
Type = Admin
Client=Watchdog
FileSet="Verify Set"
Messages = Standard
Storage = DLTDrive
Pool = Default
Schedule = "Watchdog"
RunAfterJob = "/home/kern/bacula/bin/watchdog %c %d"
}
Client {
Name = Watchdog
Address = rufus
FDPort = 9102
Catalog = Verify
Password = ""
File Retention = 1day
Job Retention = 1 month
AutoPrune = yes
}
Where I established a schedule to run the Job nightly. The Job itself is type Admin which means that it doesn't actually do anything, and I've defined a FileSet, Pool, Storage, and Client, all of which are not really used (and probably don't need to be specified). The key aspect of this Job is the command:
RunAfterJob = "/home/kern/bacula/bin/watchdog %c %d"
which runs my "watchdog" script. As an example, I have added the Job codes %c and %d which will cause the Client name and the Director's name to be passed to the script. For example, if the Client's name is Watchdog and the Director's name is main-dir then referencing $1 in the script would get Watchdog and referencing $2 would get main-dir. In this case, having the script know the Client and Director's name is not really useful, but in other situations it may be.
You can put anything in the watchdog script. In my case, I like to monitor the size of my catalog to be sure that Bacula is really pruning it. The following is my watchdog script:
#!/bin/sh cd /home/kern/mysql/var/bacula du . * | /home/kern/bacula/bin/bsmtp \ -f "\(Bacula\) abuse@whitehouse.com" -h mail.yyyy.com \ -s "Bacula running" abuse@whitehouse.com
If you just wish to send yourself a message, you can do it with:
#!/bin/sh cd /home/kern/mysql/var/bacula /home/kern/bacula/bin/bsmtp \ -f "\(Bacula\) abuse@whitehouse.com" -h mail.yyyy.com \ -s "Bacula running" abuse@whitehouse.com <<END-OF-DATA Bacula is still running!!! END-OF-DATA
By using a WriteBootstrap record in each of your Director's Job resources, you can constantly maintain a bootstrap file that will enable you to recover the state of your system as of the last backup without having the Bacula catalog. This permits you to more easily recover from a disaster that destroys your Bacula catalog.
When a Job resource has a WriteBootstrap record, Bacula will maintain the designated file (normally on another system but mounted by NSF) with up to date information necessary to restore your system. For example, in my Director's configuration file, I have the following record:
Write Bootstrap = "/mnt/deuter/files/backup/client-name.bsr"
where I replace client-name by the actual name of the client that is being backed up. Thus, Bacula automatically maintains one file for each of my clients. The necessary bootstrap information is appended to this file during each Incremental backup, and the file is totally rewritten during each Full backup.
Note, one disadvantage of writing to an NFS mounted volume as I do is that if the other machine goes down, the OS will wait forever on the fopen() call that Bacula makes. As a consequence, Bacula will completely stall until the machine exporting the NFS mounts comes back up. A possible solution to this problem was provided by Andrew Hilborne, and consists of using the soft option instead of the hard option when mounting the NFS volume, which is typically done in /etc/fstab/. The NFS documentation explains these options in detail. However, I found that with the soft option NFS disconnected frequently causing even more problems.
If you are starting off in the middle of a cycle (i.e. with Incremental backups) rather than at the beginning (with a Full backup), the bootstrap file will not be immediately valid as it must always have the information from a Full backup as the first record. If you wish to synchronize your bootstrap file immediately, you can do so by running a restore command for the client and selecting a full restore, but when the restore command asks for confirmation to run the restore Job, you simply reply no, then copy the bootstrap file that was written to the location specified on the Write Bootstrap record. The restore bootstrap file can be found in restore.bsr in the working directory that you defined. In the example given below for the client rufus, my input is shown in bold. Note, the JobId output has been partially truncated to fit on the page here:
(in the Console program)
*restore
First you select one or more JobIds that contain files
to be restored. You will then be presented several methods
of specifying the JobIds. Then you will be allowed to
select which files from those JobIds are to be restored.
To select the JobIds, you have the following choices:
1: List last 20 Jobs run
2: List Jobs where a given File is saved
3: Enter list of JobIds to select
4: Enter SQL list command
5: Select the most recent backup for a client
6: Cancel
Select item: (1-6): 5
The defined Client resources are:
1: Minimatou
2: Rufus
3: Timmy
Select Client (File daemon) resource (1-3): 2
The defined FileSet resources are:
1: Other Files
Item 1 selected automatically.
+-------+------+-------+---------+---------+------+-------+------------+
| JobId | Levl | Files | StrtTim | VolName | File | SesId | VolSesTime |
+-------+------+-------+---------+---------+------+-------+------------+
| 2 | F | 84 | ... | test1 | 0 | 1 | 1035645259 |
+-------+------+-------+---------+---------+------+-------+------------+
You have selected the following JobId: 2
Building directory tree for JobId 2 ...
The defined Storage resources are:
1: File
Item 1 selected automatically.
You are now entering file selection mode where you add and
remove files to be restored. All files are initially added.
Enter "done" to leave this mode.
cwd is: /
$ done
84 files selected to restore.
Run Restore job
JobName: kernsrestore
Bootstrap: /home/kern/bacula/working/restore.bsr
Where: /tmp/bacula-restores
FileSet: Other Files
Client: Rufus
Storage: File
JobId: *None*
OK to run? (yes/mod/no): no
quit
(in a shell window)
cp ../working/restore.bsr /mnt/deuter/files/backup/rufus.bsr
Bacula keeps a count of the number of files on each Volume in its Catalog database so that before appending to a tape, it can verify that the number of files are correct, and thus prevent overwriting valid data. If the Director or the Storage daemon crashes before the job has completed, the tape will contain one more file than is noted in the Catalog, and the next time you attempt to use the same Volume, Bacula will reject it due to a mismatch between the physical tape (Volume) and the catalog.
The easiest solution to this problem is to label a new tape and start fresh. If you wish to continue appending to the current tape, you can do so by using the update command in the console program to change the Volume Files entry in the catalog. A typical sequence of events would go like the following:
- Bacula crashes - You restart Bacula
Bacula then prints:
17-Jan-2003 16:45 rufus-dir: Start Backup JobId 13,
Job=kernsave.2003-01-17_16.45.46
17-Jan-2003 16:45 rufus-sd: Volume test01 previously written,
moving to end of data.
17-Jan-2003 16:46 rufus-sd: kernsave.2003-01-17_16.45.46 Error:
I cannot write on this volume because:
The number of files mismatch! Volume=11 Catalog=10
17-Jan-2003 16:46 rufus-sd: Job kernsave.2003-01-17_16.45.46 waiting.
Cannot find any appendable volumes.
Please use the "label" command to create a new Volume for:
Storage: SDT-10000
Media type: DDS-4
Pool: Default
(note, lines wrapped for presentation) The key here is the line that reads:
The number of files mismatch! Volume=11 Catalog=10
It says that Bacula found eleven files on the volume, but that the catalog says there should be ten. When you see this, you can be reasonably sure that the SD was interrupted while writing before it had a chance to update the catalog. As a consequence, you can just modify the catalog count to eleven, and even if the catalog contains references to files saved in file 11, everything will be OK and nothing will be lost. Note that if the SD had written several file marks to the volume, the difference between the Volume count and the Catalog count could be larger than one, but this is unusual.
If on the other hand the catalog is marked as having more files than Bacula found on the tape, you need to consider the possible negative consequences of modifying the catalog. Please see below for a more complete discussion of this.
Continuing with the example of Volume = 11 Catalog = 10, to enable to Bacula to append to the tape, you do the following:
update
Update choice:
1: Volume parameters
2: Pool from resource
3: Slots from autochanger
Choose catalog item to update (1-3): 1
Defined Pools:
1: Default
2: File
Select the Pool (1-2):
+-------+---------+--------+---------+-----------+------+----------+------+-----+
| MedId | VolName | MedTyp | VolStat | VolBytes | Last | VolReten | Recy | Slt |
+-------+---------+--------+---------+-----------+------+----------+------+-----+
| 1 | test01 | DDS-4 | Error | 352427156 | ... | 31536000 | 1 | 0 |
+-------+---------+--------+---------+-----------+------+----------+------+-----+
Enter MediaId or Volume name: 1
(note table output truncated for presentation) First, you chose to update the Volume parameters by entering a 1. In the volume listing that follows, notice how the VolStatus is Error. We will correct that after changing the Volume Files. Continuing, you respond 1,
Updating Volume "test01"
Parameters to modify:
1: Volume Status
2: Volume Retention Period
3: Volume Use Duration
4: Maximum Volume Jobs
5: Maximum Volume Files
6: Maximum Volume Bytes
7: Recycle Flag
8: Slot
9: Volume Files
10: Pool
11: Done
Select parameter to modify (1-11): 9
Warning changing Volume Files can result
in loss of data on your Volume
Current Volume Files is: 10
Enter new number of Files for Volume: 11
New Volume Files is: 11
Updating Volume "test01"
Parameters to modify:
1: Volume Status
2: Volume Retention Period
3: Volume Use Duration
4: Maximum Volume Jobs
5: Maximum Volume Files
6: Maximum Volume Bytes
7: Recycle Flag
8: Slot
9: Volume Files
10: Pool
11: Done
Select parameter to modify (1-10): 1
Here, you have selected 9 in order to update the Volume Files, then you changed it from 10 to 11, and you now answer 1 to change the Volume Status.
Current Volume status is: Error
Possible Values are:
1: Append
2: Archive
3: Disabled
4: Full
5: Used
6: Read-Only
Choose new Volume Status (1-6): 1
New Volume status is: Append
Updating Volume "test01"
Parameters to modify:
1: Volume Status
2: Volume Retention Period
3: Volume Use Duration
4: Maximum Volume Jobs
5: Maximum Volume Files
6: Maximum Volume Bytes
7: Recycle Flag
8: Slot
9: Volume Files
10: Pool
11: Done
Select parameter to modify (1-11): 11
Selection done.
At this point, you have changed the Volume Files from 10 to 11 to account for the last file that was written but not updated in the database, and you changed the Volume Status back to Append.
This was a lot of words to describe something quite simple.
The Volume Files option exists only in version 1.29 and later, and you should be careful using it. Generally, if you set the value to that which Bacula said is on the tape, you will be OK, especially if the value is one more than what is in the catalog.
Now lets consider the case:
The number of files mismatch! Volume=10 Catalog=12
Here the Bacula found fewer files on the volume than what is marked in the catalog. Now, in this case, you should hesitate a lot before modifying the count in the catalog, because if you force the catalog from 12 to 10, Bacula will start writing after the file 10 on the tape, possibly overwriting valid data, and if you ever try to restore any of the files that the catalog has marked as saved on Files 11 and 12, all chaos will break out. In this case, you will probably be better off using a new tape. In fact, you might want to see what files the catalog claims are actually stored on that Volume, and back them up to another tape and recycle this tape.
Only the File daemon needs to run with root permission (so that it can access all files). As a consequence, you may run your Director, Storage daemon, and MySQL or PostgreSQL database server as non-root processes. Version 1.30 has the -u and the -g options that allow you to specify a userid and groupid on the command line to be used after Bacula starts.
As of version 1.33, thanks to Dan Langille, it is easier to configure the Bacula Director and Storage daemon to run as non-root.
You should protect the Bacula port addresses (normally 9101, 9102, and 9103) from outside access by a firewall or other means of protection to prevent unauthorized use of your daemons.
You should ensure that the configuration files are not world readable since they contain passwords that allow access to the daemons. Anyone who can access the Director using a console program can restore any file from a backup Volume.
You should protect your Catalog database. If you are using SQLite, make sure
that the working directory is readable only by root (or your Bacula userid),
and ensure that bacula.db has permissions -rw-r--r-- (i.e. 640) or
more strict. If you are using MySQL or PostgreSQL, please note that the Bacula
setup procedure leaves the database open to anyone. At a minimum, you should
assign the user bacula a userid and add it to your Director's
configuration file in the appropriate Catalog resource.
If you normally change tapes every day or at least every Friday, but Thursday is a holiday, you can use a trick proposed by Lutz Kittler to ensure that no job runs on Thursday so that you can insert Friday's tape and be sure it will be used on Friday. To do so, define a RunJobBefore script that normally returns zero, so that the Bacula job will normally continue. You can then modify the script to return non-zero on any day when you do not want Bacula to run the job.
If you have an autochanger but it does not support barcodes, using a "trick" you can make Bacula automatically label all the volumes in your autochanger's magazine.
First create a file containing one line for each slot in your autochanger that has a tape to be labeled. The line will contain the slot number a colon (:) then the Volume name you want to use. For example, create a file named volume-list, which contains:
1:Volume001 2:TestVolume02 5:LastVolume
The records do not need to be in any order and you don't need to mention all the slots. Normally, you will have a consistent set of Volume names and a sequential set of numbers for each slot you want labeled. In the example above, I've left out slots 3 and 4 just as an example. Now, modify your mtx-changer script and comment out all the lines in the list) case by putting a # in column 1. Then add the following two lines:
cat <absolute-path>/volume-list exit 0
so that the whole case looks like:
list) # # commented out lines cat <absolute-path>/volume-list exit 0 ;;
where you replace <absolute-path> with the full path to the volume-list file. Then using the console, you enter the following command:
label barcodes
and Bacula will proceed to mount the autochanger Volumes in the list and label them with the Volume names you have supplied. Bacula will think that the list was provided by the autochanger barcodes, but in reality, it was you who supplied the <barcodes>.
If it seems to work, when it finishes, enter:
list volumes
and you should see all the volumes nicely created.
You may want to backup laptops or portables that are not always connected to the network. If you are using DHCP to assign an IP address to those machines when they connect, you will need to use the Dynamic Update capability of DNS to assign a name to those machines that can be used in the Address field of the Client resource in the Director's conf file.
At some point, you may want to be absent for a week or two and you want to make sure Bacula has enough tape left so that the backups will complete. You start by doing a list volumes in the Console program:
list volumes Using default Catalog name=BackupDB DB=bacula Pool: Default +---------+---------------+-----------+-----------+----------------+- | MediaId | VolumeName | MediaType | VolStatus | VolBytes | +---------+---------------+-----------+-----------+----------------+- | 23 | DLT-30Nov02 | DLT8000 | Full | 54,739,278,128 | | 24 | DLT-21Dec02 | DLT8000 | Full | 56,331,524,629 | | 25 | DLT-11Jan03 | DLT8000 | Full | 67,863,514,895 | | 26 | DLT-02Feb03 | DLT8000 | Full | 63,439,314,216 | | 27 | DLT-03Mar03 | DLT8000 | Full | 66,022,754,598 | | 28 | DLT-04Apr03 | DLT8000 | Full | 60,792,559,924 | | 29 | DLT-28Apr03 | DLT8000 | Full | 62,072,494,063 | | 30 | DLT-17May03 | DLT8000 | Full | 65,901,767,839 | | 31 | DLT-07Jun03 | DLT8000 | Used | 56,558,490,015 | | 32 | DLT-28Jun03 | DLT8000 | Full | 64,274,871,265 | | 33 | DLT-19Jul03 | DLT8000 | Full | 64,648,749,480 | | 34 | DLT-08Aug03 | DLT8000 | Full | 64,293,941,255 | | 35 | DLT-24Aug03 | DLT8000 | Append | 9,999,216,782 | +---------+---------------+-----------+-----------+----------------+
Note, I have truncated the output for presentation purposes. What is significant, is that I can see that my current tape has almost 10 Gbytes of data, and that the average amount of data I get on my tapes is about 60 Gbytes. So if I go on vacation now, I don't need to worry about tape capacity (at least not for short absences).
Equally significant is the fact that I did go on vacation the 28th of June 2003, and when I did the list volumes command, my current tape at that time, DLT-07Jun03 MediaId 31, had 56.5 Gbytes written. I could see that the tape would fill shortly. Consequently, I manually marked it as Used and replaced it with a fresh tape that I labeled as DLT-28Jun03, thus assuring myself that the backups would all complete without my intervention.
This tip was submitted by Marc Brueckner who wasn't sure of the case of some of his files on Win32, which is case insensitive. The problem is that Bacula thinks that /UNIMPORTANT FILES is different from /Unimportant Files. Marc was aware that the file exclusion permits wild-cards. So, he specified:
"/[Uu][Nn][Ii][Mm][Pp][Oo][Rr][Tt][Aa][Nn][Tt] [Ff][Ii][Ll][Ee][Ss]"
As a consequence, the above exclude works for files of any case.
Please note that this works only in Bacula Exclude statement and not in Include.
This tip also comes from Marc Brueckner. (Note, this tip is probably outdated by the addition of ClientRunBeforJob and ClientRunAfterJob Job records, but the technique still could be useful.) First I thought the "Run Before Job" statement in the Job-resource is for executing a script on the remote machine (the machine to be backed up). (Note, this is possible as mentioned above by using ClientRunBeforJob and ClientRunAfterJob). It could be useful to execute scripts on the remote machine e.g. for stopping databases or other services while doing the backup. (Of course I have to start the services again when the backup has finished) I found the following solution: Bacula could execute scripts on the remote machine by using ssh. The authentication is done automatically using a private key. First you have to generate a keypair. I've done this by:
ssh-keygen -b 4096 -t dsa -f Bacula_key
This statement may take a little time to run. It creates a public/private key pair with no passphrase. You could save the keys in /etc/bacula. Now you have two new files : Bacula_key which contains the private key and Bacula_key.pub which contains the public key.
Now you have to append the Bacula_key.pub file to the file authorized_keys in the \root\.ssh directory of the remote machine. Then you have to add (or uncomment) the line
AuthorizedKeysFile %h/.ssh/authorized_keys
to the sshd_config file on the remote machine. Where the %h stands for the home-directory of the user (root in this case).
Assuming that your sshd is already running on the remote machine, you can now enter the following on the machine where Bacula runs:
ssh -i Bacula_key -l root <machine-name-or-ip-address> "ls -la"
This should execute the "ls -la" command on the remote machine.
Now you could add lines like the following to your Director's conf file:
...
Run Before Job = ssh -i /etc/bacula/Bacula_key 192.168.1.1 \
"/etc/init.d/database stop"
Run After Job = ssh -i /etc/bacula/Bacula_key 192.168.1.1 \
"/etc/init.d/database start"
...
Even though Bacula version 1.32 and later has a ClientRunBeforeJob, the ssh method still could be useful for updating all the Bacula clients on several remote machines in a single script.
This tip comes from Phil Stracchino.
If you decide to blow away your catalog and start over, the simplest way to re-add all your prelabeled tapes with a minimum of fuss (provided you don't care about the data on the tapes) is to add the tape labels using the console add command, then go into the catalog and manually set the VolStatus of every tape to Recycle.
The SQL command to do this is very simple, either use your vendor's command line interface (mysql, postgres, sqlite, ...) or use the sql command in the Bacula console:
update Media set VolStatus='Recycle';
Bacula will then ignore the data already stored on the tapes and just re-use each tape without further objection.
This tip comes from Volker Sauer.
Note, this tip was given prior to implementation of ACLs in Bacula (version 1.34.5). It is left here because dumping/displaying ACLs can still be useful in testing/verifying that Bacula is backing up and restoring your ACLs properly. Please see the aclsupport FileSet option in the configuration chapter of this manual.
For example, you could dump the ACLs to a file with a script similar to the following:
#!/bin/sh
BACKUP_DIRS="/foo /bar"
STORE_ACL=/root/acl-backup
umask 077
for i in $BACKUP_DIRS; do
cd $i /usr/bin/getfacl -R --skip-base .>$STORE_ACL/${i//\//_}
done
Then use Bacula to backup /root/acl-backup.
The ACLs could be restored using Bacula to the /root/acl-backup file, then restored to your system using:
setfacl --restore/root/acl-backup
This tip was provided by Alexander Kuehn.
Bacula is a really nice backup program except that the manual tape changing requires user interaction with the bacula console.
Fortunately I can fix this. NOTE!!! This suggestion applies for people who do *NOT* have tape autochangers and must change tapes manually.!!!!!
Bacula supports a variety of tape changers through the use of mtx-changer scripts/programs. This highly flexible approach allowed me to create this shell script which does the following: Whenever a new tape is required it sends a mail to the operator to insert the new tape. Then it waits until a tape has been inserted, sends a mail again to say thank you and let's bacula continue its backup. So you can schedule and run backups without ever having to log on or see the console. To make the whole thing work you need to create a Device resource which looks something like this ("Archive Device", "Maximum Changer Wait", "Media Type" and "Label media" may have different values):
Device {
Name=DDS3
Archive Device = # use yours not mine! ;)/dev/nsa0
Changer Device = # not really required/dev/nsa0
Changer Command = "# use this (maybe change the path)!
/usr/local/bin/mtx-changer %o %a %S"
Maximum Changer Wait = 3d # 3 days in seconds
AutomaticMount = yes; # mount on start
AlwaysOpen = yes; # keep device locked
Media Type = DDS3 # it's just a name
RemovableMedia = yes; #
Offline On Unmount = Yes; # keep this too
Label media = Yes; #
}
As the script has to emulate the complete wisdom of a mtx-changer it has an internal "database" containing where which tape is stored, you can see this on the following line:
labels="VOL-0001 VOL-0002 VOL-0003 VOL-0004 VOL-0005 VOL-0006 VOL-0007 VOL-0008 VOL-0009 VOL-0010 VOL-0011 VOL-0012"
The above should be all on one line, and it effectively tells Bacula that volume "VOL-0001" is located in slot 1 (which is our lowest slot), that volume "VOL-0002" is located in slot 2 and so on.. The script also maintains a logfile (/var/log/mtx.log) where you can monitor its operation.
Bacula can run multiple concurrent jobs, but the default configuration files do not enable it. Using the Maximum Concurrent Jobs directive, you can configure how many and which jobs can be run simultaneously. The Director's default value for Maximum Concurrent Jobs is "1".
To initially setup concurrent jobs you need to define Maximum Concurrent Jobs in the Director's configuration file (bacula-dir.conf) in the Director, Job, Client, and Storage resources.
Additionally the File daemon, and the Storage daemon each have their own Maximum Concurrent Jobs directive that sets the overall maximum number of concurrent jobs the daemon will run. The default for both the File daemon and the Storage daemon is "20".
For example, if you want two different jobs to run simultaneously backing up the same Client to the same Storage device, they will run concurrently only if you have set Maximum Concurrent Jobs greater than one in the Director resource, the Client resource, and the Storage resource in bacula-dir.conf.
We recommend that you read the Data Spooling of this manual first, then test your multiple concurrent backup including restore testing before you put it into production.
Below is a super stripped down bacula-dir.conf file showing you the four places where the the file must be modified to allow the same job NightlySave to run up to four times concurrently. The change to the Job resource is not necessary if you want different Jobs to run at the same time, which is the normal case.
#
# Bacula Director Configuration file -- bacula-dir.conf
#
Director {
Name = rufus-dir
Maximum Concurrent Jobs = 4
...
}
Job {
Name = "NightlySave"
Maximum Concurrent Jobs = 4
Client = rufus-fd
Storage = File
...
}
Client {
Name = rufus-fd
Maximum Concurrent Jobs = 4
...
}
Storage {
Name = File
Maximum Concurrent Jobs = 4
...
}
This document describes the utility programs written to aid Bacula users and developers in dealing with Volumes external to Bacula.
Starting with version 1.27, each of the following programs requires a valid Storage daemon configuration file (actually, the only part of the configuration file that these programs need is the Device resource definitions). This permits the programs to find the configuration parameters for your archive device (generally a tape drive). By default, they read bacula-sd.conf in the current directory, but you may specify a different configuration file using the -c option.
Each of these programs require a device-name where the Volume can be found. In the case of a tape, this is the physical device name such as /dev/nst0 or /dev/rmt/0ubn depending on your system. For the program to work, it must find the identical name in the Device resource of the configuration file. See below for specifying Volume names.
Please note that if you have Bacula running and you ant to use one of these programs, you will either need to stop the Storage daemon, or unmount any tape drive you want to use, otherwise the drive will busy because Bacula is using it.
If you are attempting to read or write an archive file rather than a tape, the device-name should be the full path to the archive location including the filename. The filename (last part of the specification) will be stripped and used as the Volume name, and the path (first part before the filename) must have the same entry in the configuration file. So, the path is equivalent to the archive device name, and the filename is equivalent to the volume name.
In general, you must specify the Volume name to each of the programs below (with the exception of btape). The best method to do so is to specify a bootstrap file on the command line with the -b option. As part of the bootstrap file, you will then specify the Volume name or Volume names if more than one volume is needed. For example, suppose you want to read tapes tape1 and tape2. First construct a bootstrap file named say, list.bsr which contains:
Volume=test1|test2
where each Volume is separated by a vertical bar. Then simply use:
./bls -b list.bsr /dev/nst0
In the case of Bacula Volumes that are on files, you may simply append volumes as follows:
./bls /tmp/test1\|test2
where the backslash (\) was necessary as a shell escape to permit entering the vertical bar (|).
And finally, if you feel that specifying a Volume name is a bit complicated with a bootstrap file, you can use the -V option (on all programs except bcopy) to specify one or more Volume names separated by the vertical bar (|). For example,
./bls -V Vol001 /dev/nst0
You may also specify an asterisk (*) to indicate that the program should accept any volume. For example:
./bls -V* /dev/nst0
bls can be used to do an ls type listing of a Bacula tape or file. It is called:
Usage: bls [options] <device-name>
-b <file> specify a bootstrap file
-c <file> specify a config file
-d <level> specify debug level
-e <file> exclude list
-i <file> include list
-j list jobs
-k list blocks
(no j or k option) list saved files
-L dump label
-p proceed inspite of errors
-v be verbose
-V specify Volume names (separated by |)
-? print this message
For example, to list the contents of a tape:
./bls -V Volume-name /dev/nst0
Or to list the contents of a file:
./bls /tmp/Volume-name or ./bls -V Volume-name /tmp
Note that, in the case of a file, the Volume name becomes the filename, so in the above example, you will replace the xxx with the name of the volume (file) you wrote.
Normally if no options are specified, bls will produce the equivalent output to the ls -l command for each file on the tape. Using other options listed above, it is possible to display only the Job records, only the tape blocks, etc. For example:
./bls /tmp/File002 bls: butil.c:148 Using device: /tmp drwxrwxr-x 3 k k 4096 02-10-19 21:08 /home/kern/bacula/k/src/dird/ drwxrwxr-x 2 k k 4096 02-10-10 18:59 /home/kern/bacula/k/src/dird/CVS/ -rw-rw-r-- 1 k k 54 02-07-06 18:02 /home/kern/bacula/k/src/dird/CVS/Root -rw-rw-r-- 1 k k 16 02-07-06 18:02 /home/kern/bacula/k/src/dird/CVS/Repository -rw-rw-r-- 1 k k 1783 02-10-10 18:59 /home/kern/bacula/k/src/dird/CVS/Entries -rw-rw-r-- 1 k k 97506 02-10-18 21:07 /home/kern/bacula/k/src/dird/Makefile -rw-r--r-- 1 k k 3513 02-10-18 21:02 /home/kern/bacula/k/src/dird/Makefile.in -rw-rw-r-- 1 k k 4669 02-07-06 18:02 /home/kern/bacula/k/src/dird/README-config -rw-r--r-- 1 k k 4391 02-09-14 16:51 /home/kern/bacula/k/src/dird/authenticate.c -rw-r--r-- 1 k k 3609 02-07-07 16:41 /home/kern/bacula/k/src/dird/autoprune.c -rw-rw-r-- 1 k k 4418 02-10-18 21:03 /home/kern/bacula/k/src/dird/bacula-dir.conf ... -rw-rw-r-- 1 k k 83 02-08-31 19:19 /home/kern/bacula/k/src/dird/.cvsignore bls: Got EOF on device /tmp 84 files found.
If you are listing a Volume to determine what Jobs to restore, normally the -j option provides you with most of what you will need as long as you don't have multiple clients. For example,
./bls -j -V Test1 -c stored.conf DDS-4 bls: butil.c:258 Using device: "DDS-4" for reading. 11-Jul 11:54 bls: Ready to read from volume "Test1" on device "DDS-4" (/dev/nst0). Volume Record: File:blk=0:1 SessId=4 SessTime=1121074625 JobId=0 DataLen=165 Begin Job Session Record: File:blk=0:2 SessId=4 SessTime=1121074625 JobId=1 Level=F Type=B Begin Job Session Record: File:blk=0:3 SessId=5 SessTime=1121074625 JobId=5 Level=F Type=B Begin Job Session Record: File:blk=0:6 SessId=3 SessTime=1121074625 JobId=2 Level=F Type=B Begin Job Session Record: File:blk=0:13 SessId=2 SessTime=1121074625 JobId=4 Level=F Type=B End Job Session Record: File:blk=0:99 SessId=3 SessTime=1121074625 JobId=2 Level=F Type=B Files=168 Bytes=1,732,978 Errors=0 Status=T End Job Session Record: File:blk=0:101 SessId=2 SessTime=1121074625 JobId=4 Level=F Type=B Files=168 Bytes=1,732,978 Errors=0 Status=T End Job Session Record: File:blk=0:108 SessId=5 SessTime=1121074625 JobId=5 Level=F Type=B Files=168 Bytes=1,732,978 Errors=0 Status=T End Job Session Record: File:blk=0:109 SessId=4 SessTime=1121074625 JobId=1 Level=F Type=B Files=168 Bytes=1,732,978 Errors=0 Status=T 11-Jul 11:54 bls: End of Volume at file 1 on device "DDS-4" (/dev/nst0), Volume "Test1" 11-Jul 11:54 bls: End of all volumes.
shows a full save followed by two incremental saves.
Adding the -v option will display virtually all information that is available for each record:
Normally, except for debugging purposes, you will not need to list Bacula blocks (the "primitive" unit of Bacula data on the Volume). However, you can do so with:
./bls -k /tmp/File002 bls: butil.c:148 Using device: /tmp Block: 1 size=64512 Block: 2 size=64512 ... Block: 65 size=64512 Block: 66 size=19195 bls: Got EOF on device /tmp End of File on device
By adding the -v option, you can get more information, which can be useful in knowing what sessions were written to the volume:
./bls -k -v /tmp/File002 Volume Label: Id : Bacula 0.9 mortal VerNo : 10 VolName : File002 PrevVolName : VolFile : 0 LabelType : VOL_LABEL LabelSize : 147 PoolName : Default MediaType : File PoolType : Backup HostName : Date label written: 2002-10-19 at 21:16 Block: 1 blen=64512 First rec FI=VOL_LABEL SessId=1 SessTim=1035062102 Strm=0 rlen=147 Block: 2 blen=64512 First rec FI=6 SessId=1 SessTim=1035062102 Strm=DATA rlen=4087 Block: 3 blen=64512 First rec FI=12 SessId=1 SessTim=1035062102 Strm=DATA rlen=5902 Block: 4 blen=64512 First rec FI=19 SessId=1 SessTim=1035062102 Strm=DATA rlen=28382 ... Block: 65 blen=64512 First rec FI=83 SessId=1 SessTim=1035062102 Strm=DATA rlen=1873 Block: 66 blen=19195 First rec FI=83 SessId=1 SessTim=1035062102 Strm=DATA rlen=2973 bls: Got EOF on device /tmp End of File on device
Armed with the SessionId and the SessionTime, you can extract just about anything.
If you want to know even more, add a second -v to the command line to get a dump of every record in every block.
./bls -k -v -v /tmp/File002
bls: block.c:79 Dump block 80f8ad0: size=64512 BlkNum=1
Hdrcksum=b1bdfd6d cksum=b1bdfd6d
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=VOL_LABEL Strm=0 len=147 p=80f8b40
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=SOS_LABEL Strm=-7 len=122 p=80f8be7
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=1 Strm=UATTR len=86 p=80f8c75
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=2 Strm=UATTR len=90 p=80f8cdf
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=UATTR len=92 p=80f8d4d
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=DATA len=54 p=80f8dbd
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=MD5 len=16 p=80f8e07
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=UATTR len=98 p=80f8e2b
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=DATA len=16 p=80f8ea1
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=MD5 len=16 p=80f8ec5
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=UATTR len=96 p=80f8ee9
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=DATA len=1783 p=80f8f5d
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=MD5 len=16 p=80f9668
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=UATTR len=95 p=80f968c
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=32768 p=80f96ff
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=32768 p=8101713
bls: block.c:79 Dump block 80f8ad0: size=64512 BlkNum=2
Hdrcksum=9acc1e7f cksum=9acc1e7f
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=contDATA len=4087 p=80f8b40
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=31970 p=80f9b4b
bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=MD5 len=16 p=8101841
...
Normally, you will restore files by running a Restore Job from the Console program. However, bextract can be used to extract a single file or a list of files from a Bacula tape or file. In fact, bextract can be a useful tool to restore files to an empty system assuming you are able to boot, you have statically linked bextract and you have an appropriate bootstrap file.
It is called:
Usage: bextract [-d debug_level] <device-name> <directory-to-store-files>
-b <file> specify a bootstrap file
-dnn set debug level to nn
-e <file> exclude list
-i <file> include list
-p proceed inspite of I/O errors
-V specify Volume names (separated by |)
-? print this message
where device-name is the Archive Device (raw device name or full filename) of the device to be read, and directory-to-store-files is a path prefix to prepend to all the files restored.
NOTE: On Windows systems, if you specify a prefix of say d:/tmp, any file that would have been restored to c:/My Documents will be restored to d:/tmp/My Documents. That is, the original drive specification will be stripped. If no prefix is specified, the file will be restored to the original drive.
Using the -e option, you can specify a file containing a list of files to be excluded. Wildcards can be used in the exclusion list. This option will normally be used in conjunction with the -i option (see below). Both the -e and the -i options may be specified at the same time as the -b option. The bootstrap filters will be applied first, then the include list, then the exclude list.
Likewise, and probably more importantly, with the -i option, you can specify a file that contains a list (one file per line) of files and directories to include to be restored. The list must contain the full filename with the path. If you specify a path name only, all files and subdirectories of that path will be restored. If you specify a line containing only the filename (e.g. my-file.txt) it probably will not be extracted because you have not specified the full path.
For example, if the file include-list contains:
/home/kern/bacula /usr/local/bin
Then the command:
./bextract -i include-list -V Volume /dev/nst0 /tmp
will restore from the Bacula archive /dev/nst0 all files and directories in the backup from /home/kern/bacula and from /usr/local/bin. The restored files will be placed in a file of the original name under the directory /tmp (i.e. /tmp/home/kern/bacula/... and /tmp/usr/local/bin/...).
The -b option is used to specify a bootstrap file containing the information needed to restore precisely the files you want. Specifying a bootstrap file is optional but recommended because it gives you the most control over which files will be restored. For more details on the bootstrap file, please see Restoring Files with the Bootstrap File chapter of this document. Note, you may also use a bootstrap file produced by the restore command. For example:
./bextract -b bootstrap-file /dev/nst0 /tmp
The bootstrap file allows detailed specification of what files you want restored (extracted). You may specify a bootstrap file and include and/or exclude files at the same time. The bootstrap conditions will first be applied, and then each file record seen will be compared to the include and exclude lists.
If you wish to extract files that span several Volumes, you can specify the Volume names in the bootstrap file or you may specify the Volume names on the command line by separating them with a vertical bar. See the section above under the bls program entitled Listing Multiple Volumes for more information. The same techniques apply equally well to the bextract program.
The bscan program can be used to re-create a database (catalog) from the backup information written to one or more Volumes. This is normally needed only if one or more Volumes have been pruned or purged from your catalog so that the records on the Volume are no longer in the catalog.
With some care, it can also be used to synchronize your existing catalog with a Volume. Although we have never seen a case of bscan damaging a catalog, since bscan modifies your catalog, we recommend that you do a simple ASCII backup of your database before running bscan just to be sure. See Compacting Your Database.
bscan can also be useful in a disaster recovery situation, after the loss of a hard disk, if you do not have a valid bootstrap file for reloading your system, or if a Volume has been recycled but not overwritten, you can use bscan to re-create your database, which can then be used to restore your system or a file to its previous state.
It is called:
Usage: bscan [options] <bacula-archive>
-b bootstrap specify a bootstrap file
-c <file> specify configuration file
-d <nn> set debug level to nn
-m update media info in database
-n <name> specify the database name (default bacula)
-u <user> specify database user name (default bacula)
-P <password> specify database password (default none)
-h <host> specify database host (default NULL)
-p proceed inspite of I/O errors
-r list records
-s synchronize or store in database
-v verbose
-V <Volumes> specify Volume names (separated by |)
-w <dir> specify working directory (default from conf file)
-? print this message
If you are using MySQL or PostgreSQL, there is no need to supply a working directory since in that case, bscan knows where the databases are. However, if you have provided security on your database, you may need to supply either the database name (-b option), the user name (-u option), and/or the password (-p) options.
As an example, let's suppose that you did a backup to Volumes "Vol001" and "Vol002", then sometime later all records of one or both those Volumes were pruned or purged from the database. By using bscan you can recreate the catalog entries for those Volumes and then use the restore command in the Console to restore whatever you want. A command something like:
bscan -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0
will give you an idea of what is going to happen without changing your catalog. Of course, you may need to change the path to the Storage daemon's conf file, the Volume name, and your tape (or disk) device name. This command must read the entire tape, so if it has a lot of data, it may take a long time, and thus you might want to immediately use the command listed below. Note, if you are writing to a disk file, replace the device name with the path to the directory that contains the Volumes. This must correspond to the Archive Device in the conf file.
Then to actually write or store the records in the catalog, add the -s option as follows:
bscan -s -m -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0
When writing to the database, if bscan finds existing records, it will generally either update them if something is wrong or leave them alone. Thus if the Volumes you are scanning are all or partially in the catalog already, no harm will be done to that existing data. Any missing data will simply be added.
If you have multiple tapes, you should scan them with:
bscan -s -m -c bacula-sd.conf -v -V Vol001\|Vol002\|Vol003 /dev/nst0
You should, always try to specify the tapes in the order they are written. However, bscan can handle scanning tapes that are not sequential. Any incomplete records at the end of the tape will simply be ignored in that case. If you are simply reparing an existing catalog, this may be OK, but if you are creating a new catalog from scratch, it will leave your database in an incorrect state. If you do not specify all necessary Volumes on a single bscan command, bscan will not be able to correctly restore the records that span two volumes. In other words, it is much better to specify two or three volumes on a single bscan command rather than run bscan two or three times, each with a single volume.
Note, the restoration process using bscan is not identical to the original creation of the catalog data. This is because certain non-essential data such as volume reads, volume mounts, etc is not stored on the Volume, and thus is not restored by bscan. The results of bscanning are, however, perfectly valid, and will permit restoration of any or all the files in the catalog using the normal Bacula console commands.
If you wish to compare the contents of a Volume to an existing catalog without changing the catalog, you can safely do so if and only if you do not specify either the -m or the -s options. However, at this time (Bacula version 1.26), the comparison routines are not as good or as thorough as they should be, so we don't particularly recommend this mode other than for testing.
This is the mode for which bscan is most useful. You can either bscan into a freshly created catalog, or directly into your existing catalog (after having made an ASCII copy as described above). Normally, you should start with a freshly created catalog that contains no data.
Starting with a single Volume named TestVolume1, you run a command such as:
./bscan -V TestVolume1 -v -s -m -c bacula-sd.conf /dev/nst0
If there is more than one volume, simply append it to the first one separating it with a vertical bar. You may need to precede the vertical bar with a forward slash escape the shell -- e.g. TestVolume1\|TestVolume2 . The -v option was added for verbose output (this can be omitted if desired). The -s option that tells bscan to store information in the database. The physical device name /dev/nst0 is specified after all the options.
For example, after having done a full backup of a directory, then two incrementals, I reinitialized the SQLite database as described above, and using the bootstrap.bsr file noted above, I entered the following command:
./bscan -b bootstrap.bsr -v -s -c bacula-sd.conf /dev/nst0
which produced the following output:
bscan: bscan.c:182 Using Database: bacula, User: bacula bscan: bscan.c:673 Created Pool record for Pool: Default bscan: bscan.c:271 Pool type "Backup" is OK. bscan: bscan.c:632 Created Media record for Volume: TestVolume1 bscan: bscan.c:298 Media type "DDS-4" is OK. bscan: bscan.c:307 VOL_LABEL: OK for Volume: TestVolume1 bscan: bscan.c:693 Created Client record for Client: Rufus bscan: bscan.c:769 Created new JobId=1 record for original JobId=2 bscan: bscan.c:717 Created FileSet record "Kerns Files" bscan: bscan.c:819 Updated Job termination record for new JobId=1 bscan: bscan.c:905 Created JobMedia record JobId 1, MediaId 1 bscan: Got EOF on device /dev/nst0 bscan: bscan.c:693 Created Client record for Client: Rufus bscan: bscan.c:769 Created new JobId=2 record for original JobId=3 bscan: bscan.c:708 Fileset "Kerns Files" already exists. bscan: bscan.c:819 Updated Job termination record for new JobId=2 bscan: bscan.c:905 Created JobMedia record JobId 2, MediaId 1 bscan: Got EOF on device /dev/nst0 bscan: bscan.c:693 Created Client record for Client: Rufus bscan: bscan.c:769 Created new JobId=3 record for original JobId=4 bscan: bscan.c:708 Fileset "Kerns Files" already exists. bscan: bscan.c:819 Updated Job termination record for new JobId=3 bscan: bscan.c:905 Created JobMedia record JobId 3, MediaId 1 bscan: Got EOF on device /dev/nst0 bscan: bscan.c:652 Updated Media record at end of Volume: TestVolume1 bscan: bscan.c:428 End of Volume. VolFiles=3 VolBlocks=57 VolBytes=10,027,437
The key points to note are that bscan prints a line when each major record is created. Due to the volume of output, it does not print a line for each file record unless you supply the -v option twice or more on the command line.
In the case of a Job record, the new JobId will not normally be the same as the original Jobid. For example, for the first JobId above, the new JobId is 1, but the original JobId is 2. This is nothing to be concerned about as it is the normal nature of databases. bscan will keep everything straight.
Although bscan claims that it created a Client record for Client: Rufus three times, it was actually only created the first time. This is normal.
You will also notice that it read an end of file after each Job (Got EOF on device ...). Finally the last line gives the total statistics for the bscan.
If you had added a second -v option to the command line, Bacula would have been even more verbose, dumping virtually all the details of each Job record it encountered.
Now if you start Bacula and enter a list jobs command to the console program, you will get:
+-------+----------+------------------+------+-----+----------+----------+---------+ | JobId | Name | StartTime | Type | Lvl | JobFiles | JobBytes | JobStat | +-------+----------+------------------+------+-----+----------+----------+---------+ | 1 | kernsave | 2002-10-07 14:59 | B | F | 84 | 4180207 | T | | 2 | kernsave | 2002-10-07 15:00 | B | I | 15 | 2170314 | T | | 3 | kernsave | 2002-10-07 15:01 | B | I | 33 | 3662184 | T | +-------+----------+------------------+------+-----+----------+----------+---------+
which corresponds virtually identically with what the database contained before it was re-initialized and restored with bscan. All the Jobs and Files found on the tape are restored including most of the Media record. The Volume (Media) records restored will be marked as Full so that they cannot be rewritten without operator intervention.
It should be noted that bscan cannot restore a database to the exact condition it was in previously because a lot of the less important information contained in the database is not saved to the tape. Nevertheless, the reconstruction is sufficiently complete, that you can run restore against it and get valid results.
If the Storage daemon crashes during a backup Job, the catalog will not be properly updated for the Volume being used at the time of the crash. This means that the Storage daemon will have written say 20 files on the tape, but the catalog record for the Volume indicates only 19 files.
Bacula refuses to write on a tape that contains a different number of files from what is in the catalog. To correct this situation, you may run a bscan with the -m option (but without the -s option) to update only the final Media record for the Volumes read.
If you use bscan to enter the contents of the Volume into an existing catalog, you should be aware that the records you entered may be immediately pruned during the next job, particularly if the Volume is very old or had been previously purged. To avoid this, after running bscan, you can manually set the volume status (VolStatus) to Read-Only by using the update command in the catalog. This will allow you to restore from the volume without having it immediately purged. When you have restored and backed up the data, you can reset the VolStatus to Used and the Volume will be purged from the catalog.
The bcopy program can be used to copy one Bacula archive file to another. For example, you may copy a tape to a file, a file to a tape, a file to a file, or a tape to a tape. For tape to tape, you will need two tape drives. (a later version is planned that will buffer it to disk). In the process of making the copy, no record of the information written to the new Volume is stored in the catalog. This means that the new Volume, though it contains valid backup data, cannot be accessed directly from existing catalog entries. If you wish to be able to use the Volume with the Console restore command, for example, you must first bscan the new Volume into the catalog.
Usage: bcopy [-d debug_level] <input-archive> <output-archive>
-b bootstrap specify a bootstrap file
-c <file> specify configuration file
-dnn set debug level to nn
-i specify input Volume names (separated by |)
-o specify output Volume names (separated by |)
-p proceed inspite of I/O errors
-v verbose
-w dir specify working directory (default /tmp)
-? print this message
By using a bootstrap file, you can copy parts of a Bacula archive file to another archive.
One of the objectives of this program is to be able to recover as much data as possible from a damaged tape. However, the current version does not yet have this feature.
As this is a new program, any feedback on its use would be appreciated. In addition, I only have a single tape drive, so I have never been able to test this program with two tape drives.
This program permits a number of elementary tape operations via a tty command interface. The test command, described below, can be very useful for testing older tape drive compatibility problems. Aside from initial testing of tape drive compatibility with Bacula, btape will be mostly used by developers writing new tape drivers.
btape can be dangerous to use with existing Bacula tapes because it will relabel a tape or write on the tape if so requested regardless that the tape may contain valuable data, so please be careful and use it only on blank tapes.
To work properly, btape needs to read the Storage daemon's configuration file. As a default, it will look for bacula-sd.conf in the current directory. If your configuration file is elsewhere, please use the -c option to specify where.
The physical device name must be specified on the command line, and this same device name must be present in the Storage daemon's configuration file read by btape
Usage: btape [-c config_file] [-d debug_level] [device_name]
-c <file> set configuration file to file
-dnn set debug level to nn
-s turn off signals
-t open the default tape device
-? print this message.
An important reason for this program is to ensure that a Storage daemon configuration file is defined so that Bacula will correctly read and write tapes.
It is highly recommended that you run the test command before running your first Bacula job to ensure that the parameters you have defined for your storage device (tape drive) will permit Bacula to function properly. You only need to mount a blank tape, enter the command, and the output should be reasonably self explanatory. Please see the Tape Testing Chapter of this manual for the details.
The full list of commands are:
Command Description ======= =========== bsf backspace file bsr backspace record cap list device capabilities clear clear tape errors eod go to end of Bacula data for append test General test Bacula tape functions eom go to the physical end of medium fill fill tape, write onto second volume unfill read filled tape fsf forward space a file fsr forward space a record help print this command label write a Bacula label to the tape load load a tape quit quit btape rd read tape readlabel read and print the Bacula tape label rectest test record handling functions rewind rewind the tape scan read tape block by block to EOT and report status print tape status test test a tape for compatibility with Bacula weof write an EOF on the tape wr write a single record of 2048 bytes
The most useful commands are:
The readlabel command can be used to display the details of a Bacula tape label. This can be useful if the physical tape label was lost or damaged.
In the event that you want to relabel a Bacula, you can simply use the label command which will write over any existing label. However, please note for labeling tapes, we recommend that you use the label command in the Console program since it will never overwrite a valid Bacula tape.
The following programs are general utility programs and in general do not need a configuration file nor a device name.
bsmtp is a simple mail transport program that permits more flexibility than the standard mail programs typically found on Unix systems. It can even be used on Windows machines.
It is called:
Usage: bsmtp [-f from] [-h mailhost] [-s subject] [-c copy] [recipient ...]
-c set the Cc: field
-dnn set debug level to nn
-f set the From: field
-h use mailhost:port as the bsmtp server
-s set the Subject: field
-? print this message.
If the -f option is not specified, bsmtp will use your userid. If the option is not specified bsmtp will use the value in the environment variable bsmtpSERVER or if there is none localhost. By default port 25 is used.
recipients is a space separated list of email recipients.
The body of the email message is read from standard input.
An example of the use of bsmtp would be to put the following statement in the Messages resource of your bacula-dir.conf file. Note, these commands should appear on a single line each.
mailcommand = "/home/bacula/bin/bsmtp -h mail.domain.com -f \"\(Bacula\) %r\"
-s \"Bacula: %t %e of %c %l\" %r"
operatorcommand = "/home/bacula/bin/bsmtp -h mail.domain.com -f \"\(Bacula\) %r\"
-s \"Bacula: Intervention needed for %j\" %r"
Where you replace /home/bacula/bin with the path to your Bacula binary directory, and you replace mail.domain.com with the fully qualified name of your bsmtp (email) server, which normally listens on port 25. For more details on the substitution characters (e.g. %r) used in the above line, please see the documentation of the MailCommand in the Messages Resource chapter of this manual.
It is HIGHLY recommended that you test one or two cases by hand to make sure that the mailhost that you specified is correct and that it will accept your email requests. Since bsmtp always uses a TCP connection rather than writing in the spool file, you may find that your from address is being rejected because it does not contain a valid domain, or because your message is caught in your spam filtering rules. Generally, you should specify a fully qualified domain name in the from field, and depending on whether your bsmtp gateway is Exim or Sendmail, you may need to modify the syntax of the from part of the message. Please test.
When running bsmtp by hand, you will need to terminate the message by entering a ctl-d in column 1 of the last line.
dbcheck is a simple program that will search for inconsistencies in your database, and optionally fix them. The dbcheck program can be found in the <bacula-source>/src/tools directory of the source distribution. Though it is built with the make process, it is not normally "installed".
It is called:
Usage: dbcheck [-c config] [-C catalog name] [-d debug_level] []
-b batch mode
-C catalog name in the director conf file
-c director conf filename
-dnn set debug level to nn
-f fix inconsistencies
-v verbose
-? print this message
If the -c option is given with the Director's conf file, there is no need to enter any of the command line arguments, in particular the working directory as dbcheck will read them from the file.
If the -f option is specified, dbcheck will repair (fix) the inconsistencies it finds. Otherwise, it will report only.
If the -b option is specified, dbcheck will run in batch mode, and it will proceed to examine and fix (if -f is set) all programmed inconsistency checks. If the -b option is not specified, dbcheck will enter interactive mode and prompt with the following:
Hello, this is the database check/correct program.
Please select the function you want to perform.
1) Toggle modify database flag
2) Toggle verbose flag
3) Repair bad Filename records
4) Repair bad Path records
5) Eliminate duplicate Filename records
6) Eliminate duplicate Path records
7) Eliminate orphaned Jobmedia records
8) Eliminate orphaned File records
9) Eliminate orphaned Path records
10) Eliminate orphaned Filename records
11) Eliminate orphaned FileSet records
12) Eliminate orphaned Client records
13) Eliminate orphaned Job records
14) Eliminate all Admin records
15) Eliminate all Restore records
16) All (3-15)
17) Quit
Select function number:
By entering 1 or 2, you can toggle the modify database flag (-f option) and the verbose flag (-v). It can be helpful and reassuring to turn off the modify database flag, then select one or more of the consistency checks (items 3 through 9) to see what will be done, then toggle the modify flag on and re-run the check.
The inconsistencies examined are the following:
During standard purging (or pruning) of Job records, Bacula does not check for orphaned Filename records. As a consequence, over a period of time, old unused Filename records will accumulate and use space in your database. This check will eliminate them. It is strongly recommended that you run this check at least once a year, and for large database (more than 200 Megabytes), it is probably better to run this once every 6 months.
testfind permits listing of files using the same search engine that is used for the Include resource in Job resources. Note, much of the functionality of this program (listing of files to be included) is present in the estimate command in the Console program.
The original use of testfind was to ensure that Bacula's file search engine was correct and to print some statistics on file name and path length. However, you may find it useful to see what bacula would do with a given Include resource. The testfind program can be found in the <bacula-source>/src/tools directory of the source distribution. Though it is built with the make process, it is not normally "installed".
It is called:
Usage: testfind [-d debug_level] [-] [pattern1 ...]
-a print extended attributes (Win32 debug)
-dnn set debug level to nn
- read pattern(s) from stdin
-? print this message.
Patterns are used for file inclusion -- normally directories.
Debug level>= 1 prints each file found.
Debug level>= 10 prints path/file for catalog.
Errors are always printed.
Files/paths truncated is a number with len> 255.
Truncation is only in the catalog.
Where a pattern is any filename specification that is valid within an Include resource definition. If none is specified, / (the root directory) is assumed. For example:
./testfind /bin
Would print the following:
Dir: /bin Reg: /bin/bash Lnk: /bin/bash2 -> bash Lnk: /bin/sh -> bash Reg: /bin/cpio Reg: /bin/ed Lnk: /bin/red -> ed Reg: /bin/chgrp ... Reg: /bin/ipcalc Reg: /bin/usleep Reg: /bin/aumix-minimal Reg: /bin/mt Lnka: /bin/gawk-3.1.0 -> /bin/gawk Reg: /bin/pgawk Total files : 85 Max file length: 13 Max path length: 5 Files truncated: 0 Paths truncated: 0
Even though testfind uses the same search engine as Bacula, each directory to be listed, must be entered as a separate command line entry or entered one line at a time to standard input if the - option was specified.
Specifying a debug level of one (i.e. -d1) on the command line will cause testfind to print the raw filenames without showing the Bacula internal file type, or the link (if any). Debug levels of 10 or greater cause the filename and the path to be separated using the same algorithm that is used when putting filenames into the Catalog database.