Backup & Disaster Recovery How-tos & Troubleshooting

Is data on Backup & Disaster Recovery appliance encrypted?

Hard disks where Backup & Disaster Recovery appliance stores backups are encrypted to protect data from being accessed offline.

Can I transfer files over VNC connection to or from a virtual machine booted on the Backup & Disaster Recovery appliance?

No, the system does not support file transferring over the VNC connection to or from a virtual machine booted on the Backup & Disaster Recovery appliance.

Jumbo frame MTU support

The highest setting possible for Jumbo Frames is 8900.

Please note that this will work on NIC1, but not on NIC0 (the two interfaces have different e1000 chipsets and also have different feature sets). No errors shown if it does not work when changing MTU size. The MTU value simply goes back to the default on reloading the page if the setting did not take.

If you are currently using Nic0:

Transfer settings from NIC0 to NIC1.
Set your MTU on NIC1.
Enable NIC1.
Disable NIC0.
Submit and activate configuration changes.
Restart the appliance to make sure the changes applied.

What are the RAID levels in different Backup & Disaster Recovery appliances?

By default, each Backup & Disaster Recovery appliance is provisioned with a predefined RAID level.

See details.

Does disaster recovery backup agent need to run as domain admin?

The disaster recovery backup agent does not have to be run as a system or domain admin on your computer, but you do need to make sure the agent will have sufficient access to the files it needs to back up, otherwise you will get Access Denied errors or warnings in the message logs of the backup.

There are two options for successful backups:

Run the agent as an admin (easiest)
Create a specific user that will give the Backup & Disaster Recovery appliance needed access for the backups.

This may take some trial and error by viewing the message logs to make sure there are no files that are denied access, but it is a possibility if you do not wish to have the agent running as an admin.

How to delete a client from Backup & Disaster Recovery appliance

In the Management Console of the Backup & Disaster Recovery appliance, go to Clients › Summary.
Right-click the client you want to delete, and then click Edit.
Click Actions, and then click Edit.
This will take you to Clients › Edit.
Click Delete, and confirm the deletion.
Wait a few minutes to allow the appliance to complete the process.
Click Activate configuration to apply changes.

How to restrict access from the Dashboard to Backup & Disaster Recovery appliance

To restrict access from the Infrascale Dashboard to Backup & Disaster Recovery appliance:

Log in to the appliance Management Console, and go to the Settings tab.
In the Security group, click Remote Access.
Configure the remote access settings as desired.

How to configure disaster recovery image backup on Backup & Disaster Recovery appliance

A DR image backup is a type of backup designed for quick recovery of your critical systems.

DR image backups allow administrators to boot full systems on the appliance or in the cloud without the need for a boot disk.

Things to know

VMware environments do not require installing the DR imaging software when configured from the host level.
Hyper-V environments require installing the DR imaging software on the Hyper-V host.
DR image backups can be run on live systems.
Incremental backups are converted into the synthetic full backups providing independence from the rest of the incremental backup chain in the event of a full restore.
If you have other backup jobs on this machine, you may want to configure a new client just for the DR image backups, as you may want different retention policies to be set.

What you need

Access to the machines you want to set up a DR image backup job.
A configured client for the machines, for which you want to set up a DR image backup job.

Step 1. Install the DR imaging software

In the appliance Management Console, go to Settings › Downloads.
Download the DR Backup Agent.
Launch the DR Backup Agent on the machine you want to protect.
Installation is automatic and requires no additional steps.

Step 2. Configure a backup client

In the appliance Management Console, go to Clients.
Right-click the client with the installed DR backup agent, and then click Edit.
Configure the backup schedules.
Configure the retention policy.
Enter an email address to receive notifications.
Optionally, identify scripts to execute before or after the backups.
Configure the networking options.

How to back up Microsoft SQL Server database as flat file

Introduction

We have put together the information below to help you understand how SQL Server can be scripted to create a backup or “flat” file.

The concept is that you create a SQL script to start the SQL backups, then run that script from the Run Before command option in the appliance Management Console › Clients › Edit. When the backup begins, we can then capture the SQL backup file created and saved to a certain location by your script file.

Create a script file

Using a simple text editor, create a file with a descriptive name like: SqlDatabaseBackup.sql. For this example, we will save the file on C disk, so the full path would be C:SqlDatabaseBackup.sql.

The contents of this file are explained below.

Database full backup

Regardless of how many files your database uses, you can create a complete backup of your database with one simple command:

BACKUP DATABASE [dbname] to [backup_device] 

For example, BACKUP DATABASE pubs to disk = 'c:mssqlbackuppubs.bak' creates a complete backup of the pubs database using the file c:mssqlbackuppubs.bak.

The database can remain online and accessible to users while this backup is being made. To take a consistent snapshot of the database, a copy of the transaction log is also included with the database backup.

In the backup file, SQL Server stores the names and locations of the files actually used in the database. Upon restoring the database, SQL Server recreates all the necessary files, as many as there might be. A database thus restored is equivalent to the point in time that the backup finished. A complete database backup is very simple to execute and use in recovery. If your data does not change often, you might schedule a nightly full backup of your database. Even if you need more frequent backups, regularly scheduling a full database backup each hour might be sufficient (if your database is small enough).

So your SqlDatabaseBackup.sql file simply has one BACKUP DATABASE command for each database you wish to back up. Enter each command on a new line. Be sure to save each database BAK file in a directory included (either inherently or specifically) in your backup file set.

Finally, edit your SQL client on the appliance to include a Run Before Backup command. The command to run should be this: sqlcmd -i C:SqlDatabaseBackup.sql.

Make sure when you are backing up your SQL Server as a “flat” file that you are backing it up to the same file name each time. Using unique file names can cause problems with customers who use replication.

How to back up Oracle databases

Prerequisites

There are two ways to backup an Oracle database, RMAN and VSS. For either, to be able to backup the database without having to stop access to it, you will need to put the database into the ARCHIVELOG mode.

If the database is not already in the ARCHIVELOG mode, setting that mode will require restarting the database.

Back up using RMAN

Create an RMAN script file to back up your database to a location on the server’s local drive:

run { 
   allocate channel oem_backup_disk1 type disk format 'c:tempdb_backup%U'; 
   backup as COPY tag '%TAG' database; 
   backup as COPY archivelog all not backed up delete all input; 
   release channel oem_backup_disk1;
 } 
run { 
   allocate channel oem_backup_disk1 type disk format 'c:tempdb_backup%U' maxpiecesize 1000 G;
   backup as COPY tag '%TAG' current controlfile;
   release channel oem_backup_disk1;
 } 

Then add to the client setup on the appliance a run before line that runs that script:

rman target "sys/a@orcl" cmdfile "c:rman_backup.txt" 

In this example, sys is the database username being logged in as, a is the password, and orcl is the name of the database being backed up.

Be sure to include the directory that RMAN will put the database backup into in the file set for the client.

In the example above, it is c:tempdb_backup.

Once the backup job has backed up the copy of the database, it can be deleted from the server. We can do this by setting Run After Backup. In the example above, it is del /Q c:tempdb_backup*.

Back up using VSS

First, install the Oracle VSS Writer service for the database. This is done with the oraVSSw command:

oraVSSw SID /I

where SID is the database SID.

Be careful to get the SID right, as there will be no warning that you have it wrong, the service will be registered, but will not do anything.

Once the Oracle VSS Writer service has been registered, you should be able to see it in the services msc snap in services.msc.

The service will be listed as the SID added to its name.

By default, this service is configured to run under the local system user, but this user does not normally have adequate access to the database. You need to stop the service, right-click it, go to the Log On tab, and set the service to run as a user who is a member of the ORA_DBA group. Then start the service again. Check with the VSSadmin list writers command that the Oracle VSS Writer for your database is shown (if not, the user the service is running as might not have enough access to the database).

Set the client to include all non-excluded writers, and run the backup. This should backup the database. To restore the database, first you will have to stop it, then browse and restore the job you want to restore, select Oracle VSS Writer, and then click Restore. Then you will have to recover or start the database again.

How to connect to the Hybrid Cloud DR appliance via VPN

These instructions use Shrew Soft VPN Client (Standard edition) for Windows as an example. If you use another VPN client, or another operating system, the routine may differ somewhat.

In the Infrascale Dashboard, go to Disaster Recovery › Hybrid Cloud Status.
On the DR Runs tab, find and click a DR run you need to connect to, then click View Details in the left pane that opens.
The DR run must be booted, that is – must have the RUNNING status (see the Status column).
On the right side of the DR run details page, note Public IP and Pre-shared Key for VPN – these will be used later to establish the VPN connection.
Download and install the VPN client.
Open the VPN client (VPN Access Manager), and click Add (or Edit › Add).
The VPN Site Configuration dialog box opens.
On the General tab, in the Host Name or IP Address box, enter the public IP address you have noted in the DR run details page.
On the Authentication tab:
1. In the Authentication Method drop-down list, select Mutual PSK.
2. On the Credentials subtab, in the Pre Shared Key box, enter the pre-shared key for VPN you have noted on the DR run details page.
On the Phase 1 tab, in the Exchange Type drop-down list, select main.
Click Save to apply changes.
Select the newly created configuration, and then click Connect.
The VPN Connect dialog box opens.
On the Connect tab, click Connect to establish the VPN connection.
The VPN client will show tunnel opened after it establishes the connection successfully.

How to pause all scheduled jobs for all clients on Backup & Disaster Recovery appliance at once

Log in to the Management Console of the Backup & Disaster Recovery appliance.
Go to Jobs › Settings.
In the Job Schedules drop-down list box, select Disabled.
Click Apply, and then click Activate Configuration on the upper right.

This will turn off all scheduled jobs until you select Enabled again.

You can still start the jobs manually for an individual clients on the Clients › Summary tab.

How to perform system state restore

Restoring a computers system state restores the computers registry, COM+ class database, boot and system files, and so on from a system state backup. A system state backup is performed when you select the VSS and System State options.

For an Active Directory server, the system state also includes the Active Directory domain controllers. You can also restore the domain controllers separately from the rest of the system state writers.

When you select a backup job and select Browse and Restore, the file browser for the backup job will appear.

When you restore the system state to a computer, it is important that you restore the entire folder of system state writers and not individual writers. Also, you must restore the system state to the specific computer that it was backed up from (or, at least, to a computer with the exact same configuration as the backed up computer).

In the Navigation Screen pane, expand the VSS folder and select the System State folder. This will restore all of the folders under the System State folder, as required.

If you attempt to select or clear individual folders on the Entry List pane (on the right side), you will see a warning about unexpected results from a partial system state restore. Make sure that you check the System State folder and that you leave the other folder selections unchanged (all subfolders checked).

Then click Restore on the file browser screen. The system state will be restored to the client computer.

Once you have started a system state restore, you must not interrupt it in any way. This includes canceling the restore job. Any interruptions have the potential to leave the system in an non-bootable/unusable state.

How to prevent automatic deletion of backup jobs

The following information applies both to the primary and to the secondary appliances.

By default, a backup job can be deleted automatically:

under the retention settings, or
if the respective client is deleted or unregistered, and the Backup & Disaster Recovery appliance was configured to delete the orphaned jobs.

To prevent automatic deletion of a backup job in those cases, you can pin the job as follows:

In the appliance Management Console, go to Jobs › History.
Right-click a successfully completed job, and then click Pin Job.
Also, you can select a job, and then click Pin Job on the toolbar.
In the Pin Job dialog:
1. In the Date Until Pinned combo box, set the date, by which the job will stay pinned.
2. In the Comments box, enter a comment, and then click OK.

Pinned jobs are indicated by a checkmark in the Pinned column. You can sort and filter the jobs by this status.

To unpin a job, right-click the pinned job, and then click Unpin Job.

To view or edit the details of a pinned job, right-click the pinned job, and then click Pin Info.

How to reset Changed Block Tracking for VMware virtual machines

To reset Changed Block Tracking (CBT) for a VMware virtual machine (VM):

In the Management Console of the Backup & Disaster Recovery appliance, go to Clients › VMware.
Select a VM you want to reset CBT for.
CBT can be reset only for those VMs that are registered, powered on, do not have snapshots, and have CBT allowed).
On the toolbar, click Advanced, and then click Reset CBT.

After the system finishes resetting CBT, it will show a dialog box with confirmation.

How to reset password on Backup & Disaster Recovery appliance

If you lost or forgot the password to log in to the appliance Management Console, you can reset it. For this, you must have access to the appliance terminal.

8.7.0 and later

To change or reset the password on the appliance with firmware version 8.7.0 and later:

Restart the appliance.
Wait for the appliance to show the boot loader, and then press any key (except Enter and Return) to interrupt booting.
You have 5 seconds to interrupt before the appliance continues booting. Otherwise, you will have to restart the appliance again.
Select the option that starts with OS [version], and then press e to edit the commands executed before booting.
Go to the end of the line that starts with linux, enter rd.break, and then press Ctrl+X to start booting.
The appliance boots into the shell.
Check the access mode on the /sysroot mount point.
For this, run the following command:
```
mount | grep sysroot 
```
- If /sysroot is mounted in the read-only mode (see ro in the command output), remount it in the read-write mode.
  For this, run the following command:
```
mount -o remount,rw /sysroot/ 
```
- If /sysroot is mounted in the read-write mode (see rw in the command output), continue with the next step.
Change the root directory.
For this, run the following command:
```
chroot /sysroot 
```
Set the new password.
For this, run the following command, and follow the on-screen instructions to set the password:
```
passwd 
```
Relabel the file system.
For this, run the following command:
```
touch ./autorelabel 
```
Change the root directory, and then log out to continue with the normal boot.
For this, run the following commands:
```
exit 
logout 
```
Wait until the file system relabeling completes. This might take some time depending on the system performance and the number of files.

The appliance boots normally, and you can log in to the Management Console with the newly set password.

8.6.x and earlier

To change or reset the password on the appliance with firmware version 8.6.1 and earlier:

Restart the appliance.
Wait for the appliance to show the boot loader, and then press any key (except Enter and Return) to interrupt booting.
You have 3 seconds to interrupt before the appliance continues booting. Otherwise, you will have to restart the appliance again.
Select the option that starts with OS [version] (VGA), and then press e to edit the commands executed before booting.
Select the line that starts with kernel, and then press e.
Press Space, enter 1, press Enter, and then press b to start the shell.
Set the new password.
For this, run the following command, and follow the on-screen instructions to set the password:
```
passwd 
```
Restart the appliance.
For this, run the following command:
```
reboot 
```

The appliance restarts, and you can log in to the Management Console with the newly set password.

How to reset networking settings on Backup & Disaster Recovery appliance to default

Restart Backup & Disaster Recovery appliance.
Once the appliance boots, it will show the login screen.
Log in, and then select Configure Networking Option.
Select Reset to Default.
This resets the networking configuration.
Change all values as necessary—that is, short name, domain name, and default gateway.
Select Restart Networking.
Proceed with the configuration further to configure IP address (static or via DHCP).
Save changes, and then restart the appliance.

If you assigned a static address to the device, you will need to reboot the appliance to apply the correct address, but, during the initial setup the system will take a few minutes to set up the directory and storage structure and you will need to wait for this to finish before you reboot the appliance. The status of this process can be found in the system messages at the bottom of the screen. Do not restart the system until the UCAR setup is at 100%, and the system shows The system is ready. With all this done, please try to access the appliance management console through the web GUI.

How to restart backup service on Backup & Disaster Recovery appliance, or disaster recovery backup agent

If the backups seem locked or frozen (for example, not cleared when canceled), or if Backup & Disaster Recovery appliance acts strangely, or if you just need to stop all backup jobs immediately, you have to restart the backup service.

To restart the backup service on the appliance:

Sign in to the appliance Management Console.
Go to System › Director Status.
The Director Status subtab shows the details about the RVX-Backup kernel version—that is, when the backup daemons started and how many jobs have run since then.
Click Restart Backup Service.
This will cause all running, importing, and deduping jobs to stop.

Sometimes, when a backup job was canceled or is hung, it still appears in Jobs › Active Jobs or Clients › Summary. If this is due to the server-side issues, it is best to stop and restart the service on the agent (server side).

You can stop and start again the backup agent service using the built-in service manager on your server.

If you are using a Windows client, start the backup agent, and on the Service Control tab, click Stop, and then click Start.

How to restore Windows registry partially

Because editing the registry cannot be done without some risk to the system, it is recommended to create a system restore point before proceeding.

The registry is backed up using a VSS writer. VSS writers will only be backed up if the file set on the client has the following setting enabled: Use Service with All Non-Excluded Writers (recommended).

The VSS Registry Writer backs up the hive files, which backs up the registry as a single unit. A hive is a logical group of keys, subkeys, and values in the registry that has a set of supporting files containing backups of its data. Each time a new user logs on to a computer, a new hive is created for that user with a separate file for the user profile. This is called the user profile hive. A user’s hive contains specific registry information pertaining to the user application settings, desktop, environment, network connections, and printers. User profile hives are located under the HKEY_USERS key.

Traditionally, the writer would be restored in its entirety. If you wish to restore only select portions of the registry, you will need to pull the desired portions out of the registry hive files.

Obtain the hive files from the VSS Registry Writer:
1. From Jobs › History, right-click the desired job and select Browse and Restore.
2. Browse to VSS:/System State/Registry Writer/Registry/C:/Windows/System32/config/.
3. Select the names or links of the various files (these are called subkeys) located within the configuration folder one by one.
  This will prompt you to download the files to the location your web browser uses for downloaded files.
Extract the desired portions of the registry:
1. Open the hive files by temporarily loading them onto the registry.
2. Open the registry editor, select HKEY_LOCAL_MACHINE in the left panel.
  HKLM must be the active selection, or the option to Load Hive will not be available.
3. Select File › Load Hive.
4. Point to one of the Subkeys restored from the appliance and select Open.
  Different parts of the registry are located in different areas/subkeys. Determine where to find the registry key you wish to restore.
  The Registry Editor will ask for a name for the key to load the hive into. You can name it any unique string. For example, backup.
5. Browse into the key to find the portion of the registry you wish to restore.
  Due to the limitations of the registry editor, you will not be able to drag and drop the key to where you want it. You must first export the file, then import it again.
6. Right-click the desired key and chose Export.
  Save the file in a location you will be able to remember and with a name that you will recognize.
  After the desired keys are exported, you will need to close the hive file.
7. Select the top key of the hive › File › Unload Hive › Yes when it asks if you are sure.
  First right-click the REG file created by the export above, and choose edit to open the file in notepad.
8. From the edit menu, select find and replace.
  Here we will find the temporary location we had the hive file open at (in this example backup), and replace that with where the registry keys should be.
9. Save and exit.
Double-click the REG file you just edited and confirm all prompts to continue to import the file into the registry.

How to test network speed between Backup & Disaster Recovery appliance and clients

This test will only identify network bottlenecks and will not indicate how fast a backup job will run.

There are some instances where backups are running slow and is difficult to pin what the cause of the problem might be. This test will help to identify if the problem is in the network and routing.

To start this test, you will need to first SSH in to the Backup & Disaster Recovery appliance as the root user.

Once at the command line, make the appliance the listening server by running the following command:

iperf -s

Next, connect to the client that is currently showing signs of being slow.

For clients running Windows:

Download the iperf and move it to drive C.
Open Command Prompt and go to the drive C:
```
cd C: 
```
Enter the following command:
```
iperf -c <ipaddress > -t 60 
```

For clients running Linux, iPerf is usually available in any Linux distribution repository, and is often installed as part of the base installation.

To start the test, open the Shell, and enter the following command:

iperf -c <ipaddress > -t 60 

After performing the test, the Shell will show something like this after about 60 seconds:

C: > iperf -c 198.51.100.34 -t 60 
------------------------------ 
Client connecting to 198.51.100.34, TCP port 5001 
TCP window size: 8.00 KByte (default)
------------------------------ 
[132] local 198.51.100.79 port 56109 connected with 198.51.100.34 port 5001 
[ID] Interval Transfer Bandwidth 
[132] 
0.0-60.3 sec 43.5 MBytes 6.05 Mbits/sec 

This will give you the speed that it was able to transfer data back and forth between the two.

You can specify how long you want the time to run by changing the number after the -t. The longer the test runs, the better the results will your average speeds.

You can also flip the test to have the client be the server and the appliance be the client. This may show if there is a problem sending or receiving.

How to update firmware of the Backup & Disaster Recovery appliance

In the Management Console of the Backup & Disaster Recovery appliance, go to the Settings tab.
In the Basic group on the left, click Firmware Update.
- If you have Automatically download updates to appliance selected, click Install to prepare for the update.
- If you have Automatically download updates to appliance cleared, click Download to save the update file to the appliance.
  After the download is complete, click Install to prepare for the update.
After the appliance is ready to install the update, click Reboot.

After you update the appliance firmware, we strongly recommend to update the backup agents on the clients associated with the appliance. See Install DR backup agent for details.

How to use both network interface controllers on Backup & Disaster Recovery appliance

Multiple physical network interfaces on the same subnet may cause routing issues.

Configuring multiple network devices to be on the same subnet is not supported.

The proper way to set up the Backup & Disaster Recovery applioance when using both NICs is to configure one NIC to use specific static routes, and any traffic on those routes will use that NIC. (You will need to add specific routes for the second NIC using Settings › Networking › Routes. The static routes will need to point at the next hop router on the second NIC network.) The other (second) NIC will use the default gateway, which indicates “everything not specifically going to the other NIC, will go here”.

How to use UNC paths to back up network shared folders

Commonly used to backup NAS shares or partitions.

Example scenario

You have a network share on a NAS server you want to back up.

Infrascale disaster recovery backup agent cannot be installed directly on the NAS server because it runs an operating system, which is not supported by the backup agent.

In this case, you have to use UNC to back up the said share:

Log in to the Management Console of the Backup & Disaster Recovery appliance.
In Clients › Summary, add a new client or use an existing one.
Edit the backup file set of the client to include the UNC path to the NAS share.
The path must start with //?/UNC/. For example, //?/UNC/serve_name/share or //?/UNC/IP address/Folder_to_start_in.
The backup agent must be configured to run as a user, who has access to this path rather than the default Local System. You can configure this either in the RVXConfig tool or in Services of the client computer. This account must also have backup operator permissions in the network.
If the share you are backing up is not a Windows file server, you will need to select the File Set option to ignore Win32 attributes.
If the job log contains ERR=00000522: A required privilege is not held by the client for each file, you have to select the File Set option to ignore Win32 attributes.
A less optimal alternative is to set up a mapped drive from a Windows computer, and include that mapped drive as part of the backup for that Windows client. This method will require the backup agent to be manually started by the user as a program rather than as a service.

Backup fails due to database being offline

Error message

MSSQL: warning: explicitly included database 'XXX' on DEFAULT server instance appears to be offline, will try anyway 
MSSQL: error: backup of database 'XXX' on DEFAULT server instance failed unexpectedly: An exception occurred while executing a Transact-SQL statement or batch.: because Database 'XXX' cannot be opened because it is offline. 

Error description

Databases and all of their dependencies must be online to be backed up through Microsoft SQL Server.

Any backup operation that implicitly or explicitly references data that is offline fails. Some typical examples include the following: You request a full database backup, but one file group of the database is offline. Because all file groups are implicitly included in a full database backup, this operation fails.

To back up this database, you can use a file backup and specify only the file groups that are online. You request a partial backup, but a read/write file group is offline. Because all read/write file groups are required for a partial backup, the operation fails. You request a file backup of specific files, but one of the files is not online. The operation fails. To back up the online files, you can omit the offline file from the file list and repeat the operation.

See further details.

Steps to resolve

Exclude offline databases from the backup using RVXConfig, or bring the selected databases and all of their dependents online before attempting backup.

Backup job hangs or fails due to VSS writer issue on server

Symptoms

Job begins, but does not get past VSS, nor does it seem to fail:

Job hangs with 0’s across the board when viewing Client › Properties for the job and virtually no message logs available; or
Job fails due to network connectivity (reset by peer).

Troubleshooting

Disable VSS entirely to confirm if this is a VSS issue or not.

If the backup completes with VSS disabled, attempt to run a backup with VSS enabled but Include Writers disabled. This will determine if it is all of VSS or just the writers that is causing the issues.

Also, try to run the following through the client diagnostics tab to view the writers. If it does not work (hangs up), try the same from the server itself.

vssadmin list writers

If running the above command hangs as well, try the following:

Try rebooting the machine and see if that fixes the “hang” problem.
If not, make sure the service is not disabled.
1. Open Start Menu, right-click Computer, and then click Manage.
2. Go to the Services panel (Services and Applications › Services).
3. Find Volume Shadow Copy in the list.
4. The Startup Type value should not be Disabled.
5. Change Startup Type to Manual if necessary.
Try restarting the service from the Properties dialog (see steps above).
1. If service is running, click Stop to stop it.
2. Click Start.
3. Check that Service status goes from Starting to Started.
Try running vssadmin list writers again.

Backup job seems to hang or stuck

Symptom

Backup job seems to hang up or gets stuck either in Jobs › Active Jobs, or in Clients › Summary under the Active Jobs column. Possibly other jobs are waiting behind these backup jobs.

Troubleshooting

Go to Jobs › Active Jobs, right-click the job, and then click Client Properties.
Note what file the job seems to be hung up on.
It is possible that the job is performing very slowly rather than completely hung up. Monitor the Client Properties window for a moment to ensure the client is completely hung up.
Go to Jobs › Active Jobs, right-click the job, and then click Message Logs.
Note if there are any errors in the Message Logs.
Go to Support, and review the server.xml log to see if there are any error messages associated with this client and backup.
Restart the services on either the client or the appliance, or both.
For more information on how to properly do this, see How to restart backup service on the appliance or agent.

If the client seems to hang up repeatedly:

Again, note which file the client seems to be hanging up on, especially if it is the same file each time. (See Client Properties above.)

Go to Clients › Edit.
Select the affected client, and then click the “pencil” button in the Backup group.
Check to see if VSS and Include Writers are selected.

Other things to consider

Waiting for Media

Jobs › Active Jobs, see the Status column.

If status is Waiting for Media, right-click the job, and then click Message Logs.

Review any error messages present.

There is a possibility this could be tied to database problems. If so, contact Infrascale Support.

System State backup (ntbackup) running out of space. (This only affects older versions. Infrascale Disaster Recovery 3.4 and later uses VSS for System State Backup, not ntbackup.)

Jobs › Active Jobs, right-click the job, and then click Message Logs.

If the last message is: CLIENT-FileIO: WSS: System state will be saved to C:Program FilesRVX-Backupsysstatesystemstate.bkf.

Clients › Summary, right-click the client, and then click Properties.

If there is no current file (it may say ‘null’):

When looking at the Message Logs, if the client appears to be hanging in systemstate backup. The most likely cause is ntbackup running out of space, or some other error, and hanging. Please check the C: drive for space and the event log for other possible ntbackup errors. You can also log into session 0 and see if ntbackup has popped up an error dialog.

If you are unable to create enough space, you can disable System State backup using Configure Backup › System State, and clearing the option there.

Cannot find certain files and folders selected for backup using browse and restore

Symptom

You want to restore a particular file to a server, but you cannot find it when browsing to the location it should be on the server using the “Browse and Restore” feature.

Steps to resolve

There are a few things that can cause a folder to go “missing” on the Backup & Disaster Recovery appliance.

File sorting confusion

Make sure you check the entire directory the folder is supposed to reside in. Check to see if perhaps the folder is sorted differently due to the capitalization of the folder name. Currently, the appliance will sort folders alphabetically using folders with capitalized names first, then alphabetically using folders with names in all lowercase. This can cause folders with lowercase names to be several pages later than those with capitalized names.

There have been many instances where a file appears to go missing, but it is actually being sorted differently due to capitalization.

VSS writers

Check to make sure the directory or program does not require the use of VSS Writers. Sometimes, a directory or program will require the data to be backed up using a writer because of certain features/functionality of that directory or program and must be backed up using VSS. This requirement is completely on the program/server side and is something we cannot control.

For example, if data is backed up from a DFS replication share, it must be backed up via the DFS replication service writer.

The files backed up using writers will not be located under the drive they appear to be under on the server. Instead, they will be under the VSS folder in the backup. The writer must be restored as a whole unit because it must be restored using the VSS writer.

This is only backed up if you have enabled VSS with “Use Service with All Non-Excluded Writers (Recommended)” enabled in the file set. If this setting is not selected, the data will not be backed up.

There may be a way to make a “flat file” copy of the DFS share to another location and back that up without VSS or writers enabled. This would allow single file restores similar to typical flat file backups and may be the simplest way to get around the writer restore limitations.

One other option would be to write before/after scripts to get the flat file copy during the regularly scheduled backup.

Client does not run scheduled backups

Symptom

A client does not run a scheduled backup
A manual backup appears to run successfully
No scheduled backup appears in Jobs › Scheduled Jobs for the affected client

Things to check

Make sure the client has a schedule enabled under Clients › Edit (you can also see this information in Clients › Summary under the Schedule Column)
Look over the schedule in Settings › Backups › Job Schedules to confirm the schedule has the proper schedules configured for backups
Check the appliance time in Settings › Basic › Appliance Status, and change it to match the servers if needed by going to Settings › Basic › Date/Time

Steps to resolve

This issue is commonly attributed to something getting hung up during the initial client configuration.

If none of the above steps resolve the issue, it is recommended to create a new client and test to see if the same behavior is observed with the new client.

Also, you may try applying the different schedule to the client, click Apply, then Activate configuration, apply the required schedule, and activate it.

Client status. Backup agent is running but cannot communicate

Issue

In the appliance Management Console › Clients › Summary, the client shows a white and red triangle indicating that the disaster recovery backup agent is running but is unable to communicate.

Issue description

This is almost always a problem with the passwords not matching. The password in the configuration file (also displayed in the RVXConfig tool) must match the password in the appliance Management Console › Clients › Edit.

Steps to resolve

Confirm that the passwords match and then restart the service.

Another way to confirm the passwords match between the client and the agent is to download the configuration from the client and apply it to the agent.

In the appliance Management Console, go to Clients › Summary.
Right-click the client, and then select Download Config in the context menu.
Copy the downloaded configuration file bacula-fd.conf to C:Program FilesRVX-Backup.

Possible alternative causes

A firewall or antivirus is affecting communication.

Please confirm no outside factors are blocking communication between the appliance and the client.

Rarely, the default port (9102) is not working, and it is necessary to change this on both the RVXConfig side, as well as in Clients › Edit.

Try changing the port to 9204 (for example).

If none of the above options help, there could have been a problem with the initial installation of the agent. Uninstall and install the agent again.

Client status. Client is down or unreachable

Details

Status icon	Status label
	Client is down or unreachable

This status indicates one of the following immediate issues has emerged:

client is turned off
client is not on the network
firewall is blocking the bacula-fd service
configuration file is missing
backup agent is not installed

Example

Troubleshooting

To resolve the issues that raised this status, verify and fix (if needed) the following:

Ports 9102 and 9103 are open for the affected client

Check your firewall and antivirus settings. Also, check if other third-party applications use these ports

Services are up and running on the affected client

Open the Run command (press Win+R on the keyboard).
Enter services.msc to open Windows Services Manager.
Check if the following Infrascale services are running (see the Status column).

Client Status Resolution

Address not resolved

This status is typically resolved by correcting the name in the address field. Basically, Backup & Disaster Recovery appliance is unable to resolve the name to an address and so we will not be able to communicate. You might confirm that the name is actually resolvable over the network, you may also replace the name with the IP address. Additionally, you may need to confirm the correct DNS servers under the Settings tab.

Agent not running

The first step here is to confirm if the backup agent is actually running or not. The Windows Services tool is a great place to check this. You could also check by using the “Configure Backup” tool accessed via the start menu.

If the agent claims to be running, recommend restarting the service/agent.

If the agent still will not start, confirm that the configuration file is in place.

Use the RVXConfig tool and run the wizard.

Can the client and appliance ping each other?

Check the event logs for any errors regarding the RVXBRagent starting or stopping.

If the client is a Windows 2008 machine and there is a message that reads “Service is marked as an interactive service. The system is configured to not allow interactive services. This service may not function properly.”

Please see Microsoft knowledge base article.

Verify, in the Task Manager, that RVXBRagent and Bacula are running. You will need to check that you want to show processes from all users.

Confirm that the port in use by the agent (default 9102) is not blocked in any way.

Firewall? Antivirus? Other programs? Unusual networking?

If an exception needs to me made in the firewall, our RVXBRagent needs port 9103 and the Bacula process needs port 9102.

Using a telnet command, on the Support tab, can help determine if something is blocking the port that the client is running on telnet [client address] [port] (for example, 9102).

You might simply try uninstalling and installing the backup agent again.

If the agent in question is a few revs old, try installing a newer version.

Escalate and document any new discoveries.

Running but cannot communicate

This is almost always a problem with the passwords not matching. The password in the configuration file (also displayed in the RVXConfig tool) must match the password in Clients › Edit. Confirm that the passwords match and then restart the service.

If that does not do the trick, there could also be a firewall or antivirus in the way. Please check to see if anything is blocking traffic.

Sometimes the default port (9102) is not working and it is necessary to change this in both RVXConfig and Clients › Edit (for example, 9204).

Lastly, if none of this helps, the installation could have issues and you may need to just uninstall and install it again.

Down or unreachable

This status may be the result of a number possibilities, it may be that the client machine is not actually connected to the network or not powered on at all. This may also be a problem with the Windows firewall blocking the bacula-fd service. An exception to the firewall is usually the recommended solution. This status may also be caused if the configuration file is not in place at all or if the agent software has not been installed. Ensure that everything is installed as it should be and restart the service.

Error. Authoritative answer for host not found

Error message

bsock.c:237 gethostbyname() for host "XXXX" failed: ERR=Authoritative answer for host not found. 

Error description

This error indicates that the IP address of the client (see in Clients › Edit) is not resolvable. By default, the Backup & Disaster Recovery appliance uses the client name as the address, and this field needs to be adjusted.

Steps to resolve

This error can be resolved in two ways:

Verify that the host name is resolvable via DNS, and make adjustments as needed.
Change the address to the IP address rather than the host name.

Error. DR Image Backup Failure: Failed to create SuperAgent remote proxy

Error message

Message logs in the Management Console of the Backup & Disaster Recovery appliance show the following DR image backup error:

DR Image Backup Failure: Failed to create SuperAgent remote proxy

Error description

The error refers to miscommunication between the backup agent installed on the client and the associated appliance.

It can show after updating the backup agent, that is when the previous backup service had not stop before the new one started.

Steps to resolve

If the error relates to a VM client, perform the steps for the associated host.

Uninstall backup agent and install it again

Stop all related Infrascale services running on the client:
- Infrascale Backup and Restore Agent (64-bit)
- Infrascale Installation Client
- Infrascale Remote Management
If a service got stuck in the Stopping state, this indicates possible issues with the operating system. Restart the client and try stopping the service again.
Uninstall all related Infrascale software from the client.
Make sure the following folders were deleted, or delete them manually:
- C:Program FilesInfrascale
- C:Program FilesRVX-Backup
- C:Program Files (x86)Infrascale
- C:Program Files (x86)RVX-Backup
Download the backup agent installer from the appliance Management Console › Settings › Tools › Downloads.
Run the installer and follow the on-screen instructions to install the latest version of the backup agent.
Restart the client, and then check if all related Infrascale services you stopped earlier are running again on the client.

Change client operating system

In the appliance Management Console, go to Clients › Edit.
Select the client with the error.
In the General group, change the operating system to any other.
Click Apply to save changes, and then click Activate Configuration on the upper right.
Change the operating system back again.
Click Apply, and then click Activate Configuration again.

Error. Cannot connect to explicitly configured server instance

Error message

Message logs in the Management Console of the Backup & Disaster Recovery appliance show the following error related to Microsoft SQL Server backup:

MSSQL: warning: cannot connect to explicitly configured 'MSSQL10.XXXX' server instance: Failed to connect to server XXXXMSSQL10.XXXX 

Steps to resolve

You see the MSSQL10 instance when you enumerate the SQL databases in the backup agent on the Microsoft SQL Server.

This is a false error message having to do with the Microsoft SQL Server plug-in searching for a database that does not exist.

In the backup agent on the Microsoft SQL Server, make sure this database is excluded.

Error. Cannot get configuration for backup of database

Error message

Message logs in the Management Console of the Backup & Disaster Recovery appliance show the following error related to Microsoft SQL Server backup:

SNFCSQL2012-FileIO JobId 9929: MSSQL: error: can't get configuration for backup of database 'distribution' on DEFAULT server instance: GetConfiguration timed out after 300000ms 

Error description

The Microsoft SQL Server plug-in not being able to back up databases on Microsoft SQL server 2012. This appears to be an issue with the permissions of the account the agent is running as. The job does not fail, but completes with errors, and does not actually back up the SQL databases.

The requested backup cannot be completed by the SQL Server because the log chain has been broken. This is likely caused by a conflict from another tool or service.

Steps to resolve

It seems that Microsoft SQL Server 2012 has additional security requirements to enable Infrascale plug-in to run a backup that were not present in earlier versions of SQL. Depending on your SQL security settings there may be a number of places that you will have to change permissions to allow backups.

Since Infrascale backup agent runs as the local system account by default, the instructions below will reference the NT AuthoritySystem account. If you are using a different account to run the backup agent, then change these settings for that account. These instructions also assume that you have the backup agent set up according to the normal setup instructions, and also have the SMOs and the NCLI installed, but are getting the error above.

In SQL Management Studio, go to Security › Logins.
Under Login. right-click the NT AuthoritySystem account, and then select Properties.
In the Properties window, click Server Roles.
Make sure both the Sysadmin and Serveradmin roles are selected.
Click OK to close the window, and then try new backup.

If you are still getting the error after setting the roles:

Sign in to the SQL Management Studio.
Right-click the server, and then select Properties.
In the Properties window, select Permissions, and then select the NT AuthoritySystem account from the list in logins or roles.
In the Permissions group, select Grant for all allowable permissions, verifying that none of these permissions have deny the selection.
Click OK to close the window, and try new backup.

At this point, if the backup is still getting this error, then it is not likely a permission issue. Most of the time you can skip making the individual changes to the security settings above by setting the agent to run as your SQL administrator account. To do this log into the agent configuration on the SQL server, browse to the Service Control tab, stop the service, then enter the username/password of the account, then start the service again.

Error. Could not connect to storage daemon

Error message

Message logs in the Management Console of the Backup & Disaster Recovery appliance show the following error if the appliance cannot connect to a backup client:

bnet.c_1011 Could not connect to storage daemon on CFA-IP:9103. ERR=Connection refused 

Steps to resolve

Our services are not running on the client. A firewall may be blocking the connection from the client to the appliance on port 9103.

A quick backup of a reduced file set (with No Change on Full enabled) should help diagnose if the problem still exists after firewall changes or other network fixes.

Error. Could not start. The system cannot find the file specified

Generally this error means the Backup & Disaster Recovery appliance could not gain access to that file, or it does not exist, and the appliance did not back it up. This would be the case for any temporary files that existed at the time the backup began, but had been cleared out by the time the backup got to that specific file.

To eliminate that message from future backups, an exclusion can be created for that file or path from the backup file set or, if the file was already in the file set, its entry may need to be changed if the file had been moved.

Error. An existing connection was forcibly closed by the remote host

Error message

Message logs in the Management Console of the Backup & Disaster Recovery appliance shows the following error:

backup.c:666 Network send error to SD. ERR=An existing connection was forcibly closed by the remote host. 

Error description

This can happen when the Storage Daemon shuts down due to being killed by a process, which looks for out-of-space issues. It can also be due to a network connectivity problem, in which the TCP stack on the appliance eventually closes the connection with a RESET.

Error. Failure getting log and patch files

Error message

FileIO JobId 1422: ESEBackup: failure getting log and patch files: c8000232 
Some log or patch files are missing.Error reading data header from FD. ERR=Connection reset by peer 

c8000230: The database missed a previous full backup before the incremental backup 

c800020e: An incremental backup cannot be performed when circular logging is enabled 

Possible causes

Circular Logging is enabled. This is specifically recommended to be disabled for Exchange 2010, but appears to cause issues when backing up other versions of Exchange as well.

Circular logging has been enabled in the past but has since been disabled.

Another system is using VSS, backing up data or truncating logs.

Transaction logs have been manually deleted.

Steps to resolve

Disable circular logging if enabled.

Make sure you have disabled whatever else may be using VSS, backing up data or truncating logs.

Perform a full exchange backup using NTBackup. This will re-arrange the transaction logs making sure you will back up the correct ones in your next transaction log backup.

How to disable circular logging for Exchange 2000 and Exchange 2003

Click Start, go to Programs › Microsoft Exchange, and then click System Manager.

If the Administrative Groups branch exists in the left pane, expand it, expand the appropriate administrative groups branch, expand the Servers branch, and then expand the appropriate servers branch.

If the Administrative Groups branch does not exist, expand the Servers branch in the left pane, and then expand the appropriate servers branch. To expand a branch, double-click the branch or click the plus sign (+) to the left of the branch.

Right-click the storage group you want, and then click Properties.

To disable circular logging, clear Enable circular logging, and then click OK.

Restart the information store. For this:

Click Start, go to Programs › Administrative Tools › Services.

Click Microsoft Exchange Information Store in the right pane, and then in the Action menu, click Restart. If a dialog box appears stating that extra services will be restarted, click Yes.

The information store must be restarted because when it starts, it reads configuration information from Microsoft Windows 2000 Active Directory. The Active Directory attribute associated with the circular logging setting is called MSExchESEParamCircularLog.

When circular logging is enabled, this attribute is set to 1. When circular logging is disabled, it is set to 0.

How to disable circular logging for Exchange 2007

Start the Exchange Management Console.

In the console tree, expand Server Configuration, and then click Mailbox.

In the work pane, right-click the storage group, for which you want to enable or disable circular logging, and then click Properties. The Properties dialog box appears.

Select or clear Enable circular logging.

Click OK.

To make your changes to the circular logging settings effective, restart the Microsoft Exchange Information Store service, or dismount and then mount all databases in the storage group.

How to perform full Exchange backup using NTBackup on Exchange 2003

This will rearrange the transaction logs making sure you will back up the correct ones in your next transaction log backup. Please make sure all scheduled backups for this server are disabled while the NTBackup is running on this server. (Go to Clients › Edit. There is a drop-down list for Schedule › Select None.)

Click Start › Run, enter NTBACKUP and then click OK.

The backup and restore wizard appears. Click advanced mode on the welcome page.

Click the Backup tab.

On the backup tab expand server, expand servername, expand information store and select the correct storage group or entire information store.

Select file as destination, and name it so you can find and delete it later.

Click Start backup.

This process will purge old transaction logs, making sure the errors above will be solved.

If you are able to get a successful backup via NTBackup, you should be able to get a backup via the appliance. It is recommended to get a full backup immediately following the successful NTBackup.

How to perform full Exchange backup using NTBackup on Exchange 2007

This will rearrange the transaction logs making sure you will backup the correct ones in your next transaction log backup. Please make sure all scheduled backups for this server are disabled while the NTBackup is running on this server. (Go to Clients › Edit. There is a drop-down list for Schedule, select None.)

Go to Start › Administrative Tools › Windows Server Backup.

In the Actions pane, click Backup once. The backup once wizard will appear.

On the backup options page, select different options and click next.

On the select backup configuration page, select Full server, and then click Next. (Use custom if you want to select individual volumes.)

On the specify advanced options page, select VSS full backup, and then click Next.

On the Confirmation page, review the backup settings, and then click Backup.

On the Backup progress page, you can view the status and progress of the backup operation. Click Close when the backup operation has completed.

If you are able to get a successful backup via NTBackup, you should be able to get a backup via the appliance. It is recommended to get a full backup immediately following the successful NTBackup.

Error. A file access error occurred on the host or guest operating system

Error message

A file access error occurred on the host or guest operating system.

Error description

This error indicates that:

the Backup & Disaster Recovery appliance has issues communicating with the host of the virtual machine (VM) or with the datastore; or
the VM has no VMware Tools installed, or VMware Tools lacks access permissions.

Steps to resolve

First of all, run full backup of the affected VM manually:

In the appliance Management Console, go to Clients › VMware.
Select the affected VM.
On the toolbar, click Manual Backup.
In the open dialog box, select Full, and then click OK.

If previous does not help, register the affected VM again:

In the appliance Management Console, go to Clients › VMware.
Right-click the affected VM, and then click Unregister.
Click Activate configuration.
Right-click the affected VM again, and then click Register.
Click Activate configuration.
Run backup of the affected VM manually.

Also, check if the affected VM has the latest version of VMware Tools installed. If so, check if VMware Tools has all permissions required for backup.

Error. Filesystem change prohibited

Error message

Filesystem change prohibited. Will not descend into or `filepath` is a different filesystem. Will not descend from. 

Steps to resolve

This error means that the file is on a different file system than the current one.

Sometimes this can mean that the path following the ends in a symlink or a junction point would lead the backup process away from the current path, and thus may result in skipping all over the file structure, which would lead to skipping lots of data.

We do not follow such links away from the path we are currently backing up, we systematically traverse the file structure so as to not miss anything.

There is a setting to change this under Clients › Edit, but it is prudent to not cross file systems while doing a backups and to instead list them each explicitly.

Error. GetConfiguration timed out

Error message

MSSQL: error: can't get configuration for backup of database `XXX` on DEFAULT server instance: GetConfiguration timed out after 300000ms MSSQL: backup of database `XXX` on DEFAULT server instance failed 

Check debug-level job message log for:

MSSQL: information handler (instance name , db name XXX): BACKUP LOG cannot be performed because there is no current database backup. 
MSSQL: information handler (instance name , db name XXX): BACKUP LOG is terminating abnormally. 

To set a client debug level, right-click client from Clients › Summary subtab and select Advanced › Set Debug Level. Enter a larger value (such as 100) in the Debug Level text box, and then click OK. Run offending job again with new debug level to get extra log messages.

Check Microsoft SQL Server error log for:

Error: 3041, Severity: 16, State: 1. BACKUP failed to complete the command BACKUP LOG Connect. Check the backup application log for detailed messages. 

BACKUP LOG WITH TRUNCATE_ONLY or WITH NO_LOG is deprecated. The simple recovery model should be used to automatically truncate the transaction log. 

If you see this message in the error log, take note of the time(s) it has occurred. The time(s) will be useful when reviewing maintenance tasks later.

To find the ERRORLOG file, look in the LOG folder of a particular SQL Server instance. This is usually something like C:Program FilesMicrosoft SQL ServerMSSQL.xMSSQLLOG where MSSQL.x is the instance ID for the SQL Server instance.

Check Windows Application Event Log on client for:

Error 12/15/2009 5:50:04 AM MSSQL$INSTANCENAME Backup 3221228513 NT AUTHORITYSYSTEM BACKUP failed to complete the command BACKUP LOG XXX. Check the backup application log for detailed messages. 

NEED TO ADD ENTRY SHOWING "BACKUP LOG WITH TRUNCATE_ONLY or WITH NO_LOG" MESSAGE!!! 

Description

The requested backup cannot be completed by the SQL Server because the log chain has been broken. This is likely caused by a conflict from another tool or service, including maintenance tasks scheduled through the SQL Server Management Studio and the associated Maintenance Wizard. NTBackup can also cause this issue if it is enabled and doing a system state backup (we have an upcoming NTBackup plug-in scheduled for 3.4 that will negate this problem).

Steps to resolve

Remove conflicts that are causing the database log chain to become broken. Usually, the maintenance tasks scheduled for the server being backed up should be reviewed and removed (or inactivated) if determined to be at fault. Follow the steps below to review or remove maintenance tasks using SQL Server Management Studio:

Open SQL Server Management Studio on server having the backup problems.
Locate the instance containing the “problem” database(s) inside the Object Explorer pane (usually along the left side of the application window).
Find the SQL Server Agent folder for the located instance.
Make sure the Object Explorer Details tab is open in the main display area and click the Jobs folder under SQL Server Agent.
Look for offending jobs such as Truncate XXX Transaction Logs. This particular Maintenance Task causes log incremental backups to fail.
Be particularly suspicious of jobs that are scheduled to run at the same time as the messages in the SQL error logs whose time you noted from the steps above.
Disable or delete offending jobs. You can disable a job by editing the job properties.
Immediately run a full backup on the database(s), for which offending jobs were disabled or deleted.
This will prevent loss of transaction data from the lack of successful log/incremental backups.

If this is NTBACKUP-related, the only way to stop NTBACKUP from doing this is to disable to VSS writer responsible. The user will have to disable the MSDE Writer or the SQL Server Writer (their OS/SQL version determines, which one they will have).

Error. Memory allocation failed. Out of memory

Error message

VixEOUTOFMEMORY: Memory allocation failed. Out of memory.

Error description

Failed VMware VM backup due to NBD limits

Beside limitations on connections count to VC/ESXi, ESXi itself also limited by a transfer buffer for all NFC connections. This limitation is enforced by the host and cannot by bypassed or to be known in advance. The sum of all NFC connection buffers to an ESXi host cannot exceed 32 MB, and by default it is configured as 16 MB.

The primary physical appliance uses NBD protocol to backup VM disks. NBD, in turn, employs the VMware network file copy (NFC) protocol and thus is a subject of aforementioned limitations.

This happened because the ESXi server could not serve the request due to lack of enough resources (NFC connection buffer).

At the time there were N failed jobs with VixEOUTOFMEMORY error you had N+1 or more simultaneously running FULL backups. All of them were backing up VMs located on a single ESXi host. We use up to 10 MB buffer to transfer data. So there is a probability of facing with NFC buffer limitation on ESXi host. Which, as you saw, happened to occur. It does not mean the probability is always 100% with parallel backup jobs. It very depends on a lot of factors.

Steps to resolve

Nothing if there are no other failed backups with VixEOUTOFMEMORY error on your appliance on consequent backups.

You can also optimize the ESXi network (NBD) performance by increasing the NFC buffer size from 16 MB to 32 MB and reducing the cache flush interval.

Do it on all of your ESXi hosts. You can query current values using the following commands (from ESXi host): esxcfg-advcfg -get /BufferCache/MaxCapacity and esxcfg-advcfg -get /BufferCache/FlushInterval. It will not guarantee the VixEOUTOFMEMORY error never happen again but will decrease its probability. And it seems to be a good idea in general since you perform a lot of simultaneous backups.

Consider upgrading your network to 10 GbE. That should cover every network link in the chain between the appliance and the VMware host.

Error. The method is disabled

Error message

Message logs in the Management Console of the Backup & Disaster Recovery appliance shows the following error:

The method is disabled

Error description

This error may show when you try to migrate a VMware VM from one host to another.

This is a known issue in VMware. Sometimes after a backup runs, VMware will not release the migration lock even though it was asked to do so. This can happen if the backup fails, but will also sometimes happen without failed backups.

Steps to resolve

To resolve this issue, you have to manually remove the lock.

Error. Failure getting log and patch files

Error message

FileIO JobId 1422: ESEBackup: failure getting log and patch files: c8000232 
Some log or patch files are missing.Error reading data header from FD. ERR=Connection reset by peer 

c8000230: The database missed a previous full backup before the incremental backup 

c800020e: An incremental backup cannot be performed when circular logging is enabled 