Libraries needed to install ArcSight SmartConnectors on RedHat Enterprise Linux and CentOS

[Update 2016/03/11]:

Starting with SmartConnector 7.1.7 (I think, might be a rev or two earlier), there are a couple more libraries that are needed to successfully install the SmartConnector on Linux. Include libXrender.i686 libXrender.x86_64 libgcc.i686 libgcc.x86_64
yum install libXrender.i686 libXrender.x86_64 libgcc.i686 libgcc.x86_64

[Update 2014/02/04]:
Simpler syntax for the install, using yum to do the automatic dependency processing, and .. a update for CentOS 6.4 64-bit. I believe RHEL 6.4 64-bit would also need these libraries. This worked for installing ArcSight SmartConnector 6.0.7 on CentOS 6.4 64-bit.

glibc.i686
libX11.i686
libXext.i686
libXi.i686
libXtst.i686

You could install like:
yum install glibc.i686 libX11.i686 libXext.i686 libXi.i686 libXtst.i686

[Original post]
While installing an ArcSight SmartConnector 6.0.2 on RedHat Enterprise Linux 6.2 64-bit, the initial install runs successfully, however the connector configuration never kicks off, then install just claims it is done. runagentsetup.sh fails with Error occurred during initialization of VM .. java/lang/NoClassDefFoundError: java/lang/Object .. obviously a pretty major Java error.

Turns out there are some additional libraries that need to be loaded in addition to what is listed in the documentation.

Some research leads me to believe there were some base libraries that may be missing from the vanilla RHEL 6.2 64 bit install. Basic Server + Desktop configuration was selected and all libraries referenced in the ESM 6.0c Install Guide and SmartConnector User Guide were installed. Tracing through all the dependencies created this exact list of of libraries that are required to be installed on RHEL 6.2 64 bit:

glibc-2.12-1.47.el6.i686.rpm
glibc-2.12-1.47.el6.x86_64.rpm
glibc-common-2.12-1.47.el6.x86_64.rpm
libX11-1.3-2.el6.i686.rpm
libX11-1.3-2.el6.x86_64.rpm
libX11-common-1.3-2.el6.noarch.rpm
libXau-1.0.5-1.el6.i686.rpm
libXau-1.0.5-1.el6.x86_64.rpm
libxcb-1.5-1.el6.i686.rpm
libxcb-1.5-1.el6.x86_64.rpm
libXext-1.1-3.el6.i686.rpm
libXext-1.1-3.el6.x86_64.rpm
libXi-1.3-3.el6.i686.rpm
libXi-1.3-3.el6.x86_64.rpm
libXtst-1.0.99.2-3.el6.i686.rpm
libXtst-1.0.99.2-3.el6.x86_64.rpm
nss-softokn-freebl-3.12.9-11.el6.i686.rpm
nss-softokn-freebl-3.12.9-11.el6.x86_64.rpm

Note the specific X libraries versus the generic list as shown in the connector user guide. What was interesting about these is that they did NOT all install when doing a wildcard rpm install, and additionally did not report any failures. After some trial and error, on my system it appears the 32 bit X libraries needed to be installed individually for some reason. You may want to use rpm -q -a to verify each of the libraries successfully installed. Once all the above libraries were installed, the connector installation worked as expected.

A tarball with the libraries can be downloaded from here.

Extract the libraries, change into the resulting directory, then you can use the following brute force syntax to determine which libraries are not installed and install them:

rpm -ivh `ls | while read rpmfile; do rpm -q \`basename $rpmfile .rpm\`; done | egrep 'not installed' | awk '{print $2}' | xargs`

How to replay syslog events using the performance testing feature of ArcSight SmartConnectors

Aside

[Updated 2016/08/22]

For testing ArcSight SmartConnector settings or Logger and Enterprise Security Manager (ESM) content, it is quite useful to be able to replay previously captured syslog events.  The built in PerfTestSyslog class in ArcSight SmartConnectors make this easy.

There are several ways to capture syslog traffic into a text file for use in replay scenarios. Below are some methods that I have used – may not be the most elegant, but gets the job done.

Run a packet capture of syslog traffic

On the node that has inbound syslog traffic, run a packet capture using tcpdump:syslog-simulator

tcpdump -nn -i eth0 -s0 -w syslog-traffic.pcap port 514

where eth0 is the network interface receiving the syslog traffic, syslog-traffic.pcap is the resulting pcap format output file of captured events and 514 is the port that syslog traffic is expected to be received.

After capturing a suitable size of events, import the pcap file into Wireshark, click on one of the syslog packets, right click and select Follow UDP stream. A decoded content window will appear where you can select Save As .. and dump it to a sample events file. Ensure to select ASCII versus Raw format. This will be your event input file to feed the PerfTestSyslog function of the ArcSight SmartConnector.

Replaying the syslog events using an ArcSight SmartConnector is controlled via the GUI that is displayed when the PerfTestSyslog class is launched. In my example, I have a Test Connector installed on my current host (RedHat Enterprise Linux, however Windows, Solaris or AIX would work just as well) in the /opt/agents/syslog-udp-1514 directory. This connector is up and running listening on UDP 1514 for syslog messages, however we are also going to use it to feed the syslog event to the same connector. Just think of it in two separate unrelated processes, since you could just as easily use this to feed the syslog events to another host somewhere on the network.

cd /opt/agents/syslog-udp-1514/current/bin
./arcsight agent runjava com.arcsight.agent.loadable._PerfTestSyslog -H 127.0.0.1 -P 1514 -f ~arcsight/udp.txt -x 50

In this example, we are launching the connector framework (./arcsight) and telling the PerfTestSyslog class to read the ~arcsight/udp.txt file (our previously saved syslog events captured with tcpdump) and send them to Host 127.0.0.1 on Port 1514. The last argument is interesting – it configures a slider allowing the user to dynamically increase the Event Per Second (EPS) rate up to a maximum of (in our case) 50 EPS.

A sample capture file has events that look like:

<190>Jun 27 2012 12:16:53: %PIX-6-106015: Deny TCP (no connection) from 10.50.215.102/15603 to 2.3.4.5/80 flags FIN ACK  on interface outside

You can also eliminate the original timestamp if you chose:

%PIX-6-106015: Deny TCP (no connection) from 10.50.215.102/15605 to 204.110.227.10/80 flags FIN ACK  on interface outside

The PerfTestSyslog class has a number of pretty useful options, including -m to randomize the Device Address. This is really good for faking events from multiple firewalls.

Various configuration options exist on both the receiving SmartConnector (that is listening on UDP 1514) and the transmitting program, including the ability to keep the original timestamp intact or replace it with current time. This is especially useful for testing new content or performing historical analysis on previously saved event data where the original timestamp is needed.

Update:

For situations where you would like to run this without a GUI, you can add the -n option to start with No GUI.  In that case, although the rate is no longer dynamic, you do need to specify a starting event rate otherwise it appears the default is 0 .. eg. no events will be sent.  Instead of only specifying -x for max rate, also specify the starting rate with -r

./arcsight agent runjava com.arcsight.agent.loadable._PerfTestSyslog -H 127.0.0.1 -P 1514 -f ~arcsight/udp.txt -n -r 50 -x 50

See also: Common ArcSight Command Line Operations