Installation notes for Logger 6 on CentOS

[Update 2016/04/15]:  Installing Logger 6.2 on CentOS 7.1

CentOS (or RHEL) 7 changed a number of things in the OS for command and control, such as the facility to control services – for example, rather than “service” the command is now “systemctl”.  Below I outline a “quickstart” way to get HPE ArcSight Logger 6.2 installed on CentOS 7.1 (minimal distribution). Of course you want to read the Logger Installation Guide, Chapter 3 “Installing Software Logger on Linux” for the complete instructions and be sure you understand the commands I suggest below before you run them. No warranties here, just suggestions.  😉

  1. Do a base install of CentOS (or RHEL) 7.1, minimal packages.  I often suggest adding in Compatibility Libraries, however for this Logger 6.2 install, I just used the base install.  Ensure /tmp has at least 5GB of free space and /opt/arcsight has at least 50GB of usable space – I’d suggest going with at least:
    • /boot – 500MB
    • / – 8GB+
    • swap – 6GB+
    • /opt – 85GB+
  2. Ensure some needed (and helpful) utilities are installed, since the minimal distribution does not include these and unfortunately the Logger install script just assumes they are there .. if they aren’t, the install will eventually fail (such as no unzip binary).
    • yum install -y bind-utils pciutils tzdata zip unzip
    • Unlike my ESM install, for Logger, I left SELinux enabled and things appear to be working alright, but your mileage may vary.  If in doubt, disable it and try again.  To disable, edit /etc/selinux/config and set the mode to “disable” (or at least to “permissive”)
    • Disable the netfilter firewall (again, at some point I’ll update this with the rules needed to leave netfilter enabled).
    • systemctl disable firewalld; systemctl mask firewalld
    • Install and configure NTP
    • yum install -y ntpdate ntp
    • (optionally edit /etc/ntp.conf to select the NTP servers you want your new Logger system to use)
    • systemctl enable ntpd; systemctl start ntpd
    • Edit /etc/rsyslog.conf and enable forwarding of syslog events to your friendly neighborhood syslog SmartConnector (optional, but otherwise how do you monitor your Logger installation?) .. you can typically just uncomment the log handling statements at the bottom of the file and fill in your syslog SmartConnector hostname or IP address. Note the forward statement I use only has a single at sign – indicating UDP versus TCP designated by two at signs:
    • $ActionQueueFileName fwdRule1 # unique name prefix for spool files
      $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
      $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
      $ActionQueueType LinkedList # run asynchronously
      $ActionResumeRetryCount -1 # infinite retries if host is down
      # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
      #*.* @@remote-host:514
      *.* @10.10.10.5:514
    • Restart rsyslog after updating the conf file
    • systemctl restart rsyslog
    • Optionally add some packages that support trouble shooting or other non-Logger functions you run on the Logger server, such as system monitoring
    • yum install -y mailx tcpdump
  3. Update the maximum number of processes and open files our Logger software can use:
    Backup the current settings:
    cp /etc/security/limits.d/20-nproc.conf /etc/security/limits.d/20-nproc.conf.orig
    Drop in new config file (assuming you have copy/pasted the following settings into /root/20-nproc.conf):
    cp 20-nproc.conf /etc/security/limits.d/20-nproc.confContents of the /etc/security/limits.d/20-nproc.conf file becomes:
    # Default limit for number of user's processes to prevent
    # accidental fork bombs.
    # See rhbz #432903 for reasoning.
    * soft nproc 10240
    * hard nproc 10240
    * soft nofile 65536
    * hard nofile 65536
    root soft nproc unlimited

    Reboot to enable the new settings.
  4. Add an unprivileged user “arcsight” to own the application and run as:
    groupadd -g 1000 arcsight
    useradd -u 1000 -g 1000 -d /home/arcsight -m -c "ArcSight" arcsight
    passwd arcsight
  5. Ensure the *parent* directory for the Logger software exists. Standard locations for installation of ArcSight products should be /opt/arcsight, so for example, we’re going to install our Logger software at /opt/arcsight/logger.
    cd /opt
    mkdir /opt/arcsight
  6. Run the Logger installation binary as “root” user
    • ./ArcSight-logger-6.2.0.7633.0.bin
  7. After the installation script completes successfully, you should be able to login to the console via a web browser https://<hostname>
    Default username “admin” with default password “password”. You’ll be forced to change the admin password on login.
  8. If you are going to install any SmartConnectors on the system hosting your Logger, check out my post regarding required libraries for CentOS and RedHat, before you try to run the Linux SmartConnector install. This includes any Model Import Connectors (MIC) or forwarding connectors (SuperConnectors).

 

[Update 2016/03/11]: Starting with SmartConnector 7.1.7 (I think, might be a rev or two earlier), there are a couple more libraries that are needed to successfully install the SmartConnector on Linux. Include libXrender.i686 libXrender.x86_64 libgcc.i686 libgcc.x86_64
yum install libXrender.i686 libXrender.x86_64 libgcc.i686 libgcc.x86_64

These notes describe an installation of HP ArcSight Logger 6.0.1 on a CentOS 6.5 virtual machine.

For a test install of Logger 6, I built a CentOS vm with the following parameters:
Basic install from the CentOS 6.5 Minimum ISO
1 CPU with 2 cores
4GB memory
80GB virtual disk
1 bridged network adapter
Disk partition sizes:
root fs 6GB, swap 4GB, /home 2GB, /opt/arcsight 50GB, /archive 10GB, free space approximately 15GB

As soon as the system was up, I commented out the archive filesystem (will be re-mounted under the /opt/arcsight/logger directory)
vi /etc/fstab

Installed the bind-utils package so I could use dig and friends, then did a full yum update:
yum install bind-utils ntp
yum update

This turns the system into CentOS 6.6, but that’s still a supported system for Logger, so all’s good.

Next we prepare the system for Logger software install by adding a user and changing some of the system configuration.

Add a non-root user to own and run the Logger application:
groupadd -g 1000 arcsight
useradd -u 1000 -g 1000 -d /home/arcsight -m -c "ArcSight" arcsight
passwd arcsight

Install libraries that Logger depends on:
yum install glibc.i686 libX11.i686 libXext.i686 libXi.i686 libXtst.i686
yum install zip unzip

Update the maximum number of processes and open files our Logger processes can have:
cp 90-nproc.conf /etc/security/limits.d/90-nproc.conf

Contents of the /etc/security/limits.d/90-nproc.conf file becomes:
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
*          soft    nproc     10240
*          hard    nproc     10240
*          soft    nofile    65536
*          hard    nofile    65536
root       soft    nproc     unlimited

Turn off services we don’t need and turn on the ones we do need. Later we will write some iptables rules so we can turn the firewall back on when we’re done.

chkconfig iptables off
service iptables stop
chkconfig iscsi off
service iscsi stop
chkconfig iscsid off
service iscsid stop
ntpdate name-of-ntp-server-you-trust
chkconfig ntpd on
service ntpd start

All of these steps are packaged up here in centos-setup.shl:
groupadd -g 1000 arcsight
useradd -u 1000 -g 1000 -d /home/arcsight -m -c "ArcSight" arcsight
passwd arcsight
cp 90-nproc.conf /etc/security/limits.d/90-nproc.conf
yum install glibc.i686 libX11.i686 libXext.i686 libXi.i686 libXtst.i686
yum install zip unzip
chkconfig iptables off
service iptables stop
chkconfig iscsi off
service iscsi stop
chkconfig iscsid off
service iscsid stop
ntpdate 0.centos.pool.ntp.org
chkconfig ntpd on
service ntpd start

Turns out since we need 3+GB of free space in /tmp, I needed to extend the root filesystem .. I only allocated 2GB to begin with. Extend the root logical volume (lv_root) by adding 1,000 Physical Extents (4MB each):

Boot into rescue mode .. do NOT mount linux partitions, then drop to a shell

vgs
vgchange -a y vg_swlogger1
lvextend -l +1000 /dev/vg_swlogger1/lv_root
e2fsck -f /dev/vg_swlogger1/lv_root
resize2fs /dev/vg_swlogger1/lv_root

Now reboot and confirm there is at least 4GB of free space in /tmp. Could also have mounted a ram filesystem, but this will do as I’m conserving my memory on the host.

Upload the Logger installer binary and also the license file to the system into root’s home directory (or where you have space).

As root, run the Logger software install:
chmod u+x ArcSight-logger-6.0.0.7307.1.bin
./ArcSight-logger-6.0.0.7307.1.bin

Word of advice .. if doing this in a vm, run the install from the vm console since it’s possible the vm will be busy enough a remote ssh session could get disconnected – and the install will not complete properly.

After the install, we should be able to open a browser by navigating to https://name-of-vm-here

Sign in as arcsight / password then navigate to the System Administration section to change the admin password.

Building a web security lab (with VMware Fusion)

Problem: VMware machines load boot loader immediately, no BIOS banner, so can’t get into BIOS to alter boot settings.
Solution: Edit the vm’s .vmx file and add the line:

bios.bootDelay = "5000"

which adds a 5000 millisecond (5 second) delay to the boot, or add:

bios.forceSetupOnce = "TRUE"

to make the VM enter the BIOS setup at the next boot.

Problem: VMware Fusion 3.0 doesn’t give a way to edit the virtual network settings via the GUI.
Solution: To change the subnet used by the NAT or HostOnly networks, go root in Mac OS X and edit

/Library/Application Support/VMware Fusion/networking

and set the following lines to the subnets desired:

answer VNET_1_HOSTONLY_SUBNET 192.168.35.0
answer VNET_8_HOSTONLY_SUBNET 10.10.1.0

To add additional custom isolated host only VLANs, also edit the networking file and add additional VNET definitions. There can apparently only be 8 VLANs with VLAN 1 and 8 already pre-defined.

answer VNET_2_DHCP no
answer VNET_2_HOSTONLY_NETMASK 255.255.255.0
answer VNET_2_HOSTONLY_SUBNET 10.10.21.0
answer VNET_2_VIRTUAL_ADAPTER yes
answer VNET_3_DHCP no
answer VNET_3_HOSTONLY_NETMASK 255.255.255.0
answer VNET_3_HOSTONLY_SUBNET 10.10.22.0
answer VNET_3_VIRTUAL_ADAPTER yes
answer VNET_4_DHCP no
answer VNET_4_HOSTONLY_NETMASK 255.255.255.0
answer VNET_4_HOSTONLY_SUBNET 10.10.23.0
answer VNET_4_VIRTUAL_ADAPTER yes

Now create your vm with as many network interfaces as you have separate VLANs (vnet) then edit the node.vmx vm configuration file and change the interfacename.connectionType to custom, and define the VLAN (vnet) that interface will attach to:

#ethernet0.connectionType = "nat"
ethernet0.connectionType = "custom"
ethernet0.vnet = "vmnet3"

Also realize that VMware will take the .1 host address on each vmnet – so you cannot assign .1 to any of your VMs.

Problem: Ubuntu 9.10 persistent network configuration (stores the MAC address of network adapters), so if you copy a machine, by default Ubuntu will setup a new logical adapter (eth1) since the MAC address has changed (when you answer I Copied It in VMware).
Solution: Tell VMware you copied the machine, so it will chose a unique MAC address. Boot Ubuntu into single user mode (another article on that to follow) then edit the MAC address associated with eth0.

sudo vi /etc/udev/rules.d/70-persistent-net.rules

find the stanza of the network interface in question (NAME=”eth0″) and set the following ATTR tag to the new MAC address:

ATTR{address}=="new-mac-address-here"

MythTV FC7 LVM on RAID1 Configuration

MythTV PVR HDD Mirroring 2008/07/24
Host: n43 (mythtv)
– Two SATA 500GB drives sda sdb
– current production drive is sdb

Problem: I’ve done migrations of LVM2 volumes from 320GB SATA to 500GB SATA and added
a redundant 500GB SATA. Now I want to get software RAID 1 setup to protect the
root, swap and /storage filesystems from damage if/when one of the shiny new 500GB SATA
disks bite the dust.

Followed howtoforge.com linux_lvm_p1 (start of article) to free up sda from LVM
volume group VolGroup00 .. http://www.howtoforge.com/linux_lvm_p7

0. Did a file level backup to the fileserver:
[root@n59 20080724]# sshroot@192.168.1.2This e-mail address is being protected from spambots, you need JavaScript enabled to view it“tar cf – /lib” | dd of=mythtv-lib.tar
(repeat for /boot /storage /var /etc /home)

1. Free up sda2 LVM volume. I know this volume is not used anymore,
but it still has same-disk backup of /storage from when I was tweaking
MythTV.

[root@mythtv ~]# pvmove /dev/sda2
[root@mythtv ~]# vgreduce /dev/VolGroup00 /dev/sda2
[root@mythtv ~]# pvremove /dev/sda2

– now running on sdb only –

Setup RAID 1 mirroring (md)

2. Partition sda for mirroring (Auto RAID label)
[root@mythtv ~]# fdisk /dev/sda
<delete partitions>
<add primary 1 whole disk>
<set flag to fd – Auto RAID>

[root@mythtv ~]# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 19 152586 83 Linux
/dev/sda2 20 60801 488231415 fd Linux raid autodetect

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 19 152586 83 Linux
/dev/sdb2 20 60801 488231415 8e Linux LVM

Notice that sdb is still using only LVM, not RAID.

Continue reading