BlockSync Project

Welcome to the BlockSync Project

This project aims to provide an efficient way to provide mutual protection from deemed bad actors that attack Internet facing servers. The result will be an open source set of communication tools that use established protocols for high speed and light weight transmission of attacker information to a variable number of targets (unicasting to a possibly large number of hosts).


There are many open source firewall technologies in widespread use, most based on either packet filter (pf) or netfilter (iptables). There is much technology that provides network clustering (for example, OpenBSD’s CARP and pfsync; netfilter; corosync and pacemaker), however it’s difficult for disparate (loosely coupled) servers to communicate the identity of attackers in real time to a trusted community of (tightly coupled) peers. Servers or firewalls that use state-table replication techniques, such as pfsync or netfilter, have a (near) real-time view of pass/block decisions other members have made. There needs to be a mechanism for loosely coupled servers to share block decisions in a similar fashion.

Our goal is to create an open source tool for those of us that have multiple Internet facing servers to crowd source information that will block attackers via the firewall technology of choice (OpenBSD/FreeBSD pf/pfSense, iptables, others).

Project Page

All project files are still private yet, but when we publish to GitHub or SourceForge, this section will be updated.


We have published a GoFundMe page to acquire more lab equipment here at

Enabling Single Line Logging from pfSense Firewalls to ArcSight

While pfSense firewall offerings are based on the BSD packet filter (pf) functions and offer excellent performance and value, the current implementation my customers are running (2.1.5) outputs firewall rule logs in two syslog lines.  The skilled developers that maintain pfSense have indicated that in the 2.2 release they will likely move to a single line log format that is friendly to machine parsers, however I needed to provide some ArcSight parsing of this log data immediately.

Two approaches are possible here.  I could write a FlexConnector parser that expects and parses two consecutive syslog lines into one in order to pull out the relevant source, destination addresses, ports and device handling information (pass, block, reject, interface, protocol, flags, etc).  The other option I discovered was what I implemented since my customer was willing to allow a system patch to be applied to the firewalls that outputs log entries on a single line.  This will make a FlexConnector parser a breeze to create.

After looking through the pfSense support forums, I found this post by jimp (one of the developers) that provided a patch to enable single line logging.  To install the patch, you need to install the System Patch installation package on the firewall, since it’s not there by default.


Package install on pfSense Firewall


Enables installation of patches

Enables installation of patches



From the Package Manager, install the System Patch package.  This enables downloading and automating the patch installation process.  This is a safe way to apply patches, since it will validate the patch can be applied and more importantly backed out if needed.

Select the Patches option in the System menu to start the patch application process. At this point you can (and probably should) just enter the URL to the patch versus manually entering the patch diff file in the text dialog box that comes up. Entering the URL ( gives the ability to Fetch the contents of the patch (double click on the Fetch option).  pfSense-patchThe System Patch package will go download the patch at the given URL and validate it.  If the validation is successful (it can apply and also roll back the patch), then the Test and Apply links should automatically show up.  If not, only the Test link will show up .. click on the Test link to validate the patch can be applied and backed out.  This should happen automatically so the Apply link shows up.  After application of the patch there will be apfSense-rawLogFormatOption new option that shows up in the Status > System Logs > Settings pane that allows the raw logs to be written out on a single line.pfSense-block

Now the logs are coming in on a single line, creation of an ArcSight FlexConnector is simple.  I first directed logs from the firewall to a standard syslog daemon SmartConnector (7.0.5) and enabled Preserve Raw so I can grab pfSense-raw-logthe original event and write the parser to grab all the relevant fields I’m looking to correlate.

Using the ArcSight FlexConnector tool kit I was able to get it to suggest a starting point for the regex patterns to parse the single line logs.  Launched the regex GUI with arcsight regex in any connector’s current/bin directory. I copied a FlexConnector config file from an iptables parser I wrote to use to start from .. and produced a quick and dirty mapping of the pfSense events with this parser in user/agent/flexagent/syslog/  Note this is a work in progress, so no categorization map files have been provided yet.  That will come soon.


Outbound network traffic with multiple interfaces

Why does Red Hat Enterprise Linux 6 invalidate / discard packets when the route for outbound traffic differs from the route of incoming traffic?

Issue Description
Why does Red Hat Enterprise Linux 6 invalidate / discard packets when the route for outbound traffic differs from the route of incoming traffic?
Why does Red Hat Enterprise Linux 6 differ from Red Hat Enterprise Linux 5 in handling asymmetrically routed packets?

Solution posted at

Red Hat Enterprise Linux (RHEL) 6 Resolution

Temporary change
To accept asymmetrically routed (outgoing routes and incoming routes are different) packets set “rp_filter” to 2 and restart networking, by running the following commands:

echo 2 > /proc/sys/net/ipv4/conf/default/rp_filter
echo 2 > /proc/sys/net/ipv4/conf/all/rp_filter

Persistent change
To make this behaviour persistent across reboots, modify /etc/sysctl.conf and make the following change prior to reboot:

net.ipv4.conf.default.rp_filter = 2

Root Cause

RHEL6 (unlike RHEL5) defaults to using ‘Strict’ Reverse Path Forwarding (RPF) filtering.

The sysctl net.ipv4.conf.default.rp_filter selects the default RPF filtering setting for IPv4 networking. (It can be overridden per network interface through net.ipv4.interfacename.rp_filter).

Both RHEL6 and RHEL5 ship with a default /etc/sysctl.conf that sets this sysctl to 1, but the meaning of this value is different between the RHEL6 and the RHEL5 kernel.

How to reset the enable password on a Cisco ASA 5505

How to reset the enable password on an ASA 5505:

The following procedure worked for me to reset the enable password.

Connect to serial port – typically 9600,8,N,1.  On my MacBook Pro, I use a Keyspan USB-Serial adapter, so my command line is:

screen /dev/tty.USA19Hfd13P1.1 9600,8

You can eventually use <ctrl-A><ctrl-\> to kill the screen session.

Power on the device.
When it prompts to interrupt boot sequence, do so (press ESC).

It should prompt

rommon #0>

Type in:
rommon #0> confreg

Should show something like:

Current Configuration Register: 0×00000001
Configuration Summary:
boot default image from Flash

Do you wish to change this configuration? y/n [n]:

Press n (don’t change)

We can have the ASA boot a default config with no password by setting register flags 0×41, so do this:

rommon #2> confreg 0×41
rommon #2> reboot

You now can login as the password has been removed (use <return> as the password).  Be sure to set the enable password with:

config t
enable password new-password-here
config-register 0x1

Ensure you either use the config-register command or interrupt the boot sequence again and reset the boot flags back to 0x1, otherwise the boot loader will continue to boot the default configuration – ignoring your configuration.


How to secure your home PC

Whether you have a Mac or a Windows PC, there are some basic steps you can take to reduce the risk and personal impact of a malware infection.  This advise is especially impactful when you have just purchased a new Mac or Windows system. There are several steps that you can take to protect your new investment and more importantly your information. In the following detail, I mainly focus on Windows as that’s the main technology that my non-IT type friends ask about.

Basically what you should be doing is:

  1. Ensure that a hardware firewall/router is in between the internet and the PC (I’ll just call it a firewall from now on)
    • Use a recognized brand name like Linksys, avoid the no-name generics as they often have bad defaults and don’t implement the stateful-packet-inspection that you want to filter out most of the cruft on the Internet from reaching your PC
  2. Ensure all default passwords on the firewall and PC have been changed
    • When you initially turn on the power to your PC and to your firewall, do NOT have them connected to your cable or DSL modem initially.  Do the setup of your firewall and PC first in order to ensure malware doesn’t have a chance to get at your shiny new PC before you’ve turned on the needed protection
    • Point a browser to your firewall (likely or and change the default administrator password.  This is very important, as some malware will seek out your firewall and try to use the manufacturer default password to change things like your DNS server settings – inserting the bad guys in between you and the rest of the Internet (eg. forcing your traffic to them first before it goes to your bank)
  3. All normal accounts used for day-to-day business on the computer should NOT have administrator privilege (see my post on running without admin privileges)
    • On Windows XP, Vista (and I think 7), the default “user” that accesses the PC has full administrative privilege, that enables software  installation and configuration changes.  This is very dangerous, as malware that you come in contact with from infected emails or websites use this privilege to install their spyware, keyloggers, backdoors and other nasty stuff on your PC – without your explicit permission
    • Set a password for your Administrator account
    • Create a new user right away, before you setup your email, music, photos, documents, etc; ensure that new user is NOT a Computer Administrator
    • Always login with this non-Administrator username for your day-to-day use; only use the Computer Administrator username for software installation and configuration changes.
  4. Never surf the Internet with an account that has administrative privilege
  5. If this is a common PC for a business, ensure employees accounts are individually assigned (if practical). Ensure those employee accounts are not administrators (unless there is a need and a high degree of trust)
  6. Run a good commercial anti-virus program with annual software support (or a subscription)
    • There are some good free AV packages (AVG, Clamwin, Avast) .. Google them for the links
    • Sophos makes a good Mac AV package .. yes, Macs are vulnerable to malware as well; it’s just not as prevalent
  7. Finally ensure regular (daily) backups are being run to protect your business, financial, customer information from loss if there is a problem with the PC
  8. For setup of your wireless access point (if you have one .. sometimes it’s built into the router/firewall)
    • Chose wireless encryption of at least WPA or WPA2 .. never use WEP or no encryption
    • There is no significant increase in security by obscuring your network name (SSID)
    • Don’t use any personally identifiable information in your network name

If you are unsure of how to do any of these steps, get one of your computer knowledgeable friends to help you.  Of course if you are purchasing a new system right now, I’d strongly recommend you check out Apple’s Mac products.  They’re not immune to malware, but the architecture and core are by design much less vulnerable to the types of malware that plague Windows.

Sifting through Checkpoint FW1 logs

Recently I found myself in the unhappy position of needing to sift through slightly more than a billion Checkpoint Firewall-1 log lines, looking for specific patterns of access. The problem was that many of the exported fwm log files had differing column positions and there had been many ruleset changes over the course of 11 months worth of log data. Many of the excellent FW1 log summarization tools (such as Peter Sundstrom’s fwlogsum) didn’t handle the hundreds of files and differing column positions.

The final scripted solution was processing over 11,000 lines/second .. and still took over 23 hours for the first run.

Log file exports via fwm logexport can have variable column positioning, except for record ID number “num”, which is *always* column number one.  I see three viable alternatives to the changing column position in the ASCII log files exported via fwm – so we can automate the log processing:

  • Export the FW1 log file to ASCII via
    fwm logexport -i fw1-binary-logfile -o fw1-ascii-logfile.txt -n -p
    1. Parse the header line (line #1) of every log file and dynamically map (rearrange) the columns to a pre-determined standard in memory before further processing (painful, expensive)
    2. Tell Checkpoint fwm to export in a fixed column ordering
        and place in
        $FWDIR/conf directory
        eg. fwmgmtsrv:
        included_fields = num,date,time,orig,origin_id,type,action,alert,i/f_name,
        dstname,SOAP Method,category,ICMP,message_info,
        TCP flags,rpc_prog,Total logs,
        Suppressed logs,DCE-RPC Interface UUID,Packet info,
    3. Use OPSEC LEA tools to extract event log records instead of export via fwm logexport

    Once the ASCII log files are available for processing, my script can be used to find complex patterns of interest.  Any matching records found by fw1logsearch will be output with an initial FW1 header line so that fw1logsearch can be used iteratively, to build very complex search criteria.  fw1logsearch can also write out a discard file allowing completely negative logic searches resulting in 100% of the input data separated into a match file and a didn’t match file.  Some examples of how I’ve used it are shown here:

    gunzip -c fwlogs/2009*gz | \ --allinclude \
    -S '10\.1\.1[1359]\.|10\.2\.1[01]\.|192\.168\.2[245]\.' \
    -d '10\.1\.1[1359]\.|10\.2\.1[01]\.|192\.168\.2[245]\.' \
    -p '^1310$|^1411$|^1812$|^455' | \ -S '192\.168\.22\.14$|10\.2\.11\.12$' |\ --allexclude \
    -S '^192\.168\.24\.12$' -P '^1310$' --rejectfile 192-168-24-12-port-1310.txt

    Line by line:
    1. Unzip the compressed ASCII log files, feed them to the first instance of
    2. First fw1logsearch – all conditions must be true for any events to match
    Source address must NOT be in any of the following regex ranges:
    10.1.11.* 10.1.13.* 10.1.15.* 10.1.19.*
    10.2.10.* 10.2.11.*
    192.168.22.* 192.168.24.* 192.168.25.*
    Destination address must be in one of the same following regex ranges.
    Service (destination port) must be one of:
    Exactly port: 1310, 1411, 1812, or any port starting with 455
    No protocol is specified, so it will match either TCP or UDP will output any matching events to stdout, including a FW1 log header line, so the next instance of continues filtering the result set.

    3. The second specifies Source Address must not be any of the following

    4. The last excludes port 1310 from, and puts all those records into a separate reject file, while writing the other records to stdout.

    This script has been used to process over 4 billion records within the project I wrote it for – and precisely found all the use of particular business cases I needed to modify.  The result was zero outages and no unintended business interruption.

    Basic syntax/help file:

    [-a|–incaction|-A|–excaction <action regex>]
    [-p|–incservice|-P|–excservice <dst port regex>]
    [-b|–incs_port|-B|–excs_port <src port regex>]
    [-s|–incsrc|-S|–excsrc <src regex>]
    [-d|–incdst|-D|–excdst <dst regex>]
    [-o|–incorig|-O|–excorig <fw regex>]
    [-r|–incrule|-R|–excrule <rule-number regex>]
    [-t|–incproto|-T|–excproto <proto regex>]

    [–dnscache <dns-cache-file>]
    [–rejectfile <file>]
    [–debug <level>] will search a fwm logexport text file for regex patterns specified for supported columns (such as service, src, dst, rule, action, proto and orig).

    Include and exclude regex matches may be specified on the same line, although they both will include (print) a line or exclude (reject) a line based on single matches.  Allinclude or Allexclude must be specified to force a match
    only on all specified column regex patterns.

    Regex patterns can be enclosed with single quotes to include characters that are special to the shell, such as the ‘or’ (|) operator.

    Header will be output only if there are any matching lines.

    Example invocations:
    $ cat 2008-07-07*txt | \ \
    -p ’53|domain’ \
    -d ‘|host1||host2’ \
    -o ‘||’ \
    -S ‘||10.10.*|192.168.*’ \
    Will require destination port (service) to be 53, destination IP to be any of, host1,, or host2  the reporting firewall (origin) to be any of,, or  and the source IP must not be
    any of,, 10.10.*, or 192.168.*  Any lines that match this criteria, will display and the orig, src, and dst columns will use the default DNS cache file (dynamically built/managed) to perform name resolution, replacing the IP addresses where possible.

    Include regex patterns:
    -a  –incaction    Rule action (accept, deny)
    -b  –incs_port    Source port (s_port)
    -p  –incservice   Destination port (service)
    -s  –incsrc       Source IP|hostname
    -d  –incdst       Destination IP|hostname
    -o  –incorig      Reporting FW IP|hostname
    -r  –incrule      Rule number that triggered entry
    -t  –incproto     Protocol of connection

    Exclude regex patterns:
    -A  –excaction    Rule action (accept, deny)
    -B  –excs_port    Source port (s_port)
    -P  –excservice   Destination port (service)
    -S  –excsrc       Source IP|hostname
    -D  –excdst       Destination IP|hostname
    -O  –excorig      Reporting FW IP|hostname
    -R  –excrule      Rule number that triggered entry
    -T  –excproto     Protocol of connection

    Other options:
    –debug {level} Turn on debugging
    –dnscache      Specify location of DNS cache file to be used with
    the Resolve IPs option
    –resolveip     Resolve IPs for orig, src, and dst columns AFTER filtering
    –rejectfile    Write out all rejected lines to a specified file


    High availability firewalls with OpenBSD, pf and CARP

    One can now inexpensively build a fault tolerant firewall cluster that removes any single point of failure in the security policy enforcement points at your security zone boundaries. Synchronous firewall state table updates and an open source version of virtual router redundancy protocol (CARP) gives the ability to seamlessly insert or remove firewalls from a cluster. No more patching firewalls at 2am hoping for the best (or not patching because it’s too hard).


    Soekris net5501 SBC Linux installation

    Soekris Engineering net5501 SBC setup with Linux


    net5501 is a x86 SBC that I ordered with 4 10/100 ethernet ports, 512MB memory, 500MHz Geode LX CPU

    Serial console is used for setup of net5501 – BIOS writes to serial port since there is no xVGA port. <ctrl-p> to enter BIOS setup. DB9 pinout:

    2 — 3

    3 — 2

    5 — 5

    Use 19,200 bps 8 data bits, no parity, 1 stop

    With the Macbook Pro, I use a Keyspan USA-19HS USB <–> DB9 RS232 serial converter (and DB9-RJ45 adapters to implement the null modem configuration and allow me to use an ethernet cable for the serial console <–> Keyspan device.

    On OS X (10.5) I use “screen” to provide the serial terminal interface:

    $ screen /dev/tty.USA19H1a2P1.1 19200,8

    <ctrl-a><ctrl-\> to exit

    On the net5501 BIOS, PXEBoot is disabled:

    set PXEBoot=Disabled

    I setup voyage-0.5.0 on a compact flash card then installed the card into the net5501 – works great the first boot

    Default root info: root / voyage

    OpenBSD setup info:

    Linux iptables notes

    Add local redirection of low port to unpriv high port

    Remove any existing entries:

    iptables -t nat -D PREROUTING –src 0/0 -p tcp –dport 25 -j REDIRECT –to-ports 11025 2> /dev/null
    iptables -t nat -D PREROUTING –src 0/0 -p tcp –dport 80 -j REDIRECT –to-ports 8080 2> /dev/null

    Add new redirects:
    iptables -t nat -I PREROUTING –src 0/0 -p tcp –dport 25 -j REDIRECT –to-ports 11025
    iptables -t nat -I PREROUTING –src 0/0 -p tcp –dport 80 -j REDIRECT –to-ports 8080