Ever go into a config file for some application in Linux looking for one small config setting to see what it is set at (or if it is set at all) only to be presented with a hundred lines of comments and config settings that you have to wade through? Here’s a command I run to read a config file and spit out only the un-commented and non-blank lines:
grep -v “#” filename.conf | grep -v “^$”
I just learned that the French company that develops Prelude SIEM just released a new version for the first time since I started following it. It’s been nearly two years since I started to use Prelude, and over 18 months since I deployed my first production system using the free version (1.0.0) and a couple of days ago they announced the release of 1.0.1!
Should be good. I won’t know, of course, until Steve Grubb releases new packages and updated instructions for Fedora based on the new version, as I prefer to take the path of least resistance and follow in his footsteps when working with Prelude. Starting from source is a bit more work than I normally have the time or patience for. 
I’m diving back into the world of IDS/IPS/SIEM and Prelude in particular. I urge anyone considering using it for network security monitoring and alerting to go with a Red Hat derivative, as Red Hat’s Steve Grubb has put together some great packages and instructions that just work.
In my previous build I used FC13. This time around I’m using FC17. The only trouble I’ve run into so far is Prelude’s included table-building SQL script, which contains deprecated syntax in it. I received errors trying to import the script into my new Prelude database on a fresh install of Mysql 5.5. After trying a few things, I ran the SQL on a MySQL 5.0 database, which worked. I then exported the empty tables from 5.0 and the resulting sql imported fine on 5.5. Apparently 5.0 still understood the deprecated stuff but was also aware of the newer terms and syntax. Just thought this might help someone else who, like me, knows how to work with MySQL but doesn’t do it often enough to figure these types of issues out immediately when they happen.
One other suggestion. In the file /etc/prelude/default/tls.conf, I recommend dropping the key length from 2048 to 1024. I know I will take heat for that, but here’s the thing: even at 1024 bits, when you get to the steps that require key generation it will take HOURS to generate the keys. I launched 5 or 6 entropy-generating processes (rngd -r /dev/random) and it still took several hours. I don’t have that kind of time (or patience), and 1024 bits is more than secure enough for my purposes. If you’re building something to use outside of a firewall and your employer is the NSA or something, maybe you’ll want to take the extra time to generate 2048 bit keys.
I’ve been using Prelude SIEM for a month now, and have fought off and on with the prelude-manager.conf file’s idmef-criteria and threshold settings to try to fine tune the smtp alerts a bit. I had to focus mainly on tuning at the sensor level (Snort, OSSEC) because I could not, even with user community and developer assistance, get idmef criteria and thresholds to work. This morning I had my first bit of success with it, and I wanted to report it here because I know others have come to my website searching for the same info.
I wasn’t able to filter/threshold based on alert name, but using the code below I was able to set up low severity alerts to go to the database without generating email alerts, while still getting email alerts (and database logging) for medium and high severity events, which is a great first step for me in tuning my SIEM to limit the number of benign email alerts I receive.
[idmef-criteria=severitymedhigh]
rule = alert.assessment.impact.severity != low
hook = smtp[default]
hook = db[default]
[idmef-criteria=severitylow]
rule = alert.assessment.impact.severity == low
hook = db[default]
It’s weird, it seems with no special idmef-criteria set, you get emails/database for everything, but as soon as you specify any criteria, only the criteria specified is acted upon and everything else is ignored completely. Remarkably, this behavior is not documented anywhere that I have found, nor even in the prelude users mailing list activity that I searched, either. (Well, it’s in the mailing list archives now, because I posted it there.)
For example, in the code above I have two idmef-criteria statements. I tried the first of the two alone first, thinking that everything but low severity events would continue being logged to the database and to smtp alerts as well. In fact, however, I stopped getting ALL smtp alerts, and only low severity events were being databased. That seems odd to me, but whatever. That one small bit of undocumented weird behavior was the sole reason I was having so many problems figuring this out. Every time I would try to filter out a particular type of even (or set a threshold to quiet it down to a dull roar) all database and smtp activity would appear to cease completely. Also, you HAVE to include the [default] instance references even if there are no other instances in the config file. Not if you have just one entry, but when you add a second one. Oh, and you can specify more than one “hook” for each criteria.
Finally discovered a way to play around with Chromium since I am reasonably sure that I’m not going to be beta-testing one of those netbooks. There’s a virtual disk image out there for download (via torrent) that you can load up in Oracle’s free Virtual Box.
There’s an interesting/slightly suspicious (due to the fact that this is from a torrent) boot screen. Black background with a dark graphic and red Asian characters at the top:
It apparently is not really a new OS but an interface to linux, according to my trick to peruse the underlying file system:
Another interesting find, the hosts file (/etc/hosts) contains a non-standard entry:
127.0.0.1 localhost
127.0.1.1 crashpad
#67.227.134.190 dev.aerva.com
192.168.1.132 navydemo.aerva.com
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
Anyway, I look forward to some more tinkering and snooping…
EDIT: Here’s more info on hacking and tinkering with Chromium.
http://www.chromium.org/poking-around-your-chrome-os-device
On Virtualbox, I can’t get the control-alt-=> trick to work, but F1 and F2 “work” in that it cycles between shell and browser modes. But I don’t get the described login prompt when I go into the shell, just a black screen and unresponsive blinking cursor.
I just looked back 15,000 hours in Prewikka, and it’s as I suspected…something recently (3 days ago) started triggering massive numbers of alerts on our internal Snort sensor.
The alert triggered is COMMUNITY BOT Internal IRC server detected, Sig ID 1:100000241.
There have been 3 instances when the alerts (in mass quantities each instance) were triggered. The first was when a user with a roaming Windows profile logged off the network. The source port was some random port not normally associated with IRC and the destination port was 445, common on Windows networks when file copying (or just about anything else) takes place. The “payload” data given in the alerts showed SMB packets and file copy operations typical of a roaming user profile being uploaded to the central share upon user logoff. Nothing that would indicate IRC activity at all.
The other two times that the alert was triggered was during nightly NAS backup jobs. And in both cases, every alert I checked in Prewikka showed not only the same type of SMB activity, but again with those roaming user profiles. This is a strange one. Looking in the rules file, the following is the specific line triggering the alert:
alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"COMMUNITY BOT Internal IRC server detected"; flow: to_server,established; flowbits:isset,community_is_proto_irc; classtype: policy-violation; sid:100000241; rev:2;)
The thing that strikes me as odd about the rule above is that for all received alerts, both the source and destination IP are part of HOME_NET. My gut is telling me these are all false alarms, but we’ve been running Snort for about 3 weeks now and this alarm just triggered for the first time 3 days ago. I thought perhaps the rule causing this alert was a newly-added rule, but no, it’s from 2006:
# $Id: community-bot.rules,v 1.5 2006/10/23 12:49:52 akirk Exp $
I’ll just have to keep digging, but I don’t think there is a real problem here. If we really had an IRC server, I would also be seeing alerts for an internal host with port 6667 open. Not necessarily, but in all likelihood.
I have no idea if this list is complete or not, but I had been looking for a list of possible IDMEF messages, particularly as applied to Prelude IDS/SIEM. I was playing with building filters in Prewikka when I noticed that the “build a filter” tool under the settings tab had a loooong list of IDMEF “paths”. I was able to copy the entire list from the html source and thought I’d post them in case anyone else is looking for a “complete” list. I think this list probably is complete, at least as pertains to Prelude. This will be useful for doing things such as building your own smtp alert template.
alert.messageid
alert.analyzer.analyzerid
alert.analyzer.name
alert.analyzer.manufacturer
alert.analyzer.model
alert.analyzer.version
alert.analyzer.class
alert.analyzer.ostype
alert.analyzer.osversion
alert.analyzer.node.ident
alert.analyzer.node.category
alert.analyzer.node.location
alert.analyzer.node.name
alert.analyzer.node.address.ident
alert.analyzer.node.address.category
alert.analyzer.node.address.vlan_name
alert.analyzer.node.address.vlan_num
alert.analyzer.node.address.address
alert.analyzer.node.address.netmask
alert.analyzer.process.ident
alert.analyzer.process.name
alert.analyzer.process.pid
alert.analyzer.process.path
alert.analyzer.process.arg
alert.analyzer.process.env
alert.create_time
alert.classification.ident
alert.classification.text
alert.classification.reference.origin
alert.classification.reference.name
alert.classification.reference.url
alert.classification.reference.meaning
alert.detect_time
alert.analyzer_time
alert.source.ident
alert.source.spoofed
alert.source.interface
alert.source.node.ident
alert.source.node.category
alert.source.node.location
alert.source.node.name
alert.source.node.address.ident
alert.source.node.address.category
alert.source.node.address.vlan_name
alert.source.node.address.vlan_num
alert.source.node.address.address
alert.source.node.address.netmask
alert.source.user.ident
alert.source.user.category
alert.source.user.user_id.ident
alert.source.user.user_id.type
alert.source.user.user_id.tty
alert.source.user.user_id.name
alert.source.user.user_id.number
alert.source.process.ident
alert.source.process.name
alert.source.process.pid
alert.source.process.path
alert.source.process.arg
alert.source.process.env
alert.source.service.ident
alert.source.service.ip_version
alert.source.service.iana_protocol_number
alert.source.service.iana_protocol_name
alert.source.service.name
alert.source.service.port
alert.source.service.portlist
alert.source.service.protocol
alert.source.service.web_service.url
alert.source.service.web_service.cgi
alert.source.service.web_service.http_method
alert.source.service.web_service.arg
alert.source.service.snmp_service.oid
alert.source.service.snmp_service.message_processing_model
alert.source.service.snmp_service.security_model
alert.source.service.snmp_service.security_name
alert.source.service.snmp_service.security_level
alert.source.service.snmp_service.context_name
alert.source.service.snmp_service.context_engine_id
alert.source.service.snmp_service.command
alert.source.service.snmp_service.community
alert.target.ident
alert.target.decoy
alert.target.interface
alert.target.node.ident
alert.target.node.category
alert.target.node.location
alert.target.node.name
alert.target.node.address.ident
alert.target.node.address.category
alert.target.node.address.vlan_name
alert.target.node.address.vlan_num
alert.target.node.address.address
alert.target.node.address.netmask
alert.target.user.ident
alert.target.user.category
alert.target.user.user_id.ident
alert.target.user.user_id.type
alert.target.user.user_id.tty
alert.target.user.user_id.name
alert.target.user.user_id.number
alert.target.process.ident
alert.target.process.name
alert.target.process.pid
alert.target.process.path
alert.target.process.arg
alert.target.process.env
alert.target.service.ident
alert.target.service.ip_version
alert.target.service.iana_protocol_number
alert.target.service.iana_protocol_name
alert.target.service.name
alert.target.service.port
alert.target.service.portlist
alert.target.service.protocol
alert.target.service.web_service.url
alert.target.service.web_service.cgi
alert.target.service.web_service.http_method
alert.target.service.web_service.arg
alert.target.service.snmp_service.oid
alert.target.service.snmp_service.message_processing_model
alert.target.service.snmp_service.security_model
alert.target.service.snmp_service.security_name
alert.target.service.snmp_service.security_level
alert.target.service.snmp_service.context_name
alert.target.service.snmp_service.context_engine_id
alert.target.service.snmp_service.command
alert.target.service.snmp_service.community
alert.target.file.ident
alert.target.file.name
alert.target.file.path
alert.target.file.create_time
alert.target.file.modify_time
alert.target.file.access_time
alert.target.file.data_size
alert.target.file.disk_size
alert.target.file.file_access.user_id.ident
alert.target.file.file_access.user_id.type
alert.target.file.file_access.user_id.tty
alert.target.file.file_access.user_id.name
alert.target.file.file_access.user_id.number
alert.target.file.file_access.permission
alert.assessment.impact.severity
alert.assessment.impact.completion
alert.assessment.impact.type
alert.assessment.impact.description
alert.assessment.action.category
alert.assessment.action.description
alert.assessment.confidence.rating
alert.assessment.confidence.confidence
alert.additional_data.type
alert.additional_data.meaning
alert.additional_data.data
alert.tool_alert.name
alert.tool_alert.command
alert.tool_alert.alertident.alertident
alert.tool_alert.alertident.analyzerid
alert.correlation_alert.name
alert.correlation_alert.alertident.alertident
alert.correlation_alert.alertident.analyzerid
alert.overflow_alert.program
alert.overflow_alert.size
alert.overflow_alert.buffer
Just a quick post to talk about a couple of linux-y things that I always add to any Windows box that I have to use to make my life easier and more enjoyable from a geek perspective. The first thing, at work or at home, that I put on my Win boxes is cygwin. Cygwin adds a bunch of tools and command line stuff to your windows box that gives it much more power, like a linux machine. Things like wget, curl, nmap, grep, tail, and hundreds of other cool things. If you use Linux, you know what there are already. If you don’t, install cygwin on your box. Then add c:\cygwin\bin to your system path and have a heyday. Want to see what your windows firewall is doing in real time? Try this:
tail -f c:\windows\firewall.log
Want to narrow it down to watch traffic from ip address 192.168.1.1? Try this:
tail -f c:\windows\firewall.log | grep 192.168.1.1
You get the idea. That’s just the tip of the iceberg, but that’s the kind of thing I normally use it for.
Another *nix-inspired app I install on every Windows box I use is vim. Vim is inspired by the linux command line text editor, vi, which is a bit old school. I’ve been using it for about 15 years and it makes me feel like a true computer geek. The thing I like about using it in Windows as opposed to notepad and wordpad is the syntax highlighting. Based on the file extension (.php, .html, .log) it detects and highlights your syntax for you, helping you keep track of where you’re at and to detect when you might have made a typo mistake. It is better at organizing text in files for readability, particularly in simple log files than any other program I’ve tried. Even if the file doesn’t yet have an extension (new file you haven’t named/saved yet) you can turn on syntax highlighting and tell it that it’s an HTML file, for example, and it will do its thing for you. It has a couple of interesting quirks that you’ll quickly discover, but I won’t go into them here.
Here’s a quick screenshot showing vim with syntax highlighting for an html file:
Oh, almost forgot: it also automatically creates a backup of every file you edit, which is cool.
I got my second Snort sensor online tonight! I had a bit of trouble and delay due to difficulties caused by the cloning process – and my lack of experience and familiarity with same. In order to speed up the building of my second sensor, which was put onto hardware identical to the first one, I decided to clone the first one by making a .tgz file containing nearly everything on it, excluding /dev /mnt /proc and one or two others – and then restoring the backup to the new one after installing the same version (ubuntu 10.10) of linux on it. I had a few challenges afterward with grub, the ethernet interfaces, and the SSH daemon. It was cool though – in the process of researching and fixing the problems, I learned quite a bit, and the process of building the sensor and getting it online was still faster than it would have been to start from scratch.
I haven’t actually deployed the new sensor. It’s going into a rack at our collocation site (because that’s where our internet connection and firewall are), but the sensor is registered in Prelude and ready to deploy.
Still on my to-do list: deploy (or have my trusty pc/network technician deploy) OSSEC Hids agents to the rest of our laptops and desktops, physically deploy the second and final snort sensor, and then begin the long, probably never-ending, task of tuning the sensors and agents for our network (cut away as many false positives as possible) and finally, to create a custom prelude-lml regex rule file for parsing the syslog for our GTA firewall, which definitely isn’t the most common firewall on the market. But speaking of GTA, they’re good firewalls! Based on hardened BSD and ICSA labs certified. I’ve been using them for 11 years (our current one is an appliance, but our first one ran completely off a floppy in an otherwise diskless old pentium box) and would be hard-pressed to switch to something else.
Anyway, here’s a screen grab showing my prelude manager’s agent listing in prewikka:
I’m merely condensing an article that I encountered elsewhere, partially for my own benefit (as a reminder to check some of them out later!). All five of these are worth checking out. I’ll be looking into at least 3 of them myself. These are all free (some of them have free and paid versions).
1. PacketFence – GNU Licensed open source Network Access Control (NAC) system.
2. SmoothWall – GNU Licensed open source firewall running from it’s own hardened Linux image. I’m interested in this project and will follow it, but I wouldn’t protect my corporate network with a firewall that hasn’t been certified by a third party like ISCA Labs.
3. ModSecurity – Apache module for securing apache-based web servers.
4. Untangle – Their free “lite” version is a server that comes with a firewall, a web filter, multi-malware blocker, IPS, OpenVPN and reports.
5. TrueCrypt – Free open-source disk encryption for Windows, Mac, and Linux! Worth checking out, especially for security-conscious laptop and netbook users. Full disk encryption makes it impossible for someone without your encryption key/passphrase from accessing any data on your laptop if it is stolen.