Both of these tools were recently released by Utah State University under the GPL license. You can read more about them by following the links, including sample movies that demonstrate how the tools work. The tools were created by Rian Shelley.
IPVisualizer is a visualization in which a range of IP addresses are represented as dots on a screen. The shape, intensity, and color of the dot indicate the direction, count, and type of the traffic, respectively.
OIP is a visualization in which individual machine IPs are placed randomly on a display, and packets are visualized as different sized dots flowing from one machine to another.
I just wrote a blog entry about some ideas of displaying time in link graphs. This is a problem that has bugged me for a while and I still don't have a good solution. The blog entry outlines some ideas and alternatives. Maybe you have a better way to visualize relationships and time in the same graph?
I have made a minor change with regards to letting people post comments to discussion entries. It used to be the case that anyone was able to post comments on the site. Unfortunately that meant that I got spammed quite badly. I realized that I had a huge approval queue for comments. I went through some of them and published them. Sorry if I deleted a comment of yours. Please repost if your comment got lost.
From now on, new comments can only be posted when logged in. Sorry for the inconvenience, but this should help a lot to make discussions more interactive through the comments.
Thanks for everybody that commented on broken links and such. I hope I fixed everything at this point. As always, if you have any input for the site, please let me know. Either by sending me an email or posting something here. Thx!
Interested in getting a quick overview of Security Visualization? I am guest-blogging on IT World. There you can find a series of blog entries about how to generate your own security visualizations:
(Update 12/05/08: Fixed the links. Sorry!)
Today, I would like to see if the urls that are not common in the previous graph, In this graph, heatline rendering plugin is used to check with line coloration if an event is regular. In the fourth axis, you can see lines going at the bottom and red lines go there. So let's forget about this and filter to only display lines that appear above 50% of this axis.
The filter is between single quotes, just like what you'd do with tcpdump ( I actually took their code to handle this ;-) ).
This line was typed to get the graph you can see here:
pcv -Tpngcairo -Rheatline -Avirus access-wallinfire.net.pcv 'show plot > 50% on axis 4' -ra > picviz-uncommonurls.png
If we take a random IP, such as the one we clearly see on the second axis, 126.96.36.199, and googling about it, we find that this was an infected machine. The url here tells more about it.
As a conclusion for this graph, you can see that among all those lines of log, with a very empiric approach, we really discovered something. Not a very innovative attack I admit, but enough to keep searching (I will post ongoing researches here, keep following!).
Ah, and by the way Raffy, since you asked to only display lines every few times, I added the -L option, taking a number (N) as argument meaning every N lines you display the text.
This parallel coordinates graph shows 412429 lines of one of my wallinfire.net access log with generated with Picviz svn. This is the first of a set of graphs which will derivate from this one. The most complete one.
To generate such a graph, simply use the apache-access2picviz Perl script available from trunk/tools. Then, use the heatline plugin to see line frequencies: the more green the line is, the lowest it appears. When a line is in red, it means it comes often. This way you can easily see if an event is regular or not. To generate this image, you can type: pcv -Tpngcairo access.pcv -Rheatline -Avirus -rra > accesslogs.png
First axis = Time (24 hour) with 00:00 at the bottom and 23:59 on the very top.
Second axis = Source IP with 0.0.0.0 at the bottom and 255.255.255.255 on the very top.
Third axis = HTTP request type.
Fourth = Request
Tomorrow, I will post a filtered graph, on the request axis to see what are the IP addresses that are doing abnormal requests.
Picviz is available as free software on http://www.wallinfire.net/picviz
Follow SecViz on the brand new twitter feed: @SecViz.
As part of the ISSummit in Hong Kong, I will be teaching a one day workshop on security visualization. The following is the abstract of the training:
As networks become ever more complex, securing them becomes more and more difficult. The solution is visualization. Using today's state-of-the-art data visualization techniques, you can gain a far deeper understanding of what's happening on your network right now. You can uncover hidden patterns of data, identify emerging vulnerabilities and attacks, and respond decisively with countermeasures that are far more likely to succeed than conventional methods. The attendees will get an overview of visualization, data sources for IT security, and learn how to generate visual representations of IT data. The training is filled with hands-on exercises.
The talk is going over the following individual topics:
I am working on a Parser that transforms any kind of ASCII input log file into KML files used in Google Earth/Maps.
What does it do?
1. Reads the log
1a. Extracts an IP (column or Regex based)
1b. Extracts a node name (column or Regex based)
1c. Extracts a description (column or Regex based)
2. Retrieves a longitude and latitude to the given IP address
3. Writes a KML file for Google Earth
Well I was recently analyzing a big set of IDS events with abut 99% of false positives. Most of them originated from partners of the company located in England and France. Only a few of them - the real positives - originated from countries like Iran, Pakistan, Brazil, Russia and China.
That way it was really smooth to determine the real attacks and leave the false positives aside.
The picture is an animated GIF. Hope this format comes through. :)
It shows an analysis of a mod_security log file.
I'll provide a download link for the script soon. (the sticking point is the GeoIP database; most of them are commercial ones; currently I use TOR network and a GEO IP web service to determine the LAT and LON parameters. That should not be the final solution, right?!)
---- Update -----
I use a local database now which makes it very - Very - fast. It parses 8500 mod_security events in 10 secs. The resulting KML file has 5 MB. I generated a new picture set showing all events of 9 month placed in a world map. And I have to say ... "bad - bad bad Italy!". I am quite sure that this was the spreading of the MPack Malware Kit this spring.
--- Update 2 ---
I advanced the script with an option to define a source and target IP address to draw lines in the world map. Nice, isn't it? Now it is possible to parse log files with source and target to determine the connections between the nodes. Targets are marked with different icons.