Project Log:
Ken Innes(K), Chris Palmer(C) and Ryan Sackenheim(R)
03/29 K,C,R: Spent class time planning. Figured out that Chris would be the Scribe, Ryan the Tool-Smith and Ken the Architect. Also decided to try and meet the next day (Tuesday) at 4pm with Tom and/or Kevan from ECS to discuss what we're doing, what we need and what they know (Ryan will contact).
..... C: Created space in my directory and posted ~palmech/www/map/index.html, outline.html and this (plog.html).
..... C: Finished the first draft of our outline.
..... R: Experimented with Tkined. After sussing some network configuration problems
(special thanks to Chris Hardie), was able to map some of Earlham's C class addresses.
03/30 C,R: Interviewed Kevan Wogoman of the Earlham Computing
Services. Discussed the current network as well as future plans.
..... C: Posted Interview Notes.
04/04 C,R: Met over Easter brunch and ended up talking about this project.
Discussed trying to set up TKIND this week and then perhaps creating our
own web monitoring device.
..... C: Spent a couple hours on WhoIsUp.pl, a perl script that reads in a
list of IP addresses, pings them and says who is up (or down). I can't
figure out how to get ping to shutup on Unix, so I ported it to Windows.
It works there.
..... R: Found a couple of possibly useful tools "queso" and "nmap". Queso
takes an IP address for input and discovers the type of operating system that
computer is running. nmap takes an IP address as input, and outputs what
services are running on which ports.
04/05 C: Cruised the net and my linux box trying to find source code for
ping. Finally e-mailed networking group!
..... K,C: Met during class to setup Tkined on Proto. It has a strange
error, but seems to still work.
..... C: Setup Tkined on my own linux box and encountered what I think was
the same problem. Scotty assumes docs are in /usr/local/lib instead of
/usr/lib. Played around to become more familiar with Tkined.
04/06 K,C,R: Met at Proto. Ryan downloaded another version of TKINED and
installed it, but the error (not TOOLS menue) occurred again. Chris
applied the fix he'd used on his machine and things work a lot better.
We started to create maps, and ended up with one for Earlham and one
of the CS dept.
04/07 C: Read through all the man files for TKINED. They're mostly
definitions for the commands themselves, which makes some sense. There's
a little redundancy since both Tnm and snmp (mainly mib) overlap. Looked
at some of the code, but haven't really spent more than 1/2 hour bearing
down on it yet.
04/08 C: Wading through the archived scotty mail list. It seems that
several people have attempted to build some sort of web-interface to
TKINED, and none of them exist and/or have been finished. One possible
lead is http://netmon.itb.ac.id/~ismail/project/ but there is little to no
documentation. Try the "map-generator" subdirectory. I'm going to try
and d/l it later and see if I can get it
up on my machine.
04/09 C: After several hours of trying to modify map-generator's code to get the bloody thing to work (the author is Indonesian. and so are the comments and some errors...) it's working! It's created for a different version of scotty (and Tcl/Tk? I forget) so those had to be changed in Command.tcl and ip_monitor.tcl. Plus some other stuff, but most of it was pretty awesome. Oh, I had to nuke the arguments in calling Scotty for ip_monitor so I'm a bit worried about that... :) Anyhow, it outputs a webpage which shows the network in a graphical mass of ip addresses. Pretty cool, I'm going to modify it and see if we can easily add in some more details. And maybe study for my GRE's tomorrow if I have time...
..... C: Check out MAP.HTM. It was generated using
an Ip Discover on 159.28.1 in TKINED and wasn't really arranged to make a
very good visual display. Still, you get the idea!
04/11 K,C,R: Met to install map-generator on Proto. Having issues opening the IP Monitor tool and getting it to update correctly. ip_monitor.tcl seems to want to pass two commands to Scotty, but doesn't have those so gives an error and doesn't appear. It worked before the patch.
04/12 K,C,R: Met and fixed ip_monitor.tcl! We had to delete the comments in the top of the script. Why did this work? Who knows! But it did. After fixing some typoes (our own) we got everything working correctly! Added runme a script that runs the perl scripts and copies the files as needed to setup map-generator. Now we just need to keep TKINED running and set up a CRON job to run /usr/local/map/bin/map.pl /usr/local/map/log/tki.table.
..... K: Fixed the problem where we couldn't get a web page up on Proto.
04/13 C: Jumped on Proto to setup a test map that's more like how our final product may look. One machine is off the map, but I can't fix that because I don't know the root password and can't locate Ken. Root is needed to run TKINED, which is becoming a real pain in the arse. Anyhow, used old data to setup proto.cs.earlham.edu/~mapguru/map.html and linked to it.
04/14 R: Wrote to Mike and got her to translate the comments from Indonesian to English. Posted it to here
..... K: Interviewed George, but he had no ideas. Said to talk with Kevan and Tom.
..... K,C,R: Met over Proto. Spent sever hours creating a map that includes lots of key servers including IHETS and Infocom. Also discovered some quirks about TKINED and map-generator:
1) If you add in a computer by hand (using the computer icon) it will not be visible on the web page. Instead, it is out of bounds, just showing a line going off the edge of the map.
2) Check Reachability statistics are cached somewhere. Therefore, if you say, try to monitor the frame relay(?) switch(?) and it doesn't answer to pings, it will flash red and "unreachable" forever. Even deleting it, saving, and then re-discovering (and not checking its reachability again) doesn't work. Even killing the Ip-Monitor menu will just stop the refreshing of the ping time values, not set them to "unmonitored" state as it should.
3) Restarting TKINED is a good solution to most tcl errors. Things like IP Discover sometimes hang and TKINED starts to degenerate. Just restart it. Usually you have to kill the process, so perhaps just killing the malfunctioning menu would work as well.
..... C,R: Stayed on Proto 'till 3:33am trying to get map.pl to output links. It's close, but we're still pushing the links around by our HTML insertions. NOTE:We need to request that Tom activate rstatd and etherstat(d?) daemons on Yang and any other computers he wants to monitor closer details.
04/15 C: While emerging from the shower, Chris is hit with a burst of genius causing him to realize that he could have saved several hours and gotten the code to work correctly if he had made a much simpler modification to map.pl. Namely, adding the A HREF commands to the FONT additions. He subsequently almost falls over and then rushes over to the lab before class, only to find the room locked and the entire situation pointless because he doesn't have fucking root access anyhow so couldn't implement the fix.
..... C: Got JJ to change the root password on Proto so now we know it. Also implemented the fix, and our web map looks awesome!
..... K,C,R: Met over Proto to start writing the script which our HTML links will run which will use nmap to find the target computer's OS and services running. Unfortunately, we know waaaay too little about CGI. Ryan did get some good experience learning Perl, and created a parser for nmap's output.
04/16 C: Created getnames.pl which is a perl script that parses tki.table and returns all the node names and their IP address #'s. This can easily be changed to a lookup from name -> IP# if we decide to stick with user-initiated scans on specific machines.
04/17 C: Created buildhtml.pl which is a major re-write and addition to Ryan's test.pl It now parses the output of nmap even if it contains multiple sections (ie: you feed it all of the ip #'s from your TKINED display and use the nmap -Oi- command) and returns a full HTML page. I used getips.pl (a modified version of getnames.pl which just feeds back a list of ip #'s parsed out of tki.table) piped into nmap and it took 55 minutes! That's only about 17 machines... Anyhow, the output was fed to buildhtml.pl and that was saved as specs.html.
04/20 K: Found a MIB browser that uses a WWW interface. It's abstract and not well documented. Not very useful.
04/20 C,K,R: Worked on trying to track traffic on machines. Found the MIB entry that corresponds to this and are tracking it in TKINED. Ryan is going to fix some bugs in buildhtml.pl and Chris is going to try and figure out how to get those ethernet statistics using Scotty (probably the mib command).
04/21 C: Messed with using ICMP echo requests to figure out Round Trip Time as a possible indicator of network traffic. If averaged it gives a good picture, but I think it's probably somewhat mislead due to it originating at the host and not the target computer.
..... C: Was able to establish what I think is an snmp connection in Scotty with set x [snmp session -address 159.28.1.1] but I can't get any value back from the mibs. I've tried mib and $x get where $x is the variable that holds the snmp session number.
04/22 C: Trying to implement etherspeed.tcl which will return the %bandwidth of an interface using SNMP. Stuck too early to be worth describing...
04/23 C: Spent too long on a friday night and got etherspeed.tcl to take in an IP address and return the %bandwidth used as if it were single and duplex. I don't know how to differentiate them yet, but I'm going to bed.
..... C: NOTE TO KEN AND RYAN: I've been moving and adding code to www/code on proto because mapguru's root was getting way clustered. You've been warned. :)
04/25 C: Trying to figure out single vs. duplex. Changed the background color to grey (c0c0c0) in map.pl. Haven't looked at it because I'm at my house and not on Proto.
04/26 C: Changed map.pl so that the legend is corrent re: node color and the background is c0c0c0. Font size is also bigger. Re-ran map.pl so changes can be seen on map.html.
04/27 C: Check out mapMaster.cgi!!! This uses the output from the new version of etherspeed.tcl (which includes breakdown of traffic by protocol via. SNMP!) and tranNmap.pl which is a modification of buildhtml.pl (ie: Ryan- don't worry about fixing my screw-up, it's no longer important). The only problem so far? Not too happy w/tsetse (no SNMP response, I think) and I had to kill the part where it guessed the target machine's OS because mapMaster.pl doesn't act as root (is this fixable?).
..... C: Forget mapMaster.cgi, check out the bleedin' map! The links are connected (fixed map.pl) and mapMaster.cgi is now smart enough to tell if SNMP is refusing connections! I'm wondering if the SNMP dividing up the bandwidth by protocol is accurate, but it seems about right (re: TCP crushes all other traffic). Note that when you activate mapMaster.cgi you will wait about 10 seconds for etherspeed.tcl to get a decent estimate. Any shorter and it's not accurate...
04/28 C,K,R: Met over Proto to put finishing touches to WWW monitoring tool. K typed draft for
future expansion of Earlham network.
..... C,K,R: We're done! It all works (we think) and it's all there. Note that the html map won't be updated regularly because we don't have a dedicated machine to keep TKINED running and chron map.pl, but instructions on how to do so are in our notes.
04/29 C: Snuck onto Proto before class and added links to Scriptics and Scotty as well as e-mail the author of map-generator and telling him, "thank you!" and explaining a bit of what we did.
05/05 C: Spent a few hours on Proto creating getTcpInfo.tcl and getTcpPrcnt.pl. They work together to breakdown TCP's bandwidth usage by service via SNMP. I changed mapMaster.cgi to include this and removed the NMAP part (it was bit redundant).
..... C: Hacked a whole lotta crap out of our index page as well as revising the README.html that tells people how to install this thing.
..... C: Thought better of it and added NMAP back into the code and documentation. It's still pretty handy for machines w/o SNMP up. Also added pictures to links.html.
05/09 C: Added OS warning to homepage. I've already got e-mail saying, "how do I install it under Windows?"
return to index.