Log of my Senior Seminar Project:



My project is writing a network sniffer/monitor program and researching TCP/IP [and SNMP at first, but this was later abandoned] to provide me with the knowledge to make the program. In addition, I'm working on a tutorial about the scripting language Tcl and it's graphical add-on Tk. The tutorial is geared towards non-programmers. Below is the log for what I've done so far.


Date Work Completed

09/19-09/20 Read an entire book on networking. Lots of notes on TCP and IP headers, etc. Also the OSI model. Considering limiting myself to just TCP/IP.

09/23 Initial written proposal for my senior seminar. It has since been revised...

09/26 Trying to read through SNMP, SNMPv2, and CMIP: The Practical Guide to Network-Management Standards. It's dense, but some good information.

09/30 Handed in my outline for this project. Posted this set of web pages.

10/04 Installed Linux (Redhat 5.1 dist.) on my machine. Remembered that I had a Winmodem, which Linux doesn't support. Also can't get my printer to work correctly.

10/06 Turned on printer and had a redhad test page print out. Printer works fine now...

10/09 Printed out the MAN page for TCPDUMP. Looks like it does a lot of what I want it to. Played with it some, looks like it has a few easy ways to get it into C.

10/10 Recieved my outline back from Charlie. He confirmed that I should use TCPDUMP and perhaps something called SNOOP(?). I can guess what it does, but it doesn't seem to be a normal Linux command.
..... Finally had time to dig up an old 14.4 modem I had laying around. Threw it in my machine, so now I have 2 modems, one for each OS... Looking for another phone splitter. PPP is installed and working, was able to print out some tests using TCPDUMP. Looks awesome!

10/11 Trying to work out a sample interface in HTML. I may have it just present either X packets or Y amount of time for now. It occurs to me that if I fed it a constant stream of network traffic it would show me sending the stream, then record that and send it, and record *that* and send it... Just a thought.

10/12 Posted testme.html, which is where my sample interface will be located. I hope to have this listing whom has talked to whom in the last 30 seconds (or maybe packets) by Thursday... Hope I'm not just dreamin' here.
..... Added my links page. I know from experience that it's easier to update this as I go along... :)

10/13 Been researching lots of CGI stuff. Tried to keep links page accurate and current... Wondering if CGI will limit me in what I can do

10/14 Tried to get my first portion of code working, but getting stuck with some stupid C mistake that I'm making but can't identify.

10/16 While on my mid-term break I realize what I'm doing wrong and interrupt my day with my girlfriend to fix it. Thereby proving I'm a true CS geek...

10/18 Everybody is sleeping or out so I spend some time expanding my program. I hope to have it running once I spend a day or two back at Earlham.

10/19 Ok, mid-term "break" is over, so I'm back at my Linux box pluggin' away
..... Expanded sniffer.c so that it now takes in a dump from tcpdump (in file format) and gives you a list of senders and receiver's addresses. Should put it into CGI format later today... Off to class!
..... The program pulls the addresses out fine, but have been spending the last few hours trying to design a way to store the addresses quickly. I think I'm going to make a big array and hash addresses into it, thereby making it very fast. Considering all that this program does is spit the output to a webpage, the array should never be too big.

10/20 (trying to) finish up implementing the hashing of the addresses into a big array.
..... It's 2:30am and I just finished a coding spree so that this damn thing works fine, puts the addresses into a big array (hashed from their first 20 characters). If the address is already there, it increments the counter, and if it's a different address is goes to a different value (in theory, not sure that's been tested. Also I've got code to prevent infinite loops (assuming the array is big enough) which I *know* hasn't been tested). I've got class tomorrow morning and a cold still, it's time for bed. Oh, and it's not CGI yet, all C.

10/23 Spending friday night coding... Huh. I've put a few hours into hanging linked lists of received addresses off of the big array of sent addresses. I plan to fix the bug of it just adding new nodes instead of incrementing the counter when it's not the header node and then u/l the newest version of code and go to bed. When I wake up (after attending meetings) I want to have these linked lists build themselves in order of frequency (shouldn't be too tough, they're doubly linked).
..... Nothing is ever simple. U/L'ed the newest version of the code, with nodes being added correctly. I think (hope) that everything works great right now. It's 4:30am, time for bed.

10/24 It's 11:30pm, been working for a while. Adding a section so that it will output the sender's addresses in order of greatest frequency to least. Been feeling pretty weak and ill, so perhaps I'll go to bed earlier tonight.
..... Quarter after midnight, and the program does print them out as I want. Going to work on the CGI format for a bit, perhaps have a test page up before I go to bed. I'm thinking of putting it in a big table, with the sender's addres and frequency of use the only two columns. I'm not sure yet how to handle the reciever lists yet ("optional" scroll bars?)
..... WAHOOO! At 3am I finally got it to produce a nifty web page as output. I piped it to sample.html where it is currently sitting and looking beautiful. It's bed time!

10/26 After a few more hours of work (isn't this a 3 credit class?), sniffer.c now outputs the RECV lists as pulldown menues attached to each send address. This looks quite snazzy, but i just found out that it only works for IE(4??). I'm gonna figure out how to do it for Netscape and pray it's universal, but I'm really friggin' annoyed right now. I am really into this CGI stuff, though, it's a lot of fun.
..... Just passed the midnight marker and fixed that annoying netscape-IE error (in reality a stupid programmer error that IE compensated for and Netscape didn't. Go figure). Far as I can tell, the program works great!
..... 3am, heading to bed soon (I hope). I've been putting my notes into some semblence of order, and so I posted the docs as they are so far. Right now they're formatted for Office97 on my PC, I will get other versions up later. They're also in pretty rough form right now..

10/27 Just put up newly-revised version of some of my documentation. It's short, but they're in HTML format using screen captures to get the charts over.

10/28 I've just been updating my research materials, added TCP and UDP Protocol and drew and posted the charts for it. I also just updated my proposals a bit. That's about all for a few days, folks!

11/02 Started researching SNMP for my second programming project. I gave up reading SNMP, SNMPv2, and CMIP: The Practical Guide to Network-Management Standards by William Stallings because it did nothing for my general understanding of just what is SNMP. Web research has led me to seriously question the ability of creating a program that would use SNMP to find out who else is on the current network w/o those PC's running an agent...

11/04 Talked with Charlie about SNMP. He suggested the blatently obvious: that all the routers should have SNMP and can tell me what computers are jacked in.

11/08 Spent the last several hours d/ling some SNMP stuff and looking at TCL and TK. There is a plugin for Netscape (and IE, I think) which allows their use in web pages (Linux, Windows and Macintosh) so it might make the visual representation a snap...

11/09 Downloaded Tcl for Win95 and Linux. Poking around with what SNMP to use, it's looking like CMU if I go w/Linux... Spent 4 hours d/ling stuff, and got pretty annoyed at my -s-l-o-w- connection.
..... Added sniffer.cgi to make sure I could actually make a working CGI page...

11/10 Spent some time installing CMU snmp. I am way lost on most of this stuff. I would like to try using Tcl and the plugin to create a web accessable program, so I ordered a book that is all about Tcl and networking. Should be here this weekend... Until then I think I'll try to work on (finish up?) sniffer.c

11/11 Figured out what idiotic error I was making with cron. Planning some final details for sniffer.c
11/13 My book comes and I am way hyped about Tcl. This scripting language is awesome, I'm still excited about making a button. (I'm a geek, shut it)
..... I've noticed that there are no resources for non-hackers on Tcl out there, and since Tcl/Tk has been ported to Windows and Macintosh, it seems like perhaps it's time... I'm going to ask Charlie if I can switch my topic to writing a tutorial for non-programmers.

11/14 Charlie agreed and I'm way happy. It seems like Tcl/Tk is a really awesome resource for pulling in people with no programming experience who want to quickly make exciting programs and useful tools.
..... Ok, I just spent the last five hours of my friday night on this. The main tutorial index is up along with my introduction, How and Where to Get Tcl/Tk, A Few Basics, and Your First Widget.

11/15 Infocom was down all night, but I worked on the Core Rules for Tcl page. It's finished.

11/16 Infocom is back up. Posted Core Tcl Rules and changed index page.
..... Posted Tcl Commands, although it's pretty bare right now. I've added an operators chart, if.then.else's, loops, lists, and arrays to the Core Tcl Rules page. This took me quite a bit longer than I had anticipated...
..... Proof read the pages I had up. They needed it... Probably still do. Gotta stop proofing at 2am after 5 hours of work :)

11/17 Getting real tired of Infocom only letting me access Earlham and Infocom at random intervals. [expletives deleted the next day]
..... It's 3:20am, just spent 4 hours testing and adding in a slew of Tcl commands.

11/18 Decided maybe it's time to perform an overhaul of my page layout for my main index page and re-write my proposals.
..... Rewrote proposals. Checking out a tutor for Tcl, it's an actual program and tries to be for non-programmers, but obviously wasn't written from that perspective. Makes me question how well I am doing...
..... Rewrote my main index. A little more organized, and hopefuly nicer looking... Also puts a little more emphasis on the Tcl aspect as this is a fairly major component of my project

12/01 I've been away from computers for a while, but I really have been doing work! I completed reading my Tcl/Tk book and am considering buying another one more specifically geared towards what I need, but I don't think that it will be necessary. I also have started on my TK pages and updated some Tcl commands after d/ling the man pages for Tcl/Tk. I was hoping to post some Tk Basics tonight, but I've been spending a lot of time just learning Tk so I may put it off until tomorrow.

12/02 Happy Birthday to me! :)

12/03 Been plugging away at my basic rules for Tk page. Infocom is down, so things are a little hairy, but hopefully I'll be able to post the changes tonight.
..... Basic rules for Tk are up. A person following along should be able to make a calculator that outputs to the Tcl console. I have a list of Tk commands up, but they're yet to be defined (next step). Sleep first...
12/05 Fixed up some of the Tcl commands and have started to add in Tk rules.
..... Hmm, it's 6am (technically the 6th). Just spent about 8 hours coding a small game in Tcl/Tk called Land, Air, and Sea. It's based on a very simple game someone taught me, but it's damn cool. I have a meeting in less than 4 hours. To sleep, or not to sleep... Oh, LAS is complete except for instructions. I wonder how one compiles a Tcl/Tk program.

12/06 Coffee-enhanced 3 hours of sleep goes a long way. LAS has instructions and I broke it into modules so it's a bit more legible. Re-wrote some of my Core Tk rules to add in what the program taught me and have started on the commands. Hope to have excercises done shortly after and wrap this puppy up. Maybe I'll even sleep sometime.
..... 2am, why am I up? Almost finished with my Tk commands. I plan to finish them and then stop for the night.

12/07 I did complete the Tk Commands page last night, and have spent a few hours proof-reading all my pages so far tonight. I'm also trying to make the format the same (italics early on evolved to bold italics later).

12/13 So I keep a great log up until the point this project is due and it actually counts for something. Go figure. Ok, the last week has been all kinds of hell made up of mostly me cramming-because-I'm-an-idiot for my presentation that happened last friday (the 11th). A couple of 5am nights ended up with me being overly prepared. Go figure. Anyhow, I should be able to put all that into paper format without too much of a problem (fingers crossed). What have I been doing this weekend? What I alwasy do: deciding at the last minute that my program is all wrong and starting again from scratch. That's right; sniffer.c had some big issues with scaling which I never resolved so here's the newest plan: cannibilize the old sniffer.c a lot, but this time have the big array also contain a TimeToLive column, and go through and systematically flush out all those elements that only get hit once or twice. Update the web page (no longer cgi, just an oft-updated page) every flush, or more likely, every few flushes. The web page will only show the top 20 or so, so no more charts of several hundred rows. This changes the use of sniffer.c somewhat, but I think that it's now more in line with its actual output. As of now (5am, technically the 14th but don't bug me) the new sniffer.c is mostly done, I just have to add the output_to_file and FREE THOSE DAMN MALLOCs. Totally forgot about those until just now, sorry about the shouting, but I'll need to fix that. Hmm, might have to walk down and then trace back up those linked lists... Gods I need sleep. Ok, that's my current "to do" list (and clean up the Tcl/Tk stuff)

12/14 Re-wrote sniffer.c and am trying to thrash out the bugs.
.... sniffer.c 2.0 is complete! :) I'm testing it on my machine now, and assuming that it continues to work I'm going to head over to dennis and try it there. The only feature I wish it had, and tried but it's taking too long to figure out (it's just so simple, though) is to not have pull-down menues if there's only one receiver. Still, I'm happy.
.... I've given up on sniffer.c for a while. It works great on my machine, but causes a segment error on tsetse. Something to do with modifying the output.html file, I may get around it by taking it out and piping the data or something if I can't figure it out, we'll see. I'm working on other parts, trying to fix what Charlie commented on and re-writing the TCP section, including hand drawing a few pictures. I'm sort of leaning towards an all-nighter (again) to finish this mostly up so I can study tomorrow and sleep for my 2 exams on Wednesday.

12/15 Gave up on sniffer.c all of today, concentrating on getting the website looking nice(er) and organized. Printed out over 50 pages of my Tcl Tutorial and my networking research. Added in the timelines and tested a bajillion links (give or take). Sniffer.c is now the official name for the program, and it has its own website. Now if only I could get it to bloody work...
..... Wahooo! It's quarter 'till midnight and Sniffer.c is up on tsetse and running! Damn but it's cool. I'm waiting to see if it will rollover when the numbers get too huge (it's designed to), and if it does I'm gonna kill the process and go home. Charlie can set it up again when he wants to test it, because until I'm sitting with someone who can tell me if I'm shredding the hell outta TseTse I'm too paranoid to leave it running on its own. Go figure. One thing to note is that sometime when you refresh the screen, you get a "no data", that's 'cause sniffer.c is in the middle of building the file, you just have to wait 5-10 seconds and try again.
..... Hmmm, network traffic isn't steady enough to cause a rollover yet. I may just kill it and leave. Perhaps I'll set up an infinite loop on a telnet and check out what that does... >:)
..... Ok, I give up. It works, I'm going to print out everything, and then this log is probably "Complete." More will be added next semester or over break, but as far as grading goes (and doesn't school tell us that grades are the top priority?), that's it folks! It's been a great time, hope you've learned as much as I have. See ya, Chris.



return to the title page

Comments:
Chris Palmer
Ardenstone@Ardenstone.com