So for several months now my Wii has been unable to connect to the Internet. It could get updates, use the news channel and forecast channel, but could not use the browser or shop channel. The stranger thing is that my roommate is connecting to the same router with the same settings and his Wii works fine.

I decided I should fix it, envisioning a long call to tech support and if I was lucky I would be able to ship it off for some indeterminate amount of time and eventually receive my (a) Wii back. While I dreaded this I have heard that Nintendo’s technical support is quite good. Luckily I didn’t have to experience it.

If I was going to go through the trouble of calling a technical support agent I wanted to at least know why it wasn’t working and while that would be unlikely to speed up the call I would know when I could lie and say, “ok tried that and it didn’t work” versus actually trying it out.

I have always been interested in packet sniffing but have never really had time to get into it, but at some point in the recent past I installed Wireshark and played with it. I sat there watching my laptop talk to my desktop about the printer. I felt bad that they were talking about my printer behind its back, but didn’t have the heart to let him know. Last weekend I started up Wireshark again and started sniffing my Wii traffic (by the way it smelled bad). The Wii sat there querying my primary DNS server, but never received a response from it, it also never bothered to try the secondary DNS server. So I removed primary DNS and setup the configuration so its only DNS option was the secondary one and like magic everything worked. While I am still not sure why my primary DNS server didn’t respond or why my Wii didn’t try the secondary it at least works. Its a great feeling when one of your random curiosities actually ends up having a practical use. Not to mention it probably saved me an extremely long tech call that would of had a good chance of not fixing anything.

The best thing about this is I finally got to see how my site works on the Wii with The Ultimate Gamer Pack Theme and wow it looks great. I thought the theme would just be a basic plain white theme that increases the text size or something to make it more readable on a TV. Instead, the Wii theme really makes it look like a page designed for the Wii with a similar button and content layout and easy navigation.

Since I am speaking of Wordpress themes, be sure to check out the most impressive wordpress theme I have seen, the Commodore 64 Command line theme.

Lifehacker pointed to a list of software that claimed to be lightweight apps that replace certain memory hogs. The last two (original) articles are a great read, however the lightweight apps list is pretty poor.

The problem is several fold. His article is basically a rehash of the articles on The PC Spy listing the bests of specific categories, however, the original article wasn’t really about lightweight applications. It was about the programs that slow down Windows the most. There were a lot of complaints about items like Firefox (which will cache all the pages it visits until it runs out of memory) and Trillian (by no means lightweight, but an great application). Lifehacker escalated the problem by listing them not only as lightweight apps, but as open source substitutes. Not all of the programs he listed are open source and ’substitutes’ for what? Some you can assume are substitutes because there are dominate programs in that field, i.e. Foxit Reader as a substitute for Adobe. Others aren’t really substitutes at all. For example, uTorrent is the lightweight substitute for torrent downloads instead of… Sure there are popular bit torrent programs out there, but none of them hold a dominate or ‘default’ position that you would find substitutes for.

This got me thinking though. What exactly is a lightweight application and how would you measure that? In my opinion a lightweight application is something that I can quickly open to perform a quick task. The most important factor in a lightweight application then is launch time. Next to that would come factors like limited memory usage and low footprint (i.e. doesn’t try and take over my desktop, doesn’t take up much hard drive space, doesn’t pop anything up when I boot up my computer, etc.)

I wrote up a quick little c# program that launches other applications. I measure the memory usage, peak memory usage, and processing time. I’m not exactly happy with it or rather I don’t trust it right now, but I should probably try it with non-web based applications. Because I am not happy with it I spent a lot of time procrastinating talking about it. I decided I would finally finish one of my half done blog posts and this one got picked.

I decided to see what the lightest weight browser was. I used the code to take the average launching time of each of three browsers (IE 7, Opera 9.1, and Firefox 2) 10 times with three different start up options (Blank page, MSN.com, Google.com). Each browser was freshly installed and configured with no bookmarks, clean cache, no history, and set to launch a blank page by default. I ran it on a Athalon 3200+ with a gig of ram and XP home SP2.

The reason I chose this test was that I wanted to see which was the best ‘default’ browser. A major factor for the best default browser is how fast it opens. Currently I have IE7, which I never use, Firefox, which is my default but rarely gets used, and Opera, which I use all the time. The reason firefox is my default is because if I click on a link and Opera isn’t open, it is likely I just want to check out real quick, I want this to be as fast as possible and I feel Firefox does that.

Here are the results:

It’s the average of 10 runs, but I think the network causes too much distortion in the msn/google data. It doesn’t seem right that the simple google page takes longer to load than the busy busy msn page. I blame the network for this. I also can’t explain why the local blank page takes longer to load than msn in the case of Opera and Firefox. Caching maybe? Not just page caching but process caching. Not sure. However, if we just compare the local blank page between the three programs we have IE 7 as the fastest, followed by Opera, then Firefox. I am not suprised that IE is the fastest, but I was suprised that my clean, extensionless firefox install took longer to start than Opera.

So will I switch my default browser? No, I don’t trust or like IE and even if the data is correct and even if Opera is slightly faster than Firefox, Opera is set to continue browsing from where I closed it (like Session Saver for Firefox). So in the end Firefox is still the best choice for me to quickly open up a webpage.

I will probably try this on some other set of applications if for no other reason than to find out if my program really works and if the huge variance I saw in the browsers was due to the network.

The AOL search data has spawned a number of websites. Besides the one I mentioned the other day there is AOL Data Search, AOL Search Logs, and Don’t Delete which offer similar services. AOL Stalker offers a nice slick interface and was actually how I had originally thought of implementing it. It not only allows you to search the logs, but displays what other people are searching for. Its quite interesting to see what people search for on AOL, but it is equally as telling to see what people searched for in search queries.

Anyways you can easily set up your own database. The first step is to get the data, which is still available for download. You can either download the 10, 400 mb zipped text files separately or through bittorrent. Once you have the data you can do whatever you want with it, write perl scripts or examine it in any language you want. The real problem is that the files are so huge and it is hard to move around in a flat file like that. So I would recommend tossing them in a database so you can pull out the information you want, whenever you want, and in reasonable quantities. I wrote a bit of code in Java that just grabs each line from the file and puts it in the database. Simple and could probably be cleaned up to remove duplicate queries and such. If you really wanted to be efficient you could start collecting statistics as you were putting it into the database.

You can grab the java file to save some time, you will need to edit it for your configuration and grab the mySqlConnector or whatever database connector you use. Use, modify, redistribute at real it isn’t anything impressive and probably has bugs. I suggest you import into a local database, otherwise it will take forever to send all the queries over the Internet. If you want to put it on a separate server somewhere you should probably parse the file into the database locally, dump the database, FTP it to the server, and then source it in. If you don’t have MySQL installed grab XAMP its perfect for this kind of stuff.

Speaking of data, I really really want the Google N-gram data, to bad its $150.

Software is meaningless without data. This is especially in fields that rely on that data to learn/test/train the system. I have done some research into machine learning, natural language processing, AI, A-Life, etc and while the subjects are complex perhaps the hardest part of it is all is obtaining data. Data makes all the difference in the world.

In the middle of last year AOL released about 21 million search queries. When I heard about free data I raced home from work and started downloading it. Later that night I imported it all into a database and made a PHP page to search it. I didn’t have a webhost or anything so that was about all I could do with it except gather statistics. I thought I would finally toss all that on this server. It isn’t very pretty and I only added about 2.5 million records to the database. AOL Search Database already has a much nicer setup so I didn’t really see a need to completely duplicate it. I just wanted something up so I could play with it and try to come up with something a bit more interesting with it.

Here is the stats on the data:

• 36,389,567 lines of data
• 21,011,340 search queries
• 7,887,022 requests for “next page” of results
• 19,442,629 user click-through events
• 16,946,938 queries w/o user click-through
• 657,426 different users

One of the hard parts about gathering ’statistics’ on the data is that each ‘query’ shows up multiple times. For example, if someone searched for ‘cat‘ and then clicked on four links it would should up four times, even though it is only one query with four click throughs. If they click the ‘next page’ it shows up as a new query with a new time stamp, even though it is the same search. Most of these items can be found by checking the time stamp and keeping track of what the rank of the link was that they clicked on.

This is pretty old news by Internet standards, but now it is up and I can play with the search pages and maybe I can come up with some interesting statistics or experiment that others haven’t done yet. Speaking of which, I like KrazyDad’s statistics and write up of the data.