Since I’m a big fan of XKCD (and of PowerShell), I wrote a PowerShell script to download the entire XKCD archive. It’s not the most elegant script and to save time I re-used the Get-Webfile function from http://poshcode.org/417, but I thought I’d share it out broadly for the betterment of geekdom.
The entire script including the function is below. If you already have the
get-WebFile function loaded to your profile, you just need the last three lines which is just a bunch of HTML parsing regex.

function Get-WebFile {
param(
$url = (Read-Host “The URL to download”),
$fileName = $null,
[switch]$Passthru,
[switch]$quiet
)
$req = [System.Net.HttpWebRequest]::Create($url);
$res = $req.GetResponse();
if($fileName -and !(Split-Path $fileName)) {
$fileName = Join-Path (Get-Location -PSProvider “FileSystem”) $fileName
}
elseif((!$Passthru -and ($fileName -eq $null)) -or (($fileName -ne $null) -and (Test-Path -PathType “Container” $fileName)))
{
[string]$fileName = ([regex]'(?i)filename=(.*)$’).Match( $res.Headers[“Content-Disposition”] ).Groups[1].Value
$fileName = $fileName.trim(“/””‘”)
if(!$fileName) {
$fileName = $res.ResponseUri.Segments[-1]
$fileName = $fileName.trim(“/”)
if(!$fileName) {
$fileName = Read-Host “Please provide a file name”
}
$fileName = $fileName.trim(“/”)
if(!([IO.FileInfo]$fileName).Extension) {
$fileName = $fileName + “.” + $res.ContentType.Split(“;”)[0].Split(“/”)[1]
}
}
$fileName = Join-Path (Get-Location -PSProvider “FileSystem”) $fileName
}
if($Passthru) {
$encoding = [System.Text.Encoding]::GetEncoding( $res.CharacterSet )
[string]$output = “”
}
if($res.StatusCode -eq 200) {
[int]$goal = $res.ContentLength
$reader = $res.GetResponseStream()
if($fileName) {
$writer = new-object System.IO.FileStream $fileName, “Create”
}
[byte[]]$buffer = new-object byte[] 4096
[int]$total = [int]$count = 0
do
{
$count = $reader.Read($buffer, 0, $buffer.Length);
if($fileName) {
$writer.Write($buffer, 0, $count);
}
if($Passthru){
$output += $encoding.GetString($buffer,0,$count)
} elseif(!$quiet) {
$total += $count
if($goal -gt 0) {
Write-Progress “Downloading $url” “Saving $total of $goal” -id 0 -percentComplete (($total/$goal)*100)
} else {
Write-Progress “Downloading $url” “Saving $total bytes…” -id 0
}
}
} while ($count -gt 0)
$reader.Close()
if($fileName) {
$writer.Flush()
$writer.Close()
}
if($Passthru){
$output
}
}
$res.Close();
if($fileName) {
ls $fileName
}
}

$archivepage=get-WebFile http://www.xkcd.com/archive -Passthru
$a=[Regex]::Matches($archivepage, “< a href=./d+.*stitle.*< /a>” , “IgnoreCase”)|%{“http://www.xkcd.com”+[Regex]::Match($_, ‘/[0-9]*/’)}
foreach($i in $a) {$i+”:”;$curcomic=get-WebFile $i -Passthru;$comicurl=[Regex]::Match($curcomic, ‘http://imgs.xkcd.com/comics/[a-zA-Z0-9_()-]*(.jpg|.png)’ , “IgnoreCase”);get-WebFile $comicurl.Value}

A long long time ago, in an Internet far far way, in an age before Scoble, before Digg, before blogs and RSS, before Wikipedia and yes, even before Google, there was Slashdot.

At a time where I poked around on Yahoo and CNet with lynx trying to stay up to date on the latest tech news, some kind soul pointed me at Slashdot, saying “hey, you should check this site out, they have the latest tech news and Stuff that Matters”. Within a matter of weeks, I was hooked. I checked Slashdot multiple times a day and sometimes spent hours reading the comments on a particular important article or writing comments in trying to improve my Karma score. I was especially proud of my low 5 digit userid which makes me look like an old-timer now that userids are in the 7 digits!

Nowadays, I find myself reading Slashdot less and less. I rarely visit the site and it’s now just one of the dozen newsfeeds in my live.com homepage and one of the 100s of feeds in my Google reader feedlist. I remember discussing the causes for this with some other longtime slashdotter a few months back and was thinking about again this today as I saw the Google Chrome article on Slashdot a full 8 hours after I read about it elsewhere. Here’s my theory on why Slashdot is less relevant today than it was 4 or 8 years back…

Slashdot had three things going for it:

The latest News for Nerds and Stuff that Matters: I heard about some of the most important events in recent history – tech and otherwise – from Slashdot. About Apple’s fancy new music playing thingie, about the Windows’ source code leaks, Apple’s switch to Intel, about Bush defeating Gore and even about 9/11.

Over time, Slashdot has become less and less instantaneous as far as news goes. The still-manual approval process, while keeping noise low and editorial quality high, means that I’m more likely to hear about something new on digg, TechCrunch or in a barrage of new articles in my RSS reader than on Slashdot. Companies have corporate blogs where they announce their PR approved messages. Fan sites exist for every technology company, tech related product and area to help you follow your favorite tech areas in your favorite RSS reader, even if it means digesting leaked product shots and watching cameraphone videos of product demos that are faked more often than not. In essence, today, we (I?) would rather be buried by 200 possibly interesting news stories rather than wait for Slashdot to publish the one true story (and a dupe the next day).

Insightful comments and influential commenters:  The comments were always my favorite part of Slashdot. The quality of comments on Slashdot was (is still?) extremely high. Contrasting the long, thought out comments following a story on Slashdot with the one-liners on digg or or anyone else’s blog and drawing conclusions about their audiences from that is an exercise that some psychology or social behavior major should undertake for their thesis. To add to this, every other geek celebrity, from John Carmack (who I was a big fan of in my Quaking days) to Alan Cox, from ESR to Woz has commented on Slashdot at one time or another. I doubt many of them have Digg accounts 🙂

Over time, these commenters stopped frequenting Slashdot. Most of today’s geek celebrities and tech influencers have their own blogs where they publish their own raves and rants and have their own army of followers and sub-commenters who feed off of each other’s passingly insightful commentary. Comments that live on a single site are apparently soo-1999 in today’s world of webwide discussion. Hence, Slashdot commentary is dying a slow, uneventful death as everyone can comment everywhere and still be read and followed.

A sense of community and a wacky sense of humor: From the crazy CmdrTaco-flavored polls to being the site that spawned a thousand Internet memes, Slashdot had both a sense of community and an obscure sense of humor that has influenced me and probably thousands of other geeks.

With fewer comments and an aging audience, I suspect Slashdot’s audience and sense of community has mostly vaporized. The wacky sense of humor still exists, though I hear geeks referring more to XKCD these days than Slashdot for their obscure references.

Ok, this post was way longer than I ever intended it to be. I’ll blame it on the post-labor day, post-summer blues 🙂 The meta-point of this post is still what I started if off with – that web 2.0 killed Slashdot, the same way Web 1.0 killed BBSes and the Usenet.

-Umesh

PS:

1. I used to follow John Carmack’s .plan file back in the day. His thoughts on DirectX vs OpenGL were all the rage around the DX6/7 timeframe.

2. It was a difficult choice on who to link to for the geek celebrity and tech influencer terms above. I made a last minute swap 🙂

3. I know, I know this is a common debate… especially in the days when Digg was gaining in popularity, but I felt I had to write this up to get my thoughts on paper.

4. Oh yeah, I was slashdotted once. Not for something I was particularly proud of, so I didn’t link to it here.

There must be something about summer in Seattle that slows my blogging down. It’s been almost 3 months since I wrote anything at all. Looking through my blog I also noticed that I my blogging frequency was low most of last summer as well culminating in a mid-summer update exactly 365 days ago 🙂

Anyway, life has been busy over the past few months both at work and outside of it. Stuff I’ve been upto:

• Work: Nothing major to report here, though we’re getting close. The next year should be exciting, though 🙂

• Travel: In May, the wife & I hauled ourselves across the little pond called the Atlantic and on to Old World. I’ve been meaning to write up a long, detailed post about our trip (and even wrote up a few hundred words on about our first day on the flight back), but I don’t think I’ll ever get to completing it. Given that, here’s a quick summary of the trip, for posterity’s sake.

First, the logistics: We spent 8 days, across the memorial day long weekend, in Europe. We flew in and spent 3 days in Rome, took a train to Venice and spent a day there and then flew to Paris to spend 4 days (5 nights) there to fly back on the 5th day.

Rome:

We flew into Rome on an uneventful, if cramped, American Airlines flight via New York JFK. We had a classic ‘Welcome to Italy’ moment as we landed and we realized that the passport control line stretched all the way to the exit gates. It turns out 5 or so international flights had landed at the time and Rome’s Leonardo Da Vinci international airport was ill-equipped and understaffed to handle such a load (especially at 8am on a Sunday morning). That was probably the low point of the trip and the rest of our stay in Rome was amazing – from the ruins of the Roman forum to the Museums at the Vatican to lazy evening walks around the Piazza Navona, we had a ball in Rome and probably loved it the most of the three cities we visited on the trip.



Venice:

Venice is probably the most overhyped tourist location in the world. The canals and the decaying old buildings were amazing for a total of 20 minutes before we realized that we had to walk everywhere or take the slow water taxies 🙂 I described Venice to others as a maze filled with tourists and many agreed. We spent a night in Venice and spent the day at the Doge’s Palace and St Mark’s square. It was fun, but only for a day.



 

Paris:

Given how hectic the first part of our trip in Italy was, I wanted to take the it easy in the second part and so, we stayed in Paris for 5 nights. Chilling out, in no hurry, watching TV till 2 in the morning and getting up at noon to check out the Louvre and the Eiffel tower. Paris was beautiful, busy, expensive and classy. But I still preferred Rome.







Versailles:

Versailles was a day trip from Paris and we took the train there early on a Sunday morning and were back in Paris in the evening. Versailles was probably what we liked the most in France. Touring through the palace and later, though Marie Antoinette’s Domaine, we were awed by the opulence and grandeur with with the French monarchs (and their groupies) lived.

 

• More Travel: Now, if flying halfway around the world for a whirlwind tour of the Old World wasn’t hectic enough, I decided to go camping in Montana during the Independence Day long weekend. D bailed on this one, so it was Ullas, me & 7 others. This trip was waay more hectic than our last trip to Montana, but it was arguably more fun inspite of the fact that I got 12 hours of sleep during the entire long weekend (or probably because of it) 🙂

Photos from the trip:

• Photography: If it wasn’t obvious from the photos so far in this post, I’ve picked up photography in a big way over the past year. I’ve been reading and practicing a lot (10000+ photos in the past 12 months) and will probably write a blog post sometime on the stuff I’ve learned (and have yet to learn). My favorite shot so far:

Stuff I’ve not been upto:

• Blogging: Self-explanatory.
• Staying fit: I’ve reneged on my plan to run the Seattle marathon this year since I got only 2 or 3 days of running before laziness set in. I’ve also done no biking this year which makes me worried about how my legs will survive the ski season. Note to self: Fix bike up and get back on the road.
• Gaming: (Looks at my unopened copy of GTA4). ’nuff said.

Other rambling thoughts:

• My Roomba died (as in stopped sucking) almost a year back. It’s still in my living room charging itself and running around once in a while. Must figure out what to do with it next – robotic pet project?
• I finally solved the HD PVR conundrum by getting myself a TivoHD. The news, at 1080i, is beautiful 🙂 So, is Discovery HD and other channels, but somehow the local evening news looks extra sharp.

In the otherwise copycat world of Social Networking, it’s interesting to see LinkedIn do something new:

http://www.techcrunch.com/2008/03/20/linkedin-now-for-companies-2/

On Friday morning they will launch company profile pages that partly serve as fact sheets for about 160,000 companies and partly serve to reveal the connections that members have with them.

…bulk of the data shown on these company pages comes from LinkedIn’s own knowledge of people’s careers…

LinkedIn uses this knowledge to display recent hires, related companies, recent promotions, top locations for employees…

…you can see which companies employees usually come from and leave for, as well as which companies the current employees are most connected to…

I’ve always felt that the real ‘moat’ that social networking sites have is the data they hold about their users and the relationships. I suspect this data has been mined so far for generating more targeted advertising, but LinkedIn is the first network I’ve seen that’s exposing this data out in a way that’s interesting for end users:

As an example, here’s what the Microsoft profile shows me about the average Microsoft employee:

and here’s the corresponding page for Google (http://www.techcrunch.com/wp-content/linkedinbiz_shot.png):

Now, who would have thought that the most common career path for Google employees is to join (return to?) Yahoo and Microsoft!

Random Fact of the Day:

A tidbit that you can learn only by poring over random Wikipedia articles:

‘Jan‘ from The Office (NBC TV show) is the sister of the CEO of Flock (as in the browser company).

It’s all one big happy family I guess.

Random Quote of the Day:

“Oh, I get it! Silicon Valley is like Hollywood, but for nerds, and without the chicks” – from a commenter on Valleywag (which in my humble opinion has started to read like a tired rag these days).

Random Find of the Day:

http://reddit.com/user/FunFact – this guy adds a comment to every top-voted reddit submission with a ‘fun fact’ about a topic in the article. He’s gotten so popular that there’s another guy (http://reddit.com/user/did_you_know_that/) copying him.

It’s been a while since I posted anything, so I figured I’d resume on a light note 🙂 On the bright side, my HD PVR conundrum was solved yesterday and I have a Tivo HD on the way thanks to woot.com!

After reading a few reviews (and partly because I was in the mood for some change {lame, I know}), I decided to ditch BlockBuster Online and switch to Netflix for my movie fix. I’ve been with BBO almost since they started (Sep ’04?) so this was a pretty drastic move! In a few hours of playing around, I’ve found a few things I like about NFLX:

• Multiple profiles: Now, the wife & I can both have our own queue without one of us hogging the movie queue.
• Stream to PC: This is the one thing I’m most exited about – not because I want to go from watching movies from my couch on a 40″ screen to sitting at a computer chair watching movies on a 19″ screen, but because I’m fairly certain that this is just a pre-cursor for Netflix providing movie downloads on the XBox.

I’m not too impressed by Netflix’s UI yet – a couple of things that annoy me  – for one, why doesn’t the search box just take to me the movie I’m searching for when it offers me suggestions:

It offered me the right movie in the dropdown, but when I select it, it does a search on the term rather than take me to the movie directly. All that AJAXy autocomplete goodness wasted!

The other annoyance: Why is this popup displayed for every movie I add to my queue?

BBO used to let me bulk-add movies to my queue without any annoying popups! I’ll probably also miss the occasional game rental and in-store exchange from Blockbuster, but I’m hoping the larger selection and stream to PC features make up for it!

Here’s wishing everyone who sees this entry a happy new year 2008. In the spirit of a light and lively new year, here’s a new year greeting in legalese I got from someone at work:

Please accept with no obligation, implied or implicit, my best wishes for an environmentally conscious, socially responsible, low stress, non-addictive, gender neutral celebration of the winter solstice holiday, practiced within the most enjoyable traditions of the religious persuasion of your choice, or secular practices of your choice, with respect for the religious/secular persuasions and/or traditions of others, or their choice not to practice religious or secular traditions at all…and a fiscally successful, personally fulfilling, and medically uncomplicated recognition of the onset of the generally accepted calendar year 2008, but not without due respect for the calendars of choice of other cultures whose contributions to society have helped make this country great (not to imply that this country is necessarily  greater than any other country), and without regard to the race, creed,  color, age, physical ability, religious faith, or sexual orientation of  the wishee.

This wish is limited to the customary and usual good tidings for a period of one year, or until the issuance of a subsequent holiday greeting, whichever comes first. “Holiday” is not intended to, nor shall it be considered, limited to the usual Judeo-Christian celebrations or servances, or to such activities of any organized or ad hoc religious community, group, individual or belief (or lack thereof).

Note: By accepting this greeting, you are accepting these terms. This greeting is subject to clarification or withdrawal, and is revocable at the sole discretion of the wisher at any time, for any reason or for no reason at all. This greeting is freely transferable with no alteration to the original greeting. This greeting implies no promise by the wisher to actually implement any of the wishes for the wishee her/himself or others, or responsibility for the consequences which may arise from the implementation or non-implementation of same. This greeting is void where prohibited by law.

I finally gave in to geek pressure and decided to get ourselves a new TV. The gorgeous Samsung LNT4065F. 40″, LCD, 1080p. The TV’s due to arrive later this week or early next, so I thought I’d do some early research to figure out how to get the most out of it.

My current setup is a relatively ancient 27″ Sony flat-tube TV we bought as soon as we moved to Seattle back in ’04, so this is a significant upgrade in terms of screen size (27->40), technology (analog SD tube -> digital HD LCD) and most significantly, resolution (480i -> 1080p).

Our primary use of the TV is to play DVDs (thanks to BlockBuster Online and iTalkies), TV (via our trusty Series 2 Tivo and basic Comcast cable) and an XBox 360 Premium that I stood in line for 8 hours to buy on launch day.

The only HD-capable device on this list is the XBox 360 – capable of doing 1080p games and 1080i movies over component and 1080p movies over VGA. So, this one was easy – I picked up a Xbox VGA cable for cheap that I can use to get upscaled DVD playback and I’m all set of experience movies and gaming at full HD. I thought about getting the XBox HD-DVD player given the great deals we’re seeing on them this holiday season, but figured that it wasn’t wise to invest on either side of the HD format wars till there was a clear winner. Moreover, only some movies are available in HD-DVD and others are in Blu-Ray, so it didn’t make sense to confine myself to a single format till dual-format players become more common down the road.

My DVD player – a Panasonic home-theater-in-a-box from the same bygone era – doesn’t upconvert DVDs to 1080i. I figured I’ll let this one slip and keep my current DVD player, especially since it supports 480p and use the XBox for DVD playback when I really wanted upconverted video.

Cable – this one’s tricky. I don’t have digital cable , so the only HD-channels I can get right now in HD are the local channels through the TV’s built-in QAM tuner. The bigger problem for me, even if I do switch from cable to Satellite or OTA/ATSC, is how to get a PVR solution that supports HD video. The Series2 Tivo that I’ve owned for the past couple of years clearly doesn’t support HD-video of any kind. My significant other is addicted to the Tivo and would immediately disapprove of any solution that involves losing the PVR functionality (I know – I asked!).

So, my options are:

1. HD Tivo: Luckily for me, Tivo launched it’s HD-capable Series 3 last year. More recently, they launched the scaled down and cheaper HD Tivo. This PVR supports upto 20 hours of recorded HD content from OTA or digital cable (via the CableCard). Sounds perfect, except, that it costs $249 + $12.95/mo. That puts it out of this year’s tech budget!
2. Media Center PC: As I detailed in a series of earlier posts, I put together a killer quad core PC earlier in the year that runs Vista Ultimate and Windows Media Center. I’ve been using the XBox as a Media Center extender for a few months which works pretty well. So, all I need to do is get a TV tuner that supports HD and I should be all set, right? It turns that life is not that simple. Most HD tuners support decoding only QAM or ATSC signals – i.e. only free-to-air channels can be decoded. There are a handful of TV tuner cards that support CableCard so that digital cable channels can also be decoded, but get this – those cards are only sold integrated into a new PC!!! That’s right – you can’t just buy a CableCard capable decoder and plug it into your PC – you have to buy a cable-industry Certified PC that in their words “would absolutely be able to deliver on the wide range of things that you couldn’t predict with certainty would happen on a home-built PC“. Hmm.. we all know what means -thanks to DRM requirements, there’s no way a home-built PC can use a CableCard tuner right now! So, my HD-MCE dreams end there – even if I wanted to pay ComCast the extra $5 a month for the CableCard!
3. Comcast’s HD-PVR: Comcast gives^H^H^H^H^H rents an HD-capable PVR (nee DVR) for the low low price of $13.95/mo. This sounds like a good option till you read reviews of the DVR and realize that it’s a piece of cr*p. I’ll pass, and wait till Comcast starts offering Tivo’s which they’ve started doing in some regions already.
4. Do nothing: The PM in me decided to go with this option for now. As discussed above, all options have significant downsides or costs that do not align with current priorities 🙂 Therefore, I decided to do nothing for now and be content with SD cable programming and the Tivo Series 2 till such a time when the above issues are resolved. Tivo prices are sure to drop in the next few months and Comcast will probably start offering Tivo’s in the Seattle region sometime next year.

So, I’ll just wait a few months till options 1 or 3 above become viable. This is definitely the best option given current realities…

PS: The fall update for the XBox 360 introduced support for viewing Xvid/DivX streaming video. So, a 5th option, of questionable appropriateness is to download err.. place/time-shifted videos from that European site that charges € 1/month. That’s a viable option for viewing the occasionally TV show at 720p as long as € 1 < $13.95 :-)