Since I’m a big fan of XKCD (and of PowerShell), I wrote a PowerShell script to download the entire XKCD archive. It’s not the most elegant script and to save time I re-used the Get-Webfile function from http://poshcode.org/417, but I thought I’d share it out broadly for the betterment of geekdom.
The entire script including the function is below. If you already have the
get-WebFile function loaded to your profile, you just need the last three lines which is just a bunch of HTML parsing regex.

function Get-WebFile {
param(
$url = (Read-Host “The URL to download”),
$fileName = $null,
[switch]$Passthru,
[switch]$quiet
)
$req = [System.Net.HttpWebRequest]::Create($url);
$res = $req.GetResponse();
if($fileName -and !(Split-Path $fileName)) {
$fileName = Join-Path (Get-Location -PSProvider “FileSystem”) $fileName
}
elseif((!$Passthru -and ($fileName -eq $null)) -or (($fileName -ne $null) -and (Test-Path -PathType “Container” $fileName)))
{
[string]$fileName = ([regex]'(?i)filename=(.*)$’).Match( $res.Headers[“Content-Disposition”] ).Groups[1].Value
$fileName = $fileName.trim(“/””‘”)
if(!$fileName) {
$fileName = $res.ResponseUri.Segments[-1]
$fileName = $fileName.trim(“/”)
if(!$fileName) {
$fileName = Read-Host “Please provide a file name”
}
$fileName = $fileName.trim(“/”)
if(!([IO.FileInfo]$fileName).Extension) {
$fileName = $fileName + “.” + $res.ContentType.Split(“;”)[0].Split(“/”)[1]
}
}
$fileName = Join-Path (Get-Location -PSProvider “FileSystem”) $fileName
}
if($Passthru) {
$encoding = [System.Text.Encoding]::GetEncoding( $res.CharacterSet )
[string]$output = “”
}
if($res.StatusCode -eq 200) {
[int]$goal = $res.ContentLength
$reader = $res.GetResponseStream()
if($fileName) {
$writer = new-object System.IO.FileStream $fileName, “Create”
}
[byte[]]$buffer = new-object byte[] 4096
[int]$total = [int]$count = 0
do
{
$count = $reader.Read($buffer, 0, $buffer.Length);
if($fileName) {
$writer.Write($buffer, 0, $count);
}
if($Passthru){
$output += $encoding.GetString($buffer,0,$count)
} elseif(!$quiet) {
$total += $count
if($goal -gt 0) {
Write-Progress “Downloading $url” “Saving $total of $goal” -id 0 -percentComplete (($total/$goal)*100)
} else {
Write-Progress “Downloading $url” “Saving $total bytes…” -id 0
}
}
} while ($count -gt 0)
$reader.Close()
if($fileName) {
$writer.Flush()
$writer.Close()
}
if($Passthru){
$output
}
}
$res.Close();
if($fileName) {
ls $fileName
}
}

$archivepage=get-WebFile http://www.xkcd.com/archive -Passthru
$a=[Regex]::Matches($archivepage, “< a href=./d+.*stitle.*< /a>” , “IgnoreCase”)|%{“http://www.xkcd.com”+[Regex]::Match($_, ‘/[0-9]*/’)}
foreach($i in $a) {$i+”:”;$curcomic=get-WebFile $i -Passthru;$comicurl=[Regex]::Match($curcomic, ‘http://imgs.xkcd.com/comics/[a-zA-Z0-9_()-]*(.jpg|.png)’ , “IgnoreCase”);get-WebFile $comicurl.Value}

A long long time ago, in an Internet far far way, in an age before Scoble, before Digg, before blogs and RSS, before Wikipedia and yes, even before Google, there was Slashdot.

At a time where I poked around on Yahoo and CNet with lynx trying to stay up to date on the latest tech news, some kind soul pointed me at Slashdot, saying “hey, you should check this site out, they have the latest tech news and Stuff that Matters”. Within a matter of weeks, I was hooked. I checked Slashdot multiple times a day and sometimes spent hours reading the comments on a particular important article or writing comments in trying to improve my Karma score. I was especially proud of my low 5 digit userid which makes me look like an old-timer now that userids are in the 7 digits!

Nowadays, I find myself reading Slashdot less and less. I rarely visit the site and it’s now just one of the dozen newsfeeds in my live.com homepage and one of the 100s of feeds in my Google reader feedlist. I remember discussing the causes for this with some other longtime slashdotter a few months back and was thinking about again this today as I saw the Google Chrome article on Slashdot a full 8 hours after I read about it elsewhere. Here’s my theory on why Slashdot is less relevant today than it was 4 or 8 years back…

Slashdot had three things going for it:

The latest News for Nerds and Stuff that Matters: I heard about some of the most important events in recent history – tech and otherwise – from Slashdot. About Apple’s fancy new music playing thingie, about the Windows’ source code leaks, Apple’s switch to Intel, about Bush defeating Gore and even about 9/11.

Over time, Slashdot has become less and less instantaneous as far as news goes. The still-manual approval process, while keeping noise low and editorial quality high, means that I’m more likely to hear about something new on digg, TechCrunch or in a barrage of new articles in my RSS reader than on Slashdot. Companies have corporate blogs where they announce their PR approved messages. Fan sites exist for every technology company, tech related product and area to help you follow your favorite tech areas in your favorite RSS reader, even if it means digesting leaked product shots and watching cameraphone videos of product demos that are faked more often than not. In essence, today, we (I?) would rather be buried by 200 possibly interesting news stories rather than wait for Slashdot to publish the one true story (and a dupe the next day).

Insightful comments and influential commenters:  The comments were always my favorite part of Slashdot. The quality of comments on Slashdot was (is still?) extremely high. Contrasting the long, thought out comments following a story on Slashdot with the one-liners on digg or or anyone else’s blog and drawing conclusions about their audiences from that is an exercise that some psychology or social behavior major should undertake for their thesis. To add to this, every other geek celebrity, from John Carmack (who I was a big fan of in my Quaking days) to Alan Cox, from ESR to Woz has commented on Slashdot at one time or another. I doubt many of them have Digg accounts 🙂

Over time, these commenters stopped frequenting Slashdot. Most of today’s geek celebrities and tech influencers have their own blogs where they publish their own raves and rants and have their own army of followers and sub-commenters who feed off of each other’s passingly insightful commentary. Comments that live on a single site are apparently soo-1999 in today’s world of webwide discussion. Hence, Slashdot commentary is dying a slow, uneventful death as everyone can comment everywhere and still be read and followed.

A sense of community and a wacky sense of humor: From the crazy CmdrTaco-flavored polls to being the site that spawned a thousand Internet memes, Slashdot had both a sense of community and an obscure sense of humor that has influenced me and probably thousands of other geeks.

With fewer comments and an aging audience, I suspect Slashdot’s audience and sense of community has mostly vaporized. The wacky sense of humor still exists, though I hear geeks referring more to XKCD these days than Slashdot for their obscure references.

Ok, this post was way longer than I ever intended it to be. I’ll blame it on the post-labor day, post-summer blues 🙂 The meta-point of this post is still what I started if off with – that web 2.0 killed Slashdot, the same way Web 1.0 killed BBSes and the Usenet.

-Umesh

PS:

1. I used to follow John Carmack’s .plan file back in the day. His thoughts on DirectX vs OpenGL were all the rage around the DX6/7 timeframe.

2. It was a difficult choice on who to link to for the geek celebrity and tech influencer terms above. I made a last minute swap 🙂

3. I know, I know this is a common debate… especially in the days when Digg was gaining in popularity, but I felt I had to write this up to get my thoughts on paper.

4. Oh yeah, I was slashdotted once. Not for something I was particularly proud of, so I didn’t link to it here.