What's That Noise?! [Ian Kallen's Weblog]

All | Ruby | Music | Java | What's That Noise? | Commute | The Agilist | Ball | LAMP

20070406 Friday April 06, 2007

Embracing Hats At Technorati

Wow! I'm sure the executive search announcement (Embracing Change) that Dave Sifry just posted will elicit a range of responses from the blogosphere. While I expect a lot of speculation and innuendo around it, I probably have a unique perspective owing to my three years working with Dave at Technorati (this week was my third anniversary). I'd like to share some of that perspective.

It started for me in the Spring of 2004, I was trying to figure out to do next. At the time, I was frankly very skeptical of Technorati when a friend suggested that I make that my next stop. Most of the time that I visited, the website was unavailable or had PHP errors all over the place, what a trainwreck! Maybe I can fix it.

What I expected to be a short conversation with Dave that fateful day in March 2004 turned into one that lasted for hours. After much discussion about the impact technology developments have on publishing and social discourse, I was struck by Dave's insight, inspiration and passion. In turn, I committed myself to taking what I knew about scaling web sites, learning what ever I needed to scale for the blogosphere's unique requirements and fixing the technical problems that plagued Technorati; I joined Dave to make Technorati the real time engine that would provide micropublishers with the connective tissue of community.

In the years since then, I've worn many hats. Software developer, DBA, sysadmin or whatever-it-takes; I came to Technorati determined by-any-means-necessary to sustain and improve Technorati's state of the art. The company has grown (there were about five us back then). The blogosphere has grown (there were only a few millions blogs back then). And all of us working together at Technorati have grown as people. Today, I still collaborate closely with Dave and Adam. I lead the Core Services group and work with our fabulous front end (led by Dorion) and search engineering (led by Brian) teams as an architect of Technorati's evolution. In helping lead the reshaping of Technorati's infrastructure, we've sought the right path between oft conflicted goals of flexibility, economy and run time optimization. We haven't always gotten it right. Technorati's storied episodes of instability and poor performance were often the source of much sleepless grief and perhaps opportunity costs. There have been business directions that have led our attention up some blind alleys. There have been technical errors. Project execution errors. Hiring errors. And so on. But, if I may add without being too immodest, we've done a lot of things well; I think ultimately the right things have happened. It is an honor and privilege to work with these folks, except for how wonderful my kids are, I couldn't be prouder of them!

So that brings us to where we are today and my ongoing working relationship with Dave. Dave's not going anywhere else. He may do some hat trading. But I expect to enjoy the benefits of working with him... for as long as I can!

Through good times and bad, Dave provides vision and inspiration. Synthesizing new ideas, asking tough questions and supporting the creativity and enthusiasm of all us working on Technorati, Dave is a catalyzing force. Among the principles Dave demonstrates that are important to me is internal transparency. Being open with factual matters about the company, the markets we address and the competitive landscape enables myself and others at Technorati to contribute their smarts and creativity in ways that a closed environment would never benefit from. Because that openness isn't always extended to parties outside the company, Technorati has been accused of being secretive. Well, sure. We don't answer rumors or trumpet funding events. But being judicious about external transparency is part of life; we could spend all day fielding the probes and inqueries from outside parties but to what benefit?

Throughout my years working with Dave, we've made it a point to hire only great people at Technorati. "Good" isn't good enough and "can do the job" can't; we only hire great people, period (and we've passed on a lot of good people who could ostensibly do the job if there was doubt about their greatness). It seems entirely plausible that there may be great people other than Dave who can wear the CEO hat better than he can. There might not be. I support Dave's launch of this search to find out. I'm looking forward to meeting this person; if he or she exists, there are some incredible shoes to fill (and hats and a buncha good stuff in between)! However, I expect to continue collaborating with Dave irrespective of if or when there is another CEO. Regardless of what hat he's wearing and which one I am, I'm looking forward to working with Dave towards Technorati's continued success.

So, wow! I'm posting this as an embrace of Dave, a virtual hug and an assurance of my unflagging support.

( Apr 06 2007, 12:16:22 PM PDT ) Permalink View blog reactions


20070401 Sunday April 01, 2007

Test Driven Damnation

The long drive down to Visalia didn't quite go as planned. Since my daughter is doing a Shakespeare theater camp this summer, I figured listening to a "Romeo and Juliet" dramatization for three and half hours on the road would provide enriching entertainment. But by the time we got to The Five, I sayeth unto myself, as if Yoda did Shakespeare, "Patience for lengthy dialog in British accents, thou hath not." Eject that CD. We ended up yacking and partaking in Dave Matthews, Green Day and Queen instead.

Besides the California state Odyssey of the Mind competition this weekend, Visalia seemed to be hosting a soccer event and a gathering of evangelical christians. I'm sorry, but really. The latter are very silly people, they take themselves and their world view way too seriously. But I guess if you're convinced you have a divine and ultimate truth, that could happen.

Odyssey of the Mind is a creative problem solving competition. It's an exercise in questioning boundaries.

Evangelicals seem to think their prophetic fantasy has to unfold, and I... have some problems with that. Having a grand Test Case scripted in advance for all validation to be held against is a set of boundaries that are probably only best applied to technology development. Test-driven damnation is just too wacky for moi. At least some of the evangelicals have a sense of humor, evidenced by the license plate frames and bumper stickers in a Visalia hotel parking lot:

Maybe those were actually Odyssey families who've tired of their "My Boss is a Jewish Carpenter" sloganed regalia. Whatever. Buick, Cadillac, Jaguar: Jahweh or the highway.
Um. Anyway.

On to the outcomes. My daughter's Odyssey team has, for the second year in a row, achieved victory in the state arena and are headed to the World Finals. I understand that the Michigan State campus will be much more of a class act than last year's in Ames, Iowa (there's really not far to go to make that claim).

OK, perhaps I'm not doing too good tonight. Since I've only written a few words but have already managed to offend my Iowan, British and Christian friends, perhaps I should exit my Highway to Hell and call it a night. Forgive me while I grasp at excuses: too much driving and sleep deprivation precipitates rambling snark.

But seriously folks, I'm still blown away by the kids' victory, they're a young team for division II (four 5th graders, 3 in 6th). I was keeping the faith but also expecting to quietly go home and have that be the end of the OM road. However, their long term scores dominated the competition despite a poor showing in spontaneous. East Lansing, here we come!

                   

( Apr 01 2007, 10:06:07 PM PDT ) Permalink View blog reactions


20070329 Thursday March 29, 2007

WiFi on BART

I wasn't in the station long enough to use it but my laptop picked up a signal from Wifi_Rail_BART at the Embarcadero BART station. Hmmm... perhaps this is a new service that BART is rolling out. Now if they'd just install access points uplinked to EVDO connections on the trains, that'd be awesome! I'd heard that there were RFP's out last year from Amtrak to provide WiMax to the Capitol Corridor line, that's cool. But there's a much larger population of BART riders who I'd imagine would be as grateful as I would be to read feeds, mail ...and access source control.

             

( Mar 29 2007, 08:02:22 PM PDT ) Permalink View blog reactions


20070328 Wednesday March 28, 2007

I'm calling "Bullshit!"

I've been using social software for a few decades now. It started with dial-up BBS' (300 baud modems worked pretty good when you had tweets of text) and compuserve (user ID's were numbers and commas, IIRC). In the 1990's I used usenet and The WeLL to great delight, chagrin and all sorts of other things. In all of these goings about, the motivations to participate have ranged from voyeuristic curiosity, to chit-chatting and connecting, to utilitarian knowledge sharing, to being of service to others, to showing off in front of others or something else altogether social.

Lately, it seems to me that the blogosphere is buckling under the corrosive forces of greed. You can't trust voices to be authentic. PR firms and marketing departments are hiring bloggers or ghost writing for pretenders. You have buffoons (who may describe themselves as a "Dot Com Moguls") selling links to other blogs with prices based on their Technorati and Alexa ranks. There are all of these schemes like ReviewMe, Algoco and PayPerPost twisting our trust of the things we read. Is that blogger really talking about a great bike or was there some payola slipped in to pimp it? I don't know. My cynicism rises (further).

What's a "pro blogger" anyway? Are folks really discussing or dialing for dollars? SEO is increasingly turning publishing into a perverse sport. If I wanted to watch infomercials, Comcast is already piping a couple of hundred channels of crap through the coax in my living room, I could passively drink the nonsense all day if that's what I wanted.

But that's not what I want. It's a total cluetrain derailment. Instead of using social media to empower meme sharing and synthesis, as a way to expand the collective consciousness, I'm seeing it turned into another exploitation ploy.

Our country is in a brain-dead war, podcasts are published, urban public schools are in shambles, photos are being uploaded, illiteracy is rampant, a funny thing happened at the dog park, baseball season is starting soon, technology innovations are blossoming in our midsts, pandemics threaten us every flu season, tuition is going up (again), omega-3 fatty acids, look at this video of a 4-way tricycle collision, Osama is still who-the-hell-knows-where, Dick Cheney is still satan, Jimi Hendrix is still god, god is still dead, lives are beginning and ending all around us... lives are being lived. The rhythmic thrum of life, that's what I want.

(Disclosure: there are links on this site from AdSense and Amazon. They don't do much for me but then, I don't think they should, that's not why I blog. If you click 'em: thanks, maybe I'll get pizza next week)

Putting a few ads or Amazon links to benefit from the accidental tourist that Google sends you from time to time is one thing. Superficially making it the object of your publishing is just, well, bullshit!

BTW, this is not necessarily the opinion of my employer. In fact, it's probably not. I may be a cynical bastard but this is my authentic voice. You can't buy it.

( Mar 28 2007, 09:52:17 PM PDT ) Permalink View blog reactions


20070319 Monday March 19, 2007

Disintermediation

If you've seen Charlie Hunter play guitar, you know he's just fantastic. I first saw him play at The Nightbreak on Haight Street about 16 years ago or so (Primus was headlining, Kirk Hammett jammed with Primus too that evening, IIRC). I thought about how it's been too long since I've seen Charlie Hunter play whilst putting down some alaska rolls last night at Yoshi's. I missed Charlie Hunter's shows there last December. But now I don't have to miss them.

Here's a sample:

But that's not a recording you'd take with you.

However, you can buy a recording of the performance directly from the artist, Charlie Hunter Trio: Recorded live at Yoshi's in Oakland, CA (12/15/2006). I confess to knowing little about Fast Atmosphere. But I do know the media establishment needs a severe beating with a clue-stick. Providing services for recording artists that enhances their ability to sell their creations directly to their audience sounds like an appropriate truncheon. Fast Atmosphere has other Charlie Hunter recordings (and more names you might know, like Norah Jones and others you might not), check them out and support direct-sale art.

Next time Charlie Hunter stops at Yoshi's, I'm there.

             

( Mar 19 2007, 09:54:29 PM PDT ) Permalink View blog reactions


20070318 Sunday March 18, 2007

Space-Time Continuum Continued

1979: Black Flag played at the Temple Beautiful
2007: Henry Rollins is on Twitter


       

( Mar 18 2007, 08:08:31 AM PDT ) Permalink View blog reactions


20070315 Thursday March 15, 2007

Deja Vu

Do you ever meet somebody who reminds you of someone else and have to remind yourself, "No, don't mix up the two"? An odd moment fell upon me when I accidentally found myself watching Some Kind of Monster on TV the other night. There was a flashback. And then a flashforward. Ya see, when I was a teenager, our crowd, we hated the music industry and wanted to subvert it by any means necessary. Disco music and the just-following-orders spineless vermin that ran the press establishment and radio stations (the program managers) amused and sickened us. I had a radio show on KUSF and actively tapped underground music scenes, looking for that undiscovered stroke of worldchanging noise. Perhaps my efforts of the last 15 years or so building applications on the web are an extension of the same jaded disdain I have for the media establishments. They're in-bred pay-for-play following-orders brain-washers. Let me tell you how I really feel. Complacency is not an option.

I haven't seen them for a few years but when the lot of us were teenagers, I used to fervantly headbang with my pals in Metallica. They lived in a house on Carlson Street in El Cerrito. We drank a lot. And listened to New Wave of British Heavy Metal acts like Tank and Angelwitch. And though they weren't Britons, we listened to a lot of Merciful Fate, too. Here's a picture of us, long straggly hair, pimples, listening to Venom no doubt and expressing a whole cargo ship full of fervor.

Photo courtesy of umlaut, thanks again Brian!
Fast forward to the 21st century. In the past few years, I've run into Jeff Veen a number of times in professional and social situations. I don't hob-nob in the web 2.0 "scene" much but when I do, there's Veen. He's not famous for too-loud/too-fast/too-go-f*k-yerself sounds of pummeling. He's not famous for alienating fans trading his goods on peer to peer networks. But he's known for his web design sophistication. He's usually a bit happier, too. See, look at the picture. Happy guy!

What does Jeff Veen have to do with James Hetfield? "He looks like James Hetfield of Metallica." I mean, they both have first names that start with "J". Coincidence? I think not.

Hetfield rides tricked out choppers. I don't make it to San Rafael much except to visit Bubbe (I hope I make it through the long haul the way she has!). But if I lived in San Rafael I'd probably run into Hetfield all over the place; our kids are similar ages so we'd probably share Thin Lizzy on a boombox while the offspringlings are in the sandbox. When I was living in The City, I'd run into Kirk Hammett all of the time (like, the vegetable isle at Whole Foods).

On the other hand, I work in the South-of-Market part of The City and Veen is oft spotted tooling around in a mini-cooper. I don't think he's into Merciful Fate or Thin Lizzy. But then, I've never asked him.

"Hey, what's James doing driving 'round here in that? Oh, yea... that's Jeff Veen."

Hetfield and company have composed sonic booms such as "The Frayed Ends Of Sanity" (And Justice For All)

Veen on the other hand published about The Art and Science of Web Design

Hetfield goes on stage and exhorts about anger, misery and suffering. 50 thousand pairs of finger-and-pinkie horns are raised in response. And 50 bucks will get you a t-shirt.
Veen goes on stage, dressed a little smarter and discusses design principles. 50 million web pages get their style sheets revised (a subject I'm inept with, judging from my markup skillz). I don't know about the web design merchandising potential, would you pay 50 bucks for a microformats t-shirt?

Hetfield doodles on his guitars.

Veen draws Venns.


So they're about the same height and share some conversational mannerisms. Maybe their hairlines are about the same. The spectacles and goatee look might do it, too. But (rational voice) these guys are nothing alike and the context I know them from are completely different. And yet the introverted sensing part of me always draws the association when I run into one of them face to face or flipping TV channels and stumbling upon VH1; there is the odd familiarity of Hetfield when I run into Veen. I should invite Veen to share a bottle of stolichnaya and crank up Black Sabbath to see how he takes to it, maybe it'd just come naturally.

Silly grins. Coincidence? I think not.
  < Hetfield   |   Veen >

Of course, Metallica did change the music industry. But it changed them, too. And damn, I've got it coming and going: young enough to get pimples still but old enough to be all salt-n-peppa gray on top. OK, 'nuf musing. Back to subversion.

         

( Mar 15 2007, 11:52:20 PM PDT ) Permalink View blog reactions


20070314 Wednesday March 14, 2007

Sprung

Clocks have received (or are overdue for) day-light savings time adjustments, flags with Matt Cain and Barry Bonds are on the lamp posts of 3rd Street, kids are at play; recreational softball/baseball on the weekend and Odyssey of the Mind state championships are coming up. The hills are green and blossoms and blooms abound. Spring has sprung.

Have a nice day!

( Mar 14 2007, 01:30:52 PM PDT ) Permalink View blog reactions


20070225 Sunday February 25, 2007

The OpenID Snowball

In case you haven't noticed, there's been an unmistakable groundswell around OpenID in recent months. The proliferation of new web 2.0 services and the resulting "password fatigue" (except for those who are using OpenID) are contributing mass to the movement but the adoption of OpenID by established services (Technorati, Digg) and big players (AOL, Microsoft) are catalyzing acceleration. I'll cite these headlines as evidence:

Tim Bray raised a lot of great questions about OpenID. I'll summarize a response by simply saying that one of the virtues of using OpenID for URL-based user-centric authentication is the granularity of control available, it's all "opt-in":

  1. relying parties can maintain their own white/black listing policies for identity providers, define what user attributes they require, etc
  2. users are in control of who they allow their identity provider to provide URL ownership verification to, which of their attributes are allowed to be shared, etc
  3. the identity provider can implement policies around which relying parties they'll authenticate for or send user attributes to

Given current implementations, I'm probably not ready to use OpenID for online banking or verification that I'm old enough to buy wine (but I'll be grateful if you ask, I miss getting carded). However, I see no reason why the standards and practices can't be advanced to support those activities. The potential for phishing and man-in-the-middle attacks are a concern but there a lot of applications today where there are many benefits to the opting-in parties but few for the attacker.

Right now, if Tim's comment authentication system was OpenID-enabled, I'd be able to use my Technorati profile URL (http://technorati.com/profile/spidaman) to sign-in to post a comment on his blog. For "low-gravity" authentication requirements (blog comments), OpenID works great, today. For more rigorous authentication , user-attribute verification and trust requirements (like credit score lookups) there's a lot of great discussion underway. What I find heartening is that there seems to be broad acceptance of the Laws of Identity and increasing understanding that there's a big difference between the identity requirements for uploading photos and trading stocks in your IRA account.

URL-based authentication will likely go through the same growing pains raised by using email addresses to identify people. Back in the day when there were email addresses that people paid for and those were distinguishable from free ones, mailing list policies against subscribers with, say, hotmail addresses were often implemented. We may be approaching an era where some URLs are more equal than others, I dunno. But in the meantime, there's a lot of useful services you can use with OpenID today. If you haven't tried OpenID, do so right now by logging in to your Technorati account and then use your profile URL to log into Zooomr; this stuff is easier to use than it is to explain. I wouldn't be surprised if Yahoo! and/or Google get on the bus in the next few months. As the snowball gains mass, you should know how and when to utilize user-centric authentication systems such as OpenID.

               

( Feb 25 2007, 08:17:17 AM PST ) Permalink View blog reactions


20061202 Saturday December 02, 2006

So Long, Sam

In my wild, youthful daze of ... indulgence, I could count on Sam Kress to goad me on to indulge more. We shared a common bond, loathing all that spankles and poofs, reveling in the too-loud-and-too-fast-so-suck-it-up sounds of the day. I learned this day via my bud @ Umlaut of Sam's passing. Crap! Sam was one of those people who leaves on indelable mark on your memory with his fervent exhortations and swaggering enthusiasm. Even just recently (at the Godsmack show), I thought of Sam when I put down a shot of Jack Daniels (yea, I know, I should stick to the cabernet, they say resveratrol is much healthier, good 'nuff excuse for me). I last saw Sam at a reunion party of sorts (a gathering of old-schoolers) about a year ago, I hadn't seen him in maybe 15 years or so but that mischievious gleam in his eye was still there.

Heathen were formed in 1984 by guitarist Lee Altus and drummer Carl Sacco. Even without a bassist, this lineup played a single gig, on April 21st, 1985. Then, Jim Sanguinetti left to form Mordred and was replaced by Doug Piercey on guitars. Sam Kress, who was a better songwriter than vocalist, was kicked out in late 1985 and David Godfrey of Blind Illusion was asked to join.
read more
Yea, Dave's a better singer but Sam could growl like a mofo.

So long, Sam. I don't drink much whiskey any more but next time I have the occasion, I'll be raising it for you. I'll probably drink straight out of the bottle and pass it around, just for old times sake.

       

( Dec 02 2006, 04:20:45 PM PST ) Permalink View blog reactions


20061117 Friday November 17, 2006

Counting Links To Your Posts

The new Technorati link count widget provides a way for bloggers to display how many links a blog post gets. Doing it in Roller is easy, add the velocity macro below to WEB-INF/classes/weblog.vm and then call that macro from the default blog page template (Weblog). 

#macro( showCosmosLink $entry )
    <script src="http://embed.technorati.com/linkcount"
        type="text/javascript"></script>
        <a href="http://technorati.com/search/$absBaseURL/page/$userName/#formatDate($plainFormat $entry.PubTime )"
        rel="linkcount">View blog reactions</a>
#end

Tantek has more details about the release on the Technorati Blog.

     

( Nov 17 2006, 08:31:30 PM PST ) Permalink View blog reactions


20061115 Wednesday November 15, 2006

Scaling PostgreSQL

Last night, Casey Duncan gave a nice presentation on how Pandora partitions and populates their PostgreSQL data warehouse. The San Francisco PostgreSQL Users Group meeting discussion, Horizontal Scalability with Postgresql: A Case Study, covered how Pandora segments their data set, uses pgpool connection aggregation and Slony for replication and log shipping. I also had great conversation with Greenplum CTO Luke Lonergan, they're definitely pushing the envelope of what's possible with open source database technologies. Kudos to Pandora and Greenplum, great stuff.

           

( Nov 15 2006, 10:50:56 AM PST ) Permalink View blog reactions


20061112 Sunday November 12, 2006

CardSpace and the Identity Ecosystem

An interesting introduction came over the transom recently. I've read Kim Cameron's blog before but the honest truth is: I've really been flumoxed by the wide range in the cast of characters and agendas in the identity fray. Some seem overly concerned with identity as a line of business, others concerned with seeing themselves at the center of the discussion. Meeting Kim was a treat, even though he had the cards stacked against him coming from Microsoft, we had a great conversation. When I think of Microsoft I think of the many aspersions; "the Borg", "the evil empire", "The Man", "the big cathedral", "stifling monopolists", "makers of the Blue Screen Of Death", "vendor lock-in creeps", "virus and security-hole mongering dumbos." OK, I'll stop. Of course the reality is that good people also show up in bad places and they make good things happen nonetheless. C# looks and the .Net framework does great stuff for developer productivity. There's a lot of innovation happening in Microsoft's search and online services divisions. To be fair, a lot of Microsoft bashing is another form of bigotry that we have to get beyond. Microsoft has a lot great people and their executive leadership has done a lot of really bad things, so move along. The good guys inside the cathedral need constructive engagement lest they never prevail over the Matrix; more than anyone they (and Melinda) have the capacity to draw the Sith away from the Dark Side (re "constructive engagement": I'm thinking Clinton's Sino-American oppositional/collaborative stance that rides on the inevitable, not Reagan's failure vis-a-vis South Africa, which was wimpy coddling of the anti-divestment movement).

Speaking of the Jedi and Neo architype, characters and ranches in Santa Barbara, endorsements from Doc Searls always get my attention:

When the conversation started to heat up after DIDW, the Neo role was being played by a character with the unlikely title of "Architect", working inside the most unlikely company of all: Microsoft. Kim Cameron is his name, and his architecture is the Identity Metasystem. Note that I don't say "Microsoft's Identity Metasystem". That's because Kim and Microsoft are going out of their way to be nonproprietary about it. They know they can't force an identity system on the world. They tried that already with Passport and failed miserably.
I prefer to think of the various roles of Identity Providers, Relying Parties and People as part of an ecosystem. But metasystem is fine, let's just stick to that vernacular. Kim is the author of Laws of Identity. Again citing the same article from Doc for a nice summarization:

  1. User Control and Consent: digital identity systems must reveal information identifying a user only with the user's consent.

  2. Limited Disclosure for Limited Use: the solution that discloses the least identifying information and best limits its use is the most stable, long-term solution.

  3. The Law of Fewest Parties: digital identity systems must limit disclosure of identifying information to parties having a necessary and justifiable place in a given identity relationship.

  4. Directed Identity: a universal identity metasystem must support both "omnidirectional" identifiers for use by public entities and "unidirectional" identifiers for private entities, thus facilitating discovery while preventing unnecessary release of correlation handles.

  5. Pluralism of Operators and Technologies: a universal identity metasystem must channel and enable the interworking of multiple identity technologies run by multiple identity providers.

  6. Human Integration: a unifying identity metasystem must define the human user as a component integrated through protected and unambiguous human-machine communications.

  7. Consistent Experience across Contexts: a unifying identity metasystem must provide a simple consistent experience while enabling separation of contexts through multiple operators and technologies.

This is powerful stuff. I'm very pleased with our implementation of OpenID to support blog claiming but I know that this is the tip of the iceberg. There are people on the web who aren't authoring and sharing; they may not have nor want a URL that they can use for their identity. So while I'm committed to extending our support for OpenID, I'm also looking beyond it. The Laws are exemplary guiding principles in my exploration of the topic. Kim and Doc joined Kristopher Tate (the Zooomr dude), Tantek and myself to talk about CardSpace, Microsoft's implementation of an identity metasystem. After discussing some of the high-level issues facing the web, the blogosphere and user generated content participant created artifacts in general, we dived deep on CardSpace. Since CardSpace will be shipping with Vista (as well as distributed for Windows XP), by my estimation the coming ubiquity of user-centric identity isn't something to ignore. As we worked through the CardSpace workflow with Kim, Tantek and I came up with this diagram (Glossary: "IDP" = "Identity Provider", "RP" = "Relying Party", CardSpace is a page embedded app so there's both interaction via the browser and directly in the OS). This is of course just Microsoft's implementation but the Good Thing is that they aren't clutching it tightly, folks working on open source implementations (keep an eye on the OSIS working group) will make sure that the identity metasystem isn't a Borg in sheeps clothing.

Identities on the contemporary web suffer from a lot of accountability, authenticity and siloization deficiencies. Pings, trackbacks and comments all suffer from these and in turn we all do in the form of web spam. Reputation systems (such as Technrati's authority ranking) mitigate some of these problems but there is still much to do. I'm really pleased to have met Kim, he's one of the good guys and I look forward to working more folks pushing the online identity envelope. If you're going to be joining Internet Identity Workshop coming up, I'll see you there!

       

( Nov 12 2006, 01:25:24 PM PST ) Permalink View blog reactions


20061029 Sunday October 29, 2006

Hacking Into Movable Type

Everyone knows what a great product Movable Type is. But if you find yourself in care of a Movable Type deployment that nobody seems to be able to login to with superuser privileges, it may seem pretty hopeless; if you need to perform privileged operations, especially if the installation is backended by a sleepycat, er, Oracle BerkelyDB database, the data is somewhat opaque. AFAIK, MT doesn't seem to ship with any "break glass with this little hammer if the superuser was hit by a bus" contingencies and with BerkelyDB there's no SQL command prompt; in fact, the only way to dig into it is to write some code. So I was fiddling with just such a MT-3.33 installation; I had an account but not much in the way of privileges. After opening the BerkeleyDB files with DB_File, dumping contents with Data::Dumper and going through some of the MT libraries, I found what I was looking for. Here's the Perl I hacked up to grant myself superuser privileges: 

#!/usr/bin/perl

use strict;
use DB_File;
use lib qw( /path/to/MT-3.33/lib );
use MT;
use MT::Serialize;
use MT::ConfigMgr;

my $serializer = MT::Serialize->new(MT::ConfigMgr->instance->Serializer);
my %hash;
tie %hash,  'DB_File', '/path/to/MT-3.33/author.db', O_CREAT|O_RDWR, 0666, $DB_BTREE or die $!;
my $data;
while (my($k,$v) = each %hash) {
    my $rec = $serializer->unserialize($v);
    if (${$rec}->{'name'} eq 'Ian Kallen') {
        $data = ${$rec};
        last;
    }
}
$data->{'is_superuser'} = 1;
my $frozen = $serializer->serialize( \$data );
$hash{'12'} = $frozen;
untie %hash;
For other fixes to Movable Installations, consider MT-Medic.

     

( Oct 29 2006, 09:35:21 PM PST ) Permalink View blog reactions


20061019 Thursday October 19, 2006

OpenID on Technorati

As I announced on the Technorati Weblog, we rolled out support for blog claiming with OpenID. I'm really proud of the work that Chris and the team have done to make this a reality. If you're not familiar with OpenID, here is one good place to start. Sure, I'm well aware of the concerns about phishy user interface vulnerabilities. The idea of logging in without a password may seem weird.

One weird thing, for new users, is that instead of logging into an OpenID-using site (like Zooomr) with a user name and password, you just give it your personal OpenID URL -- and no password. Then your browser pops over to your authenticating site (like myopenid.com) to verify that you want to use your persona on the new site. This is bound to initially confuse people, and since users may not be asked for a password, it can also appear to be less secure, although it is not.
ZDNet: OpenID has a potential cure for Website password overload - Rafe Needleman
Frankly, I'm not certain what the best resolutions are for those concerns. However I'm more comfortable with adopting OpenID "as-is" and evolving as the technology advances then sitting around waiting for it to be perfected. Welcome to now.

Distributed identity ideas have been gestating for a long time while identity cathedrals have been built and fallen. If your blog is your voice, your URL can be your identity.

   

( Oct 19 2006, 11:42:04 PM PDT ) Permalink View blog reactions


Thinking about linking

Whenever I look at page to page, post to post, blog to blog and domain to domain relationship statistics (and permutations across them) interesting things often emerge. Microsoft's Live Search recently released a linkfromdomain operator that can help dig into these linking relationships. For instance, linkfromdomain:arachna.com ruby returns the pages that I've linked to that have ruby in the text. Combined with the site operator, I can do a search of the pages I've linked to on Technorati with linkfromdomain:arachna.com site:technorati.com.

Looks like the blogosphere is noticing, within the last two days Technorati has seen 57 links to the linkfromdomain announcement blog post. Kudos to MSN's search team for a cool innovation.

One apparent problem with their crawls is javascript/flash-plugin handling, the site:youtube.com linkfromdomain:technorati.com SERP shows pages referenced from Technorati's most linked-to YouTube videos, however all of the SERP items have the text

Hello, you either have JavaScript turned off or an old version of Macromedia's Flash Player. Click here to get the latest flash player.
heh!
Anyway, combine programmatic access (you can get a feed of that search with this link) with these link operators and Live Search is a very powerful and useful product. Read more about it on Live Search's WebLog

       

( Oct 19 2006, 06:56:16 AM PDT ) Permalink View blog reactions


20061018 Wednesday October 18, 2006

Saturn Eclipse

This was on NASA's Astronomy Picture Of The Day site a few days ago, I haven't been able to close the browser tab with it... I just keep gazing at the surreality of it.

In the shadow of Saturn, unexpected wonders appear. The robotic Cassini spacecraft now orbiting Saturn recently drifted in giant planet's shadow for about 12 hours and looked back toward the eclipsed Sun. Cassini saw a view unlike any other. First, the night side of Saturn is seen to be partly lit by light reflected from its own majestic ring system. read on
NASA goes on to explain that the eclipse revealed newly detected strata of rings around Saturn.

       

( Oct 18 2006, 10:45:16 AM PDT ) Permalink View blog reactions


20061017 Tuesday October 17, 2006

More Greening at Google

Between Google's extensive use of employee shuttles, their green data centers proposal last month and yesterday's announcement Google to Convert HQ to Solar Power, I'm really impressed with the ecologically conscientious initiatives they're taking! Personal note: the solar installation will be led by Energy Innovations, EI president Andrew Beebe is a friend from years ago who I've long lost touch with but I was very pleased to see his name associated to this project.

     

( Oct 17 2006, 06:52:10 AM PDT ) Permalink View blog reactions


20061007 Saturday October 07, 2006

Scaling Down

It's broadly appreciated how scaling up is usually driven by business demand, but the requirements for scaling down are rarely as appreciated. Questions about how web 2.0 business scale up abound these days. As the challenges of service growth and business plans stress technical infrastructure, startups try to squeeze everything they can out of their architecture with a number of widely accepted practices. However, scaling considerations for the other direction are oft neglected.

Why should you be thinking about scaling down?
  1. Isolated functional testing to mitigate the riskiness of change

    End-to-end testing that doesn't require duplication of production infrastructure is a strategic advantage. I know of a financial analytics system run by a large institution that is untestable. This system has cron jobs, data feeds and query systems built on top of Perl code going back at least a decade. The inputs and outputs are so convoluted, that the system is untestable. So if this code is making the bank that owns it tens of millions of dollars every day (it is!), what's wrong with that? Well, it could be probably be more profitable if it could be changed and optimized safely. As it stands, the folks maintaining the code don't really know what modifications might break the system and with income produced at that scale, who wants to risk it? So look at the systems you're working on now, think about the "scaling up" considerations you've made and ask yourself: Is a system testable in a developer's environment? Can they unit test? Can they perform functional tests? Do the tests require access to resources only available at the data center? Is "now" hardcoded to the present in your code? Using scaled down database, messaging, caching and application runtimes that have no dependencies on a connected network and production infrastructure should be considered up front in your design consideration.

  2. Operational costs of vertical vs horizontal scaling

    If a system makes assumptions about the process space it runs in that allows for functionality to be accessed from other runtimes, bravo: you may be headed in the right direction of service oriented architecture and horizontal scaling. But can the application stack be collapsed? This is like the OMG-moment when folks first started running J2EE application tiers over remote interfaces and realized that they've ended up with so much complexity and overhead, they have no choice but to scale up. That complexity can have all kinds of expensive side effects with how effectively systems can be triaged when they ail.

  3. Business agility or just changing your mind

    Businesses are run be people. People make mistakes. Wetware is imperfect. When you buy a long term commitment to a data center, you may be assuming liabilities that will outlive the business proof. Make sure the hardware footprint you're signing up for is one you can sustain it or you can get out of it. When you build gratuitous tiers, the costs of taking them out when it's time to consolidate functionality can be stifling. So ask yourself: If systems scaled up to meet business objective that aren't met, can you "retreat" from the scale-up offensive?

Every time I see a system that's hard to test, has sysadmins overwhelmed or are not meeting business objectives and has to be reeled in, I'm reminded of the importance of thinking about scaling in both directions. No, I haven't read the book yet but as someone burdened with too much stuff at home, I've got it on my list.

           

( Oct 07 2006, 03:53:58 PM PDT ) Permalink View blog reactions


20060930 Saturday September 30, 2006

Publishing Little Bits: More than micropublishing, less than big bytes

I find it really fascinating to see the acceptance of a publishing paradigm that lies in between the micropublishing realm of blogging, posting podcasts and videos and "old school" megapublishing. There are of course magazines; your typical piece in the New Yorker is longer than a blog post but shorter than a traditional book. But there's something else on the spectrum, for lack of a better term I'll call it minipublishing.

If you want to access expertise on a narrow topic, wouldn't it be cool to just get that, nothing more, nothing less? For instance, if you want to learn about the user permissions on Mac OS X, buy Brian Tanaka's Take Control of Permissions in Mac OS X. TidBITS Publishing has a whole catalog of narrowly focused publications that are bigger than a magazine article but smaller than your typical book. O'Reilly has gotten into the act too with their Short Cuts series. You can buy just enough on Using Microformats to get started; for ten bucks you get 45 pages of focused discussion of what microformats are and how to use them. Nothing more, nothing less. That's cool!

What if you could buy books in part or in serial form? Buy the introductory part or a specific chapter, if it seems well written, buy more. Many of us who've bought technical books are familiar with publish bloat, dozens of chapters across hundreds of pages that you buy even though you were probably only interested in a few chapters. Sure, sometimes publishers put a a few teaser chapters online hoping to entice you to buy the whole megilla. Works for me, I've definitely bought books after reading a downloaded PDF chapter. But I'm wondering now about buying just the chapters that I want.

           

( Sep 30 2006, 07:04:31 PM PDT ) Permalink View blog reactions


20060927 Wednesday September 27, 2006

You Can't Handle The Truth

Colonel Jessup has assumed control of Newsweek:

Ignorance is bliss
How meta:

See ya at the gulag.

           

( Sep 27 2006, 04:16:31 PM PDT ) Permalink View blog reactions


20060926 Tuesday September 26, 2006

Green Data Centers

At today's Intel Developer Forum, Google is presenting a paper that argues that the power supply standards that are built into today's PCs are anachronistic, inefficient and costly. With the maturing of the PC industry and horizontal scaling becoming a standard practice in data center deployments, it's time to say good-bye to these standards from the 1980's.

John Markoff reported in the NY Times today

The Google white paper argues that the opportunity for power savings is immense, by deploying the new power supplies in 100 million desktop PC's running eight hours a day, it will be possible to save 40 billion kilowatt-hours over three years, or more than $5 billion at California's energy rates.
Google to Push for More Electrical Efficiency in PCs
Nice to see Google taking leadership on the inefficiencies of the PC commodity hardware architectures.

       

( Sep 26 2006, 06:02:09 PM PDT ) Permalink View blog reactions


20060925 Monday September 25, 2006

Greater than the sum of its parts

The other week I reflected on the scaling-web-2.0 theme of the The Future of Web Apps workshop. Another major theme there was how social software is different, how transformative architectures of participation are. There was one talk that stood out from Tom Coates, Greater than the sum of its parts. A few days ago the slides were posted; I poked through 'em since and they jogged some memories loose, I thought I'd share Tom's message, late though it is, and embellish with my spin.

Tom's basic thesis is that social software enables us to do "more together than we could apart" by "enhancing our social and collaborative abilities through structured mediation." Thinking about that, isn't web 1.0 about structured mediation? Centralized services, editors & producers, editorial staff & workflow, bean counting eyeballs, customer relationship management, demographic surveys and all of that crap? Yes, but what's different is that web 2.0 structured mediation is about bare sufficiency in that it's better to have too little than too much, the software should get out of the way of the user, make him/her a participant, not lead him/her around by the nose.

Next, Tom highlighted that valuable social software should serve

Individual Motives
An individual should get value from their contribution
Social Value
The individual's contributions should provide value to their peers as well
Business/Organizational Value
The organization that hosts the service should enable the user to create and share value and then derive aggregate value to expose this back to it users. I thought that was really well considered.
Tom outlined a spectrum of social software, on the one hand concensus focused and fact oriented where many contributions make one voice and, on the other hand, a social contribution focus and polyphony where many voices produce emergent order. Wikipedia, MusicBrainz and openstreetmap.org are illustrative of the former, Flickr, Plazes, YouTube and Last.fm the latter. Tom discussed the motives for contributing to the community:
anticipate reciprocity
by offering value, it's reasonable to expect others to contribute value as well
reputation
by showing off a little, highlighting something uniquely yours to contribute, you gain prestige
sense of efficacy
by being able to make an impact, a sense of worth is felt
identification with a group
be it for altruism or attachment, contributing to a group makes you part of it
Think about every mailing list you've been on, every online forum and simulated environment you've used and you know it's true, these are among the basic underpinnings of virtual community

Citing The Success of Open Source , he likened social software participants motivations to this ranked list of open source contributor's motivations

At a meta-level then, commodization of memes is driven similarly to open source's commodization of software capabilities. I think this analogy requires exploration, particularly now. While Mark Pilgrim counts all Non-Commercial-Use-Only licenses as overly restrictive, I disagree. I don't think we need to remove all encumbrances on our words in order to freely disseminate memes. On the contrary, if every n'er-do-well kleptotorial spammer has free reign of your words, it seems more likely that your meanings authenticity will get lost as it gets reposted on legions of AdSense-laden splogs. So while many of the motivations for contribution inspire analogy, the licensing ramifications are very different. I own my own words. Feel free to quote, excerpt or otherwise use them for non-commercial use. Everything else is a negotiation.

Here are some social software "best practices":

And here's what to watch out for: So, what's the business? Where's the money? Well, AFAICT, the business models still need to prove themselves. We've seen virtual communities become viral communities; driven by social networking, peer to peer technologies and other bindings but apart from Fox' MySpace acquisition, where's the big money? Hopefully we'll see "IPO 2.0" events, web 2.0 companies enjoying financial vigor and going public, in the next year or so. Ultimately, it's liquidity that will provide commercial validation. Anyway, you'll find a lot of this in Tom's slides but unfortunately, what's online is just shadow of his live preso @ The Future of Web Apps.

     

( Sep 25 2006, 10:24:40 AM PDT ) Permalink View blog reactions


20060921 Thursday September 21, 2006

Community Policing In The Blogosphere

I mused about people-powered topic classification for blogs after playing with the Google Image Labeller the other week. It seems like a doable feature for Technorati because the incentives to game topic classification are low.

That same week, Rafe posed a question about community driven spam classification:

Why couldn't Blogger or Six Apart or a firm like Technorati add all of the new blogs they register to a queue to be examined using Amazon's Mechanical Turk service? I'd love to see someone at least do an experiment in this vein. The only catch is that you'd want to have each blog checked more than once to prevent spiteful reviewers from disqualifying blogs that they didn't agree with.
(read the rest)
The catch indeed is that the incentive is high for a system like this to be gamed. Shortly after blogger implemented their flag, spammers fired laughed back with bloggerbowling:
"Bloggerbowling" - the practice of having robots flag multiple random blogs as splogs regardless of content to degrade the accuracy of the policing service.
As previously cited from Cory, all complex ecosystems have parasites. So I've been thinking about what it would take to do this effectively, what would it take overcome the blogosphere's parasites bloggerbowling efforts? The things that come to mind for any system of community policing are about rewards and obstacles. For example I've participated in virtual communities of many flavors for years (in fact, Cory and Rafe are familiar faces from over a dozen years ago on The WeLL, back then I was newbie amongst oldtimers). Virtual communities work well when there are social bonds, when there is accountabiity and reputational capital that gets put on the line. The stronger those factors, the greater the motivation for community policing. Who's motivated to police the blogosphere? Obviously, Technorati is motivated; if the up-to-the-minute is up-to-it's-neck-in-crap, the value diminishes quickly. Another class of motivated users are folks like Doc, authors of narrative ripped off by kleptotorial sploggers. The last class of motivated ecosystem participants that comes to mind are the victims of click fraud, from what I've heard their outcomes to date have been lotsa free ads and their lawyers fetching fat fees.

At the end of the day, I don't have the answers. But I think Rafe, Doc and so many others concerned with splog proliferation are asking great questions. Technorati is currently keeping a tremendous volume of spam out of its search results but, at the end of the day, there's still much to do. And this post is the end of my day, today.

             

( Sep 21 2006, 11:06:22 PM PDT ) Permalink View blog reactions


20060913 Wednesday September 13, 2006

Everybody Hurts, Sometimes

A few weeks ago, Adam mentioned some of the shuffling going on at Technorati's data centers. Yep, we've had our share of operational instability lately, when you have systems that expect consistent network topologies and that has to change, I suppose these things will happen. It seems a common theme I keep hearing in conversations and presentations about web based services: the growing pains.

This morning, Kevin Rose discussed The digg story: from one idea to nine million page views at The Future of Web Apps workshop. Digg has had to overcome a lot of the "normal" problems (MySQL concurrency, data set growth, etc) that growing web services face and have turned to some of the usual remedies, rethinking the data constructs (they hired DBA's) and memcached. This afternoon, Tantek was in fine form discussing web development practices with microformats where he announced updates to the search system Technorati's been cooking, again a growth induced revision. Shortly thereafter, I enjoyed the stats and facts that Steve Olechowski presented in his 10 things you didn't know about RSS talk. And so it goes, this evening it was Feedburner having an episode. "me" time -- heh, know how ya feel <g>

While Feedburner gets "me" time, Flickr gets massages when they have system troubles. Speaking of Flickr, I'm looking forward to Cal Henderson's talk, Taking Flickr from Beta to Gamma at tomorrow's session of The Future of Web Apps. I caught a bit of Scaling Fast and Cheap - How We Built Flickr last spring, Cal knows the business. I've been meaning to check out his book, Building Scalable Web Sites.

Perhaps everybody needs a therapeutic message for the times of choppy seas. When Technorati hurts, it just seems to hurt. Should it be getting meditation and tiger balm (hrm, smelly)? Some tickling and laughter (don't operate heavy machinery)? Animal petting (could be smelly)? Aromatherapy (definitely smelly)? Data center feng shui? Gregorian chants? R.E.M. samples?

               

( Sep 13 2006, 09:26:42 PM PDT ) Permalink View blog reactions


« Applying Security Tactics to W... Speaking of Upgrades »