What's That Noise?! [Ian Kallen's Weblog]

Tuesday April 12, 2005

Hello Berkeley DB This morning, I wanted to get familiar with the Berkeley DB "Java Edition" API (that's a mouthful, can't I just call it "sleepycat"?). I was in a carpool and I don't think the dude driving realized I was hacking-in-traffic. While I've happily used used DB_File in Perl for years-n-years, I haven't had time/opportunity to mess with the Java stuff from Sleepycat. However, at work we're cooking up a durable message buffer with an embedded servlet container, fun! As with poking into any new API, I like to start with Hello World.

Here's my Sleepycat Hello World

import com.sleepycat.je.Cursor;
import com.sleepycat.je.Database;
import com.sleepycat.je.DatabaseEntry;
import com.sleepycat.je.DatabaseConfig;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.Environment;
import com.sleepycat.je.EnvironmentConfig;
import com.sleepycat.je.LockMode;
import com.sleepycat.je.OperationStatus;
import java.io.File;

public class HelloBdb {

    public static void main(String[] args) throws Exception {
        String key = args[0];
        String value = args[1];

        File dir = new File("db");
        dir.mkdirs();

        Environment env = new Environment(dir, new EnvironmentConfig());
        Database database = env.openDatabase(null, "foobar", new DatabaseConfig());
        database.put(null, 
            new DatabaseEntry(key.getBytes()), new DatabaseEntry(value.getBytes()));

        DatabaseEntry foundKey = new DatabaseEntry();
        DatabaseEntry foundData = new DatabaseEntry();

        Cursor cursor = database.openCursor(null, null);
        while (cursor.getNext(foundKey, foundData, LockMode.DEFAULT) == 
                OperationStatus.SUCCESS) {
            String keyString = new String(foundKey.getData());
            String dataString = new String(foundData.getData());
            System.out.println("Key | Data : " + keyString + " | " + 
                dataString + "");
        }
        cursor.close();
        database.close();
        env.close();
    }
}

Of course, the real fun will be running this in a multi-threaded environment and the concurrency issues therein. With Hello World done, it's time to move on to see what else needs to be added to the cookbook.

( Apr 12 2005, 08:57:09 PM PDT ) Permalink View blog reactions

Sunday April 10, 2005

del.icio.us investment details Joshua Schachter just announced the specifics of the del.icio.us investment. It's an intriguing cast of characters.

Here's the roster:

On the mound: Union Square Ventures: specializes in infomediation
Catcher: AMZN: Amazon is certainly interested in how you tag the products your interested in
1st base: Marc Andreesen: what's he up to now?
2nd base: BV Capital: Bertelsmann Ventures are playahs
short stop: Esther Dyson: another leading light
3rd base: Seth Goldstein: another
Right field: Tim O'Reilly: book mogul
Center field: Bob Young: founder of RedHat
Left field: Josh Koppelman: The half.com guy
DH: Howard Morgan: Not sure, IdeaLab?

I've been a del.icio.us fan for a long time, this seemed inevitable. Looks like a lot of good folks, congrat's to del.icio.us!

( Apr 10 2005, 06:38:37 PM PDT ) Permalink View blog reactions

Tuesday April 05, 2005

Talk To The Blog Cause The Hand Isn't Typing If you'd rather dictate audio than type a word composition, check out Audioblogger. It works with a blogger account by posting audio that you call in to a number that Audioblogger provides.

With all of the phones these days supporting crude digital video features, it seems like just a matter of time before vblogging comes of age. Er, hold the phone, Google is archiving video clips now. In the meantime, I'm imagining James T. Kirk rambling philosophically to his blog about a recently visited corner of a previously uncharted galaxy and posting video mashups of interstellar foibles.

( Apr 05 2005, 07:55:31 AM PDT ) Permalink View blog reactions

Monday April 04, 2005

How Not to Learn Japanese I've been collaborating with a team in Tokyo a lot over the previous few months. They've been very gracious about speaking English with me but I've been unable to reciprocate. Anyway, I've had my eye on the w3c conference in Chiba. I should probably learn to speak a little Japanese.

This is what I'm not going to be doing to learn Japanese:
Kanji Quiz Toilet Paper

This is an extremely extraordinary item, toilet paper with the power to teach! For those who prefer to "sit in the library" this great paper can provide... hours (?) of fun and prepare them for that pop quiz in Japanese class the next morning. Three different methods of learning are provided "multiple choice", "reading", and "philosophical fill-in-the-blank". Written in a soft blue color on white. A superb item for anyone interested in studying Japanese, it's really cool on many different levels.

- eBay blurb where this was offered for $4.80

Maybe I'll just take the phrase book or something.

( Apr 04 2005, 01:43:29 AM PDT ) Permalink View blog reactions

Thursday March 31, 2005

Core Competencies Has Google jumped the shark and been eclipsed by Yahoo!?

Clearly Yahoo!360 demonstrates Yahoo!'s competency: building a relationship with its users. All of the web mail, 'My Yahoo!' and 'Yahoo! Groups' user features are basically good personalized outward looking experiences with (more or less) single sign-on. Contrast this with Google's user features: the disconnected islands of GMail, Orkut and Blogger each have separate identities and user populations.

360 adds to Y!'s feature set social networking, blogging and integrates with (some) other Y! features seamlessly. I anticipate seeing Flickr and other innovative features making their way into this soon enough. So it's transforming from a personalized outward looking platform to one for personalized networking. Google's core competency seems to be stuck at "sooner or later, we're indexing everything" -- not that that's a bad thing. Unlocking the world's accumulated knowledge is cool.

So while I don't know if I agree that Yahoo! has overtaken Google, they're both clearly magnifying what they're good at lately. And I'm sure the Googlers aren't sitting still, they've got some smarty pants people there. In the meantime, Yahoo! clearly knows mores about providing a platform for people.

( Mar 31 2005, 12:36:39 PM PST ) Permalink View blog reactions

Wednesday March 30, 2005

Rob Zombie Becomes More Human How fitting that the week I found myself prowling around the grinding heavy metal equivalent of a nascar-for-nerds rally I've learned of Rob Zombie's blog. What's next? Lars Ulrich releasing music under a creative commons license?

I am the ripper
Man a locomotion
Mind love american
Style

More human than human
...stranger, sicker things have happened

( Mar 30 2005, 10:51:43 PM PST ) Permalink View blog reactions

Tuesday March 29, 2005

EVDB The Events and Venues Database just went beta.

I'm looking forward to subscribing to a tag to follow specific types of events and seeing what EVDB does to identify important events.

( Mar 29 2005, 09:04:27 PM PST ) Permalink View blog reactions

Monday March 28, 2005

RoboGames 2005

There was almost a Homebrew Computer Club feel (not that I was there but I can imagine the ambient excitement and ferment) to the mix of people who turned for RoboGames 2005. This is the event formerly known as robolympics and it was heaps 'o fun!

In a future time Children will work together To build a giant cyborg Robot Parade Robot Parade Wave the flags that the robots made Robot Parade Robot Parade Robots obey what the children say There's electric cars There's electric trains Here comes a robot with electric brains Robot Parade Robot Parade Wave the flags that the robots made Robot Parade Robot Parade Robots obey what the children say		Boy versus Robot The boy has a better CPU but the robot has a blue samurai sword that lights up!
I took a few pictures. This is the kind of endeavor I could imagine sucking up all of my time. For now, I'm content to take an aloof armchair interest in robotics.

robogames

( Mar 28 2005, 01:12:05 AM PST ) Permalink View blog reactions

Sunday March 27, 2005

Tagbacks Just wanted to post a quick clarification from some of the discussion last month about tagbacks.

The comment about Technorati's robots.txt is correct. There are no rules there for the tag pages there but in the tag pages themselves have instructed the googlebot with nofollow, this is in the markup:

<meta name="robots" content="index,nofollow" />

This won't stop tag spam that's just trying to get accidental clickthroughs (i.e. we know we have some more work to do) but it will deny google juice rewards (which is what they really want).

( Mar 27 2005, 06:17:23 PM PST ) Permalink View blog reactions

Wednesday March 16, 2005

The Web Spammed By Jesus There is no shortage of funny, sad and scary things fizzling out of the magma of blogs.

The recent interest Technorati has taken in web spam has perhaps inspired the divine. Note the recent appearance of http://jesus-the-lord.blogspot.com/ but linking to it without nofollow is most certainly a sin.

nofollow jesus spam
( Mar 16 2005, 01:59:38 PM PST ) Permalink View blog reactions

Tuesday February 15, 2005

A Java i18n Checklist I've worked on Java projects with the human language aspects abstracted out but now that I've had to get down and dirty with real localization problems, I'm starting to take mental notes in the "if I had known that earlier it would've saved me a lot of grief" folder.

It seemed pretty straight forward going into the project that I'm working on:

Text is stored as UTF-8 character data
Use ResourceBundle property files to manage display strings
Maintain the set of property keys in the properties file
Let the browser's Accept-Language request headers drive what property file to prefer

See, it's easy! Well, for simple proofs of concept with Western characters, it's just about that easy. When dealing with multibyte strings for asian languages, there's a whole lot more to consider.

Make sure the servlet container is handling UTF-8 appropriately

For instance, if Tomcat is serving on the HTTP tier edit server.xml and make sure the URIEncoding attribute (absent by default) is set for the connector.
    <Connector port="8080"
               maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
               enableLookups="false" redirectPort="8443" acceptCount="100"
               debug="0" connectionTimeout="20000"
               disableUploadTimeout="true"
                URIEncoding="UTF-8"
    />
The same holds true for letting Apache do the HTTP dirty work and connecting with mod_jk
    <Connector port="8009"
               enableLookups="false" redirectPort="8443" debug="0"
               URIEncoding="UTF-8"
               protocol="AJP/1.3" 
    />
And, by the way, if you have static content served by an Apache server, you probably want this as well
AddDefaultCharset utf-8

Wire up the native2ascii ant task into the build system early in the project.
Manually dealing with the ASCII escaping is a nuisance. If the conversion can't be transparent, at least automate it.
Make sure the database connection drivers are being gentle with their data handling.
In the case of MySQL, changing the JDBC URLs from this
```
jdbc:mysql://localhost/fubar
```
to this
```
jdbc:mysql://localhost/fubar?useUnicode=true&characterEncoding=UTF-8
```
made a world of difference.
Check the HTTP response headers to assure that the Content-type header value is appropriate
If the charset isn't set to UTF-8 when it really is, you could be confusing the client. This can set in a servlet, in a JSP and IIRC the struts-config.xml allows you to set it declaritively. You want to set the Content-type before writing to the response object's PrintWriter. Apparently if you have multibyte characters in your JSP page components, you need to set the pageEncoding i.e. in the JSP file itself, something like this:
```
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8 %>
```
Though my whole motivation for using Java on this project was to have page components have only markup and display code; all of the lanugage is abstracted. Anyway, I'm preferring Velocity over JSP these days.
Be prepared to convert request parameter values.
In my experience, doing this
```
request.setCharacterEncoding("UTF-8");
```
before getting the parameter values is not reliable (could be Tomcat bugs though). However, this appears to be a fairly standard idiom
```
String formValue = new String(request.getParameter("formParam").getBytes("ISO8859_1") /* bytes */, "UTF-8");
```

There are other i18n traps to beware of; seems like every place data is passed from one subsystem to another there's an opportunity for the encoding to get mangled.

( Feb 15 2005, 10:17:29 PM PST ) Permalink View blog reactions

Monday February 14, 2005

Time Slips Away Another day of end-to-end frenzy and the day is still not done. Short meetings turn long. Interruptions and blocking issues. Who has time for anything? Who has time to blog?

I just took my trusty old swiss army watch out of my pocket, I shoved it in there on the way out the door this morning and have had nary a chance to adorn my wrist with it all day. And now it's too late. Where goes the time?

( Feb 14 2005, 09:24:56 PM PST ) Permalink View blog reactions

Thursday January 27, 2005

Speed Isn't Everything and It's Crap Without Automated Testing The software project management triangle has these three things at the corner:

scope
speed
cost

The third item refers to allocating more resources (typically, people) to projects. The idea is that if you're resource starved (short handed) you need to reduce scope and/or the schedule (sacrifice speed) to compensate. If you increase scope, either the schedule slips or costs increase (or both). Of course, throwing more resources at the problem is often counter-productive.

There's another triangle associated with software development:

rigor
automated testing
quality

Most software projects involving technology innovation can ill-afford to lean to close to the rigor corner; problems that haven't been identifiably solved before are intrinsically riskier. Rigorous development practices such as old-school waterfall processes are usually a bad fit where the emphasis has to be on speed of execution and technical creativity. Culturally, automated testing can't just be a function for quality assurance staff; unit testing, integration testing and functional testing have to built into the DNA of a software project's build system or else the engineering costs for writing and running tests grows too high to sustain. So the third option is winging it in an ad-hoc fashion and live with the associated reduction in quality. Obviously for things like flight control and medical monitoring where lives are at stake, you want rigor and automated testing otherwise bad things happen. Space crafts blowing up and/or becoming astro-debris, missile guidance systems failing or heart transplant patients dying due to software defects are tragic. But most software projects don't have those kinds of consequences attached. Thus the incentive is high for making test driven development part of the software engineering culture wherever speed of execution and breadth of innovation are among the primary motivations.

So if low quality is crap, perhaps there's a mathematic expression here

  SPEED - AUTOMATED TESTING = CRAP

...and perhaps it's even transitive: high quality + automated testing = speed.

Speaking of which, it must be time for more coffee.

( Jan 27 2005, 10:11:18 AM PST ) Permalink View blog reactions

Sunday January 23, 2005

Technorati + Firefox Search tools in browsers are usually pre-populated with all of the usual suspects: Google, Yahoo, Amazon and so on. That's all well and good but what about when you want to know about what's being said on the real-time web?

There's now a Firefox plugin that adds a "Technorati Engine" to the pulldown list. Sweet!

( Jan 23 2005, 09:45:08 AM PST ) Permalink View blog reactions

Saturday January 22, 2005

Start Up Pains I've posted here about Technorati's misadventures in recent months: power outages, disk upgrades, data center moves and so on. In the life of a startup, these kinds of foibles seem to be just a part of growing up.

So I've empathized and enjoyed recent readings of other's mishaps. Not in celebration perhaps in feeling the bonds of shared trauma.

FlickrBlog's Growing Pains

unbalanced load/capacity distributions
bottlenecked queues
non-specific system instabilities

Word to your mutha, I know that.

LiveJournal's Power-loss post-mortem

hardware configuration/system problems that exhibited themselves as hosts that didn't came back up correctly
MySQL databases getting hosed
Databases tuned for speed over safety

Been there, done that.

So is it the destiny of all web service start-ups to have fabulous disasters? Probably. A lot of times, these kinds of things are predicated on having Innovative architectures, on doing some things that aren't widely known to have been done that particular way. Can you really guard against finding yourself with arrows in your back when you're out on the frontier? I don't think so. Count on the topsy turvy. It's not always fun when you're in the thick of it but if you adapt, you'll be better for it.

Of course, you could just laugh about it. Or post to your blog about it. Or both. So far, from what I've stumbled across, this is the funniest of the bunch.

10:11 pm: So far so good. Things are checking out, but we're wearing tinfoil hats. A few annoying LJ users, but nothing that's not fixable. We're going to be buying a bunch of weed on Monday so that, if this happens again, we'll just be too baked to care.

This weekend's disastrophe for me is relatively mild: sore throat and congestion. So I'm drinking tea. And laughing about it. And posting to my blog.

( Jan 22 2005, 08:42:19 PM PST ) Permalink View blog reactions

Saturday January 15, 2005

Booting linux in single user mode Today, we had to bring up a Debian host in single-user mode and it didn't want to cooperate. Note to self:
Next time LILO doesn't want to stop for a boot prompt, hold down the shift key to force to stop for a prompt.

I don't know if that's a Debian default or something, but it's really annoying that something special is required to get a boot prompt.

( Jan 15 2005, 11:03:22 PM PST ) Permalink View blog reactions

Wednesday January 12, 2005

Jack of All Trades I thought I was pretty broadly skilled with my work spanning software engineering, network and systems operations, some DBA-duties and variety of other do-it-cause-it-needs-to-be-done endeavors.

Well here's a something funny: Chef and Missile Treaty Compliance Inspector. Yea, this is an actual job that is posted right now:

Job Description: Responsible (sharing duties with one other chef) for the preparation of three daily meals (brunch & dinner only on Sundays) for approximately 20-25 personnel at remote, self-contained site in Russia. Duties include menu planning, food ordering & resupply, inventory management, kitchen maintenance and sanitation and catering support to infrequent special functions on and offsite. As treaty monitor, may occasionally operate the On-Site Continuous Monitoring System in accordance with official START treaty missile inspections at Votkinsk.

Where can you sign up? HotJobs (what do you mean by "hot"?). I took the liberty of mirroring the job ad, since I'm not really familiar with the lifecycle of job ads posted there. I thought it was good enough to keep!

If that's not your cup-of-tea, Technorati has some great jobs open right now. Most of them don't even require you to

walk, cross-country ski, or walk with snowshoes, a distance of 2.5 - 3 miles without rest in all seasons, including winter temperatures as low as -20 degrees Fahrenheit

nor

shoveling sidewalks and building entrances and digging out snow around satellite dish area and other equipment

Not that there's anything wrong with that.

( Jan 12 2005, 12:58:23 AM PST ) Permalink View blog reactions

Monday January 10, 2005

Eclipse and iTunes Something's really goofy with my Powerbook!

If I'm listening to iTunes and then starting working in Eclipse, I get static popping and scratching in my ears. It hurts! It sucks! It just isn't right! Is this Apple's way of telling me they don't want me to develop code with Eclipse?

I'm marching over to MacWorld to protest!

( Jan 10 2005, 02:29:46 PM PST ) Permalink View blog reactions

Saturday January 08, 2005

Developer Contests Over the last few weeks I've been paying more attention to the API's that blog and taggregator services offer. I've also starting looking more into the API's that blog tools offer.

The Technorati Developer's Contest results are in. I was especially impressed with the visualization effort by Michael Dale'sTechnorati Touchgraph application. While it is a little rough around the edges in places (some of the implementation's PITA'ness is just AWT and crapplets being what they are), these kinds of graphical renderings of the Technorati cosmography are really great. Wondering now if anybody has tried something like this with Macromedia Flash, hmm....

If anybody is entering the Blojsom Developer Contest and wants help with the Technorati API, there's a Java client in the SDK and I'd be happy to lend assistance with it.

( Jan 08 2005, 12:04:12 PM PST ) Permalink View blog reactions

Managing Multi-Timezone Conference Calls I'm ringmastering a project that requires coordination with parties near and far. Most of the coordination for these kinds of things functions well with email, IM, wiki and CVS. But sometimes a plain-old conference call is in order.

Getting all of the parties lined with

a number to dial into
invitations and acknowledgements (RSVP's)
conference call times with the time and timezone contextual to the recipient

It turns out that FreeConference.com offers tools for all of this for, well as the name implies, free. They make money up-selling premium services but I've got my first conference call with parties in three timezones setup through them and haven't paid a dime for it (which BTW, is their cost-per-minute-per-connection to have the dial-in number be an 800 number); I'll find out how it goes next week!

( Jan 08 2005, 10:26:54 AM PST ) Permalink View blog reactions

Monday January 03, 2005

PHP and mod_perl w/mysql breakage on Mac OS X I'm working with code in PHP, Perl and Java that all have to access MySQL. I haven't had any problem using PHP configured "--with-mysql" and mod_perl with DBD::mysql on Linux but on Mac OS X, it just doesn't work.

My setup is: Mac OS X (Panther) with Perl 5.8.1, MySQL (v4.0.17) installed with Fink. Apache (v1.3.31) was compiled with both mod_perl and PHP. When I deployed a mod_perl module that connected to MySQL, the connections always failed with this error:

DBI connect('bjorkdb;port=3306','hamster',...) failed: Protocol mismatch. Server Version = 0 Client Version = 10 at ...

which was really confounding because the connections in other runtime contexts were fine. I double checked to make sure I didn't have multiple libmysqlclient's around or additional Perl installations. All of that checked out.

I recompiled Apache without PHP and everything works great. So my conclusion is that something funky happens on Mac OS X when linking libmysqlclient. Looks like I'll have to keep another set of Apache binaries around should the need to run PHP locally recur, otherwise I'll just stick to the compile that has mod_perl but not PHP.

( Jan 03 2005, 12:01:11 AM PST ) Permalink View blog reactions

Sunday January 02, 2005

In Every Season: Turn, Turn, Turn I still feel a small sense of propriety for things I've worked on in the past. Even way in the past. Even when not an inkling of anything I'd worked on back then is likely to have survived any of the incarnations since then. On the other hand, I'm probably doing better than my friends whose efforts over the years completely dot-bombed-in-flames. I guess this is the time of year when I mark the time by life's events (including gigs) and reflect.

Hearing talk of Gamespot's decline (for me, Gamespot.com was a gig that started nine years ago and ended three later), like "Gamespot going down hill" is kinduva bummer. I was concerned when I read of Scott Rosenberg's book break but a lot of the fine folks I have fond memories of at Salon (four years ago) are still there and doing a great job! And like myself, a lot of folks that I worked with over the years have moved on to work on and accomplish great and interesting things, so it's all good.

OK, reflection time is over now.

Thankfully, I really dig what I'm doing now! Happy 2005 to all of my friends and colleagues past and present!

( Jan 02 2005, 11:02:38 AM PST ) Permalink View blog reactions

Saturday January 01, 2005

The Top Tsunami Sites I did a little digging around and found some of the top sites for relief information, reactions, pictures and videos of Sunday's Tsunami.

The South-East Asia Earthquake and Tsunami Blog: On going coverage of relief news, publications and other linkable resources
Tsunami Relief: Google's thumnail list of relief organizations
QuickBird Images of Tsunami Sites: Stunning satellite photos with before and after shots of Sri Lanka and Indonesia
Cheese and Crackers: Tsunami Video: Tons of links to amateur video's from the region that capture the power and immensity of the tsunami
Blog: I can't read Japanese but people that do are reading this a lot
Raw Print | Westboro Baptist Tsunami Statement: Those Kansas right-wing nutjobs (ya know, the "god hates fags" zealots) are at it again with their Sodom and Gomorrah crap. "Thank God for Tsunami. Thank God for 3,000 dead Americans!" Wow, you'd think the pastor at Westboro Baptist Church is really Osama Bin Laden. More and more, I think that seceding from the union sounds like a good idea.
Support UNICEF's Tsunami Relief Efforts: Ya know, it's for the kids. $50, $100, $250, $500 or any amount you can give will make a difference.
Network for Good :: Help Support Disaster Relief in Southeast Asia: A good collection of emergency response and on-going relief organizations. Certainly there must be one for you.
CNN.com - Aid groups accepting donations for victims: Who says that big media has no soul? They care too!
MoveOn.org: Tsunami Relief: The Bush administration stepped forward first with $15M, then $35M and today brought it up to $350M. MoveOn's petition says
President Bush and Congress must offer whatever assistance is necessary to prevent further human suffering in the wake of the tsunami disaster.

I arrived at this list with some simple pokes at the data coming through at Technorati, I might try to spiff this up a little bit and turn it into a feature.

( Jan 01 2005, 03:51:00 AM PST ) Permalink View blog reactions

Friday December 31, 2004

Pictorial Fallout Seems like there are no events without photostreams anymore.

Here are some from last night's fiesta and after party that followed Scobleizer's Technorati visit.

( Dec 31 2004, 12:49:48 PM PST ) Permalink View blog reactions

Hey 2004, don't let the door hit you on the butt on your way out! I have a lot of things to be thankful for this past year. However, 2004 brought on a sufficient number of painful kicks-in-the-teeth, I say good riddance!

The bad stuff has been damn bad:

The loss of my grandfather
The prevalence of red states
Widespread devastation striking people on a massive scale
Local devastation striking my friends and neighbors

But there was good stuff too:

My family and my kiddos continue to blow me away with how wonderful they are, my love and pride swells
Working with all of the terrific folks at Technorati has been a thrill
And life in general has been quite good, albeit hectic
People are trying to help

Wishing for a good 2005!

( Dec 31 2004, 10:56:36 AM PST ) Permalink View blog reactions