What's That Noise?! [Ian Kallen's Weblog]
Wednesday December 15, 2004
Tool to defeat a DOS against an Apache server farm Throttling the number of requests that come in to a single web server instance isn't rocket science but it requires keeping track of a state shared amongst processes/threads. But how would you efficiently throttle requests that come to a set of servers?
Apparently, there's some Apache goodness available for this now. At least I think it sounds good! Ian Holsman has written mod_ip_count for Apache 2.0. It uses the APR portability layer and memcached for shared state (actually apr_memcache from Paul Querna). This would enable a whole server farm to keep track of request rates from and throttle specific IP addresses.
( Dec 15 2004, 04:24:13 PM PST ) Permalink View blog reactions
Tuesday December 14, 2004
Greener Pastures This weekend Technorati's network and server infrastructure is going to move. In one big fell swoop. Well, hopefully nothing will fall.
The home page sez: "Movin' on up" cause Technorati is substituting the Jefferson's theme song for the old ops/facilities anthem, the Talking Heads' "Burning Down The House"
( Dec 14 2004, 12:47:43 AM PST ) Permalink View blog reactions
Sunday December 12, 2004
l10n development practices I've used Java's handy-dandy ResourceBundles to do some proof-of-concept localizations. But my past proofs were limited; they only used other European languages that utilized ISO-8859-1 characters. I don't recall having to do anything special with the property files in those cases.
Working on a recent Japanese localization project was an eye opening experience. It turns out the java.util.Properties expects ISO-8859-1 characters. I guess that's the downside of having a super-simple file format. I got the localized display boostrapped by using native2ascii to get the UTF-8 localization text rendered as escaped unicode. On a one-off basis, that's easy enough. But collaborative development always begs the tools question, how do folks typically manage this?
- Give the localization team instructions on how to use native2ascii on a command line basis?
- Put it in the build system with the native2ascii ant task?
- Use a light weight GUI tool like resourcebundleeditor (or a heavy weight GUI tool ...)?
What about input encoding? If there's an HTML form on a page and the input has multibyte characters in the query string (or POST data), are characters escaped to ISO-8859-1? My recollection was that HTTP headers must be ISO-8859-1.... but looking at the docs for PHP's mbstring and the encoding_translation parameter, it looks like server-side handling of the request needs to account for other character set encodings. Do browsers honor charset specification as a form attribute, like
<form action=... method=... accept-charset="UTF-8">(looks like Struts supports this) or is it presumed that the browser always escapes unicode? Or perhaps they simply URL encode the characters so it's a non-issue? On the server side the must the request handling do this
request.setCharacterEncoding("UTF-8");
String raw = request.getParameter("foo");
String clean = new String(raw.getBytes("ISO-8859-1"), "UTF-8");
or is it all supposed to transparently just work (obviating String cleansing) if request.setCharacterEncoding("UTF-8") is used?
...for all of the hand-waving in the docs for ResourceBundle, etc establishing a clear practice for input String handling in a webapp remains murky.
As far as sending responses, is it safe to always just send UTF-8 and include "charset=UTF-8" in the Content-type header? Is it standard practice to presume that the client will send a request header Accept-Charset (which indicates what an acceptable response is)? If they send it and UTF-8 isn't on the list, must the server go through a big String re-writing exercise to encode response to the browser's preference or is UTF-8 presumed to be implicitly acceptable at all times?
So many questions... I'm still digging for anwers.
( Dec 12 2004, 11:51:01 PM PST ) Permalink View blog reactions
Tuesday December 07, 2004
Runtime inseration of struts tiles in Velocity I've been impressed with TilesTool, it comes in the Velocity Tools package. It runs Velocity views through the struts MVC machine for processing reusable "subviews". However, there's no support for runtime insertion of components!
You can do this in tiles-defs.xml
<definition name=".dog" extends=".animal.layout">
<put name="body" value=".dog.display" />
<put name="head" value=".dog.head" />
</definition>
<definition name=".cosmos.head" extends=".head">
<put name="titleKey" value="dog.title" />
</definition>
<definition name=".dog.display"
controllerUrl="/dog.do"
path="/tile/dog.vm"
/>
and so forth. Declaritive tile composition works just fine. But what about programmatic composition at runtime?
With JSTL and struts, I can do this:
<c:forEach var="bit" items="${kibble}">
<tiles:insert page="/tile/bark.jsp">
<tiles:put name="bit" beanName="bit" />
</tiles:insert>
</c:forEach>
I would imagine that the Velocity equivalent would look like this:
<ol>
#foreach ($bit in $kibble)
$tiles.put("/tile/bark.vm", { "bit" : $bit })
#end
</ol>
but alas, it's not implemented by TilesTool. I can work around this by moving "bark.vm" to its own velocimacro but that it fugly as hell. I would prefer parameterized components.
( Dec 07 2004, 06:53:07 AM PST )
Permalink
View blog reactions
Monday December 06, 2004
Servlet container forward from inside Velocity I ported some JSP UI code to Velocity, it's been fun learning the Velocity paradigm (being able to cleanly process template components outside the container rocks). One of the things in the JSP UI handled forwarding requests that bypassed the struts controller back through struts.
In JSP with struts tags, it looks like this (assume web.xml has "struts-logic" mapped):
<%@ taglib uri="struts-logic" prefix="logic" %> <logic:redirect forward="home"/>But what about Velocity? Well, it turns out that the VelocityViewServlet stuffs the basic servlet container things into the Velocity context, much like JSTL does in JSPville. Ergo, the $request object itself can be invoked like this:
$request.getRequestDispatcher("/home.do").forward($request,$response)
Seems kinda grotty to not be able to use struts symbolic name, but so far that's where my read of the Velocity docs has taken me. As I unpeel the onion, I may be inspired to subclass the VelocityViewServlet as a StrutsViewServlet... it seems like however you're invoking the rendering, you should be able to access, if present, other runtime services such as struts, spring, etc.
( Dec 06 2004, 10:05:35 AM PST )
Permalink
View blog reactions
Sunday December 05, 2004
memcached in a service oriented functionality ecosystem Among my efforts over recent months have been those focused on decoupling. Technorati has a very high update rate as it taps the ping streams, fetches update contents, analyzes links and keyword indexes the substance of posts in the blogsphere. Such a system doesn't work well when components are closely coupled; the availability of the whole system is subject to the whim of the system's weakest links. Often, weaknesses are combinatorial; the weakness of the whole is greater than the weakness of the parts. That's what I'm focused on undoing. Fixing weaknesses in the components is important but decoupling them first is more so.
When folks say "service oriented architecture" it still cannotes monolithicism to me. An architecture implies a level of structure definition that sounds rigid; can you re-pour that foundation to adapt redrawn plans? Software development agility and loose coupling should reinforce each other. I prefer to think of architectures and ecosystems. A service oriented functionality ecosystem supplies application functionality as a suite of services. Supporting requirements (as opposed to the core business requirements) such security, logging, persistence, redundancy and caching are each handled independently; they in turn may be provisioned as services that higher level services rely on. This is part of the evolution under way at Technorati; some of the changes are evident in Dave's recent posts but some are just revisions that we're quietly rolling out.
Queues and distributed memory caches are natural elements of a such an environment. In the December issue of Linux Journal, Technorati's use of open source building blocks such as memcached is discussed by Doc Searls.
This is the game:
A memcached server (or a set of servers) can be accessed over the network to store things in a table kept in RAM. When storing things, you can specify a maximum age for the cache entry -- if you go back to fetch it and the elapsed time since it was stored exceeds that age, it gets treated as a cache miss.
Storing things in memcached with the timeout parameter and invalidating cache entries works as long as you have consistent mechanism for calculating the key. If internally you're managing "stories" and each one has an "id" attribute that is unique (a primary key), that's a good candidate to store them with. So for instance putting memcache inside a content management system (CMS) "content service" seems natural. In babytalk code:
public Story fetchStory(int storyId) {
Story story = memc.get(storyId);
if (story == null) // perhaps more rigorous validation of the fetched object
return story;
story = StoryDB.findById(storyId);
memc.put(storyId, story, AGE);
return(story);
}
If it's difficult to determine whether something is new or an update because it doesn't have an id and uniqueness is determined by some combination of attributes, then the lookup cycle can be helped by caching with composite keys. It gets a little more complicated:
public Story fetchStory(Map atts) {
// encapulate whatever attributes uniquely identify a thing
CacheKey key = new CacheKey(attrs);
Story story = memc.get(key);
if (story == null)
return story;
story = StoryDB.findByAttrs(attrs);
memc.put(key, story, AGE);
return(story);
}
We're in the process of evolving Technorati's infrastructure to one that is loosely coupled, redundant and robust. Our use of memcached is one of the enabling technologies of that evolution.
( Dec 05 2004, 09:22:23 AM PST ) Permalink View blog reactions
Tuesday November 23, 2004
Adding a file system hierarchy into a CVS module I had a whole bunch of code that needed to added to a CVS repository. Normally with just a new directory or two and a few files, doing "cvs add" for each one is no big deal. But when there's a whole file system hierarchy that can be a real PITA. This is where "cvs import" comes in handy.
I usually only use "cvs import" to create a new CVS module but it can also be used to do a "bulk add." Maybe it's common knowledge for CVS jockies but it's easy to forget about unless oft-used. Here's the scenario:
- module webservice already exists with subdirectories perl, python and php
- a new subdirectory java is in your sandbox and needs to be added and it has a complex hierarchy of source packages, jar files and test data
- to add webservice/java do this
- cd webservice/java
- cvs import -m "adding java implementation" webservice/java vendorname tag
Friday November 19, 2004
Editing Perl with Eclipse and EPIC I've been bouncing between development in Perl and Java a lot this, two languages that I both love and loath at various times. One of the things that I love about developing in Java is using Eclipse. This week I decided to give the Eclipse support for Perl a spin. Well, I'll be saving my love for the Java support.
The only gripe I've heard about Eclipse that I haven't had a good answer for is the absense of Emacs key bindings. Otherwise, what's there not to dig about Eclipse?
I was hopefull that the EPIC plugin would provide at least some of those things for Perl development. This is what I found:
- Syntax highlighting
- Module outlining
- Dependency checking (i.e. "use" statements validated)
- The semantic checking hangs sometimes (a popup with "Building Workspace (Blocks User Operation) [thermometer bar] Invoking Perl Auto Builder" will just put it under spell for a brief period)
- Sometimes the dependency checking makes mistakes... it'll falsely indicate a problem with a an import (a "use" statement)
- No refactoring, build system, testing or perldoc support
In the meantime, you can enjoy the fruits of this week's labor by pulling it off of CPAN; that's where you can get WebService::Technorati. It's also part of the latest release of the Technorati web services SDK. Thanks to David Wheeler for turning me on to Pod::Simple::HTML ...I'm still trying to figure how he gets it to output nice docs from pod, mine didn't come out nearly that purty. Ah well, I guess that'll be part of next week's Perl fun.
( Nov 19 2004, 10:59:25 PM PST ) Permalink View blog reactions
Wednesday November 17, 2004
XML::Parser on Mac OS X I needed to fiddle with XML::XPath on my powerbook today, it depends on XML::Parser. Complacent with how most unixy things I want to do JFW on Mac OS X, I dropped down to my CPAN shell and typed "install XML::Parser" -- bzzzt!
It turns out that expat is not installed, grrr. So I fired up Fink Commander and had it gimme some expat lovin'. Tried it again -- bzzzt! This is what I did in the CPAN shell
cpan> o conf makepl_arg "EXPATLIBPATH=/sw/lib EXPATINCPATH=/sw/include" cpan> install XML::Parser-- ding-ding-ding! We have a winner! XML::Parser installed! Thereafter, XML::XPath JFW'd and I'm on my way. ( Nov 17 2004, 04:53:44 PM PST ) Permalink View blog reactions
Sunday November 07, 2004
Tomcat's "Content-type" header parsing busted? One bit fun this week was trying to figure out why some XML output I was working was mangling characters. I thought I was doing all of the right things as far as handling the data goes. Well, I think I was but Tomcat 5.0.28 wasn't.
I poked around the Jakarta bug database and the only mention I could find that close was PR 31442, which described having this
<%@ page language="java" contentType="text/html; charset=UTF-8" %> <%@page pageEncoding="UTF-8"%>and saying that the text was coming back ISO8859-1 when the page is requested as a GET but not as a POST. Well, someone from the Jakarta project marked the bug INVALID glibly saying to ask on the user's mailing list and look at the Connector configuration because it's not a bug. WTF? Are you kidding?
Now I looked around in the Connector stanza's that come in the server.xml and see no mention of encoding configuration attributes. I've got a real simple test case.
<% response.setContentType("text/xml"); %>
triggers no funny encoding behavior, I get the data out as good old utf8 just as I wanted but if I did this
<% response.setContentType("text/xml; charset=UTF-8"); %>
....kablooey! Mangled encoding! That's just wrong. And if it's not wrong, I think it warrants a better answer than RTFM on the Connectors.
And the problem may not just be isolated to JSP handling. Judging from other reports that are turning up in Google's index pertaining to SetCharacterEncodingFilter, it's affecting the filter implemetation as well.
( Nov 07 2004, 02:46:58 AM PST ) Permalink View blog reactions
Saturday November 06, 2004
What are all of these stupid people doing in my country? There were some severe system problems last week that pretty much knocked this site out of commission, I'm hoping it's all in the past now.
In the meantime, the Big Lie that waging war on Iraq has some relationship to 9/11 and terrorism apparently has been successfully Jedi mind-tricked into the American psyche and we're destined to have four more years of high crimes and misdemeanors. It just makes me wonder what is up with the rest of the country. Plenty of folks abroad are, evidently, equally perplexed by this election, as we see in a recent Daily Mail cover.
If you're single, there are some Canadians offering asylum. I'm thinking of packing up the family and moving to New Zealand or something. Just to keep track of where I don't want to be, I'm reckoning with the map: 
Source: http://www.electoral-vote.com/
|
Do you live in a state of stupity? Apparently 59,054,087 of you do.
|
I was never particularly enamored with John Kerry, in fact I would've been happy with a Wesley Clark-Howard Dean ticket. Nonetheless, I don't think Kerry would have been so driven to a war that he would have disregarded counter-indicative intelligence and the advice of allies to wage one.
Another idea that no longer seems entirely ridiculous is to secede from union. Seriously, who wants to be part of this country when California's vote is under represented in the electoral college and yet our youth are being sent to Fallujah to wage war against a culture and people most people here know little of. We're the country's vegetable stand and it's cannon fodder. I don't think so. Suddenly, Ecotopia sounds like a reasonable proposition.
Independence for California! ( Nov 06 2004, 01:12:57 PM PST ) Permalink View blog reactions
Saturday October 23, 2004
Rock stars and actresses What is it with rock stars and actresses? Hopefully there's no home made videos of them but apparently Lars Ulrich and Connie Nielsen are a thing.
Remember Lucilla in Gladiator? Yea, that's Connie Nielsen.
| Here's a guy with two sons and a wife of 7 or 8 years going to fashion shows, art auctions and movie premiers with his Danish girlfriend. Oh, Lars: you're so damned hollywood! Apparently the paparazzi in Denmark have kept tabs on them as well. |
|
Back in the old days o' Metallica we had loads of fun but didn't go to fashion shows, art auctions and movie premiers. We didn't sip fine wines either. Oh well, I hope the dude is happy.
( Oct 23 2004, 07:29:35 PM PDT ) Permalink View blog reactions
Friday October 22, 2004
Technorati's referer-driven cosmos There's a it's-so-simple-it's-cool feature on Technorati. You can construct a link to get the cosmos for a page simply by linking from it a specific URL.
After raising the notion with Tantek, he plugged the trivial bit to enable this on the Technorati site..
Check it out http://www.technorati.com/cosmos/referer.html (ok, so I'm not very popular in this big 'ol cosmos but anyway...). This is what you do:
Try it! ( Oct 22 2004, 07:54:11 PM PDT ) Permalink View blog reactions
Thursday October 21, 2004
Technorati Party All work and no play makes Ian a dull boy, so to keep me jazzed Technorati is having a party!
OK, I lied. It ain't about me, it's about our new office and the major milestones that Technorati is achieving, the agony of startup setbacks and the ecstacy of... having fun! The details:
- WHEN
- Thursday, October 28, 7 p.m.
- WHERE
- Technorati, 665 Third Street #207, San Francisco
- WHAT
- A party to catch up, and celebrate the move to our new offices!
- RSVP
- rsvp@technorati.com. As space is always limited, please be sure to RSVP.
Here's Dave's original post.
( Oct 21 2004, 04:34:11 PM PDT ) Permalink View blog reactions
Wednesday October 20, 2004
Eclipse and Test Driven Development I've got some coding going on in Java, Perl, Python and PHP at the same (thus often self-query frequently: "Oh wait, which language am I working in for this bit?"). I've been using, at least partially, Test Driven Development with my current Java effort and It Is Good.
I've used Eclipse extensively in the past but since there was kinduva a long hiatus in Java development for me, it felt both novel and familiar to install it on my powerbook for the stuff I'm currently working on. I think that liberated feeling of knowing that That Thing Works So I Can Move On Now is referred to as being "Test Infected." Which, unlike the flu, is a good kind of infection. One of the things I've been trying to be more consistent about is writing tests prior to, or if not, concurrently with, application code. I strongly recommend reading Kent Beck's Test Driven Development (who BTW appears to have a work in progress about Eclipse), TDD is a quick read but more than any particular valuable agile techniques per se it offers a good outlook on how to think about what you're doing when you sit down to write code.
| Well, I was writing a few initial test cases for a class I was working on and noticed something new (to me anyway) when I clicked on an error that Eclipse flagged. The non-existence of the method in the called class was identified and Eclipse offered to stub it out on my behalf, this is a huge win as far as expressing an API in a test case and continuously filling in the functionality development "TODO" list. Everytime I want to add a new public method, I'll write a new test case and let Eclipse bootstrap the implementation. | |
|
|
Read it:
|
Now if I could get that level of simplicity and automation working with Perl, Python or PHP, I'd be jammin'! Sure, there are testing frameworks for each of these but Eclipse really streamlines that whole TDD cycle. |
Forget about the flu shot hysteria, get "Test Infected"
( Oct 20 2004, 11:04:06 AM PDT ) Permalink View blog reactions
Tuesday October 19, 2004
The MUNI gods frown on me I specifically didn't walk from BART today because of the downpour of rain, I transferred to MUNI to take the N-Judah down towards the Caltrain station. What a mistake!
Sure, I have a rain parka but that's not the point, I wanted to avoid having my backpack get saturated. So after waiting about 10 minutes, a two car N-Judah rolls into Embarcadero station at aroun 8:40 or so with the robotic audio announcement "two car. mission bay. now approaching...." A crowd piles on; I don't think the MUNI scheduling people have realized that a lot of people work near the ball park.
Out on the surface streets at Harrison, the driver announces that it's the last stop and that the next N would arrive in 3 minutes! Why would they cut the route 3 stops short?! Everyone spills out into the deluge of wind and rain; the shelter there hardly has adequate room, so most of us are getting wet. BTW, that was train cars number 1481 and 1483, in case any MUNI hacks are reading this. Almost 10 minutes later, I ended up getting back on the first train as it showed up on the opposite side of the platform, going the other way -- I was getting concerned that the rain would soak through my backpack and get my powerbook wet. On the way back to Embarcadero station, I saw the "next train" -- it was packed; looked like there would barely be room for all of the folks who'd withstood the rain waiting for it. I finally got another train and got to the office by 9:10am. Thanks for the convenient adventure, MUNI!
( Oct 19 2004, 09:36:51 AM PDT ) Permalink View blog reactions
Saturday October 16, 2004
Enhancing web application component "testability" with IoC I heard from some friends who are mid-flight on a project using Spring, Hibernate, Struts and Tomcat. A lot of the complaints that can be lodged against J2EE are philosophical and technical but there are some down and dirty practical reasons to un-EJB-ify.
- The objections on principal may be technical
-
- Intrusive inheritance hierarchy and mandatory interfaces, narrowing and recasting, gratuitous class proliferation and so forth (does the framework work for you or do you work for the framework?)
- Container managed debugging nightmares
- The philosophical objections are perhaps a general complaint about the Software Industrial Complex
-
- J2EE is handed down from The Cathedral
- JSR's and the JCP don't constitute inclusive openness, it's a "club" that you're either in or out of
- Java is not open source
- The practical problems of EJBs
-
- Using the containers can be a morass of deployment descriptors and dependencies, slowing down project spikes
- Component dependency on the container requires extra plumbing for testing, which also anti-agile. Specifically, if you want to use CruiseControl and Cactus together, get ready for a visit from an old friend named the OutOfMemory exception .
While the practical problems of test-friendliness seem to be addressed by IoC, it sounds to me like some of the morass-of-deployment-descriptors problem are inescapable. Mapping services into POJOs declaratively (as Spring, Hibernate and Struts all require) isn't a walk in the park; until there's a linter and/or metadata/xdoclet support for easing how services can be mapped in given a specific runtime context, the pain of diddling XML config files and finding errors at runtime are going to be part of the landscape. Nonetheless, automated unit testing without cactus is a huge win in my book -- leaving CruiseControl to run unit tests and having Cactus runs reserved for manually-invoked integration tests sounds like the way to go.
I'll be steering clear of the traditional J2EE containers such as JBoss or Weblogic.
( Oct 16 2004, 09:33:02 AM PDT ) Permalink View blog reactions
Thursday October 14, 2004
Add To The Reading Queue I should take a week off of work so I can sit around and read (like that could ever happen).
So many books, so little time
- Hibernate in Action
- If the slashdot review glowed any brighter, it'd spontaneously combust.
- Agile Database Techniques : Effective Strategies for the Agile Software Developer
- Weighing YAGNI against the PITA'ness of data model changes has bit me before. Data modeling and iterating over model revisions are often the hard part in agile development cycles. I'd expect Scott Ambler to offer useful insights into lightening what is often the heaviest part of a projects development.
- The Innovator's Solution: Creating and Sustaining Successful Growth
- As one who actively looks for opportunities in subverting the dominant paradigm, I'm interested in Clayton Christensen's follow-up to The Innovator's Dilemma.
Someday real soon, I'll build a little app with Amazon's ListLookupOperation to integrate wishlist items with my blog and perhaps fold it into an Attention.XML data source.
( Oct 14 2004, 10:36:56 AM PDT ) Permalink View blog reactions
Friday October 08, 2004
Who is the Master of Puppets? Could this be a picture of a wired George Bush on Salon.com?
See for yourself, Bush's mystery bulge I hope Kerry brings it up in tonight's debate, may be pat him down.
I will occupy- Metallica, 1986 ( Oct 08 2004, 11:49:12 AM PDT ) Permalink View blog reactions
I will help you die
I will run through you
Now I rule you too
Come crawling faster
obey your Master
your life burns faster
obey your Master
Master
Master of Puppets I'm pulling your strings
Interfaces and wire protocols for web services A lot of great feedback emerge during and after the Technorati hackathon.
The hackathon percolated a lot of interesting ideas. One of the bits of feedback that caught my attention was from Chris Fry:
There are some drawbacks however to not having a WSDL and to not using SOAP. (1) You are bound to HTTP; (2) If you version the contract how do you notify your clients? (3) Related to 1, no SOAP Headers; (4) No public contract other than your documentation.
This sounds great, getting Technorati developers out of the wire-protocol-awareness business (unless they want to be in it) is one of my goals for future development efforts with the Technorati API. The direction I'd like to take it is an "all of the above" implementation where API consumers can fiddle with the low level if they want to (via REST w/XML, REST w/XOXO, xml-rpc or whatever interface to du jour is desired) but also provide a SOAP interface for those who want to use WSDL to skip all of that.
We have some work to do internally at Technorati to get us to that point though.
( Oct 08 2004, 10:58:31 AM PDT ) Permalink View blog reactionsGoogle's Velocity Accelerating If you measure development velocity as the number of funcitonality points per time interval, then Google is definitely accelerating.
Check it out:
- Gmail Atom Feed Coming?
- According to internetnews.com, Google is enabling feeds of gmail mailboxes to make feed readers lightweight mail clients (well, readonly clients). I'm imagining subscribing to mailing list digests through gmail and reading them through a feed reader will become an adopted practice. Additionally, gmail's search capabilities are getting some augmentation.
- Google Searches Books: Google Print
- As has been widely publicized, Google Print seems to be Google's response to A9 ...so far Google Print has been down half the time I've tried to use it but I wouldn't bet against them executing well on it.
- Google queryable via SMS
- I'm imagining all kinds of goodness out of this... querying Froogle for prices while you're shopping bricks-n-mortar venues, looking up local bars and restaurants (and Dodgeballing them!)
- Gbrowser
- John Doerr says that he doesn't see browser in Google's future but still, if it quacks like a browser builder, it's probably a browser building duck.
So what's next from Google? An Orkut that isn't all-Brazil-all-the-time?
( Oct 08 2004, 10:31:29 AM PDT ) Permalink View blog reactions
Tuesday October 05, 2004
Mount St. Helens VolcanoCam Wow, the pictures this morning are fantastic!
If you haven't checked it out, stop what you're doing and check out Mount St. Helens right now!
( Oct 05 2004, 10:03:09 AM PDT )
Permalink
View blog reactions
Monday October 04, 2004
The Bad Time Of Year With the Astro's defeat of the Rockies yesterday, the Bad Time of Year has begun.
Sure, this is always the time of year when the weather gets a little colder, the sun sets earlier and the volume of junk mail catalogs arriving from the postal service goes up. But what makes this Bad is obvious: the Giants aren't playing baseball until the spring. I just can't wait until spring! Any claims that the lack of October play is Felipe Alou's fault is just silly. The last few years' bad deals with Sidney Ponson and Damien Moss and bad luck Rob Nenn have taken away payroll that coulda been used more productively -- those situations are more blameworthy. But hope springs eternal and this spring we should see Jessie Foppert and Jerome Williams in the rotation and if some good luck (and payroll availability is on our side) Moises should be a Giant and batting behind Bonds.
( Oct 04 2004, 10:04:14 PM PDT )
Permalink
View blog reactions
Wednesday September 29, 2004
Coming Up For Air Who has time to blog? The procession of emergencies that has afflicted Technorati lately has left me little time for peace (or even, as in the case of the only previous post I've had time to make in nearly two weeks, civility).
As reported on by Dave, the chaos really crescendoed last weekend with an electrical outage at the colo facility. The service is on the mend but we still have a ways to go. The database repairs are proceeding. The hardware upgrades are mostly completed and it looks like we're going to setup camp someplace that will be a huge step up from the ghetto colo we've been in.
Here comes the sun. It's alright.
( Sep 29 2004, 10:37:10 AM PDT ) Permalink View blog reactions
Monday September 20, 2004
The Optimal Time to Optimize in Operations One of the recent hassles I've had recently was with a hardware migration that needed to proceed quickly. The clock was ticking down on the disk capacity utilization on some key database hosts. Now suppose one of the sysadmins wanted to perform "preventitive fsck's" and "table consistency checks" -- when you're dealing with over 100 GB (closer to 200 GB, actually) of data, these are not quick propositions. In fact, they might take days. Ergo, just not feasible. Given the time, would it be optimal to sanity check every subsystem's functionality? Perhaps. But when struggling to beat the clock, you just gotta say, "Not now, Poncho!" Sometimes the only effective action is fast action.
First of all, the only times I've ever needed to do a reiserfsck has been after a cold power loss (and reiser is usually fine even after one of those). So the fact that this sysadmin wanted to do a reiserfsck "preemptively" made even less sense. As far as doing a table consistency check, with innodb this is never needed on an anticipatory basis. In my experience, innodb either is able to keep itself consistent with its own journaling or it's just hosed... not a lot of grey in between. Again, the only exception has been in cases of a cold power loss. Sure, sometimes other hardware problems, low level disk defects, will manifest themselves as problems with the filesystem or a database's data file. But usually there are other indicators as well (kernel complaints in syslog, etc). But even with the dependency stack accounted for and checked, it's no guarantee against failure.
Sometimes the optimal course is just the fastest one between where you are and where you need to be. Choosing the deliberate and cautious route, dwelling on unnecessary optimizations, may in fact be the slow and steady road to.... failure! In this case, if we'd followed the course of doing every unnecessary system check possible, we'd have run out disk space and crashed these particular databases.
Stop optimizing. Just shut up and get it done already.
( Sep 20 2004, 11:51:27 PM PDT ) Permalink View blog reactions
![[Valid RSS]](/images/valid-rss.png "Validate my RSS feed")