Ohhhh… geeks don’t usually have girlfriends. ;-)

Oh how times have changed. Here’s an old photo of the device, courtesy of about.com:

So. I’ve been carrying around a Pearl for about a year and recently purchased a shiny new iPhone 3G. It’s been reviewed all over the place, and if you read my blog, you almost certainly already know about, ahem, visual voicemail. So this article won’t mention SMS messaging and the “ooh” factor of the entire thread in view when text messaging. (Besides, my pearl does that, too, albeit with less animation.)

I’ll start with the cons first. Everyone and their Mom wants a friggin’ iPhone, and these things are supposed to be the most innovative devices since some goofball thought selling stuff (starting with porn) on the darpanet using a simple markup language might be a good idea. (If you don’t know the lore to which I’m referring, please do comment saying so. I’ll be happy to post the appropriate Wikipedia links if it’s that unclear.)

In other words, everyone wants one. I’m choosing to not hop on that bandwagon. At least not up front.

Why iPod Sucks

1. Calendar, schmalendar. Blackberry talks to my google calendar seamlessly. I can transfer an ical invitation in my inbox to my calendar with a single click, and before I can open 3 browser windows on my desktop, it’s already in my online google calendar and my offce manager knows about it. iPhone, on the other hand, a year after initial release, uses MobileMe, a subscription based service on top of the $40 minimum I have to pay AT&T every month. And then it talks only to Exchange. How well does it work? I dunno. I’m not gonna go install Exchange just so I can get a locked-to-outlook version of my perfectly good Google calendar. Sure, there’s an alternative out there, but it looks — ahem — a bit dodgy to me. Plus that ain’t gonna be free as soon as they can hire a designer for their web site.
2. I can sync contacts. To Outlook. With a Cable. Wireless? Just kidding!
3. Oh, wait, I can’t actually sync. I can DELETE EVERY CONTACT ON MY PHONE and then REWRITE them from Outlook. WTF? I’m sorry, I know that my little boutique software firm would have a hard time writing a seamless contact synchronization algorithm in less than three months. Hell, it might take four if you (gasp) asked for synchronization against more than one contact management system. But you’re Apple. You have $25 billion in the bank. C’mon, guys.
4. I can’t sync using Bluetooth. Sure, it might be slow. Sure, it’s not standard on all laptops. Heck, do MacBooks have them? Maybe that’s why there was this subtle omission. But what about that inevitable road trip where I forget my USB cable? Having Bluetooth-to-laptop connectivity at least as a backup would have been a nice value-add without a lot of additional engineering resources.
5. Speaking of cables, why do I need one? EVER? Let me repeat that question with a little more context. Why does my bluetooth-having, wi-fi sensing, 3G Network boasting, so cool it helps me get laid, star trek communicators are so 20th century, always-connected device EVER NEED A F***ING CABLE? You want to use one because it’s faster the first time you load up 14 Gigabytes of totally legal digital music on your brand new “I’m a Steve Jobs groupie” badge, sure. But a device with so many connectivity options should have an over-the-air (OTA) option for everything. EVERYTHING.
6. I can’t download a song from iTunes unless I use Wi-fi. Thanks, AT&T.
7. Mail has no search function. And, yes, my crackberry does.
8. Sideways keyboard on web pages, but not in (ahem) the mail application. I don’t much like the software keyboard, anyway.
9. I can’t delete podcasts. I can’t even mark podcasts for deletion on my next cable synchronization. Insert another expletive here for the cable issue. Yes, again. This is meaning of life stuff here, Jack.
10. I can’t download podcasts. Not even from iTunes over wifi. So I have to cable synch to my laptop connected to the same stinkin’ wifi network just so I can get my latest Robert Reich commentary and last week’s This American Life. Yes, I know, I’m a bourgeouis pampered bastard, but I grew up poor and I work harder than you, so bite me. =)
11. The software keyboard, take two. On a Blackberry with a full keyboard (e.g. the Curve), I can type probably 35 words per minute. No, not the 75-90 I can blast out when blogging passionately on a full-blown keyboard, but still not too shabby. Good enough, in fact, to manage a software project completely from a blackberry, especially with solid administrative help. I can reply to an email with three paragraphs in two minutes, including the requisite backspaces. The apple predictive capabilities with the software keyboard are top-notch, but it’s still dependent on software to guess at your intention after you fat-finger your way to gobbledy-gook heaven. You will remember predictive texting, right? You know, that feature you turned off on your old Nokia the minute you started trying to send your third SMS ever back when too much of the country still thought W. was a good president? (I told you so, btw. Yes. I did. 5 years ago. Check my archives. =)
12. Camera — old news, yes, but there’s no light. No zoom. My Blackberry has that.
13. Camera — wish list. It’s a good camera. Good enough to replace my desire for a Canon/Sony/Nikon. I’m not a professional photographer, and I’m well aware that I don’t need more than 2 megapixels to warrant sending Shutterfly a few dollars. But there’s no video (my 5 year old Sony Walkman phone did that for crying out loud), there’s no flash, and there’s no — and this is the biggest issue I have with all camera phones — lens protection. Would it be so hard to allow us consumers the ability to protect that lens so the damn thing can take a good picture for more than 3 weeks after you unwrap the thing?
14. The software keyboard, take three. You can’t send a text with one hand. This is pretty much across the board, don’t even try. Not that I’d ever consider sending a text message or email while doing something (cough) Holding a steering wheel (cough) with my other hand.

Later in the week, I’ll post an entry on what I actually like about my iPhone. Despite all of the above complaints, yes, it still has its merits… but the verdict is still out on whether or not the Blackberry gets retired.

(It’s a thinly veiled ad for google products, but it is food for thought about (a) market opportunities for your own pursuit, and (b) what some google engineers are probably working on.)

• Today, only the largest companies can afford to hire consultants and experts. In the future, even small companies will be able to purchase on-demand expertise and other services via the Internet.
• Today, marketing intelligence are costly reports describing data many months or years old. In the future, small businesses will have access to real-time data on market conditions.
• Today, only the largest companies can run expensive experiments with their advertising campaigns. In the future, even small business will be able to run carefully controlled marketing experiments that will enable them to better reach their potential customers.
• Today, only large companies can sell products in many countries. Tomorrow, businesses of any size can use online services and outsourced logistics to buy and sell in every corner of the globe.

Bummer. Here’s a quick refresher on some things to make sure you audit on your own site.

• Use a unique account for your web-originating database access. This means a unique account with its own password and credentials as limited as possible. Making your DB access credentials the same as anything SSH-capable is asking for trouble. You don’t use your ATM access code to login to your email account, do you? (You’d better not!)
• Lockout all but the necessary functions (usually just insert, select, and update) at the database level for your new database access account. If you need to give this user “delete” access, too, maybe you need to re-think your application’s logic. (Deleted should be handled via a flag in most applications, not by actual removal of the data.)
• Place all of your common core server-side includes outside of your root web context. So if your web server’s root is “public_html”, put all of your shared libraries, especially your core database connection library, above public_html in your directory structure. If you can’t do this on your web host and you give two craps about the security of your site, get a new host. In English, this means that no one on the web should be able to surf to any of your core libraries. Requiring authentication to get to these files might feel just as good, but it isn’t.
• Chmod 777 is a Very Bad Idea.

This is by no means definitive, nor should it take the place of a professional audit by a skilled professional. However, if your development staff consists of your left brain, your pooch, and white board, this is a decent place to start.

After about 20 seconds of discussion, someone said, “so they have a whole lot of servers then, huh?”

I retorted that it’s probably the single biggest application on the planet in terms of computing power. Of course, I followed this up by silently asking said google cache about this question. Interestingly, after a half dozen tries (”largest server farms”, “largest application”, “largest global applications”, etc), I couldn’t find a definitive list. Heck, I really couldn’t find any list.

So I thought I’d put one out there for public consumption. Please debate and correct me. It’s all guesswork.

(This is based on total flops devoted to a single application infrastructure, not based on other trivialities, like number of CPUs, number of racks, or raw storage capacity. There’s also probably a distinction between different branches of government, but I’m way out of my league there.)

1. Google. The www cache, youtube, picasa, and all the rest.
2. The DoD. You know, those guys that invented the internet?
3. Amazon. S3, alexa, the wayback machine, and of course their tiny product database.
4. Microsoft. Hotmail, Live, windowsupdate, msdn.
5. Yahoo. Their tombstone aside, they’ve got a web cache and tons of services. All of that info has to be stored somewhere.
6. China’s IT Infrastructure. Suppressing all of that free speech has to take some computing power, right?

Let the debate ensue…

Using Google for shared calendar, address book, and email is the avenue I’ve been exploring. I also considered installing Kolab, an open source groupware solution, but that seemed to me analogous to installing my own Exchange server on a hosted Windows box. The only benefit there would be the presumption that an open source solution like this would have support for a wide variety of products, including my Thunderbird and Blackberry requirements.

Here’s what I’ve found works in favor of the big G.

Pros of Using Google as a Groupware Platform

• Calendar Sync to Blackberry is solid. I can only sync on calendar at a time, so I still have the problem of merging my personal gmail account with my newer google apps account (hereafter referred to as “corporate”), but that strikes me as a relatively small problem since my schedule planning is typically only a few weeks out at the most. I can also display my own personal calendar by “sharing” it with my corporate self and then sync it down that way. I now have both calendars on my Blackberry happily syncing both ways. I do wonder if there’s a limit to the number of calendars I can share…
• Mail is a no-brainer. Imap support means that, aside from the 5-10 minute lag, thunderbird and blackberry work just fine with both the personal and the corporate account. On the merging side of the equation, I’m still waiting for the corporate account to finish fetching my 50K messages from the personal account via POP (it’s google fetching from google, so it’s obviously throttled heavily), but I can continue to work from both accounts for the next few days.

Cons of Using Google as a GroupWare Platform

• They claim ownership to you data. Sure, “do no evil” is a great edict, and sure, they’ve done nothing major (although I suppose the “caving” to China’s censorship demands is arguable) to violate the public trust, but the details in the Terms of Use are pretty alarming if you’re developing any sort of intellectual property, which I happen to do every day.
• Contacts are a mess. Aside from my earlier complaints, I’ve been trying out Zindus for Thunderbird-to-Google sync, and while I’m very impressed on their development speed (the Google Contacts API was only published 45 days ago or so), the plugin still needs some work. I attempted to move my contacts from plaxo over to my corporate gmail account, since the plaxo data set should contain all the data from my legacy personal mail account plus data from plaxo itself, linkedin, and my mobile phone contact list (via blackberry –> outlook –> plaxo manually). However, I ran into a number of issues with the Zindus merge:
  • The Plaxo export to Thunderbird felt a little off. I did it at about 4am last night, so I can’t recall what was wrong with it, but I ended up tossing the Thunderbird LDIF from plaxo and merging with Outlook 2007 instead. Then I imported from there from within Thunderbird. That’s fine, but it’s worth mentioning.
  • The Thunderbird-Google sync halts when google denies a contact. So my plaxo data, which contains duplicates all over the place (a single contact might be represented as a record from the old gmail account, linkedin, and plaxo, and all contain the identical primary email, which google denies and Zindus pushes back on the user. So, basically, I’ll need to manually clean up my contact list in thunderbird before pushing it upstream. Good thing I have administrative help now.
  • Zindus does not “match up” contacts from my mobile phone. These contacts are generally an abbreviated name (i.e., a nickname, like “johnnyf” instead of “Johnny Fuery”) and a single number. The location of the number varies with regard to specific field (mobile, business, home, etc.), largely because I’ve migrated phone contacts several times over the years. Only 1% contain an email address at all. It would seem that Zindus relies on that email address for keying, becaues after attempting a dozen or so syncs, which all halted at various points in the process because of Google denying the contact add based on a duplicate primary email, the contacts that originated in my Blackberry are now duplicated a dozen times over.
  • Google won’t let me delete more than 20 contacts at a time. So starting over is going to be a major pain in the you-know-where. I now have 3500 contacts in my corporate gmail account when I started with 1900 or so in plaxo.
• The Google Apps “shared address book” is actually only intra-domain, so I can’t share my “personal” contacts. In other words, it ain’t a shared address book. I know the 8 people with accounts on my domain already and they’re IN my personal address book.
• The workaround would be to sync multiple Thunderbird installations via Zindus against the same “personal” address book. One could conceivably setup a Google account solely for contacts, configure Thunderbird + Zindus to sync against this single account, and voila — group contacts. The scalability of this model is cause for concern, however. I’ve yet to try it on more than two desks with my personal account. Zindus does authenticate against both my personal and corporate accounts, by the way.

At this point, I need quite a few utilities to make this work.

Market Opportunities

Please let me know (via the now-working Contact Form or a comment below) if you’re interested in these. Most of these I foresee being donationware and/or ad supported, but I’d also be interested in the economic value to you.

• Google “Delete all contacts” script. Go to a web page, enter you gmail login data, click a button, and flush your account. Google probably has some throttling on it, so it would probably be a “we’ll email you a confirmation in a few minutes” type of experience.
• Google contact migrator. Move contacts from an old account into a new one. See previous item for proposed interface.
• Improvements on the Zindus tool. I’d rather not write my own, but minimally, it should key against more than just the email address (the FAQ says it does, but my experience indicates otherwise) and provide a preference for skipping past problematic records. Problematic records should be outputted in a summary or log of some kind so they can be handled manually. A re-sync should not create still more duplicates (keying against more than just an email address would satisfy this).
• Thunderbird (Lightning)-Google Sync remains unanswered. GCalDaemon can do it, but it’s an awful lot of code sitting on clients for that one feature, and I’ve yet to be able to get it to work with a non-personal (Google Apps based) calendar.

I’ve used their FVS318 Router/Firewall product for years, more or less standardizing on it as the platform of choice for all of my client’s offices. That backfired for me this week.

On Monday, I got a call from a brand new client from an office I’d never visited before. It was an emergency. Their internet was offline. Great, an opportunity to be a hero, right?

Well, I diagnosed pretty quickly that their existing gateway was dead. No worries; that’s what Best Buy is for — overpriced networking gear in a hurry. After getting a little lost first, of course.

Everything worked fine for two days. Then, today, the Firewall would intermittently drop it’s connection to the internet. A reboot fixed it every time… for 5 minutes. I troubleshooted my happy little arse off for 90 minutes on it, then decided to take it back and exchange it. What the heck?

In the meantime, naturally, I grab a spare Linksys Wireless Router WRT54G device and get it working in… oh, six minutes.

I get the second Netgear Firewall unit in, configure it identically, it works for about fifteen minutes, then ceases to route. Heck, it doesn’t even respond to a ping from the internal network any longer, let alone show me it’s own built-in web utility. Or, you know, work.

Screw it. My $35 old school Linksys device wins. I am never buying a Netgear device again.

I’ve been scratching my head trying to think of what went wrong with the device, and the only difference I can think of in terms of configuration is the fact that it’s using a static IP instead of PPPoE like the other dozen I’ve installed in the past few years.

So, sell your NetGear stock. Ditto on whomever manufactures D-Link products for the same reason. Buy Cisco.

P.S., Yes I’m still alive. Just been busy.

I caught a posting from the google blog earlier today that Google’s legal big shot posted on the topic. In short, he basically says that the internet is fun place, it’s fun because it’s based on open standards, google’s all about open standards, and Microsoft is bad mmmmmmKay. Actually, you can skip all the prefacing. He’s pretty bold with the damning rhetorical questions, asking “Could Microsoft now attempt to exert the same sort of inappropriate and illegal influence over the Internet that it did with the PC?” (You can read the full article here.)

Now, I like Google as much as the next guy, and I had fun beating up on Microsoft in the late 90s, too, even as jumped up a socioeconomic quintile by servicing Microsoft products. But the notion that poor little Google is worried about big, bad Microsoft coming in and peeing in the collective bowl of Cheerios is a pretty big stretch, even if Google really could claim that it has never done any evil (Can it?). Google has $15 billion in cash, a market cap of $150 billion or so, and commands between 56% and over 77% of the internet search marketplace depending on who you ask. Oh, and if that weren’t enough, what verb do you use every day to describe the action of performing a text-based search on the internet?

No, sorry Google, but as much as I respect every single person I know that collects a paycheck from you (damn that’s impressive), as much as I trust the details of my life to you (email, desktop search, docs… oh wow do I feel vulnerable now), and as little as I use any of those other search engines, I’m gonna have to go ahead and side with Mr. Ballmer on this one.

Someone needs to check Google’s power. No matter how good your king is, there is something inherently questionable about monarchy isn’t there? Unbridled, unchecked power, even if laced with good intentions, even if executed extraordinarily well, even if absolutely magnanimous, is very, very dangerous. Kings have spoiled kids. And corporations have to answer to finicky shareholders. It’s not right when the president consolidates power away from the people, and it’s not right when any one single company has the power to control the internet.

And if Microsoft can’t become a worth competitor, I’m not sure who can. So, c’mon, Redmond! Get your act together, harness Jerry and the Yahoo boys, and start releasing some great products! You’ve got a lot of catching up to do…

Description:
A small (handheld or slightly larget) device that can tell where metal is in the walls of your home. It’s tuned to ignore [most] nails and other metal fastening devices, like picture hangers and staples. You walk around your home following some basic directions presented on an LCD screen. The device is emitting some low-level radiation that allows it to pick up conducitive materials with reasonable accuracy. It then assembles a schematic for the wiring in your house. This would be plugged in to your PC or network (via USB or ethernet) and upload the raw data to a web-based service that crunches the numbers and assembles pretty graphics.

Pricing:
$400-$600 retail. Sold at Lowe’s, Home Depot, and Sears.

Value:
Troubleshooting electrical problems in your house would be a cinch. You’d be more willing to cut into your walls to add desired items, too, because you could go into the job with a keen awareness of just how big a mess you’re going to make.

There’s also the commercial angle (probably where something like this would start). Sell the unit to electricians only at a higher price and let them offer web-based schematics to all their clients for free. The web site and software would be paid for via the add-on services and monetization methods (below).

Add-ons:
Specific recommendations could be provided once data is gathered. This could be with an eye towards environmental ends (replacing something could save XX tons of carbon, for instance), financial savings (e.g., save $300 on energy this year by spending $250 now), or even simply safety (because plugging in 17 appliances and a hairdryer on your tired, old wiring could start a fire).

There’s certainly an angle where the uploaded data gets sold to local vendors (privacy and permission rights would have to be observed, of course). I’m certain there’s a long list of electricians that would like to get a list of 20 leads monthly that have knob-and-tube electrical wiring or are still running with old-school fuses. The list of homeowners generated, too, would be extremely valuable as a marketing tool, even without the schematic details. (Homeowners who dabble in home improvement and have disposable income… yum! Not to mention the fact that we can make certain conclusions about their net worth based on their physical address.)

Cool Geek Factor:

• Mash up the data with Google Maps/Earth.
• Scanning for metal through sheetrock while ignoring other materials and packing it into a consumer device is a non-trivial engineering problem.
• Hello? Adobe Flex based graphs of your home’s wiring? Yeah!

Screw the “HTMLSelectElement object”. Whatever that is. ;-) Just append a text child and set the appropriate attribute.

This works in both IE and Firefox:

myOpt = document.createElement('option')myOpt.appendChild(document.createTextNode('hello');myOpt.setAttribute('value', 'hello was the choice');

As for these methods, don’t bother. Firefox likes this one:

myOpt.value = 'hello was the choice, too bad IE users cannot see it';myOpt.text  = 'hello Firefox users only';

…and Internet Explorer can deal with (this is the #1 Google hit, by the way! Dammit.):

document.forms['testform'].testselect.options[i] = new Option(’new text’,'but Firefox cannot see it!’);

While I’m recording notes from hours of browser battles, here’s how to get around the limitations (features?) of innerHTML and get to just the text:

element.firstChild.nodeValue

…works in both browsers.

If you have only one child object there, of course, if you don’t, we’ll, you’ll just have to actually program. As for the other variations that are not cross-browser compatible:

element.textContent

…works in Firefox. not IE. And

element.innerText

…works in Internet Explorer. Not FF.

This works on my local WAMP development server:

$timestamp = date(DATE_RFC822);

It doesn’t when I post the same code to my dedicated virtual server at MediaTemple.  Looks like their date constants are all disabled, and I don’t have time to wait around for support. So, here’s the workaround:

// Manual translation into RFC822 format, e.g. “Mon, 15 Aug 2005 15:52:01 UTC”
$timestamp    = gmdate(”D, d F Y H:i:s”);
$timestamp .= ” UTC”;

Enjoy.

So I’m going to talk about something else I have experience in. Having just come off a nightmare of a software development phase, it’s on the brain.

There’s a couple of sayings that I’ve heard tossed around in software development circles:

You can have it on time. You can have it on budget. You can have it bug free. Pick any two.

…and…

The first half of the project takes 90% of the time. The second half takes the other 90%.

Both are allusions to the difficulty in managing software projects. Of course, the reason software is difficult to manage has a whole lot to do with psychology.

Developers wear their brains on their sleeves like trophies. They are smart. They like to be reminded that they’re smart. They like to feel smart. They like you to know that their smart. Some of them even cheekily name their blogs “Smart”. ;-)

No matter what a developer might think of a problem, the speed at which they solve said problem is an indication of their intelligence. And so, a developer will always give you his best possible estimate for how quickly he can solve a problem. And, in fact, if a developer were to write bug-free code on the first pass all the time, that estimate wouldn’t be half bad.

But even the smartest of developers make mistakes. I bet there are even bugs in the googlebot. I know, I’m risking a lot saying that (alienation of the googlebot is a death sentence for any blog, right?), but it (gulp) must be true.

Those mistakes go up when the developer is under the wrong kind of pressure. I, for instance, thrive on stress. The more things you throw at me, the more productive I become. Basically, I insist on dropping 2-3 balls at all times, so if you throw 53 at me, you’ll watch a crazy man juggle with 50 items at once. But if you throw only 3, I’ll just sit around, drink beer, and play a lot of basketball. (lol)

Other developers are the opposite. One fellow I know handles tasks in an extremely serial fashion — one thing at a time, and you’ll get the best work you’ve ever seen. Throw six things at him, and he serializes the tasks, then gets his work done almost as well. 50? Well…

Let’s just say that spontaneous human combustion does exist. I’ve seen it.

But back to the psychology of developers. They don’t figure in bug fixing time by default. If they say they can write you a script in 2 minutes, they can. It just might not work! Good developers re-think their estimates real time, but they’re still optimistic. And they always forget their biology. If you’ve worked them the last 12 hours straight, it’s the red bull talking. Take whatever they say and triple the time.

The other major reason software is so hard is because of bad project managers. Now, developers are “programmed” to say yes to everything. Ask a software engineer to build a fully functional Apollo rocket using nothing but LISP, a rubber band, an Atari 2600, and two alligator clips and you’ll get it. It might take a billion bucks, 3 years of engineering, and some minor embezzlement, but you’ll get it.

Project managers, on the other hand, have to say no to everything. It’s their job to protect developers. There is a reason engineers are often not naturally “sociable”. That’s how they got good. Solving a good problem is, in it’s own way, more rewarding than getting laid. Well, maybe not quite that rewarding, but you get the picture. So don’t bother them. They’re working.

The thing is, because engineers like to be smart heroes, and because to an engineer every problem is solvable, the folks driving the development generally ask for too much. The people footing the bill on a development project commonly engage in “scope creep”, which is when they tap a developer on the shoulder and whisper a little, tiny request into his ear.

A good Project Manager barricades the engineer and tells said shoulder-tapper to take a hike. Yes, sure you can have that extra icon, but I’m taking this one away instead. Or, frankly, no. You can’t have it. Piss off. We release in a week. Come talk to me after that about the release planned two months from now. Furthermore, that good PM makes the stakeholder appreciate her all while letting her developers get on with their work uninterrupted.

But, alas, so many PMs are simply glorified secretaries… and bad ones at that. At the last big company I worked for, every project where I was a developer under another PM either suffered from scope creep at a Cheetah’s pace or never got finished because they had me attending 4 hours of meetings every day. You can’t write code if you’re in a meeting! Oh, and by the way — if I have to attend every meeting and interact with the stakeholders every step of the way, how the heck are you adding value? I’m managing the project directly!

In my last gig, regrettably, I was stuck wearing both hats. The developer in me wanted to say yes to everything, while the project manager in me screamed “no” every 23 seconds. One of the stakeholders on the project even called me on it, saying I was contradicting myself. And, indeed, I was. But that’s the nature of the beast. This was one place where wearing multiple hats didn’t work so well.

Pay is $80-100 an hour. Low end for W-2; higher end for 1099 (since you pay your own payroll taxes, of course). The ideal candidate, in addition to being a JavaScript Jedi, has the people skills to chameleon themselves nicely into an existing development team and can provide support, both in matters technical and soft, to his or her peers. In other words, you have to know how to communicate with both Star Trek jargon and the everyday business folk.

It’s a good gig at a solid company whose name you’d recognize. I’d take it myself, but I’m booked solid for the time being and on-site in SF is a commitment I can’t make right now.

It’s a contract, so the standard stuff applies — you’re on your own when it comes to having a legal right to work in the United States, etc.

And no, I don’t have a direct financial interest on the backside (although I may ask for a kickback if it works out). This post is simply a favor on both ends.

Well, this weekend, how about taking a breather from all of that and contemplating how Google generates those search results? Those pages you get back when you punch keywords into the search engine are dynamically generated in real time. That means that every search index effectively has an entire copy of the internet in its data center.

Now, this may not have much to do with your business, but trust me, this is a pretty amazing feat of engineering. Not a Great Wall of China kinda feat or anything, but certainly a Herculean task.

If you’re a computer science fellow, take a look at Google’s BigTable white paper. If you’re not, but still vaguely comprehend what a pointer is and want to try to get your head around data relationships in n-dimensions (the visual model tends to break down for me somewhere around three, personally :-) ), enjoy.

There’s a quick solution to this problem, however. In my case, I had a section of text that was positioned about 60 pixels higher in IE7 than in the other two browsers. So I just added the following to my source file:

<!--[if gt IE 6]>
<style>
#message_left {
padding-top: 60px;
}

</style>
<![endif]–>


Thanks to quirksmode.org, where I found this helpful workaround.

(The reason is because no two web browsers render html and css identically. So your beautifully designed masterpiece that rivals the Mona Lisa in Firefox might be a huge mess in IE6, slightly skewed in Safari, and Martian in IE7. Not to mention the handful of browsers with even less market share.)

I have two Windows 2000 systems I use side-by-side on a daily basis. Both boxes are a couple of years old, but are plenty fast for my needs. I’m not a gamer, my primary development environment consists primarily of cygwin and a text editor, and my most-used application is Firefox.

I nevertheless need to evaluate my work on at least IE6, IE7, and Firefox 2. It would better to include Firefox 1.5 and Safari as well.

Two Problems

1. Internet Explorer is part of Windows. So separating it, or, as I want to do, compartmentalizing so I can run both IE6 and IE7 side-by-side is a monstrous effort all by itself. There are a number of other Smart Guys out there who have discussed it in detail, but I wasn’t too interested in that solution. Besides, I have two boxes, so running IE6 on one and IE7 on the other seemed like the best approach. Except that…
2. IE7 cannot be installed on Win2K. Call it Microsoft’s encouragement to upgrade your OS. Call it a Microsoft cost-savings effort (because it’s a lot less Quality Assurance labor). Call it whatever you like… getting around it is yet another Herculean effort. And I do this sort of thing for a living.

A Free and Easy-to-Install Solution

Enter Virtual PC from Microsoft. It’s a poor man’s Windows-only, less efficient implementation of VMWare, but it’s free and it solves the problem in 30 minutes, 20 of which was downloading the “Virtual Hard Drive” from M$ containing an installation of Windows XP SP2 and IE7. I installed the Virtual PC package (download link is in the first paragraph, above), downloaded the Virtual Hard Drive (VHD), ran the “New Virtual Machine Wizard”, and 5 minutes later I was running IE7 on XP within a window on my Windows 2000 box.

Here’s a nice screen shot for you (click through to see it at full size):

I’ve also been exploring the technical requirements for a bigger project while getting tapped on the virtual shoulder on a regular basis with web development questions. This led to the articulation of something that, as a software and support guy, comes as second nature, but seems to escape so many whom I speak to.

Your Web Site is a Software Application.

That means that the rules of software development apply. It means that where it makes sense to apply processes associated with software manufacturing, you should wholeheartedly do so. And it means that if all you’ve ever done is blog about how poor you are, you probably ought to change your mindset before going out and trying to do much with web-based businesses other than blogging.

If you’re using the web to extend your offline business, you don’t need to think about software. It’s perfectly ok to use a word processor to place a virtual business card on the web and place the URL on your business card. If you’re a writer, then use tools like blogger and wordpress. If you’re a broker of used goods, then yes, craigslist and ebay are web tools you should be using. If you need to do something more complicated that increases the value of your business through custom software, that’s easy! Hire me.

But if you’ve never touched html, if you don’t know the fundamental differences — and I’m not talking about code syntax — between Java and JavaScript, and if you haven’t the foggiest what I’m referring to when I say “data abstraction” or “presentation layer”, then you’ve got some reading and studying to do before you start thinking about building Facebook plugins and online storefronts.

All of this being said, if you want to ignore my rant and forge ahead anyway, here are some basics you need to know about software development.

1. You need to think about your business before even considering your application. Software people — usually with a title like “Product Manager” — call this a Roadmap. The application you want to develop is serving a purpose, right? Well, like any investment, the costs associated with the application need to be weighed against the reward, both potential (optimistic) and likely (realistic). Is there a better place you should be spending your time and money? More importantly, are you interested in a given feature, application, or business simply because you’ve fallen in love with the idea? Don’t do it. The three shower rule applies.
2. You need to document your functional requirements before you start writing code. If you happen to be doing the development, then maybe you can take some of shortcuts here, but that amounts to shorthand notation, not skipping this step. For instance, consider a user management application. You know, those things you use every day when you need to login to something. Great, so you need some place to store the user and password, you need a form that accepts these inputs, and you need a way of providing feedback like “Sorry, bad password, stop trying to hack into your boyfriend’s account.” Anything else? No? How about security concerns? How do you reset a password? How do you register a new user? What about when a user name is already taken? What about password requirements, like length or a rule against using dictionary words? The point is this: until you start making a list (at least… what we software guys call “use cases” would be even better), you’ll forget lots of stuff that needs to be in there. Make your list first, and you won’t find yourself rewriting code you should have constructed properly the first time. Or worse, paying your $100 an hour IT guy to do it twice. Oh, and yes, there’s an acronym for this one: “FRD”.
3. You need to capture your technical details. Just as I’ve described above with your features, making a list of your technical decisions — both in terms of architecture (e.g, the server operating system, the development tools used, how data is stored/cached/handled) and your details, like the type of data you need to maintain in your database, will save you a great deal of heartache later. On bigger projects, both your FRD (functional requirements document) and TRD (technical requirements document) go through several iterations before any development begins. (Aside from, perhaps, some user interface design for inclusion any these documents, which happens often with web applications, because a skilled developer can do web UI mock-ups faster than a designer can create mock-ups in Photoshop.)
4. You need a guide, young padawan. It doesn’t have to be me. I’m admittedly pretty expensive. That being said, I’ll happily answer simple questions from any reader and would be delighted to critique your business plan off-the-cuff. If you want a formal evaluation or some actual development work, of course, the first thing I’ll ask for is if you’ve been through the first three points above. :-)

Problem Description

PHP 5 has some built in functions for handling and parsing xml files. In typical PHP style, these are simple and straightforward to use. The simplexml_load_file() function in PHP is commonly used to load xml files and ready them for parsing using associative array (that’s a hash for your perl coders) syntax.

There’s a catch, though. With simplicity comes limitations. The error checking built in to the simplexml_load_file() function is difficult to use. In point of fact, I couldn’t get it to work.

I even found [incorrect] documentation that implied that this would do the trick:

$xml = simplexml_load_file($url);
if($xml) {
// parse it


That may work if the file doesn’t exist altogether, but it still through plenty of runtime errors with my malformed xml.

Digging further, I discovered that, based on the API, I should be able to supply a libxml constant as the third argument and thus gain a high level of control, but after toying with the constant LIBXML_DTDVALID without achieving the expected (or desired) results, I opted for my own methods.

The problem in my particular case was that my script was trying to retrieve a dynamically produced xml document that contained Java Runtime Error output instead of the expected well-formed xml. Sure, it’s the data provider’s problem to fix this sort of thing, but in the meantime, I’m the one displaying PHP errors to my customers. And, in all probability, losing them. Better to tell them the data is currently unavailable and give them a phone number to call, no?

Here’s the exact error message:

Warning: simplexml_load_file() [function.simplexml-load-file]: I/O warning : failed to load external entity “../my/server/directory/structure/myfile.xml” in /my/server/directory/strcuture/xml-parsing-script.php on line 34

And yes, of course the file is there. I told you, it contains an html-formatted page full of Java Errors. Can’t you Java coders parse some simple data into some clean, well-formed xml? ;-)

The Solution

The problem arises because simplexml_load_file is trying to both retrieve and parse all at once. The solution is to read the file using file_get_contents() first, test for validity, then parse it as xml using simplexml_load_string. If you’re still reading, you’re probably done with the explanation and want to see the code:

The function below is based on some code snippet I found out in the wild somewhere. I’d credit the author, but I can’t seem to find it again. Feel free to comment if you are the proud originator and I’ll update this post accordingly.

function isWellFormed($xmlString)
{
libxml_use_internal_errors(true);

return $message;
}


Once you have functionized the validation step, just load, test, and parse:


$xmlString = file_get_contents($url);
if (isWellFormed($xmlString))
{
$xml = simplexml_load_string($xmlString);
}