ArticlesWhen I have been stupid enough to make a mistake, and then smart enough to realise it, I try to be even smarter. I try to learn from the mistake so I don’t make it again. Better still, I don’t want to make that class of mistakes again.
I made a mistake at work late last week, and I am still scratching my head trying to decipher what the lesson might be. I can’t see one. I am posting it here either so others can provide me guidance, or at the very least so it can stand as a warning for others.
The War Story
A customer visited our web-site, and received some status about his account. He was confused by it, and decided to email our help desk. He included a screenshot of the page, so that his complaint was clear. Rather than take a screenshot, he elected to pasted the rendered HTML page into a Word document. (I have no complaints about the customer. He needed help, and he requested it in an intelligent way.)
The first tier of the Help Desk looked at the page, and were confused by it. They could not figure out the error. They escalated the problem, including a copy of the Word document. (I have no complaints about the Help Desk. They couldn’t figure out the problem, and rather than make the problem worse, they escalated it in an intelligent way.)
And so it went, escalating through a number of tiers until it hit my desk. (Again, all intelligently following the support process.)
The next part gets technical. I was unaware that the web-site uses a common trick. The HTML sent to the browser includes the text of an error message. However, through Javascript magic, it is hidden. The text included in the HTML but is not displayed on the screen. If the right circumstances occur, the Javascript makes the text appear, in bright red, without having to re-load the page. (I have no complaints about the way the web-developers wrote this. It is an intelligent way to solve a real issue.)
I opened up the Word document, which contained a copy of the web-site page. Inside this page was the error message, which was not relevant to this particular customer’s situation: it was marked as hidden, and was not visible…. except on my machine, where, for reasons I can’t adequately explain, I had Word configured to display hidden text by default.
So picture it from my perspective. I have an request to explain what a message means, and I open up the attached Word document, and there is a bright red error message. “Aha!” I think, and I proceed to confidently explain to everyone on the support chain what the situation is, and what my recommendations are to resolve the problem. I was more than happy that I could resolve this customer’s concern – all based on a completely incorrect view of what the customer was seeing.
Hilarity ensued.
I have since completed my red-faced apologies. I have raised a request to the web team to make a small change to the web-site that I think will prevent other users from suffering from the initial confusion. However, I don’t feel that I have learnt my lesson. I can foresee the same scenario happening again.
What can I do to stop me (and you!) from making a similar mistake again?
Comment by Alan Green on March 2, 2006
Your problem could have been avoided if you were able to reconstruct what was on the user’s browser. You could probably make a sketchy guess, based on the server logs, but better would be a tool that sits on the public webserver, monitoring all traffic in and out. Such a tool could/would/should have a little GUI that allows you to select a user and a time, and then shows you, on your browser, what they would have seen on their browser.
So, to stop this from happening again, all you have to do is implement some interesting spyware and deploy it on the company webserver.
Comment by Julian on March 2, 2006
I heard another suggestion last night. Wherever the web-site has some hidden text, it should also include another piece of hidden text. That text is never officially displayed. It says “If you can read this hidden text, be careful! The nearby text might also be hidden.”
Both this suggestion and Alan’s suggestion assume that it is somehow the web-developers’ responsibility – they caused the confusion, and they should expend effort to fix it. I am not sure I agree. It doesn’t seem to me that this is a use-case (scenario) that the developers should be expected to handle.
Alan, your idea would require copious disk-space and probably slow the server to a crawl, but it would be such a cool debugging tool! So many strange claims from the users could be quickly verified.
Comment by Alan Green on March 2, 2006
A quick google along these lines finds Carnivore, and, err.. Carnivore.
As to your objections:
1. “copious disk-space”. It’s probably only necessary to keep the last week’s worth of traffic, and a simple algorithm would deal with static text and images very well.
2. “slow the server to a crawl”. It doesn’t nescessarily have to run on the server. It could run off a proxy or – even better – just reconfigure the network switch to snag traffic off the network via a specially configured switch. (This of course assumes that your company’s web servers are attached to some kind of managed switch :))
Comment by Alastair on March 2, 2006
Julian, when I read your war story I thought about a similar solution to the one you posted in the above comment. Instead of burying the error message and having it revealed by Javascript, howabout having the Javascript actually _set_ the error message as well as reveal it. This is a more flexible approach in the event that there are lots of possible error conditions.
And without knowing too much about the error itself, would it not have been possible to express it more clearly so that the user would not have needed to escalate? Perhaps it could have provided a link (being a webapp and all) to a support site knowledgebase article or similar?
If this sounds too hard, there’s always the google as knowledgebase approach.
Comment by Sunny Kalsi on March 2, 2006
Perhaps you didn’t read the email properly, or didn’t get a copy of it when you were asked to solve the problem? In general, when I’m putting in requests from support, I clearly state my problem in the email. I sometimes even put in caps that the problem is not something obviously wrong in the attached material.
I still usually get an email back saying “ah this is a known problem blah blah blah”. I have to send back another email saying “no, no, I knew about that, can you solve my real problem”.
Comment by Aristotle Pagaltzis on March 3, 2006
Having Word configured to always display hidden text is very sensible. The media have reported many cases of unintentional disclosure reported where a company or even the US gov’t published a document that contained more data than met the eye. It’s also very interesting the things you will sometimes see if you surf the web with a filter that makes HTML comments and meta tags visible. Etc.
I think the lessons here are:
• You can never be sure that what you see on your screen is what the sender or the recipient saw or will see on theirs, when you’re looking at a Word document.
• Therefore, ask for a real screenshot next time someone sends you a Word “screenshot†as part of a problem report.
• Relying on Word for anything is stupid.
• It would be nice if you could allow
ul/olandlitags so people could type actual lists into your comments. ☺Comment by Julian on March 5, 2006
Alastair,
While I haven’t asked the web-developers this, my assumption would be that by making the text present, but invisible, would still take up space. The page layout wouldn’t change (and hence have elements jumping around the screen) when the error is displayed. I certainly agree that this doesn’t scale for many error messages.
Also, I am loathe to blame and/or make more work for the web-developers here. I am not yet prepared to describe this as a bug in their code, and I know they are busy and have more important things to change on the web-site, because I have asked for several more important (unrelated) changes!
Yes, a usability improvement could have reduced the likelihood of the escalation in the first place (and I have made such a request to the web-team.)
Sunny,
I appreciate that you try hard to make your support requests clear. Thank you. Many users don’t, especially when they are not using their first language.
This request was along the lines of “I try to do X, and I get this error message.” Unfortunately, the message they were referring to wasn’t (in my opinion) an error message, but merely indicating a status. I can see that it could be seen an error message.
Let me give an unrelated example. If you list a directory and the system says “No files found.” is that an error? Normally no, but if someone was expecting files there, they might described it as an error message.
So the user asked “What does this error message mean?” and I saw an error message under the status message… and I naturally responded to the wrong one.
Aristotle,
Very true, but too limited in scope. I would expand to “looking at a Word document, HTML document, Acrobat document, working with photocopiers or colour-blind people, working with non-Metric page sizes, different screen-resolutions (including colour-depths), or (from bitter experience) working with Europeans that use pre-punched holes in their stationery.
I agree that that would help, but given the length of the communication cycle (Think time zones!), it would probably add a day to the issue’s resolution time (in the cases where there is no problem with hidden text being displayed.) The risks involved are too small to bother.
In fact, that echoes my current position on the matter. It was an unfortunate incident, but the cost of protecting against it is greater than the cost of occassionally falling for the same mistake.
I dunno. I find it the best tool for collaborating on document production with the subset of people who use Word to produce documents.
I also notice that this subset is by far the majority of people I have worked with for over a decade.
OK. I’ll look into it during the next round of blog style changes. The coincidence that your smiley face doesn’t display correctly in IE6, while talking about the impact of documents that appear differently to different people, is delicious!