OK, I'll tell you again. There's The Truth, and then there's “the truth.”
If you heard Lionel Hutz’s voice when you read that, then give yourself a high-five. As a sysadmin (let alone a human), we need to be truthful. This doesn’t mean that we are so transparent that we walk around broadcasting every task we’re about to perform like some NetBEUI skin tube. A lesson that I learned early in my career as a sysadmin is that end-users have little to no concept of the difference between correlation and causation. There is a well known logical fallacy called post hoc ergo propter hoc. Loosely translated, it states, "It happened after therefore because." Auto mechanics understand this all too well as they constantly hear complaints like, “You replaced my brakes last week, and now my trunk won't open; therefore, YOU BROKE MY TRUNK!”
I’ve done my own little sociological tests in this arena. Years ago, I decided to tell about 100 end users that I was going to update a telnet client on their computers. My email stated that I would deploy the new update, "this Saturday." Sure enough, the next Monday I started getting emails and help desk tickets stating the following (this is just a sample from memory):
My MS Word templates are missing!
I can't print to the color HP LaserJet!
My computer was powered on even though I left it powered off on Friday!
I can’t access my U: drive!
Had there been a family member kidnapped among the 100 users, I'm sure I would have been brought in for questioning and some light water-boarding. Anyway, what I didn't tell the end-users in my original email was that I had already applied the update several weeks before to well over 3,000 computers (including theirs). Yet, weeks after the covert update, only people in the small sample who had been given a fake one-week notice seemed to have problems that were immediately diagnosed as having been caused by yours truly.
Right after this happened, I went out for drinks with several sysadmin buddies, and I told them what I had done. They all got on board and started doing similar things at their companies. Over the years, we have followed up with each other, and I have always loved hearing the stories from their own little experiments. I assure you, my experience wasn’t an outlier.
The lesson (which should be obvious) is this: Don't broadcast every time you intend to take a leak in the yard. If you are new to an organization you can also use similar methods to suss out users who will blame you for everything and those who will actually try to troubleshoot a little. It’s a great way to learn who your co-workers are, who to avoid, and (if you’re Lex) who to mess with.
It's important, however, to still follow logical change management processes and procedures. Of those 3,000 computers, there wasn't one production server in the mix. I'm not Lex. I do give special dispensation to production servers or mission-critical systems. Obviously, if your organization has specific P&P’s that prohibit this type of covert “just proving a point” administration then don’t do it. Or, at least, don’t mention my name.
My next article will discuss those moments where we, the sysadmin community, have tendencies to over-correct and not take the blame even when it is warranted.