Friday night, I was responsible for a maintenance upgrade to our document imaging system, ImageRight. This upgrade was required to repair a potentially serious data corruption issue that was discovered by the vendor. We weren’t affected by the corruption at the time it was discovered but some functionality had been disabled as a work-around, so we had to schedule time to perform the fix.
First off, let me say that I really like the vendor and I like how the product works overall. However, we always seem to be the client who has issues that the vendor never seems to encounter before. It was almost refreshing when they called about the corruption issue and it wasn’t something we’d found first.
Friday morning, I had exchanged a few emails with the vendor support tech who was going to do the upgrade to firm up the planned roll-back procedures (for our change management documents) and to clarify any last minute items. He mentioned a known bug related to environments with ImageRight users that spanned multiple Active Directory domains, fondly referred to as the “AD dual domain bug” and how the upgrade shouldn’t be performed if we had an environment with those characteristics.
Yes, we have two domains. But no, we don’t have accounts that are used by ImageRight from the second domain. We confirmed those details and I mentioned in one of my reply emails that the AD bug had me a bit worried anyway. I was told my environment wasn’t going to be an issue based on their testing. (Okie–dookie then.)
So away we went with the upgrade. That was the easy part. Then came the testing.
Exactly half the program worked – literally half. The program launches two windows when it starts – one window acts as the file manager, for searching and loading image files and the other is the image viewer. I could see and use the file manager portion, but the viewer never loaded, only returning an error that was cryptic overall but referenced active directory about 15 times.
So they uninstalled and reinstalled just to make sure some random DLL wasn’t left behind or something. But that didn’t solve the problem. I chilled out on hold for a while. It was already after 7pm here on the west coast, so I felt a little bad for the tech on the east coast. “Are we sure this isn’t a different manifestation of the AD bug?”, I asked.
I chilled out on hold for a while longer while the support tech consults some developers on his cell phone. No, it shouldn’t be the AD bug, we are getting an error “too soon” in the loading of the program, but there is a hotfix for that bug that should be released on Monday. A developer was working on getting a copy for us now if we wanted to try that.
Why not? It’s already broken, might as well toss one more thing at it before we roll everything back. Sure enough the hotfix did the trick, avoiding a roll-back and saving me another late night at the office. I wasn’t surprised that it was yet another instance of something no other client they have has ever experienced.
I’m not sure how I feel about always being the “one-off” case, but it always seems to work out fine in the end. Though I’m thinking of framing my “I’m a bit worried about the AD bug” email that I sent out before we even began.