New Hires – My System Admin Wish List

Last week I wrote about employee separations and the list of things that often need to be considered when an employee leaves. To balance that, here is the list of information I’d like to have handy when a new employee needs access to corporate resources.

  • Proper spelling of the person’s name (and if they have a preferred nickname) – Your company might insist that user accounts and email addresses be based on legal names, but if “Robert” always goes by “Bob” he may prefer “bjones” over “rjones” for his username. If your email global address book is sorted by first names, other staffers might look for Bob first under B, instead of R.
  • Start Date – I want to make sure everything is ready on the proper day. But, it’s also important to let me know if the start date is change or delayed. Most accounts are created with a generic, easy password and I would prefer to not have an active account hanging out there for an extra 2 weeks before the new hire can select a more secure password.

After the name and the start date, everything else tends to drift quickly from the ideal “standard” setup and slips to every employee being just a little bit different. By default, I give every new employee a personal home directory on the file server, access to their departments file share and membership on their department distribution list and any generic office lists, like “All Staff” or “San Francisco Office”.

  • Specialty Distribution Lists – Which other DLs do they need to be on? Contractors and employees might use different DLs. Managers, supervisors, special project lists, etc.
  • Phone Number – Will any available phone number do? Sometimes a hire is destined to replace someone who needs immediate coverage. If that’s not the case, I like to give out a fresh DID or at least one that hasn’t been used in a while. No one likes to spend their first weeks on the job fielding calls that don’t pertain to them. If I’m reusing numbers, I like to keep recycled numbers in the same department if possible. This way the new person in accounting won’t be getting calls directed to the person who retired from HR. And what about calling privileges – local only, long distance, international calling? How about membership in special hunt groups, dial-by-name directory? Do they need call appearances to pick up calls for manager or executive?
  • Applications and Security Groups – Which applications will they be using their first week or so? I know roles evolve and users always need their access adjusted. New hires usually will be learning 2-4 new applications immediately, so concentrate on finding out what those are. I don’t like to “make Bob the same as Joe”, because I know that Joe probably has membership in some security groups that Bob will never need. If the hiring manager can’t give you a list of which applications and data the new hire needs, remind them that security groups and application access are areas that are often looked at closely by auditors.
  • Hardware – What’s standard for others in that department or role? If you have options for different mice or keyboards, let the newcomer know so they can request changes sooner than later. Make sure they are connected to the closest printer to their workspace, etc.
  • Helpdesk Communications – Make sure they know the appropriate ways to submit help desk tickets or report problems. Should they use a ticketing-system? Send email? Call a special number? Pop into you cube? It’s a safe bet that people new to the office don’t want to annoy the IT folks, so set them up for success.
  • Training Documentation – Many departments have manuals or documentation about how various tasks are performed, IT is no different. Voice Mail instructions, conference bridge information, document management system procedures, “how-to” information for common FAQs related to Outlook or other applications… make sure the new hire knows how and where to find those things. It’s much easier help someone do something right the first time than to bother your DBA with bulk corrections to database information that was improperly entered.

Finally, document, document, document! File any forms or emails related to access needs and who authorized the access. Note the date you added or changed access going forward. Not only will this help with any audit needs, proper documentation can make it easier to remove access completely when someone leaves the company in the future.

Managing Employee “Separations”

It happens at every workplace. Employees leave – layoffs, retiring, or just moving on to new things. As a systems administrator, I wish that managers understood how deeply integrated a staff-person is with the computer systems they work on daily. It’s not always a simple process to undo someone’s existence.

Processing exiting employees without identity lifecycle tools can be tedious, but it’s often the way things are done in small and medium sized businesses. I realize that several days notice isn’t always possible, but I can hope. I’ll even take a few hours of notice. However, we’ve all gotten that call at ten minutes to 5:00pm letting use know that someone won’t be coming in the next day.

I have my list of basic things I’d like a department manager to think about when it comes to seeing an employee off. The first couple can get me out out the door on time, the rest of them tie things up in a nice package.

Before beginning, it’s important to make sure the employee REALLY is leaving. It’s not unheard of to get several days of notice about a separation, complete account closure process, and then find out that the employee will be contracting from time to time and needs access when they are on-site.

  • What time should their network account be disabled? – Ideally this is before someone in the NetOps department leaves for the day. Worst case is having to set an account expiration, as midnight often comes a long time after the employee has walked out with their final paycheck.
  • Do they have remote access? – If yes, I disable that ASAP. This way if the network account has to stay active for longer than I’d like at least they have to be physically in the office log on.
  • Email Forwarding – Is it needed? If so, I like to turn that on as soon as possible so that any incoming emails (especially over a weekend) are not missed.
  • Phone and Voicemail – Is any call forwarding needed? For the same reasons as email, I don’t want any voicemail messages missed or left unchecked for too long.
  • Building Access – Has the access to office space been removed? Network Operations isn’t always responsible for physical access and that needs to coordinated as well.

Now those are just my “get-things-under-control” checklist. Then comes the rest of the things that need to be considered, but most managers really don’t know to mention them ahead of time.

  • Email History – Do someone need a copy of their email box? Does the user have any PST files that need to be located and preserved?
  • Distribution Lists – Is the user the sole member of any distribution lists? If so, removing them and leaving the DL empty will cause messages to go undelivered and lost. A new contact person needs to be designated.
  • Work Files – Does the user have a home folder or area where they store work products? Do these files need to be preserved?
  • Phone System – Is the user a destination for any phone tree options, a member of a workgroup or hunt groups?
  • Application Management – Is the user the sole owner/manager of other important enterprise products like databases or SharePoint sites? Those roles will need to be assigned to someone else. Is there any applications that regularly delegate specific tasks that would need to be reassigned to a co-worker?
  • External Systems – Does the user have any accounts with third-party systems (not AD or Windows-integrated) or external systems with other partners or clients where access would need to be removed separately?
  • Locally Installed Applications or Hardware – Do they have some special applications or hardware installed on their workstations that need to be set up for another staff member?

Finally, there is usually a change control process that documents what was done to close the network account of the user so items weren’t overlooked. In a perfect world, the manager in question would have filled out the necessary forms ahead of time, but I’ll settle for some quick answers over email that I can file in our document management system.

Every company will have it’s only list of tasks, but the premise is the same. Securing critical data and making sure that customers continue to be served after the departure of an employee are important aspects of any systems administrator’s job.

Who’s Geeky? She is.

Happened across the She’s Geeky conference while surfing around the web. “She’s Geeky” is an event specifically for women interested in and/or working in the technology, math and science industries. Actually, it’s an “un”conference – 3 days of geek-minded women gathered together with a daily agenda of tracks and sessions generated fresh every morning.

I’m always up for an interesting tech conference, plus it’s hard to pass up an event being held at the Computer History Museum in Mountain View, CA. Seems like a great chance to check out the Babbage Engine, too!

Always Enjoy Lunch

I once received some sage advice from another System Administrator I worked with years ago regarding working with potentially troublesome servers.

It was back in the Exchange 5.5 days. I had a cranky server with a potentially unsolved hardware problem in the disk subsystem. Every time I powered off the server, it damaged the OS and I was forced to restore Exchange from tape. The manufacturer always replaced a part when I called for support, but I had ended up rebuilding it several times and had not yet confirmed that the latest hardware replacement resolved the issue.

My co-worker was on-site to help me set up a new server room after we relocated the office. Because of the history of the server, I was very anxious about possibly having to restore Exchange again. It was approaching lunch time and we were at the point where it was time to power on the mail server.

He turns to me and says, “We are going to press the power button and then walk out to eat without looking back.” His theory was that if the server was going to be fine, it would be fine without us watching it boot. If was going to have a problem, the problem would still be there when we returned. At the very least we would have had a relaxing lunch break and would be better able to solve a problem without the additional stress of hunger pains.

Turns out the server was fine.

To this day, I still heed that advice. If I’m about to do something to a system that has the potential to backfire, I make sure I’ve already enjoyed my lunch.

Internet Monitoring – Good, Bad or just Ugly?

A good friend of mine works at an academic institution where she teaches literature. Her specialization revolves around romance literature. Research in that area often spans into topics that are considered to be NSFW and she’s often thwarted by internet filtering when doing research in her office. She objects to this and we shared an exchange about possible reasons for these type of restrictions. As a systems administrator, I can argue bits on both sides.

For me, intentions mean everything.
First off, monitoring and filtering meet different needs. Most appliances and applications available today can do both functions and are adjustable to allow various exceptions. I define monitoring as simply logging sites visited, the length of time spent and the amount of bandwidth used. Filtering is when a site is restricted outright or portions of the site are prevented from loading.

I agree that in an academic institution, internet filtering should be kept to a minimum on the staff network. Education institutions thrive on the fact that professional staff produce new works and having unlimited access to the internet and even access to potentially taboo or questionable material could easily be justified. Being that most university professors have private offices, the risk of offending someone who walks by is minimal.

However, general monitoring is often needed to track bandwidth usage and some light filtering may be reasonable to reduce the impact of sites infiltrated with with malware. In a location where the general public or children use the Internet, clearly more strict monitoring and filtering is necessary to block age inappropriate content and prevent abuses. In either case, there needs to be a system that allows for users to request review of websites that are blocked, as most out-of-the-box filtering systems can categorize some sites strangely.

In the classic business world, internet access gets even more slippery. I stand behind my opinion that light filtering to reduce malware and basic monitoring (for bandwidth tracking) is an important part of keeping control of IT costs. Also, I understand that it’s helpful to block obvious non-work related or NSFW sites. Unless your business has a specific need to access gambling, online games or other clearly “entertainment” sites, I don’t fault management for asking IT to limit access.

Home banking, personal email, news and some social networking sites can be a gray area. I feel that employees work more effectively if they can access some personal conveniences from the office. I can quickly handle an urgent bill or respond to a family member online and then get back on my work task, instead of having to take out of office break time to visit the bank or run another errand that could be completed online faster. Also, many corporations now have identities on social networking sites that need to be maintained.

The big disconnects start to occur when managers start looking at internet usage as a way to determine employee productivity. Using amount of time an employee is online as a sole reason for a write-up, reprimand or worse is inappropriate. If an employee is not completing their required tasks, blaming internet usage shouldn’t be necessary. There should be clear areas of suffering in that employee’s work product that can be documented.

If an employee IS completing work tasks and still has time to surf the web, either a manager should look to assign additional tasks or examine ways to utilize that employee’s efficiency methods. Controlling some of what flows from the public networks to a private network is a necessary component of good IT practices. However, when those same controls start hampering employee’s ability to work or are used as poor indicator of productivity no one is gaining anything from the information available online.

System Adminstration – The "YouTube" Way

Don’t miss the Windows 7 72-Hour Film Fest on YouTube. All the videos had to include a character called “CIO Wiggins”, the line of dialog “The guys in IT are going to like this” and mention “Windows 7”. My favorite is Installation, a fun mix of Office Space meets “Flight of the Concords” with a little throw back to the 80’s hit, “Say Anything”, touting the joys of system administration and Windows 7.

Speaking of fun tech videos, there have been some great ones over the years. If you are looking to kill a little bit of time, I’ve got some “classics” for you. First off, no one can forget Internet Tech Support, harking way back to 2001 from deadtroll.com. And then there is the ever popular Medieval Helpdesk. I’m hoping Windows 7 isn’t as difficult of a transition!

Also, don’t forget this great, gamer-themed performance by Tripod – “Gonna Make You Happy“. It’s about 3 years old, but never ceases to entertain me. Txt Msgs is also good one.

Enjoy!

Paper vs. Electronic – The Data Double Standard

One of the main enterprise applications I’m partly responsible for administering at work is our document imaging system. Two years have passed since implementation and we still have some areas of the office dragging their feet about scanning their paper. On a daily basis, I still struggle with the one big elephant in the room – the double standard that exists between electronic data and data that is on paper.

The former is the information on our Exchange server, SQL servers, financial systems, file shares and the like. The the latter is the boxes and drawers of printed pages – some which originally started out on one of those servers (or a server that existed in the past) and some which did not. In the event of a serious disaster it would be impossible to recreate those paper files. Even if the majority of the documents could be located and reprinted any single group of employees would be unable to remember everything that existed in a single file, never mind hundreds of boxes or file cabinets. In the case of our office, many of those boxes contain data that dates back decades, containing handwritten forms and letters.

Like any good company, we have a high level plan that dictates what information systems are critical and the amount of data loss that will be tolerated in the event of an incident. This document makes it clear that our senior management understands the importance of what the servers in the data center contain. Ultimately, this drives our IT department’s regular data backup policies and procedures.

However, IT is the only department required by this plan to ensure the recovery of the data we are custodians of. What extent of data loss is acceptable for the paper data owned by every other department after a fire or earthquake? A year of documents lost? 5 years? 10 years? No one has been held accountable for answering that question, yet most of those same departments won’t accept more than a day’s loss of email.

Granted, a lot of our paper documents are stored off site and only returned to the office when needed, but there are plenty of exceptions. Some staffers don’t trust off site storage and keep their “most important” papers close by. Others in the office will tell you that the five boxes next to their cube aren’t important enough to scan, yet are referenced so often they can’t possibly be returned to storage.

And there lies the battle we wage daily as the custodians of the imaging system, simply getting everyone to understand the value of scanning documents into the system so they are included in our regular backups. Not only are they easier to organize, easier to access, more secure and subject to better auditing trails, there is a significant improvement in the chance of the survival when that frayed desk lamp cord goes unnoticed.

Dusting off the Disaster Recovery Plan

This week, I started testing our department’s disaster recovery plan. The goal is to use the contents of our existing “disaster recovery box” that we keep off-site combined with our current backup tapes to restore some key parts of our infrastructure.

Success or failure will be measured by what road bumps we encounter and most importantly, our ability to work around them using only the resources in the box. If I have to go “outside the box” for some critical piece of software or some undocumented configuration detail it would be a black mark in our preparations that needs to be remedied.

Our testing scenario includes the domain, Exchange, the document imaging system, the financial system, the primary file server and the time card application. We are also going to provide remote access to restored applications so staff from other departments can test out the results and give us feedback on changes that could improve the end-user experience during this type of event. As an added bonus, we’ll be able to try out Server 2008 R2 Remote Desktop Services.

In the last 6 months we started using VMWare ESX to consolidate some of our servers in production, but none of the machines needed for this scenario are virtual yet. I will be doing “classic” restores where the OS has to be installed before restoring our data from backup tapes. However, we are using VMWare to host several of the machines in the disaster lab, so I will be able to save time by cloning my first installation of Windows Server a few extra times before installing specific applications.

Depending on how this project goes, I’d like to see us take more advantage of virtualization within our disaster recovery planning and maybe start looking into backup solutions that are easier and faster than tape.

When Things Work.

This morning, I’ve been at the office. I needed to make a key change with our imaging system that affects the user’s logons, so it’s one of those things you can’t do during the business day.

And due to the additional security features we have turned on for the system, sometimes regular changes to the system actually break things. I don’t really like broken things, thus have given myself the entire weekend to fix anything that could have potentially gone wrong.

But it worked. Just like the documentation was supposed to. I appreciate that the tech just sent me their internal documentation, instead of making me rely on them to hand me information only when things start going wrong. Plus I didn’t have to make one of them actively work on the weekend and I end up understanding the system better overall because I was doing the work myself.

I did have a tech available via email – but that was more for moral support. He would have only jumped on if things went badly and we had to roll back the changes. But I hate rolling things back – I really like to just fix the problem and keep moving forward.

A Thought on Network Administration

This is nearly a perfect quote to describe Network Administration. I found it on Evan Erwin’s blog, at www.misterorange.com where he wrote a post thanking Google for it’s existence.

“While most of my job does not require Google, a good majority of it does. The better you are at analyzing what keywords are important to your search, the better geek you can be. Sometimes Network Administrator really means what it says. Other times it just means I Can Google Better Than You.

I used to compete with another colleague on our abilities to find solutions to our problems fastest. He’d call me and say “I’d bet you can’t figure out what’s wrong with my server…”