In part one of this article, I discussed some of the common pitfalls for sysadmins who are suddenly pulled into projects way too late in the game. Recognizing these warning signs can help you anticipate whether a project is in trouble. Here in part two, I'll continue with the remaining six signs and have provided you with a checklist that you can use to make your life as a sysadmin better.
Integration, a lonely story
Many POCs focus—rightfully so—on the new product that brings many good things. Limited tests are made just to see how the new product integrates with some of the legacy software products. Very few products live their lives in isolation, so expect many integration and dependency issues, and not just at a first-tier layer but also to a second and possibly even a third-tier layer. These deep and entangled complexities are seldom tested during the POC. Expect the integration tests to focus on those that are obvious and most likely already a part of the new software product. In production, the complex stack of legacy software has to be integrated, ways of communication have to be developed, or intricate workarounds have to be invented.
Sysadmins have control of all APIs within the technology landscape. Together with system owners and developers, the impact and possibilities can be reviewed and explored.
Power and speed
With cloud services, it is easy to get performance across the globe, and the services can grab more resources when needed. It sounds like all our problems of slow systems and long response times are gone forever. As the sysadmin, you know that new solutions need to interact with not just one but probably several back-end systems that could be located in one or more data centers. This means that even if the new solution is fast as lightning, the actual speed is determined by how fast the communication both with and in-between the back end systems is. If this is not properly tested in the POC, it is sure to show its ugly head in production.
Sysadmins, together with developers, can provide real-world experience to anticipate expected performance and also indicate what changes could be necessary to accommodate the expectations of the new software product.
NIH - Not Invented Here
This section addresses the internal corporate politics of trying to bring a new product into the company. The fact that one department came up with the idea and not another can be enough to spark animosity and counterproductive activities. Same thing with the IT department being brought in too late in the process, which can lead to a focus on finding faults instead of solutions.
Sysadmins need to have the balance and vision to see beyond these things and stick to facts. In the end, this is the most valuable contribution that brings the project forward.
Help desk doesn't know anything
When the solution is in production and the provider has just signed off, the daily routines are quietly dropped on the sysadmin desk. About this time somebody realizes that it would be good to inform the help desk that there is a new solution in production and they need to take care of it. So just send some documents and the help desk will take care of the rest. This is, of course, not reasonable. The help desk manager (outsourced or in-house) must ensure sufficient documentation and system readiness (CMDB, directories, workflow, support contracts, etc.) and make plans for training the help desk crew.
Localized support in different languages and time zones could also be on the agenda. So as a sysadmin, you are most likely tasked to set up or re-structure the accounts and groups that need access to the new solution. If not already done, you most likely need to build a first response challenge for the help desk crew to follow to help them pinpoint potential technical issues.
The help desk can help if given a fair chance, and that means involving them as part of the POC to ensure they are ready if and when the new software product moves into production.
The solution is so intuitive
Whenever training is not considered necessary, you know that the support organization (and you as a sysadmin) takes a substantial hit. Furthermore, there is no plan for how to implement the solution in the organization or how to make the organization adopt and effectively use the new solution. The help desk will benefit from training a group of regional or local super-users who, in turn, can provide some initial support to the local users. This is also a great way to build localized communities around the new solution. The challenge with not training users is also that their expectations range from everything to nothing, thus placing even more pressure on sysadmins and developers to deliver.
Being a sysadmin, you should ensure the boundaries of the new software product are defined, or you will have to support mobile connectivity in one region and fax solutions in another.
New solutions enter, but older applications aren't decommissioned. This means that whatever older product that provides some or most of the functionality the new solution brings remains within the organization and continues to be used.
So instead of one challenge with a new solution, you now have multiple. The user community is inevitably divided between the legacy solution and the new. This means the new solution has also contributed to complexity and confusion - not to mention cost. Support, license, backup, antivirus, etc. must work double-time to support the two solutions that provide the same or similar solution.
From a sysadmin point of view, it is important to raise questions early in the process regarding which solutions will be decommissioned as a result of using the new software solution. This brings better balance and helps keep the technical landscape up to date.
New software products are great, and they have the potential to bring the whole company forward and provide many competitive advantages. Implemented the wrong way, they can cause confusion and delay. The solution could bring unpleasant financial surprises and send cost spiraling. It might also increase complexity and add unnecessary workload.
One of my key points is that as a sysadmin, you must be involved early on in a software implementation project to help your organization avoid many of the caveats outlined in this document. There are potentially more issues, and some specific to your company, but through this document, you can at least track some of the most important and potentially damaging events.
I will leave you with a warning checklist for IT projects. I have added my own subjective score to each line item so you can add up and see where your current projects end up. Suggested actions follow the checklist.
IT project warning checklist:
❏ IT and/or sysadmin not involved from the start [2 points]
❏ Project active for more than 6 months without sysadmin/SME involved [2 points]
❏ Substantial modifications to original product [2 points]
❏ Modifications not documented [1 point]
❏ Localization (time zone and language) not part of POC [1 point]
❏ Risk & Opportunity analysis not made [1 point]
❏ Financial analysis of full scale production missing [1 point]
❏ Test does not reflect production [2 points]
❏ POC have morphed into production [10 points]
❏ No handover to production [2 points]
❏ Life cycle management planning missing [1 point]
❏ Backup not considered [1 point]
❏ Antivirus not considered [2 points]
❏ Security not considered, security department not involved [5 points]
❏ Integrations not thoroughly tested or even possible [2 points]
❏ Not-Invented-Here politics [1 point]
❏ Help desk not informed [2 points]
❏ User training not necessary [1 point]
❏ Nothing out [5 points]
My advice to the IT project based on the accumulated score:
0 - 5 Action the line items that generated points and the project should be fine
5 - 10 This is going to hurt, but the project is salvageable through significant efforts
>10 Pull the break immediately or cost will spiral until the inevitable crash
[ Want to test your sysadmin skills? Take a skills assessment today. ]