From Traditional IT Ops to Automated AIOps

CIO Talk – Hexaware Blogcast: Welcome listeners! This is Sanjog Aul, your host and the topic for conversation is:

From Traditional IT Ops to Automated AIOps

Organizations are adopting modern application delivery models increasingly to support accelerated time–to-market for their products and service updates to their customers. However, it has tipped over the traditional IT operations execution methods.

To keep pace with continuous code deployment requirements, we must provision IT infrastructure almost instantaneously. And that too with minimal or no downtime. Trying to do the same with a typical enterprise cloud environment can get extremely complicated.

As a result, many enterprise IT operations leaders are looking at investing in automation along with AI and ML to build these capabilities now. So, how can they move from traditional IT Ops to automated AIOps in the quickest way possible?

To discuss this, I have with me Siddharth Dhar. Sid is the Executive Vice President and Global Head for Infrastructure Management Services at Hexaware, a consulting firm focused on transforming IT solutions and solving complex business problems using a combination of human creativity and intellect. Their three-pronged strategy of Automate Everything^®, Cloudify Everything^® and Transform Customer Experiences^® enables enterprises fast-track into the digital era.

Hello Sid…Thank you for joining us.

Siddharth Dhar: Thank you for having me Sanjog. Looking forward to our discussion today.

If you wish to directly listen to the Podcast: From Automation to Autonomous Testing, Click here

Sanjog: Great! So, my first question for you Sid is, why should a company invest in automation and AIOps to optimize their IT infrastructure and operations management efforts? In short, what business problem will this solve?

Siddharth Dhar: That’s a great question to begin with, Sanjog! Too often in IT infrastructure, just given the nature of our job and operations, we tend to not focus on the bigger picture of what business benefits we are driving.

Actually, AIOps and automation, really are geared towards business objectives more than anything else. To my mind, there are really four business problems that we hope to solve by deploying something like this.

The first one is time-to-market. Delivering infrastructure these days has to be done almost instantaneously. Over the last five years or so, most enterprise organizations have chosen to change their software lifecycle to Agile from Waterfall and that modern application delivery really needs speed, in terms of delivering infrastructure and that speed can only be guaranteed by extreme automation and AIOps. So that is a really good reason to do it.

Second, I would say a great reason to do it is, as organizations adopt more modern application delivery, they are looking to deploy code almost as soon as it is committed so that, from the point the code is committed to the point the code is deployed is one automated cycle, which we call ci/cd. That, by definition, means that the underlying infrastructure platforms must be available all the time, in order to accept that code. If you are not, then you are actually reducing or impacting your release cycles. So again, that is a business problem that we need to solve and again, it is a complicated problem to solve, especially if organizations, like most do, have a hybrid – private and public cloud footprint. So that is another problem that automation and AIOps help us resolve.

The third issue, again, very related to the first two, is as organizations and enterprises are making plans or already having footprints in private as well as public cloud infrastructures, our security organizations are extremely worried. They are worried whether we will be able to maintain our security stance, whether we will be able to identify drifts from our security stance and be able to correct them in production, and that is what automation does. We have systems that can deploy security guard rails as soon as any workload is deployed in your public or private footprint. We have the monitoring capability to identify drifts from your security compliance status and actually fix those in an automated fashion. So, again, a great reason to do that because without really making your security team comfortable, I can tell you, very few CIOs will have the ability to adopt cloud at the base that they need to. So, again, helps you resolve a business issue.

Finally, just given the amount of automation and the amount of Artificial Intelligence and Machine Learning that we deploy, your cost per unit for operations delivery is also impacted down, which means you are able to optimize your support costs. That is a good reason as any as I can think of. That is like, almost, a cherry on the top. So really, those are the four reasons why somebody should do it and quite frankly, all four reasons can be directly drawn to a business problem that each of them solves.

Sanjog: That is a lot of good promise from AIOps and automation. So, next question is – what all are the prerequisites for an organization to exploit this automation and AIOps in their environment and what are the pitfalls and gotchas?

Siddharth Dhar: Great question again, Sanjog! I think any deployment of this nature is complex. This one especially so, because the footprint of change is across, almost every aspect of IT operations. Anytime you have a change that impacts such a large footprint of your IT organization, you can bet you will have challenges of execution. You may have technical and technology challenges, you probably will have people challenges and you will certainly have political challenges.

The number one prerequisite in my mind is – management buy-in and sponsorship, because that is what you will count on, when you hit those roadblocks and I promise, you will hit those roadblocks.

The second important prerequisite really is that you have to have a mentality to embrace change. Anything that has such a large footprint of change across the organization and in this case, there are really three dimensions that we are looking to change. We are looking to change the underlying technology, to make it more modern, we are looking to change the skills of the people. Most infra and operations organizations have very little coding capabilities, and this is all about coding. So, you have to reskill and retool your people. Finally, the old organization structures of technology-based organizations or operations versus engineering, some of those will get collapsed, when you deploy them. So, you have to be ready to embrace significant change across your organization and sometimes, I have seen all organizations underestimate the quantum of that change and then struggle with it.

The third prerequisite, really, is that you have to invest time upfront in defining what your goals from this are. You have to be very clear of why you are doing it and if you can’t draw a straight line to a business benefit then you’re probably getting mired in cost-based or cost-related goals, which quite frankly, are not the right goals. Cost is a consequence of doing this, not the reason to do this. Finally, if I have to give you a word of caution on what would be a gotcha, really, the one gotcha is there is no magic bullet. The quantum of change and I just described it to you is pretty significant across the organization and across dimensions. There are companies out there that will say, “Hey you know this is a problem that my tool will solve or if this is a problem I can solve, outsource your operations to me and I will build this for you.” There is no alternative but for you to work hard at it. A lot of the change you have to drive is internal, the technology and the deployment of technology is the easy bit, that is not the hard bit. So, please, if there is one thing to avoid, it is to think of this as a magic bullet. You got to work hard to get there.

Sanjog: Welcome back! So, based on your experience Sid, how successful have these investments related to automation and AIOps been for the organizations that tried it, and have they truly been able to realize the benefits it was intended for?

Siddharth Dhar: You know, there are no guarantees in life, Sanjog. You have to have commitment to an endeavor of this nature for it to deliver success. I am going to give you three examples of my recent past engagements, where we have seen a great amount of success. That is something that has been recognized internally in those organizations as well as externally by the market.

The first example, I will give you is of a healthcare insurance provider. They are one of the largest in our country. So, this company had been grappling with, you know, should they move to public cloud, will they be able to deploy public cloud in a secure way? While they are grappling with that, one thing was clear to them that their current internal in-house data center had to be upgraded to behave more like a private cloud, more software defined, if you may. So, they did that process last year and in that process what we did was we deployed AIOps and automation in-built into the new private cloud data center. What we have been able to do as a consequence of that is pre-deployment of this new private cloud infrastructure, they would have taken anywhere from 27 days to more than a month to deploy the environment that their application teams needed to code. Today, the deployment of an entire environment takes less than a day and mind you I am not talking about just a VM or you know, may be just the network address. The entire dev environment including deploying the treated architecture of the application that they have to code on is done in less than a day. So, that is a great result. You have saved almost 26 days of time and you are enabling your business to make releases that are much faster.

The second example I will give you is of a large Europe-based bank. They are very committed to automation, they, in fact, have had an automation program, a very successful one for the past three years or so. One of their most successful automation deployments has been to automate their end-to-end patching process. Now, this is an organization which has more than 50, 000 servers globally and they are now able to patch each and every one of them across the globe in less than eight hours flat! Think about just over a year ago, we had the hard blade zero-day vulnerability. I mean I know of organizations that have taken days, if not weeks, to patch all of their estates to get rid of that vulnerability and an organization of their size and scale of 50, 000 plus servers can now do it in eight hours flat! Again, a great outcome for them.

The third and the final example I will talk about is another healthcare company based here, in Illinois. It is a mid-sized company. We did a full AIOps and automation platform deployment for them last year and one of the biggest benefits that they have seen is almost a 50% reduction in their Mean Time To Detect (MTTD) or identify a seven issue today and seven issues, as we know lead to business loss, eventually. You know in a traditional setup, the way you identify the root cause of a seven is to open a bridge, have multiple teams in there, and by a process of elimination, you kind of come down and zero in on what the actual root cause is. Today in this organization, even as the seven incident is happening, the system itself is intelligent enough to give us the root cause of the issue, so that we can then solely focus our time on fixing it. In fact, this year, we are going to even attempt to automate the actual fixing, as well. So, in a way you get to auto-healing from there.

You know, these are all great results. I mean these are results that allow deployment of infrastructure quickly. These are results that allow your security postures to be better, and these are results that allow you to keep your downtime minimized. And these are real-life examples that I have seen just in the past year.

Sanjog: So, what holds back companies from being able to fully exploit this automated AIOps and what are the challenges and the suggested remedies you would like to present to the audience?

Siddharth Dhar: Sanjog, I am actually going to refer to the previous question around what the prerequisites are, because to me, it is not getting those prerequisites right that eventually holds back organizations from getting this right.

The number one thing is, most organizations that I have seen, struggle with this. They underestimate the quantum of change, the footprint of the change and the dimensions of the change, across technology people and arch structure and that is something I would say, you know, really holds people back from realizing the true benefits of this program.

The second reason, I see all the time, and I spoke about it a little earlier as well, is not getting your success criteria right. If you are overly focused on the cost benefits of doing something like this, you know, ignoring the cost of missing out on the actual business benefits that the business is looking from you, from something like this, then you are likely to fail or even if you are likely to succeed, you are likely to succeed only in that dimension, not truly impacting what the business needs from you.

The last thing I will say is organizations have to take a pragmatic approach. The quantum of change is significant, therefore, you need to give yourself time and you need to give yourself investments. And to be sure those investments will pay you back, fairly quickly, you have to give an upfront time and investment allocation for a program like this in order to succeed.

So, I would say those are things that I generally see and things that can potentially be avoided early on in the program.

Sanjog: So, Sid, it looks like that automation and AIOps can be a very involved effort and very possible, an organization may require a partner. So, as part of doing due diligence and asking the tough but fair questions in order to select a partner, what would you say would be those questions that one must ask and if Hexaware, which is your organization, ends up being the selected one, the selected partner to achieve this extreme automation objective why should they do so?

Siddharth Dhar: Sanjog, I am going to wear two hats for the two parts of this question. Let me wear my subject matter expert hat for the first part. Obviously, there are many questions that you can ask a potential partner, as you are kind of going through evaluation of potential partners. To me, if I were doing it for myself, the one question that I would ask the potential partner is – “Are you willing to get into an output-based compensation agreement? This means we are not going to pay you for deploying an automation platform or for deploying an AIOps platform. We are not going to pay you for coding any use cases that enable, you know, some of the promises to be made good. We are only going to pay you, when the actual promises that a use case has, are delivered in production.”

I can tell you, only organizations that have done this before, that have the experience and the confidence that comes from having delivered those results, only those organizations will be willing to say yes to something like this.

So, that, to me, is really the most important question that if I were doing it, I would ask my potential partners.

Coming back to the second half of your question, and for a minute, wearing my Hexaware cap on. So, if you think of Hexaware, right, over the past six years, we have transformed ourselves into an organization that has an ‘Automation-First’ mindset. It has not been an easy journey. The cultural change has been rocky, but over the past five years, we have managed to actually transform every Hexawarian to be able to automate themselves out of a job.

We have provided them the mindset and the cultural alignment, first and then we have combined that with retraining our organization, especially on the operation side, providing the coding skill sets required to automate themselves out of a job. Only a few organizations have made that level of cultural and training investments.

The second thing, I will tell you is, especially for organizations that are combining a program like this with operations and execution, you have to realize that all of the account teams that your partners have, they are goaled on customer satisfaction, they are goaled on revenue growth and they are goaled on profitability.

Anytime you take undertake a program like this, you are going to negatively impact the revenue for your partner. This is clearly a disincentive for your partner’s account teams to attempt to do this, because they are cannibalizing their revenues, you know, come the end of the year, they are not going to meet their numbers.

What we have done at Hexaware is – we have actually reversed the paradigm, we have actually incentivized our teams to cannibalize their own revenues. If any account team can show me that the reduction in revenue is directly the result of an automation use case that they have deployed, we simply add that back to the achievement. It is a simple solution but not a lot of organizations have gone down that path.

Finally, I will tell you, experience matters. Today we are the automation partners of choice for over five Fortune 100 organizations, and each of these organizations are large enough to have other large IT service providers in the system, but despite that they have elected to work with Hexaware, as an automation partner for them and that speaks for itself.

Anytime, you can say you have been chosen by a Fortune 50 or a Fortune 100 organization and you have been a part of their program in some time for over three years. That kind of experience is hard to get out there. So, I would say those are three reasons why you should look out at Hexaware as your partner of choice.

Sanjog: Once again, thank you Sid for sharing your thoughts and insights about how an organization can quickly move from traditional IT Ops to Automated AIOps.

Siddharth Dhar: Thank you so much, Sanjog.