How this company took its DevOps capabilities to the next level

“It’s not my machines, it’s your code!” “It’s not my code, it’s your machines!”

This was the argument between John Allspaw and Paul Hammond, two Flickr employees who gave a presentation on cooperation between development and operations at the O’Reilly Velocity Conference 12 years ago. These arguments encapsulated the divide between tech developers and infrastructure operations teams, which caused issues such as slow deployment times and difficulties in problem solving.

The need for companies to iron out these issues eventually gave rise to DevOps – an approach to the delivery of IT services that tries to bridge the gap between software development and IT operations. With this, companies could smooth out internal workflows and improve efficiency with potentially staggering results: organizations proficient at DevOps can deploy new code up to 973x more frequently and recover from IT-related incidents up to 6570x faster.

One company that knows the importance of DevOps is Hyphen Group, a leading fintech company in Greater Southeast Asia. The firm initially tried beefing up its DevOps capabilities with the help of a central team to help maintain smooth business operations – however, both the engineering and DevOps teams struggled to communicate effectively, which often led to bottlenecks that delayed delivery times.

Shannon Murphy, CTO of Hyphen Group / Photo credit: Splunk

“We’ve always had a strong engineering culture and ability to code our own infrastructure, but it was more of an enterprise view where infrastructure and operations were separate from engineering,” explains Shannon Murphy, CTO at Hyphen Group.

And, as the company accelerated its growth last year by acquiring personal finance community platform Seedly and launching several new products, not having fully-integrated DevOps created some growing pains in maintaining the firm’s IT environment.

An overwhelming issue

As the company introduced more code and products, the additional infrastructure made its IT environment more complex. Eventually, it became harder for its developers and infrastructure operations teams to react quickly when problems arose. When major infrastructure outages occurred, it took the company several hours on average to diagnose and fix the problem.

While the company initially tried leveraging its existing engineering and DevOps teams to solve such issues, it ended up hurting these employees.

“My heads of engineering had to come in at two or three in the morning because they were the ones who knew how to dig through the [error] logs,” Murphy shares. Additionally, employees were stuck doing less meaningful work, focusing on solving these errors.

Photo credit: Unsplash

“If you’re spending your time troubleshooting problems, then you’re not doing what you love to do,” he adds. “As a result, you’re not delivering the value that you want to the business.”

Moreover, having DevOps as a centralized effort meant that its DevOps team became increasingly “overwhelmed” as new products and services were added.

“It got to the point where I was pretty sure I was going to lose many of them because it’s just mind-sapping work to sit there and dig through troubleshooting logs, especially when you don’t really know the application that well because you’re not part of the engineering team,” says Murphy.

Empowering its teams

To solve these issues, Hyphen Group had to shore up its DevOps capabilities.

It started with decentralizing its DevOps and giving its engineering teams the responsibility to manage it on their own while retaining platform services – such as networking, security, computation, and IT service management – as centralized functions.

“[It] moves DevOps to where it means the most: the team that creates the product,” says the chief technology officer. “It creates a much better sense of ownership.”

As part of this change, the company also needed to give its engineering and infrastructure operations teams the right tools to succeed. This was when Murphy started to explore potential solutions and connected with Splunk, a data platform that provides observability tools for organizations and DevOps teams.

Murphy says he was sold on the platform’s capabilities centering around end-to-end observability – a term in DevOps that refers to the ability to identify and classify oftentimes unexpected errors in complex environments, helping streamline the problem-solving process and allowing both developers and infrastructure operations teams to focus on adding value to the business such as updating existing software or coming up with new products.

Murphy highlights how having these observability capabilities came in handy for Hyphen Group’s customers, who experienced isolated errors – such as specific web pages being down – on certain content management platforms. In such cases, businesses might not even realize there were errors on their platform or website for several hours.

Photo credit: Unsplash

With Splunk’s Observability Cloud, which provides software solutions to empower observability, Hyphen Group could automatically identify, track, and analyze error reports from these content management platforms. Through this, the firm identified a clear pattern within these error reports and built an automatic script that would resolve similar problems in the future in just under three minutes.

Additionally, these tools helped Hyphen Group integrate error monitoring and observability into newly released software updates for existing products and services. This allowed the company to improve the stability of new releases from the get-go, and the firm was able to conduct its last 275 releases automatically, with none of the issues it had experienced before.

Similarly, this also allowed the company to improve the stability of new infrastructure and reduce the time needed to deploy it. For instance, Hyphen Group launched its new insurance platform in just 45 days – a process that would have earlier taken it up to a year to complete. Hyphen Group is now looking to capitalize on these improved deployment capabilities to facilitate the more efficient release of new products and services in the future.

Clearing the path to success

Moving forward, the company is looking to improve its DevOps processes even further through widespread automation, allowing it to achieve better stability, scalability, and security.

For this, it will need even greater DevOps observability into the inner workings of its applications, which the firm is working on by modifying its applications to accommodate Splunk’s Observability Cloud even better. In the future, Murphy expects these efforts to pave the way for Hyphen Group’s continual growth.

“Where I’m hoping to go with this is understanding system events, business events, and deployment operations in one consistent way, with Splunk providing the infrastructure and application side of that,” he adds. “That is going to be quite awesome and a differentiator for us.”

Splunk Observability Cloud is the only full-stack, analytics-powered and OpenTelemetry-native platform which provides a seamlessly integrated user experience, context and workflow across metrics, traces and logs to solve problems in seconds.

To find out more about Splunk Observability Cloud, visit its website.

This content was produced by Tech in Asia Studios, which connects brands with Asia’s tech community. Learn more about partnering with Tech in Asia Studios.