Building Dependable IT Systems

About Crossjoin

In today’s technology-driven world, ensuring reliable, efficient, and secure IT systems is paramount to business success. Crossjoin Solutions, with over 15 years of industry experience and partnerships with more than 30 international customers, specializes in Dependable Systems Engineering (DSE)—a comprehensive approach to creating Dependable IT systems.

With a strong presence in Tier 1 Telecommunications and Financial Services companies, Crossjoin guarantees the fast, reliable, and secure operation of IT systems, where every millisecond counts. This white paper delves into the concept of Dependable Systems Engineering, its critical significance, and how Crossjoin’s methodologies enable businesses to optimize performance while achieving operational excellence.

Executive Summary

Managing Non-Functional Requirements (NFRs) effectively is critical to the success of IT systems and programs. While functional requirements often dominate project budgeting and planning, neglecting NFRs can lead to serious issues such as poor performance, security vulnerabilities, and system unreliability—ultimately compromising the intended business outcomes.

This white paper emphasizes the importance of proactively addressing NFRs throughout the project lifecycle. Crossjoin Solutions’ Dependable Systems Engineering (DSE) approach balances proactive and reactive strategies to optimize system performance, security, scalability, and cost efficiency. DSE ensures operational excellence and alignment with business goals by integrating key elements such as performance management, observability, and structured methodologies.

Crossjoin brings a proven Performance Management Framework (PMF) to guide organizations in transitioning from reactive problem-solving to proactive management. This framework resolves immediate issues and drives long-term improvements in processes, team structures, and technology usage.

Crossjoin’s holistic approach to DSE delivers tangible benefits: reduced operational costs, improved system reliability, enhanced customer experience, and alignment of IT capabilities with strategic business objectives. By following the principles outlined in this white paper, organizations can transform their IT systems into reliable, secure, and efficient enablers of business success.

What are Non-Functional Requirements (NFRs)?

Non-Functional Requirements (NFRs) are essential for IT systems success, even though they are not directly tied to business functions. Key NFRs include:

1. Response time: Ensuring systems performance enables them to respond quickly and meet business KPIs.
2. Uptime assurance: Maximizing reliability and minimizing failures.
3. Security: Safeguarding systems and data against vulnerabilities and breaches.
4. Cost Efficiency: Utilizing resources optimally to control operational expenses.
5. Observability: Capturing telemetry – logs, traces, dumps – from running systems to understand their behavior and correlating this information to monitor IT and Business KPIs.

Dependable IT Systems

A “Dependable IT System” is a system designed to consistently meet the expectations of users and the business, ensuring the dimensions of availability, reliability, maintainability, security, scalability, resilience, and response time.

Dependable systems focus on a healthy combination of Business Functionality, Non-Functional Requirements, proper Architecture design, and Infrastructure planning. A project cannot be successful with complete business functionality and infrastructure design but lacks performance, security, or scalability.

The primary reasons an IT system needs to be dependable are:

1. Business Continuity: Dependable IT systems ensure uninterrupted business operations. Downtime or failures can disrupt workflows, lead to revenue losses, and damage customer trust.
2. Data Integrity and Security: A dependable IT system protects sensitive data from loss, corruption, or breaches. This is essential for maintaining confidentiality, compliance with regulations, and safeguarding intellectual property.
3. Customer Satisfaction: Dependable IT systems provide consistent service to customers, ensuring positive experiences, fostering loyalty, and reducing churn.
4. Operational Efficiency: Dependable IT systems allow employees and processes to function without delays or errors, optimizing productivity and minimizing wasted resources.
5. Reputation Management: System failures or breaches can harm an organization’s reputation. Dependable IT systems protect against such risks, preserving trust among stakeholders.
6. Cost Savings: Preventing system failures and ensuring regular operation reduces unplanned maintenance, downtime penalties, and the financial impact of resolving critical incidents.
7. Regulatory Compliance: Many industries require adherence to strict standards for IT reliability and security. Dependable systems help meet these requirements and avoid legal consequences.
8. Scalability and Growth: As businesses grow, they need IT systems that can scale without compromising performance or reliability. Dependability ensures a foundation for sustainable expansion.
9. Support for Decision-Making: Dependable IT systems provide accurate and timely data for decision-making processes, enabling organizations to respond effectively to challenges and opportunities.
10. Risk Mitigation: Dependability reduces the risk of operational failures, data loss, and security incidents, ensuring resilience against unforeseen challenges.

Proactive vs. Reactive NFR Management

Organizations can address NFRs through two primary approaches:

• Proactive Management: Involves planning, measuring, and improving systems during the program development lifecycle to prevent production issues. Activity examples include creating performance and observability streams for the project, baselining existing metrics, business KPI design, and a quality program that covers NFR testing, including code reviews, load and security testing.
• Reactive Management: Focuses on resolving issues after deployment. In this approach, NFRs are usually bundled with functional requirements within the system’s support and operations team roles. This bakes unpredictability into the team’s tasks, creating long-term sustainability problems. Additionally, it has a compound effect: due to their specialized nature, most operations and support teams are ill-equipped to deal with NFR problems and require assistance from development to deal with them. This generates an unpredictability cascading effect into the development lifecycle of projects, impacting the whole IT department.

These two approaches are both important and mandatory. The discussion is not about following one or the other, it is about how to balance them to achieve the best results.

Building a Dependable IT Systems

The most common scenario for a new program is to include some aspects of NFRs but in a non-structured/inconsistent way. The program may include load testing, but lack KPI targets, or have observability without correlating the gathered data with business processes. The most common root causes are:

• Lack of understanding or experience of the importance of NFRs
• Lack of structure and methodology on the way NFRs are approached
• Lack of budget or time to implement the NFRs within the program

This scenario involves a new program allocating 20% of its “NFR budget” (if such a concept exists) during the program’s development and leaving the remaining 80% to be sorted out after the program goes live. In our view, this approach would be characterized as 20% proactive and 80% reactive.

This approach may have far-reaching consequences, including increased operational and infrastructure costs, degraded business KPIs, lack of capability to perform timely Root Cause analysis, compromised customer experience, and higher security risks. For more details on this subject, please refer to Crossjoin’s white paper on Cost-efficient IT (link).

Crossjoin addresses these challenges through its Dependable Systems Engineering (DSE) offering to minimize program risks and maximize efficiency.

What is Dependable Systems Engineering?

DSE is the strategic end-to-end management of Non-Functional Requirements to build Dependable IT systems that are aligned with the program’s business goals. It integrates elements of performance, reliability, security, and cost-effectiveness into a cohesive and holistic framework. 

DSE: Critical success factors

Crossjoin’s DSE offering enables organizations to optimize their IT systems through a balanced approach to NFRs and considering different, but complementary viewpoints for the problem:

• Processes: assess the end-to-end processes that underlie the software development, lifecycle, security compliance, observability, quality assurance, DevOps, Release Management, Architecture, Support, Operations and many more.
• People: assess how the teams are organized to achieve the processes and the company’s objectives. It is very common to find duplications, overlaps, gray-areas, misaligned objectives and KPIs, …
• Technology: assess how the company is using technology to support its processes and projects in relation to its objectives and KPIs. 

By taking a holistic approach instead of just looking at technology, Crossjoin achieves much greater efficiency goals than what would be possible otherwise. This is also part of providing Dependable Systems Engineering. As a result of this activity, Crossjoin will provide recommendations on how to improve each one of the areas and work with the customer to implement them. It is this last part that guarantees a successful approach instead of a failed consulting effort.

DSE: How do we do it?

With years of experience, Crossjoin has developed a Performance Management Framework (PMF) that provides a structured approach to effectively managing IT NFRs. The PMF includes tools to measure the maturity of existing DSE practices and a methodology to improve them. 

The PMF is at the center of our approach. Crossjoin’s offer covers the full lifecycle of a program and includes specific activities at each stage, including both proactive and reactive activities for multiple domains. 

From Reactive to Proactive

Most projects cannot be initiated in the perfect scenario where every NFR is considered and planned upfront. In fact, most projects are exactly the opposite: NFRs are an afterthought considered once the program has reached the Quality Assurance phase (best-case) or Production (worst-case). This leads to delays, cost overruns, and a difficult-to-manage program. In the worst-case scenario, the program will generate a Non-Dependable IT system, leading to the problems highlighted in previous chapters.

Crossjoin’s vast experience enables a smooth transition from focus on Reactive to Proactive for existing projects by following these steps:

1. Reactively control production: work with the operations and maintenance teams to solve the most urgent and critical incidents. This is done by quickly diagnosing and performing root cause analysis for production incidents and working with the development, infrastructure, database, operations, and network teams to resolve them by providing recommendations and insights.
   Benefit: improved turnaround time for problem resolution and avoided problem recurrence.
2. Proactively control production: together with the customer, create a performance management team that will proactively look at the production systems and prevent problems from happening. This is done by looking at patterns in production, creating alarms and solving issues before they become major incidents. Crossjoin’s work is to implement and monitor, together with the operations team, and provide root cause analysis and recommendations for the other teams.
   Benefit: avoiding certain types of problems affecting production, reducing downtime and incidents.
3. Understand – Leverage observability for operations: re-use, automate, or implement observability tools to provide easily accessible troubleshooting for production issues.
   Benefit: increased capability to perform 1 and 2. Ability to perform cost/benefit analysis for Proactive measures.
4. Introducing proactive measures: acknowledging the most likely NFR areas impacting performance, security, reliability, and scalability. Provide recommendations, root cause analysis, skills or tools to proactively prevent problems from occurring in production. This can include load testing and performance-focused code reviews for performance problems, security testing and code reviews for security issues, architecture, devops and software development lifecycle recommendations or implementation for quality or development speed problems. This is usually done on a project basis to avoid disruption. Progressively recommend changes to people, processes and technology so that tactical solutions for “Project A” are converted into strategic changes for teams, processes, and technology on all projects. Over time, this will result in structural changes to the way software development is done, converting a reactive approach into a proactive approach.
   Benefit: structurally avoiding problems from creeping into production during the system’s development and maintenance activities. Lowers OPEX, infrastructure costs and improves performance and reliability. Mature software development practices that include NFRs, lead to all the benefits highlighted in this paper.
5. Automate operations and support to move away from firefighting.
   Benefit: less OPEX and faster resolution times for critical issues. Happier IT teams, business and customers.

Rebalancing the Scale

Beyond NFRs implementation, another crucial role played by Crossjoin’s approach is balancing the scales between Customers and Providers (systems integrators, software vendors, and cloud vendors). Our facts-based approach derived from the Performance Management Framework and our G2P2EC methodology allows for straightforward, no-nonsense, facts-based discussions between all parties. This avoids endless discussions on who’s responsible for fixing a specific issue and allows the responsible parties to quickly solve the problem either internally OR with the solution provider.   

This approach allows managed services to be operated based on KPIs and metrics from an independent source, eliminating finger-pointing and responsibility avoidance. Additionally, it includes the validation of the NFR results from the providers’ work, establishing a crucial verification point essential for systems acceptance and compliance before going live.

Conclusion

Implementing Dependable Systems is not just a technical necessity but a strategic imperative for organizations in today’s fast-paced, technology-driven world. As described, effective management of Non-Functional Requirements (NFRs) and the adoption of a balanced proactive and reactive approach to IT program management can significantly reduce operational risks, improve business performance, and enhance customer satisfaction.

Other related articles