How we designed a 5G/6G-ready business support system for telco operators

Business support system (BSS) tools allow telecom, media, and entertainment (TME) operations to connect with partners and customers, create offerings, issue bills to customers, and generate interbusiness transactions, such as settlements and point-of-interconnect information.

[ Learn more about modernizing 5G operational and business support systems for the cloud. ]

An enterprise-grade, dynamically scalable, 5G-ready BSS built on bare metal and the cloud can offload platform responsibilities and eliminate headaches for TME companies. It allows those organizations to deploy, test, benchmark, and deliver service-level agreement (SLA)-backed services for the same application catalogs wherever they are used.

Converged charging system (CCS) specifications

Converged charging systems (CCSs) enable TME providers to manage their users and services, including converging "payment methods like prepaid and postpaid, as well as access methods and services like fixed telephony, mobile telephony, broadband, and TV," in real time. They can authenticate users, terminate services when a bill isn't paid, deliver notifications to users, and more.

The 3rd Generation Partnership Project's (3GPP) TS 32.290 provides specifications for CCS.

To support this spec, Red Hat and i2i Systems are collaborating on a distributed 5G/6G-ready BSS solution blueprint using an enterprise-grade, reliable Kubernetes stack as a common platform. The solution leverages private and public cloud infrastructures that can be easily swapped for each other.

The goal is to:

• Help TME providers prepare for 5G and 6G, with its ultrafast and efficient charging grid providing low latency, on-demand scalability, and telco-grade SLAs.
• Provide agility and elasticity to leverage cloud-native development and deployment patterns.

Solution components

This 5G/6G ready BSS solution deployment consists of a 3GPP-compliant centrally hosted converged charging system (CCS) deployment blueprint and an additional expansion/coverage blueprint (5G charging function, or CHF):

• The central CCS blueprint will be deployed on ocp@hyperscaler main regions (see figure 2).
• The expansion/coverage blueprint can be spread across hyperscaler extension points or joints.

Both blueprints can leverage the same OpenShift Container Platform (OCP) release with configuration models, so there is no drift from a platform-as-a-service (PaaS) layer perspective. Alternatively, they can be handled independently from an operational perspective.

In this solution design, the central 5G-ready BSS stack is hosted on a Kubernetes cluster on a cloud (SiteA). The expansion/coverage blueprint (SiteB) is hosted on additional cloud regions. Each deployment blueprint will have its own underlying Kubernetes cluster, interconnected using Submariner (cross-cluster L3 connectivity using encrypted or unencrypted connections) for 5G traffic management.

As the existing BSS product portfolio evolves, some components of the solution will be based on virtual machines (VMs) and some on container packaging. Kubernetes with KubeVirt supports both types of application packaging and deployment on the same converged platform.

[ Learn more about how containers and VMs compare. ]

5G CHF service network design

5G CHF is a Kubernetes application deployment consisting of a deployment manifest that oversees ReplicaSets, pods, services, and network policy. 5G CHF will be exposed through a Kubernetes service construct. It will be discoverable through cluster service discovery by other 5G cloud-native network functions (CNFs) such as session management function (SMF), access and mobility management function (AMF), or user plane function (UPF). (See Edge computing: How to architect distributed scalable 5G with observability for information about these functions.)

The CHF pod and CCS VM use the following interfaces and protocols.

CHF pod:

• 5G charging traffic
  • Pod interface: ETH0
  • Protocol: Nchf/Http2
  • Port: 9191
• BSS upstream
  • Pod interface: ETH0
  • Protocol: Akka/TCP
  • Port: Ephemeral
• Management
  • Pod interface: ETH0
  • Protocol: SSH
  • Port: 9099

CCS VM:

• BSS downstream
  • VM interface: ETH0
  • Protocol: Akka/TCP
  • Port: 25781
• Management
  • VM interface: ETH0
  • Protocol: SSH

  • Port: 2599

5G Service discovery (intracluster)

CHF <-> NRF/SMF/AMF/UPF

As specified in 3GPP 5G specifications, network function (NF) and NF service registration, discovery, and OAuth2-based authentication are realized through NRF integration. CHF instances register with NRF for service discovery that allows 5G NFs to discover which CHF instance is up and ready to serve.

Although there are mechanisms within OpenShift to select services using cluster internal DNS, the NRF-based logic provides the cluster independence, as well as conformance to 3GPP specifications for extra flexibility for NF and NF service selection based on various 5G-specific parameters such as slice, data network name (DNN), public land mobile network (PLMN) identifier, and supported features.

[ Learn why open source and 5G are a perfect partnership. ]

CCS discovery (intercluster)

CHF <-> CCS

The 3GPP specification does not explicitly define the interface and discovery between CHF and the converged charging system (CCS). We are using the Akka remoting mechanism. A fully qualified domain name (FQDN) or cluster IP-based integrations can be used.

CCS container-native virtualization (KubeVirt) can be accessed using a Kubernetes service. In broader deployments, you can use a diameter routing agent (DRA) to perform discovery of charging system entry points.

[ Get the complimentary eBook Kubernetes Patterns: Reusable elements for designing cloud-native applications. ]

BSS features

The i2i BSS system:

• Enables 5G service monetization through integration with 5G Core networks through an Nchf interface (3GPP Release 16.4). This allows charging for new 5G digital bundles, such as network slicing with legacy online charging systems (OCS) or i2i OCS, along with diameter support to integrate with legacy (2G, 3G, 4G) networks.
• Supports dynamic pricing policies that provides highly configurable rate plans, packages, and scenarios. This helps communication service providers (CSPs) speed up the time to market.

• Allows CSPs to manage different services, including voice, video, data, and multimedia, on a single platform and charge customers on a prepaid or postpaid basis.
• Adheres to 3GPP and TMForum industry standards.
• Helps Tier-1 operators deliver performance, reliability, and scalability. It provides a unified platform for converged charging across multiple markets and supports horizontal scalability and "five 9s" availability to scale with operators' businesses.
• Enables the CCS VM to be downsized and scale horizontally by leveraging Kubernetes service constructs (Figure 6) for load balancing.

Setting up the 5G-ready BSS solution sandbox

We are using the following setup in our sandbox for a 5G-ready BSS solution.

BSS central platform (minimum cluster sizing)

The following table shows the minimum requirements for an OpenShift Container Platform (OCP) 4.x release. (Please follow OpenShift's recommended cluster scalability and performance practices.)

NOTE: You must add the CPU, memory, and storage requirements from CNF deployments for 5G core central deployment to the following sandbox cluster specs:

• Bootstrap (ephemeral)
  • 4 CPU cores
  • 16GB memory
  • 120GB storage
  • 1 count
• Master and worker nodes:
  • 4 CPU cores
  • 16GB memory
  • 120GB storage
  • 3 count

The central BSS specs are:

• VM: OCS
• Affinity: Yes
• DPDK/SRIOV: No
• CPU cores: 16
• Memory: 48GB
• Total storage: 500GB

The expansion/coverage platform specs are:

• CNF: CHF
• Affinity: Yes
• DPDK/SRIOV: No
• CPU cores per module: 2
• Memory per module: 4GB
• Attached storage: 15GB

Benchmarking and cost calculations

In our sandbox environment, we used an open source traffic generator function (TGF) from i2i Systems to load the solution to observe the behavior of OpenShift as an application platform as well as CCS/OCS.

TGF is a traffic tool that generates HTTP, diameter, or Akka messages and sends them to desired NFs, then it collects responses and prints statistics. To achieve this, TGF needs to be supplied with a set of user equipment (UE).

To test CHF performance for this application, TGF acts as an SMF and executes three steps to generate traffic:

1. Generates create requests, which include requested units for each UE, sends them to CHF, and establishes charging sessions
2. Generates x number of update requests, including requested and used units for the established sessions at the previous step, and sends them to CHF
3. Generates release requests, including used units, sends them to CHF, and releases established sessions

We performed three different test phases with incremental 5G traffic profiles with x number of concurrent charging sessions or transactions. We measured the following performance:

Average end-to-end (E2E) response time with 150 transactions per second (tps) load: 9.56ms

Average E2E response time with 500tps load: 10.51 ms

Average E2E response time with 1000tps load: 30ms

We also tested the total number of connected charging endpoints per CCS/OCS. We found no explicit limitation on the number of connections to CCS/OCS as long as the total tps capacity was not exceeded.

Benchmark conclusions

In a production environment (with an actual customer field reference);

• A 1728 CPU core (bare metal x18 machines) CCS/OCS solution capacity delivers 100,000tps with a 30ms response time.
• As much as we want higher tps, we must be conscious about response time to prevent fraud. Keeping response time below a certain threshold (<50ms) is acceptable for telecom service providers, as most prepaid offerings are measured using minutes of service given or taken.

As seen in the sandbox environment tests in figures 7, 8, and 9, we reach the same level of capacity with a sandbox VM (16 cores, 48GB memory) to 1000tps with a 30ms response time, which is in line with (actually slightly better) in a live production environment with a bare metal deployment.

Capacity expectations vs. measurements:

• Formula: [TestBed-CPUCount / Production-CPUCount] * ProductionTPSBenchmark -> Expected Sandbox Performance
• Calculation: [16/1728] * 100 = 925.92tps <- Expected value
• Observed: 1000tps <- Real test outcome (see figure 9)

Summary

In this study, we showed that Red Hat OCP with CNV can deliver equivalent (if not better) performance for BSS solutions with mixed container and VM workloads against bare metal. It also provides better total cost of ownership (TCO), optimized infrastructure (on-demand scalability), and consistent deployment experience across different infrastructure types.

The i2i BSS solution stack does not have to run on a particular infrastructure. It can run anywhere (private and public clouds) consistently and reliably by leveraging a common abstraction layer provided and supported by Red Hat.

If you want more information about this project, please watch these presentations.