Leveraging Big Data Technology for High-Performance Subscriber Data Management

The Active Resource Manager (ARM) leverages Big Data technology and an innovative data management architecture to address the unique performance and scalability demands of usage-based subscriber service management.

The platform includes three integrated storage systems:

• A robust relational database for functions requiring transactional integrity
• A highly scalable, noSQL Big Data database for storing and retrieving massive volumes of usage data
• A fast access, in-memory, key-value store for rapid retrieval of information in real-time

Scaling To Billions of Records

The ARM and the Internet Statistics System (ISS) Internet usage accounting module compute subscriber usage and network statistics from raw instrumentation data collected from the network.  Data for system use and functions requiring transactional integrity is stored in the relational database. The ARM's noSQL Big Data database scales in a linear, pay-as-you-grow fashion, and is capable of storing billions of records collected from millions of subscribers, while maintaining a modest hardware footprint. As there may be a need to reference original collected records at a later date, the ARM compresses all records and retains them for a long period of time (typically one year). It uses proprietary compression algorithms specifically designed for this application, achieving an overall compression ratio of 12:1 for DOCSIS 3.0 IPDRs. Internet usage data required for web-based subscriber service portals and real-time service management is maintained in a fast access, in-memory, key-value storage cache for rapid storage and retrieval.

ARM ETL for Highly-Efficient Data Exporting

ARM ETL is a specialized, high-performance, file transfer based, extract, transform and load (ETL) function for streamlining the process of exporting data to other database management systems, including billing systems and enterprise data warehouses. ARM ETL provides a highly efficient and cost effective solution for usage-based billing by selectively transferring only the account, service and usage information required for the billing system, eliminating the need to upgrade the billing database to accommodate the influx of large amounts of additional charging data.