Hide this content.

Introduction

BlackBerry service applications have grown and continue to grow in scope, scale and function at an exponential pace. Application log data is a rich and useful source of information for planning, reporting, measurement and fault monitoring; however, it has become a major challenge for RIM and BlackBerry to take advantage of this data source, due to the challenges imposed by the sheer scale of application log data generated by BlackBerry services, which is in the 10’s and approaching the 100’s of terabytes (trillions of bytes) of raw daily log data. This architecture began as a homegrown solution in the Relay service environment; however, it has quickly become apparent that a global standard architecture is required to efficiently support the needs of BlackBerry Operations and its RIM corporate stakeholders.

The efficient extraction and processing of relevant log data from these environments is critical to BlackBerry’s success as an organization, and directly impacts our ability to plan, scale, predict, and respond to subscriber growth, usage trends, application issues and various business reporting and compliance requirements. This document provides a standard global architecture that attempts to meet the current and future log data collection and distribution needs of the organization in a single, sustainable, flexible and supportable framework.

Architecture Overview

A services oriented architecture defines a solution in terms of its services and interfaces. This approach provides the flexibility needed to maintain a consistent design footprint in the face of widespread and continuously changing information processing needs. The same set of services can be used to build different solutions across the organization, thus addressing the needs of many customers and consumers using the same set of tools and with minimal development effort or re-work.

While the key challenge that BlackBerry Operations faces is the sheer scale of data delivery, consumer requirements can often change very quickly as well. RIM continues to introduce new products and to grow into new markets worldwide, where compliance and regulatory requirements, usage trends and application deployment strategies can vary widely. A service- based architecture enables us to address these different and changing requirements without redesigning our core data delivery architecture. Insofar as that core architecture meets our scalability needs, the services oriented approach provides the agility needed to quickly and easily address the information needs of various consumers.

Much as is the case in other service-oriented frameworks, defining strict standards is the key to providing this flexibility, particularly with regard to interfaces between services and applications that are developed and owned by separate organizations. The definition of these standards is a critical component of this architecture, requiring buy-in and blessing from all service owners and data consumers in order for the architecture to achieve its goals.

Consumer Subscription

Consumer services can easily be built to interface with middle tier data streams in a subscription model. Examples of this ‘live’ logstream approach include some Relay monitoring functions and the Relay Business Systems Critical (BSC) logstream for extraction of reporting data. Relevant information can be filtered from log streams as they are written to the syslog pool, thus reducing the cost of searching through archived log data and preventing unnecessary duplication of data for storage and retention.

Currently, most consumer services are using storage systems to collect and process relevant logstream data; however, there are solutions in the works that will be able to receive and process the filtered logstream directly from a network socket. Some ad-hoc monitoring is already in place that uses this approach, employing a syslog service to receive filtered events and scripts to analyze and respond to application events.

Filtering of interesting application logs is an important contributor to the efficiency of data consumption across the organization. This feature can be used, preferrably in combination with structured loglines, to pull only data that is specific to a particular consumer’s requirements, thereby reducing network usage, storage overhead and data duplication. While some data duplication is certainly unavoidable, consumers are encouraged to perform the necessary legwork to identify specific information needs so that only relevant data need be delivered, and any potential synergies with other consumer environments can be identified and taken advantage of.

Hide this content.

"The sky remained a brilliant blue, its clouds still floating warily beneath it. The leaves still whispered in the breeze. The world still turned, the mob still broiled in the streets. There was nothing to mark his passing."

Hide this content.

Children For Mr. Leary

I’m at the punch bowl wondering when they’ll figure out how to put Stephen Hawking on a unicycle when Denis Leary grabs my crotch and whispers “I’m going to rape your children.”

“Jesus!” I say, knocking his hand away. “I don’t have children.”

“That’s not necessary,” he says, and puts his hand back.

“What the hell are you doing?”

He gets close to me again and whispers in my ear.

I wake up in a cold sweat and desperately reach for my wife.

“Honey-” I say, “Honeybun, we need to have children.”

“What are you talking about?” she says, half asleep.

“We need to start now,” I say, everything becoming clear. “Denis Leary needs them.”

Go to site.

Hide this content.

"She says nothing more as the car winds around the mountain beneath the cool fuzz of the moon...." Read More

"I jigged around her words in my head, watched the cat sharpen his claws on a tree and lick his lips while watching my wife bend over the dead Sneezeweed, my smile fading a little..." Read More

"She raises a hand to brush back her hair, then stops, remembering I shaved her bald last night while she slept..." Read More

Hide this content.