How ElasticSearch and Kibana make the c|g BI IT solution “just work”

Lets start this off with an intro of what the c|g BI IT products are designed to do.  We looked for a way to index several hundred thousand data points and provide easy access to them in real time to OUR end users that needed them.  We needed real-time, agent-less collection from MANY different types of sources (Servers, Routers, Switches, Tape Robots, Arduino boards, etc).  Some data points are simple to track and trend, ie. “How many tickets related to backup jobs were created?”  Others are much more complex, “How many backup jobs are failing due to traffic timeout in Dallas, TX  during inclement weather on Saturdays?”

This led us to to investigate several leading commercially available solutions, SAP BusinessObjects, IBM Cognos, Oracle OBIE/Hyperion, and Microsoft BI to name a few.  For us, it came down to a combination of products that needed to be integrated and accessible on a global scale.  The result of this effort is three fold.

  1. We have purpose designed hardware tuned for the unique work loads of BI/EDW clusters
  2. We have a custom designed BI platform that actually gives us faster and more specific information than splunk (an industry standard).
  3. We have installe

Private Cloud Solutions and You

A Microsoft private cloud transforms your datacenter. Building on your existing investments and skill sets, you can now get true cloud computing capabilities that help deliver new levels of agility, focus, and cost-efficiency. With the advanced capabilities of Windows Server 2012 and System Center 2012, a Microsoft private cloud provides a range of powerful benefits across your IT environment.

All About the App

To gain a real edge, you need to go beyond just managing infrastructure. You need to manage your applications in a new way. A Microsoft private cloud helps you deliver apps faster, keep them up and running more reliably, and ultimately enable more predictable service level agreements.

Cross-Platform from the Metal Up

IT environments today encompass a wide range of OS, hypervisor, and development tools. The Microsoft private cloud is designed to let you comprehensively support your heterogeneous IT environment, leveraging your investments and maximizing your development skill sets. Now you can keep what you have and make the move to a new kind of agility.

Foundation for the Future

Cloud computing offers the promise of new levels of agility through advanced capabilities, ultimately opening new avenues of innovation for your business. The Microsoft private cloud enables a flexible, highly-available infrastructure that can scale to meet the most advanced enterprise requirements. With built-in automation and new file and storage services, you can get more power from your datacenter and realize greater value from your investments.

Cloud on Your Terms

The move to cloud computing involves more than just building a private cloud. The challenge is to leverage your existing investments to build the right mix of private and public cloud solutions for your business—one that will work for you today and in the future. With Microsoft, you have the freedom to choose. Our solutions are built to give you the power to construct and manage clouds across multiple datacenters, infrastructures, and service providers—on terms that you control, across cloud implementations.

5 Myths for Social Business Intelligence

5 Social Business Intelligence Myths Holding Businesses Back

With significant advancements in both technology and methodology surrounding the ability to filter, classify and structure the daily flood of open social commentary at a scale never before possible, several one-time truths related to social business intelligence (SBI) have now evolved into myths. For many organizations, these myths are serving as gating factors to achieving the actionable, strategic SBI to enhance their decision-making and drive overall strategy and innovation.Let’s dispel some of these major SBI myths.

  • Myth 1: Basic “buzz” on our business is the best we can get from social media.For years, companies have relied on basic “buzz,” or the general sentiment consumers have on a business, brand or product. Essentially, this answers the simple question as to whether consumers like something or not. With questionable accuracy and a lack of related actionable insight related to “buzz,” many companies have thrown up their hands assuming that this “thumbs up / thumbs down” gauge is the best they can do when it comes to SBI. Many of these same companies are realizing that understanding whether they would win a popularity contest does not strategically aid their organizations to effectively make decisions, solve problems, set strategy or drive innovation.Today, advanced streaming big data processing solutions are able to filter, classify, structure and report on the continual flood of open social commentary across the Internet in real-time. For example, ListenLogic’s (listenlogic.com) streaming big data platform processes over one billion operations per second, delivering real-time answers to incredibly complex questions across massive amounts of streaming open social commentary. This is revolutionizing the valuable insight businesses immediately gain from social media and online channels, allowing companies to receive and leverage genuine, unbiased, timely consumer intelligence on their markets, products, services and competitors through the unsolicited opinions of millions of consumers. Today it’s no longer about the simplistic and misleading concept of “buzz” or “sentiment,” rather it’s about deep understanding and actionable insight to facilitate better decision-making and enhanced strategy. To gain true value out of an SBI solution leading corporations are going  way beyond “buzz.” And with advanced technology it’s now possible.
  • Myth 2: SBI is a “nice to have” for our business.This is actually an understandable stance for companies who believe in the first myth above; that extracting simple “buzz” is the best their organization can hope for from open social commentary. However, given the technology advancements that now deliver deep, real-time, actionable insight on valuable aspects like consumer personas and decision points, path-to-purchase, competitive shifts and influencer analysis, this is now a dangerous myth resulting in some businesses missing a huge strategic competitive advantage; an advantage that in many cases their competitors are already adopting.Pioneering businesses are exploiting this advantage to understand their consumers, markets, products, services and competitors better than ever before. They realize that superficially “monitoring” a snippet or sample of the conversation rather than deeply surveying the entire market discussion places their organization at a major disadvantage. Companies investing in this strategic insight are able to leverage these market truths to take strategic or tactical action in order to gain tremendous advantages in growing and protecting their business, immediately transforming advanced SBI into a necessity rather than a “nice to have.”
  • Myth 3: Boolean keyword search is the only solution available.Although they can be highly specific in the results they provide, Boolean keyword search technology is very limiting in what it delivers, typically providing only 10 percent of the relevant results out there and 90 percent noise (irrelevant results or spam). Yes, if specific keywords like “Chevrolet,” “Sonic” and “service” are used in a search they will find the owner who posts “Having a service issue with my Chevrolet Sonic.” However, most consumers don’t speak and certainly don’t post, message, chat or tweet this way. Plus, if the keyword “Sonic” is run alone it will result in a massive amount of irrelevant content on everything from fast food to video games to broadband services.What this approach won’t find is the owner who posts “Heard the grinding again this AM for the third time.” In this example there is no specific reference to a car or a make or model or “service,” rather it provides a concept. Our brains understand that this is referring to a car and that the poster is having a service issue and is probably at best annoyed and at worst irate. This is parallel processing, which the human brain is capable of. Technology is also capable of this with the use of advanced conceptual models, which recognize how people communicate beyond a select set of keywords. This allows for, as an example, a company to identify irate owners of their products without specifically declaring themselves “irate owners” of their products. This modeling has made Boolean keywords obsolete for accurately discovering conversation concepts from consumers within the complex, unstructured social media universe.
  • Myth 4: Our company has to hire a team of experts to achieve true SBI.With a myriad of self-service, keyword tools available, many companies are realizing that you in fact “get what you pay for” with these first generation solutions. They are also coming to understand that although the cost of the tool may be relatively cheap, the subsequent personnel and quality costs skyrocket financially, particularly when they realize that this approach delivers 90 percent noise and 10 percent signal, missing 90 percent of the relevant signal they’re in search of. What’s worse are the potential reputational and financial costs related to regularly missing key insights.Companies are also realizing that the cost and quality benefits of partnering with an advanced SBI provider with advanced technology and expertise quickly delivers strong ROI related to the level of real-time actionable intelligence they receive to set strategy, make decisions and adjust tactics across their enterprise. Leveraging an advanced Social Listening Intelligence Center like the one pictured below, staffed by expert analysts provides an extension to Marketing, Brand, Sales, Product, Corporate Communication, Risk, Legal and Compliance Teams within the organization.
  • Analysts like these deliver the qualitative “art” of advanced social intelligence which supplements the quantitative “science” of the technology. Both the art and science combined deliver the speed and accuracy businesses require of actionable social business intelligence to gain deep, real-time insight to take immediate action on emerging threats and opportunities.
  • Myth 5: The costs to achieve true SBI is out of our company’s reach.Yes, there are big data platforms that cost in the tens of millions of dollars and require millions more in personnel investment to program and operate. Unfortunately, in most cases, even with this sort of massive investment, these sorts of systems will not give an organization the ability to handle streaming big social data on a real-time basis since they are largely designed to query static data. So, going down this path is largely pointless for the organization looking for real-time, actionable, advanced SBI.Major corporations have achieved advanced social intelligence at a cost of pennies per billion operations per second – without the need to make massive hardware, software or personnel investments like general-purpose data systems require. In fact, leading companies across the pharmaceutical, food and beverage, technology, media and consumer packaged good segments are using this technology today at cost that is typically lower than that of an analyst.

    Given the major competitive advantages true advanced social intelligence delivers, pioneering companies who have overcome these myths are gaining deeper understanding of their consumers, markets, products and competitors to strategically impact their acquisition and retention marketing, promotions, customer service, risk management and product development. All in all they are leveraging a major first-mover competitive advantage which is helping them own markets, drive growth and even protect their operations against risks and threats.

    These organizations are now setting strategies built on market truths extracted from millions of consumers allowing them to make decisions and understand market movements better than ever before.