New startup uses Internet to predict the future

By Darren Quick

4 Comments

4 Pictures

An analysis of the Nasrallah person network over 30 days with the curve at the top showing a spike in online momentum

Image Gallery (4 images)

There’s no doubt that most people would like to know the future. It’s a desire that has kept palm readers, astrologists and tealeaf readers in business for hundreds of years. Now there’s a company called Recorded Future that says it can use information scoured from tens of thousands of websites, blogs and Twitter accounts to predict the future. And before you laugh, it’s got some heavyweight backers including Google and the CIA.

Recorded Future uses the term “temporal analytics” to describe what it does. It extracts information including entities, events and the time that these events occur from the thousands of news publications, blogs, niche sources, trade publications, government web sites, financial databases and more that the company continually scans. Using this information the company says it is able to find the relationships between people, organizations, actions and incidents, not only in the past and present, but also in the future.

Google with a difference

In a white paper describing the underlying philosophy and overall system architecture of Recorded Future the company says that a comparison with traditional search engines is inevitable, but that search is only one aspect of its temporal analytics engine.

Unlike the PageRank algorithm that is at the heart of Google’s search engine that uses explicit link analysis by analyzing links between pages and ranking pages based on the number of links pointing to it as well as the rank of the pages pointing to it, Recorded Future adds implicit link analysis to its system. It does this by separating the documents and their content from what they talk about – the “canonical” entities and events – so it is able to look at the “invisible links” between documents that talk about the same, or related, entities and events.

A ranking measure called momentum – based on the number of documents referring to these canonical entities and events while taking into account the credibility of the documents (or document sources) along with several other factors – is the company’s aggregate judgment of how interesting or important an entity or event is at a certain point in time. This momentum measure will change over time.

Predicting the future

So how is this type of analysis supposed to predict the future? In addition to extracting event and entity references, Recorded Future also analyzes the “time and space dimension” of documents – references to when and where an event has taken place, or even when and where it will take place – since many documents actually refer to events expected to take place in the future. A company’s plans to open a production plant in a particular country in the next two years, for example.

This time and space analysis by aggregating weighted opinions about the likely timing of future events is the first way in which Recorded Future says it makes predictions about the future. Additionally, the company says it can also use statistical models to predict future happenings based on historical records of chains of events of similar kinds.

It is this combination of automatic event, entity, time and location extraction, implicit link analysis, and statistical prediction models that forms the basis for Recorded Future’s temporal analytics engine.

The company is also adding more components such as sentiment analyses, which determine an author’s attitude towards his/her topic, and how strong that attitude is.

Big things predicted

Recorded Future has attracted funding from Google’s investment arm, Google Ventures, as well as In-Q-Tel, a not-for-profit venture capital firm that invests in high-tech companies on behalf of the CIA. Although the exact amounts haven’t been disclosed, both were apparently under US$10 million each.

With businesses always looking at ways to gather more information to make better-informed decisions and, more importantly, willing to pay for it, it’s likely investors will see a handsome return on their investment. Individuals hoping to use Recorded Future to predict next week’s lotto numbers probably won’t be as lucky.

Anyone curious can sign up at Recorded Future to receive free “broad” email alerts on events in the future, (which Recorded Future calls “futures”), while a premium subscription costing US$149 provides more detailed temporal analyses including highly specific alerts on events in the future, present or past and dynamic timeline visualizations.

Via Wired