March 20, 2013 – Decrypted Matrix Radio: CIA Spook’s Message, Data Mining Exposed, Cyprus Explained, Internet Surveillance, Contract Security War Crimes

March 20, 2013 – Decrypted Matrix Radio: CIA Spook’s Message, Data Mining Exposed, Cyprus Explained, Internet Surveillance, Contract Security War Crimes

A Motivational Message From Robert Steele – Former CIA

Why the Cyprus Situation Might become a big deal

The Internet Is a Surveillance State

Extreme Data Mining – What we know about tracking users Online

The Privatization of War: Mercenaries, Private Military and Security Companies

3-20

Every Week Night 12-1am EST (9-10pm PST)

– Click Image to Listen LIVE –

1 Billion Invested: FBI Launches Facial Recognition Project

1 Billion Invested: FBI Launches Facial Recognition Project

The Next Generation Identification programme will include a nationwide database of criminal faces and other biometrics

“FACE recognition is ‘now’,” declared Alessandro Acquisti of Carnegie Mellon University in Pittsburgh in a testimony before the US Senate in July.

It certainly seems that way. As part of an update to the national fingerprint database, the FBI has begun rolling out facial recognition to identify criminals.

It will form part of the bureau’s long-awaited, $1 billion Next Generation Identification (NGI) programme, which will also add biometrics such as iris scans, DNA analysis and voice identification to the toolkit. A handful of states began uploading their photos as part of a pilot programme this February and it is expected to be rolled out nationwide by 2014. In addition to scanning mugshots for a match, FBI officials have indicated that they are keen to track a suspect by picking out their face in a crowd.

Another application would be the reverse: images of a person of interest from security cameras or public photos uploaded onto the internet could be compared against a national repository of images held by the FBI. An algorithm would perform an automatic search and return a list of potential hits for an officer to sort through and use as possible leads for an investigation.

Ideally, such technological advancements will allow law enforcement to identify criminals more accurately and lead to quicker arrests. But privacy advocates are worried by the broad scope of the FBI’s plans. They are concerned that people with no criminal record who are caught on camera alongside a person of interest could end up in a federal database, or be subject to unwarranted surveillance.

The FBI’s Jerome Pender told the Senate in July that the searchable photo database used in the pilot studies only includes mugshots of known criminals. But it’s unclear from the NGI’s privacy statement whether that will remain the case once the entire system is up and running or if civilian photos might be added, says attorney Jennifer Lynch of the Electronic Frontier Foundation. The FBI was unable to answer New Scientist‘s questions before the magazine went to press.

The FBI hasn’t shared details of the algorithms it is using, but its technology could be very accurate if applied to photographs taken in controlled situations such as passport photos or police shots.

Tests in 2010 showed that the best algorithms can pick someone out in a pool of 1.6 million mugshots 92 per cent of the time. It’s possible to match a mugshot to a photo of a person who isn’t looking at the camera too. Algorithms such as one developed by Marios Savvides’s lab at Carnegie Mellon can analyse features of a front and side view set of mugshots, create a 3D model of the face, rotate it as much as 70 degrees to match the angle of the face in the photo, and then match the new 2D image with a fairly high degree of accuracy. The most difficult faces to match are those in low light. Merging photos from visible and infrared spectra can sharpen these images, but infrared cameras are still very expensive.

Of course, it is easier to match up posed images and the FBI has already partnered with issuers of state drivers’ licences for photo comparison. Jay Stanley of the American Civil Liberties Union urges caution: “Once you start plugging this into the FBI database, it becomes tantamount to a national photographic database.”

SOURCE: NewScientist

Pre-Crime Software Moves One Step Closer to Reality

Pre-Crime Software Moves One Step Closer to Reality

The era of Big Data is upon us. Major corporations in the areas of advertising, social media, defense contracting, and computing are forming partnerships with government agencies to compile virtual dossiers on all humans.

This data integration initiative is taking place across the board in our largest federal agencies and departments as part of an Office of Science and Technology outline (PDF) that includes a $200 million upfront investment, as well as a $250 million annual investment by military departments into human-computer interaction.

The $200 million in the Obama program will be spread among the National Institutes of Health, Department of Defense, National Science Foundation, the Department of Energy, the U.S. Geological survey, and DARPA to see that the information they collect will move quickly “from data to decisions.”

Seeing the vast potential of Big Data management and applications, Oblong Industries – the actual creator of the software that appeared in the movie Minority Report, (known by its propriety name g-speak) – is now offering a commercial version in the marketplace. An AFP article posted at Raw Story is quick to point out that the software has been stripped of its “pre-crime” detection analytics. But should this blanket dismissal by a mainstream news agency be comforting in light of stated U.S. government goals that seek to turn science fiction into science reality?

John P. Holdren, Obama’s science czar, and author of the controversial eugenics tome, Ecoscience, is one of those directing the solution to the data overload problem. This alone should force us to be hyper-vigilant.

We also know that the NSA is constructing a massive new $2 billion data center that aims to expand its spy activities by September, 2013:

A project of immense secrecy, it is the final piece in a complex puzzle assembled over the past decade. Its purpose: to intercept, decipher, analyze, and store vast swaths of the world’s communications as they zap down from satellites and zip through the underground and undersea cables of international, foreign, and domestic networks.

(…)

Flowing through its servers and routers and stored in near-bottomless databases will be all forms of communication, including the complete contents of private emails, cell phone calls, and Google searches, as well as all sorts of personal data trails—parking receipts, travel itineraries, bookstore purchases, and other digital ‘pocket litter. It is, in some measure, the realization of the “total information awareness” program created during the first term of the Bush administration… (Source)

The FBI has recently announced that facial recognition will be coming to a state near you:

Recently-released documents show that the FBI has been working since late 2011 with four states—Michigan, Hawaii, Maryland, and possibly Oregon—to ramp up the Next Generation Identification (NGI) Facial Recognition Program. When the program is fully deployed in 2014, the FBI expects its facial recognition database will contain at least 12 million “searchable frontal photos.” (p. 6) (Source)

While Oblong Industries claims to have disabled the analytics portion of the software that could be used in pre-crime screening taken right from the Minority Report film, they state that private companies and law enforcement agencies could certainly augment their scaled-back version by introducing their own analytics.

Furthermore, the earlier-mentioned AFP article downplays the significance of the software by stating that “Oblong currently has no government customers in the United States or abroad but offers itself as ‘a core technology provider.'”

However, two of Oblong’s current clients are Boeing – the world’s second largest defense contractor and drone specialists; and #12 General Electric – specialists in electronic warfare components and military communication systems. Their combined annual sales are more than $35 billion, which makes it quite likely that new technology will be embraced and proliferated. (Source)

Oblong’s own website states that their company “works with Global Fortune 500 companies, government agencies, and universities to develop custom g‑speak solutions.” Their application areas include:

  • financial services
  • network operations centers
  • logistics and supply-chain management
  • military and intelligence
  • automotive
  • natural resource exploration
  • data mining and analytics
  • bioinformatics
  • trade shows and theatrical presentations
  • medical imaging
  • consumer electronics interfaces

(Source: Oblong Industries “Client Solutions”)

To reinforce the obvious, Oblong’s Chief Executive, Kwin Kramer, has stated flatly that, “We think law enforcement and intelligence are big data users and we think our technology is the leader.”

It is also worth noting that the g-speak “Minority Report” technology was developed at MIT and has been 30 years in the making. MIT has quite a storied history of connections to intelligence agencies; the CIA having actually started the Massachusetts Institute of Technology Center for International Studies (MIT-CIS) in 1950. (Source)

The implied connections are in plain view on MIT’s own website: http://web.mit.edu/policies/14/14.5.html

And Orwell smiles.
Until recently, we might have held out hope that it was the vastness of data collection that was the weakest point for potential tyrants, as it was nearly impossible to sift through, manage, and apply all of it. With the birth of Big Data applications that are literally coming straight from science fiction, however, we might do well to accept the possibility of a truly dystopian reality descending upon us even faster than the speed of thought.

For a glimpse into the cool sales pitch behind Oblong Industries’ Big Data management software, please view the following video. While we should keep an open mind to how advancements in computer tech can be exciting and rewarding, while offering novel ways of human interaction, it is most important to ensure that these technologies are not hoarded by governments and elites who have a clear history of negatively impacting true human development, as well as freedom of movement, expression and self-determination.

 

ByNicholas West

SOURCE: ActivistPost.com

 

14 Incredibly Creepy Surveillance Technologies That Big Brother Will Be Using To Spy On You

14 Incredibly Creepy Surveillance Technologies That Big Brother Will Be Using To Spy On You

Most of us don’t think much about it, but the truth is that people are being watched, tracked and monitored more today than at any other time in human history. The explosive growth of technology in recent years has given governments, spy agencies and big corporations monitoring tools that the despots and dictators of the past could only dream of.

Previous generations never had to deal with “pre-crime” surveillance cameras that use body language to spot criminals or unmanned drones watching them from far above. Previous generations would have never even dreamed that street lights and refrigerators might be spying on them. Many of the incredibly creepy surveillance technologies that you are about to read about are likely to absolutely astound you. We are rapidly heading toward a world where there will be no such thing as privacy anymore. Big Brother is becoming all-pervasive, and thousands of new technologies are currently being developed that will make it even easier to spy on you. The world is changing at a breathtaking pace, and a lot of the changes are definitely not for the better.

The following are 14 incredibly creepy surveillance technologies that Big Brother will be using to watch you….

#1 “Pre-Crime” Surveillance Cameras

A company known as BRS Labs has developed “pre-crime” surveillance cameras that can supposedly determine if you are a terrorist or a criminal even before you commit a crime.

Does that sound insane?

Well, authorities are taking this technology quite seriously. In fact, dozens of these cameras are being installed at major transportation hubs in San Francisco….

In its latest project BRS Labs is to install its devices on the transport system in San Francisco, which includes buses, trams and subways.

The company says will put them in 12 stations with up to 22 cameras in each, bringing the total number to 288.

The cameras will be able to track up to 150 people at a time in real time and will gradually build up a ‘memory’ of suspicious behaviour to work out what is suspicious.

#2 Capturing Fingerprints From 20 Feet Away

Can you imagine someone reading your fingerprints from 20 feet away without you ever knowing it?

This kind of technology is actually already here according to POPSCI….

Gaining access to your gym or office building could soon be as simple as waving a hand at the front door. A Hunsville, Ala.-based company called IDair is developing a system that can scan and identify a fingerprint from nearly 20 feet away. Coupled with other biometrics, it could soon allow security systems to grant or deny access from a distance, without requiring users to stop and scan a fingerprint, swipe an ID card, or otherwise lose a moment dealing with technology.

Currently IDair’s primary customer is the military, but the startup wants to open up commercially to any business or enterprise that wants to put a layer of security between its facilities and the larger world. A gym chain is already beta testing the system (no more using your roommate’s gym ID to get in a free workout), and IDair’s founder says that at some point his technology could enable purchases to be made biometrically, using fingerprints and irises as unique identifiers rather than credit card numbers and data embedded in magnetic strips or RFID chips.

#3 Mobile Backscatter Vans

Police all over America will soon be driving around in unmarked vans looking inside your cars and even under your clothes using the same “pornoscanner” technology currently being utilized by the TSA at U.S. airports….

American cops are set to join the US military in deploying American Science & Engineering’s Z Backscatter Vans, or mobile backscatter radiation x-rays. These are what TSA officials call “the amazing radioactive genital viewer,” now seen in airports around America, ionizing the private parts of children, the elderly, and you (yes you).

These pornoscannerwagons will look like regular anonymous vans, and will cruise America’s streets, indiscriminately peering through the cars (and clothes) of anyone in range of its mighty isotope-cannon. But don’t worry, it’s not a violation of privacy. As AS&E’s vice president of marketing Joe Reiss sez, “From a privacy standpoint, I’m hard-pressed to see what the concern or objection could be.”

You can see a YouTube video presentation about this new technology right here.

#4 Hijacking Your Mind

The U.S. military literally wants to be able to hijack your mind. The theory is that this would enable U.S. forces to non-violently convince terrorists not to be terrorists anymore. But obviously the potential for abuse with this kind of technology is extraordinary. The following is from a recent article by Dick Pelletier….

The Pentagon’s Defense Advanced Research Projects Agency (DARPA) wants to understand the science behind what makes people violent, and then find ways to hijack their minds by implanting false, but believable stories in their brains, with hopes of evoking peaceful thoughts: We’re friends, not enemies.

Critics say this raises ethical issues such as those addressed in the 1971 sci-fi movie, A Clockwork Orange, which attempted to change people’s minds so that they didn’t want to kill anymore.

Advocates, however, believe that placing new plausible narratives directly into the minds of radicals, insurgents, and terrorists, could transform enemies into kinder, gentler citizens, craving friendship.

Scientists have known for some time that narratives; an account of a sequence of events that are usually in chronological order; hold powerful sway over the human mind, shaping a person’s notion of groups and identities; even inspiring them to commit violence. See DARPA proposal request HERE.

#5 Unmanned Drones In U.S. Airspace

Law enforcement agencies all over the United States are starting to use unmanned drones to spy on us, and the Department of Homeland Security is aggressively seeking to expand the use of such drones by local authorities….

The Department of Homeland Security has launched a program to “facilitate and accelerate the adoption” of small, unmanned drones by police and other public safety agencies, an effort that an agency official admitted faces “a very big hurdle having to do with privacy.”

The $4 million Air-based Technologies Program, which will test and evaluate small, unmanned aircraft systems, is designed to be a “middleman” between drone manufacturers and first-responder agencies “before they jump into the pool,” said John Appleby, a manager in the DHS Science and Technology Directorate’s division of borders and maritime security.

The fact that very few Americans seem concerned about this development says a lot about where we are as a nation. The EPA is already using drones to spy on cattle ranchers in Nebraska and Iowa. Will we eventually get to a point where we all just consider it to be “normal” to have surveillance drones flying above our heads constantly?

#6 Law Enforcement Using Your Own Cell Phone To Spy On You

Although this is not new technology, law enforcement authorities are using our own cell phones to spy on us more extensively than ever before as a recent Wired article described….

Mobile carriers responded to a staggering 1.3 million law enforcement requests last year for subscriber information, including text messages and phone location data, according to data provided to Congress.

A single “request” can involve information about hundreds of customers. So ultimately the number of Americans affected by this could reach into “the tens of millions” each year….

The number of Americans affected each year by the growing use of mobile phone data by law enforcement could reach into the tens of millions, as a single request could ensnare dozens or even hundreds of people. Law enforcement has been asking for so-called “cell tower dumps” in which carriers disclose all phone numbers that connected to a given tower during a certain period of time.

So, for instance, if police wanted to try to find a person who broke a store window at an Occupy protest, it could get the phone numbers and identifying data of all protestors with mobile phones in the vicinity at the time — and use that data for other purposes.

Perhaps you should not be using your cell phone so much anyway. After all, there are more than 500 studies that show that cell phone radiation is harmful to humans.

#7 Biometric Databases

All over the globe, governments are developing massive biometric databases of their citizens. Just check out what is going on in India….

In the last two years, over 200 million Indian nationals have had their fingerprints and photographs taken and irises scanned, and given a unique 12-digit number that should identify them everywhere and to everyone.

This is only the beginning, and the goal is to do the same with the entire population (1.2 billion), so that poorer Indians can finally prove their existence and identity when needed for getting documents, getting help from the government, and opening bank and other accounts.

This immense task needs a database that can contain over 12 billion fingerprints, 1.2 billion photographs, and 2.4 billion iris scans, can be queried from diverse devices connected to the Internet, and can return accurate results in an extremely short time.

#8 RFID Microchips

In a previous article, I detailed how the U.S. military is seeking to develop technology that would enable it to monitor the health of our soldiers and improve their performance in battle using RFID microchips.

Most Americans don’t realize this, but RFID microchips are steadily becoming part of the very fabric of our lives. Many of your credit cards and debit cards contain them. Many Americans use security cards that contain RFID microchips at work. In some parts of the country it is now mandatory to inject an RFID microchip into your pet.

Now, one school system down in Texas actually plans to start using RFID microchips to track the movements of their students….

Northside Independent School District plans to track students next year on two of its campuses using technology implanted in their student identification cards in a trial that could eventually include all 112 of its schools and all of its nearly 100,000 students.

District officials said the Radio Frequency Identification System (RFID) tags would improve safety by allowing them to locate students — and count them more accurately at the beginning of the school day to help offset cuts in state funding, which is partly based on attendance.

#9 Automated License Plate Readers

In a previous article, I quoted a Washington Post piece that talked about how automated license plate readers are being used to track the movements of a vehicle from the time that it enters Washington D.C. to the time that it leaves….

More than 250 cameras in the District and its suburbs scan license plates in real time, helping police pinpoint stolen cars and fleeing killers. But the program quietly has expanded beyond what anyone had imagined even a few years ago.

With virtually no public debate, police agencies have begun storing the information from the cameras, building databases that document the travels of millions of vehicles.

Nowhere is that more prevalent than in the District, which has more than one plate-reader per square mile, the highest concentration in the nation. Police in the Washington suburbs have dozens of them as well, and local agencies plan to add many more in coming months, creating a comprehensive dragnet that will include all the approaches into the District.

#10 Face Reading Software

Can computers tell what you are thinking just by looking at your face?

Don’t laugh.

Such technology is actually being actively developed. The following is from a recent NewScientist article….

IF THE computers we stare at all day could read our faces, they would probably know us better than anyone.

That vision may not be so far off. Researchers at the Massachusetts Institute of Technology’s Media Lab are developing software that can read the feelings behind facial expressions. In some cases, the computers outperform people. The software could lead to empathetic devices and is being used to evaluate and develop better adverts.

#11 Data Mining

The government is not the only one that is spying on you. The truth is that a whole host of very large corporations are gathering every shred of information about you that they possibly can and selling that information for profit. It is called “data mining“, and it is an industry that has absolutely exploded in recent years.

One very large corporation known as Acxiom actually compiles information on more than 190 million people in the U.S. alone….

The company fits into a category called database marketing. It started in 1969 as an outfit called Demographics Inc., using phone books and other notably low-tech tools, as well as one computer, to amass information on voters and consumers for direct marketing. Almost 40 years later, Acxiom has detailed entries for more than 190 million people and 126 million households in the U.S., and about 500 million active consumers worldwide. More than 23,000 servers in Conway, just north of Little Rock, collect and analyze more than 50 trillion data ‘transactions’ a year.

#12 Street Lights Spying On Us?

Did you ever consider that street lights could be spying on you?

Well, it is actually happening. New high tech street lights that can actually watch what you do and listen to what you are saying are being installed in some major U.S. cities. The following is from a recent article by Paul Joseph Watson for Infowars.com….

Federally-funded high-tech street lights now being installed in American cities are not only set to aid the DHS in making “security announcements” and acting as talking surveillance cameras, they are also capable of “recording conversations,” bringing the potential privacy threat posed by ‘Intellistreets’ to a whole new level.

#13 Automated ISP Monitoring Of Your Internet Activity

As I have written about before, nothing you do on the Internet is private. However, Internet Service Providers and the entertainment industry are now taking Internet monitoring to a whole new level….

If you download potentially copyrighted software, videos or music, your Internet service provider (ISP) has been watching, and they’re coming for you.

Specifically, they’re coming for you on Thursday, July 12.

That’s the date when the nation’s largest ISPs will all voluntarily implement a new anti-piracy plan that will engage network operators in the largest digital spying scheme in history, and see some users’ bandwidth completely cut off until they sign an agreement saying they will not download copyrighted materials.

Word of the start date has been largely kept secret since ISPs announced their plans last June. The deal was brokered by the Recording Industry Association of America (RIAA) and the Motion Picture Association of America (MPAA), and coordinated by the Obama Administration.

Spying On Us Through Our Appliances

Could the government one day use your refrigerator to spy on you?

Don’t laugh.

That is exactly what CIA Director David Petraeus says is coming….

Petraeus says that web-connected gadgets will ‘transform’ the art of spying – allowing spies to monitor people automatically without planting bugs, breaking and entering or even donning a tuxedo to infiltrate a dinner party.

‘Transformational’ is an overused word, but I do believe it properly applies to these technologies,’ said Petraeus.

‘Particularly to their effect on clandestine tradecraft. Items of interest will be located, identified, monitored, and remotely controlled through technologies such as radio-frequency identification, sensor networks, tiny embedded servers, and energy harvesters – all connected to the next-generation internet using abundant, low-cost, and high-power computing.’

Petraeus was speaking to a venture capital firm about new technologies which aim to add processors and web connections to previously ‘dumb’ home appliances such as fridges, ovens and lighting systems.

For many more ways that Big Brother is spying on you, please see these articles….

Every Breath You Take, Every Move You Make – 14 New Ways That The Government Is Watching You

30 Signs That The United States Of America Is Being Turned Into A Giant Prison

The things that I have written about above are just the things that they admit to.

There are also many “black box technologies” being developed out there that the public does not even know about yet.

So how far will all of this go?

Has Big Brother already gone way too far?

Please feel free to post a comment with your opinion below….

Source: The American Dream

Everything You Wanted to Know About Data Mining

Everything You Wanted to Know About Data Mining

A guide to what data mining is, how it works, and why it’s important.

Big data is everywhere we look these days. Businesses are falling all over themselves to hire ‘data scientists,’ privacy advocates are concerned about personal data and control, and technologists and entrepreneurs scramble to find new ways to collect, control and monetize data. We know that data is powerful and valuable. But how?

This article is an attempt to explain how data mining works and why you should care about it. Because when we think about how our data is being used, it is crucial to understand the power of this practice. Without data mining, when you give someone access to information about you, all they know is what you have told them. With data mining, they know what you have told them and can guess a great deal more. Put another way, data mining allows companies and governments to use the information you provide to reveal more than you think.

Data mining allows companies and governments to use the information you provide to reveal more than you think.

To most of us data mining goes something like this: tons of data is collected, then quant wizards work their arcane magic, and then they know all of this amazing stuff. But, how? And what types of things can they know? Here is the truth: despite the fact that the specific technical functioning of data mining algorithms is quite complex — they are a black box unless you are a professional statistician or computer scientist — the uses and capabilities of these approaches are, in fact, quite comprehensible and intuitive.

For the most part, data mining tells us about very large and complex data sets, the kinds of information that would be readily apparent about small and simple things. For example, it can tell us that “one of these things is not like the other” a la Sesame Street or it can show us categories and then sort things into pre-determined categories. But what’s simple with 5 datapoints is not so simple with 5 billion datapoints.

And these days, there’s always more data. We gather far more of it then we can digest. Nearly every transaction or interaction leaves a data signature that someone somewhere is capturing and storing. This is, of course, true on the internet; but, ubiquitous computing and digitization has made it increasingly true about our lives away from our computers (do we still have those?). The sheer scale of this data has far exceeded human sense-making capabilities. At these scales patterns are often too subtle and relationships too complex or multi-dimensional to observe by simply looking at the data. Data mining is a means of automating part this process to detect interpretable patterns; it helps us see the forest without getting lost in the trees.

Discovering information from data takes two major forms: description and prediction. At the scale we are talking about, it is hard to know what the data shows. Data mining is used to simplify and summarize the data in a manner that we can understand, and then allow us to infer things about specific cases based on the patterns we have observed. Of course, specific applications of data mining methods are limited by the data and computing power available, and are tailored for specific needs and goals. However, there are several main types of pattern detection that are commonly used. These general forms illustrate what data mining can do.

Anomaly detection : in a large data set it is possible to get a picture of what the data tends to look like in a typical case. Statistics can be used to determine if something is notably different from this pattern. For instance, the IRS could model typical tax returns and use anomaly detection to identify specific returns that differ from this for review and audit.

Association learning: This is the type of data mining that drives the Amazon recommendation system. For instance, this might reveal that customers who bought a cocktail shaker and a cocktail recipe book also often buy martini glasses. These types of findings are often used for targeting coupons/deals or advertising. Similarly, this form of data mining (albeit a quite complex version) is behind Netflix movie recommendations.

Cluster detection: one type of pattern recognition that is particularly useful is recognizing distinct clusters or sub-categories within the data. Without data mining, an analyst would have to look at the data and decide on a set of categories which they believe captures the relevant distinctions between apparent groups in the data. This would risk missing important categories. With data mining it is possible to let the data itself determine the groups. This is one of the black-box type of algorithms that are hard to understand. But in a simple example – again with purchasing behavior – we can imagine that the purchasing habits of different hobbyists would look quite different from each other: gardeners, fishermen and model airplane enthusiasts would all be quite distinct. Machine learning algorithms can detect all of the different subgroups within a dataset that differ significantly from each other.

Classification: If an existing structure is already known, data mining can be used to classify new cases into these pre-determined categories. Learning from a large set of pre-classified examples, algorithms can detect persistent systemic differences between items in each group and apply these rules to new classification problems. Spam filters are a great example of this – large sets of emails that have been identified as spam have enabled filters to notice differences in word usage between legitimate and spam messages, and classify incoming messages according to these rules with a high degree of accuracy.

Regression: Data mining can be used to construct predictive models based on many variables. Facebook, for example, might be interested in predicting future engagement for a user based on past behavior. Factors like the amount of personal information shared, number of photos tagged, friend requests initiated or accepted, comments, likes etc. could all be included in such a model. Over time, this model could be honed to include or weight things differently as Facebook compares how the predictions differ from observed behavior. Ultimately these findings could be used to guide design in order to encourage more of the behaviors that seem to lead to increased engagement over time.

The patterns detected and structures revealed by the descriptive data mining are then often applied to predict other aspects of the data. Amazon offers a useful example of how descriptive findings are used for prediction. The (hypothetical) association between cocktail shaker and martini glass purchases, for instance, could be used, along with many other similar associations, as part of a model predicting the likelihood that a particular user will make a particular purchase. This model could match all such associations with a user’s purchasing history, and predict which products they are most likely to purchase. Amazon can then serve ads based on what that user is most likely to buy.

Data mining, in this way, can grant immense inferential power. If an algorithm can correctly classify a case into known category based on limited data, it is possible to estimate a wide-range of other information about that case based on the properties of all the other cases in that category. This may sound dry, but it is how most successful Internet companies make their money and from where they draw their power.

SOURCE:
http://www.theatlantic.com/technology/archive/2012/04/everything-you-wanted-to-know-about-data-mining-but-were-afraid-to-ask/255388/

By: Alexander Furnas April 3, 2012