ADVISE
From SourceWatch
ADVISE (Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement) is a "research and development program within the Department of Homeland Security (DHS), part of its three-year-old 'Threat and Vulnerability, Testing and Assessment' portfolio. The TVTA received nearly $50 million in federal funding this year," Mark Clayton reported in the February 9, 2006, Christian Science Monitor.
ADVISE is "at the core" of a "massive computer system" "being developed by the US government ... that can collect huge amounts of data and, by linking far-flung information from blogs and e-mail to government records and intelligence reports, search for patterns of terrorist activity," Clayton wrote.
"The system - parts of which are operational, parts of which are still under development - is already credited with helping to foil some plots. It is the federal government's latest attempt to use broad data-collection and powerful analysis in the fight against terrorism. But," Clayton wrote, "by delving deeply into the digital minutiae of American life, the program is also raising concerns that the government is intruding too deeply into citizens' privacy."
Lee Tien, a "staff attorney with the Electronic Frontier Foundation, said that programs like ADVISE "are about connecting" the dots of the "traces" we leave behind everywhere, "analyzing and aggregating them - in a way that we haven't thought about, ... as we live our lives and make little choices, like buying groceries, buying on Amazon, Googling," Clayton said.
"A major part of ADVISE involves data-mining - or 'dataveillance,' as some call it. It means sifting through data to look for patterns. If a supermarket finds that customers who buy cider also tend to buy fresh-baked bread, it might group the two together. To prevent fraud, credit-card issuers use data-mining to look for patterns of suspicious activity.
"What sets ADVISE apart is its scope," Clayton reported. "It would collect a vast array of corporate and public online information - from financial records to CNN news stories - and cross-reference it against US intelligence and law-enforcement records. The system would then store it as 'entities' - linked data about people, places, things, organizations, and events, according to a report summarizing a 2004 DHS conference in Alexandria, Va. The storage requirements alone are huge - enough to retain information about 1 quadrillion entities, the report estimated. If each entity were a penny, they would collectively form a cube a half-mile high - roughly double the height of the Empire State Building.
"But ADVISE and related DHS technologies aim to do much more, according to Joseph Kielman, manager of the TVTA portfolio. The key is not merely to identify terrorists, or sift for key words, but to identify critical patterns in data that illumine their motives and intentions, he wrote in a presentation at a November conference in Richland, Wash.," Clayton wrote.
Contents[hide]
|
How ADVISE works
According to the "Data Sciences Technology for Homeland Security Information Management and Knowledge Discovery" Report of the DHS Workshop on Data Sciences conducted September 22-23, 2004, which was jointly released in January 2005 by Sandia National Laboratories and Lawrence Livermore National Laboratory:
- ADVISE is "a system that is under 'spiral' development (meaning that it is being deployed simultaneously with development) and will provide a common platform that supports scalable knowledge management across multiple missions."
- The system "includes tools for ingesting and canonicalizing massive quantities of information from many different sources. ... Some of the data comes from other databases ... Other data comes from free-form text document sources that must be processed to discover the entities and their relationships. Automatic tools for event extraction are used for some reports but are not yet very good."
- "At ADVISE’s core, semantic graphs are used to organize the data entities and their relationships. ... A semantic graph organizes relational data by using nodes to represent entities and edges to connect related entities. Hidden relationships in the data are uncovered by examining the structure and properties of the semantic graph. Privacy and support policies are enforced by a security infrastructure. Several interfaces for browsing, querying, and viewing the results of queries are under development, including IN-SPIRE and Starlight, from the DHS National Visualization and Analytics Center (NVAC). The key to fusing disparate data from many sources in ADVISE is the exploitation of 'precomputed' relationship information by storing the data in a semantic graph. All nodes are related by the links between them on the graph."
-
- For example, "a simple semantic graph" links "people (black nodes), workplaces (red nodes), and towns (blue nodes). The different link (or edge) types indicate different relationship types. For example, the fact that Person 13 and Person 15 have a green link between them indicates that they are friends with one another, while the orange link from Workplace 19 to Town 22 indicates that Workplace 19 is located in Town 22. In this example, the links are all bidirectional, but directed links can also be used."
- "Confidences (or uncertainties) are attributes of both the nodes and edges. Studying such graphs can help in understanding the relationships between entities (e.g., what’s the shortest path between Persons 16 and 26?) and in making intelligent hypotheses (e.g., Persons 15 and 14 are linked by a common workplace and a common friend, so we may hypothesize that there is a good chance that they should also be connected by a 'Friends with' link)."
- "Several systems are built on top of the ADVISE architecture ... including the Threat Vulnerability Information System (TVIS) for the Information Analysis (IA) organization, the Regional Threat Analysis System (RTAS) for Border and Transportation Security (BTS), and the Biodefense Knowledge Center (BKC) for the National Biodefense Analysis and Countermeasures Center (NBACC)."
Resources and articles
Related SourceWatch articles
- Bush administration warrantless wiretapping
- domestic spying
- George W. Bush's domestic spying
- Government Information Awareness
- Information Awareness Office (IAO) at Defense Advanced Research Projects Agency (DARPA)
- Intelligence Community
- internet surveillance
- National Security Branch Analysis Center
- Novel Intelligence from Massive Data (NIMD)
- Office of Net Assessment
- System to Assess Risk (STAR)
- TALON
- Topsail
- Total Information Awareness (TIA)
External articles
2002
- John Markoff, "Threats and Responses: Intelligence; Pentagon Plans a Computer System That Would Peek at Personal Data of Americans," New York Times, November 9, 2002: "'This could be the perfect storm for civil liberties in America,' said Marc Rotenberg, director of the Electronic Privacy Information Center in Washington 'The vehicle is the Homeland Security Act, the technology is Darpa and the agency is the F.B.I. The outcome is a system of national surveillance of the American public.'"
2003
- Farhad Manjoo, "Total Information Awareness: Down, but not out. Congress may have put the brakes on the most ambitious government surveillance program ever. But for citizens worried about their privacy, TIA still means trouble," Salon, January 28, 2003: But while Congress asks for reports, TIA is already steaming forward. According to people with knowledge of the program, TIA has now advanced to the point where it's much more than a mere 'research project.' There is a working prototype of the system, and federal agencies outside the Defense Department have expressed interest in it."
- Declan McCullagh, "Pentagon spy database funding revealed," C|Net News, February 27, 2003. Includes information on contractors and programs.
- "Total/Terrorism Information Awareness (TIA): Is It Truly Dead? EFF: It's Too Early to Tell," Electronic Privacy Information Center, October 2003.
2004
- George Cahlink, "Security agency doubled procurement spending in four years," GovExec.com, June 1, 2004.
2005
- Ted Rall, "The Return of Total Information Awareness - Bush Asserts Dictatorial 'Inherent' Powers," Ted Rall (Common Dreams), December 28, 2005.
2006
- William A. Arkin, "NSA Expands, Centralizes Domestic Spying," Washington Post, January 30, 2006.
- Mark Clayton, "US plans massive data sweep. Little-known data-collection system could troll news, blogs, even e-mails. Will it go too far?" Christian Science Monitor, February 9, 2006.
- rktect, "Breaking- White House Gives Details on Surveillance," Daily Kos, February 9, 2006.
2007
- Ryan Singel, "DHS Data Mining Program Suspended After Evading Privacy Review, Audit Finds," WIRED Blog, August 20, 2007.
External resources
U.S. Government Documents
- "Privacy: Total Information Awareness Programs and Related Information Access, Collection, and Protection Laws" prepared by Gina Marie Stevens, Legislative Attorney, American Law Division, Congressional Research Service, March 21, 2003 (updated).
- "Data Sciences Technology for Homeland Security Information Management and Knowledge Discovery," Report of the DHS Workshop on Data Sciences, September 22-23, 2004, Jointly released by Sandia National Laboratories and Lawrence Livermore National Laboratory (unlimited release; 41-page pdf printed January 2005). "Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement" is on page 7 (pdf page 17).
- Robert Burleson, "Image to Insight in a Counterterrorism Context," Lawrence Livermore National Laboratory, 2005 (23-page pdf).
- 98–563PS 2005. "An Overview of the Federal R&D Budget for Fiscal Year 2006," Hearing Before the Committee on Science, U.S. House of Representatives, 109th Congress, First Session, February 16, 2005: "Created the knowledge management architecture, known as ADVISE (Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement) to integrate the various information analysis and synthesis, visualization, and knowledge discovery component capabilities. ADVISE will incorporate a comprehensive encyclopedia of chemical, biological, radiological, nuclear and explosive (CBRNE) threat and effects data. Pilot ADVISE systems for the BTS Directorate will be installed in FY 2005. Update the initial TVIS system at the Biodefense Knowledge Center with the enhanced ADVISE capability."
- DHS Science and Technology Directorate FY 2006 Budget Brief, March 1, 2005 (49-page pdf).
- Fiscal Year 2006 Congressional Justification, Department of Homeland Security, Science and Technology Directorate, undated (156-page pdf). See pdf page 93:
-
- FY 2005 Plan: "Create a knowledge management architecture, known as ADVISE (Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement) to integrate the various information analysis and synthesis, visualization, and knowledge discovery component capabilities. ADVISE will also create and incorporate a comprehensive encyclopedia of CBRNE threat and effects data." and "Pilot ADVISE systems for the BTS Directorate will be installed. The ADVISE system comprises computer hardware, networking hardware, and a suite of analytical and visualization tools. A pilot system was installed at IAIP in FY 2004, that is, it was connected to their internal networks and made available for use by all IA analysts. The same process will be used for BTS. Replace the initial TVIS system at the Biodefense Knowledge Center with the enhanced ADVISE capability."
- FY 2006 Plan: "A National Homeland Security Support System (NH3S) will be created using the ADVISE architecture and providing quantitative risk analysis and decision support capabilities."