Intelligence Relational Database?

BY Herschel Smith
14 years ago

Captain Tim Hsia, U.S. Army, has written a thoughtful and provocative article at the Small Wars Journal Blog entitled Intelligence Collection and Sharing. Captain Hsia begins by cataloging the sundy reasons for the paucity of good local and regional intelligence and other information.

When a unit assumes battle space within Iraq the first thing that a commander receives from his higher headquarters is a plethora of maps detailing major avenues of approach, religious divides, key figures, demographics, key infrastructure, etc. However, much of the intelligence is outdated or watered down, and the source of this data is often unattributed. The source of this intelligence is necessary in order to winnow the chaff from the wheat. The intelligence received from higher headquarters can come from multiple sources, which oftentimes can be suspect and unverifiable. For example, is this intelligence derived from an Iraqi Army soldier, Iraqi policeman, neighborhood councils, street vendor, coalition signal assets, or from the previous military units who have operated within the current Area of Operations? Additionally, this initial trove of intelligence oftentimes provides just the basics and does not delve into more important issues that commanders need to know, such as the amount of money U.S. forces have spent developing the local infrastructure, the number of discontinued projects and reasons for their discontinuance, the quality of local leaders, and the attitudes of those leaders toward the U.S. military.Counterinsurgencies are not won by more soldiers, cutting edge technology, or more lethal weapon systems. Rather insurgents are defeated when the pacifying force fully understands the local citizenry, when the people identify with the pacifying force, and when there is an abundance of timely information which allows the pacifying force to apply their intelligence to operations that result in overturning and disrupting insurgent activity. Despite the great advances in the U.S. military’s ability to leverage technology to gain intelligence, it has been less successful in storing and synchronizing the historical data compiled during the past several years in its campaigns in the Middle East.

When a unit redeploys to the states they usually dump all of their electronic files to their counterparts in no systematic or coherent manner. This is the ideal situation, though if they are on a more limited timeline they might just pass off the most essential information. With units being continually shifted around Iraq with little or no notice to respond to increased violence in different areas, it has been almost impossible for units to properly pass off their intelligence to the next battle space owner or more importantly to future units that will operate in their sector. At best the problem a commander faces is an abundance of information that is improperly cataloged. Oftentimes however it is the worse case scenario in which commanders and diplomats encounter, a difficult situation where they have little to no information regarding a region or locale

As a sidebar note, if Captain Hsia is arguing for a small footprint model of counterinsurgency, he has history with which to contend (the problems in Operation Iraqi Freedom II and III caused by inadequate force size and force projection, as well as the problems in Afghanistan from the same). But continuing with the point of his article, he advocates a rather remarkable solution to this problem.

A way to remedy the chaotic state of intelligence management is to create a central intelligence collection platform that will allow any unit to upload operation summaries, economic analysis, tribal networks, environmental analysis, and graphical overlays into a central site that future commanders can access when they assume an assigned battle space. Currently all military units in Iraq and Afghanistan have access to a worldwide SIPR (secure Internet protocol router) network which allows them to access, view, and transmit secret information. Expanding this network to encompass a more centralized program of data sharing would not require any additional hardware. A fusion of geography and intelligence within a centralized network can ensure that commanders arrive at any location with the necessary intelligence derived from years of work by previous agencies and military units that have already provided a framework for understanding the enemy and the people in his assigned area. Commanders could then be spared the countless man hours recollecting data that has already been captured thru blood, sweat, and tears. A solution to the current intelligence blackhole would be to collect, store, and sift this data into a intel site organized in a manner replicating stock market data.

The appeal for such a system is strong. I started the category The Anbar Narrative in order to begin to capture snippets of perspectives and information regarding the campaign in Anbar in its various manifestations. I have exchanged perspectives with Lt. Col. Gian Gentile at the Small Wars Journal in which I have taken the position that the campaign in Iraq can be at least partially categorized as a counterinsurgency, while he has clearly said multiple times that it is only a civil war. Gentile’s perspective doesn’t affect the Anbar narrative, but it does go to show that the region and locale within Iraq can deeply affect the way a participant sees the situation. In Anbar and to the East around Ramadi, the Anbar narrative became all about tribes “flipping.” In Fallujah, tribal sheikhs were irrelevant and kinetic operations, gated communities and biometrics were the order of the day, and the muktars were much more important than the sheikhs. To the West in Haditha, fighters from Syria were problematic and earthern berms were necessary to isolate the area from outside influences. In Basra, the story is one of competing Shi’a gangs and thugs in a struggle for power. In Kirkuk there is a mixed Sunni, Shi’a and Kurdish population, and like Baghdad, civil war might be a better description of the circumstances. No single narrative fully explains the complexity of the Iraq experience. Combined with personalities, monies spent successfully and wasted, and other exigencies of the battle space, one can easily be overwhelmed by the data and information. And if we cannot even get our history right while the campaign is ongoing, how can we expect to pass on more particular and detailed information with precision?But the solution Captain Hsia profers is overwhelming as well. Note well what is being advocated. Graphical overlays, potential enemy information, (probably) census information, operational summaries, and on the list goes. All of this would have to be in a database, searchable on name (enemy), operational details (e.g., what were the locations and patrol sizes when IEDs were encountered, were distributed operations successful against enemy snipers, etc.). This means that such a database would have to be a relational database. This means that those who enter the information and access the information would have to be trained in this relational database (search query criteria, required entry information and formatting, etc.). This means that in order to deploy such a system across the armed forces in a consistent manner, a defense contractor will be at work for years to develop such a system and training for its operation would be implemented only over subsequent years in order to put it to use.We have noted with lament that the U.S. armed forces is at war, and the public has not yet mobilized for this war. Defense Secretary Gates is having trouble deciding to wean the Army off of a fifty year old cold war by re-deploying from the European theater, and the Afghanistan campaign suffers from a lack of force projection. And yet we are discussing millions and years more to deploy an integrated relational database for battle space intelligence!We like the idea, but we are realistic about it. It pays to profer the idea when its need is seen, but it also pays to point out the scope of the project. This scope is likely to kill the project before it ever gets off the ground.

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment

You are currently reading "Intelligence Relational Database?", entry #799 on The Captain's Journal.

This article is filed under the category(s) Department of Defense,Intelligence and was published November 27th, 2007 by Herschel Smith.

If you're interested in what else the The Captain's Journal has to say, you might try thumbing through the archives and visiting the main index, or; perhaps you would like to learn more about TCJ.

26th MEU (10)
Abu Muqawama (12)
ACOG (2)
ACOGs (1)
Afghan National Army (36)
Afghan National Police (17)
Afghanistan (704)
Afghanistan SOFA (4)
Agriculture in COIN (3)
AGW (1)
Air Force (36)
Air Power (9)
al Qaeda (83)
Ali al-Sistani (1)
America (21)
Ammunition (172)
Animals (116)
Ansar al Sunna (15)
Anthropology (3)
Antonin Scalia (1)
AR-15s (303)
Arghandab River Valley (1)
Arlington Cemetery (2)
Army (80)
Assassinations (2)
Assault Weapon Ban (27)
Australian Army (7)
Azerbaijan (4)
Backpacking (2)
Badr Organization (8)
Baitullah Mehsud (21)
Basra (17)
BATFE (121)
Battle of Bari Alai (2)
Battle of Wanat (18)
Battle Space Weight (3)
Bin Laden (7)
Blogroll (3)
Blogs (23)
Body Armor (21)
Books (3)
Border War (14)
Brady Campaign (1)
Britain (38)
British Army (35)
Camping (4)
Canada (3)
Castle Doctrine (1)
Caucasus (6)
Center For a New American Security (8)
Charity (3)
China (15)
Christmas (12)
CIA (29)
Civilian National Security Force (3)
Col. Gian Gentile (9)
Combat Outposts (3)
Combat Video (2)
Concerned Citizens (6)
Constabulary Actions (3)
Coolness Factor (3)
COP Keating (4)
Corruption in COIN (4)
Council on Foreign Relations (1)
Counterinsurgency (218)
DADT (2)
David Rohde (1)
Defense Contractors (2)
Department of Defense (190)
Department of Homeland Security (26)
Disaster Preparedness (4)
Distributed Operations (5)
Dogs (12)
Donald Trump (26)
Drone Campaign (3)
EFV (3)
Egypt (12)
El Salvador (1)
Embassy Security (1)
Enemy Spotters (1)
Expeditionary Warfare (17)
F-22 (2)
F-35 (1)
Fallujah (17)
Far East (3)
Fathers and Sons (2)
Favorite (1)
Fazlullah (3)
FBI (32)
Featured (186)
Federal Firearms Laws (18)
Financing the Taliban (2)
Firearms (1,440)
Football (1)
Force Projection (35)
Force Protection (4)
Force Transformation (1)
Foreign Policy (27)
Fukushima Reactor Accident (6)
Ganjgal (1)
Garmsir (1)
general (15)
General Amos (1)
General James Mattis (1)
General McChrystal (44)
General McKiernan (6)
General Rodriguez (3)
General Suleimani (9)
Georgia (19)
Google (1)
Gulbuddin Hekmatyar (1)
Gun Control (1,380)
Guns (1,943)
Guns In National Parks (3)
Haditha Roundup (10)
Haiti (2)
Haqqani Network (9)
Hate Mail (8)
Hekmatyar (1)
Heroism (4)
Hezbollah (12)
High Capacity Magazines (16)
High Value Targets (9)
Homecoming (1)
Homeland Security (1)
Horses (1)
Humor (37)
ICOS (1)
IEDs (7)
Immigration (94)
India (10)
Infantry (4)
Information Warfare (2)
Infrastructure (2)
Intelligence (23)
Intelligence Bulletin (6)
Iran (170)
Iraq (379)
Iraq SOFA (23)
Islamic Facism (64)
Islamists (95)
Israel (18)
Jaish al Mahdi (21)
Jalalabad (1)
Japan (2)
Jihadists (80)
John Nagl (5)
Joint Intelligence Centers (1)
JRTN (1)
Kabul (1)
Kajaki Dam (1)
Kamdesh (9)
Kandahar (12)
Karachi (7)
Kashmir (2)
Khost Province (1)
Khyber (11)
Knife Blogging (4)
Korea (4)
Korengal Valley (3)
Kunar Province (20)
Kurdistan (3)
Language in COIN (5)
Language in Statecraft (1)
Language Interpreters (2)
Lashkar-e-Taiba (2)
Law Enforcement (4)
Lawfare (7)
Leadership (6)
Lebanon (6)
Leon Panetta (2)
Let Them Fight (2)
Libya (14)
Lines of Effort (3)
Littoral Combat (8)
Logistics (50)
Long Guns (1)
Lt. Col. Allen West (2)
Marine Corps (271)
Marines in Bakwa (1)
Marines in Helmand (67)
Marjah (4)
Media (62)
Medical (110)
Memorial Day (6)
Mexican Cartels (36)
Mexico (52)
Michael Yon (6)
Micromanaging the Military (7)
Middle East (1)
Military Blogging (26)
Military Contractors (4)
Military Equipment (24)
Militia (6)
Mitt Romney (3)
Monetary Policy (1)
Moqtada al Sadr (2)
Mosul (4)
Mountains (25)
MRAPs (1)
Mullah Baradar (1)
Mullah Fazlullah (1)
Mullah Omar (3)
Musa Qala (4)
Music (24)
Muslim Brotherhood (6)
Nation Building (2)
National Internet IDs (1)
National Rifle Association (78)
NATO (15)
Navy (24)
Navy Corpsman (1)
NCOs (3)
News (1)
NGOs (2)
Nicholas Schmidle (2)
Now Zad (19)
NSA (3)
NSA James L. Jones (6)
Nuclear (58)
Nuristan (8)
Obama Administration (221)
Offshore Balancing (1)
Operation Alljah (7)
Operation Khanjar (14)
Ossetia (7)
Pakistan (165)
Paktya Province (1)
Palestine (5)
Patriotism (7)
Patrolling (1)
Pech River Valley (11)
Personal (67)
Petraeus (14)
Pictures (1)
Piracy (13)
Pistol (2)
Pizzagate (21)
Police (549)
Police in COIN (3)
Policy (15)
Politics (913)
Poppy (2)
PPEs (1)
Prisons in Counterinsurgency (12)
Project Gunrunner (20)
PRTs (1)
Qatar (1)
Quadrennial Defense Review (2)
Quds Force (13)
Quetta Shura (1)
RAND (3)
Recommended Reading (14)
Refueling Tanker (1)
Religion (248)
Religion and Insurgency (19)
Reuters (1)
Rick Perry (4)
Rifles (1)
Roads (4)
Rolling Stone (1)
Ron Paul (1)
ROTC (1)
Rules of Engagement (75)
Rumsfeld (1)
Russia (32)
Sabbatical (1)
Sangin (1)
Saqlawiyah (1)
Satellite Patrols (2)
Saudi Arabia (4)
Scenes from Iraq (1)
Second Amendment (442)
Second Amendment Quick Hits (2)
Secretary Gates (9)
Sharia Law (3)
Shura Ittehad-ul-Mujahiden (1)
SIIC (2)
Sirajuddin Haqqani (1)
Small Wars (72)
Snipers (9)
Sniveling Lackeys (2)
Soft Power (4)
Somalia (8)
Sons of Afghanistan (1)
Sons of Iraq (2)
Special Forces (28)
Squad Rushes (1)
State Department (23)
Statistics (1)
Sunni Insurgency (10)
Support to Infantry Ratio (1)
Supreme Court (30)
Survival (58)
SWAT Raids (55)
Syria (38)
Tactical Drills (2)
Tactical Gear (6)
Taliban (168)
Taliban Massing of Forces (4)
Tarmiyah (1)
TBI (1)
Technology (17)
Tehrik-i-Taliban (78)
Terrain in Combat (1)
Terrorism (95)
Thanksgiving (11)
The Anbar Narrative (23)
The Art of War (5)
The Fallen (1)
The Long War (20)
The Surge (3)
The Wounded (13)
Thomas Barnett (1)
Transnational Insurgencies (5)
Tribes (5)
TSA (22)
TSA Ineptitude (13)
TTPs (4)
U.S. Border Patrol (5)
U.S. Border Security (14)
U.S. Sovereignty (17)
UAVs (2)
UBL (4)
Ukraine (3)
Uncategorized (58)
Universal Background Check (3)
Unrestricted Warfare (4)
USS Iwo Jima (2)
USS San Antonio (1)
Uzbekistan (1)
V-22 Osprey (4)
Veterans (3)
Vietnam (1)
War & Warfare (318)
War & Warfare (40)
War Movies (4)
War Reporting (21)
Wardak Province (1)
Warriors (6)
Waziristan (1)
Weapons and Tactics (73)
West Point (1)
Winter Operations (1)
Women in Combat (21)
WTF? (1)
Yemen (1)

November 2021
October 2021
September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006

about · archives · contact · register

Copyright © 2006-2021 Captain's Journal. All rights reserved.