Software to Predict "Troubled Youths"
MOSAIC-2000 is being developed for a national field test in 25 schools. Though
the field test is not complete, here's an informal summary of where we are:
What is MOSAIC-2000?
MOSAIC is a computer-assisted method for conducting high-stakes evaluations of
persons who might act violently (such as when students make threats to harm
others). MOSAIC systems have been in use for a decade by many federal and
state law enforcement agencies and major universities. The same assessment
strategies are now being made available to schools through MOSAIC-2000,
co-developed by the Los Angeles County District Attorney's Office, the Bureau of
Alcohol, Tobacco & Firearms, the Los Angeles County Office of Education, and
Gavin de Becker, Incorporated.
Gavin de Becker Incorporated designed the MOSAIC-2 system used for screening
threats to Justices of the US Supreme Court (and also used by eleven states as part
of protecting governors). Another system, MOSAIC-20, is used by police
departments all over the country for identifying which domestic violence
perpetrators are most likely to escalate their violence. It has been credited with
major improvements in the safety of domestic violence victims. (See Los Angeles
Times article, October 21st, 1996.)
Two other MOSAIC systems were co-developed with the U.S. Marshals Service,
including MOSAIC-3, used for screening threats to Federal judges and
The MOSAIC method has been widely applied for a decade, already having
screened tens of thousands of cases. The method is now being made available for
the first time to help schools evaluate situations that might escalate to violence.
Every principal in America already has a method for evaluating students who make
threats - it's just that most of those methods are unorganized, idiosyncratic, and
cannot be expressed or documented. MOSAIC-2000 is intended to bring
uniformity, structure, expert opinion, and validity to high stakes evaluations.
Public pressure on schools has led many communities to respond to the fear of
Columbine-style incidents by improving real estate instead of improving children
and education. The buildings have been enhanced, to be sure (new locks, security
alarms, cameras, etc), and because of that, some parents may chose to conclude
that school-safety is no longer a problem. But Columbine had cameras - cameras
that recorded the tragedy, but contributed nothing to preventing it.
Gavin de Becker's book, Protecting the Gift is currently the best selling parenting
book in America. In both this and his previous bestseller (The Gift of Fear), he
notes that many parents are inadequately involved with their children's schools.
Parents are fast to blame schools for anything wrong, but slow to participate in
making things right - fast to develop outlandish expectations about what principals
can do, but slow to invest schools with the resources needed to do their jobs well.
As a society, we don't pay school professionals enough, we don't praise them
enough, we don't prepare them enough - and then we expect magical abilities in all
fields. Today, principals are expected to be threat-assessment experts able to
instantly make fair and accurate predictions -able to identify which students might
act out violently- yet we haven't given them the tools to help them do it.
MOSAIC-2000 is one of those tools.
How Does it Work?
MOSAIC is not a computer program, but rather an evaluation method that is
computer-assisted. It is a way of breaking down a case to its elements, then
organizing and identifying the most important factors. MOSAIC suggests to the
user which questions are most likely to produce a quality evaluation. Once a case
is broken down to its elements, it can be instantly compared to others where the
outcome is known. The case can also be weighed against the opinions of experts in
many relevant fields.
Imagine a student has made a threat which alarms others, and it falls to you to
evaluate the situation and the student. In a perfect world, you'd be able to instantly
confer with all the leading experts in threat-assessment, law enforcement,
psychology, and behavioral science, and ask:
What is most important for me to learn about this situation?
What information will most inform my evaluation?
How can I organize the information I gather to weigh it all most effectively?
What factors and warning signs are most relevant to future behavior?
How can I express and document my conclusions?
MOSAIC-2000 provides the guidance of leading experts, presented in a step by
step form that lets the evaluation process begin immediately.
Specifically, MOSAIC-2000 presents a series of questions, along with a range of
possible answers. Users are offered extensive, in depth explanations of what
factors must be present in order to stimulate selection of a given answer. Different
answers have different weights, so they can be weighed against each other, against
past cases where the outcome is known, and against expert opinion. Each
evaluation is rated on a scale of one to ten, with ten representing cases most similar
to those that have escalated, and thus, most in need of intervention
The system produces an automatic report that documents and presents exactly
what questions were asked, how they were answered, and what comments the
user chose to add along the way. Both the rating and the process help inform the
school administrator's evaluation of the situation.
Each evaluator brings his or her own intuition and experience, and MOSAIC
assures that different evaluators approach their cases from a shared foundation.
Can MOSAIC Label Kids?
MOSAIC-2000 cannot label anyone as anything. People unfamiliar with the
method may jump to the worry that principals will use it to unfairly label kids, but
the objective process resists bias. MOSAIC-2000 is vastly more likely give a low
rating in a situation to which people are over-reacting - than to give a high rating in
a situation people are not concerned about.
MOSAIC-2000 seeks to identify those students most in need of the interventions
and resources that are available, and in the school setting, all appropriate
intervention is good.
Is MOSAIC-2000 A Computerized Checklist Of Warning Signs?
The use of checklists in high-stakes evaluations is the antithesis of the MOSAIC
method. Checklists reduce to Yes/No answers elements of behavior and
circumstance that do not lend themselves to being limited to just two answers.
With Yes/No limitations, evaluators check off answers to global questions and
decide which answer they'll give - consciously or unconsciously influenced by
what they feel is the "right" overall result for the evaluation. Yes/No checklists do
not work for assessments of human behavior.
Imagine being asked to describe a movie you saw last night, but being required to
answer by saying either:
BEST MOVIE I EVER SAW, or
WORST MOVIE I EVER SAW
Those answers wouldn't produce a very fair appraisal of your opinion about the
movie - and situations involving human beings are far more complex than movies.
A range of answers is far more likely to stimulate accuracy fairness, and
completeness. For example, if asking about firearms, a Yes/No question could not
stimulate as fair or complete an exploration as a range:
__No known possession of a firearm
__Friends known to have ready access to a firearm
__There are firearms in the home
__There are firearms in a home frequented by the student
__The student owns his own firearm
__The student recently acquired a firearm
A range not only encourages accurate and complete evaluations; it also recognizes
that different answers have different value. To use a checklist that gives the same
weight to all answers would be like evaluating a passenger jet and giving the same
weight to the in-flight magazine as to the landing gear.
Here are some fast answers to frequently asked questions:
What is new about MOSAIC-2000?
For the first time, schools at the elementary, middle, and high school levels will
have access to technology and methods that have long been used for many of our
nation's highest stakes assessments.
Is MOSAIC-2000 for use on all students?
School administrators would use MOSAIC-2000 only in situations that reach a
certain threshold (e.g., a student makes a threat, brings a weapon to school,
teachers or students are concerned a student might act out violently).
Does MOSAIC-2000 invade the privacy of students?
The information gathered for each evaluation is held at the school only, and is
never communicated over the Internet. MOSAIC is a stand-alone system, secure
at each school, with no central combining of cases. The system isn't a "Big
Brother" approach. MOSAIC-2000 merely brings organization and expert opinion
to a process every principal already has.
Has the MOSAIC method been tested?
MOSAIC systems have been in daily use for a decade. Society faces many types
of high-stakes evaluations (threats to public officials, hazards to domestic violence
victims, workplace violence cases, etc). MOSAIC systems are used by the United
States Supreme Court, the Federal Reserve Board, the Central Intelligence
Agency, Governors of eleven states, and many others.
Has MOSAIC been used in the school setting?
MOSAIC systems have been in use by schools for many years, including Yale
University, Boston University, Penn State, and throughout the University of
What will MOSAIC do for school administrators?
MOSAIC-2000 will help schools identify students most in need of intervention,
and in the school setting, all appropriate interventions are favorable (i.e., no
adverse results come to a student when a principal concludes that there may be a
risk of violence). A student in need of intervention benefits, and of course the
student population benefits in terms of enhanced safety.
How does it work?
MOSAIC-2000 guides school administrators through the questions that most
inform an evaluation, then provides a range of possible answers. Answers that
have been developed and weighted by experts are then calculated by the system to
produce an overall rating, expressed on a scale of one to ten.
Does the computer make decisions?
MOSAIC-2000 does not make decisions; it is a tool that helps school
administrators by identifying the areas of inquiry that experts feel will produce the
best evaluation of the situation.
Does the computer tell schools what to do?
MOSAIC-2000 does not suggest any specific case-management actions. It offers
diagnosis of a situation, more triage than treatment plan.
Can schools use MOSAIC-2000 in cases where a student is already
In cases where students have been expelled as a result of safety concerns, when
they are considered for re-enrollment, some schools may use MOSAIC-2000 to
help evaluate if the risk has lessened.
Can the system brand a student as dangerous?
Most often, MOSAIC-2000 will help establish that a student does not pose an
elevated risk of violence.
Is the system biased?
MOSAIC-2000 brings a shared language to assessments, so that all users
objectively apply similar methods when exploring these situations. This ensures
that critical situations are evaluated in a fair, objective, consistent, and
What will MOSAIC-2000 cost?
The 25 schools participating in the field test will pay nothing for the system. The
cost for the final version of MOSAIC-2000 (due in February, 2000) will be
determined by the M-2000 Advisory Board. It is likely to be a small monthly fee
for each school.
How does MOSAIC express evaluation results?
At a keystroke, the system automatically produces reports in regular English. They
include the questions that were asked, the answers that were selected, what
comments were added by the evaluator, the value of the information that was
evaluated, and the overall rating.
What are the technical requirements?
The system operates on entirely standard and traditional hardware, including an
IBM-PC compatible 486 computer. It uses very little disk space.
How was the expert opinion within MOSAIC-2000 identified and captured?
In order to identify what questions experts feel are most important to ask in one of
these situations, three groups were established:
1.A pool of experts and practitioners - 125 experts;
2.The M-2000 Advisory Board - 57 experts;
3.The M-2000 Development Team - 17 experts;
Concern that a student might act out violently can be triggered in any of several
a student makes a threat;
alarming writings are observed;
a student brings a firearm to school;
a student gets into trouble with police;
a teacher, counselor, psychologist, parent, or fellow student becomes
concerned and makes a report
Each of these categories was represented at the Expert and Practitioner Pool
(EPP) meeting at the Skirball Cultural Center in Los Angeles. The 125 members of
the EPP included experts in threat-assessment, law enforcement, education,
psychology, behavioral science, the judiciary, and even some high school students.
The EPP heard presentations by Gavin de Becker and other experts:
Dr. James McGee, an advisor to the Baltimore State Police who recently headed
up a comprehensive study of 16 students who have committed multiple shootings
Paul Mones, author of When A Child Kills, a seminal work in the field of violence
Gregory Gibson, author of GONE BOY, whose son was killed during a mass
shooting by another student at his school;
Barbara Nelson, Dean of UCLA's School of Public Policy and Social Research;
Deputy District Attorney Scott Gordon, a founding member of the Stalking and
Threat Assessment Team of the Los Angeles. County District Attorney's Office;
Robert Martin, former Commanding Officer of LAPD's Detective Headquarters
Division, and founder of the Department's Threat Management Unit;
Gil Garcetti, District Attorney for the County of Los Angeles;
After these presentations, the 125 participants broke into groups of twenty, tasked
to identify the questions they'd ask if faced with a threat-assessment challenge in a
school setting. Following extensive discussions, each group was asked to rank
proposed questions in order of importance.
The entire EPP then reconvened for a case-management workshop, where the
very questions they'd just identified were applied.
Months later, after the questions of all groups were combined, analyzed, and
ranked, they were presented to the MOSAIC-2000 Advisory Board (57 experts
representing the fields of threat-assessment, education, law enforcement, the
judiciary, school administration, and behavioral sciences). The Advisory Board
analyzed the questions for clarity, applicability, relevance-to-outcome,
answerability, and fairness. They then refined the list and developed a range of
possible answers for each question.
The Advisory Board's results were combined and further analyzed against what is
known about past cases that escalated to violence. The questions and answers
were converted into Artificial Intuition format and entered into a MOSAIC
Next, the MOSAIC-2000 Development Team (16 people) gathered at the UCLA
Conference Center for an intensive two-day session. The group included senior
representatives from the Los Angeles County Office of Education (coordinating
programs for 1700 schools), the Bureau of Alcohol, Tobacco & Firearms, the Los
Angeles County District Attorney's Office, the Cook County District Attorney's
Office, as well as Dr. James McGee, Gavin de Becker, Robert Martin, Jennifer
Mitchell, (co-developer of the Child Lures safety programs currently used in more
than 1000 schools), and several threat-assessment experts.
During two long days and nights, the Development Team worked with the
preliminary MOSAIC program, drafting language and further refining questions
and answers. In a daylong round-table session, Development Team members
reviewed all questions, omitted those that did not meet the group's criteria, and
added some that emerged during the discussions.
The draft version of MOSAIC-2000 is being tested against known cases that
escalated to violence, as well as being used in the field by schools around the
nation. The Advisory Board and Development Team will take the results from
these tests, as well as comments from schools using the system, and implement