Think big. Start small. Preparing for machine learning.

The big idea

Imagine how powerful it will be if we can upload and analyze images of offenders committing crimes to automatically fill out a report for us?

Would it be faster and more accurate in the long run?

Where we’ve started

When our customers report crime, they fill in the usual stuff like height, build, and gender. There’s a text field asking for more information on appearance. Text fields allow people to write as little or as much as they have time. It is entirely dependant on how confident the reporter is at remembering incidents and translating them into well-written descriptions. 

Is it accurate? Is it fast? And more importantly, is it consistent?

Ask people to describe an object. Is it big or small? What color is it? Do you describe the same object using the same words all the time? Through the data, we can see that people describe the same things in many different ways. This is perfectly normal, but there is huge potential for improvements.

Don’t take my word for it, let’s take a look at the data.

Two immediate benefits of structured data: simplicity and speed.

Putting our user experience hat on

Writing up crime reports is time consuming. Writing them well even more so. We compared reports from experienced Auror users against newer users. The quality of reporting is quite different, with experienced Reporters consistently writing better reports than newer users.

Writing is a core skill for security personnel roles. What could we do to make this process better for non-security personnel?

Structured data

An early hypothesis was that structured data could be captured faster than writing and would help construct better quality narratives. Couple this with a great user interface we felt that we will capture better quality intel. We also felt that if there were a range of options presented to reporters then it could trigger a memory, for example ‘Beard, Shoes, Bags’ may trigger a memory that could be critical.

A huge benefit of structured data is that it’s easily indexable, which makes it incredibly searchable! This means more accurate matches along with a huge gain in speed.

Structuring the content

Through data analysis we could clearly see how we could break down appearance to 7 main categories:

  1. Head
  2. Upper body
  3. Lower body
  4. Feet
  5. Accessories
  6. Bags
  7. Scars or tattoos

We set up a spreadsheet of all the possible attributes that could match each category and we ended up with something like this:

Early designs

User experience is the phrase we use to describe how something performs for our customers. It’s a way of saying - we made this for you, not for us.

The early designs were well received by our users. The use of icons and labels on the buttons invited them to add content.

Assumptions tested

User experience testing

We conducted some in depth testing with a variety of users and customers. 

Here’s what we learned:

Updating the designs

We reduced the amount of categories to 5 which had a remarkable effect on reducing the cognitive load. The new appearance function suddenly became more powerful whilst being quick and easy to use. Nice!

So what does this have to do with machine learning?

Our customers want to report crime fast, and accurate data to solve and prevent crime. We need to be able to teach the machines what is good data and what isn’t. Remember when Facebook used to tag everyone’s faces to ask you if this is Person X or Person Y? That’s machine learning at scale. The user interface is the convergence of the two, so let’s make this the best, most simple process possible.

We recognize that when all the amazing capabilities of machine learning are unlocked, it will take some time for people to adapt, trust, and believe that the outcomes are better than people themselves.

We’re preparing to humanize the interface between machine learning to help report, solve, and prevent crime.

How we imagine it could work...

Our new reporting forms start with uploading photos first. We see this as the start point for asking the following:

Is this John Doe? – 97% match based on facial features and beanie. Confirm (Yes/No)

Head –Dark beard. Confirm (Yes/No)

Clothing – Black beanie with with NYC logo. Confirm (Yes/No)

Clothing – White jacket. Confirm (Yes/No)

Height – 182cm / 5’ 11”. Confirm (Yes/No)

Build – Muscular. Confirm (Yes/No)

We expect reporting to become faster, more accurate, and most importantly, a magical experience for our customers. We can’t wait.