Limited beside my pleasure to introduce.

Our similar speaker today
Professor Tim Farley from years.

In assistant professor.

Guard unit from.

Washington.

And frankly one of the things that
is actually very well known for

is the motion of.

Clinician and they're doing a procedure.

Should not use or you know a lot of today.

You also commercialized the work.

Or wanted to.

Do this he said.

Today we're going to hear from.

Surgeries surgeons mistakes.

That are.

So.

Wonderful.

Thank you so much data for the invitation
it's been wonderful to actually

see your campus and see the amazing
projects you guys are working on here and

fellow roboticist today and
clinicians yesterday.

So before I begin I just want to say
two things have a little disclosure

one is you'll see some information from
C.S.S. incorporated we created a company

in this area so there's a financial
concept of interest there and the other

thing is you might see so this is a talk
about surgical robotics so you'll see some

footage of surgery and whenever you
see that little element show up.

I learned my lesson to always do this
whenever you see this little symbol

that means the next slide is going
to have some footage of surgery and

so about maybe less than ten percent
of the population they get queasy

when they see footage of
basically blood and guts and

that's you if you see that sign just look
down check your email do something else.

I learned my lesson because once I
gave a talk to a math department and

they all got really queasy and
not good news.

All right so let's dive in with a little
bit of history about surgery so

I really think the field of surgery
began as a scientific discipline with

John Hunter who for the first time started
using the scientific method in surgery and

that's really where it took off this
is back in the day when for you to be

a surgeon the idea is you were a maverick
you actually created part part of your

surgical training was how to create all of
your own tools you make all of your own

prototypes and everything you put into a
patient was stuff that you made yourself.

The application of the scientific method
give a rise of proliferation of progress

and surgery with the rise of anesthesia
finally some people really think

surgery really only started once and

if he was getting used it was only
in the eight hundred fifty S.

as the advice did his work scientifically
proving that surgeons should wash their

hands to avoid spreading infection and

he was actually met with a lot of ridicule
at the time people thought it was.

A useless idea but scientific evidence
won out this is before germ theory and

things really picked up and
then ultimately.

The way surgery was trained is still
the way we do it today you have to go to

medical school you get a medical degree an
M.D. and then if you want to be a surgeon

you go into residency for maybe four or
five more years maybe ten if you're doing

neurosurgery and it looks kind of like
this you have an attending surgeon looking

over you you're doing a task maybe for the
first time and he lets you do it and or

she might let you do it until you make a
mistake and the idea is you see one being

done you do one until you make mistakes
and then you start teaching them and

it's called the house stayed in model as
it really is remained largely unchanged

until the last decade or so because the
field of surgery has been changing this

historical perspective where you had
open surgery where the surgeon was

directly interfacing with the patient in
open surgery the connection is one to one

you see in the left half picture
in the one nine hundred eighty S.

surgeons started working
with engineers to.

New tools you can see these long slender
tools here called laparoscopic tools

the surgeon got further
removed from the patient but

this is a major benefit to
patients instead of going.

Having an open procedure for
let's say a removal of an appendix you

have to stay in the hospital for
a few days afterwards

last topically you can go home within an
hour because it's so minimally invasive.

And then fast forward to today where
the Davinci surgical robot you can

see the geometric almost
exponential rate of adoption.

Of this technology where
the surgeon is actually completely

removed from the patient they
can actually be transcontinental

removed across an ocean and doing these
procedures it's been done before.

This sort of geometric rise
was actually due to your so

today we haven't actually seen
this penetration of robotics and

cardiovascular supervised bascule
are much much larger markets much bigger

opportunities but
notice was going on not only have you now

removed the surgeon even further from
the patient there's no force feedback.

There might be in different rooms
something else is also crept in

that traditional interface has been
replaced by a digital interface so

everything going from the master
console is fiber optic digital signals

about one hundred twenty eight
dimensions of data running in real time.

And so
sometimes you don't even need a patient so

increasingly what's happening is surgeons
are training in virtual reality or

in simulators so you may make your
mistakes they are sort of on humans and

get your skills quantified before
you actually go on a human

being what you see here is some work
on a virtual trainer or this is just

having real surgical tools doing a mock
trial of task and measuring your skill.

So the words are it's no longer blood and
guts it's bits and bytes it's a classic

quote from one of the sort of godfathers
of modern surgical robotics and that's

really the feel of computational surgery
which is really exciting because for

the first time the field of surgery is now
rigorously quantitative or it can be and

we can use methods from machine learning
from Systems Engineering Systems Science.

To grapple with problems in health care
longstanding problem with health care so

it's an exciting field because it's.

Potentially going to do some great things
in the field of surgery so let's look at

some problems facing health care like what
kind of stuff can we potentially help

with so in the turn of the millennium
this publication came out landmark public

publication from the Institute of Medicine
where the field of medicine of health care

as a whole took a really cold hard
look at itself using numbers and

not opinions what they found was that
the rate of I after Genesis that

is human error preventable
human error was rampant and

those medical errors led to approximately
one hundred thousand deaths a year

preventable medical errors and about
thirty billion dollars of annual costs.

But a third of that was surgery even
though surgery is nowhere near a third of

health care but they are responsible for

a significant portion of those errors
consider that the average American

has seven thirty in their lifetime this is
a risk that we're all kind of exposed to.

Consider also that well I'll back up
in a second when this came out this was

somewhat earth shattering because nobody
really expected these numbers were so

high so there are a major policy shift
in health care things like the wait

list The Checklist Manifesto came
out where surgeons learn from pilots

that one of the best things that pilots
have done to avoid trouble and landing and

takeoff is they'll just go through
checklists and they started incorporating

that but despite all these
massive shift in policy changes.

These error rates kept creeping up they
kept being publications that would

come out that actually show
the numbers keep increasing and

people would discredit those publications
because they weren't publishing very

reputable sources until last summer.

The British Medical Journal a very
reputable source dropped this bombshell

that in fact medical errors were so
grossly under reported

that they're actually one computer
correctly they account for

roughly the third leading cause
of death in the United States

just phenomenal which means that medical
errors are rampant we're talking about

you know a surgeon taking out the wrong
kidney a nurse giving the wrong dose of.

Pharmacist prescribing
the incorrect medication or

passing out the incorrect medication.

This is a pretty serious problem and
healthcare has been trying to grapple

with this problem for millennia the fact
that human error it's rampant and

surgery especially is very difficult
little in medicine so let's dive in.

Caution there's going to be some footage.

In two thousand and
thirteen Burke Meier and

his colleagues writing in The New
England Journal of Medicine hypothesized

that if you measure the technical
skills of a surgeon

you would correlate directly to patient
outcomes shown here is footage from

two of the surgeons in his
twenty baryonic surgery study

these are practicing surgeons this
is their full time job their.

Procedure and

he asked that they voluntarily take
one video of their surgery and

submit it to their peers for and on of
my assessment of their technical skills.

So you can decide who you'd prefer
to do your surgery this is a top and

bottom quartile performer.

What they found was that as a technical
skill rating increased along this axis

the risk adjusted complication
rate decreased so much so

that if you were in the bottom quartal of
performers you had a five fold increase in

death a three fold increase in blood loss
and a three fold increase in hospital

hospital stay nobody anticipated that
technical skills would matter that much

and again this is for practicing surgeon
this is not trainees is not residence.

And again that was from one video IT
project outcomes surgical outcomes of that

surgeon's entire year of practice
which is pretty interesting so

how hard can this problem be measuring
technical skill I want to show you two

surgeons these are actually residents but
they are doing surgery and I want to

ask you as quickly as you can to tell me
who you would prefer to do your surgery.

Anybody want the person on the right
it's a notice you don't have to wait

ten minutes it takes this person
actually finish this task to tell

within seconds you can tell you
don't need a medical degree to tell

that one of these individuals looks more
skilled than the other interesting well if

can we quantify that can we can we do
something a little more rigorous and

scientific instead of just a gut feeling.

So the video that you just saw we're
actually capturing the two motion for

the surgical tools is what they
look at plotted out in time

you can see a lot more sort of entropy and
chaos than one versus the other but

this is still pretty qualitative this
is still not a not actual number.

And you can look at things like
the force profile of grasping so

on the top here you have the left and
right hand grasping of and somebody we

called an expert will get to that and
it's not that the expert has less force or

more force than the novice It seems
that they have a lot more consistent

force behavior and
the novice just has a lot of variability.

So anyway these are sort of nice
observations but they tend not to

scale very well to let's say looking at
one hundred surgeons maybe comparing two

very extreme cases we need to do
something a little bit more rigorous.

And so for this we collected some
massive databases there called Blues and

Red Dragon Age I won't get into those
details but we have over fifty hours of

these procedures recorded where
we have to emotion and video and

the idea is can we quantitatively measure
the skill and do better than a stopwatch

So it turns out that measuring how long it
takes somebody to do a task as very hard

to compete with a good surgeon can
do stuff more quickly period and

if our skill metric our machine learning
amazing skill metric gives you nothing

better than the correlation to a stopwatch
it's useless you can get a stopwatch for

ten bucks and you're done and
so what we try to do is

have a quantitative measure of skill
that's better than a stopwatch So

in ten seconds can you tell me how good
skill is just like you guys could and

that's been kind of under explored and
so a little bit of footage coming up so

what you see here is our early
work a machine learning.

This is just a hit or Marcus model.

Being trained in real time on the on
the live survey that you see running here.

Evaluating in real time pre-trained
on the surgery that you see here and

this allowed us to actually
quantify the skill of the surgeon.

I won't get too detailed
into the method it's just

pretty much boilerplate
hit a mark off models.

And I apply this in my Ph D.

work and I was actually disappointed
because it didn't work very well.

Like yes you could do surgeons but
when you take the first ten or twenty or

thirty seconds it actually work and
I'm really confused I was like well

why would that be so
I started questioning the assumptions

that the surgeons gave us they told us
who the experts were they said hey these

guys are the best in our department
this guy is the maverick use him and

I trusted them I said OK I'll use this
as my ground truth labeling and I start

to get really suspicious of that so what
I did is I took fifty six of the videos.

I took with what they use
as a gold standard so

this is an example of a tool
called that subjective structured

assessment technical skills is just a set
of Likert scale domains so want to kill

of want to five How well do you use both
hands on a scale of one to five with your

tool motion efficiency how confident are
you that kind of stuff and this has been

validated in the surgical literature
this is a tool that they that actually

faculty surgeons will use to evaluate
their residence before they accredited.

Anyway I sat down my surgeons by
collaborating surgeons and I put

in front of them fifty six videos and
I asked them to evaluate these surgeons as

if they were evaluating their own
students to be as objective as possible.

What I didn't tell them is that the fifty
six videos are looking at videos of

all the people they told me with
a top experts and they proceeded to

disqualify over half as non competent
surgeons It was interesting.

I do I first I was kind of shocked I
didn't really know what to do with that.

He would define competence so
you had to have both a score above three.

So the average score for motion
quality and about three for by Man U.

ality and there were very few
that actually ended up in both.

So we were suspicious of those results but

when we finally used a true sort
of ground truth that wasn't noisy

then the skill evaluation really really
worked so the machine learning problem is

very ill posed if you have two sets
of data that are very noisy So

let's say you have labeled data and
you sort of cross contaminate a bunch

of those labels any machine learning
algorithm is not going to work very well.

So when we got a better ground truth it
worked and we could actually identify

skill with and I think it took forty
seconds to disambiguate novice from X.

or behavior for
a task in the cool thing was at least for

these three training tasks and
these three tasks by the way.

Are correlated with surgical
outcomes sort of performance.

The cool thing is it's a good market model
so we can crunch this in real time so

for every frame every ten milliseconds or
so every five milliseconds you can ask

are you doing good surgery or poor
surgery how well do you fit the model so

what you're seeing here is a probability
that you're acting like an expert versus

the probably really that you're acting
like a novice on that extra video that you

saw and what happens is sometimes you're
looking like an expert sometimes the model

doesn't know sometimes it's confident that
both of them are doing the same thing so

you can't really use that information and

then sometimes it inverts sometimes
a person starts acting like a novice and

this is the other problem with
the ground ground truth data is that for

an hour of surgery from an expert for
about ten seconds he'll slip up and

do something very good that's going to
pollute your ground truth labels and

a novice like for three seconds they can
get lucky and just look awesome for three

seconds and that again pollutes the data
so where this fall and we end up coming

up with a thing called the minimally
acceptable classification criterion so

one is you have to provide something
over test Times got to be better

of a test I'm well great we can get real
time metrics this I will get into we

published recently that it takes
about ten to fifteen seconds.

Of data from the beginning of machine
learning will give you everything you need

in the ten to fifteen seconds it's not
going to give you much more improvement to

on the entire task and
that he was a harder one we said that.

Machine learning will fail unless you
meet this criteria and if I show you guys

an obvious novice somebody that should not
be touching a patient versus an obvious

expert you will unanimously agree that
these are extremes of a category right.

And so our minimally acceptable criterion
was that a machine learning algorithm

has to be able to give you that
hundred percent accuracy and

that no brainer case and we found is that
every single machine learning algorithm

technique that we've employed or our
colleagues competitors in the field have

employed has failed to meet
that which is interesting.

Maybe maybe it's because there isn't
enough information in the tool motion

maybe because we don't have enough machine
learning background but it seems like if

we can't even defend this basic case
there's no point in going further.

Also consider that there's fifty
one million surgeries done and

it states every year by about one
hundred thousand surgeons and

the majority of those are not
robotic we don't have data for.

But those big errors are happening with
these big numbers how do we actually get

a method that can scale and
be correct to those big numbers so can

we actually scale this can we do objective
assessment as scale it accurately and

this is where we proposed this
somewhat controversial hypothesis that

you guys seem to tell me very well within
ten seconds who was a better surgeon and

so we had this idea well can we just
crowdsource this to non-experts aggregate

their ratings and will they be equivalent
to the back to the top by top experts.

So crowdsourcing has been around for
a long time it was founded by

Lord Francis Galton the sort of
grandfather of modern statistics

in one thousand seven Nature paper
it's an interesting story this is his

story of the ox where he was.

Kind of a controversial story he he
was actually you Genesis he was trying

to prove that there's a better
class of human beings than another.

And he wanted to scientifically prove that
you shouldn't let the average person for

example vote because they're
an educated and incapable and so

he found a little carnival where the game
was somebody would bring out a huge box

and if you could guess the rendered weight
of the ox how much it weighs after all

the good stuff is butchered how much is
that way to within a pound you get it and

people are lining up by the hundreds
it's like a ha I got my experiment so

he got somebody to
tabulate all the guesses

he went home analyzed that nobody was even
close so he was going to say OK finally I

could sort of quantify how wrong people
can be but when you computed the median

it was within a pound the exact
rounded weight of the ox and

that's basically by the way the founding
of the central limit theorem where if you

sample from a distribution of any any
size it doesn't have to be normal and

you keep averaging or
you keep aggregating or taking the median

that distribution is normal and all you
need is enough data and equals thirty or

more and you'll converge the true mean.

And this has been around again for a while
it's kind of like a fad comes comes and

goes every ten twenty years recently
sort of reviewed by sort of.

In this book one of my
favorite cases is where

the folder probably folded
a group in Stanford.

Shown that you can get crowds of
non-experts to outperform groups of

top Ph D.'s on a complex protein
folding problem that the crowd doesn't

even understand but on average that
they could actually outperformed them.

So.

Who are these crowd workers over six
million are ready to work at any time

through platforms like
Amazon Mechanical Turk and

others more than half are female more than
half are educated with an advanced degree

and that means that they
are sufficiently diverse.

To me the criteria for
France a goal to crowdsourcing

as long as your variables are independent
and aggregated you're going to be good.

So here's how we tested
the hypothesis we gave.

A bunch of crowd workers this one minute
video this is a video of a robotic

training task and

and given the same exact survey instrument
that expert faculty experts use.

And ask them to objectively
quantify the skill on exposed here

had no idea what we were doing we gave
them a dollar to look at this task which

turned out to be way too much for
these people.

It turns out there's there's a lot
of people with massive amount of

free time on their hands that want
nothing more than to do tasks for you for

ten cents on the on
Amazon Mechanical Turk which is great.

And I kid you not the more you give them
the more they like tell their friends and

there seems to be no bottom to this.

Well so then we test the hypothesis
we got the scores from Turkey and

we got the scores from
ten faculty experts and

let me draw your attention
to this the means of the two

were statistically identical pretty
much just like the Francis Galton study.

Which to us was really
surprising we were we had

question mark exclamation point that
we didn't really trust at first.

Was even more interesting is
that it took less than twenty

four hours to get the four hundred
rating from the from the turkey

again they've got nothing better to do and
they're around for

twenty four seven where is it took almost
twenty four days almost a month to beg and

plead ten surgeons just to watch a one
minute video and give us the raid

even though they're involved in the study
but this sort of underscores the problem

faculty surgeons they don't have the time
they do not have the time to sit there and

give their trainees all the attention or
evaluation that they need.

All right so we published that just
remember presenting it at clinical

conferences and it would just immediately
polarize a room like nobody would be like

OK that's interesting you are either that
is amazing we need this yesterday or

the other half would be like you're insane
nobody is going to evaluate my skills with

like a group of Homer Simpson
off the street right.

But the central limit theorem says
that statistically this is going

to work you're going to
convert to the true meaning.

So then the feedback we've got is like
yeah but look this is a simulation task

this is not surgery when you know what
about when you go to live and that surgery

surgery is so much more complex everybody
gets what's supposed to happen here.

So here comes some more life surgery
the first idea where we actually tested

an animal cases we had twelve poor sign
bladder closures the pigs being used to

train surgeons to use a dementia
robot across of skill level

you can see the ratings from the surgeons
the faculty aggregated on this axis so

for each dot you have about
five to twelve surgeons

that were averaging their scores for
and the crowds about thirty to fifty.

And we got a cone back
south of point one two

as a measure of concordance
kind of like correlation.

But numbers of point nine or
above are considered good enough for

high stakes assessment so
we're really really surprised by this

when we actually went to live
surgery we got better numbers

than than when we were doing simulated
tasks and again the concordance was strong

enough to actually merit some
some really thorough study.

And people who sort of caught wind of
this we're getting a lot of phone calls

from all sorts of people like
the american your logical associate and

they had this massive study where they're
trying to validate a new curriculum

to credential their surgeons and then did
it with four hundred fifty four videos

to evaluate and there was in the words
of else with MacDougal the director of

education at your Olivier never in my
lifetime will I get my surgeons the people

that she's the boss of to the way that
many videos so they turn to us and

I'll just going to get through this we got
really good agreement with their experts

and long story short they ended up using
the crowd scores to validate their

entire curriculum for the entire
American people as the nation and

they were able to so in about nine days
we actually evaluated all those videos.

Over sixteen thousand ratings in nine days
which which is now we can actually scale

up to those numbers you know
hundred thousand students a year.

We publish this it repeated
locations there was a recent

sort of summary of some
results from JAMA surgery.

But other people have now really
picked this up there's now actually

a review paper written by a clinical group
about all the people using crowdsourcing

in health care started
from from the stuff.

Then the first real study we did on
live human cases was with this group

the Michigan it's called The music group
you can see what is the answer of there.

It's actually really remarkable
consortium over ninety percent

of all of the year all that is the surgeon
surgeons in the state of Michigan

voluntarily nobody nobody is forcing
them to do this voluntarily.

Contribute pool their
outcomes resources so

they'll pool things like how they're doing
just for the sake of self-improvement

which is really remarkable So we partnered
with this group they're running a study

to try to replicate Burke Myers work to
try to replicate the birth my hypothesis.

And hypothesis was that our crowd we
would be concurrent with their experts so

the twelve videos were assessed.

And what we found was as they ranked
the scores from the expert reviewers in

the middle column and discours with
a review from the crowds in the far

right column they agreed on the bottom
five performers which is interesting now

remember these are live cases
on humans with outcomes and we

are showing agreement between crowds that
this very quickly and expert surgeons.

Which is remarkable now.

Actually work a lot better
than we anticipated but

you can also see there's some
disagreement and here so for

example they don't agree on the top
performers and people say well yeah but

the top performers don't matter what we're
worried about are the people in the bottom

quintile those are the people we need
to help with with more training but

we started getting suspicious because
statistically the statistics is

in favor of the crowds that's where you
have the high powered the high numbers so

we start getting suspicious about these
experts scores like maybe maybe the gold

standard in surgery is actually not
the ground truth maybe the gold standard

that they think is the best thing for

scale is not accurate remember
we're using a web a survey tool for

all this stuff and so the clinicians are
using the same web survey to every mouse

click we're recording so we got suspicious
and for an average video of thirty

minutes in length we looked at how long
it takes the faculty to review it which

is how long it takes the crowd workers to
review it and we caught him in the act

the faculty weren't even watching the
whole video they were barely washing and

half were at
the Mechanical Turk hers again

they're spending more than thirteen
minutes watching this video getting

really excited about putting in
comments on that kind of stuff.

So clearly this is not the gold standard.

You know what if they missed something in
it in the middle of that video and for

all we know you know maybe the faculty
maybe they only need ten seconds

to tell you exactly the skills so

you know this doesn't say that
they're making a mistake here.

So if we're trying to compare a gold
standard to ground truth we really need

some sort of arbitrator and

what is that you know we have crowd versus
experts who is correct and that's where

we bring in patient outcomes patient
outcomes are the trump card does this

actually benefit patients can you predict
improvements in outcomes for patients.

And it's a much harder problem but
we've been working with

a number of hospital networks and we've
been collecting this data in long term and

we've been able to show again this
with these at so disclosure there.

Incorporated the company we've been
able to show that that scores correlate

directly to patient outcomes so if we
have crowd workers looking at your video.

We can correlate to hospital readmission
rates excess blood loss and length of stay

and significant complications
what was even more surprising.

So it seems like you know the crowds
agree with the outcomes and the and

the faculty agree of the outcomes as
well when this study in certain cases

the crowds actually had better predictive
power of the outcomes than the experts

which is interesting we would which
is a surprising result to us.

And I'll just wrap it up here that
this thing is like spread like

wildfire there's.

There are now entire hospital networks
that are actually using this I'll

just highlight two of them here one is out
of business health us the sixth largest

hospital network in the United States
within two years every surgeon will be

evaluated by this and by the way it's not
the surgeons evaluating this is the risk

assessment apartment every hospital every
major network has a risk assessment

apartment they have budget and a mandate
to minimize risk to improve outcomes and

they don't have tools to do it so

they got a hold of this and they're going
to implement this over their entire.

Network and within the third year they
will make credentialing decisions

they will grant to revoke operating
privileges based on the scores.

Also mentioned briefly that we've been
able to show that this works in not just

you know high profile
research institutions or

expensive hospitals
McCauley white Stanford.

Actually did this on a fellowship in
Ethiopia so he went out to a local clinic

got a local smartphone that he got off
the street took a bunch of videos and

he showed that there for their medical
school and suturing you can get the same

level of accuracy as standard medical
school for a particular valuation.

So I'll just end this part with this the
problem of quantifying surgical skill and

surgery is not close it's not done
I think we've just started and

from machine learning it's
been data starved for

this entire time we can actually solve the
problem of machine learning until we get

really clean data but I think now we're
finally getting to the point where we

have massive databases they're very well
labelled they correlate to outcomes and

now we have a well formed problem for
machine learning so we're putting together

the DACs alliance if you're interested
at all we're going to actually put these

on public forums so machine learning
experts you just get the data you get

the labels and see how well you can make
your algorithm perform and we have like a.

An annual contest where we'll get out
you know ten thousand dollars something

in a prize.

So if you're interested please email me
what will put you on the list of people

and when we actually publicize it all
right that's the first half of the talk.

First part of the talk the major part
the other part is this will look

even if we make every single surgeon
an absolute expert in a master they'll

still make mistakes humans make
mistakes can we ever get that risk all

the way down to zero because human errors
and is guaranteed we're just human and so

that's where this idea sort of comes in so
we've seen how the surgeon patient

interface is getting replaced by a digital
interface and the rise of CO robots so

if you've got a surgical robot
you're at once a CO participant.

A coworker with a surgeon but
you can start acting as a codefendant to

the patient what if a robot could
actually detect an imminent error

about to happen and not just alert the
surgeon because they could ignore that but

actually prevent it from
happening in the first place or

by design make it impossible for an error
to happen so this kind of our goal is

an error avoiding surgical robot and the
idea is you detect something in real time

before it happens and you automatically
prevent it and we started with a very

non-controversial area so if you go to
surgeons and tell them Look we're going

to take over take control of your emotions
your directories they will have issues

with that that's very controversial
historically the surgeon is like a deity

in the operating room they are in command
of everything absolute authority and

challenging that is is
quite controversial.

So we decided to go with
a non-controversial problem it turns

out when people use or tickle robots in
practice there's no force feedback so

you'd crush tissue all the time to
crush injury is far more prevalent with

a surgical robot have about a fifty
percent increase just from using a tissue

crush nobody's really dying from that your
question is just really uncomfortable

during recovery like like Question three
hypothesis was that we could in real time

using the data that the robot already has
no new sensors using the robot data that's

already present identify what tissue you
have and based on that tissue throttle the

force so that you get the traction that
you need but you know you don't crush it.

And here comes some
sunlight footage careful.

And the idea is you can see the case here
we've got a surgical robot a grasper and

as a grasping during the early part of
the grasp we're just punching on the data

trying to automatically prevent
the question to be from happening

we've got some models in place so
the field of biomechanics has a bunch of

classic work on static models non-linear
models it looks like this and

the problem is early on in the curve
you can't dissemble you ate.

The difference between let's say
a liver and spleen but if you build in

a dynamic model a real kind of
manic model with the ridge time to.

It is and what not there's a lot of
discriminating information that happens

much earlier and think about this you know
sort of coming in they're going to grasp

you have maybe hundred two hundred
milliseconds to identify that tissue

before it's too late so if we're using
this it's not going to work we need to use

a nonlinear dynamics potentially to
do it and then we use this idea.

Like computers freezing up we
start the program Scuse me.

So well that's happening the notion of
shared control so in shared control

you know you have a robot that
let's say is autonomy and

it's kind of like the autonomous driver
idea and a human Now right now the human

has absolute control and
the notion of shared control is to.

Open this up we go.

If you and your.

All right sorry about that
here we go share control.

When you have a human driver
driving a plant like a robot

which is what we've got now.

There we go human driving a plant if Alpha
the control variable is zero the human has

absolute hundred percent control standard
Center case in the operating room if Alpha

is one human has no control and a computer
artificial intelligence takes over and

if Alpha is any number between zero and

one you're smoothly
blending your averaging or.

Changing in time and
Alpha can change very quickly so

we've actually been able to show that this
works you can see here adventure robot

grasper taken right out
of the operating room.

This was used for some of the in vivo
studies and you can see here if there's no

share control on the human will
pretty much always graft too hard so

you could potentially entertain you that
way but if you turn the shared control on

within about one hundred milliseconds
the identification happens we

identify what issue it is throw the force
fast enough and the surgeon doesn't know

the difference will basically automated
this lower level control loop or

automated low level force control and
they're just grasping tissue but

it's not getting crushed and
by the way this has been around for

a long time share control
we wouldn't invent this.

Scheme and
there's a bunch of work in the area but

what we did find is that
when we use the scheme

right off the shelf just this alpha scheme
the only requirement is that Alpha has to

be monotonic with confidence that's
the only like real between zero and one.

What we found is that sometimes a system
would just go unstable spontaneously

which is not a good thing to
be happening in surgery and

what was going on is it's actually really
sensitive to Alpha dot Alpha is changing

very quickly your balance of stability
are out so we'll be able to show is for

example here is two controllers two
simple stable if you are controllers

if you blend them in any static way with
a constant alpha stable Alpha changes very

very slowly it's stable but it changes
quickly you can not only lose performance

you can destabilize the entire system
which seems to me like in the literature

people have just gotten lucky so far
with slower experiments but we ended up

with something across this because we need
Alpha to change very quickly so we have.

Trevor here is actually working on some
theory to use passivity to be able to.

Stabilize this no matter what.

Let's move on to like a bigger problem so
heart disease is the number one cause of

death in the world in the US One out of
four people that on average as you or

the person next to you.

The year is twenty seventeen and

our best clinical options are still
full open heart surgery coronary bypass.

Invented in the one hundred fifty S.

one of the most dramatic procedures or
stenting so you get a little to put in

to expand the stent there the underlying
condition is this you get slower.

You get plaque to slowly build up in your
arteries shown here that yellow stuff

starts as early as ten years of age in
American children depending on diet and

when that starts to grow over time you
slow the flow of blood to the heart that's

result in heart disease and

when it gets clogged by either
a piece of plaque that dislodges or

a clot that's a heart attack and
the downstream part of that flow whatever.

Tissue that blood was
feeding starts to die and

that's why a heart attack is so
dangerous that the same condition

same mechanism occurs in
the brain that is called a stroke.

Again and this by the way is a leading
contributor to the leading cause of death.

Heart disease that rise of surgical
robotics that you saw before

Intuitive Surgical was going after this
procedure and other minimally invasive

heart procedures in the early two thousand
a little known fact is they were on

the verge of bankruptcy because
they were not succeeding

until the year all of us started
using it and then it really took off.

So let's look at the market it turns out
everything is either bypass surgery or.

Stents the underlying problem
getting the plaque out of there

it's called Etheredge to me removing
the actual plaque is almost never done and

there's another population here called
no option patients which is not small

there's actually roughly the same number
of no option patients as prostate patients

prostate removal patients which
drove the explosion of sort of.

Robotics So there's a tremendous
opportunity here if we can get a surgical

robotics to treat this condition you
could really disrupt a massive amount

of health care in a positive way but
you can't compete with stents

you see there are actually companies
right now that make stents that for

example dissolve and there are they're
failing because you can't compete with us

that if you're stent cost three times as
much and you can prove that it's better

they're going to going to go
into the cheap stent anyway.

And so our strategy is to actually
go after these new Often patients or

individuals who arteries are so torturous
you can't get a stent there's no catheter

that's going to go in and
navigate their torturous turning.

And they're not good candidates for
surgery but

if you can bring the precision of robotics
to safely imprecisely walk through those

arteries take out the plaque in no option
patients if you can do all those three

potentially have a disruptive technology
first of all there's no competitor

there's no competitive market if you could
do it for the new option patients you

get your foot in the door in the clinic
and then the cardiologist interventionists

will start using it for
all the other procedures and

you could potentially disrupt cabbage
a coronary bypass and stenting.

And by the way I think it's a matter of
time before somebody does it you know it's

not if somebody's going to do this but
when and who so our approach to this

has been a little bit different we decided
to sort of reinvent how catheters work

catheters have been around they've been in
horses since the eighty's hundreds and for

the entire time people have been pushing
catheters of the tips of a kind of what it

looks like you get this long slender
catheter introduced in a blood vessel

Let's listen to this a plebian here and it
goes into the vessels of the heart under

it and you're pushing at the back end and
trying to get control of the front end and

once you're going through a bunch of
anatomy that becomes complicated and

difficult so
we decided to sort of reinvent this and

say what if we took our catheter that's
a catheter inside of a blood vessel and

sort of pushing it from the back and
we let it pull itself at the tip So

imagine this is a catheter
walking through a blood vessel.

And this will motion mechanism
this kind of motion if you will.

It's predicated on those
here with these so

we're using a little helix that
spiral because if you use a balloon

you block blood flow and that's a heart
attack that we're trying to treat.

But if you use the helix you
can anchor into the vessel and

still have blood flow so you are not
coming out on procedure time and

the idea that this is a soft robot so
it's inherently soft and

the compliant you can't damage the blood
vessels even if you tried when you get to

where the plaque is yet another degrees
of freedom that will articulate and

police are removing the plaque I'll show
you a technique for that little bit later.

For this we're using hydraulics and
McKibben so

I'll actually get to some of this stuff
for the sake of time we've built a few of

them the basic idea is you're
wrapping a tube with fibers and

you can make that to turn to whatever
shape you want and be very strong building

a looks kind of like this we have a little
lay that we hacked so that we could wind

fibers onto them we can model this thing
the model is actually pretty accurate for

putting in pressure so this is sailing
that's going to hear the relationship

between volume and pressure as a fourth
order polynomial not very complicated.

Now the question is how do we get serial
locomotion the blood vessels of a beating

heart we're talking about like
a two millimeter diameter so

this has to be small there's no way we're
going to get parallel actuation we're not

going to get four lines of pressure
two in the Bentley control for five.

Actuators we get one how do we control
how do we get this motion out of one

actuator of one line of pressure and the
idea was this imagine I had a little R.C.

circuit if I apply a square wave of
voltage this capacitor charges first and

then this one and then this one very
simple idea passive circuit and

if I discharge it get it to zero
volts this one destroyed is first and

then this one and then this one and by
tuning the resistances I get to the timing

of this thing so our system was
a mechanical system this was a resistor it

was just a simple flow restrictor So
a little valve a little set screw

that pinches this and is governed by the
or physical Asian fits the model pretty.

Well and you end up with a pretty
hairy non-linear model each P.C.

is a fourth order polynomial but
with that you can optimize.

To basically say one of the two
settings on these two knobs one of

my two resistance values to give me the
motion that I want that's most effective.

And it works so this is a large scale
model this is that just under a centimeter

in diameter for the first work but you
can see it working under robotic control.

And.

It was simple enough actually
that you can drive it by hand so

this is actually being driven
by a syringe and failing.

All the parts or M.R.I. compatible
disposable is very inexpensive So

now the only challenges is to scale
down which is which is still.

A challenge but doable and.

We point out in body intelligence what
you're seeing here is a fish there's

a well known fact that if
you take a fish gut it and

maybe even take out his brain you put
it back in the water will swim away.

Is this notion that we most of the control
and also like in the eighty's there was

there's some weird work on cats where
they would get the capitated cat and

put it on a treadmill and
we still keep walking.

And this notion is that there's

in a million control groups there's
embodied intelligence you don't do

it like your brain doesn't close all the
control loops you do supervisory things

like tell your limbs to walk and they just
magically do it because they have a body

intelligence there's a lower level control
it's doing all the complicated stuff

as if it's in this example you
take out the central system and

the lower level loops they just interact
with their environment to create swimming.

And so our sort of contention is that
this is the same thing going on in our

little spiral mechanism here that
as long as you're putting energy in

to figure out how to local note there's
no need for advanced sensing or control.

All right so I mentioned walking through
Anatomy what about no option patients so

these are patients again
whose anatomy is so

torturous there is no single device that
can that can navigate their anatomy.

And there were three D.
printing comes in so we get a C.T.

scan these are actually now available.

Tools like from V.M.

T.K. The vascular modeling toolkit
you get a C.T. scan and they could.

For example compute the centerlines
automatically from the from the C.T. scan.

So you can get the geometry of the
particular patient a really challenging

geometry Well let's say from
that challenging geometry

we have the initial shape of my catheter
which is always going to be a small small

cylinder as I can get it and
the final shape so

here I'm showing us a spiral but
it could be any weird shape you want.

Any weird amorphous shape
right that it can conform that

anatomy that's really unique.

So here's our idea we're going to do in
versus line and sort of starting out with.

With a mechanism or

a manufacturing technique we're
going to know who the patient needs

how do we get this thing to actually get
built in versus I'm here's what they need.

Make it happen and here's an idea if I
know that initial and final shape then for

any point on that body I can see how
much local differ mation there has to be

an isotropic defamation if I know how much
local determination there has to be I can

back project that onto the material and
say the material stiffness has

to be has to have the certain properties
in that little pixel of voxel of material

well that's pretty exciting because it
lets it go directly from a patient's needs

to a blueprint the ideal blueprint
if we can give this to a three D.

printer we can print a patient specific
actuator that would be ideal for

this patient is the idea.

And we've been sort of working with
the with the group in chemical engineering

on this.

And then get through this to say that
we've talked about this is so great it

looks like we could actually look I'm
a through arteries and possibly do it and

no option patients in and keep in mind
that the economies of scale are there

to actually do this there's a lot of
a large number of these patients questions

now is how do we remove the plaque so
we get there how you take stuff out.

The extra laser This is the same
technology and Lasik eye surgery it.

Fascinating technology you get a pull
of that light that is pico seconds and

duration there's some bubbling and
so that happens afterwards and

it doesn't heat up tissue
a regular laser just cooks tissue

it's a thermal ablation process
this laser pulses tissue with light

at a such a high intensity that it
is the middle of the molecular bonds

the particles are smaller than a blood
cell that result and you get these fifty

micron you can basically mill tissue
with Micron accuracy that's why they say

they use a school to lens doesn't heat
things up well this thing is F.D.A.

approved already it was actually a big
deal the one nine hundred ninety S.

and it flopped because
it was a little unsafe.

It's already F.D.A.

approved for use in patients plaque you
can see one of them working here and

we had a really simple idea right now
that the reason why this is not used.

To me surgeons really
clinicians do not like it

because when you're using this thing you
can't see anything you can see soft tissue

you're looking at basically an X.

ray and
real time soft tissues will show up on X.

ray so you can very quickly through plaque
and start leading through the vessel and

other patients bleeding out and you have
an emergency procedure you have to do so

is perceived as unsafe so
we said well what if during the off time

we have a twenty four millisecond that's
like a lifetime for engineering right what

if we just queried the light on some light
down to look at a light coming back most

black is visible to the naked eye these
are not small signals we're talking about

our political results so
that we could actually in real time detect

the difference between plaque and artery
wall at least for a specific patient.

And then actually we have this
thing up into the eczema laser So

what you're seeing here is the notion
of an error avoiding surgical robot

we have a surgeon in this case the.

Grad student.

Doing a mock task the idea
is the task is so

easy any human expert with decent
visual acuity can sit down and

do that task expertly Now this
task is we're saying so the idea

is you want to only want to leave the
white stuff and leave the yellow stuff.

Anybody can do this.

We're showing here is that when you
put the laser peddle on and you just

indiscriminately lazing everything we can
automatically shut it off when you get to

the sensitive areas of tissue so even if
a surgeon is trying to make a mistake and

cut tissue that they shouldn't
be we won't this algorithm won't

let him actually do it and
then we did a small user study.

Where with five different people for

different trials each we asked them
to us as carefully as possible

lays as quickly as possible as if they
were trying to get all this tissue and

showed that even on the super
simple task everybody is an expert.

We could get improved performance so
you had lower errors even

though people shouldn't be making zero
errors they're still making errors and

we can reduce those errors at that margin
so the clinician got excited about this we

got to be a little controversial but
they said no we know it's hard

we actually want this now you
know in the operating room.

All right.

And I'll just wrap up
with this the final slide

is kind of a completely left
turn you know big idea.

What if robots instead of us trying
to incrementally improve surgery

because surgeon say that they
want you know a sharper scalpel

that there's that classic quote Miss
attributed to Henry Ford if I asked people

what they would have wanted they
would have told me a faster horse.

But sometimes you could have different
ideas and the question is maybe with

with robotics we can actually do something
drastically different so surgery has been

a subtractive manufacturing process we
give her disk out and they traditionally

take tissue out what if we can give them
a little wand that could add tissue back

in the right location and irrespective of
human motion you know a beating heart.

And so that's kind of our idea
to do dynamic bio printing and

the field by opening by the way
is doing very well there's

they've been successful transplants into
mice you can get stem cells from mouse.

An organ there was a study I think in
nature on successfully transplanting

uterus into five mice and they could
actually carry pops successfully So

there are some very early work I'll just
skip through this but long story short.

We were able to.

Turn down.

This is.

Where you can three D.

printer directly onto really
moving human anatomy.

Now we're not by opening people we're
actually working with Anthony Attala

who's a bio printing sort of
figure they do the chemistry

in the cells what we're really
interested in is tracking motion and

getting a robot to subtract
out the motion right.

And the application here would be for
burn patients so for

burn patients actually at Intel's been
doing this and a group in Australia they

can take stem cells out of your stomach
make basically a little concoction for

second degree burn they could airbrush
that on and your tissue will spontaneously

grow back now the downside is you can't
get really good you can get thickness and

you can get good geometries but this to
me at all it was actually really excited

about this because now you can you
could potentially do the stuff in

vivo you could you could bring
the benefit of bio printing.

Onto a living patient.

So I think.

I think I will stop there just
to think our sponsors and

all the grassroots to actually
make this work possible.

And all wrapped up with saying that.

Computational surgery so you know we
started this talk with Hunter showing up

applying the scientific method to surgery
and I think now there's kind of a new

renaissance happening because there's so
much more American quantitative rigor

penetrating the operating room with
all these new technologies I think

we're poised with feel that compositional
surgery and robotics to have a new

renaissance in that area so that I'll say
thank you and I'll take any questions.

So why so yeah.

Right.

The.

Yes.

I will I love what.

I see.

Now the great question.

So it's a bit to the first comma but the
spirit was actually really tragic because

even though he I think this is what
you're referring to even though he

got religion in a scientific data
he was ridiculed for it and for

the rest of his life I think he ended up
like losing and he had a really sad story.

But she was fascinating so
historically you Robert G.

has traditionally been a leader
in a lot of technologies so

they're the first and
the scopic procedure for

example part of that is to fall I
think they have a culture of being.

Let's say not risk averse you know they're
interested in new technologies and

trying it out and also when you're all of

your chances of doing some really serious
damage are in some ways way lower

like if you got first of all beating
heart it's very difficult to get in there

if you mess anything up you know you start
bleeding that's a really serious going to.

Right Whereas in your ology you
can put things through the year or

prettiness Li and do a do some damage and
the patient is still be fine so

it tends to be a safer proving ground for
a lot of these technologies so

I think the answer is twofold is that it's
easier to get in there and the to and

the cultures there where they
actually want to use it and

then the other reason is the division of
robot didn't really meet the needs of what

cardiovascular surgeon wanted you know if
you think about you have a stiff dangerous

tool next to a delicate beating heart it's
a bad design decision right whereas in

prostate surgery first of all they give
surgeon the wrist which was a revolution

and a prostate surgery it's really really
hard to access you've got a bone that

you're contending with need to be able to
reach around and start tying knots and

getting a wrist was just the best
thing ever and I think that's part

of why it's such a good match for
that particular robot that it took off.

Thank you

yes and so so that's a great question that
there's a great slide by Apollo Daria was

it one of these huge figures in our field
where we can address a map it all out so

open surgery was macro scale and

then you know biomedical biological
cell biology was coming up of this.

So they were nano scale and
the future they're going to

sort of meet together and I'm kind
of still in this mess of scale you

know I'm still thinking about catheters
roughly a millimeter in diameter but

there are folks working so that like
best seller and campus she's working on.

Gold macro particles but basically nano
particles that you can sort of to do

with to basically identify factors that
made them attracted to cancer cells and

then you know it in with therapy I
think there's probably always going

to be a place for for a macro to
mess with scale stuff especially and

congenital defects but I think the future
is probably going to see a lot more

of the nano scale stuff it's
just hard it's hard to get and

I think one of the fundamental limitations
in this group try to do is get stuff down

that awesome robot for example Bob Webster
with was involved in this where.

You have a bunch of capsules in a cup you
drain the capsules they sell someone to

a robot it's out of your stomach to do a
surgery disassemble and come out right and

the fundamental issue they had was
power to generally get surgery done

you need a certain amount of mechanical
power and force and displacement and

so you need the energy density and
that's why we're stuck with hydraulics

that we think hydraulics you have
the highest energy density of

any technology known to man right so
higher than anything electromechanical So

if you need any forces you're stuck with
that only with a nano scale you couldn't

necessarily get accomplished so it's over
the procedures that you need to like you

know Rip tear tissues out a blade
tissues is probably going to get enough

All right thank you all.