it's a great honor and a pleasure today
to introduce dr. Jesse grizzle from the
University of Michigan he did his PhD at
the University of Texas Austin now he's
currently a professor at the University
of Michigan he's actually the Elmer
Gilbert distinguished professor a
university professor and then he is also
Jesse's made a lot of really cool
contributions to automotive engineering
emissions control motion and as a result
he's won two very prestigious prizes in
our control systems community the George
axel beautiful ward and the bode prize
so some really distinguished work in the
past as a legged locomotion personal
robotics person I just really appreciate
the work Jesse does for me he's sort of
captured the three really important
things in robotics he does really
rigorous mathematical theory then he
does cool innovative and fun experiments
and lastly he still has a passion for
what he does his videos you really
enjoyed themselves as students enjoy
yourselves
thank you so much it is named Georgia
Tech is such a powerhouse in control in
robotics so someone who's doing robotics
from a control point of view this is
this is the top okay it's really great
to be here this this is my group the
work today I'll be talking about is
essentially then all the new stuff is by
Dennis dog he's now at Nvidia the paper
is up on the archive and like any
sufficiently novel thing that's almost
unpublishable okay much easier to
publish an incremental result than
something totally new so we'll see how
this this works out and then a key
players you will see is a young go who
did his PhD here at Georgia Tech with
Aaron we have a new Institute going up
the building is actually under
construction so it's no longer just a
pipe dream that we've been telling
everybody so look forward to about two
years you come there we'll have
everything open classes running etc so
should open April 2020
it's gonna be gorgeous the main thing is
you know our faculty being scattered in
multiple departments not that much of a
problem our students not being together
that's the problem so we're gonna have
this swell of creativity when we get the
equivalent of this many students all
working together and sharing knowledge
okay so watch out for Michigan and it's
just going to be a fun place to work and
to be so I get last axle lots of
questions about why work on bipeds and
my favorite answer is the truth I do it
because it's fun I started out as a
theoretical nonlinear control person no
one ever came to my office and watch me
do a proof never happened okay I never
got a call from a high school saying can
we come and do a tour of watching you do
one of your proofs one of the more
interesting proofs never happened okay
but I we do hundreds and hundreds of
kids every summer in the lab we also do
important things we are teaching the
younger generation how to protect
themselves during the coming robot pop
apocalypse so if you're not doing your
part Georgia Tech you better get on on
your game okay now the other reason for
why I like to work on bye pads I always
in the past had to refer to others work
like the great work Bobby Gregg is doing
where he's taken some of the algorithms
that we developed on virtual constraints
and hybrid zero dynamics and implemented
it on prosthetic limbs here is the
exoskeleton designed by wonder craft to
start up in Paris their objective is
walking you're paralyzed from the waist
down and you have empty hands no
crutches so you can be tapping on your
cell phone as you go down the hall just
like you and I do it seemed totally
crazy they started using results from a
book I published in 2007 and we got wind
of that when Erin put together an NRI
with cow shell I said
join and I'll bring wander craft and
having the collaboration with Aaron and
Cal has been just fantastic this is a
year old so the walking is awkward
everyone in the exoskeleton is paralyzed
when their waist down so that's Oscar
the safety people are required by the
IRB they're absolutely not needed just
watch his face as he kind of starts
getting into it first time he's walked
since his accident
paralyzed the waist down the gloves are
required by the IRB I don't know why
totally relaxed getting up from her
wheelchair okay so that was kind of like
version 1 this is the walking now so
this was April of this year at a tech
fair in Paris
marine is not paralyzed okay she's not
paralyzed she's one of the engineers
anybody know anybody famous there
[Music]
so president of France has shaken hands
with marine who's in an exoskeleton that
has my algorithm that's as close as I
can get to a distance to a president
okay so it's much better walking with
better mathematics okay so the rest of
the talk what I want to do will be
focusing on our previous robot to Marlow
built with with Jonathan Herst so the
robots roughly human size 60 kilograms
so 140 something pounds okay it's
simplified it has no feet so it's under
actuated we do put shoes on it often
just so that it's not yawning imagine
you're walking on a hockey puck on ice
not very fun so this at least removes
the rotation at the feet but you still
have all the rolling motion no vision
and no lidar at this I'll talk about
what we're doing at the very very end to
add that and then the other thing this
robot has is it it's it can move the
legs sideways so it can do lateral
stabilisation but it doesn't have a
twisting motion so what that means is
when we get it out in rough ground and
the and the ground causes it to rotate
we can't command it to rotate back so we
implemented omnidirectional walking if I
have to walk like this in order to go
here the robot will do it okay so that's
the kind of agility we were forced to
build into this robot this is Cassie you
may have seen her walking a little bit
earlier it's one of the more spectacular
things so playing it back at 0.1 X and
you'll have a really good time okay so
robotics at the speed of life that's one
of our tag lines at Michigan robotics at
the speed of life so Cassie's
spectacular it's half the mass of Marlow
at least as powerful and it does have
the twisting motion and it has some
actuation in the in the foot so it has
these blade feet you can't see very well
here so I'm going to go through the
models that we use
feedback control design give you a
little bit of the hint of the past what
we were able to accomplish and then what
motivated us to make this leap for know
what motivated my student Dennis to make
this leap it's not my leaf okay it's the
work of my students Dennis and then
videos are always fun everybody likes
them and goals wrap up so walking
alternating phases of one foot on the
ground and two feet on the ground so
week one foot on the ground single
support if the foot is not sliding then
it's a standard mechanical equations
that you use for a manipulator even when
these swing foot now comes forward and
impacts the ground it decelerates in
about 20 milliseconds so you have two
choices here you can model that with a
very detailed nonlinear spring model
with who knows how many parameters or
you can take the limit as that time goes
to zero and replace it with an impulse
so mostly the field does the latter and
that's what we're going to do so what
that does is if you will go back to your
Laplace transforms and you take a
second-order system and you put an
impulse into it you will get a jump in
the velocity and the guinea positions
are continuous so that's what this
nonlinear model does so that's it that's
the model it's this little simple
trivial hybrid system got the Lagrangian
dynamics here it flows along till the
foot hits the ground it gets a reset map
you get a new initial condition and you
flow it's no problem okay except that's
the equations for this little tiny robot
the first one I ever worked on and if
you do the equations for the exoskeleton
it's about a thousand pages like that so
no human has ever looked at every line
in that model okay so the reaction of
the field is the dominant reaction is
wow that's just way too complicated it's
crazy you can't do anything with it so
they replace the Dyne
a mix with various types of pendula
spring-loaded inverted pendulum is very
famous the linear inverted pendulum etc
we don't do that we work now with the
full floating base model the only change
i've made in models i used to work with
a pin model instead of the floating base
model but we've done that from rabbit to
Mabel to Marlowe to Cassie to the
exoskeleton and the reason I think this
is important is for these robots to walk
properly
it's all about the right physical
constraints on the joint limits for
safety reasons the joint limits on the
ankle of this exoskeleton are crazy
tight and I don't know how to reflect
any of that over here okay and then you
have all of your limits of your
actuators in terms of power torque etc
so we work with the full models okay so
now let me tell you what we'd kind of
accomplished until Denis came on the
scene okay as all stories it's a little
bit twisting the facts to make the story
short and more fun but it's basically
true okay I'm from Oklahoma there's not
too many stories I've ever told that are
fully true it would be and ethical to my
background okay I'm sorry about that so
we use optimization it's really
parameter optimization because we
parameterize potential trajectories in
the state space of the robot and we put
a cost function on it roughly energy
consumed for distance travel so that's
the first thing we do but then we put
all the constraints I was talking about
in terms of you want periodic motion or
we transitioning to something the speed
we want all the ground reaction forces
what's my reasonable coefficient of
friction the torque limits joint limits
we put all of those in there and of
course we may not find a solution but we
hope we find one and if we find one it
may not be truly optimal but we hope
it's not too bad okay there's a lot of
hoping going on in here but that's where
numerical optimization of non
Beks problems is that's what these are
the method weight had that made me well
known is that I figured out how to take
a single periodic orbit and from that
build a controller for the full system
one periodic trajectory a controller
that could be applied on the full
dynamical model and get you some
reasonable stability properties so
that's what's the story of hybrid zero
dynamics it's the work of Alberto
isidoro and Chris Burns taken into the
hybrid world but turned into a design
framework instead of an analysis
framework and the design made it much
more powerful so we'd use a variable
instead of time to parameterize the
orbits because this is monotonically
increasing when you're walking and then
we would impose what are called virtual
constraints these are things I will
achieve with feedback control so the
torso angle as a function of where I am
in the gate the knee angle is a function
of where I am in the gate the swing leg
angle is a function of where I am in the
gate to etc and the mathematical theory
says if I'm able to achieve that I will
get an invariant surface in the
Lagrangian dynamics of the model and for
certain class of models if I also have a
periodic orbit that is satisfying these
constraints I automatically got
invariance in the hybrid model totally
amazing ok when Eric was JaVale
discovered that I was blown away it
could not be true Eric it is true Jessie
show me Eric you're right it's true this
is amazing ok we have to publish that
and that led to a whole bunch of other
work so the theory says if we can find
one of these hybrid invariant surfaces
and within this low dimensional surface
perturbations of the orbit converge back
to it so we have a stability problem and
the transverse dynamics are reasonably
nice this is a little stronger than you
need but suppose their feedback
linearizable then we can design a
feedback that will make the orbit stable
in the full order model so one periodic
orbit and we get all of that
seems too good to be true okay so this
is rabbit the robot in France and
Grenoble I worked on Mabel she can't see
the drop-down students that's hanging on
Park and Allie raised the Ramazan e
well-known to a few people when they
were much younger and then we got it
from the planar robots to a 3d robot
Marlowe okay and you know it's pretty
good walking it's in the lab it's the
most under actuator walking ever
achieved because she even had her hoof
feet on so she's rotating like this okay
so if I'd retired there I would have
said hey we we did a lot but I wasn't
young enough old enough to retire us two
young stills we kept going okay and then
you know all this stuff in the lab what
happens when you go outdoors so it's
pretty smooth
but it's super cold okay so it's like
minus 10 Celsius okay we're dying out
there and there were there's just a
little bit of extra slope from what's in
the laboratory and stuff and it's not
robust
so um next student
Brent Griffin and we read some stuff
that came out of rust Hendrix groups is
particularly the work by honk I died and
we tried to build robustness into the
trajectory design process so this is
kind of how it goes before we're just
designing say walking on a particular
slope or flat but just do periodic
walking okay and we'd put all of these
equality constraints and stuff on it but
then what brent was it would do is that
he would then challenge the controller
with a disturbance and then measure how
well the when the disturbance is removed
the robot converges back to the nominal
thing okay and so you want to optimize
that so if you know about Eduardo
sonntag input to state stability this is
a numerical version of disturbance to
state stability okay so I mean it's it's
it's not a beautiful theory like Eduardo
had but it's trying to build in
robustness and so you have is nominal
cost but then you'd add to that however
many disturbances you thought you had
the computing power to stand and then
measuring the convergence back to the
nominal orbit okay let's see how it
works
so conceptually here's my original orbit
I challenged it with perturbations and
they don't want to go back to the orbit
very well and so that I give that a high
cost and then I get to something where
when I challenge it with the disturbance
it behaves much better okay so that's
the idea that's the idea and so
now this is Marlo
totally blind okay it's also ten degrees
below zero Celsius out here okay all
those extra stiction and a harmonic
drive nothing has changed in the
controller okay it's spring time nothing
not a single parameter has changed in
the controller the impacts are totally
different yes we're optimizing
parameters we parameterize the potential
surfaces that I showed you with Bezier
polynomials it's a parameter
optimization problem that was solved
there okay this looks pretty good so
it's what Brent did also if you notice
up here
he's parameterizing his orbits with
something not just as mechanical phase
but he's bringing in more of the under
actuated variables of the state he's got
angular momentum and so Brent taught us
how to do much better now that would
have been fine except if you've seen
this monument it's right behind the
aerospace building at the University of
Michigan it's called the wave field done
by the same sculpture who did the
Vietnam Veterans Memorial ok what do you
see when you look at this I see robot
torture back ok
and it's been driving me knots that we
couldn't do it ok
so we're in the lab we've done all this
stuff and the Discovery Channel says
what else do you have we've always
wanted to try this it didn't work out so
well so yeah the robot literally caught
fire
electrical fire two hours later we had
it back working so the students are
totally amazing okay and if you read the
abstract that's where this quote comes
from okay
I'm never willing to turn down a bet to
show the feedback can concur very
complicated terrain but we couldn't then
and so Dennis then I went to work okay
Dennis went to work and so he's now
gonna I'm gonna show you what he did to
design a controller to have various
speeds outdoor terrain and then crazy
kung-fu external forces okay so so from
now on this is the work of Dennis so
there's before we start on Dennis though
a younga is like super important in this
so that optimization problem for
robustness that got us so much more it
was really cool but just to do the
periodic orbit with the techniques we
were using was taking about three hours
and then each perturbation would be
another two or three hours okay so how
many more perturbations you want so it's
basically a 24-hour game to get a new
controller
a younga took the three hours for
periodic orbit down to about five
minutes so this is a game changer okay
this is 60 times faster 60 times faster
now it's fast but it's still not fast
enough because if you could just do all
these calculations in real time I mean
why why not okay but our gate times are
about half a second so three minutes
half a second that doesn't work okay so
it's not fast enough the other thing you
could say well I can do all these
calculations offline let's just store
the controller that's called
explicit NPC the problem is you've got
all these states let's suppose you only
use five points in each dimension and
you've got sixteen dimensions five to
power 16 is that I'll show you later is
152 billion okay you can do some
sampling and reduce that down to 1
billion or something but it's still you
can't you can't calculate it offline and
16 is actually smaller than the
exoskeleton so this is what's this is
this is Dennis's question what to do so
he's now at Nvidia so in his papers or
that one's published and this one is on
the archives waiting for a reviewer to
be nice to us so here's the story
trajectory optimization thanks to a
younger is almost free in fact just
recently we've taken that three minutes
and it's now 15 seconds for the
exoskeleton and for Cassie okay so part
of it so a youngest using direct
collocation and that makes it much
faster than he had before he also what
does slowing down the codes before is
they were have to invert the mass
inertia matrix and he made that one of
the slack variables in his optimizations
that gave him the speed-up and then this
last speed-up is taking everything from
MATLAB to C++
you're not surprised it's much less
convenient in some sense but it's still
good
so trajectories we know how to do but
trajectories are terrible because if I
get perved off perturbed off the
trajectory I don't really know what to
do now vector fields are wonderful
they're an instantaneous replan ER in
the language of computer science I don't
really like that language I like
language of vector fields and manifolds
but as soon as I get perturbed off I
just follow the next set of arrows
that's the instantaneous replan ER so if
I can find a state variable realization
of this behavior in the sense of young
Willems that would be amazing okay yes
in the end we will get a feedback
control law but the key idea here
trajectories are not good vector fields
are good okay and then we'll use the
hybrid zero dynamics if we can endow
this surface with the right properties
we can use properties of transverse
dynamics and physics and this is where
the machine learning in the title comes
from so there's real mathematics about
if you can find a function that
satisfies this property and interpolates
in a certain way on these trajectories
then this exists and the machine
learning is what is a way to find that
function it's just solving an implicit
equation okay so that's what we're doing
okay so now I need to set things up so I
can quantify stability in this setting
when I start building my trajectories
this is what I want to do so here's my
hybrid model how do i what how do I find
these periodic solutions and then how do
I check for and for me I'm a design guy
it's really how do I build instability
that's the question I always want to ask
so this is a plonker a map so that pink
thing is the surface that has dimension
one less than the ambient space you
guess where your trajectory might lie
and you make sure that that surface is
parallel to it that's called transversal
it's not orthogonal because we have no
no structure here to even talk about
orthogonal but it's not parallel so
plunk are a map is I just take a point
on that surface I'm gonna apply my
impact map from my hybrid model I now
have an initial condition for the
mechanical model and I flow back until
the leg touches the ground to give so I
can work from XK to XK plus 1 that's the
punk terrain map if I start and I come
back where I started
that's a periodic orbit yeah it's called
a fixed point ok now how do I check if I
have one of those whether it's stable or
not easy way to do it is you do a
numerical Jacobian if the eigenvalues
are inside the unit circle you get local
exponential stability small
perturbations come back exponentially
fast another way to check stability is
give yourself a contraction rate and a
small ball and they just check as I
evolve to my point XK plus 1 and my
closer to the fixed point then I started
by a contraction factor so this is a
sufficient condition to get this so
that's the one I'm going to do because I
don't want to linearize any punk array
maps so so then you get exponential
convergence ok so let me start telling
you the story of how we build these maps
now there's two facets to the story so
I'm going to break it up into two parts
the first one is the longest I'm going
to ignore the fact that my model is high
dimensional and I can't deal with 16 or
24 or 48 dimensions I'm going to pretend
when I'm telling this first part of the
story that the dimension of the space is
not the issue it is how can I take
trajectories and build feedback control
policies that's what I'm going to focus
on now and then we'll come back to the
fact that that story is still impossible
to implement you got
another small improvement okay I'm going
to break it up into two parts so to make
everything simple sorry
Duras you're no longer a hybrid machine
okay you're just an Eau de so we're just
going to throw away all of the reset and
apps and just pretend we're playing with
Oh des and that's how we wrote our IJ
are our paper we did first Oh des and
then the second part of the paper we put
it the hybrid stuff yet so we now have a
differential equation modelling our
robot we design a periodic solution that
meets all of our constraints using
numerical parameter optimization it's
going to have a given period and it has
an initial condition I'm going to calls
AI star okay really important here this
is a point this is the entire trajectory
okay V sub Z star is the trajectory so
this is my periodic trajectory so now
I'm going to make a guess at how fast I
can contract and how big of a ball
around that I can extend my optimization
I'm going to start building other
transient solutions now so these are
just other in open-loop inputs
corresponding trajectories on the same
interval of time okay so they're defined
on the same time interval as the
periodic orbit they don't have to be but
it's much simpler if we do that they
have an initial condition in this ball
they terminate in the ball and in the
language of computer science I now have
infinite executions okay starting the
ball I come back to the ball I can keep
going
well you look for that before it is in
video you like it you like the computer
science language okay and this is my
stability condition
I'm gonna build in stability okay now
could be that you know I start on the
ball I go farther out and then I can't
find these trajectories anymore that's
good information to have
okay that's telling me I'm physically
limited in what the
but can do so it's all good if my solver
returns infeasible when I start
branching this out so I'm assuming I've
successfully built these up we use
optimization and a younga but let me
just skip whoops I hit the wrong button
really hit the wrong button we're not
back at the beginning are we okay
anybody bored is the time to get up and
leave let's go
yeah just at the fun part okay bingo
bingo bingo bingo bingo bingo bingo
okay now I'm being very honest here with
some luck a little bit of skill up to
you how much luck and how much skill my
trajectories look like this you're going
why I'm very interesting what I really
mean is they don't cross okay what I'm
doing open-loop trajectories you know
they can do anything but if I want this
to come from a feedback controller and
it's gonna be reasonably smooth like
it's locally Lipschitz continuous I need
unique solutions and these are going to
be solutions of my closed loop system
if they cross game over okay there is no
solution okay so you may have to do some
work to go back and do that so you can't
always make this work so now I'm going
to build two feedback policies it'll be
fun to note along the way they have the
same clunker a map but they have totally
different performance so the first one
is going to be a sampled data feedback
control law okay question I have a
solution of the model I need an initial
condition an input and then the state
trajectory yep
yeah that was that optimization problem
that I skipped through yes
[Music]
there's no controller yet that's all
open-loop but Panos yes so optimal
control people try to do this all the
time no no that's why the word luck was
up there it was a better with a better
person here with more skill that you
have more skill there's less luck okay
it's the product of skill and luck
that's a constant okay so I'm building
my first feedback now everything else
was open-loop i justjust trajectories
just trajectories okay because that's
what I know I can compute and thanks to
a younger boom 15 seconds 15 seconds and
I can run them in parallel be computed
1331 trajectories in less than five
minutes for 87 cents on the Amazon Cloud
okay so they're free it's free okay come
to consider the trajectory calculations
free so I'm going to build this first
feedback this is how it works
it's periodically time varying and it
gets reset in the state every time I
take a step at the beginning of the step
I take a new sample and then I apply
this open-loop trajectory okay so it's
it's like a zero order hold except from
varying in between think Pierre Kabam
bah once upon a time when you was doing
robust control okay so this is how it
works so these are my trajectories these
are the state trajectories these are the
corresponding inputs to the thing and
they're all parameterize by the initial
conditions in my ball
they're just parameterised by that so I
sample the state
I apply this input and it moves me to
here okay then I get a new state value I
go over and I read and I apply this
input and I apply it I play it who
jumped over to here okay and then I get
a new thing etc so that's how this
feedback works it resets every TP
seconds otherwise it's in between it's
open loop that's how sample data control
works usually you're a constant I'm
little more varying let's check the
punker a map and see if it's stable okay
so my impact map here is trivial because
I don't have impacts okay so I start on
my surface I start out of ice yeah my
impact map is trivial so if I take X I
just keep the same acts that I've reset
time to zero that's my reset map I
follow that around I get XK plus one
that's my plunker a map but the way I
designed the trajectories this is
contracted with respect to this this is
precisely my punker a map and so I have
exponential stability okay now most
people they say hey your locally
exponentially stable they just go okay
game over time to go home right let's go
I'll meet you at the bar or I'll meet
you at the coffee shop it's game over
but this one is this is a bad game okay
I'm setting you up so this probably this
thing it doesn't react to a disturbance
in between so here's the inverted
pendulum we're gonna do the upright
equilibrium so it does not have a
fundamental period so we can use any
periods we want that's what we teach our
sophomores right your Georgia Tech - I
hope and so it we start off the
equilibrium and this controller boom you
know every two seconds we're updating
and after six seconds we've crushed the
initial error but now at half a second
before I do my next update I apply a
disturbance and it just goes crazy okay
control updates at 12 seconds and then
it drives it back so here's a smarter
feedback and this is what Dennis did so
our little gird um can experiment we're
sliding a trajectory and we get pushed
off before we just kept applying the
same input and that was really bad what
input would you like to apply at this
time that's see lots of people you want
to apply the input that's the trajectory
that passes through this but that's an
implicit problem okay
so that's the formula for the feedback
policy that you want okay so it's an
interpolation condition on these
trajectories so that's the feedback
because you know at least this
controller is driving this thing to be
closer to the policy okay and this
updates the control continuously and you
have an implicit equation and that's
where we use machine learning is just to
solve this equation from data so this is
how we do it we sample the state here's
a state and now we sample time and we
spit them in a table and I didn't know
this language of features and labels but
I do now okay so the features are the
time and the states and the labels or
the control action I take a new state
okay that gives me a trajectory i sample
the time on that trajectory it gives me
a whole bunch of values and so now I
know at all the different times and a
whole bunch of different states what the
correct control values are and I keep
doing that
tada
and you learn a policy because if you
call it regression it seems boring okay
so it's called learning
might be mean or something okay but you
know and then you check that you're not
overfitting and stuff like that we just
use the MATLAB toolboxes for this okay
so it's totally amazing that you know
this there's a lot of data that we've
generated and this takes like two
minutes and even in MATLAB okay super
faster the codes for this was just
amazing okay
and so here's the theorem that goes
along with it okay it says assume I have
a system for which I can generate all
those trajectories I was telling you
about assume when I checked those
trajectories that there exists a
function that satisfies this
interpolation condition that all you
guys intuited immediately was the right
one
okay assume it's locally Lipchitz and
stuff but when I'm using my neural nets
it's actually usually smooth and oh very
nice okay so that's a problem then the
origin of the closed-loop system it is
locally exponentially stable so I built
a feedback policy this feedback policy
has exactly the same clunker a map as
the other one but this one is has an ISS
property it's it rejects disturbances
much much better so and then you can see
how the controller's work there's the
old one and then this is the
perturbation with the new one okay so
totally different because it's reacting
immediately okay now the problem is this
is the other part of the story that I
said I'm breaking it up is that the
sampling of the states if I have Marlowe
the robot I showed you which is eight
degrees of freedom in single support I
would have then sixteen states positions
and velocities and five to the sixteen
is a hundred and fifty two billion so I
can do it on the inverted pendulum our
car but that's not a robot okay so this
is where dennis was amazing
and he's doing it with 25 trajectories
okay 152 billion so I don't care what
kind of sampling you're using you
weren't going to get 25 okay I don't
care I don't go away with your sampling
I mean it's important but it was not
going to get 25 so what idea did he have
there that's what I want to go in and
this is just to remind you we want real
things that act well in complicated
terrain okay so okay so the big tool
that's still valid is invariant
manifolds you know that those who have
been invented for over a hundred years
and I've just been playing with them
more in the hybrid domain than many
people but so invariant manifolds so
it's still true if I can build an
invariant manifold with a nicely
contractive orbit contained within it
and I can make the transverse dynamics
nice I can stabilize into full model so
that did not go away now this is how we
were doing it we'd start with a single
periodic trajectory i define all these
virtual constraints and from that
trajectory
I was really building an immersion
between the under actuated States and
the and the fully actuated States and so
that would sweep out this surface so
what I want to do now is not do that I
still need to do a little bit I'm just
going to start out with a periodic
trajectory and I have to have something
on that my initial condition is a
hypothesis of what a good low
dimensional surface will be and that's
where I use the physical intuition we
have on these models that part needs
more theory into how you choose it but
what I'm not going to do is just say
once I know the boundaries and this I
fill it all in I was filling it in with
trajectories that were
valid for the mechanical model but
didn't satisfy necessarily joint
conditions motor power constraints
ground reaction forces all the things
that keep a robot walking okay so we're
gonna now fill in these trajectories
using the optimization procedure I was
showing before so we're going to start
out with the boundary of a manifold and
fill it in to a surface it'll be
foliated with trajectories and then
we'll turn those trajectories into
vector fields if we can and then we'll
build a controller for the full MOPP
okay so this is just defining the
boundary T equals zero and T equals TP
instead of the whole surface so we're
going to do that by assuming where we
know something about the physics so we
decompose the robot into weakly actuated
parts like the center of mass position
and velocity they're far from the ends
of the legs where the ground forces are
being generated and then this strongly
actuated part which is all the joints
the knees and the hips and things like
that so the physics gives us a natural
decomposition of the robot and that
physics is such that the dimension of x1
is far far far smaller than the rest and
now we can use the machinery that I just
showed you for when we ignore dimension
we'll start out with a periodic orbit
that meets all of our things we'll have
an initial condition we'll have a period
we'll have a trajectory okay we'll then
build other trajectories that start in
the orbit excuse me start in the
boundary of the manifold the end in the
boundary of the manifold giving me my
infinite executions and they contract to
the periodic orbit I'm going to build
instability I'm going to create an
invariant orbits low-dimensional that
contains a periodic or
of it and its contractive in terms of
trajectories okay so that's my picture I
wanted to see can I find a state
variable realization of that so find the
vector field and as before I can't have
this but this is hard there's three
dimensions these two trajectories they
don't cross I project them the two
dimensions you what happened here oh
okay so this is harder this is harder
okay we have to avoid this but this is a
lot harder to do just because once again
here's two solutions they're unique I
project them down to try to parameterize
by lower dimensional set and now they
cross so I couldn't find a smooth
solution so this this is harder so we
need to get x2 projecting nicely onto x1
we just we just test it we build up our
tables just as before we do the fitting
and if we have a terrible fitting we
know that we had problems and we have to
go back in and adjust the cost function
and things like that to build surfaces
that don't have crossing trajectories
the theory for how to do that
systematically is missing this is just
the initial part and so if we have the
same type of learning condition as
before it's expressed in a little bit
more complicated notation but then we
get this zero dynamics that I've talked
about all the time for the past 15 years
but Dennis is now designing one that is
specifically going to contain an
exponentially stable periodic orbit
there's a T missing there and the final
step is to embed it in the model and we
use physical structure so that's pretty
cool
okay so here's Dennis in the lab
that's a replication of the wave field
the student is supposed to be practicing
the buddy system paying attention to
what he's doing Oh something interesting
just happens that's not how the buddy
system works but but that's Dennis there
with the robot now can he knock the
robot over yeah so he caused a robot to
spin and anytime that happens we just
killed the power so we save it okay it
was completely safety reaction then we
took it out to the wave field and we're
able to go up and down that's all
documented on YouTube up and down the
small pumps and they're much tougher
than it looks because when people are
mowing they mow it to look fat flat but
there's a big hole there so Marlowe
found all those holes and she still got
through them this is that
omnidirectional walking I mean I was
telling you about so all the
disturbances are just rotating the robot
remember we have no twist control in
this in this robot we still want to go
this direction so that's amazing okay
that's how robust the controller sees
heat Dennis is just giving it direction
and speed from though from the telecom
and okay see those holes he steps in you
know he can't see him in the grass but
you could see the robot reacting to it
now she's rotated around and she's
facing the beginning here but of course
the Discovery Channel was not there when
we did this okay no life is not that
that good okay and then we tried the big
hops and we snapped the leg completely
into it the knee
so I think we reached the limit of what
you can do with any kind of blind
walking
I don't think Seth would do that I
wouldn't do it okay maybe these young
guys like Aaron could pull it off okay
these students okay so now you need you
need perception and so that's where
we're moving now you have the first
Cassie now is a with a torso we built
ourselves one of the lied ours is still
on order but there'll be two lied ours
two cameras there's a GPU a router small
battery Cassie can walk about four hours
the battery we put in the little hobby
battery we put in the perception unit
we'll go about two hours and we're
working on mapping and autonomy so
that's where we're going and the goal is
to walk upstairs and the also to be able
to recognize when we're on grass versus
snow so we take much smaller more
careful steps in the snow things like
that but when you want the geometry so
we can walk up stairs at human speed
like Atlas can do except we'll publish
how we do it so this is Cassie at
Michigan he said everything that was
interesting right there's I just cut it
out okay
I think Cassie policy Mouse
okay that's it thank you for your
attention
[Applause]
yeah if you need to get the class and
stuff no shame yeah
I know you want to yes highly under
actuated
well so so this is where the hybrid
nature is helping us because we're only
traveling a small part of the state
space before the foot goes down and then
we're repeating the motion okay so if we
have to go like this in like a space
robot all of that then we're in big
trouble but we're actually piecing
together trajectories over a much
smaller system my observation this is
not my theorem my observation is that
allows smoothness to to work its
piecewise smoothness right because
hybrid reset so I think that's what
gives us yes no no so if we could solve
those optimization problems in real time
so calculating a trajectory from current
initial conditions and suppose I can do
that in 50 microseconds okay 50 50
milliseconds even okay
so now I've got another 400 milliseconds
to use the controller that's not bad the
thing is the model that I'm using is
good but it still doesn't have all of
the little subtleties of the power
amplifiers the gearing and things like
that
so part of the thing that I didn't tell
today is once we have this how is it
deployed on the robot and there it's
still roughly trajectory tracting where
it's model free human tuned PD control
that is a totally embarrassing admission
okay and I totally hope that we can
solve that problem but on the robots
that I have the uncertainty from the
motors a time delay in the EtherCAT
Network and the dynamics and the power
amplifier is still big so we'd have to
include that into the model and the
trajectory optimization and then and
then then we can do it so there's two
steps improving our modeling of the
drivetrain the actuation train and then
the speeding up the calculations and
then so I think it's going to take
sufficiently long that I can finish my
career with this okay be honest with you
but it would it would be a controller if
as panels was discussing with you this
morning I can give guarantees on the
computation time and then I still need
to do some kind of robustness analysis
like I was talking about here because I
could end up with these very life I was
computing that first controller where we
went outside with Marlo and she fell
down after 21 steps I could compute that
as fast as I want and it would not have
kept her up right so there's still
control theory needed to get the
robustness and it's not just the
computations but I'd love to have those
computations okay yes
yeah we use symbolic methods okay so how
we know the zero dynamics is stable is
the surface we we roughly fill it with
trajectories then we use machine
learning with that data and I fit a
surface to it and then I check the
quality of my fit okay the trajectories
I know were all converging to the
nominal operating point the periodic
orbit by design and so if I had
perfectly computed this surface and this
function was computed perfectly I know
that the punker a map is such that I
have exponential stability here I'm
relying on the automatics small amount
of robustness that I get with
exponential stability so did I use the
quality of my fit doesn't have to be you
know point zero zero one centimeters off
of the joint angles and radians or
whatever so so we actually computed when
you when you do the machine learning the
supervised machine learning the surface
that comes out that is the zero dynamics
manifold and it from that we're
extracting the vector field as well
system is not full state linearizable
it's under actuated you can prove that
but the transverse dynamics consists of
the actuator joints so those are all
relative degree two outputs and so the
transverse dynamics is input-output
linearizable yeah we take advantage of
that let's go with the student you know
you can each step you reset back to the
same point so it's a stable periodic
orbit and then you optimize doctor
that day but why they're kind of
disagree with the summary okay the pump
the punker a map is only there to guide
the creation of trajectories so that if
I have this convergent property I know
that stability will will come in the
optimization is because my tool my
hammer the only one I have right now is
trajectories they are parameterised by
my initial conditions and then I try to
take that information and convert it
into an entire manifold an entire set of
vector fields well for example back in
Brent's work he would take three steps
and measure that he got 80% back so the
recovery was still asymptotic so he
weren't trying to do it in one step
deadbeat is very expensive and very
fragile so we definitely were not doing
deadbeat yeah alluded to yeah
so there's a couple of things you can
change your selection of the variables
you're putting into the feature set so
in this paper on the archive we go
through three different ways that we
solve that problem okay so that's one I
can't remember some of the others right
away I do remember one that one that
we're working on now and we need more
powerful mathematicians to work with
this if these trajectories were coming
from solutions of hamilton-jacobi Belma
and we'd have a better chance of them
being feedback realizable okay we're
just minimizing a to simple cost
function that doesn't represent that
doesn't bring the constraints properly
into the cost function and stuff so
they're just individual trajectories and
I I'm pretty sure we need a more
compliment arresting cost function to do
that
if we I know that there are examples
where if we pay too much attention to
energetics it does cause crossing so I
haven't looked at P implies Q but I've
looked at not Q implies piece was I
think they're equivalent
almost yeah we just haven't had the time
Denis defended in April he's now at
Nvidia earning my salary okay he went
from $2,500 a month
- okay so so he hasn't had a whole lot
of time to come back and work on that
and so now it's a question of just
another one of the PhD students want to
pick up on that
exactly and especially I mean if this is
the periodic thing and one of the
perturbations is slowing you down and
you say what to get there that's crazy
okay they need to be scaled in a much
more intelligent way yeah it's it's it's
a 35-page paper like it is okay you can
I think is I think it's the way to turn
it into a book if somebody wants to pick
it up I don't know if anybody at
Michigan well I'm hoping somebody will
with me but panelist you want to work on
that with me evangelist you want to work
on that with me okay
there we go well Oh way in the back so
you have to speak loudly
the zero dynamics is a lower dimensional
model but every trajectory of it is also
a trajectory of the full dimensional
model so every trajectory in that zero
dynamics is respecting every joint limit
every motor torque limit everything that
I needed to make a robot work and so I
think there is potentially a very
strange pendulum with different crazy
masses rotating but they're all linked
together because it's still
parameterised as a four a three degree
of freedom system okay so it is maybe
some mechanical analog of it but one
fact is that when you have more than two
degrees of under actuation you can prove
that the dynamics on the zero dynamics
is not Lagrangian so it's not really a
pendulum of any kind
oh what i lose is i get pushed off it
and i'm using a simple feedback
controller to try to converge back to it
and these trajectories are not optimized
and they're not respecting necessarily
all of my constraints okay so those
trajectories which are you know in the
case of marlo 12 of the 16 dimensions so
the
much more numerous in some sense okay
that's potentially very dangerous no
reason it works is that if this zero
dynamics we have the robustness built-in
the perturbations don't push us very far
away and so the trajectories near there
are still if we have some
conservativeness when we designed them
they're still okay and our feet aren't
slipping and you saw those tremendous
kicks that Dennis was applying to the
robot and it's still staying upright but
you're hoping you're hoping that those
satisfy your constraints yes if you
could use more of your state space for
your solutions then you can do more
things but if the motion that I'm
interested in is one of my is my orbit
that's transitioning from point A to
point B and I'm still stabilizing around
it I'm still I'm not restricted into
this a priori the robot has this big of
a state space and I can only use these I
get to choose where this surface is
lying in the big thing when I build it
so that's why I don't lose as much as
you might think
okay but I still have the problem that
with this really crazy very fun
acrobatic motion I can only guarantee in
a spin surface around that whereas if I
had more powerful tools and I get this
great big domain of Attraction I'm much
more robust so that's what I lose I
don't lose individual motions because I
can work around any solution of the
robot yeah
wow you guys are hardcore that's cool
yeah oh oh robotics yeah yeah but you
can have quite a few of those surfaces
of the same dimension and the complexity
of everything grows linearly mm-hmm and
then you can do interpolation between
the terms no we just use linear
interpolation and it works it works well
we haven't had to do anything more
complicated so we have these a technique
called gate libraries and we have gates
for you know flat ground sloped up
sloped down walking forwards walking
backwards walking sideways walking at a
diagonal we have all of these gates and
we can just move amongst the surface and
you're right that does help fill out the
space as well so that's another thing
dentists didn't mean the guy's his like
I said he just completely flipped what
how everybody in my lab thinks about
gait does I know one student brilliant
so it's on you the onus is on you to do
the same no you get one of those in a
career maybe two if you're lucky you
know so but all these ideas are Genesis
I may be telling the story differently
than he would but it's all his work
super proud of him
yeah so I think if I know the ground
terrain I can compute solutions but I
have to take you know what happened
there is marlowe literally stepped down
into a hole that was a meter deep okay
that is not good okay a meter deep okay
so everybody here would have torn
something there's not a single person in
the room would have come up without
damage so but if I know it's there I
know she could have taken small enough
steps and headed down the thing but if
my perception system can do it the thing
that drives me crazy is four and
five-year-olds show up on that they've
never seen the field before they just
take off running okay and they cross the
field they never break anything so we
can do it to question
um I don't think my advice is good
enough to give to anyone I and I'm
certainly not bio-inspired in any
possible way I just take the mechanical
equations and I look for solutions of
models and I try to meet physical
conditions with friction coefficients
and stuff like that but it does drive me
crazy that little simple biological
organisms are just killing my methods
okay so I respect my colleagues who are
by all bio-inspired and so I guess my
advice would be yeah go work with the
shy retsyn go work with greg just our
wiki go work work with Erin okay I'm
gonna work with ani I'm gonna work with
the young people who kind of move
transparently from the biomechanics the
neural mechanics to the real mechanics
to the feedback control I don't I'm a
dinosaur okay I was trained in pure
nonlinear control theory I'd like to
compartmentalize everything to be a
theorem and some geometry and stuff I've
got machine learning in a title but it
works it works really well but the
theorem doesn't depend upon the machine
learning the machine learning is used to
extract the function that the theorem
says should do it so I think that's a
small difference but one of my postdocs
is using reinforcement learning in a
really cool way so he took feedbacks
that were parameterised with these
Bezier polynomials and things I did and
then instead of trying to let the
reinforcement learning learn the torques
which is super complicated when you just
have a few parameters and nominal
parameters are giving you almost walking
then it's much easier to get really
robust walking much with far fewer
iterations on there reinforcement
learning and so that paper just got
submitted
ikura without my name on it cuz I did
nothing okay but he was working with
colleagues at Ohio State so I think
there's lots of ways to combine all
these things I'm not against any method
I would just like to have some analysis
and justification for what calculations
to do and some intuition that's just me
I'm a dinosaur okay so let me ask you
this and then I'll answer the question
when I ask a clip you a question back so
you are you have a segue and you are
designing the control system before the
segue have you ridden a Segway but
you've seen people write one okay so is
the segue rejecting the action of the
body yeah I say no because I know how
the inverted pendulum on our cart works
I need to have the mast leaning forward
that I need to have my acceleration such
that I keep it okay and so that's how
this exoskeleton is controlled in the
end it's a glorified Segway except it
doesn't have wheels it has legs that
move underneath now also for maintaining
a standing position it has fully
actuated ankles so it's not like the
Segway where it has to have a small
oscillation to take care of balance it
can actually stand quietly they will
also potentially have a hand controller
but so once again so the the human is
going to be used
their trunk position now the spasticity
in the legs and stuff that is rejected
by the controller they're not typically
providing useful input so the problem
I'm solving even though it's so
spectacular that you can walk paralyzed
on the waist down and stuff is easier
than trying to recover motion and people
who've had stroke where like things you
are doing where they'll have maybe 60%
of their motion is correct and 40% of
you and you try to use the good stuff
and only fight the bad stuff but then
you have a lot of trouble of discerning
what there is so yeah so ever I've had
this thing people say well you're just
rejecting the human as a disturbance
well their lower body that's pretty much
true their upper body is an actuator but
then they could have buttons and other
things so you know I want to lock this
so I can lean and pick something up okay
wow that is not how you don't want to be
rolling towards the desk when you're
doing this okay so there's some things
that you're gonna have to do other ways
to get information into the exo so that
what the how the upper body is
interpreted is it fair
[Applause]