Mechanics for robot simulation and reinforcement learning
Instead of counting how many kinds of mechanics there are, learn how to pick the framework that describes what you are dealing with and can actually be computed.
Written for people starting out in robot simulation and reinforcement learning. The goal is a map of the concepts, not drill practice in solving equations, so that the words thrown around in a design review land somewhere specific. Every number and default in here was checked against primary documentation or measured directly.
Chapter 0
"How many kinds of mechanics are there?" has no clean numeric answer. Change the classification criteria and it subdivides forever. So change the question instead: which framework describes the phenomenon I am dealing with, and can actually be computed? Ask it that way and the answer settles immediately.
To simulate a robot and train it with reinforcement learning, you need four things: Newton and Lagrange inside classical mechanics, plus rigid-body dynamics and contact, which are those methods applied to robots. This document puts all four on one map and walks through them in order.
When someone says "dynamics", "the Lagrangian", "contact parameters", or "forward dynamics" in a meeting, you can place the word on the map instantly. Solving the equations by hand is not the goal.
Chapter 1
Mechanics is the physics of why and how things move. It has several branches, and the world at human scales and human speeds is covered almost entirely by classical mechanics. Robots live here.
Approach the speed of light and you need relativistic mechanics. Go down to the size of atoms and you need quantum mechanics. When there are too many particles to track individually, statistical mechanics handles them probabilistically, and that is where heat comes from. For robotics, none of those three are needed right now.
Next to classical mechanics sits continuum mechanics, for things that bend or flow. The bending side is solid mechanics, the flowing side is fluid mechanics. Robot arm links are treated as rigid, so you do not need this yet. It comes back the day you work on soft grippers or cables.
Chapter 2
"There is classical mechanics, and under it Newton, Lagrange, and Hamilton." That is a fair way to hold it. One more distinction keeps it from getting tangled later.
Newton, Lagrange, and Hamilton are not kinds of mechanics. They are three different formulations of the same physics. Solve the same pendulum or the same robot arm with any of the three and you get the same answer. Only the starting point and the route differ. In accounting terms, it is the difference between reading a company through its income statement or its cash flow statement. There is still only one company.
Rigid-body dynamics, on the other hand, is an application area: those formulations applied to a system of stiff bodies connected by joints. It sits one layer down. Contact dynamics is the part of it that deals with what happens when one body touches another.
| Layer | Name | Question it answers |
|---|---|---|
| 1. The big frame | Classical mechanics | Why do things move? Everything at human scales and speeds |
| 2. Formulation | Newtonian, Lagrangian, Hamiltonian | How do I write that motion down and compute it? All three give the same answer |
| 3. Application | Rigid-body dynamics, and contact dynamics inside it | What happens when I apply those methods to a real robot made of links and joints? |
| 4. Tools | MuJoCo, Isaac Sim, Isaac Lab | Software that does the layer-3 computation for you, every step |
Chapters 4 to 6 are layer 2. Chapters 7 and 8 are layer 3. Chapters 9 and 10 are layer 4.
Chapter 3
The word dynamics in rigid-body dynamics and contact dynamics means the relationship between force and motion, over time. Its counterpart is statics, where nothing moves and you only check balance.
So rigid-body dynamics is "how do stiff bodies move over time under force", and contact dynamics is "what force appears when bodies touch, and what happens afterwards". Both have time in them.
Robotics documents always mention kinematics alongside statics and dynamics. Kinematics leaves force out entirely and deals only with the geometry of position, velocity, and acceleration — where the tool tip ends up given the joint angles. If force appears, it is dynamics. If it does not, it is kinematics.
Chapter 4
The Newtonian formulation starts from force. Draw every force acting on one body, add them up, and you get how that body accelerates.
F = ma
How to read it: the sum of forces equals mass times acceleration. Know the forces and you get acceleration; accumulate acceleration over time and you get velocity and position.
Rotation takes the same shape. For a body turning about a fixed axis, the sum of torques equals its moment of inertia times angular acceleration. Robot joints rotate, so in practice you use the rotational version far more. A body tumbling freely in three dimensions needs extra terms.
Hang a bob from the ceiling on a string. Two forces act on it: gravity and string tension. What you want is how the angle evolves in time, but the tension is also unknown, so you have to solve for both together. With one body that is manageable.
Note: the Newtonian formulation, tidied up for rotating rigid bodies and arranged into a fast recursive order, is called Newton–Euler. It is the standard way to solve robot inverse dynamics in real time, and it is commonly what runs inside a controller.
Chapter 5
The Lagrangian formulation starts from energy, not force. You never draw individual forces. You write down the system's kinetic and potential energy and nothing else.
L = T − V
How to read it: subtract potential energy (energy from position) from kinetic energy (energy from motion) and write down that single quantity. Feed this Lagrangian L into a fixed procedure and the equations of motion fall out mechanically.
Instead of the x, y, z position of every link, take only the joint angles q as your variables. On a six-axis arm that is six numbers. This set of variables is called generalized coordinates. The reasoning is simple: knowing six joint angles already fixes where every link is, so use only those.
Do this and the internal forces that exploded in chapter 4 drop out of the equations by themselves. The force a joint exerts to hold a link is already encoded in the constraint "only the joint angle can change", so it never has to be written down. What drops out is the idealized joint reaction force. Motor torque and joint friction remain and still have to be added separately. But the number of unknowns falls sharply, which is why the Lagrangian formulation became the textbook standard for robotics.
M(q) q̈ + C(q, q̇) q̇ + g(q) = τ
How to read it: inertia term + Coriolis and centrifugal term + gravity term = joint torque. The left side is the itemized bill for moving the robot; the right side is what the motors supply.
| Term | Symbol | Meaning |
|---|---|---|
| Inertia | M(q) q̈ | Torque needed to accelerate the arm. M is the mass matrix. An extended arm is harder to swing, so M changes with the pose q |
| Coriolis, centrifugal | C(q, q̇) q̇ | Side effects that appear only when the arm is already moving. They grow with speed |
| Gravity | g(q) | Torque gravity applies at each joint. Largest when the arm is stretched out sideways |
| Torque | τ | What the motors actually deliver. Forces that cannot be written as energy, like friction and contact, are added here |
Open a real robot controller and you will find this equation laid out one module per term. There is a place that computes the gravity term, a place for the Coriolis term, and a place for the inertia term; the three are summed and feedback is layered on top to produce the torque command. Because the model cancels what it knows and feedback only cleans up the remainder, this is where the textbook name computed torque and the shop-floor name gravity compensation both come from. Read chapter 5 and you have read the skeleton of most robot controllers.
Chapter 6
The third formulation is Hamiltonian mechanics, one more transformation away from the Lagrangian. It describes the state as position q and momentum p rather than position and velocity. Momentum is roughly mass times velocity.
q̇ = ∂H/∂p, ṗ = −∂H/∂q
How to read it: the change in position is set by momentum, and the change in momentum is set by how steeply H varies with position. You get two equations instead of one, but each is simpler.
H is usually called "the total energy of the system", and that comes with conditions. H equals kinetic plus potential energy only when the coordinate frame and the constraints do not change with time and the potential does not depend on velocity. Robot arms usually satisfy this. Also, motor forces and friction are not automatically present in the two equations above and must be added.
You rarely use this directly in robotics. It shows up in three places: designing symplectic integrators that conserve energy well, in optimal control theory, and as the bridge to quantum mechanics. Worth noting: the Hamiltonian in optimal control is a separate function built from the cost, not the energy described here. For now it is enough to know that a third formulation exists and it sees the world as position and momentum.
| Newtonian | Lagrangian | Hamiltonian | |
|---|---|---|---|
| Starts from | Force | Energy (kinetic − potential) | Energy (total) |
| Variables | Position and velocity per body, plus internal forces | Joint angles q and their velocities | Position q and momentum p |
| Shape of the equations | One force balance per body | One second-order equation per joint | Twice as many first-order equations |
| Good at | Force is visible. Contact force, ground reaction | Internal forces vanish, equations stay clean | The energy-conserving structure is explicit |
| In robotics | Contact and collision. Real-time controller math (Newton–Euler) | Source of the robot arm equation of motion | Rare. Integrator design, optimal control theory |
| Needed now? | Yes | Yes | Name only |
Once more: all three give the same answer to the same problem. Choosing between them is a question of whether you need the forces visible or the equations clean.
Chapter 7
A rigid body is one that does not change shape under load. The links of a robot arm, the leg segments of a quadruped, the torso of a humanoid are all treated as rigid bodies. They flex a little in reality, and that little is ignored.
Rigid-body dynamics is the field that takes those bodies connected by joints and applies the formulations from chapters 4 and 5 to compute motion over time. It is the engine room of a simulator.
A single rigid body floating in space can move six ways: three translations and three rotations. That is six degrees of freedom (DOF). Joints tie those degrees of freedom down. A six-axis arm bolted to the floor has six joints that each rotate one way, so the whole pose is captured by six numbers, the joint angles q.
The MuJoCo humanoid model on an ordinary laptop has 28 position variables. The floating torso is one free joint written as 3 position numbers plus a 4-number quaternion, so 7, and the remaining 21 are joint angles. That is what "nq = 28" means. But the actual DOF count is 27 (nv = 27). Three rotational degrees of freedom are written with four numbers, which adds one. The number of coordinates and the number of degrees of freedom are not always the same.
The chapter 5 equation M(q) q̈ + C q̇ + g = τ can be solved in either direction. Which direction you solve is what separates a simulator from a controller.
There are algorithms that do this in time proportional to the number of joints. Knowing the names is enough: ABA (articulated body algorithm) on the forward side and RNEA (recursive Newton–Euler algorithm) on the inverse side. Libraries such as MuJoCo, Pinocchio, and KDL carry them. You normally pick one rather than writing your own.
If a link genuinely flexes, the rigid-body assumption fails. Cables, silicone grippers, and thin plates do. Then you move to continuum mechanics from the chapter 1 map, and the computation gets much more expensive.
Chapter 8
Contact dynamics deals with the force that appears when two bodies touch and what happens afterwards. A foot landing, a finger closing on a cup, a part sliding into a hole are all contact. Almost every moment a robot does useful work is a contact moment. It is also the hardest part of simulation.
| Remedy | How | Good | Bad |
|---|---|---|---|
| Hard contact | Impose non-penetration as a constraint and solve every step for forces that satisfy it | Physically accurate | Expensive, and can become unstable with many constraints |
| Soft contact | Allow a little penetration and push back in proportion to the overlap, like a spring | Stable and fast | Bodies sink into each other. Wrong stiffness makes things bounce |
MuJoCo defaults to the soft side, but rather than a crude spring it solves a convex optimization each step for the forces that best satisfy the physics while keeping penetration small, which is how it gets stability and accuracy together. The values a human has to choose are the contact parameters: the friction coefficient, and the values that set how stiff the contact is and how quickly the oscillation dies out. The names differ by engine. MuJoCo specifies these through solref and solimp rather than stiffness and damping directly, and has no separate field for restitution.
"Things sink through each other in sim" and "it grips but then slips" are both contact parameter problems. There is no correct answer, so each asset gets tuned by hand, and that makes a person the bottleneck. A large share of the sim-to-real gap comes from here. That is why reinforcement learning randomizes these values during training so the policy survives any of them. That is domain randomization, and what gets randomized goes beyond friction to mass, actuator response, and signal delay.
Chapter 9
This ties chapters 4 through 8 together. A simulator cuts time into very short slices and repeats the five stages below on each slice. One turn around this loop is a physics step.
Integration is not a difficult idea. Multiply acceleration by the time slice and add it to velocity; multiply velocity by the time slice and add it to position. The order in which you do this gives the integrator its name — Euler, semi-implicit Euler, RK4 — and they differ in accuracy and stability.
Timestep is the single most important knob in a simulation. Make it large and it runs fast, but a foot can already be deep in the floor within one step, producing the penetration from chapter 8. Make it small and it is accurate but slow.
On an ordinary laptop CPU, 5,000 steps of the humanoid model took about 0.09 seconds. That is roughly 57,000 turns of the five stages per second. This model sets its timestep to 0.005 s rather than the engine default of 0.002 s, so 5,000 steps is 25 seconds of simulated time. That works out to about 280 times real time. Whenever someone quotes a real-time factor, ask what timestep the model uses. For a single robot this is more than enough; where it stops being enough is chapter 10.
Chapter 10
Reinforcement learning (RL) gives no correct answers, only a score. The robot tries millions of times inside a simulator and a policy — a network that takes an observation and emits an action — is nudged in whatever direction increases a defined reward, such as "points for moving forward without falling". Each of those attempts is some number of physics steps from chapter 9.
One robot at chapter 9 speeds takes seconds per episode, and millions of episodes turns into weeks. So you run thousands of robots at once on a GPU. Each tries something slightly different, and the results are pooled to update the policy in one shot.
From the official Isaac Lab benchmark (v2.1.0, RTX 4090 with a Ryzen 9 7950X, Isaac-Velocity-Rough-G1-v0, headless): about 82,000 environment steps per second across 4,096 environments. That figure covers policy inference and training, not just physics. It is roughly 1.5 times a single laptop MuJoCo instance, with 4,096 times as many robots.
| MuJoCo | Isaac Sim, Isaac Lab (PhysX engine) | |
|---|---|---|
| Where it runs | The base build runs on CPU and works on a laptop right away. There is also MJX, a separate GPU-parallel build | Requires an NVIDIA GPU, and one with RT cores. Even a datacenter H100 has none, so rendering will not run |
| Strength | Fast on a single robot, good for understanding and experimenting with control and RL algorithms themselves. Widely regarded as stable on contact, though accuracy still has to be checked per task | Thousands of environments in parallel on a GPU. Suited to quadruped and humanoid locomotion RL, where attempts are needed in bulk |
| Weakness | The base build is one robot at a time, so large-scale RL is slow. Using a GPU means switching to MJX | Needs hardware and a lot of configuration. What happens behind the button is invisible |
| Use it for | Getting the concepts, small experiments, seeing chapters 4 to 9 with your own eyes | Actually running locomotion RL |
Being good at Isaac and understanding simulation are different things. Get Newton, Lagrange, rigid bodies, and contact straight before you press the button and run PPO, and you will not be shaken when the tooling changes. Skip the concepts and learn only the tool, and when a result looks wrong you will not be able to tell a contact parameter problem from a timestep problem from a reward design problem. So the order is MuJoCo for the concepts, Isaac for the scale.
Chapter 11
On a real robotics team the material above splits three ways. The names differ, but all of it is in this document.
1. "What were the contact parameters in this simulation tuned against?" Without a reference, the result rests on one person's eye.
2. "Is the policy's action a torque or a target angle?" If it is a target angle there is another controller underneath, and its behaviour drives the result.
3. "What is the ratio of physics steps to policy steps?" Without that number you cannot match control rates when you move to hardware.
Chapter 12
Answer out loud, then expand to check. Eight or more and you are done here.
There is no fixed number; change the classification criteria and it subdivides indefinitely. Replace the question with "which framework best describes what I am dealing with and can actually be computed?"
They are formulations. They write the same physics from different starting points — force, energy, position and momentum — and give the same answer to the same problem.
No. It is an application area one layer down: those formulations applied to stiff bodies connected by joints. Contact dynamics is the part of it dealing with bodies touching.
Statics checks, with no time involved, that forces sum to zero and torques sum to zero. Dynamics covers the whole evolution in time once torque is applied and things accelerate.
You must draw every force on every body, and each joint drags in an unknown internal force holding the links together, so the equations blow up.
Because you take joint angles as the variables (generalized coordinates). The role of the joint reaction force is already inside the constraint "only the joint angle can change", so it never gets written down.
The inertia term (torque needed to accelerate the arm, with M the mass matrix), the Coriolis and centrifugal term (side effects that appear only when already moving), and the gravity term (torque gravity applies at each joint). The right side is the torque the motors deliver.
A simulator solves forward dynamics (torque in, motion out). A controller solves inverse dynamics (desired motion in, required torque out). Same equation, opposite directions.
It switches on and off instantly. Penetration must be prevented, but computing real stiffness directly makes the force explode. And sideways, static and kinetic friction differ, so whether it slips is itself a decision every step.
MuJoCo runs a single robot fast, which suits learning concepts and testing algorithms. Isaac runs thousands of environments on a GPU, which suits large-scale training such as locomotion RL. Tools hide what happens behind the button, so without the concepts you cannot diagnose a wrong result.
Chapter 13
Spoken aloud it can sound like "rigid control dynamics", but it is rigid-body dynamics. The word control is not in it. It is the dynamics of bodies — of the links.
Chapter 14
Rather than stopping at reading, work through these in MuJoCo on a laptop. Each step is a chance to see one chapter with your own eyes.
The natural next topic after chapter 8 is domain randomization: shaking contact parameters during training to narrow the sim-to-real gap. It is the first thing you meet when you actually run locomotion RL.