walker documentation and frep wann

02_Presentation/AI_that_grows/AI_grow.md

@@ -5,43 +5,116 @@ Research and development of workflows for the co-design reconfigurable AI softwa
 
 
 
-## [Weight Agnostic Neural Networks (WANN)](https://weightagnostic.github.io/)
+## Weight Agnostic Neural Networks (WANN)
 
+### Introduction
 
-- [Demo!](https://amiraa.pages.cba.mit.edu/physical-computing-design-tools/01_Code/physical_computing_interface/demos/indexAI.html)
+- [Paper](https://weightagnostic.github.io/)
   - "focus on finding minimal architectures".
   - "By deemphasizing learning of weight parameters, we encourage the agent instead to develop ever-growing networks that can encode acquired skills based on its interactions with the environment".
 
 
-<img src="./WANN_schematic.png" width="70%" />
-<img src="./WANN_operators.png" width="70%" />
+<img src="./WANN_schematic.png" width="75%" />
+<img src="./WANN_operators.png" width="75%" />
 
 <img src="./square_biped.png" width="35%" /><img src="./square_biped.gif" width="35%" />
 
-## Case Study Implementation: Cart-Pole Swing Up
+### Case Study: Cart-Pole Swing Up
 
-![](./swing.gif)
+<img src="./swing.gif" width="50%" />
 
 One of the most famous benchmarks of non-linear control, there is lots of approaches including standard q-learning using a discretized state space, [deep Q-learning](http://ml.informatik.uni-freiburg.de/former/_media/publications/riesmc05.pdf) or [linear Q-learning](https://pdfs.semanticscholar.org/94ef/b6e4238c078ca0c443a1a538a52a54fa1e73.pdf) with continuos state space. 
 
 WANN is interesting as it tries to get the simplest network that uses the input sensors (position, rotation and their derivatives) to the output (force). It focuses on learning **principles** and **not only tune weights**. 
 
-<img src="./swing_best.png" width="75%" />
+<img src="./swing_best.png" width="50%" />
 
 This is one of the outputs of the network and you can see because of it's simplicity it's not a black box and one can deduce the principles learnt [[1]](https://towardsdatascience.com/weight-agnostic-neural-networks-fce8120ee829):
 - the position parameter is almost directly linked to the force, there is only an inverter which means that if the cart is on right or left of the center (+- x), it always try to **go to in the opposite direction** to the center.
 - Based on the weight (shared weight between them all) it learned that one inverter is not enough, so it **doubled** it.
 - It shows that it discovered **symmetry**: Most of them pass by a gaussian filter which basically gives the same result for -x and x, which means it's agnostic to sign of the input.
 
-## Evolution and DICE Integration
 
-<img src="./evol1.gif" width="50%" />
+----
+
+## Example Implementation: Frep Search
+
+As a first step to understand the code and WANN training, I implemented a toy problem where I am trying to learn the functional representation of target image, which is, given the x,y position of every pixel in an image, try to find the distance function that represent how far is this pixel from the edge of the shape.
+
+In the following training the target shape is a circle, and the input is the x and y positions of the pixels, and it found a minimal neural network architecture (given a library of given non linear functions) that maps the input position into the target shape.
+
+Graph Evolution:
+
+<img src="./frep_evol.gif" width="60%" />
+
+Target Evolution:
+
+<img src="./frep_circle.gif" width="40%" />
+
+
+----
+## Rover and Walker Training
+
+Now that I have a good understanding how to extend and train new WANN models, I started to see what is the best strategy to train rover and walker control.
+
+First, since the MetaVoxel code is in julia and WANN library is in python, I used the [pyjulia](https://pyjulia.readthedocs.io/en/latest/usage.html) library in python to efficiently compile and run the simulation from python, and smoothly send information back and forth between them.
+
+Now that this is working, the next step is to properly formulate the problem and define the variables and the target reward function.
 
-<img src="./wann_run.png" width="50%" />
+
+### 1- Rover
+[Function Graph](https://amiraa.pages.cba.mit.edu/physical-computing-design-tools/01_Code/physical_computing_interface/graph/control/control.html), [Simulation Demo](https://amiraa.pages.cba.mit.edu/metavoxels-code/demos/indexRover.html), [Progress](https://gitlab.cba.mit.edu/amiraa/metavoxels/-/blob/master/02_Presentation/robotics/rover/rover.md)
+  
+<img src="../macro_dice/1rover4.gif" width="30%" />
+<img src="../macro_dice/1rover2.gif" width="30%" /><br/>
+<img src="../macro_dice/1rover1.gif" width="30%" />
+<img src="../macro_dice/1rover3.gif" width="30%" />
+
+- Variables
+  - Modularity and Hierarchy
+    - Number of Variables
+      - For rover, the robot has four legs, and each leg has three degrees of freedom, so the most direct way of implementing search is to assume there are **12 independent degrees of freedom**
+      - Since we already have kind of a sense that similar robots and animals usually each the two legs that move identically and independently from the other two legs, we can assume that there are **6 independent degrees of freedom**
+      - We can further assume that all legs will move in the same manner, but each two will have have a shift in phase, se we can **just reduce the degrees of freedom to 3** and find the best phase shift
+      - I will explore the three routes and find the speedup and performance in the training between the different strategies
+    - Time frame
+      - The most straight forward way is to give the robot a time series (0 => total time) and see what it can do in this time  
+      - However, since I want to the movement to be modular and extendable, I will divide the total time into N time frames and repeat the movement
+      - this should push the robot to learn cyclic smooth motions
+- Objective Function
+  - for now, I will use the max distance traveled in the x direction 
+  - next I explore steering right and left
+  - next it will be trying to follow a target
+
+### 2- Walker
+- [Progress](https://gitlab.cba.mit.edu/amiraa/metavoxels/-/blob/master/02_Presentation/robotics/walking/walking.md)
+  
+<img src="../macro_dice/walker.gif" width="60%" />
+
+- Variables
+  - Modularity and Hierarchy
+    - Number of Variables
+      - Similar to rover, I can explore the benefit of hierarchy and modularity by taking advantage of the symmetry and hierarchy in the problem
+      - Each leg has 2 independent degrees of freedom
+- Objective function
+  - this problem will be a bit harder as it has to learn ro stand first then walk
+  - I can try to make it easier by restraining some degrees of freedom
+  - I want to explore if the graphs of both behaviors are related or not
+
+
+-----
+
+## DICE Integration
+
+<!-- <img src="./evol1.gif" width="50%" /> -->
 
 - [Demo!](https://amiraa.pages.cba.mit.edu/physical-computing-design-tools/01_Code/physical_computing_interface/demos/indexAI.html)
   
-<img src="./quick.gif" width="100%" />
+<img src="./quick.gif" width="75%" />
+<img src="./wann_run.png" width="75%" />
+
+
+----
 
 ## Next Steps:
 - Implement WANN for rover

02_Presentation/macro_dice/macro_dice.md

@@ -5,7 +5,6 @@ Case study of the design and optimization of Macro-DICE.
 Hardware Implementation:
 - [VoxRob](https://gitlab.cba.mit.edu/bej/voxrob)
 - [rover](https://gitlab.cba.mit.edu/ccameron/rover)
-- [DM3](https://gitlab.cba.mit.edu/bej/dm3)
 
 
 
@@ -14,7 +13,7 @@ Hardware Implementation:
 
 <img src="./Dice_Voxrob_2.png" width="100%" />
 
-
+----
 ## Steps
 1. Find Goal
    - CNN to learn target (instructions) from Image
@@ -31,9 +30,12 @@ Hardware Implementation:
    - [Robotic assembly](https://gitlab.cba.mit.edu/amiraa/swarm-morphing/-/blob/master/Science_Robotics_Paper/LOI.md)
 
 
-
+---
 ## Simulation Environment
 
+### Rover
+
+
 - [Check Control Function Graph](https://amiraa.pages.cba.mit.edu/physical-computing-design-tools/01_Code/physical_computing_interface/graph/control/control.html)
   
 <img src="./1rover4.gif" width="40%" />
@@ -42,30 +44,41 @@ Hardware Implementation:
 <img src="./1rover3.gif" width="40%" />
 
 - [Simulation Demo](https://amiraa.pages.cba.mit.edu/metavoxels-code/demos/indexRover.html)
+- [Progress](https://gitlab.cba.mit.edu/amiraa/metavoxels/-/blob/master/02_Presentation/robotics/rover/rover.md)
 
+### Wal
 
+<img src="./walker.gif" width="80%" />
 
-## Integration
+- [Progress](https://gitlab.cba.mit.edu/amiraa/metavoxels/-/blob/master/02_Presentation/robotics/walking/walking.md)
 
-Using the [MetaVoxel simulation tool](https://gitlab.cba.mit.edu/amiraa/metavoxels):
 
-<img src="./200114_simulation.PNG" width="80%" /><br/>
+----
+
+## Optimization
+
+- Two level optimization
+  - Shape (body) => [Hierarchal Local Online Shape Optimization](https://gitlab.cba.mit.edu/amiraa/metavoxels/-/blob/master/02_Presentation/top_opt/search.md)
+  - Control (mind) => [AI that grows](../AI_that_grows/AI_grow.md)
+  - Simultaneous optimization of shape and control
+
+---
+
 
 ## Robotics Design
+Using the [MetaVoxel simulation tool](https://gitlab.cba.mit.edu/amiraa/metavoxels):
 
 <img src="./dice_assembly.gif" width="80%" /><br/>
 
+---
+
 ## Assembly
 - Option for different kinds of assembly (swarm assembly)
   
 <img src="./assembly.gif" width="80%" /> <br/>
+---
 
 
-## Optimization
 
-- Two level optimization
-  - Shape (body) => [Hierarchal Local Online Shape Optimization](https://gitlab.cba.mit.edu/amiraa/metavoxels/-/blob/master/02_Presentation/top_opt/search.md)
-  - Control (mind) => [AI that grows](../AI_that_grows/AI_grow.md)
-  - Simultaneous optimization of shape and control