Skip to content
Snippets Groups Projects
AI_grow.md 2.81 KiB
Newer Older
  • Learn to ignore specific revisions
  • Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    # AI that Grows
    
    Research and development of workflows for the co-design reconfigurable AI software and hardware.
    
    
    ## [Weight Agnostic Neural Networks (WANN)](https://weightagnostic.github.io/)
    
    - "focus on finding minimal architectures".
    - "By deemphasizing learning of weight parameters, we encourage the agent instead to develop ever-growing networks that can encode acquired skills based on its interactions with the environment".
    
    
    
    Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    <img src="./WANN_schematic.png" width="70%" />
    <img src="./WANN_operators.png" width="70%" />
    
    <img src="./square_biped.png" width="35%" /><img src="./square_biped.gif" width="35%" />
    
    Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    
    ## Case Study Implementation: Cart-Pole Swing Up
    
    ![](./swing.gif)
    
    One of the most famous benchmarks of non-linear control, there is lots of approaches including standard q-learning using a discretized state space, [deep Q-learning](http://ml.informatik.uni-freiburg.de/former/_media/publications/riesmc05.pdf) or [linear Q-learning](https://pdfs.semanticscholar.org/94ef/b6e4238c078ca0c443a1a538a52a54fa1e73.pdf) with continuos state space. 
    
    WANN is interesting as it tries to get the simplest network that uses the input sensors (position, rotation and their derivatives) to the output (force). It focuses on learning **principles** and **not only tune weights**. 
    
    
    Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    <img src="./swing_best.png" width="75%" />
    
    Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    
    This is one of the outputs of the network and you can see because of it's simplicity it's not a black box and one can deduce the principles learnt [[1]](https://towardsdatascience.com/weight-agnostic-neural-networks-fce8120ee829):
    - the position parameter is almost directly linked to the force, there is only an inverter which means that if the cart is on right or left of the center (+- x), it always try to **go to in the opposite direction** to the center.
    - Based on the weight (shared weight between them all) it learned that one inverter is not enough, so it **doubled** it.
    - It shows that it discovered **symmetry**: Most of them pass by a gaussian filter which basically gives the same result for -x and x, which means it's agnostic to sign of the input.
    
    ## Evolution and DICE Integration
    
    
    Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    <img src="./evol1.gif" width="50%" />
    
    Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    
    
    Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    <img src="./wann_run.png" width="50%" />
    
    Amira Abdel-Rahman's avatar
    Amira Abdel-Rahman committed
    
    
    <img src="./quick.gif" width="100%" />
    
    ## Next Steps:
    - Implement WANN for rover
    - Beyond WANN
      - [Structure Learning](http://fab.cba.mit.edu/classes/864.20/people/amira/project/dynamic_system_optimization.html)
        - Information gain using the ensemble method
          - [Notebook](http://fab.cba.mit.edu/classes/864.20/people/amira/project/examples/Pendulum_Ensemble.html)
        - Data driven differential equations
          - [Sparse Identification of Nonlinear Dynamics](https://arxiv.org/abs/1906.10612)
          - [Notebook](http://fab.cba.mit.edu/classes/864.20/people/amira/project/examples/Pendulum_Ensemble.html)
    - Integrate shape and control search