323 lines
12 KiB
HTML
323 lines
12 KiB
HTML
<html>
|
|
<head>
|
|
<title>Neataptic.js - Target-seeking AI</title>
|
|
|
|
<!-- Meta data -->
|
|
<meta charset="UTF-8">
|
|
|
|
|
|
<meta name="description" content="Neural agents learn to seek targets through neuro-evolution">
|
|
|
|
|
|
|
|
<meta name="keywords" content="target seeking, AI, genetic-algorithm, NEAT, Neataptic">
|
|
|
|
|
|
|
|
<meta name="author" content="Thomas Wagenaar">
|
|
|
|
|
|
<!-- CSS -->
|
|
<link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
|
|
<link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css">
|
|
<link rel="stylesheet" type="text/css" href="../../custom.css">
|
|
|
|
<!-- Favicon -->
|
|
<link rel="shortcut icon" href="http://i.imgur.com/rPZtAOc.png">
|
|
|
|
<!-- Fonts -->
|
|
<link href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700" rel="stylesheet">
|
|
|
|
<!-- Google Analytics -->
|
|
<script async>
|
|
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
|
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
|
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
|
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
|
|
|
ga('create', 'UA-51480408-2', 'auto');
|
|
ga('send', 'pageview');
|
|
</script>
|
|
|
|
|
|
|
|
<!-- Scripts -->
|
|
<script src="https://code.jquery.com/jquery-3.2.1.min.js"></script>
|
|
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
|
|
|
|
<script type="text/x-mathjax-config">
|
|
MathJax.Hub.Config({
|
|
tex2jax: {
|
|
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
|
|
processEscapes: true
|
|
}
|
|
});
|
|
</script>
|
|
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
|
|
</head>
|
|
<body>
|
|
<!-- Fixed navbar -->
|
|
<nav class="navbar navbar-default navbar-fixed-top">
|
|
<div class="container">
|
|
<div class="navbar-header">
|
|
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
|
|
<span class="sr-only">Toggle navigation</span>
|
|
<span class="icon-bar"></span>
|
|
<span class="icon-bar"></span>
|
|
<span class="icon-bar"></span>
|
|
</button>
|
|
<a class="navbar-brand" href="../.."><p class="brand">Neataptic</p></a>
|
|
</div>
|
|
<div id="navbar" class="navbar-collapse collapse">
|
|
<ul class="nav navbar-nav">
|
|
<li class="current active"><a href="../..">Home</a></li>
|
|
<li class="current"><a href="../../docs/">Docs</a></li>
|
|
<li class="current"><a href="../">Articles</a></li>
|
|
</ul>
|
|
<ul class="nav navbar-nav navbar-right">
|
|
<li><a href="https://github.com/wagenaartje/neataptic" target="blank"><i class="fa fa-github fa-1x"></i><b> GITHUB</b></a></li>
|
|
</ul>
|
|
</div>
|
|
</div>
|
|
</nav>
|
|
|
|
|
|
<div class="container" style="margin-top: 60px">
|
|
<div class="row">
|
|
<!-- Sidebar -->
|
|
<div class="col-sm-2 col-md-2 col-lg-2">
|
|
|
|
<ul id="sidebar" class="nav">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<li class="">
|
|
<a href="../neuroevolution/">Neuroevolution</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
<li class="active">
|
|
<a href="./">Target-seeking AI</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
<li class="">
|
|
<a href="../agario/">Agar.io AI</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
<li class="">
|
|
<a href="../classifycolors/">Classify colors</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
<li class="">
|
|
<a href="../playground/">Playground</a>
|
|
</li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<!-- Content -->
|
|
<div class="col-sm-10 col-md-10 col-lg-10 content">
|
|
<h1 style="margin-bottom: 0px;">Target-seeking AI</h1>
|
|
<p><a style="text-decoration: none" href="https://github.com/wagenaartje/neataptic/edit/master/mkdocs/templates/articles\targetseeking.md"><i class="fa fa-github fa-1x"></i> Edit on Github</a></p>
|
|
<hr>
|
|
<p>In the simulation below, neural networks that have been evolved through roughly
|
|
100 generations try to seek a target. Their goal is to stay as close to the target
|
|
as possible at all times. If you want to see how one of these networks looks like,
|
|
check out the <a href="https://wagenaartje.github.io/target-seeking-ai/">complete simulation</a>.</p>
|
|
<div id="field" height="500px">
|
|
</div>
|
|
|
|
<p><em>Click on the field to relocate the target! Source code <a href="https://wagenaartje.github.io/target-seeking-ai/">here</a>.</em></p>
|
|
<p>The neural agents are actually performing really well. At least one agent will have
|
|
'solved the problem' after roughly 20 generations. That is because the base of the solution
|
|
is quite easy: one of the inputs of the neural networks is the angle to the target, so all it
|
|
has to do is output some value that is similar to this input value. This can easily be done
|
|
through the identity activation function, but surprisingly, most agents in the simulation above
|
|
tend to avoid this function.</p>
|
|
<p>You can check out the topology of the networks <a href="https://wagenaartje.github.io/target-seeking-ai/">here</a>.
|
|
If you manage to evolve the genomes quicker or better than this simulation with different settings, please
|
|
perform a pull request on <a href="https://wagenaartje.github.io/target-seeking-ai/">this</a> repo.</p>
|
|
<h3 id="the-making-of">The making of</h3>
|
|
<p>In the previous article I have gone more into depth on the environment of the algorithm, but in this article
|
|
I will focus more on the settings and inputs/outputs of the algorithm itself.</p>
|
|
<p>If you have any questions about the code in the linked repo, please create an issue on <a href="https://github.com/wagenaartje/neataptic">this</a> repo.</p>
|
|
<h3 id="the-agents">The agents</h3>
|
|
<p>The agents' task is very simple. They have to get in the vicinity of the target which is set to about
|
|
100 pixels, once they are in that vicinity, each agents' score will be increased proportionally `(100 - dist)``
|
|
to the distance. There is one extra though: for every node in the agents' network, the score of the agent will
|
|
be decreased. This has two reasons; 1. networks shouldn't overfit the solution and 2. having smaller networks
|
|
reduces computation power.</p>
|
|
<p>Agents have some kind of momentum. They don't have mass, but they do have acceleration, so it takes a small
|
|
amount of time for a agent to reach the top speed in a certain direction.</p>
|
|
<p><strong>Each agent has the following inputs</strong>:</p>
|
|
<ul>
|
|
<li>Its own speed in the x-axis</li>
|
|
<li>Its own speed in the y-axis</li>
|
|
<li>The targets' speed in the x-axis</li>
|
|
<li>The targets' speed in the y-axis</li>
|
|
<li>The angle towards the target</li>
|
|
<li>The distance to the target</li>
|
|
</ul>
|
|
<p>The output of each agent is just the desired movement direction.</p>
|
|
<p>There is no kind of collision, except for the walls of the fields. In the future, it might be interesting to
|
|
add collisions between multiple agents and/or the target to reveal some new tactics. This would require the
|
|
agent to know the location of surrounding agents.</p>
|
|
<h3 id="the-target">The target</h3>
|
|
<p>The target is fairly easy. It's programmed to switch direction every now and then by a random amount. There
|
|
is one important thing however: <em>the target moves with half the speed of the agents</em>, this makes sure
|
|
that agents always have the ability to catch up with the target. Apart from that, the physics for the target
|
|
are similar to the agents' physics.</p>
|
|
<h3 id="the-genetic-algorithm">The genetic algorithm</h3>
|
|
<p>The genetic algorithm is the core of the AI. In the first frame, a certain
|
|
amount of players are initialized with a neural network as brain. The brains
|
|
represent the population of a generation. These brains are then evolved by
|
|
putting the entire population in óne playing field and letting them compete
|
|
against each other. The fittest brains are moved on the next generation,
|
|
the less fit brains have a high chance of being removed.</p>
|
|
<pre><code class="javascript">// Networks shouldn't get too big
|
|
for(var genome in neat.population){
|
|
genome = neat.population[genome];
|
|
genome.score -= genome.nodes.length * SCORE_RADIUS / 10;
|
|
}
|
|
|
|
// Sort the population by score
|
|
neat.sort();
|
|
|
|
// Draw the best genome
|
|
drawGraph(neat.population[0].graph($('.best').width(), $('.best').height()), '.best', false);
|
|
|
|
// Init new pop
|
|
var newPopulation = [];
|
|
|
|
// Elitism
|
|
for(var i = 0; i < neat.elitism; i++){
|
|
newPopulation.push(neat.population[i]);
|
|
}
|
|
|
|
// Breed the next individuals
|
|
for(var i = 0; i < neat.popsize - neat.elitism; i++){
|
|
newPopulation.push(neat.getOffspring());
|
|
}
|
|
|
|
// Replace the old population with the new population
|
|
neat.population = newPopulation;
|
|
neat.mutate();
|
|
|
|
neat.generation++;
|
|
startEvaluation();
|
|
</code></pre>
|
|
|
|
<p>The above code shows the code run when the evaluation is finished. It is very similar
|
|
to the built-in <code>evolve()</code> function of Neataptic, however adapted to avoid a fitness
|
|
function as all genomes must be evaluated at the same time.</p>
|
|
<p>The scoring of the genomes is quite easy: when a certain amount of iterations has been reached,
|
|
each genome is ranked by their final score. Genomes with a higher score have a small amount of nodes
|
|
and have been close to the target throughout the iteration.</p>
|
|
<p><strong>Some (configurable) settings</strong>:</p>
|
|
<ul>
|
|
<li>An evaluation runs for 250 frames</li>
|
|
<li>The mutation rate is 0.3</li>
|
|
<li>The elitism is 10%</li>
|
|
<li>Each genome starts with 0 hidden nodes</li>
|
|
<li>All mutation methods are allowed</li>
|
|
</ul>
|
|
<h3 id="issuesfuture-improvements">Issues/future improvements</h3>
|
|
<ul>
|
|
<li>... none yet! <a href="https://github.com/wagenaartje/neataptic">Tell me your ideas!</a></li>
|
|
</ul>
|
|
<p><strong>Forks</strong></p>
|
|
<ul>
|
|
<li><a href="https://corpr8.github.io/neataptic-targetseeking-tron/">corpr8's fork</a>
|
|
gives each neural agent its own acceleration, as well as letting each arrow
|
|
remain in the same place after each generation. This creates a much more
|
|
'fluid' process.</li>
|
|
</ul>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
<!-- Javascript (only for blogs) -->
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<script src="../../js/articles/target-seekingai/import.js"></script>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<script>
|
|
$("table").each(function(){
|
|
$(this).addClass('table table-striped');
|
|
});
|
|
$(".current").each(function(){
|
|
var text = $(this).text().toLowerCase();
|
|
if(window.location.pathname.indexOf(text) >= 0){
|
|
$('.active').removeClass('active');
|
|
$(this).addClass('active');
|
|
}
|
|
})
|
|
</script>
|
|
</body> |