satori/sakuya
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

整理桌面

Browse Source
This commit is contained in:
satori
2022-04-10 00:37:53 +08:00
parent 82e3f2623f
commit e25c8bb318
728 changed files with 986384 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

109
node_modules/neataptic/mkdocs/templates/docs/tutorials/evolution.md generated vendored Normal file
View File

@@ -0,0 +1,109 @@
description: A simple tutorial on how to get started on evolving neural networks with Neataptic
authors: Thomas Wagenaar
keywords: evolution, machine-learning, neural-network, neuro-evolution, neat
Neuro-evolution is something that is fairly underused in the machine learning
community. It is quite interesting to see new architectures develop for
complicated problems. In this guide I will tell you how to set up a simple
supervised neuro-evolution process. If you want to do an unsupervised
neuro-evolution process, head over to the [NEAT](../neat.md) page.
### The training set
You always have to supply a training set for supervised learning. First of all,
you should normalize your data. That means you have to convert all your input and
output data to values within a range of `0` to `1`. If you are unsure how to do
this, visit the [Normalization](normalization.md) tutorial. Each training sample
in your training set should be an object that looks as follows:
```javascript
{ input: [], output: [] }
```
So an example of a training set would be (XOR):
```javascript
var myTrainingSet = [
{ input: [0,0], output: [0] },
{ input: [0,1], output: [1] },
{ input: [1,0], output: [1] },
{ input: [1,1], output: [0] }
];
```
### The network architecture
You can start off with _any_ architecture. You can even use the evolution process
to optimize already trained networks. However, I advise you to start with an empty
network, as originally described in the [NEAT](http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf)
paper. The constructor of an empty network is as follows:
```javascript
var myNetwork = new Network(inputSize, outputSize);
```
So for the training set I made above, the network can be constructed as follows:
```javascript
var myNetwork = new Network(2, 1); // 2 inputs, 1 output
```
Now we have our network. We don't have to tinker around anymore; the evolution
process will do that _for_ us.
### Evolving the network
Be warned: there are _a lot_ of options involved in the evolution of a process.
It's a matter of trial and error before you reach options that work really well.
Please note that most options are optional, as default values have been configured.
Please note that the `evolve` function is an `async` function, so you need to wrap
it in an async function to call `await`.
The evolve function is as follows:
```javascript
yourNetwork.evolve(yourData, yourOptions);
```
Check out the evolution options [here](../architecture/network.md) and [here](../neat.md). I'm going to use the following options to evolve the network:
* `mutation: methods.mutation.FFW` - I want to solve a feed forward problem, so I supply all feed forward mutation methods. More info [here](../methods/mutation.md).
* `equal: true` - During the crossover process, parent networks will be equal. This allows the spawning of new network architectures more easily.
* `popsize: 100` - The default population size is 50, but 100 worked better for me.
* `elitism: 10` - I want to keep the fittest 10% of the population to the next generation without breeding them.
* `log: 10` - I want to log the status every 10 iterations.
* `error: 0.03` - I want the evolution process when the error is below 0.03;
* `iterations: 1000` - I want the evolution process to stop after 1000 iterations if the target error hasn't been reached yet.
* `mutationRate: 0.5` - I want to increase the mutation rate to surpass possible local optima.
So let's put it all together:
```javascript
await myNetwork.evolve(myTrainingSet, {
mutation: methods.mutation.FFW,
equal: true,
popsize: 100,
elitism: 10,
log: 10,
error: 0.03,
iterations: 1000,
mutationRate: 0.5
});
// results: {error: 0.0009000000000000001, generations: 255, time: 1078}
// please note that there is a hard local optima that has to be beaten
```
Now let us check if it _actually_ works:
```javascript
myNetwork.activate([0,0]); // [0]
myNetwork.activate([0,1]); // [1]
myNetwork.activate([1,0]); // [1]
myNetwork.activate([1,1]); // [0]
```
Notice how precise the results are. That is because the evolution process makes
full use of the diverse activation functions. It actually uses the [Activation.STEP](../methods/activation.md)
function to get a binary `0` and `1` output.
### Help
If you need more help, feel free to create an issue [here](https://github.com/wagenaartje/neataptic/issues)!

108
node_modules/neataptic/mkdocs/templates/docs/tutorials/normalization.md generated vendored Normal file
View File

@@ -0,0 +1,108 @@
description: A tutorial on how to standardize/normalize your data for neural networks
authors: Thomas Wagenaar
keywords: normalize, 0, 1, standardize, input, output, neural-network
Although Neataptic networks accepts non-normalized values as input, normalizing your input makes your network converge faster. I see a lot of questions where people ask how to normalize their data correctly, so I decided to make a guide.
### Example data
You have gathered this information, now you want to use it to train/activate a neural network:
```javascript
{ stock: 933, sold: 352, price: 0.95, category: 'drinks', id: 40 }
{ stock: 154, sold: 103, price: 5.20, category: 'foods', id: 67 }
{ stock: 23, sold: 5, price: 121.30, category: 'electronics', id: 150 }
```
So some information on the above data:
* `stock`: the amount of this item in stock
* `sold`: the amount of this item sold ( in the last month )
* `price`: the price of this item
* `category`: the type of product
* `id`: the id of the product
### Normalize
So we want to represent each of these inputs as a number between `0` and `1`, however, we can not change the relativity between the values. So we need to treat each different input the same (`stock` gets treated the same for every item for example).
We have two types of values in our input data: numerical values and categorical values. These should always be treated differently.
#### Numerical values
Numerical values are values where the distance between two values matters. For example, `price: 0.95` is twice as small as `price: 1.90`. But not all integers/decimals are numerical values. Id's are often represented with numbers, but there is no relation between `id: 4` and `id: 8` . So these should be treated as categorical values.
Normalizing numerical values is quite easy, we just need to determine a maximum value we divide a certain input with. For example, we have the following data:
```javascript
stock: 933
stock: 154
stock: 23
```
We need to choose a value which is `>= 933` with which we divide all the `stock` values. We could choose `933`, but what if we get new data, where the `stock` value is higher than `933`? Then we have to renormalize all the data and retrain the network.
So we need to choose a value that is `>=933`, but also `>= future values` and it shouldn't be a too big number. We could make the assumption that the `stock` will never get larger than `2000`, so we choose `2000` as our maximum value. We now normalize our data with this maximum value:
```javascript
// Normalize the data with a maximum value (=2000)
stock: 933 -> 933/2000 -> 0.4665
stock: 154 -> 154/2000 -> 0.077
stock: 23 -> 23/2000 -> 0.0115
```
#### Categorical data
Categorical data shows no relation between different categories. So each category should be treated as a seperate input, this is called [one-hot encoding](https://en.wikipedia.org/wiki/One-hot). Basically, you create a seperate input for each category. You set all the inputs to `0`, except for the input which matches the sample category. This is one-hot encoding for our above training data:
<table class="table table-striped">
<thead>
<tr>
<th>Sample</th>
<th>Drinks</th>
<th>Foods</th>
<th>Electronics</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>1</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<td>2</td>
<td>0</td>
<td>1</td>
<td>0</td>
</tr>
<tr>
<td>3</td>
<td>0</td>
<td>0</td>
<td>1</td>
</tr>
</tbody>
</table>
But this also allows the addition of new categories over time: you just a need input. It has no effect on the performances of the network on the past training data as when the new category is set to `0`, it has no effect (`weight * 0 = 0`).
### Normalized data
So applying what I have explained above creates our normalized data, note that the relativity between inputs has not been changed. Also note that some values may be rounded in the table below.
```javascript
{ stock: 0.4665, sold: 0.352, price: 0.00317, drinks: 1, foods: 0, electronics: 0, id40: 1, id67: 0, id150: 0 }
{ stock: 0.077, sold: 0.103, price: 0.01733, drinks: 0, foods: 1, electronics: 0, id40: 0, id67: 1, id150: 0 }
{ stock: 0.0115, sold: 0.005, price: 0.40433, drinks: 0, foods: 0, electronics: 1, id40: 0, id67: 0, id150: 1 }
```
Max values:
* `stock`: 2000
* `sold`: 1000
* `price` : 300
Please note, that these inputs should be provided in arrays for neural networks in Neataptic:
```javascript
[ 0.4665, 0.352, 0.00317, 1, 0, 0, 1, 0, 0 ]
[ 0.77, 0.103, 0.01733, 0, 1, 0, 0, 1, 0 ]
[ 0.0115, 0.005, 0.40433, 0, 0, 1, 0, 0, 1 ]
```

81
node_modules/neataptic/mkdocs/templates/docs/tutorials/training.md generated vendored Normal file
View File

@@ -0,0 +1,81 @@
description: A simple tutorial on how to get started on training neural networks with Neataptic
authors: Thomas Wagenaar
keywords: training, machine-learning, neural-network, backpropagation
Training your network is not that hard to do - it's the preparation that is harder.
### The training set
First of all, you should normalize your data. That means you have to convert all your input and output data to values within a range of `0` to `1`. If you are unsure how to do this, visit the [Normalization](normalization.md) page. Each training sample in your training set should be an object that looks as follows:
```javascript
{ input: [], output: [] }
```
So an example of a training set would be (XOR):
```javascript
var myTrainingSet = [
{ input: [0,0], output: [0] },
{ input: [0,1], output: [1] },
{ input: [1,0], output: [1] },
{ input: [1,1], output: [0] }
];
```
### The network architecture
There is no fixed rule of thumb for choosing your network architecture. Adding more layers makes your neural network recognize more abstract relationships, although it requires more computation. Any function can be mapped with just one (big) hidden layer, but I do not advise this. I advise to use any of the following architectures if you're a starter:
* [Perceptron](../builtins/perceptron.md) - a fully connected feed forward network
* [LSTM](../builtins/lstm.md) - a recurrent network that can recognize patterns over very long time lags between inputs.
* [NARX](../builtins/narx.md) - a recurrent network that remembers previous inputs and outputs
But for most problems, a perceptron is sufficient. Now you only have to determine the size and amount of the network layers. I advise you to take a look at this [StackExchange question](https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw) for more help on deciding the hidden size.
For the training set I provided above (XOR), there are only 2 inputs and one output. I use the rule of thumb: `input size + output size = hidden size`. So the creation of my network would like this:
```javascript
myNetwork = architect.Perceptron(2, 3, 1);
```
### Training the network
Finally: we're going to train the network. The function for training your network is very straight-forward:
```javascript
yourNetwork.train(yourData, yourOptions);
```
There are _a lot_ of options. I won't go over all of them here, but you can check out the [Network wiki](../architecture/network.md) for all the options.
I'm going to use the following options:
* `log: 10` - I want to log the status every 10 iterations
* `error: 0.03` - I want the training to stop if the error is below 0.03
* `iterations: 1000` - I want the training to stop if the error of 0.03 hasn't been reached after 1000 iterations
* `rate: 0.3` - I want a learning rate of 0.3
So let's put it all together:
```javascript
myNetwork.train(myTrainingSet, {
log: 10,
error: 0.03,
iterations: 1000,
rate: 0.3
});
// result: {error: 0.02955628620843985, iterations: 566, time: 31}
```
Now let us check if it _actually_ works:
```javascript
myNetwork.activate([0,0]); // [0.1257225731473885]
myNetwork.activate([0,1]); // [0.9371910625522613]
myNetwork.activate([1,0]); // [0.7770757408042104]
myNetwork.activate([1,1]); // [0.1639697315652196]
```
And it works! If you want it to be more precise, lower the target error.
### Help
If you need more help, feel free to create an issue [here](https://github.com/wagenaartje/neataptic/issues)!

11
node_modules/neataptic/mkdocs/templates/docs/tutorials/tutorials.md generated vendored Normal file
View File

@@ -0,0 +1,11 @@
In order to get you started on Neataptic, a few tutorials have been written that
give a basic overview on how to create, train and evolve your networks. It is
recommended to read all of them before you start digging in your own project.
* [Training](training.md)
* [Evolution](evolution.md)
* [Normalization](normalization.md)
* [Visualization](visualization.md)
If you have any questions, feel free to create an issue [here](https://github.com/wagenaartje/neataptic/issues).
If you feel like anything is missing, feel free to create a pull request!

82
node_modules/neataptic/mkdocs/templates/docs/tutorials/visualization.md generated vendored Normal file
View File

@@ -0,0 +1,82 @@
description: A simple tutorial on how to visualize neural-networks in Neataptic
authors: Thomas Wagenaar
keywords: graph, model, neural-network, visualize, nodes, connections
This is a step-by-step tutorial aimed to teach you how to create and visualise neural networks using Neataptic.
**Step 1**
Create a javascript file. Name it anything you want. But make sure to start it off with the following:
```javascript
var Node = neataptic.Node;
var Neat = neataptic.Neat;
var Network = neataptic.Network;
var Methods = neataptic.Methods;
var Architect = neataptic.Architect;
```
This makes the whole developing a whole lot easier
**Step 2**
Create a html file. Copy and paste this template if you want:
```html
<html>
<head>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://marvl.infotech.monash.edu/webcola/cola.v3.min.js"></script>
<script src="https://rawgit.com/wagenaartje/neataptic/master/dist/neataptic.js"></script>
<script src="https://rawgit.com/wagenaartje/neataptic/master/graph/graph.js"></script>
<link rel="stylesheet" type="text/css" href="https://rawgit.com/wagenaartje/neataptic/master/graph/graph.css">
</head>
<body>
<div class="container">
<div class="row">
<svg class="draw" width="1000px" height="1000px"/>
</div>
</div>
<script src="yourscript.js"></script>
</body>
</html>
```
**Step 3** Create a network. You can do that in any of the following ways:
```javascript
var network = architect.Random(2, 20, 2, 2);
```
```javascript
var network = architect.Perceptron(2, 10, 10, 2);
```
Or if you want to be more advanced, construct your own:
```javascript
var A = new Node();
var B = new Node();
var C = new Node();
var D = new Node();
var E = new Node();
var F = new Node();
var nodes = [A, B, C, D, E, F];
for(var i = 0; i < nodes.length-1; i++){
node = nodes[i];
for(var j = 0; j < 2; j++){
var connectTo = nodes[Math.floor(Math.random() * (nodes.length - i) + i)];
node.connect(connectTo);
}
}
var network = architect.Construct(nodes);
```
**Step 4** Retrieve data and draw a graph
```javascript
drawGraph(network.graph(1000, 1000), '.draw');
```
See a working example [here](https://jsfiddle.net/ch8s9d3e/30/)!
See more info on graphs [here](https://github.com/wagenaartje/neataptic/tree/master/graph)!