satori/sakuya
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

整理桌面

Browse Source
This commit is contained in:
satori
2022-04-10 00:37:53 +08:00
parent 82e3f2623f
commit e25c8bb318
728 changed files with 986384 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
node_modules/neataptic/mkdocs/templates/docs/architecture/architecture.md generated vendored Normal file
View File

@@ -0,0 +1,10 @@
If you want to built your completely custom neural network; this is the place to be.
You can build your network from the bottom, using nodes and connections, or you can
ease up the process by using groups and layers.
* [Connection](connection.md)
* [Node](node.md)
* [Group](group.md)
* [Layer](layer.md)
* [Network](network.md)
* [Construct](construct.md)

30
node_modules/neataptic/mkdocs/templates/docs/architecture/connection.md generated vendored Normal file
View File

@@ -0,0 +1,30 @@
description: Documentation of the Connection instance in Neataptic
authors: Thomas Wagenaar
keywords: connection, neural-network, architecture, synapse, weight
A connection instance defines the connection between two nodes. All you have to do is pass on a from and to node, and optionally a weight.
```javascript
var B = new Node();
var C = new Node();
var connection = new Connection(A, B, 0.5);
```
Connection properties:
Property | contains
-------- | --------
from | connection origin node
to | connection destination node
weight | the weight of the connection
gater | the node gating this connection
gain | for gating, gets multiplied with weight
### Connection methods
There are three connection methods:
* **methods.connection.ALL_TO_ALL** connects all nodes from group `x` to all nodes from group `y`
* **methods.connection.ALL_TO_ELSE** connects every node from group `x` to all nodes in the same group except itself
* **methods.connection.ONE_TO_ONE** connects every node in group `x` to one node in group `y`
Every one of these connection methods can also be used on the group itself! (`x.connect(x, METHOD)`)

47
node_modules/neataptic/mkdocs/templates/docs/architecture/construct.md generated vendored Normal file
View File

@@ -0,0 +1,47 @@
description: Documentation on how to construct your own network with Neataptic
authors: Thomas Wagenaar
keywords: neural-network, architecture, node, build, connection
For example, I want to have a network that looks like a square:
```javascript
var A = new Node();
var B = new Node();
var C = new Node();
var D = new Node();
// Create connections
A.connect(B);
A.connect(C);
B.connect(D);
C.connect(D);
// Construct a network
var network = architect.Construct([A, B, C, D]);
```
And voila, basically a square, but stretched out, right?
![Square](https://i.gyazo.com/c91f9ce9df69f6e085535a642355b88a.png)
The `construct()` function looks for nodes that have no input connections, and labels them as an input node. The same for output nodes: it looks for nodes without an output connection (and gating connection), and labels them as an output node!
**You can also create networks with groups!** This speeds up the creation process and saves lines of code.
```javascript
// Initialise groups of nodes
var A = new Group(4);
var B = new Group(2);
var C = new Group(6);
// Create connections between the groups
A.connect(B);
A.connect(C);
B.connect(C);
// Construct a network
var network = architect.Construct([A, B, C, D]);
```
Keep in mind that you must always specify your input groups/nodes in **activation order**. Input and output nodes will automatically get sorted out, but all hidden nodes will be activated in the order that they were given.

115
node_modules/neataptic/mkdocs/templates/docs/architecture/group.md generated vendored Normal file
View File

@@ -0,0 +1,115 @@
description: Documentation of the Group instance in Neataptic
authors: Thomas Wagenaar
keywords: group, neurons, nodes, neural-network, activation
A group instance denotes a group of nodes. Beware: once a group has been used to construct a network, the groups will fall apart into individual nodes. They are purely for the creation and development of networks. A group can be created like this:
```javascript
// A group with 5 nodes
var A = new Group(5);
```
Group properties:
Property | contains
-------- | --------
nodes | an array of all nodes in the group
connections | dictionary with connections
### activate
Will activate all the nodes in the network.
```javascript
myGroup.activate();
// or (array length must be same length as nodes in group)
myGroup.activate([1, 0, 1]);
```
### propagate
Will backpropagate all nodes in the group, make sure the group receives input from another group or node!
```javascript
myGroup.propagate(rate, momentum, target);
```
The target argument is optional. An example would be:
```javascript
var A = new Group(2);
var B = new Group(3);
A.connect(B);
A.activate([1,0]); // set the input
B.activate(); // get the output
// Then teach the network with learning rate and wanted output
B.propagate(0.3, 0.9, [0,1]);
```
The default value for momentum is `0`. Read more about momentum on the
[regularization page](../methods/regularization.md).
### connect
Creates connections between this group and another group or node. There are different connection methods for groups, check them out [here](connection.md).
```javascript
var A = new Group(4);
var B = new Group(5);
A.connect(B, methods.connection.ALL_TO_ALL); // specifying a method is optional
```
### disconnect
(not yet implemented)
### gate
Makes the nodes in a group gate an array of connections between two other groups. You have to specify a gating method, which can be found [here](../methods/gating.md).
```javascript
var A = new Group(2);
var B = new Group(6);
var connections = A.connect(B);
var C = new Group(2);
// Gate the connections between groups A and B
C.gate(connections, methods.gating.INPUT);
```
### set
Sets the properties of all nodes in the group to the given values, e.g.:
```javascript
var group = new Group(4);
// All nodes in 'group' now have a bias of 1
group.set({bias: 1});
```
### disconnect
Disconnects the group from another group or node. Can be twosided.
```javascript
var A = new Group(4);
var B = new Node();
// Connect them
A.connect(B);
// Disconnect them
A.disconnect(B);
// Twosided connection
A.connect(B);
B.connect(A);
// Disconnect from both sides
A.disconnect(B, true);
```
### clear
Clears the context of the group. Useful for predicting timeseries with LSTM's.

77
node_modules/neataptic/mkdocs/templates/docs/architecture/layer.md generated vendored Normal file
View File

@@ -0,0 +1,77 @@
description: Documentation of the Layer instance in Neataptic
authors: Thomas Wagenaar
keywords: LSTM, GRU, architecture, neural-network, recurrent
Layers are pre-built architectures that allow you to combine different network
architectures into óne network. At this moment, there are 3 layers (more to come soon!):
```javascript
Layer.Dense
Layer.LSTM
Layer.GRU
Layer.Memory
```
Check out the options and details for each layer below.
### Constructing your own network with layers
You should always start your network with a `Dense` layer and always end it with
a `Dense` layer. You can connect layers with each other just like you can connect
nodes and groups with each other. This is an example of a custom architecture
built with layers:
```javascript
var input = new Layer.Dense(1);
var hidden1 = new Layer.LSTM(5);
var hidden2 = new Layer.GRU(1);
var output = new Layer.Dense(1);
// connect however you want
input.connect(hidden1);
hidden1.connect(hidden2);
hidden2.connect(output);
var network = architect.Construct([input, hidden1, hidden2, output]);
```
### Layer.Dense
The dense layer is a regular layer.
```javascript
var layer = new Layer.Dense(size);
```
### Layer.LSTM
The LSTM layer is very useful for detecting and predicting patterns over long
time lags. This is a recurrent layer. More info? Check out the [LSTM](../builtins/lstm.md) page.
```javascript
var layer = new Layer.LSTM(size);
```
Be aware that using `Layer.LSTM` is worse than using `architect.LSTM`. See issue [#25](https://github.com/wagenaartje/neataptic/issues/25).
### Layer.GRU
The GRU layer is similar to the LSTM layer, however it has no memory cell and only
two gates. It is also a recurrent layer that is excellent for timeseries prediction.
More info? Check out the [GRU](../builtins/gru.md) page.
```javascript
var layer = new Layer.GRU(size);
```
### Layer.Memory
The Memory layer is very useful if you want your network to remember a number of
previous inputs in an absolute way. For example, if you set the `memory` option to
3, it will remember the last 3 inputs in the same state as they were inputted.
```javascript
var layer = new Layer.Memory(size, memory);
```
The input layer to the memory layer should always have the same size as the memory size.
The memory layer will output a total of `size * memory` values.
> This page is incomplete. There is no description on the functions you can use
on this instance yet. Feel free to add the info (check out src/layer.js)

269
node_modules/neataptic/mkdocs/templates/docs/architecture/network.md generated vendored Normal file
View File

@@ -0,0 +1,269 @@
description: Documentation of the network model in Neataptic
authors: Thomas Wagenaar
keywords: neural-network, recurrent, layers, neurons, connections, input, output, activation
Networks are very easy to create. All you have to do is specify an `input` size and an `output` size.
```javascript
// Network with 2 input neurons and 1 output neuron
var myNetwork = new Network(2, 1);
// If you want to create multi-layered networks
var myNetwork = new architect.Perceptron(5, 20, 10, 5, 1);
```
If you want to create more advanced networks, check out the 'Networks' tab on the left.
### Functions
Check out the [train](../important/train.md) and [evolve](../important/evolve.md) functions on their separate pages!
<details>
<summary>activate</summary>
Activates the network. It will activate all the nodes in activation order and produce an output.
<pre>
// Create a network
var myNetwork = new Network(3, 2);
myNetwork.activate([0.8, 1, 0.21]); // gives: [0.49, 0.51]
</pre>
</details>
<details>
<summary>noTraceActivate</summary>
Activates the network. It will activate all the nodes in activation order and produce an output.
Does not calculate traces, so backpropagation is not possible afterwards. That makes
it faster than the regular `activate` function.
<pre>
// Create a network
var myNetwork = new Network(3, 2);
myNetwork.noTraceActivate([0.8, 1, 0.21]); // gives: [0.49, 0.51]
</pre>
</details>
<details>
<summary>propagate</summary>
This function allows you to teach the network. If you want to do more complex
training, use the <code>network.train()</code> function. The arguments for
this function are:
<pre>
myNetwork.propagate(rate, momentum, update, target);
</pre>
Where target is optional. The default value of momentum is `0`. Read more about
momentum on the [regularization page](../methods/regularization.md). If you run
propagation without setting update to true, then the weights won't update. So if
you run propagate 3x with `update: false`, and then 1x with `update: true` then
the weights will be updated after the last propagation, but the deltaweights of
the first 3 propagation will be included too.
<pre>
var myNetwork = new Network(1,1);
// This trains the network to function as a NOT gate
for(var i = 0; i < 1000; i++){
network.activate([0]);
network.propagate(0.2, 0, true, [1]);
network.activate([1]);
network.propagate(0.3, 0, true, [0]);
}
</pre>
The above example teaches the network to output <code>[1]</code> when input <code>[0]</code> is given and the other way around. Main usage:
<pre>
network.activate(input);
network.propagate(learning_rate, momentum, update_weights, desired_output);
</pre>
</details>
<details>
<summary>merge</summary>
The merge functions takes two networks, the output size of <code>network1</code> should be the same size as the input of <code>network2</code>. Merging will always be one to one to conserve the purpose of the networks. Usage:
<pre>
var XOR = architect.Perceptron(2,4,1); // assume this is a trained XOR
var NOT = architect.Perceptron(1,2,1); // assume this is a trained NOT
// combining these will create an XNOR
var XNOR = Network.merge(XOR, NOT);
</pre>
</details>
<details>
<summary>connect</summary>
Connects two nodes in the network:
<pre>
myNetwork.connect(myNetwork.nodes[4], myNetwork.nodes[5]);
</pre>
</details>
<details>
<summary>remove</summary>
Removes a node from a network, all its connections will be redirected. If it gates a connection, the gate will be removed.
<pre>
myNetwork = new architect.Perceptron(1,4,1);
// Remove a node
myNetwork.remove(myNetwork.nodes[2]);
</pre>
</details>
<details>
<summary>disconnect</summary>
Disconnects two nodes in the network:
<pre>
myNetwork.disconnect(myNetwork.nodes[4], myNetwork.nodes[5]);
// now node 4 does not have an effect on the output of node 5 anymore
</pre>
</details>
<details>
<summary>gate</summary>
Makes a network node gate a connection:
<pre>
myNetwork.gate(myNetwork.nodes[1], myNetwork.connections[5]
</pre>
Now the weight of connection 5 is multiplied with the activation of node 1!
</details>
<details>
<summary>ungate</summary>
Removes a gate from a connection:
<pre>
myNetwork = new architect.Perceptron(1, 4, 2);
// Gate a connection
myNetwork.gate(myNetwork.nodes[2], myNetwork.connections[5]);
// Remove the gate from the connection
myNetwork.ungate(myNetwork.connections[5]);
</pre>
</details>
<details>
<summary>mutate</summary>
Mutates the network. See [mutation methods](../methods/mutation.md).
</details>
<details>
<summary>serialize</summary>
Serializes the network to 3 <code>Float64Arrays</code>. Used for transferring
networks to other threads fast.
</details>
<details>
<summary>toJSON/fromJSON</summary>
Networks can be stored as JSON's and then restored back:
<pre>
var exported = myNetwork.toJSON();
var imported = Network.fromJSON(exported);
</pre>
<code>imported</code> will be a new instance of <code>Network</code> that is an exact clone of <code>myNetwork</code>.
</details>
<details>
<summary>standalone</summary>
Networks can be used in Javascript without the need of the Neataptic library,
this function will transform your network into a function accompanied by arrays.
<pre>
var myNetwork = new architect.Perceptron(2,4,1);
myNetwork.activate([0,1]); // [0.24775789809]
// a string
var standalone = myNetwork.standalone();
// turns your network into an 'activate' function
eval(standalone);
// calls the standalone function
activate([0,1]);// [0.24775789809]
</pre>
The reason an `eval` is being called is because the standalone can't be a simply
a function, it needs some kind of global memory. You can easily copy and paste the
result of `standalone` in any JS file and run the `activate` function!
Note that this is still in development, so for complex networks, it might not be
precise.
</details>
<details>
<summary>crossOver</summary>
Creates a new 'baby' network from two parent networks. Networks are not required to have the same size, however input and output size should be the same!
<pre>
// Initialise two parent networks
var network1 = new architect.Perceptron(2, 4, 3);
var network2 = new architect.Perceptron(2, 4, 5, 3);
// Produce an offspring
var network3 = Network.crossOver(network1, network2);
</pre>
</details>
<details>
<summary>set</summary>
Sets the properties of all nodes in the network to the given values, e.g.:
<pre>
var network = new architect.Random(4, 4, 1);
// All nodes in 'network' now have a bias of 1
network.set({bias: 1});
</pre>
</details>
<details>
<summary>clear</summary>
Clears the context of the network. Useful for predicting timeseries with LSTM's. `clear()` has little to no effecton regular NN, use on RNN's!
</details>
### Properties
Each network only has a small number of properties.
<details>
<summary>input</summary>
Input size of the network
</details>
<details>
<summary>output</summary>
Output size of the network
</details>
<details>
<summary>nodes</summary>
Array of nodes
</details>
<details>
<summary>connections</summary>
Array of connections
</details>
<details>
<summary>gates</summary>
Array of gated connections
</details>
<details>
<summary>selfconns</summary>
Array of self connections
</details>

203
node_modules/neataptic/mkdocs/templates/docs/architecture/node.md generated vendored Normal file
View File

@@ -0,0 +1,203 @@
description: Documentation of the Node instance in Neataptic
authors: Thomas Wagenaar
keywords: node, neuron, neural-network, activation, bias
Nodes are the key to neural networks. They provide the non-linearity in the output. A node can be created as follows:
```javascript
var node = new Node();
```
Node properties:
Property | contains
-------- | --------
bias | the bias when calculating state
squash | activation function
type | 'input', 'hidden' or 'output', should not be used manually (setting to 'constant' will disable bias/weight changes)
activation | activation value
connections | dictionary of in and out connections
old | stores the previous activation
state | stores the state (before being squashed)
### activate
Actives the node. Calculates the state from all the input connections, adds the bias, and 'squashes' it.
```javascript
var node = new Node();
node.activate(); // 0.4923128591923
```
### noTraceActivate
Actives the node. Calculates the state from all the input connections, adds the bias, and 'squashes' it.
Does not calculate traces, so this can't be used to backpropagate afterwards.
That's also why it's quite a bit faster than regular `activate`.
```javascript
var node = new Node();
node.noTraceActivate(); // 0.4923128591923
```
### propagate
After an activation, you can teach the node what should have been the correct
output (a.k.a. train). This is done by backpropagating the error. To use the
propagate method you have to provide a learning rate, and a target value
(float between 0 and 1).
The arguments you can pass on are as follows:
```javascript
myNode.propagate(learningRate, momentum, update, target);
```
The target argument is optional. The default value of momentum is `0`. Read more
about momentum on the [regularization page](../methods/regularization.md). If you run
propagation without setting update to true, then the weights won't update. So if
you run propagate 3x with `update: false`, and then 1x with `update: true` then
the weights will be updated after the last propagation, but the deltaweights of
the first 3 propagation will be included too. For example, this is how you can
train node B to activate 0 when node A activates 1:
```javascript
var A = new Node();
var B = new Node('output');
A.connect(B);
var learningRate = .3;
var momentum = 0;
for(var i = 0; i < 20000; i++)
{
// when A activates 1
A.activate(1);
// train B to activate 0
B.activate();
B.propagate(learningRate, momentum, true, 0);
}
// test it
A.activate(1);
B.activate(); // 0.006540565760853365
```
### connect
A node can project a connection to another node or group (i.e. connect node A with node B). Here is how it's done:
```javascript
var A = new Node();
var B = new Node();
A.connect(B); // A now projects a connection to B
// But you can also connect nodes to groups
var C = new Group(4);
B.connect(C); // B now projects a connection to all nodes in C
```
A neuron can also connect to itself, creating a selfconnection:
```javascript
var A = new Node();
A.connect(A); // A now connects to itself
```
### disconnect
Removes the projected connection from this node to the given node.
```javascript
var A = new Node();
var B = new Node();
A.connect(B); // A now projects a connection to B
A.disconnect(B); // no connection between A and B anymore
```
If the nodes project a connection to each other, you can also disconnect both connections at once:
```javascript
var A = new Node();
var B = new Node();
A.connect(B); // A now projects a connection to B
B.connect(A); // B now projects a connection to A
// A.disconnect(B) only disconnects A to B, so use
A.disconnect(B, true); // or B.disconnect(A, true)
```
### gate
Neurons can gate connections. This means that the activation value of a neuron has influence on the value transported through a connection. You can either give an array of connections or just a connection as an argument.
```javascript
var A = new Node();
var B = new Node();
var C = new Node();
var connections = A.connect(B);
// Now gate the connection(s)
C.gate(connections);
```
Now the weight of the connection from A to B will always be multiplied by the activation of node C.
### ungate
You can also remove a gate from a connection.
```javascript
var A = new Node();
var B = new Node();
var C = new Node();
var connections = A.connect(B);
// Now gate the connection(s)
C.gate(connections);
// Now ungate those connections
C.ungate(connections);
```
### isProjectingTo
Checks if the node is projecting a connection to another neuron.
```javascript
var A = new Node();
var B = new Node();
var C = new Node();
A.connect(B);
B.connect(C);
A.isProjectingTo(B); // true
A.isProjectingTo(C); // false
```
### isProjectedBy
Checks if the node is projected by another node.
```javascript
var A = new Node();
var B = new Node();
var C = new Node();
A.connect(B);
B.connect(C);
A.isProjectedBy(C); // false
B.isProjectedBy(A); // true
```
### toJSON/fromJSON
Nodes can be stored as JSON's and then restored back:
```javascript
var exported = myNode.toJSON();
var imported = Network.fromJSON(exported);
```
imported will be a new instance of Node that is an exact clone of myNode.
### clear
Clears the context of the node. Useful for predicting timeseries with LSTM's.