diff options
| author | Matthieu <matthieu@developershouse.xyz> | 2021-10-08 14:40:29 +0400 | 
|---|---|---|
| committer | Matthieu <matthieu@developershouse.xyz> | 2021-10-08 14:40:29 +0400 | 
| commit | dbfc18ed3cef55a4433870ddee77c921b996cc2b (patch) | |
| tree | 57dbbb3b1d39afa2910b751fb388dc2e99772e89 /docs | |
| parent | 4a08838b902d7f7c0c4daec5fe121707d794ec6e (diff) | |
docs: add docusaurus base
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/README.md | 1 | ||||
| -rw-r--r-- | docs/common.md | 6 | ||||
| -rw-r--r-- | docs/components/auto-scaler.md | 0 | ||||
| -rw-r--r-- | docs/components/cache.md | 0 | ||||
| -rw-r--r-- | docs/components/gateway.md | 0 | ||||
| -rw-r--r-- | docs/components/ratelimiter.md | 0 | ||||
| -rw-r--r-- | docs/components/webhook.md | 8 | ||||
| -rw-r--r-- | docs/docs/LICENSE.md | 201 | ||||
| -rw-r--r-- | docs/docs/deployment/_category_.json | 4 | ||||
| -rw-r--r-- | docs/docs/deployment/production.md | 1 | ||||
| -rw-r--r-- | docs/docs/internals/_category_.json | 4 | ||||
| -rw-r--r-- | docs/docs/internals/components/_category_.json | 4 | ||||
| -rw-r--r-- | docs/docs/internals/components/gateway.md | 34 | ||||
| -rw-r--r-- | docs/docs/internals/index.md | 51 | ||||
| -rw-r--r-- | docs/docs/internals/protocols/index.md | 7 | ||||
| -rw-r--r-- | docs/docs/intro.md | 29 | ||||
| -rw-r--r-- | docs/docs/quickstart.md | 21 | ||||
| -rw-r--r-- | docs/exemples/config.yaml | 0 | ||||
| -rw-r--r-- | docs/internals/management.md | 5 | ||||
| -rw-r--r-- | docs/structure.md | 15 | 
20 files changed, 356 insertions, 35 deletions
diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index fcc815a..0000000 --- a/docs/README.md +++ /dev/null @@ -1 +0,0 @@ -# Nova documentation
\ No newline at end of file diff --git a/docs/common.md b/docs/common.md deleted file mode 100644 index 3dce3ac..0000000 --- a/docs/common.md +++ /dev/null @@ -1,6 +0,0 @@ -# Common packages - -Nova exposes common packages for all the different projects in the mono-repo.  -Since Nova uses two different languages, it has two different common libraries, -one built in rust in [the common/rust](../common/rust) directory and one in Go -in [the common/](../common) directory.
\ No newline at end of file diff --git a/docs/components/auto-scaler.md b/docs/components/auto-scaler.md deleted file mode 100644 index e69de29..0000000 --- a/docs/components/auto-scaler.md +++ /dev/null diff --git a/docs/components/cache.md b/docs/components/cache.md deleted file mode 100644 index e69de29..0000000 --- a/docs/components/cache.md +++ /dev/null diff --git a/docs/components/gateway.md b/docs/components/gateway.md deleted file mode 100644 index e69de29..0000000 --- a/docs/components/gateway.md +++ /dev/null diff --git a/docs/components/ratelimiter.md b/docs/components/ratelimiter.md deleted file mode 100644 index e69de29..0000000 --- a/docs/components/ratelimiter.md +++ /dev/null diff --git a/docs/components/webhook.md b/docs/components/webhook.md deleted file mode 100644 index 76361ef..0000000 --- a/docs/components/webhook.md +++ /dev/null @@ -1,8 +0,0 @@ -# Webhook - -> TD;TR The webhook component is an implementation of the discord interaction webhooks; You can either use the Gateway or the Webhooks. The webhooks __require__ an external https endpoint to work. - -The webhook source code is located in the [webhook](../../webhook) folder and is implemented in Rust. It's a simple http web server which implements the webhook signature verification and deserialization. Like the gateway, the messages are redirected using the [relaying system](../common#relaying_trait). - -The signature verification is done using libsodium via the libsodium-sys trait. -Subsequently, it uses code marked as "unsafe" in rust. It's built into the binary statically. Any route can be used to receive webhook messages.
\ No newline at end of file diff --git a/docs/docs/LICENSE.md b/docs/docs/LICENSE.md new file mode 100644 index 0000000..7342771 --- /dev/null +++ b/docs/docs/LICENSE.md @@ -0,0 +1,201 @@ +                                 Apache License +                           Version 2.0, January 2004 +                        http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + +   "License" shall mean the terms and conditions for use, reproduction, +   and distribution as defined by Sections 1 through 9 of this document. + +   "Licensor" shall mean the copyright owner or entity authorized by +   the copyright owner that is granting the License. + +   "Legal Entity" shall mean the union of the acting entity and all +   other entities that control, are controlled by, or are under common +   control with that entity. For the purposes of this definition, +   "control" means (i) the power, direct or indirect, to cause the +   direction or management of such entity, whether by contract or +   otherwise, or (ii) ownership of fifty percent (50%) or more of the +   outstanding shares, or (iii) beneficial ownership of such entity. + +   "You" (or "Your") shall mean an individual or Legal Entity +   exercising permissions granted by this License. + +   "Source" form shall mean the preferred form for making modifications, +   including but not limited to software source code, documentation +   source, and configuration files. + +   "Object" form shall mean any form resulting from mechanical +   transformation or translation of a Source form, including but +   not limited to compiled object code, generated documentation, +   and conversions to other media types. + +   "Work" shall mean the work of authorship, whether in Source or +   Object form, made available under the License, as indicated by a +   copyright notice that is included in or attached to the work +   (an example is provided in the Appendix below). + +   "Derivative Works" shall mean any work, whether in Source or Object +   form, that is based on (or derived from) the Work and for which the +   editorial revisions, annotations, elaborations, or other modifications +   represent, as a whole, an original work of authorship. For the purposes +   of this License, Derivative Works shall not include works that remain +   separable from, or merely link (or bind by name) to the interfaces of, +   the Work and Derivative Works thereof. + +   "Contribution" shall mean any work of authorship, including +   the original version of the Work and any modifications or additions +   to that Work or Derivative Works thereof, that is intentionally +   submitted to Licensor for inclusion in the Work by the copyright owner +   or by an individual or Legal Entity authorized to submit on behalf of +   the copyright owner. For the purposes of this definition, "submitted" +   means any form of electronic, verbal, or written communication sent +   to the Licensor or its representatives, including but not limited to +   communication on electronic mailing lists, source code control systems, +   and issue tracking systems that are managed by, or on behalf of, the +   Licensor for the purpose of discussing and improving the Work, but +   excluding communication that is conspicuously marked or otherwise +   designated in writing by the copyright owner as "Not a Contribution." + +   "Contributor" shall mean Licensor and any individual or Legal Entity +   on behalf of whom a Contribution has been received by Licensor and +   subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of +   this License, each Contributor hereby grants to You a perpetual, +   worldwide, non-exclusive, no-charge, royalty-free, irrevocable +   copyright license to reproduce, prepare Derivative Works of, +   publicly display, publicly perform, sublicense, and distribute the +   Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of +   this License, each Contributor hereby grants to You a perpetual, +   worldwide, non-exclusive, no-charge, royalty-free, irrevocable +   (except as stated in this section) patent license to make, have made, +   use, offer to sell, sell, import, and otherwise transfer the Work, +   where such license applies only to those patent claims licensable +   by such Contributor that are necessarily infringed by their +   Contribution(s) alone or by combination of their Contribution(s) +   with the Work to which such Contribution(s) was submitted. If You +   institute patent litigation against any entity (including a +   cross-claim or counterclaim in a lawsuit) alleging that the Work +   or a Contribution incorporated within the Work constitutes direct +   or contributory patent infringement, then any patent licenses +   granted to You under this License for that Work shall terminate +   as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the +   Work or Derivative Works thereof in any medium, with or without +   modifications, and in Source or Object form, provided that You +   meet the following conditions: + +   (a) You must give any other recipients of the Work or +   Derivative Works a copy of this License; and + +   (b) You must cause any modified files to carry prominent notices +   stating that You changed the files; and + +   (c) You must retain, in the Source form of any Derivative Works +   that You distribute, all copyright, patent, trademark, and +   attribution notices from the Source form of the Work, +   excluding those notices that do not pertain to any part of +   the Derivative Works; and + +   (d) If the Work includes a "NOTICE" text file as part of its +   distribution, then any Derivative Works that You distribute must +   include a readable copy of the attribution notices contained +   within such NOTICE file, excluding those notices that do not +   pertain to any part of the Derivative Works, in at least one +   of the following places: within a NOTICE text file distributed +   as part of the Derivative Works; within the Source form or +   documentation, if provided along with the Derivative Works; or, +   within a display generated by the Derivative Works, if and +   wherever such third-party notices normally appear. The contents +   of the NOTICE file are for informational purposes only and +   do not modify the License. You may add Your own attribution +   notices within Derivative Works that You distribute, alongside +   or as an addendum to the NOTICE text from the Work, provided +   that such additional attribution notices cannot be construed +   as modifying the License. + +   You may add Your own copyright statement to Your modifications and +   may provide additional or different license terms and conditions +   for use, reproduction, or distribution of Your modifications, or +   for any such Derivative Works as a whole, provided Your use, +   reproduction, and distribution of the Work otherwise complies with +   the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, +   any Contribution intentionally submitted for inclusion in the Work +   by You to the Licensor shall be under the terms and conditions of +   this License, without any additional terms or conditions. +   Notwithstanding the above, nothing herein shall supersede or modify +   the terms of any separate license agreement you may have executed +   with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade +   names, trademarks, service marks, or product names of the Licensor, +   except as required for reasonable and customary use in describing the +   origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or +   agreed to in writing, Licensor provides the Work (and each +   Contributor provides its Contributions) on an "AS IS" BASIS, +   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or +   implied, including, without limitation, any warranties or conditions +   of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A +   PARTICULAR PURPOSE. You are solely responsible for determining the +   appropriateness of using or redistributing the Work and assume any +   risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, +   whether in tort (including negligence), contract, or otherwise, +   unless required by applicable law (such as deliberate and grossly +   negligent acts) or agreed to in writing, shall any Contributor be +   liable to You for damages, including any direct, indirect, special, +   incidental, or consequential damages of any character arising as a +   result of this License or out of the use or inability to use the +   Work (including but not limited to damages for loss of goodwill, +   work stoppage, computer failure or malfunction, or any and all +   other commercial damages or losses), even if such Contributor +   has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing +   the Work or Derivative Works thereof, You may choose to offer, +   and charge a fee for, acceptance of support, warranty, indemnity, +   or other liability obligations and/or rights consistent with this +   License. However, in accepting such obligations, You may act only +   on Your own behalf and on Your sole responsibility, not on behalf +   of any other Contributor, and only if You agree to indemnify, +   defend, and hold each Contributor harmless for any liability +   incurred by, or claims asserted against, such Contributor by reason +   of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + +APPENDIX: How to apply the Apache License to your work. + +      To apply the Apache License to your work, attach the following +      boilerplate notice, with the fields enclosed by brackets "[]" +      replaced with your own identifying information. (Don't include +      the brackets!)  The text should be enclosed in the appropriate +      comment syntax for the file format. We also recommend that a +      file or class name and description of purpose be included on the +      same "printed page" as the copyright notice for easier +      identification within third-party archives. + +Copyright 2021 Developer's House + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +       http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/docs/docs/deployment/_category_.json b/docs/docs/deployment/_category_.json new file mode 100644 index 0000000..27fb1fc --- /dev/null +++ b/docs/docs/deployment/_category_.json @@ -0,0 +1,4 @@ +{ +  "label": "Deployment", +  "position": 3 +}
\ No newline at end of file diff --git a/docs/docs/deployment/production.md b/docs/docs/deployment/production.md new file mode 100644 index 0000000..38ca6b8 --- /dev/null +++ b/docs/docs/deployment/production.md @@ -0,0 +1 @@ +# Production guide
\ No newline at end of file diff --git a/docs/docs/internals/_category_.json b/docs/docs/internals/_category_.json new file mode 100644 index 0000000..ed12ab8 --- /dev/null +++ b/docs/docs/internals/_category_.json @@ -0,0 +1,4 @@ +{ +  "label": "Internals", +  "position": 99999 +}
\ No newline at end of file diff --git a/docs/docs/internals/components/_category_.json b/docs/docs/internals/components/_category_.json new file mode 100644 index 0000000..a50cc69 --- /dev/null +++ b/docs/docs/internals/components/_category_.json @@ -0,0 +1,4 @@ +{ +  "label": "Components", +  "position": 4 +}
\ No newline at end of file diff --git a/docs/docs/internals/components/gateway.md b/docs/docs/internals/components/gateway.md new file mode 100644 index 0000000..9aab5c2 --- /dev/null +++ b/docs/docs/internals/components/gateway.md @@ -0,0 +1,34 @@ +--- +title: Gateway +position: 1 +--- + +The gateway if the component responsible for the real-time events +using the Discord Gateway API (websocket), the component is written +in rust using [`tokio-tungstenite`](https://github.com/snapview/tokio-tungstenite) and his job is to simply maintain +a gateway connection (heartbeat, reconnect, etc...) and send events +directly to the cache using nats. + +The project is located in the `gateway` folder at the root +of the GitHub repo and is published using a statically linked +library (except libc) and docker images. + +## Structure + +Internally, the gateway is divided in multiple rust modules + +### Connection (src/connection) + +This module implements a [`futures_core::stream::Stream` trait](https://docs.rs/futures-core/0.3.17/futures_core/stream/trait.Stream.html) +and [`futures::sink::Sink` trait](https://docs.rs/futures/0.3.17/futures/sink/trait.Sink.html) that streams deserialized discord +packets to the stream. It does not implement any reconnect mechanism and returns +an error when the connection encounters a deserialization error or i/o error. +If the connection is closed, the stream simply closes. + +You can send any `BaseMessage` struct through the connection and receive +any `BaseMessage` struct. + + +### Payloads (src/payload) + +This module implements the deserialization of discord packets diff --git a/docs/docs/internals/index.md b/docs/docs/internals/index.md new file mode 100644 index 0000000..d2bbd9f --- /dev/null +++ b/docs/docs/internals/index.md @@ -0,0 +1,51 @@ +--- +sidebar_position: 1 +--- + +# Nova Internals + +## Definitions + +### Cache + +A cache is an instance of the `cache` program, it managed everything that is related to data-management +of the discord events, webhooks and more. + +### Gateway +*todo* +### Webhook +*todo* +###  + +## Communication + +``` +    ┌───────────────┐              ┌───────────────────┐ +    │    Gateway    │   event(s)   │                   │                   ┌───────────────────────┐ +┌──►│   (gateway)   ├─────────────►│                   ├──────────────────►│                       │ +│   └───────────────┘ (nats queue) │   Cache instance  │  (redis channel)  │     Caching server    │ +│                                  │      (cache)      │                   │        (redis)        │ +│   ┌───────────────┐   event(s)   │                   │◄──────────────────┤                       │ +│   │    Webhook    ├─────────────►│                   │                   └───────────────────────┘ +│   │   (webhook)   │ (nats queue) └─────────────────┬─┘ +│   └───────────────┘                  ▲             │ +│                       cache requests │             │   event(s) +│                          (grpc)      │             │ (nats queue) +│                                      │             ▼ +│                                  ┌───┴───────────────┐ +└──────────────────────────────────┤   User programs   │ +         gateway commands          │   (client libs)   │ +              (grpc)               └───────────────────┘ +``` + +> We try to use protocol buffers everywhere even inside the event broker (nats), +> this ensures we have the same date representation across all the programs written +> in different languages + +> The only dependency of the nova architecture is a reliable nats & redis cluster for now + +#### Protocols + +All the protocols a detailed in the [protocols folder](protocols) + +#### 
\ No newline at end of file diff --git a/docs/docs/internals/protocols/index.md b/docs/docs/internals/protocols/index.md new file mode 100644 index 0000000..4de3a69 --- /dev/null +++ b/docs/docs/internals/protocols/index.md @@ -0,0 +1,7 @@ +# Protocols + +This section of the internal documentation describes the protocols used by nova +to communicate between all the components. + + +* 
\ No newline at end of file diff --git a/docs/docs/intro.md b/docs/docs/intro.md new file mode 100644 index 0000000..451d2ab --- /dev/null +++ b/docs/docs/intro.md @@ -0,0 +1,29 @@ +# Introduction + +## What is nova ? + +Nova is a complete framework for building reliable and stable discord bots +using a services based system to operate all the components needed to operate +a discord such as the discord gateway, new discord webhooks for receiving interactions. +Using a traditional infrastructure (sharder / process), we can quickly reach bottlenecks +when we scale to multiple thousands of shards. Nova is a way to avoid these problems by +building a scale-first discord framework that creates an environment that allow  +better testing, reliability and operations easier. + +### Advantages + +With the help of Nova, you can achieve a number of things, such as + +* Scaling of workers independent of the number of shards +* zero-login updates +* Automatic shards scaling +* Shared cache for the whole bot (without broadcastEval and other unsafe methods) +* Stateless workers, easier to test +* Distributed rest rate-limiting +* Easier fine-tuned monitoring using cloud-native technologies +* Languages agnostic apis + +### How did we solve this ? + +Nova separates the gateway into multiple smaller components corresponding to multiple +discord apis diff --git a/docs/docs/quickstart.md b/docs/docs/quickstart.md new file mode 100644 index 0000000..216552e --- /dev/null +++ b/docs/docs/quickstart.md @@ -0,0 +1,21 @@ +--- +sidebar_position: 2 +--- + +# 5 Minutes quickstart + +This page shows you how to start a new nova project +under five minutes using a typescript project, +hold tight this is going to be fast. + +## Requirements + +* A discord bot application available +* [Docker](https://docker.io/) (or alternatives) available to you. +* A domain name / [ngrok.io](https://ngrok.com/) domain (for webhooks only) + +> If you are deploying nova to production, consider following the +> production guide instead. + +## Setting up a nova instance + diff --git a/docs/exemples/config.yaml b/docs/exemples/config.yaml deleted file mode 100644 index e69de29..0000000 --- a/docs/exemples/config.yaml +++ /dev/null diff --git a/docs/internals/management.md b/docs/internals/management.md deleted file mode 100644 index 7bff71b..0000000 --- a/docs/internals/management.md +++ /dev/null @@ -1,5 +0,0 @@ -# Nova Internals - Management - -In order to run in multiple environments, Nova uses a scheduling system to balance shards between "cluster" instances. -Each cluster instance is in fact a gateway instance. All the management is handled in the "manager" component.  -The discord shards are dynamically scaled and restarted if needed.
\ No newline at end of file diff --git a/docs/structure.md b/docs/structure.md deleted file mode 100644 index 1f82ae3..0000000 --- a/docs/structure.md +++ /dev/null @@ -1,15 +0,0 @@ -# Workspace/System structure - -## System structure - -In the nova repository, two different types of projects exist,  - -* The management projects primarly in Go \ -    They manage the other components of the nova infrastructure. -* The data-path projects \ -    They handle all the data transfer / management. - -### Gateway - -> The gateway interfaces with the discord gateway to retrive events in real time -It's implemented in rust and is in the gateway folder.
\ No newline at end of file  | 
