diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/README.md | 1 | ||||
| -rw-r--r-- | docs/common.md | 6 | ||||
| -rw-r--r-- | docs/components/auto-scaler.md | 0 | ||||
| -rw-r--r-- | docs/components/cache.md | 0 | ||||
| -rw-r--r-- | docs/components/gateway.md | 0 | ||||
| -rw-r--r-- | docs/components/ratelimiter.md | 0 | ||||
| -rw-r--r-- | docs/components/webhook.md | 8 | ||||
| -rw-r--r-- | docs/docs/LICENSE.md | 201 | ||||
| -rw-r--r-- | docs/docs/deployment/_category_.json | 4 | ||||
| -rw-r--r-- | docs/docs/deployment/production.md | 1 | ||||
| -rw-r--r-- | docs/docs/internals/_category_.json | 4 | ||||
| -rw-r--r-- | docs/docs/internals/components/_category_.json | 4 | ||||
| -rw-r--r-- | docs/docs/internals/components/gateway.md | 34 | ||||
| -rw-r--r-- | docs/docs/internals/index.md | 51 | ||||
| -rw-r--r-- | docs/docs/internals/protocols/index.md | 7 | ||||
| -rw-r--r-- | docs/docs/intro.md | 29 | ||||
| -rw-r--r-- | docs/docs/quickstart.md | 21 | ||||
| -rw-r--r-- | docs/exemples/config.yaml | 0 | ||||
| -rw-r--r-- | docs/internals/management.md | 5 | ||||
| -rw-r--r-- | docs/structure.md | 15 |
20 files changed, 356 insertions, 35 deletions
diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index fcc815a..0000000 --- a/docs/README.md +++ /dev/null @@ -1 +0,0 @@ -# Nova documentation
\ No newline at end of file diff --git a/docs/common.md b/docs/common.md deleted file mode 100644 index 3dce3ac..0000000 --- a/docs/common.md +++ /dev/null @@ -1,6 +0,0 @@ -# Common packages - -Nova exposes common packages for all the different projects in the mono-repo. -Since Nova uses two different languages, it has two different common libraries, -one built in rust in [the common/rust](../common/rust) directory and one in Go -in [the common/](../common) directory.
\ No newline at end of file diff --git a/docs/components/auto-scaler.md b/docs/components/auto-scaler.md deleted file mode 100644 index e69de29..0000000 --- a/docs/components/auto-scaler.md +++ /dev/null diff --git a/docs/components/cache.md b/docs/components/cache.md deleted file mode 100644 index e69de29..0000000 --- a/docs/components/cache.md +++ /dev/null diff --git a/docs/components/gateway.md b/docs/components/gateway.md deleted file mode 100644 index e69de29..0000000 --- a/docs/components/gateway.md +++ /dev/null diff --git a/docs/components/ratelimiter.md b/docs/components/ratelimiter.md deleted file mode 100644 index e69de29..0000000 --- a/docs/components/ratelimiter.md +++ /dev/null diff --git a/docs/components/webhook.md b/docs/components/webhook.md deleted file mode 100644 index 76361ef..0000000 --- a/docs/components/webhook.md +++ /dev/null @@ -1,8 +0,0 @@ -# Webhook - -> TD;TR The webhook component is an implementation of the discord interaction webhooks; You can either use the Gateway or the Webhooks. The webhooks __require__ an external https endpoint to work. - -The webhook source code is located in the [webhook](../../webhook) folder and is implemented in Rust. It's a simple http web server which implements the webhook signature verification and deserialization. Like the gateway, the messages are redirected using the [relaying system](../common#relaying_trait). - -The signature verification is done using libsodium via the libsodium-sys trait. -Subsequently, it uses code marked as "unsafe" in rust. It's built into the binary statically. Any route can be used to receive webhook messages.
\ No newline at end of file diff --git a/docs/docs/LICENSE.md b/docs/docs/LICENSE.md new file mode 100644 index 0000000..7342771 --- /dev/null +++ b/docs/docs/LICENSE.md @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + +APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + +Copyright 2021 Developer's House + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/docs/docs/deployment/_category_.json b/docs/docs/deployment/_category_.json new file mode 100644 index 0000000..27fb1fc --- /dev/null +++ b/docs/docs/deployment/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Deployment", + "position": 3 +}
\ No newline at end of file diff --git a/docs/docs/deployment/production.md b/docs/docs/deployment/production.md new file mode 100644 index 0000000..38ca6b8 --- /dev/null +++ b/docs/docs/deployment/production.md @@ -0,0 +1 @@ +# Production guide
\ No newline at end of file diff --git a/docs/docs/internals/_category_.json b/docs/docs/internals/_category_.json new file mode 100644 index 0000000..ed12ab8 --- /dev/null +++ b/docs/docs/internals/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Internals", + "position": 99999 +}
\ No newline at end of file diff --git a/docs/docs/internals/components/_category_.json b/docs/docs/internals/components/_category_.json new file mode 100644 index 0000000..a50cc69 --- /dev/null +++ b/docs/docs/internals/components/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Components", + "position": 4 +}
\ No newline at end of file diff --git a/docs/docs/internals/components/gateway.md b/docs/docs/internals/components/gateway.md new file mode 100644 index 0000000..9aab5c2 --- /dev/null +++ b/docs/docs/internals/components/gateway.md @@ -0,0 +1,34 @@ +--- +title: Gateway +position: 1 +--- + +The gateway if the component responsible for the real-time events +using the Discord Gateway API (websocket), the component is written +in rust using [`tokio-tungstenite`](https://github.com/snapview/tokio-tungstenite) and his job is to simply maintain +a gateway connection (heartbeat, reconnect, etc...) and send events +directly to the cache using nats. + +The project is located in the `gateway` folder at the root +of the GitHub repo and is published using a statically linked +library (except libc) and docker images. + +## Structure + +Internally, the gateway is divided in multiple rust modules + +### Connection (src/connection) + +This module implements a [`futures_core::stream::Stream` trait](https://docs.rs/futures-core/0.3.17/futures_core/stream/trait.Stream.html) +and [`futures::sink::Sink` trait](https://docs.rs/futures/0.3.17/futures/sink/trait.Sink.html) that streams deserialized discord +packets to the stream. It does not implement any reconnect mechanism and returns +an error when the connection encounters a deserialization error or i/o error. +If the connection is closed, the stream simply closes. + +You can send any `BaseMessage` struct through the connection and receive +any `BaseMessage` struct. + + +### Payloads (src/payload) + +This module implements the deserialization of discord packets diff --git a/docs/docs/internals/index.md b/docs/docs/internals/index.md new file mode 100644 index 0000000..d2bbd9f --- /dev/null +++ b/docs/docs/internals/index.md @@ -0,0 +1,51 @@ +--- +sidebar_position: 1 +--- + +# Nova Internals + +## Definitions + +### Cache + +A cache is an instance of the `cache` program, it managed everything that is related to data-management +of the discord events, webhooks and more. + +### Gateway +*todo* +### Webhook +*todo* +### + +## Communication + +``` + ┌───────────────┐ ┌───────────────────┐ + │ Gateway │ event(s) │ │ ┌───────────────────────┐ +┌──►│ (gateway) ├─────────────►│ ├──────────────────►│ │ +│ └───────────────┘ (nats queue) │ Cache instance │ (redis channel) │ Caching server │ +│ │ (cache) │ │ (redis) │ +│ ┌───────────────┐ event(s) │ │◄──────────────────┤ │ +│ │ Webhook ├─────────────►│ │ └───────────────────────┘ +│ │ (webhook) │ (nats queue) └─────────────────┬─┘ +│ └───────────────┘ ▲ │ +│ cache requests │ │ event(s) +│ (grpc) │ │ (nats queue) +│ │ ▼ +│ ┌───┴───────────────┐ +└──────────────────────────────────┤ User programs │ + gateway commands │ (client libs) │ + (grpc) └───────────────────┘ +``` + +> We try to use protocol buffers everywhere even inside the event broker (nats), +> this ensures we have the same date representation across all the programs written +> in different languages + +> The only dependency of the nova architecture is a reliable nats & redis cluster for now + +#### Protocols + +All the protocols a detailed in the [protocols folder](protocols) + +####
\ No newline at end of file diff --git a/docs/docs/internals/protocols/index.md b/docs/docs/internals/protocols/index.md new file mode 100644 index 0000000..4de3a69 --- /dev/null +++ b/docs/docs/internals/protocols/index.md @@ -0,0 +1,7 @@ +# Protocols + +This section of the internal documentation describes the protocols used by nova +to communicate between all the components. + + +*
\ No newline at end of file diff --git a/docs/docs/intro.md b/docs/docs/intro.md new file mode 100644 index 0000000..451d2ab --- /dev/null +++ b/docs/docs/intro.md @@ -0,0 +1,29 @@ +# Introduction + +## What is nova ? + +Nova is a complete framework for building reliable and stable discord bots +using a services based system to operate all the components needed to operate +a discord such as the discord gateway, new discord webhooks for receiving interactions. +Using a traditional infrastructure (sharder / process), we can quickly reach bottlenecks +when we scale to multiple thousands of shards. Nova is a way to avoid these problems by +building a scale-first discord framework that creates an environment that allow +better testing, reliability and operations easier. + +### Advantages + +With the help of Nova, you can achieve a number of things, such as + +* Scaling of workers independent of the number of shards +* zero-login updates +* Automatic shards scaling +* Shared cache for the whole bot (without broadcastEval and other unsafe methods) +* Stateless workers, easier to test +* Distributed rest rate-limiting +* Easier fine-tuned monitoring using cloud-native technologies +* Languages agnostic apis + +### How did we solve this ? + +Nova separates the gateway into multiple smaller components corresponding to multiple +discord apis diff --git a/docs/docs/quickstart.md b/docs/docs/quickstart.md new file mode 100644 index 0000000..216552e --- /dev/null +++ b/docs/docs/quickstart.md @@ -0,0 +1,21 @@ +--- +sidebar_position: 2 +--- + +# 5 Minutes quickstart + +This page shows you how to start a new nova project +under five minutes using a typescript project, +hold tight this is going to be fast. + +## Requirements + +* A discord bot application available +* [Docker](https://docker.io/) (or alternatives) available to you. +* A domain name / [ngrok.io](https://ngrok.com/) domain (for webhooks only) + +> If you are deploying nova to production, consider following the +> production guide instead. + +## Setting up a nova instance + diff --git a/docs/exemples/config.yaml b/docs/exemples/config.yaml deleted file mode 100644 index e69de29..0000000 --- a/docs/exemples/config.yaml +++ /dev/null diff --git a/docs/internals/management.md b/docs/internals/management.md deleted file mode 100644 index 7bff71b..0000000 --- a/docs/internals/management.md +++ /dev/null @@ -1,5 +0,0 @@ -# Nova Internals - Management - -In order to run in multiple environments, Nova uses a scheduling system to balance shards between "cluster" instances. -Each cluster instance is in fact a gateway instance. All the management is handled in the "manager" component. -The discord shards are dynamically scaled and restarted if needed.
\ No newline at end of file diff --git a/docs/structure.md b/docs/structure.md deleted file mode 100644 index 1f82ae3..0000000 --- a/docs/structure.md +++ /dev/null @@ -1,15 +0,0 @@ -# Workspace/System structure - -## System structure - -In the nova repository, two different types of projects exist, - -* The management projects primarly in Go \ - They manage the other components of the nova infrastructure. -* The data-path projects \ - They handle all the data transfer / management. - -### Gateway - -> The gateway interfaces with the discord gateway to retrive events in real time -It's implemented in rust and is in the gateway folder.
\ No newline at end of file |