1 | <div align="center" style="text-align: center;">
|
2 | <h1 style="border-bottom: none;">TotalJS Application</h1>
|
3 |
|
4 | <p>Building Scalable Applications with TotalJS Flow</p>
|
5 | <img src="images/ApplicationComponent.png" alt="Application Component" width="163" />
|
6 | <img src="images/ModuleComponent.png" alt="Module Component" width="163" />
|
7 | </div>
|
8 |
|
9 | <h1>UNDER CONSTRUCTION: DO NOT USE</h1>
|
10 | <hr />
|
11 |
|
12 | [//]: # ([![Follow me][follow-me-badge]][follow-me-url])
|
13 |
|
14 | [![Version][version-badge]][version-url]
|
15 | [![Node version][node-version-badge]][node-version-url]
|
16 | [![MIT License][mit-license-badge]][mit-license-url]
|
17 |
|
18 | [![Downloads][downloads-badge]][downloads-url]
|
19 | [![Total downloads][total-downloads-badge]][downloads-url]
|
20 | [![Packagephobia][packagephobia-badge]][packagephobia-url]
|
21 | [![Bundlephobia][bundlephobia-badge]][bundlephobia-url]
|
22 |
|
23 | [//]: # ([![Dependency Status][daviddm-badge]][daviddm-url])
|
24 |
|
25 | [//]: # ([![codecov][codecov-badge]][codecov-url])
|
26 |
|
27 | [//]: # ([![Coverage Status][coveralls-badge]][coveralls-url])
|
28 |
|
29 | [//]: # ([![codebeat badge][codebeat-badge]][codebeat-url])
|
30 |
|
31 | [//]: # ([![Codacy Badge][codacy-badge]][codacy-url])
|
32 |
|
33 | [//]: # ([![Code of Conduct][coc-badge]][coc-url])
|
34 |
|
35 |
|
36 | ## Table of contents <!-- omit in toc -->
|
37 |
|
38 | - [Description](#description)
|
39 | - [Dependencies](#dependencies)
|
40 | - [Application Configuration](#application-configuration)
|
41 | - [Application Bootstrapping](#application-bootstrapping)
|
42 | - [Module Configuration](#module-configuration)
|
43 | - [Module Bootstrapping](#module-bootstrapping)
|
44 | - [Application Flow Architecture](#application-flow-architecture)
|
45 | - [Pre-requisites](#pre-requisites)
|
46 | - [Setup](#setup)
|
47 | - [Install](#install)
|
48 | - [Install Application](#install-application)
|
49 | - [Install Module](#install-module)
|
50 | - [License](#license)
|
51 |
|
52 | ## Description
|
53 | This project provides a set of components to help build applications using [TotalJS Flow][totaljs-flow-url].
|
54 | The two main components are the [Application][application-component-url] and the [Module][module-component-url].
|
55 | The Application is responsible for processing requests and managing the modules. The Module is responsible for
|
56 | handling all the operational logic.<br /><br />
|
57 | The TotalJS Flow project is a great tool for developing applications using flow based programming.
|
58 | When building larger applications where inter-module communication, multiphase processing, and
|
59 | scaling need to be taken into account it can be difficult to manage. This project provides a
|
60 | set of components to facilitate the building of larger applications using TotalJS Flow.<br /><br />
|
61 | Both the Application and the Module need to communicate with a redis server. Redis is used
|
62 | as the backend for how Application and Module instances communicate with each other.<br /><br />
|
63 | The Application component needs to communicate with a MongoDB server. The MongoDB server
|
64 | is primarily used to store details about the modules and their status. This is used so that
|
65 | the Application instances can be scaled out horizontally and keep in sync when it comes to
|
66 | processing.<br /><br />
|
67 |
|
68 |
|
69 | ## Dependencies
|
70 | - TotalJS Flow
|
71 | - Redis Server
|
72 | - RethinkDB Server
|
73 |
|
74 |
|
75 | ## Application Configuration
|
76 | - Application Name (Required: String)
|
77 | - Processing Mode (Required: String)
|
78 | - "Everything" (all) = Both input and API requests are processed
|
79 | - "Input" (input) = Only input requests are processed
|
80 | - "API" (api) = Only API requests are processed
|
81 | - Redis Connection (Required: String)
|
82 | - RethinkDB Connection (Required: String)
|
83 |
|
84 | ## Application Bootstrapping
|
85 | - Application does sanity checks on the configuration<br />
|
86 | - Check connection to MongoDB and Redis<br />
|
87 | - Then application executes the "init" phase.<br />
|
88 | - Initialize the Application Worker<br />
|
89 | - Initialize the Module Manager<br />
|
90 | - Create a new FlowMessage (with config as data) then send it to the "init" output and store message data on 'end' as Application Data (AppData)<br />
|
91 | - Then application executes the "boot" phase.<br />
|
92 | - Boot the Application Worker<br />
|
93 | - Boot the Module Manager<br />
|
94 | - Create a new FlowMessage then send it to the "boot" output with AppData and update message data on 'end' as AppData<br />
|
95 | - Then application executes the "start" phase.<br />
|
96 | - Start the Application Worker<br />
|
97 | - Start the Module Manager<br />
|
98 | - Create a new FlowMessage then send it to the "start" output with AppData and update message data on 'end' as AppData<br />
|
99 | - Then application executes the "ready" phase.<br />
|
100 | - Ready the Application Worker<br />
|
101 | - Ready the Module Manager<br />
|
102 | - Create a new FlowMessage then send it to the "ready" output with AppData and update message data on 'end' as AppData<br />
|
103 | - Start the status monitor which sends status to UI at interval<br />
|
104 | - The application is now ready to process requests.<br />
|
105 |
|
106 |
|
107 | ## Module Configuration
|
108 | - Label (Required: String)
|
109 | - Name (Required: String)
|
110 | - Application (Required: String)
|
111 | - Mode (Required: String)
|
112 | - "Everything" (all) = Both input and API requests are processed
|
113 | - "Input" (input) = Only input requests are processed
|
114 | - "API" (api) = Only API requests are processed
|
115 | - Redis Connection (Required: String)
|
116 |
|
117 | ## Module Bootstrapping
|
118 | - Module does sanity checks on the configuration<br />
|
119 | - Check connection to Redis<br />
|
120 | - Then module connects to the application and registers itself.<br />
|
121 | - Module sends a AppRegisterMessage to the application informing it of its existence.<br />
|
122 | - Application responds with a AppRegisterResponse.<br />
|
123 | - If AppRegisterResponse.success is true, then the module is registered.<br />
|
124 | - Module executes the "init" phase.<br />
|
125 | - Initialize the Module Worker<br />
|
126 | - Create a new FlowMessage (with config as data) then send it to the "init" output and store message data on 'end' as Module Data (ModData)<br />
|
127 | - Module executes the "boot" phase.<br />
|
128 | - Boot the Module Worker<br />
|
129 | - Create a new FlowMessage then send it to the "boot" output with ModData and update message data on 'end' as ModData<br />
|
130 | - Module executes the "start" phase.<br />
|
131 | - Start the Module Worker<br />
|
132 | - Create a new FlowMessage then send it to the "start" output with ModData and update message data on 'end' as ModData<br />
|
133 | - Module executes the "ready" phase.<br />
|
134 | - Ready the Module Worker<br />
|
135 | - Create a new FlowMessage then send it to the "ready" output with ModData and update message data on 'end' as ModData<br />
|
136 | - Start the status monitor which sends status to UI at interval<br />
|
137 | - Start heartbeat monitor which sends heartbeat to application at interval<br />
|
138 | - If AppRegisterResponse.success is false, then the module is not registered.<br />
|
139 | - Module is disabled.<br />
|
140 |
|
141 |
|
142 | ## Application Flow Architecture
|
143 | - Application Input (Input -> Phases -> Output)
|
144 | - Application Input is the entry point for the application which is the input labeled "input".<br />
|
145 | - Application Input is then sent to the "phases" output with the result being an array of strings used for that inputs processing phases.<br />
|
146 | - Application Input metadata is sent to each module requesting its involvement (phases and priority).<br />
|
147 | - Application Input is processed by each module involved according to priority with the results becoming the updated Application Input.<br />
|
148 | - Resulting Application Input is sent to "output" output for final processing.<br />
|
149 | - Application Message (Message -> Application -> Response)
|
150 | - Application Message is the message that is sent to the application from a module.<br />
|
151 | - Application Messages are sent to the application via the BullMQ job queue system.<br />
|
152 | - AppRegisterMessage is sent to the application to register a module.<br />
|
153 | - AppUnregisterMessage is sent to the application to unregister a module.<br />
|
154 | - AppHeartbeatMessage is sent to the application to inform it that a module is still alive.<br />
|
155 | - AppAPIMessage is sent to the application to execute an API call (Handled by "api" output. When the message ends the data is sent back as the result).<br />
|
156 | - AppModuleMessage is sent to the application to execute a module API call.<br />
|
157 | - Application Messages are sent to the "api" output with the result being sent back to the module as an AppMessageResponse.<br />
|
158 | - Module Message (Module <-> Application <-> Module)
|
159 | - Module Message is the message that is sent to a module from the another module through the Application.<br />
|
160 | - Module Messages are sent to the application via the BullMQ job queue system.<br />
|
161 | - The application sends the message to the appropriate module for processing and the result is sent back to the requesting module.<br />
|
162 |
|
163 | ## Pre-requisites
|
164 |
|
165 | - [Node.js][nodejs-url] >= 8.16.0
|
166 | - [NPM][npm-url] >= 6.4.1 ([NPM][npm-url] comes with [Node.js][nodejs-url] so there is no need to install separately.)
|
167 |
|
168 | ## Setup
|
169 |
|
170 | ### Install
|
171 |
|
172 | #### Install Application
|
173 | - Create a new TotalJS Flow.
|
174 | - Create a new component and containing the contents of [Application Component][application-component-url].
|
175 | - Add the Application component under the "Application" group to the Flow.
|
176 | - Attach desired input and outputs to the Application component.
|
177 | - Configure Application component.
|
178 |
|
179 | #### Install Module
|
180 | - Create a new TotalJS Flow.
|
181 | - Create a new component and containing the contents of [Module Component][module-component-url].
|
182 | - Add the Module component under the "Module" group to the Flow.
|
183 | - Attach desired input and outputs to the Module component.
|
184 | - Configure Module component.
|
185 |
|
186 | ## Messages
|
187 |
|
188 | ### Application Messages
|
189 | Theses are messages sent TO the application.
|
190 |
|
191 | ## License
|
192 |
|
193 | [MIT License](https://ColtonMcInroy.mit-license.org/) © Colton McInroy
|
194 |
|
195 |
|
196 | [typescript-url]: https://github.com/Microsoft/TypeScript
|
197 | [nodejs-url]: https://nodejs.org
|
198 | [npm-url]: https://www.npmjs.com
|
199 | [node-releases-url]: https://nodejs.org/en/download/releases
|
200 |
|
201 |
|
202 | [array-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array
|
203 | [boolean-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Boolean
|
204 | [function-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function
|
205 | [map-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
|
206 | [number-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number
|
207 | [object-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object
|
208 | [promise-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
|
209 | [regexp-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp
|
210 | [set-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set
|
211 | [string-mdn-url]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String
|
212 |
|
213 |
|
214 | [follow-me-badge]: https://flat.badgen.net/twitter/follow/ColtonMcInroy?icon=twitter
|
215 |
|
216 | [version-badge]: https://flat.badgen.net/npm/v/@nexgin/totaljsapplication?icon=npm
|
217 | [node-version-badge]: https://flat.badgen.net/npm/node/@nexgin/totaljsapplication
|
218 | [mit-license-badge]: https://flat.badgen.net/npm/license/@nexgin/totaljsapplication
|
219 |
|
220 | [downloads-badge]: https://flat.badgen.net/npm/dm/@nexgin/totaljsapplication
|
221 | [total-downloads-badge]: https://flat.badgen.net/npm/dt/@nexgin/totaljsapplication?label=total%20downloads
|
222 | [packagephobia-badge]: https://flat.badgen.net/packagephobia/install/@nexgin/totaljsapplication
|
223 | [bundlephobia-badge]: https://flat.badgen.net/bundlephobia/minzip/@nexgin/totaljsapplication
|
224 |
|
225 | [daviddm-badge]: https://flat.badgen.net/david/dep/ColtonMcInroy/@nexgin/totaljsapplication
|
226 | [codecov-badge]: https://flat.badgen.net/codecov/c/github/ColtonMcInroy/@nexgin/totaljsapplication?label=codecov&icon=codecov
|
227 | [coveralls-badge]: https://flat.badgen.net/coveralls/c/github/ColtonMcInroy/@nexgin/totaljsapplication?label=coveralls
|
228 |
|
229 | [codebeat-badge]: https://codebeat.co/badges/123
|
230 | [codacy-badge]: https://api.codacy.com/project/badge/Grade/123
|
231 | [coc-badge]: https://flat.badgen.net/badge/code%20of/conduct/pink
|
232 |
|
233 |
|
234 | [follow-me-url]: https://twitter.com/ColtonMcInroy?utm_source=github.com&utm_medium=referral&utm_content=ColtonMcInroy/@nexgin/totaljsapplication
|
235 |
|
236 | [version-url]: https://www.npmjs.com/package/@nexgin/totaljsapplication
|
237 | [node-version-url]: https://nodejs.org/en/download
|
238 | [mit-license-url]: https://github.com/ColtonMcInroy/@nexgin/totaljsapplication/blob/master/LICENSE
|
239 |
|
240 | [downloads-url]: https://www.npmtrends.com/@nexgin/totaljsapplication
|
241 | [packagephobia-url]: https://packagephobia.now.sh/result?p=@nexgin/totaljsapplication
|
242 | [bundlephobia-url]: https://bundlephobia.com/result?p=@nexgin/totaljsapplication
|
243 |
|
244 | [daviddm-url]: https://david-dm.org/ColtonMcInroy/@nexgin/totaljsapplication
|
245 | [codecov-url]: https://codecov.io/gh/ColtonMcInroy/@nexgin/totaljsapplication
|
246 | [coveralls-url]: https://coveralls.io/github/ColtonMcInroy/@nexgin/totaljsapplication?branch=master
|
247 |
|
248 | [codebeat-url]: https://codebeat.co/projects/github-com-ColtonMcInroy-@nexgin/totaljsapplication-master
|
249 | [codacy-url]: https://www.codacy.com/app/ColtonMcInroy/@nexgin/totaljsapplication?utm_source=github.com&utm_medium=referral&utm_content=ColtonMcInroy/@nexgin/totaljsapplication&utm_campaign=Badge_Grade
|
250 | [coc-url]: https://github.com/ColtonMcInroy/@nexgin/totaljsapplication/blob/master/CODE_OF_CONDUCT.md
|
251 |
|
252 | [application-component-url]: https://gitlab.mcinroy.info/npm/totaljsapplication/-/blob/main/components/Application.html
|
253 | [module-component-url]: https://gitlab.mcinroy.info/npm/totaljsapplication/-/blob/main/components/Module.html
|
254 | [totaljs-flow-url]: https://github.com/totaljs/flow
|