Shing Architecture
Shing's core systems are:
- MinIO
- MongoDB
- Redis
- Node.js
Shing's primary application components are:
- Web Application Server
- Stream Ingest Server
- Stream Relay Server
- WebRTC Server
- Worker Node(s)
Core Systems
Shing's core architecture is selected and optimized for replicating the system's setup in nearly any hosting environment quickly while operating efficiently.
MinIO
MinIO is an open source implementation of a scalable and available persistent storage system that is compatible with Amazon Simple Storage Service (S3). It enables Shing to store absolutely ridiculous amounts of data without having to worry very much.
MongoDB
MongoDB is a free NoSQL database that excels at building content-focused sites such as Shing. MongoDB is not the best choice for an airline reservation system, banking system, or real-time stock trading network. That's not really what it's built to do. Mongo was built to power posts on timelines and a calendar of videos and streams to watch with associated "rich media" - including blogs (like this) - at scale.
Redis
Redis is a database of sorts that keeps all data in RAM. For certain use cases that are more of a challenge to solve with systems like MongoDB, Redis simply keeps websites performant by being able to quickly answer certain common and high-frequency requests for information.
Node.js
Node.js is a JavaScript application framework and runtime environment. It provides libraries and "modules" that make the implementation of network services a happier experience.
Application Components
All application components are written in JavaScript and intended to run in the Node.js runtime environment. The browser also uses JavaScript. So Shing is a homogeneous development environment written in one (1) language with some extensions and custom bits written in C/C++.
Web Application Server
The Web Application Server is what delivers the Shing user interface, accepts user input, presents stream channels and chat rooms and Community and the rest of the application.
There is more than one Web Application Server running at any given time to handle load and provide high availability in case of a hardware failure or similar situation.
Stream Ingest Server
The Stream Ingest Server is what you connect to when you press "Start Stream" in OBS or similar. It receives your stream, does some math on it, optionally saves it to disk, and produces your live content stream.
There is more than one Ingest Server running at all times for load handling and to provide high availability in case of a hardware failure.
Stream Relay Server
The Stream Relay Server receives your browser's request for a live stream and provides that stream. A Relay will be pulling one stream from an ingest server, and delivering that stream to many viewers.
There is more than one Relay Server. We add and remove relays as needed to changes in demand and load (elastic scale).
WebRTC Server
The WebRTC server is Shing's Selective Forwarding Unit (SFU). It is the process to which your VoIP calls will communicate and probably also connect while using Shing Chat X-Stream. It does no math on the streams. It simply receives data from producers and routes it as configured to consumers.
There is more than one WebRTC Server. Shing adds and removes WebRTC servers in response to changing levels of demand for the service (elastic scale).
Worker Nodes
Worker Nodes are servers that YOU don't connect to. They instead listen to job queues for jobs that need done, do the job, and go back to waiting. They send notifications, encode DVR episodes, extract clips, distribute posts to timelines, perform general housekeeping, expire things, and more. Worker Nodes are pure workhorse machines, and some have GPUs in them to make the video work as fast as possible.
Other Components
There are other components such as DTP Radio, which runs *inside of* OBS to provide the platform's 24/7 radio stream with live headlines so there is always at least one live channel with a chat for folks who like to hang out there. And there are some other projects I tinker on from time to time as a break from working on the main platform.
This doesn't even mention Venue, which is a whole separate environment with a similar list of components minus the stream/video services. It just directly uses Shing for those.
There's a lot going on. And this was a long post about it as a test of long posts :)
