Network Outline

Introduction to network mechanics

This document aims to provide insight into future network components, mechanics, and more. From managing network diversity to term definitions, this paper is for curious individuals who want to learn more about future network features and capabilities.

Table of Contents:

Overview:

Were planning to develop three versions of software, enterprise, opensource, and the mainstream application. Enterprise will be a barebones model our team will shape to fit any enterprise needs; from small businesses to large corporations, our enterprise product will aim to connect those with extreme storage needs. Our opensource software will be for any tech-savvy developers who want to take a shot at developing and experiencing the future dataspace. Finally, our mainstream application is our primary focus in these early stages, and will act within a hybrid social/data model enabling sem-prem storage between friends, family, and peers. For the most primitive of understandings, our network acts as a 1:1 p2p networking system. Extra space that you have on your hard drive can be traded for space on someone else’s, enabling redundancy for both parties without 3rd party managers.

Sem-Prem:

Semi Premises, or Sem-Prem for short, is a storage method that leverages both your local storage (prem), and the local storage of your peers (sem). Similar to a P2P network, by linking peers in a network and sharing excess storage, users can trade extra space on their hard drive for space one another or multiple others. In short we aim to create a marketplace of sorts for users to trade excess space on their local hard drive for remote storage and, in turn, redundancy; A marketplace of redundancy.

With Semi-Premises storage, a user neither has to trust some remote data farm nor rely solely on local storage. Through pool management, redundancy and system load can be balanced to provide a streamline and passive data redundancy system.

Clique Storage:

The difference between our networks and a traditional P2P network is pool sizes and network flexibility. Most P2P networks run on massive scales with thousands of computers and nodes. With such a large network size, user freedom is compromised: one user can’t customize the entire network and others aren’t held socially or personally responsible for data dishonesty. While most Peer-to-Peer networks can still use certain encryption methods and reward systems to prevent data dishonesty, each computer has less at stake and are only held there by their personal storage. This can lead to high turnover and a high level of network churn.

We wouldn’t want our data in such a turbulent environment, so why would you? By shrinking pool sizes, enabling custom network participants, and giving network control back to the users, we can bring sovereignty back to the dataspace. This method is known as Cliquing, or in our case, Clique storage. The storage system will look much like a modern day social media platform: you’ll start by by choosing friends, or ‘peers’ that you want to do a redundancy swap with, of course they will never see your actual data so your just initiating a redundancy agreement. This choice of peers will be informed by a reliability rating that all participants will be issued based on their uptime, reliability, and storage capacity. After a peer request is accepted (and terms are negotiated in opensource and enterprise versions) a two way storage trade begins, an encrypted copy of the desired data is swapped to either person; the limit of data transfer between the two is defined by a percentage. This tolerance is the percent difference that is permitted between peers to prevent one party from oversharing, this percentage can be adjusted if both peers agree and is coined: ‘overshare tolerance’. A multi person storage clique only forms if multiple people all have a mutual peer connection with each other. If three people all had mutual connections with each other, then data would be triangulated between each peer leading to increased scalability and potential data fragmentation.

Network Diversity:

Your data needs are 24/7 but we don’t expect your peers to run full time servers, so how do we provide timely retrieval without requiring unreasonable uptime? Well, first off, you’ll only need to rely on this network and its redundancy in case of a corruption or local file damage. Your data, under normal circumstances, is locally available. Secondly, in the unfortunate circumstance that you may need to rely on the redundancy network, your request will be queued and fulfilled once a peer comes online. A peer won’t even have to open the application or upload anything, if their device turns on, your data will migrate from their device back to yours, fully recovering any data lost on a damaged device! This queuing system doesn’t apply to always-on enterprise networks or servers.

The problem with this system comes in the form of peer diversity. If a local power outage takes out a neighborhood, this neighborhood held all of your peers, no individual had backup power/internet access or even device batteries, and your device somehow lost all of its valuable data, then your data would be unavailable until a peer comes back online. It’s a relatively rare occurrence, but its still important to remember. In order to combat this problem its recommended that you choose a geographically diverse set of peers as to prevent physical data barriers and a set of peers that use different computer stacks as to prevent digital data barriers. By including different devices in different locations an individual has a lower average que time, and a near zero chance that a bug or physical barrier keeps you from your backups.

Enforced inclusion/diversity is being considered at the moment. By adding some random peers to each network we can: 1) improve que times and even remove the que in some cases 2) manage nefarious network activities by incentivising randoms to report suspicious activity.

Participation/Load Balancing:

As mentioned earlier, users that begin a redundancy agreement or those who qualify for group networking will start by, among other things, agreeing upon an overshare tolerance. The program proposes an initial tolerance based on the nature of the agreement and both parties negotiate until both agree on the percentage. Most of the technical details listed regarding network architecture will be abstracted and backended in the mainstream application.

Once the tolerance is determined, both parties complete any other negotiations and begin the redundancy leveraging process. An upload will not silently begin if the calculated upload size exceeds the overshare tolerance. If an upload size is still to big even with the most aggressive compression algorithms, the user will be prompted to continue a partial upload. This partial upload will be documented and the storing party will be informed of any partial coverage. Under ideal circumstances, peers will be matched to such a degree of accuracy that the overshare tolerance is near zero and neither party has to deal with partial coverage. In this agreement both parties host full encrypted copies of each others data and local storage is converted into distributed sem-prem storage. The number of peers you device can support is directly proportional to both the space available on your local drive and the general amount of data you need to store. This equality framework is adopted across most other network resource distribution; any resource that can’t be managed this way will be discussed in a later document.

The mainstream application is designed to work in the background and consume minimal resources. Periodic checking of the que pool should be the only task consuming power on the regular. We can prevent que swamping by reducing endpoint network access, preventing users from requesting data if a local version is found on their device or if they requested recently. Using these methods alongside countless other defense mechanisms, que pools should almost always be empty and user devices should never use unneeded resources. In the end using these main methods and countless other periphery developments will lead to a streamline, non-intrusive, and simple data backup method. No more expensive or vulnerable cloud storage. Use the space you have and bring true sovereignty back to the dataspace.